October 2014 - Devel - libvirt List Archives

[libvirt] [PATCH v2] domain: fix parsing of memory tunables on 32-bit machines
by Eric Blake 01 Nov '14

01 Nov '14

Commit 6c9a8a4 (Oct 2014) exposed a long-standing issue on 32-bit machines: code related to virDomainSetMemoryParameters has always been documented as using a 64-bit limit, but it was implemented by calling virDomainParseMemory which enforced an 'unsigned long' limit. Since VIR_DOMAIN_MEMORY_PARAM_UNLIMITED capped to a long is -1, but virDomainParseScaledValue no longer accepts negative values, an attempt to use 2^53-1 as a hard memory limit started failing the testsuite. However, the problem with capping things artificially low has existed for much longer - ever since commits 4888f0fb and 2e22f23 (Mar 2012) switched internal tracking from 'unsigned long' to 'unsigned long long' (prior to that time, the cap was a side-effect of the choice of types). We _have_ to cap the balloon memory values, (no thanks to baked in 'unsigned long' of API such as virDomainSetMaxMemory or virDomainGetInfo with no counterpart API that guarantees 64-bit access to those numbers) but memory parameters have never needed the artificial limit. At any rate, the solution is to make the parser function gain a parameter, and only do the reduced 32-bit cap for the values that are constrained due to API. * src/conf/domain_conf.h (_virDomainMemtune): Add comments. * src/conf/domain_conf.c (virDomainParseMemory): Add parameter. (virDomainDefParseXML): Adjust callers. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- src/conf/domain_conf.c | 25 ++++++++++++++----------- src/conf/domain_conf.h | 14 ++++++++------ 2 files changed, 22 insertions(+), 17 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 39befb0..c594325 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -6373,19 +6373,22 @@ virDomainParseScaledValue(const char *xpath, /* Parse a memory element located at XPATH within CTXT, and store the - * result into MEM. If REQUIRED, then the value must exist; - * otherwise, the value is optional. The value is in blocks of 1024. - * Return 0 on success, -1 on failure after issuing error. */ + * result into MEM, in blocks of 1024. If REQUIRED, then the value + * must exist; otherwise, the value is optional. The value must not + * exceed VIR_DOMAIN_MEMORY_PARAM_UNLIMITED once scaled; additionally, + * if CAPPED is true, the value must fit within an unsigned long (only + * matters on 32-bit platforms). Return 0 on success, -1 on failure + * after issuing error. */ static int virDomainParseMemory(const char *xpath, xmlXPathContextPtr ctxt, - unsigned long long *mem, bool required) + unsigned long long *mem, bool required, bool capped) { int ret = -1; unsigned long long bytes, max; /* On 32-bit machines, our bound is 0xffffffff * KiB. On 64-bit * machines, our bound is off_t (2^63). */ - if (sizeof(unsigned long) < sizeof(long long)) + if (capped && sizeof(unsigned long) < sizeof(long long)) max = 1024ull * ULONG_MAX; else max = LLONG_MAX; @@ -12215,11 +12218,11 @@ virDomainDefParseXML(xmlDocPtr xml, /* Extract domain memory */ if (virDomainParseMemory("./memory[1]", ctxt, - &def->mem.max_balloon, true) < 0) + &def->mem.max_balloon, true, true) < 0) goto error; if (virDomainParseMemory("./currentMemory[1]", ctxt, - &def->mem.cur_balloon, false) < 0) + &def->mem.cur_balloon, false, true) < 0) goto error; /* and info about it */ @@ -12339,19 +12342,19 @@ virDomainDefParseXML(xmlDocPtr xml, /* Extract other memory tunables */ if (virDomainParseMemory("./memtune/hard_limit[1]", ctxt, - &def->mem.hard_limit, false) < 0) + &def->mem.hard_limit, false, false) < 0) goto error; if (virDomainParseMemory("./memtune/soft_limit[1]", ctxt, - &def->mem.soft_limit, false) < 0) + &def->mem.soft_limit, false, false) < 0) goto error; if (virDomainParseMemory("./memtune/min_guarantee[1]", ctxt, - &def->mem.min_guarantee, false) < 0) + &def->mem.min_guarantee, false, false) < 0) goto error; if (virDomainParseMemory("./memtune/swap_hard_limit[1]", ctxt, - &def->mem.swap_hard_limit, false) < 0) + &def->mem.swap_hard_limit, false, false) < 0) goto error; n = virXPathULong("string(./vcpu[1])", ctxt, &count); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 9908d88..fbb3b2f 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1966,8 +1966,10 @@ typedef struct _virDomainMemtune virDomainMemtune; typedef virDomainMemtune *virDomainMemtunePtr; struct _virDomainMemtune { - unsigned long long max_balloon; /* in kibibytes */ - unsigned long long cur_balloon; /* in kibibytes */ + unsigned long long max_balloon; /* in kibibytes, capped at ulong thanks + to virDomainGetMaxMemory */ + unsigned long long cur_balloon; /* in kibibytes, capped at ulong thanks + to virDomainGetInfo */ virDomainHugePagePtr hugepages; size_t nhugepages; @@ -1975,10 +1977,10 @@ struct _virDomainMemtune { bool nosharepages; bool locked; int dump_core; /* enum virTristateSwitch */ - unsigned long long hard_limit; /* in kibibytes */ - unsigned long long soft_limit; /* in kibibytes */ - unsigned long long min_guarantee; /* in kibibytes */ - unsigned long long swap_hard_limit; /* in kibibytes */ + unsigned long long hard_limit; /* in kibibytes, limit at off_t bytes */ + unsigned long long soft_limit; /* in kibibytes, limit at off_t bytes */ + unsigned long long min_guarantee; /* in kibibytes, limit at off_t bytes */ + unsigned long long swap_hard_limit; /* in kibibytes, limit at off_t bytes */ }; typedef struct _virDomainPowerManagement virDomainPowerManagement; -- 1.9.3

2 2

[libvirt] [PATCH 0/2] Fix two coverity issues
by Pavel Hrdina 01 Nov '14

01 Nov '14

Both are false positive but its nice to make coverity happy. Pavel Hrdina (2): vbox_storage: fix coverity issue with overwriting value hotplug: fix char device detach src/qemu/qemu_hotplug.c | 12 ++++++++---- src/vbox/vbox_storage.c | 36 ++++++++++++++++-------------------- 2 files changed, 24 insertions(+), 24 deletions(-) -- 2.0.4

2 7

[libvirt] [PATCH 1/2] qemu: migration:Release nbd port from migrationPorts instead of remotePorts
by weiwei li 01 Nov '14

01 Nov '14

commit 3e1e16aa8d4238241a1806cb9bdb3b9ad60db777(Use a port from the migration range for NBD as well) change ndb port allocation from remotePorts to migrationPorts, but does not change the port releasing process, which makes an error when migration seveal times(above 64). Signed-off-by: Weiwei Li <nuonuoli(a)tencent.com> --- src/qemu/qemu_migration.c | 8 ++++---- src/qemu/qemu_process.c | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index ca70e35..0ad8054 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1351,7 +1351,7 @@ qemuMigrationStartNBDServer(virQEMUDriverPtr driver, unsigned short port = 0; char *diskAlias = NULL; size_t i; - + VIR_DEBUG("DEBUG: in qemuMigrationStartNBDServer"); for (i = 0; i < vm->def->ndisks; i++) { virDomainDiskDefPtr disk = vm->def->disks[i]; @@ -1389,7 +1389,7 @@ qemuMigrationStartNBDServer(virQEMUDriverPtr driver, cleanup: VIR_FREE(diskAlias); if (ret < 0) - virPortAllocatorRelease(driver->remotePorts, port); + virPortAllocatorRelease(driver->migrationPorts, port); return ret; } @@ -1595,7 +1595,7 @@ qemuMigrationStopNBDServer(virQEMUDriverPtr driver, qemuDomainObjExitMonitor(driver, vm); - virPortAllocatorRelease(driver->remotePorts, priv->nbdPort); + virPortAllocatorRelease(driver->migrationPorts, priv->nbdPort); priv->nbdPort = 0; } @@ -2813,7 +2813,7 @@ qemuMigrationPrepareAny(virQEMUDriverPtr driver, VIR_FORCE_CLOSE(dataFD[1]); if (vm) { if (ret < 0) { - virPortAllocatorRelease(driver->remotePorts, priv->nbdPort); + virPortAllocatorRelease(driver->migrationPorts, priv->nbdPort); priv->nbdPort = 0; } if (ret >= 0 || vm->persistent) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index ef258cf..ba8ba82 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -4839,7 +4839,7 @@ void qemuProcessStop(virQEMUDriverPtr driver, } } - virPortAllocatorRelease(driver->remotePorts, priv->nbdPort); + virPortAllocatorRelease(driver->migrationPorts, priv->nbdPort); priv->nbdPort = 0; if (priv->agent) { -- 1.7.1

4 4

[libvirt] [PATCH 0/3] Avoid rare race with virDomainUndefine()
by Martin Kletzander 31 Oct '14

31 Oct '14

First patch just fixes 'virsh list' for a rare case that is similar to the one fixed in the third patch. Second patch is just a helper for reproducing the race. This was found in qemu driver but is fixed on upper level, so it should be more general. Martin Kletzander (3): virsh: don't list unknown domains DO NOT APPLY UPSTREAM: Reproducer helper Avoid rare race when undefining domain src/conf/domain_conf.c | 26 +++++++++++++++++++++++--- src/conf/domain_conf.h | 1 + tools/virsh-domain-monitor.c | 5 +++++ 3 files changed, 29 insertions(+), 3 deletions(-) -- 2.1.2

3 9

[libvirt] [PATCH] Do not probe for power mgmt capabilities in lxc emulator
by Ján Tomko 31 Oct '14

31 Oct '14

It fails after 30 seconds with this error: error : virDBusCall:1429 : error from service: CanSuspend: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken. Only probe for the power mgmt capabilities when driver is non-NULL. This speeds up domain startup by 30 seconds. https://bugzilla.redhat.com/show_bug.cgi?id=1159227 --- src/lxc/lxc_conf.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/lxc/lxc_conf.c b/src/lxc/lxc_conf.c index 17df7a8..e713ff8 100644 --- a/src/lxc/lxc_conf.c +++ b/src/lxc/lxc_conf.c @@ -82,7 +82,9 @@ virCapsPtr virLXCDriverCapsInit(virLXCDriverPtr driver) VIR_WARN("Failed to query host NUMA topology, disabling NUMA capabilities"); } - if (virNodeSuspendGetTargetMask(&caps->host.powerMgmt) < 0) + /* Only probe for power management capabilities in the driver, + * not in the emulator */ + if (driver && virNodeSuspendGetTargetMask(&caps->host.powerMgmt) < 0) VIR_WARN("Failed to get host power management capabilities"); if (virGetHostUUID(caps->host.host_uuid)) { -- 2.0.4

3 2

[libvirt] [PATCH] qemu: make advice from numad available when building commandline
by Martin Kletzander 31 Oct '14

31 Oct '14

Particularly in qemuBuildNumaArgStr(), there was a need for the advice due to memory backing, which needs to know the nodeset it will be pinned to. With newer qemu this caused the following error when starting domain: error: internal error: Advice from numad is needed in case of automatic numa placement even when starting perfectly valid domain, e.g.: ... <vcpu placement='auto'>4</vcpu> <numatune> <memory mode='strict' placement='auto'/> </numatune> <cpu> <numa> <cell id='0' cpus='0' memory='524288'/> <cell id='1' cpus='1' memory='524288'/> </numa> </cpu> ... Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1138545 Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- src/qemu/qemu_command.c | 10 ++++++---- src/qemu/qemu_command.h | 3 ++- src/qemu/qemu_driver.c | 3 ++- src/qemu/qemu_process.c | 3 ++- tests/qemuxml2argvtest.c | 3 ++- tests/qemuxmlnstest.c | 2 +- 6 files changed, 15 insertions(+), 9 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 2e5af4f..207e2b0 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -6636,7 +6636,8 @@ static int qemuBuildNumaArgStr(virQEMUDriverConfigPtr cfg, const virDomainDef *def, virCommandPtr cmd, - virQEMUCapsPtr qemuCaps) + virQEMUCapsPtr qemuCaps, + virBitmapPtr nodeset) { size_t i, j; virBuffer buf = VIR_BUFFER_INITIALIZER; @@ -6796,7 +6797,7 @@ qemuBuildNumaArgStr(virQEMUDriverConfigPtr cfg, virBufferAsprintf(&buf, ",size=%dM,id=ram-node%zu", cellmem, i); - if (virDomainNumatuneMaybeFormatNodeset(def->numatune, NULL, + if (virDomainNumatuneMaybeFormatNodeset(def->numatune, nodeset, &nodemask, i) < 0) goto cleanup; @@ -7764,7 +7765,8 @@ qemuBuildCommandLine(virConnectPtr conn, virNetDevVPortProfileOp vmop, qemuBuildCommandLineCallbacksPtr callbacks, bool standalone, - bool enableFips) + bool enableFips, + virBitmapPtr nodeset) { virErrorPtr originalError = NULL; size_t i, j; @@ -7992,7 +7994,7 @@ qemuBuildCommandLine(virConnectPtr conn, } if (def->cpu && def->cpu->ncells) - if (qemuBuildNumaArgStr(cfg, def, cmd, qemuCaps) < 0) + if (qemuBuildNumaArgStr(cfg, def, cmd, qemuCaps, nodeset) < 0) goto error; if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_UUID)) diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index aa40c9e..f7d3c2d 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -79,7 +79,8 @@ virCommandPtr qemuBuildCommandLine(virConnectPtr conn, virNetDevVPortProfileOp vmop, qemuBuildCommandLineCallbacksPtr callbacks, bool forXMLToArgv, - bool enableFips) + bool enableFips, + virBitmapPtr nodeset) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(11); /* Generate '-device' string for chardev device */ diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 373daab..5ac638a 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -6472,7 +6472,8 @@ static char *qemuConnectDomainXMLToNative(virConnectPtr conn, VIR_NETDEV_VPORT_PROFILE_OP_NO_OP, &buildCommandLineCallbacks, true, - qemuCheckFips()))) + qemuCheckFips(), + NULL))) goto cleanup; ret = virCommandToString(cmd); diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index ef258cf..6cbd608 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -4362,7 +4362,8 @@ int qemuProcessStart(virConnectPtr conn, priv->monJSON, priv->qemuCaps, migrateFrom, stdin_fd, snapshot, vmop, &buildCommandLineCallbacks, false, - qemuCheckFips()))) + qemuCheckFips(), + nodemask))) goto cleanup; /* now that we know it is about to start call the hook if present */ diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 0e9fab9..9b8e3e2 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -362,7 +362,8 @@ static int testCompareXMLToArgvFiles(const char *xml, migrateFrom, migrateFd, NULL, VIR_NETDEV_VPORT_PROFILE_OP_NO_OP, &testCallbacks, false, - (flags & FLAG_FIPS)))) { + (flags & FLAG_FIPS), + NULL))) { if (!virtTestOOMActive() && (flags & FLAG_EXPECT_FAILURE)) { ret = 0; diff --git a/tests/qemuxmlnstest.c b/tests/qemuxmlnstest.c index b3a608c..2f37a26 100644 --- a/tests/qemuxmlnstest.c +++ b/tests/qemuxmlnstest.c @@ -119,7 +119,7 @@ static int testCompareXMLToArgvFiles(const char *xml, vmdef, &monitor_chr, json, extraFlags, migrateFrom, migrateFd, NULL, VIR_NETDEV_VPORT_PROFILE_OP_NO_OP, - &testCallbacks, false, false))) + &testCallbacks, false, false, NULL))) goto fail; if (!virtTestOOMActive()) { -- 2.1.2

2 2

[libvirt] [PATCH 0/2] Improve error messages about blkio values
by Martin Kletzander 31 Oct '14

31 Oct '14

There was one message used for all parsing errors in the blkio code. This series fixes it in both qemu and lxc drivers where the code is used. Martin Kletzander (2): qemu: improve error message for invalid blkiotune settings lxc: improve error message for invalid blkiotune settings src/lxc/lxc_driver.c | 29 +++++++++++++++++++---------- src/qemu/qemu_driver.c | 29 +++++++++++++++++++---------- 2 files changed, 38 insertions(+), 20 deletions(-) -- 2.1.2

3 5

[libvirt] Entering freeze and release of cadidate release 1 of 1.2.10
by Daniel Veillard 31 Oct '14

31 Oct '14

As planned beginning of this week, I just tagged 1.2.10-rc1 in git and pushed signed tarballs and rpms to the usual place: ftp://libvirt.org/libvirt/ The plan would be to get the rc2 out this friday, and push the release on monday if everything looks okay. In my own tests, my guest lost network connectivity, I'm not sure if it is a temporary problem on my side or something more serious in the current code, so please give it a try and report, thanks, Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

2 3

[libvirt] [PATCH v3 0/3] add nodeset check in numatune
by Chen Fan 31 Oct '14

31 Oct '14

when setting elements memnode and nodeset in attribute numatune more than the host nodes in XML file, VM boot should fail. so add check for that. Chen Fan (3): numatune: add check for numatune nodeset range lxc controller: add check for numatune virnuma: remove redundant check for numanode src/conf/numatune_conf.c | 28 +++++++++++++ src/conf/numatune_conf.h | 1 + src/libvirt_private.syms | 2 + src/lxc/lxc_controller.c | 1 + src/qemu/qemu_command.c | 3 ++ src/qemu/qemu_command.h | 1 + src/util/virbitmap.c | 49 ++++++++++++++++++++++ src/util/virbitmap.h | 3 ++ src/util/virnuma.c | 48 ++++++++++++++------- src/util/virnuma.h | 2 + ...rgv-numatune-static-nodeset-exceed-hostnode.xml | 35 ++++++++++++++++ tests/qemuxml2argvmock.c | 9 ++++ tests/qemuxml2argvtest.c | 1 + tests/virbitmaptest.c | 26 +++++++++++- 14 files changed, 193 insertions(+), 16 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-numatune-static-nodeset-exceed-hostnode.xml -- 1.9.3

3 9

[libvirt] virsh migrate
by Chateigner Nicolas 31 Oct '14

31 Oct '14

Hi, I would like to use this command to backup my virtual machine on my secondary server virsh migrate --live --suspend --copy-storage-inc virtual_machine qemu+ssh://root@secondary_server/system But we don't have option to disable shut off the virtual machine on the source server. Would you add an option to offer this functionality ? Best regards Nicolas Chateigner <http://www.inserm.fr>

1 0