February 2015 - Devel - libvirt List Archives

[libvirt] [PATCH v2 0/3] Add support for vCPU and I/O thread scheduler setting
by Martin Kletzander 12 Feb '15

12 Feb '15

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1178986 v2: - No dependency on sched.h unless really needed - Docs fixed according to John - RNG schema matches parsing - Possibility to specify one element for whole mask of vcpus/iothreads - leaks fixed (VIR_FREE(nodes);) O:-) - not accepting "other" as a scheduler when parsing - Rebased, of course Martin Kletzander (3): util: Add virProcessSetScheduler() function for scheduler settings docs, schema, conf: Add support for setting scheduler parameters of guest threads qemu: Add support for setting vCPU and I/O thread scheduler setting configure.ac | 4 +- docs/formatdomain.html.in | 16 ++ docs/schemas/domaincommon.rng | 44 +++++ src/conf/domain_conf.c | 183 ++++++++++++++++++++- src/conf/domain_conf.h | 24 +++ src/libvirt_private.syms | 1 + src/qemu/qemu_driver.c | 5 + src/qemu/qemu_process.c | 57 ++++++- src/qemu/qemu_process.h | 5 +- src/util/virprocess.c | 104 +++++++++++- src/util/virprocess.h | 20 ++- .../qemuxml2argv-cputune-iothreadsched-toomuch.xml | 38 +++++ .../qemuxml2argv-cputune-iothreadsched.xml | 39 +++++ .../qemuxml2argv-cputune-vcpusched-overlap.xml | 38 +++++ tests/qemuxml2argvtest.c | 2 + .../qemuxml2xmlout-cputune-iothreadsched.xml | 39 +++++ tests/qemuxml2xmltest.c | 1 + 17 files changed, 612 insertions(+), 8 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cputune-iothreadsched-toomuch.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cputune-iothreadsched.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-cputune-vcpusched-overlap.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-cputune-iothreadsched.xml -- 2.3.0

4 13

[libvirt] Google Summer of Code application submitted
by Stefan Hajnoczi 12 Feb '15

12 Feb '15

Hi, I have submitted our Google Summer of Code application for QEMU, libvirt, and KVM. Accepted organizations will be announced on March 2nd at 19:00 UTC on http://google-melange.com/. You can still add project ideas if you wish to mentor: http://qemu-project.org/Google_Summer_of_Code_2015 Mentors must be active contributors and willing to commit 5 hours per week during the summer. Please let me know if you have any questions. Stefan

1 0

[libvirt] [PATCH v2 0/3] Propagate storage errors on migration
by Michal Privoznik 12 Feb '15

12 Feb '15

This is practically v2 to: https://www.redhat.com/archives/libvir-list/2015-January/msg00230.html The approach is merely the same - listening to BLOCK_JOB_* events instead of asking on the monitor repeatedly. But the code looks more polished. Michal Privoznik (3): qemuProcessHandleBlockJob: Set disk->mirrorState more often qemuProcessHandleBlockJob: Take status into account qemuMigrationDriveMirror: Listen to events src/qemu/qemu_migration.c | 37 +++++++++++++++++-------------------- src/qemu/qemu_process.c | 36 +++++++++++++++++++++--------------- 2 files changed, 38 insertions(+), 35 deletions(-) -- 2.0.5

3 8

[libvirt] vm live storage migration with snapshots
by Edward Young 11 Feb '15

11 Feb '15

Hi all, I'm investigating the ways to improve the live migration performance in libvirt. I have a question about the vm live storage migration. the platform is libvirt + qemu + kvm If we want to migrate a running vm with its virtual disk images to another node. we can use 'virsh migrate ....' commands. What if this vm has a number of disk-only external snapshots? In the current version, how can live migrate this vm? Is it possible to blockcommit the snapshots to the base image and later perform the migration, without shutting down the running vm? Or is it possible to iteratively transfer all the snapshots to the destination and later live migrate only the latest new data? Thanks in advance. Best, Ed

2 4

[libvirt] [PATCH] qemu: fix setting of VM CPU affinity with TCG
by Daniel P. Berrange 11 Feb '15

11 Feb '15

If a previous commit I fixed the incorrect handling of vcpu pids for TCG mode QEMU: commit b07f3d821dfb11a118ee75ea275fd6ab737d9500 Author: Daniel P. Berrange <berrange(a)redhat.com> Date: Thu Dec 18 16:34:39 2014 +0000 Don't setup fake CPU pids for old QEMU The code assumes that def->vcpus == nvcpupids, so when we setup fake CPU pids for old QEMU with nvcpupids == 1, we cause the later code to read off the end of the array. This has fun results like sche_setaffinity(0, ...) which changes libvirtd's own CPU affinity, or even better sched_setaffinity($RANDOM, ...) which changes the affinity of a random OS process. The intent was that this would merely disable the ability to set per-vCPU affinity. It should still have been possible to set VM level host CPU affinity. Unfortunately, when you set <vcpu cpuset='0-1'>4</vcpu>, the XML parser will internally take this & initialize an entry in the def->cputune.vcpupin array for every VCPU. IOW this is implicitly being treated as <cputune> <vcpupin cpuset='0-1' vcpu='0'/> <vcpupin cpuset='0-1' vcpu='1'/> <vcpupin cpuset='0-1' vcpu='2'/> <vcpupin cpuset='0-1' vcpu='3'/> </cputune> Even more fun, the faked cputune elements are hidden from view when querying the live XML, because their cpuset mask is the same as the VM default cpumask. The upshot was that it was impossible to set VM level CPU affinity. To fix this we must update qemuProcessSetVcpuAffinities so that it only reports a fatal error if the per-VCPU cpu mask is different from the VM level cpu mask. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- src/qemu/qemu_process.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index d5df60d..e8c532f 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -2497,9 +2497,18 @@ qemuProcessSetVcpuAffinities(virDomainObjPtr vm) return 0; if (priv->vcpupids == NULL) { - virReportError(VIR_ERR_OPERATION_INVALID, - "%s", _("cpu affinity is not supported")); - return -1; + /* If any CPU has custom affinity that differs from the + * VM default affinity, we must reject it + */ + for (n = 0; n < def->vcpus; n++) { + if (!virBitmapEqual(def->cpumask, + def->cputune.vcpupin[n]->cpumask)) { + virReportError(VIR_ERR_OPERATION_INVALID, + "%s", _("cpu affinity is not supported")); + return -1; + } + } + return 0; } for (n = 0; n < def->vcpus; n++) { -- 2.1.0

3 4

[libvirt] [PATCH] bhyvexml2argvmock: change int to size_t for tapfdSize
by Pavel Hrdina 11 Feb '15

11 Feb '15

Commit c5b6a4a5 forget to update also this mock function. Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com> --- Pushed as build breaker fix (FreeBSD). tests/bhyvexml2argvmock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/bhyvexml2argvmock.c b/tests/bhyvexml2argvmock.c index 0cbea29..41058ca 100644 --- a/tests/bhyvexml2argvmock.c +++ b/tests/bhyvexml2argvmock.c @@ -24,7 +24,7 @@ int virNetDevTapCreateInBridgePort(const char *brname ATTRIBUTE_UNUSED, const unsigned char *vmuuid ATTRIBUTE_UNUSED, const char *tunpath ATTRIBUTE_UNUSED, int *tapfd ATTRIBUTE_UNUSED, - int tapfdSize ATTRIBUTE_UNUSED, + size_t tapfdSize ATTRIBUTE_UNUSED, virNetDevVPortProfilePtr virtPortProfile ATTRIBUTE_UNUSED, virNetDevVlanPtr virtVlan ATTRIBUTE_UNUSED, unsigned int fakeflags ATTRIBUTE_UNUSED) -- 2.0.5

1 0

[libvirt] [PATCH] domain: include portgroup in interface status xml
by Laine Stump 11 Feb '15

11 Feb '15

Prior to commit 7d5bf484747 (first appearing in libvirt 1.2.2), the status XML of a domain's interface was missing a lot of important information; mainly it just output the config of the interface, plus the name of the tap device and qemu device alias. Commit 7d5bf484747 changed the status XML to include many important bits of information that were required to make network "hook" scripts useful - bandwidth information, vlan tag, the name of the bridge (or physical device in the case of macvtap) that the tap/macvtap device was attached to - the commit log for 7d5bf484747 has a very detailed explanation of the change. For quick reference - in the example given there, prior to the change, status XML looked like figure [C]: <interface type='network'> <source network='testnet' portgroup='admin'/> <target dev='macvtap0'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> and after the change, it looked like figure [E]: <interface type='direct'> <source dev='p4p1_0' mode='bridge'/> <bandwidth> <inbound average='1000' peak='5000' burst='1024'/> <outbound average='128' peak='256' burst='256'/> </bandwidth> <target dev='macvtap0'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> You'll notice that bandwidth info, physdev, and macvtap mode have been added, but the network and portgroup names are now missing - I didn't think that this information was of any use once the needed bandwidth/vlan/etc config had been pulled from the network/portgroup. I was wrong. A few months after that change a user on IRC asked what happened to portgroup in the status XML and described how he used it (more or less as a tag to decide what external information to use in a hook script that was run at startup/migration time - see http://wiki.libvirt.org/page/OVS_and_PVLANS ). At that time I planned to make a patch to re-add portgroup, but life intervened as that was just prior to a transatlantic move involving several weeks of "vacation". During this time I somehow forgot to make the patch, and also mistakenly remembered that I *had* made it. Subsequent to this, as a part of mprivozn's work to add support for network-specific hooks, I did re-add the output of the network name in status XML, but once again completely forgot about portgroup. This was in commit a3609121 (first appearing in libvirt 1.2.11). This made the status XML from the above example look like this: <interface type='direct'> <source network='testnet' dev='p4p1_0' mode='bridge'/> <bandwidth> <inbound average='1000' peak='5000' burst='1024'/> <outbound average='128' peak='256' burst='256'/> </bandwidth> <target dev='macvtap0'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> *This* patch just adds the portgroup back to the status XML, so the same example interface will look like this: <interface type='direct'> <source network='testnet' portgroup='admin' dev='p4p1_0' mode='bridge'/> <bandwidth> <inbound average='1000' peak='5000' burst='1024'/> <outbound average='128' peak='256' burst='256'/> </bandwidth> <target dev='macvtap0'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> The result is that the status XML now contains all information about how the interface is setup (bandwidth, physical device, tap device, etc), in addition to pointers to its origin (the network and portgroup). --- I had previously asked about this topic here: https://www.redhat.com/archives/libvir-list/2014-June/msg00416.html but then forgot to follow up on it... src/conf/domain_conf.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 8600eff..d870bb6 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -17591,14 +17591,18 @@ virDomainActualNetDefContentsFormat(virBufferPtr buf, if (def->type == VIR_DOMAIN_NET_TYPE_NETWORK && !inSubelement) { /* When we're putting our output into the <actual> * subelement rather than the main <interface>, the - * network name isn't included in the <source> because the - * main interface element's <source> has the same info - * already. If we've been called to output directly into - * the main element's <source> though (the case here - - * "!inSubElement"), we *do* need to output the network - * name, because the caller won't have done it). + * network name and portgroup don't need to be included in + * the <source> here because the main interface element's + * <source> has the same info already. If we've been + * called to output directly into the main element's + * <source> though (the case here - "!inSubElement"), we + * *do* need to output network/portgroup, because the + * caller won't have done it). */ - virBufferEscapeString(buf, " network='%s'", def->data.network.name); + virBufferEscapeString(buf, " network='%s'", + def->data.network.name); + virBufferEscapeString(buf, " portgroup='%s'", + def->data.network.portgroup); } if (actualType == VIR_DOMAIN_NET_TYPE_BRIDGE || actualType == VIR_DOMAIN_NET_TYPE_NETWORK) { -- 2.1.0

2 1

[libvirt] [PATCH v2 0/2] Followup for NFS root squashed domain save issue
by John Ferlan 11 Feb '15

11 Feb '15

v1 here: http://www.redhat.com/archives/libvir-list/2015-January/msg01029.html Patch 1 of 2 replaces patch 4 from the original series Patch 2 of 2 is the same from the original series John Ferlan (2): virfile: Adjust error path for virFileOpenForked qemu: qemuOpenFileAs - set flag VIR_FILE_OPEN_FORCE_MODE src/qemu/qemu_driver.c | 3 +++ src/util/virfile.c | 65 ++++++++++++++++++++++++++++---------------------- 2 files changed, 40 insertions(+), 28 deletions(-) -- 2.1.0

2 5

[libvirt] [PATCHv2] qemu: fix crash in migrate when migrateuri do not have a scheme
by Luyao Huang 11 Feb '15

11 Feb '15

https://bugzilla.redhat.com/show_bug.cgi?id=1191355 When we migrate a vm with migrateuri option with a uri do not have scheme like this: # virsh migrate test4 --live qemu+ssh://lhuang/system --migrateuri 127.0.0.1 target libvirtd will crashed because uri->scheme is NULL in qemuMigrationPrepareDirect this line: if (STRNEQ(uri->scheme, "tcp") && add a value check before this line. Also fix a place like this in doNativeMigrate. Signed-off-by: Luyao Huang <lhuang(a)redhat.com> --- src/qemu/qemu_migration.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 879b1bf..d4ffe6c 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3281,6 +3281,13 @@ qemuMigrationPrepareDirect(virQEMUDriverPtr driver, if (!(uri = qemuMigrationParseURI(uri_in, &well_formed_uri))) goto cleanup; + if (uri->scheme == NULL) { + virReportError(VIR_ERR_INVALID_ARG, + _("missing scheme in migration URI: %s"), + uri_in); + goto cleanup; + } + if (STRNEQ(uri->scheme, "tcp") && STRNEQ(uri->scheme, "rdma")) { virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, @@ -4083,6 +4090,13 @@ static int doNativeMigrate(virQEMUDriverPtr driver, if (!(uribits = qemuMigrationParseURI(uri, NULL))) return -1; + if (uribits->scheme == NULL) { + virReportError(VIR_ERR_INVALID_ARG, + _("missing scheme in migration URI: %s"), + uri); + goto cleanup; + } + if (STREQ(uribits->scheme, "rdma")) { if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATE_RDMA)) { virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", -- 1.8.3.1

2 2

[libvirt] [PATCH] conf: Fix libvirtd crash and memory leak caused by virDomainVcpuPinDel()
by Zhang Bo 11 Feb '15

11 Feb '15

The function virDomainVcpuPinDel() used vcpupin_list to stand for def->cputune.vcpupin, which made the codes more readable. However, in this function, it will realloc vcpupin_list later. As the definition of realloc(), it may free vcpupin_list and then points it to a new-realloced address, but def->cputune.vcpupin doesn't point to the new address(it's freed however). Thus, 1) When we refer to the def->cputune.vcpupin afterwards, which was freed by realloc(), an INVALID READ occurs, and libvirtd may crash. 2) As no one will use vcpupin_list any more, and no one frees it(it's just alloced by realloc()), memory leak occurs. Part of the valgrind logs are shown as below: ==1837== Thread 15: ==1837== Invalid read of size 8 ==1837== at 0x5367337: virDomainDefFormatInternal (domain_conf.c:18392) which is : virBufferAsprintf(buf, "<vcpupin vcpu='%u' ", def->cputune.vcpupin[i]->vcpuid); ==1837== by 0x536966C: virDomainObjFormat (domain_conf.c:18970) ==1837== by 0x5369743: virDomainSaveStatus (domain_conf.c:19166) ==1837== by 0x117B26DC: qemuDomainPinVcpuFlags (qemu_driver.c:4586) ==1837== by 0x53EA313: virDomainPinVcpuFlags (libvirt.c:9803) ==1837== by 0x14CB7D: remoteDispatchDomainPinVcpuFlags (remote_dispatch.h:6762) ==1837== by 0x14CC81: remoteDispatchDomainPinVcpuFlagsHelper (remote_dispatch.h:6740) ==1837== by 0x5464C30: virNetServerProgramDispatchCall (virnetserverprogram.c:437) ==1837== by 0x546507A: virNetServerProgramDispatch (virnetserverprogram.c:307) ==1837== by 0x171B83: virNetServerProcessMsg (virnetserver.c:172) ==1837== by 0x171E6E: virNetServerHandleJob (virnetserver.c:193) ==1837== by 0x5318E78: virThreadPoolWorker (virthreadpool.c:145) ==1837== Address 0x12ea2870 is 0 bytes inside a block of size 16 free'd ==1837== at 0x4C291AC: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1837== by 0x52A3D14: virReallocN (viralloc.c:245) ==1837== by 0x52A3DFB: virShrinkN (viralloc.c:372) ==1837== by 0x52A3F57: virDeleteElementsN (viralloc.c:503) ==1837== by 0x533939E: virDomainVcpuPinDel (domain_conf.c:15405) //doReset为true时才会进到。 ==1837== by 0x117B2642: qemuDomainPinVcpuFlags (qemu_driver.c:4573) ==1837== by 0x53EA313: virDomainPinVcpuFlags (libvirt.c:9803) ==1837== by 0x14CB7D: remoteDispatchDomainPinVcpuFlags (remote_dispatch.h:6762) ==1837== by 0x14CC81: remoteDispatchDomainPinVcpuFlagsHelper (remote_dispatch.h:6740) ==1837== by 0x5464C30: virNetServerProgramDispatchCall (virnetserverprogram.c:437) ==1837== by 0x546507A: virNetServerProgramDispatch (virnetserverprogram.c:307) ==1837== by 0x171B83: virNetServerProcessMsg (virnetserver.c:172) Steps to reproduce the problem: 1) use virDomainPinVcpuFlags() to pin a guest's vcpu to all the pcpus of the host. This patch uses def->cputune.vcpupin instead of vcpupin_list to do the realloc() job, to avoid invalid read or memory leaking. Signed-off-by: Zhang Bo <oscar.zhangbo(a)huawei.com> Signed-off-by: Yue Wenyuan <yuewenyuan@huawei.com@huawei.com> --- src/conf/domain_conf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index f0b715d..e1a3024 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -16276,7 +16276,7 @@ virDomainVcpuPinDel(virDomainDefPtr def, int vcpu) if (vcpupin_list[n]->vcpuid == vcpu) { virBitmapFree(vcpupin_list[n]->cpumask); VIR_FREE(vcpupin_list[n]); - VIR_DELETE_ELEMENT(vcpupin_list, n, def->cputune.nvcpupin); + VIR_DELETE_ELEMENT(def->cputune.vcpupin, n, def->cputune.nvcpupin); return; } } -- 1.7.12.4

2 1