- Devel - libvirt List Archives

[libvirt] [PATCH v3] Fixes virsh save-restore/migration when memory detach not in LIFO
by Nitesh Konkar 31 Oct '16

31 Oct '16

Currently the migration stream references the memory blocks by name (which is supplied by libvirt) rather than by there order. With the current code that is assigning aliases for memory backend objects this won't happen and since qemu is treating the memory object links differently migration does not work in such case. This patch ensures slot number alocation for the memory modules beforehand and assign alias accordingly. This keeps slot numbers consistent with the aliases always. Signed-off-by: Nitesh Konkar <nitkon12(a)linux.vnet.ibm.com> --- src/conf/domain_conf.h | 1 + src/qemu/qemu_alias.c | 36 +++++++++++++++++++++++++----------- src/qemu/qemu_alias.h | 6 ++++-- src/qemu/qemu_domain.c | 3 +++ src/qemu/qemu_hotplug.c | 5 ++++- 5 files changed, 37 insertions(+), 14 deletions(-) diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index fd3ae8e..22b5fe1 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -2145,6 +2145,7 @@ struct _virDomainDef { virDomainBlkiotune blkio; virDomainMemtune mem; + virBitmapPtr memslotsptr; virDomainVcpuDefPtr *vcpus; size_t maxvcpus; diff --git a/src/qemu/qemu_alias.c b/src/qemu/qemu_alias.c index cc83fec..8deb054 100644 --- a/src/qemu/qemu_alias.c +++ b/src/qemu/qemu_alias.c @@ -332,21 +332,34 @@ qemuAssignDeviceRNGAlias(virDomainDefPtr def, } -int -qemuAssignDeviceMemoryAlias(virDomainDefPtr def, - virDomainMemoryDefPtr mem) +void +qemuAssignDeviceMemorySlot(virDomainDefPtr def, + virDomainMemoryDefPtr mem) { size_t i; - int maxidx = 0; - int idx; + int minidx = 0; - for (i = 0; i < def->nmems; i++) { - if ((idx = qemuDomainDeviceAliasIndex(&def->mems[i]->info, "dimm")) >= maxidx) - maxidx = idx + 1; + if (mem->info.addr.dimm.base) { + minidx = mem->info.addr.dimm.slot; + } else { + for (i = 0; i < def->mem.memory_slots; i++) { + if (!virBitmapIsBitSet(def->memslotsptr, i)) { + minidx = i; + break; + } + } } - if (virAsprintf(&mem->info.alias, "dimm%d", maxidx) < 0) - return -1; + ignore_value(virBitmapSetBit(def->memslotsptr, minidx)); + mem->info.addr.dimm.slot = minidx; +} + + +int +qemuAssignDeviceMemoryAlias(virDomainMemoryDefPtr mem) +{ + if (virAsprintf(&mem->info.alias, "dimm%d", mem->info.addr.dimm.slot) < 0) + return -1; return 0; } @@ -475,7 +488,8 @@ qemuAssignDeviceAliases(virDomainDefPtr def, virQEMUCapsPtr qemuCaps) return -1; } for (i = 0; i < def->nmems; i++) { - if (virAsprintf(&def->mems[i]->info.alias, "dimm%zu", i) < 0) + qemuAssignDeviceMemorySlot(def, def->mems[i]); + if (virAsprintf(&def->mems[i]->info.alias, "dimm%d", def->mems[i]->info.addr.dimm.slot) < 0) return -1; } diff --git a/src/qemu/qemu_alias.h b/src/qemu/qemu_alias.h index 11d9fde..c6cb568 100644 --- a/src/qemu/qemu_alias.h +++ b/src/qemu/qemu_alias.h @@ -57,8 +57,10 @@ int qemuAssignDeviceRedirdevAlias(virDomainDefPtr def, int qemuAssignDeviceRNGAlias(virDomainDefPtr def, virDomainRNGDefPtr rng); -int qemuAssignDeviceMemoryAlias(virDomainDefPtr def, - virDomainMemoryDefPtr mems); +void qemuAssignDeviceMemorySlot(virDomainDefPtr def, + virDomainMemoryDefPtr); + +int qemuAssignDeviceMemoryAlias(virDomainMemoryDefPtr mems); int qemuAssignDeviceShmemAlias(virDomainDefPtr def, virDomainShmemDefPtr shmem, diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 9b1a32e..263e78f 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -2386,6 +2386,9 @@ qemuDomainDefPostParse(virDomainDefPtr def, if (qemuDomainDefVcpusPostParse(def) < 0) goto cleanup; + if (def->mem.memory_slots) + def->memslotsptr = virBitmapNew(def->mem.memory_slots); + ret = 0; cleanup: virObjectUnref(qemuCaps); diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 72dd93b..5a3af10 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -1898,7 +1898,9 @@ qemuDomainAttachMemory(virQEMUDriverPtr driver, if (qemuDomainDefValidateMemoryHotplug(vm->def, priv->qemuCaps, mem) < 0) goto cleanup; - if (qemuAssignDeviceMemoryAlias(vm->def, mem) < 0) + qemuAssignDeviceMemorySlot(vm->def, mem); + + if (qemuAssignDeviceMemoryAlias(mem) < 0) goto cleanup; if (virAsprintf(&objalias, "mem%s", mem->info.alias) < 0) @@ -4427,6 +4429,7 @@ qemuDomainDetachMemoryDevice(virQEMUDriverPtr driver, } mem = vm->def->mems[idx]; + ignore_value(virBitmapClearBit(vm->def->memslotsptr, memdef->info.addr.dimm.slot)); if (!mem->info.alias) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", -- 1.9.3

3 2

[libvirt] Entering freeze for libvirt-2.4.0
by Daniel Veillard 30 Oct '16

30 Oct '16

As planned I just tagged the release in git and pushed signed tarball and rpms to the usual place: ftp://libvirt.org/libvirt/ Not in my normal environment this week so I hadn't had a chance to test the build before pushing, sorry about this but things went smoothly, I noticed a warning in virsh however: CC virsh-virsh-host.o vsh.c: In function 'vshReadlineParse': vsh.c:516:9: warning: 'opts_need_arg' may be used uninitialized in this function [-Wmaybe-uninitialized] i = ffsl(*opts_need_arg) - 1; ^~~~~~~~~~~~~~~~~~~~ vsh.c:2658:14: note: 'opts_need_arg' was declared here uint64_t opts_need_arg, opts_seen; ^~~~~~~~~~~~~ that probably ought to be fixed for release. I hope to be able to push RC2 during the week-end and possibly the final 2.4.0 on the 1st Nov if all goes well. please give it some testing, thanks ! Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

3 5

[libvirt] [PATCH v2] vsh: Using VSH_REQUIRE_OPTION rather than virReportError
by Kothapally Madhu Pavan 28 Oct '16

28 Oct '16

Correcting the error reporting method by using VSH_REQUIRE_OPTION instead of virReportError Signed-off-by: Kothapally Madhu Pavan <kmp(a)linux.vnet.ibm.com> --- tools/virsh-domain.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 050e7fb..e1cb2ac 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -10443,6 +10443,7 @@ cmdMigrate(vshControl *ctl, const vshCmd *cmd) VSH_EXCLUSIVE_OPTIONS("live", "offline"); VSH_EXCLUSIVE_OPTIONS("timeout-suspend", "timeout-postcopy"); + VSH_REQUIRE_OPTION("postcopy-after-precopy", "postcopy"); if (!(dom = virshCommandOptDomain(ctl, cmd, NULL))) return false; @@ -10474,12 +10475,6 @@ cmdMigrate(vshControl *ctl, const vshCmd *cmd) } if (vshCommandOptBool(cmd, "postcopy-after-precopy")) { - if (!vshCommandOptBool(cmd, "postcopy")) { - virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, "%s", - _("--postcopy-after-precopy can only be used with " - "--postcopy")); - goto cleanup; - } iterEvent = virConnectDomainEventRegisterAny( priv->conn, dom, VIR_DOMAIN_EVENT_ID_MIGRATION_ITERATION,

2 1

[libvirt] [PATCH 00/17] Avoid races with udev
by Michal Privoznik 28 Oct '16

28 Oct '16

I've came across interesting bug recently. The problem was that user tried to start a domain, but qemu was denied access to some device. Even though we relabelled it initially. By debugging I found the root cause: while we were starting qemu, udev came and restored original security labels. Sigh. We have two options here: a) write out series of udev rules so that whenever it tries to relabel something our rule will stop it from doing so b) write a small helper binary that will udev call in order to: 1) detect whether device is in use by libvirt 2) get seclabel that was set by libvirt These patches implement the latter approach. While these patches make life easier for us, there is still a race when udev might restore the device's seclabel before we had chance to flush internal database of seclabels for the helper binary. This is something I'm currently focusing on. But before I get there, here are patches that makes the problem much more bearable. In case you want to try these patches, here are some scratch builds: https://mprivozn.fedorapeople.org/udev/ Also, you can find them on my branch: https://github.com/zippy2/libvirt/commits/udev_labels2 This beast is turned off by default, to turn it on you'll need to add: write_udev=1 to qemu.conf. Michal Privoznik (17): virseclabel.h: Include stdbool.h virseclabel: Introduce virSecurityDeviceLabelDefNewLabel security_dac: Pass manager to virSecurityDACSetImageLabel security_dac: Pass manager to virSecurityDACRestoreFileLabelInternal virudev: Introduce basic skeleton virudev: Implement virUdevMgrAddLabel and virUdevMgrRemoveAllLabels virudev: Introduce virUdevMgrDump tests: Introduce virudevtest virudev: Parse virUdevMgr from JSON virudev: Introduce virUdevMgrLookupLabels util: Introduce libvirt_udevhelper security: Wire up virUdevMgr qemu.conf: Introduce write_udev qemu: Wire up virUdevMgr qemu: Reload virUdevMgr on start virudevtest: Introduce device filtering qemu: Filter uninteresting paths for virUdevMgr libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + po/POTFILES.in | 2 + src/Makefile.am | 21 ++ src/libvirt_private.syms | 15 + src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf | 5 + src/qemu/qemu_conf.c | 3 + src/qemu/qemu_conf.h | 5 + src/qemu/qemu_domain.c | 12 +- src/qemu/qemu_domain.h | 3 +- src/qemu/qemu_driver.c | 40 +- src/qemu/qemu_hotplug.c | 35 +- src/qemu/qemu_process.c | 47 ++- src/qemu/qemu_process.h | 3 + src/qemu/test_libvirtd_qemu.aug.in | 1 + src/security/security_dac.c | 103 ++++-- src/security/security_manager.c | 16 + src/security/security_manager.h | 5 + src/security/security_selinux.c | 47 ++- src/util/udevhelper.c | 137 +++++++ src/util/virseclabel.c | 14 + src/util/virseclabel.h | 6 + src/util/virudev.c | 588 ++++++++++++++++++++++++++++++ src/util/virudev.h | 63 ++++ tests/Makefile.am | 12 + tests/virudevmock.c | 29 ++ tests/virudevtest.c | 312 ++++++++++++++++ tests/virudevtestdata/complex.json | 30 ++ tests/virudevtestdata/empty.json | 5 + tests/virudevtestdata/simple-dac.json | 13 + tests/virudevtestdata/simple-selinux.json | 13 + 32 files changed, 1535 insertions(+), 54 deletions(-) create mode 100644 src/util/udevhelper.c create mode 100644 src/util/virudev.c create mode 100644 src/util/virudev.h create mode 100644 tests/virudevmock.c create mode 100644 tests/virudevtest.c create mode 100644 tests/virudevtestdata/complex.json create mode 100644 tests/virudevtestdata/empty.json create mode 100644 tests/virudevtestdata/simple-dac.json create mode 100644 tests/virudevtestdata/simple-selinux.json -- 2.8.4

5 35

[libvirt] [PATCH] network: fix endless loop when starting network with multiple IPs and no dhcp
by Laine Stump 28 Oct '16

28 Oct '16

(From the "How the Hell did I not see this?" files) commit 9065cfaa added the ability to disable DNS services for a libvirt virtual network. If neither DNS nor DHCP is needed for a network, then we don't need to start dnsmasq, so code was added to check for this. Unfortunately, it was written with a great lack of attention to detail (I can say that, because I was the author), and the loop that checked if DHCP is needed for the network would never end if the network had multiple IP addresses, and none of them had a <dhcp> section (which would have contained a <range> or <host> element). This patch rewrites the check to be more compact and (more importantly) finite. This bug was present in release 2.2.0 and 2.3.0, so will need to be backported to any relevant maintainence branches. Reported here: https://www.redhat.com/archives/libvirt-users/2016-October/msg00032.html https://www.redhat.com/archives/libvirt-users/2016-October/msg00045.html --- src/network/bridge_driver.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/src/network/bridge_driver.c b/src/network/bridge_driver.c index 9d7fc31..a3ee3f3 100644 --- a/src/network/bridge_driver.c +++ b/src/network/bridge_driver.c @@ -1410,20 +1410,22 @@ networkStartDhcpDaemon(virNetworkDriverStatePtr driver, int ret = -1; dnsmasqContext *dctx = NULL; - if (!(ipdef = virNetworkDefGetIPByIndex(network->def, AF_UNSPEC, 0))) { - /* no IP addresses, so we don't need to run */ - ret = 0; - goto cleanup; - } - /* see if there are any IP addresses that need a dhcp server */ - for (i = 0; ipdef && !needDnsmasq; - ipdef = virNetworkDefGetIPByIndex(network->def, AF_UNSPEC, i + 1)) { + i = 0; + while ((ipdef = virNetworkDefGetIPByIndex(network->def, AF_UNSPEC, i))) { + i++; if (ipdef->nranges || ipdef->nhosts) needDnsmasq = true; } + if (i == 0) { + /* no IP addresses at all, so we don't need to run */ + ret = 0; + goto cleanup; + } + if (!needDnsmasq && network->def->dns.enable == VIR_TRISTATE_BOOL_NO) { + /* no DHCP services needed, and user disabled DNS service */ ret = 0; goto cleanup; } -- 2.7.4

2 1

[libvirt] [PATCH 00/10] qemu: agent: bugfix and improvments
by Nikolay Shirokovskiy 28 Oct '16

28 Oct '16

Patch 1 fixes the bug I dealt with. Patches 2-3 is a somewhat protection from similar bugs. Patches 5-6 drop error flag from the agent altogether, so now there are less places for bugs to lurk. Rest patches cleanup loop function. I think 1-3 can be dropped if 6 is approved. It is just 6 can be considered controversial. Nikolay Shirokovskiy (10): qemu: agent: unset error flag in EOF handler qemu: agent: dont set error after agent cleanup qemu: agent: clear error flag upon init qemu: agent: handle agent connection errors in one place qemu: agent: straigten up failed agent start case qemu: agent: drop agent error flag (6) qemu: agent: simplify io loop qemu: agent: exit early if error is already set qemu: agent: fix loop error messages qemu: agent: drop redundant referencing in loop src/qemu/qemu_agent.c | 183 ++++++++++++++++++------------------------- src/qemu/qemu_agent.h | 2 - src/qemu/qemu_domain.c | 41 +++++----- src/qemu/qemu_domain.h | 1 - src/qemu/qemu_driver.c | 9 +-- src/qemu/qemu_migration.c | 13 +-- src/qemu/qemu_process.c | 90 ++++----------------- tests/qemumonitortestutils.c | 1 - 8 files changed, 119 insertions(+), 221 deletions(-) -- 1.8.3.1

2 31

[libvirt] [RFC PATCH v2 1/1] qemu: host NUMA hugepage policy without guest NUMA
by Sam Bobroff 28 Oct '16

28 Oct '16

At the moment, guests that are backed by hugepages in the host are only able to use policy to control the placement of those hugepages on a per-(guest-)CPU basis. Policy applied globally is ignored. Such guests would use <memoryBacking><hugepages/></memoryBacking> and a <numatune> block with <memory mode=... nodeset=.../> but no <memnode .../> elements. This patch corrects this by, in this specific case, changing the QEMU command line from "-mem-prealloc -mem-path=..." (which cannot specify NUMA policy) to "-object memory-backend-file ..." (which can). Note: This is not visible to the guest and does not appear to create a migration incompatibility. Signed-off-by: Sam Bobroff <sam.bobroff(a)au1.ibm.com> --- Hello libvirt community, This patch is a RFC of an attempt to understand and fix a problem with NUMA memory placement policy being ignored in a specific situation. Previous discussion on this issue: https://www.redhat.com/archives/libvir-list/2016-October/msg00033.html https://www.redhat.com/archives/libvir-list/2016-October/msg00514.html (Sorry, this is a long description because the situation is complicated and there are several ways of approaching it and I need to explain my reasoning for choosing this specific method.) The issue I'm trying to fix: The problem is apparent when using this combination of features: * The guest is backed by hugepages in the host. * The backing memory is constrained by NUMA memory placement policy. * The guest either does not use NUMA nodes at all, or does but does not specify a per-node policy. The guest XML would therefore contain blocks similar to the following: <memoryBacking><hugepages/></memoryBacking> <numatune><memory mode='strict' nodeset='0'/></numatune> (And does not use <numatune><memnode ..>... .) (Policy works correctly without hugepages, or when memnode is used.) What happens is that the guest runs successfully but the NUMA memory placement policy is ignored, allowing memory to be allocated on any (host) node. Attempted solutions: The most obvious approach to me is a trivial patch to qemuBuildMemoryBackendStr(). The test that checks to see if a memory-backend is necessary (around line 3332 in qemu_command.c) is this: /* If none of the following is requested... */ if (!needHugepage && !userNodeset && !memAccess && !nodeSpecified && !force) { /* report back that using the new backend is not necessary * to achieve the desired configuration */ ret = 1; } else { This test does not consider the nodemask, so when no other reason exists to use a memory-backend (which is the case in point) the policy is lost. Adding nodemask to this test appears to easily fixe the issue. However: * It only works when the guest uses NUMA nodes (because otherwise the whole function is skipped at a higher level). * When the guest does use NUMA nodes, it causes a memory-backend object to be added to each one (when they did not previously have one) and this causes migration incompatability. So it seems worthwhile to look for other options. Other ways of controlling NUMA memory placement are numactl (libnuma) or cgroups but in practice neither can work in this situation due to the interaction of several features: * numactl and cgroups can only apply policy to all memory allocated by that thread. * When a guest is backed by hugepages, memory needs to be preallcated. * QEMU performs memory pre-allocation from it's main thread, not the VCPU threads. This seems to mean that policy applied to the QEMU VCPU threads does not apply to pre-allocated memory and policy applied to the QEMU main thread incorrectly applies to all of QEMU's allocations, not just the VCPU ones. My testing seems to confirm this. If that is true, then neither of those approaches can work (at least, not without changes to QEMU). At this point it appeared that the only ways to proceed would all require a migration-breaking change, so I started to look at QEMU to see if that could be worked around somehow. ... another option: While investigating QEMU I discovered that it is possible to use a QEMU memory backend object without attaching it to any guest NUMA node and when it is used that way it does not break migration. This is (to me at least) a new way of using a memory-backend object, but it appears to directly replace -mem-path and -mem-prealloc and additionally allow NUMA memory placement policy to be specified. (The ID field is still required even though it is not used.) For example, this... -m 4G -mem-path /dev/hugepages -mem-prealloc ... seems to be equavalent to this: -m 4G -object memory-backend-file,id=xxx,prealloc=yes,path=/dev/hugepages,size=4G But this is now possible as well: -m 4G -object memory-backend-file,id=xxx,prealloc=yes,path=/dev/hugepages,host-nodes=0-1,size=4G ... so it seems to be a way of solving this issue without breaking migration. Additionally, there seems to be code specifically in QEMU to prevent the migration data from changing in this case, which leads me to believe that this is an intended use of the option but I haven't directly consulted with the QEMU community. Implementation: My patch is based on this last case. It is intended to work by switching the "-mem-path" style argument to an un-attached memory-backend-file, which is why the change is done within qemuBuildMemPathStr(). In the case where QEMU doesn't support the needed option, I've opted to raise a warning rather than an error, because existing guests may have this configuration and although their policy is being ignored they do at least run and I assume it's important to maintain this. I realize that this policy can't be changed at run-time by libvirt, unlike policy set by cgroups or libnuma, but memory-backend objects are already in use in more common situations, and already have this limitation so I don't think this is adding significantly to that problem. Questions: So does this line of reasoning make sense? Is this a good way to fix it? Are there better solutions? Are there concerns with using memory-backend in this way: do I need to consult with the QEMU community before a libvirt patch that uses it could be accepted? Should the warning be an error? It seemed safest to me to only use the backend object when necessary, but there doesn't seem to be a reason to avoid using it whenever it is available. Is there one? Should I have done that instead? Thanks for reading! Sam. v2: Incorporated review feedback from Peter Krempa <pkrempa(a)redhat.com> and Martin Kletzander <mkletzan(a)redhat.com>. src/qemu/qemu_command.c | 56 +++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 47 insertions(+), 9 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 405e118..0dea9fd 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -7105,12 +7105,18 @@ qemuBuildSmpCommandLine(virCommandPtr cmd, static int qemuBuildMemPathStr(virQEMUDriverConfigPtr cfg, - const virDomainDef *def, + virDomainDefPtr def, virQEMUCapsPtr qemuCaps, - virCommandPtr cmd) + virCommandPtr cmd, + virBitmapPtr auto_nodeset) { const long system_page_size = virGetSystemPageSizeKB(); char *mem_path = NULL; + virBitmapPtr nodemask = NULL; + const char *backendType = NULL; + char *backendStr = NULL; + virJSONValuePtr props = NULL; + int rv = -1; /* * No-op if hugepages were not requested. @@ -7135,18 +7141,50 @@ qemuBuildMemPathStr(virQEMUDriverConfigPtr cfg, if (qemuGetHupageMemPath(cfg, def->mem.hugepages[0].size, &mem_path) < 0) return -1; - virCommandAddArgList(cmd, "-mem-prealloc", "-mem-path", mem_path, NULL); + if (virDomainNumatuneMaybeGetNodeset(def->numa, auto_nodeset, + &nodemask, -1) < 0) + return -1; + if (nodemask && virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_MEMORY_FILE)) { + if (qemuBuildMemoryBackendStr(virDomainDefGetMemoryInitial(def), + 0, -1, NULL, auto_nodeset, + def, qemuCaps, cfg, &backendType, + &props, false) < 0) + goto cleanup; + /* The memory-backend object created here is not going to be + * attached anywhere and there is only one so the ID can be hard coded. + * It acts like -mem-prealloc -mem-path ... but allows policy to be + * set. */ + if (!(backendStr = virQEMUBuildObjectCommandlineFromJSON(backendType, + "mem", + props))) + goto cleanup; + virCommandAddArgList(cmd, "-object", backendStr, NULL); + rv = 0; + cleanup: + virJSONValueFree(props); + VIR_FREE(backendStr); + } else { + if (nodemask) + /* Ideally this would be an error, but if it was it would + * break existing guests already using this configuration. */ + VIR_WARN("Memory file backend objects are " + "unsupported by QEMU binary. Global NUMA " + "hugepage policy will be ignored."); + virCommandAddArgList(cmd, "-mem-prealloc", "-mem-path", mem_path, NULL); + rv = 0; + } VIR_FREE(mem_path); - return 0; + return rv; } static int qemuBuildMemCommandLine(virCommandPtr cmd, virQEMUDriverConfigPtr cfg, - const virDomainDef *def, - virQEMUCapsPtr qemuCaps) + virDomainDefPtr def, + virQEMUCapsPtr qemuCaps, + virBitmapPtr auto_nodeset) { if (qemuDomainDefValidateMemoryHotplug(def, qemuCaps, NULL) < 0) return -1; @@ -7170,7 +7208,7 @@ qemuBuildMemCommandLine(virCommandPtr cmd, * there is no numa node specified. */ if (!virDomainNumaGetNodeCount(def->numa) && - qemuBuildMemPathStr(cfg, def, qemuCaps, cmd) < 0) + qemuBuildMemPathStr(cfg, def, qemuCaps, cmd, auto_nodeset) < 0) return -1; if (def->mem.locked && !virQEMUCapsGet(qemuCaps, QEMU_CAPS_REALTIME_MLOCK)) { @@ -7307,7 +7345,7 @@ qemuBuildNumaArgStr(virQEMUDriverConfigPtr cfg, } if (!needBackend && - qemuBuildMemPathStr(cfg, def, qemuCaps, cmd) < 0) + qemuBuildMemPathStr(cfg, def, qemuCaps, cmd, auto_nodeset) < 0) goto cleanup; for (i = 0; i < ncells; i++) { @@ -9385,7 +9423,7 @@ qemuBuildCommandLine(virQEMUDriverPtr driver, if (!migrateURI && !snapshot && qemuDomainAlignMemorySizes(def) < 0) goto error; - if (qemuBuildMemCommandLine(cmd, cfg, def, qemuCaps) < 0) + if (qemuBuildMemCommandLine(cmd, cfg, def, qemuCaps, nodeset) < 0) goto error; if (qemuBuildSmpCommandLine(cmd, def) < 0) -- 2.10.0.297.gf6727b0

1 0

[libvirt] [PATCH 5/5] docs: Add Qt Virtual machines manager and Qt Remote viewer
by Alexander Vasilenko 27 Oct '16

27 Oct '16

--- docs/apps.html.in | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/apps.html.in b/docs/apps.html.in index 1a138b3..4efb318 100644 --- a/docs/apps.html.in +++ b/docs/apps.html.in @@ -208,6 +208,17 @@ to remote consoles supporting the VNC protocol. Also provides an optional mozilla browser plugin. </dd> + <dt><a href="http://f1ash.github.io/qt-virt-manager">qt-virt-manager</a></dt> + <dd> + The Qt GUI for create and control VMs and another virtual entities + (aka networks, storages, interfaces, secrets, network filters). + Contains integrated LXC/SPICE/VNC viewer for accessing the graphical or + text console associated with a virtual machine or container. + </dd> + <dt><a href="http://f1ash.github.io/qt-virt-manager/#virtual-machines-viewer">qt-remote-viewer</a></dt> + <dd> + The Qt VNC/SPICE viewer for access to remote desktops or VMs. + </dd> </dl> <h2><a name="iaas">Infrastructure as a Service (IaaS)</a></h2> -- 2.10.1

2 1

[libvirt] [PATCH 0/3] wireshark: Further build system fixes
by Andrea Bolognani 27 Oct '16

27 Oct '16

This takes care of the few remaining nits. All use cases I could think of are covered; if any more issues are discovered, we'll take care of them then. Andrea Bolognani (3): wireshark: Don't redefine ws_plugindir wireshark: Try a bunch of possible prefixes wireshark: Use ${exec_prefix} for $ws_plugindir m4/virt-wireshark.m4 | 23 ++++++++++++++++------- tools/Makefile.am | 1 - 2 files changed, 16 insertions(+), 8 deletions(-) -- 2.7.4

2 7

[libvirt] [RFC] Toward a better NEWS file
by Andrea Bolognani 27 Oct '16

27 Oct '16

Hi, there's an idea that has been kicking around in my head for a while, and I'd like to share it with the list to gather some feedback before I forget about it :) Right now, each entry in our NEWS file contains what is basically the output of git log \ --pretty=oneline \ vX.Y-1.0..vX.Y.0 with the commits organized somewhat arbitrarily into a bunch of sections with partially overlapping scopes. I believe the current form is less than useful, as it is too detailed for end users and distro packagers, who only care about the high-level user visible changes, and not detailed enough for developers, who are always going to refer to the proper git log anyway. Moreover, we ship a ChangeLog file that contains very detailed information already. Ideally, the NEWS file would contain a summary of notable changes (new APIs, significantly improved features, etc.) laid out in an easy to digest fashion, such that people who are not knee-deep into libvirt development can grasp them and hopefully get excited about upgrading to our latest and greatest release :) Of course, it would take an insane amount of time for any single one of us to turn the git log into such a document, and the result would still be sub-par because we simply can't expect anyone to have full insight in every single change to the project. My solution for this is to ask the people with the most knowledge of the changes - the authors themselves! The workflow I'm envisioning would look like this: * DV, at the same time as he announces that libvirt has entered freeze, will put out a Call for NEWS and ask people who have contributed code to the upcoming release to post a summary of their changes * the authors will go over git log \ --author=$(git config user.email) \ vX.Y-1.0..master and come up with a short (one-three sentences) summary for each of the changes, if they are notable. Commits that are part of a larger series would not be described on their own: a short summary of the series would be used instead, akin to the one you would put in your cover letter. To give a practical example: I've mostly been busy with reviews this cycle, but if I were to go over my commits since v2.3.0 right now I would write something like * Bug fix: don't restart libvirt-guests.service when libvirtd.service is restarted for commit 2273bbd, and omit both 61e1014 and a0da413 as they're neither notable enough on their own, nor part of a larger series: we'll always have a "various bug fixes and improvements" bullet point in a NEWS file entry to take care of that kind of small cleanups and improvements. * the authors would post the resulting summaries to the list. We could simply post them as regular patches to docs/news.html.in (potentially without requiring review before pushing them), or post them as plain text and have someone collect them and prepare a single commit * DV will tag the release and push the tarballs, and everyone will be able to enjoy the NEWS file :) Some light editoral work might be needed throughout the process, eg. fix typos or post one or two reminders during the freeze: I volunteer to take such tasks upon myself. I'm looking forward to feedback about this idea, especially from people who might be part of any community where anything like this is already happening. -- Andrea Bolognani / Red Hat / Virtualization

6 14

[libvirt] [PATCH 0/3] vz: nettype=bridge and other corrections
by Maxim Nestratov 27 Oct '16

27 Oct '16

Maxim Nestratov (3): vz: support type=bridge network interface type correctly vz: remove Bridged network name and rename Routed vz: add MIGRATION_V3 capability src/vz/vz_driver.c | 1 + src/vz/vz_sdk.c | 100 ++++++++--------------------------------------------- src/vz/vz_utils.h | 3 +- 3 files changed, 16 insertions(+), 88 deletions(-) -- 2.4.11

2 10

[libvirt] [REPOST] regarding cgroup v2 support in libvirt
by Tejun Heo 27 Oct '16

27 Oct '16

(reposting w/ libvir-list cc'd, sorry about the delay in reposting, was traveling and then on vacation) Hello, Daniel. How have you been? We (facebook) are deploying cgroup v2 and internally use libvirt to manage virtual machines, so I'm trying to add cgroup v2 support to libvirt. Because cgroup v2's resource configurations differ from v1 in varying degrees depending on the specific resource type, it unfortunately introduces new configurations (some completely new configs, others just a different range / format). This means that adding cgroup v2 support to libvirt requires adding new config options to it and maybe implementing some form of translation mechanism between overlapping configs. The upcoming systemd release includes all that's necessary to support v1/v2 compatibility so that users setting resource configs through systemd don't have to worry about whether v1 or v2 is in use. I'm wondering whether it would make sense to make libvirt use dbus calls to systemd to set resource configs when systemd is in use, so that it can piggyback on systemd's v1/v2 compatibility. It is true that, as libvirt can be used without systemd, libvirt will probably want its own direct implementation down the line, but I think there are benefits to going through systemd for resource settings in general given that hierarchy setup is already done through systemd when available. What do you think? Thanks! -- tejun

3 4

[libvirt] [PATCH] qemu_driver: don't set cpu pin for TCG domain
by Chen Hanxiao 27 Oct '16

27 Oct '16

From: Chen Hanxiao <chenhanxiao(a)gmail.com> We don't support pinning cpu for TCG domain. But we could set it by vcpupin command, which result in a failed startup. Signed-off-by: Chen Hanxiao <chenhanxiao(a)gmail.com> --- src/qemu/qemu_driver.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 93ea5e2..98cfcab 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5177,6 +5177,14 @@ qemuDomainPinVcpuFlags(virDomainPtr dom, if (virDomainObjGetDefs(vm, flags, &def, &persistentDef) < 0) goto endjob; + if ((def && def->virtType == VIR_DOMAIN_VIRT_QEMU) || + (persistentDef && persistentDef->virtType == VIR_DOMAIN_VIRT_QEMU)) + { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("Virt type 'Qemu'(TCG) did not support CPU pin")); + goto endjob; + } + if (persistentDef && !(vcpuinfo = virDomainDefGetVcpu(persistentDef, vcpu))) { virReportError(VIR_ERR_INVALID_ARG, -- 1.8.3.1

2 3

[libvirt] [RFC PATCH] libxl: add tunnelled migration support
by Bob Liu 27 Oct '16

27 Oct '16

Tunnelled migration doesn't require any extra network connections beside the libvirt daemon. It's capable of strong encryption and is the default option in openstack-nova. This patch add the tunnelled migration(Tunnel3params) support to libxl. The data flow in the src side is: * libxlDoMigrateSend() -> pipe * libxlTunnel3MigrationFunc() poll pipe out and then write to dest stream. While in the dest side: Stream -> pipe -> 'recvfd of libxlDomainStartRestore' The usage is the same as p2p migration, execpt adding one more '--tunnelled' to the libvirt p2p migration command. Signed-off-by: Bob Liu <bob.liu(a)oracle.com> --- src/libxl/libxl_driver.c | 58 ++++++++++- src/libxl/libxl_migration.c | 241 +++++++++++++++++++++++++++++++++++++++++--- src/libxl/libxl_migration.h | 9 ++ 3 files changed, 292 insertions(+), 16 deletions(-) diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index b66cb1f..a01bbff 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -5918,6 +5918,61 @@ libxlDomainMigrateBegin3Params(virDomainPtr domain, } static int +libxlDomainMigratePrepareTunnel3Params(virConnectPtr dconn, + virStreamPtr st, + virTypedParameterPtr params, + int nparams, + const char *cookiein, + int cookieinlen, + char **cookieout ATTRIBUTE_UNUSED, + int *cookieoutlen ATTRIBUTE_UNUSED, + unsigned int flags) +{ + libxlDriverPrivatePtr driver = dconn->privateData; + virDomainDefPtr def = NULL; + const char *dom_xml = NULL; + const char *dname = NULL; + const char *uri_in = NULL; + +#ifdef LIBXL_HAVE_NO_SUSPEND_RESUME + virReportUnsupportedError(); + return -1; +#endif + + virCheckFlags(LIBXL_MIGRATION_FLAGS, -1); + if (virTypedParamsValidate(params, nparams, LIBXL_MIGRATION_PARAMETERS) < 0) + goto error; + + if (virTypedParamsGetString(params, nparams, + VIR_MIGRATE_PARAM_DEST_XML, + &dom_xml) < 0 || + virTypedParamsGetString(params, nparams, + VIR_MIGRATE_PARAM_DEST_NAME, + &dname) < 0 || + virTypedParamsGetString(params, nparams, + VIR_MIGRATE_PARAM_URI, + &uri_in) < 0) + + goto error; + + if (!(def = libxlDomainMigrationPrepareDef(driver, dom_xml, dname))) + goto error; + + if (virDomainMigratePrepareTunnel3ParamsEnsureACL(dconn, def) < 0) + goto error; + + if (libxlDomainMigrationPrepareTunnel3(dconn, st, &def, cookiein, + cookieinlen, flags) < 0) + goto error; + + return 0; + + error: + virDomainDefFree(def); + return -1; +} + +static int libxlDomainMigratePrepare3Params(virConnectPtr dconn, virTypedParameterPtr params, int nparams, @@ -6017,7 +6072,7 @@ libxlDomainMigratePerform3Params(virDomainPtr dom, if (virDomainMigratePerform3ParamsEnsureACL(dom->conn, vm->def) < 0) goto cleanup; - if (flags & VIR_MIGRATE_PEER2PEER) { + if ((flags & (VIR_MIGRATE_TUNNELLED | VIR_MIGRATE_PEER2PEER))) { if (libxlDomainMigrationPerformP2P(driver, vm, dom->conn, dom_xml, dconnuri, uri, dname, flags) < 0) goto cleanup; @@ -6501,6 +6556,7 @@ static virHypervisorDriver libxlHypervisorDriver = { .nodeDeviceReset = libxlNodeDeviceReset, /* 1.2.3 */ .domainMigrateBegin3Params = libxlDomainMigrateBegin3Params, /* 1.2.6 */ .domainMigratePrepare3Params = libxlDomainMigratePrepare3Params, /* 1.2.6 */ + .domainMigratePrepareTunnel3Params = libxlDomainMigratePrepareTunnel3Params, /* 2.3.1 */ .domainMigratePerform3Params = libxlDomainMigratePerform3Params, /* 1.2.6 */ .domainMigrateFinish3Params = libxlDomainMigrateFinish3Params, /* 1.2.6 */ .domainMigrateConfirm3Params = libxlDomainMigrateConfirm3Params, /* 1.2.6 */ diff --git a/src/libxl/libxl_migration.c b/src/libxl/libxl_migration.c index 534abb8..88c9bb8 100644 --- a/src/libxl/libxl_migration.c +++ b/src/libxl/libxl_migration.c @@ -44,6 +44,7 @@ #include "libxl_migration.h" #include "locking/domain_lock.h" #include "virtypedparam.h" +#include "fdstream.h" #define VIR_FROM_THIS VIR_FROM_LIBXL @@ -484,6 +485,90 @@ libxlDomainMigrationPrepareDef(libxlDriverPrivatePtr driver, } int +libxlDomainMigrationPrepareTunnel3(virConnectPtr dconn, + virStreamPtr st, + virDomainDefPtr *def, + const char *cookiein, + int cookieinlen, + unsigned int flags) +{ + libxlMigrationCookiePtr mig = NULL; + libxlDriverPrivatePtr driver = dconn->privateData; + virDomainObjPtr vm = NULL; + libxlMigrationDstArgs *args = NULL; + virThread thread; + int dataFD[2] = { -1, -1 }; + int ret = 0; + + if (libxlMigrationEatCookie(cookiein, cookieinlen, &mig) < 0) + goto error; + + if (mig->xenMigStreamVer > LIBXL_SAVE_VERSION) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, + _("Xen migration stream version '%d' is not supported on this host"), + mig->xenMigStreamVer); + goto error; + } + + if (!(vm = virDomainObjListAdd(driver->domains, *def, + driver->xmlopt, + VIR_DOMAIN_OBJ_LIST_ADD_LIVE | + VIR_DOMAIN_OBJ_LIST_ADD_CHECK_LIVE, + NULL))) + goto error; + + /* + * The data flow of tunnel3 migration in the dest side: + * stream -> pipe -> recvfd of libxlDomainStartRestore + */ + if (pipe(dataFD) < 0) + goto error; + + /* Stream data will be written to pipeIn */ + if (virFDStreamOpen(st, dataFD[1]) < 0) + goto error; + + if (libxlMigrationDstArgsInitialize() < 0) + goto error; + + if (!(args = virObjectNew(libxlMigrationDstArgsClass))) + goto error; + + args->conn = dconn; + args->vm = vm; + args->flags = flags; + args->migcookie = mig; + /* Receive from pipeOut */ + args->recvfd = dataFD[0]; + args->nsocks = 0; + if (virThreadCreate(&thread, false, libxlDoMigrateReceive, args) < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("Failed to create thread for receiving migration data")); + goto error; + } + + goto done; + + error: + VIR_FORCE_CLOSE(dataFD[1]); + VIR_FORCE_CLOSE(dataFD[0]); + virObjectUnref(args); + /* Remove virDomainObj from domain list */ + if (vm) { + virDomainObjListRemove(driver->domains, vm); + vm = NULL; + } + ret = -1; + + done: + /* Nobody will close dataFD[1]? */ + if (vm) + virObjectUnlock(vm); + + return ret; +} + +int libxlDomainMigrationPrepare(virConnectPtr dconn, virDomainDefPtr *def, const char *uri_in, @@ -710,9 +795,90 @@ libxlDomainMigrationPrepare(virConnectPtr dconn, return ret; } -/* This function is a simplification of virDomainMigrateVersion3Full - * excluding tunnel support and restricting it to migration v3 - * with params since it was the first to be introduced in libxl. +typedef struct _libxlTunnelMigrationThread libxlTunnelMigrationThread; +struct _libxlTunnelMigrationThread { + virThread thread; + virStreamPtr st; + int srcFD; +}; +#define TUNNEL_SEND_BUF_SIZE 65536 + +/* + * The data flow of tunnel3 migration in the src side: + * libxlDoMigrateSend() -> pipe + * libxlTunnel3MigrationFunc() polls pipe out and then write to dest stream. + */ +static void libxlTunnel3MigrationFunc(void *arg) +{ + libxlTunnelMigrationThread *data = (libxlTunnelMigrationThread *)arg; + char *buffer = NULL; + struct pollfd fds[1]; + int timeout = -1; + + if (VIR_ALLOC_N(buffer, TUNNEL_SEND_BUF_SIZE) < 0) { + virReportError(errno, "%s", _("poll failed in migration tunnel")); + return; + } + + fds[0].fd = data->srcFD; + for (;;) { + int ret; + + fds[0].events = POLLIN; + fds[0].revents = 0; + ret = poll(fds, ARRAY_CARDINALITY(fds), timeout); + if (ret < 0) { + if (errno == EAGAIN || errno == EINTR) + continue; + virReportError(errno, "%s", + _("poll failed in libxlTunnel3MigrationFunc")); + goto abrt; + } + + if (ret == 0) { + VIR_DEBUG("poll got timeout"); + break; + } + + if (fds[0].revents & (POLLIN | POLLERR | POLLHUP)) { + int nbytes; + + nbytes = read(data->srcFD, buffer, TUNNEL_SEND_BUF_SIZE); + if (nbytes > 0) { + /* Write to dest stream */ + if (virStreamSend(data->st, buffer, nbytes) < 0) { + virReportError(errno, "%s", + _("tunnelled migration failed to send to virStream")); + goto abrt; + } + } else if (nbytes < 0) { + virReportError(errno, "%s", + _("tunnelled migration failed to read from xen side")); + goto abrt; + } else { + /* EOF; transferred all data */ + break; + } + } + } + + if (virStreamFinish(data->st) < 0) + virReportError(errno, "%s", + _("tunnelled migration failed to finish stream")); + + cleanup: + VIR_FREE(buffer); + + return; + + abrt: + virStreamAbort(data->st); + goto cleanup; +} + +/* This function is a simplification of virDomainMigrateVersion3Full and + * restricting it to migration v3 with params since it was the first to be + * introduced in libxl. */ static int libxlDoMigrateP2P(libxlDriverPrivatePtr driver, @@ -737,6 +903,10 @@ libxlDoMigrateP2P(libxlDriverPrivatePtr driver, bool cancelled = true; virErrorPtr orig_err = NULL; int ret = -1; + /* For tunnel migration */ + virStreamPtr st = NULL; + libxlTunnelMigrationThread *libxlTunnelMigationThreadPtr = NULL; + int dataFD[2] = { -1, -1 }; dom_xml = libxlDomainMigrationBegin(sconn, vm, xmlin, &cookieout, &cookieoutlen); @@ -764,29 +934,62 @@ libxlDoMigrateP2P(libxlDriverPrivatePtr driver, VIR_DEBUG("Prepare3"); virObjectUnlock(vm); - ret = dconn->driver->domainMigratePrepare3Params - (dconn, params, nparams, cookieout, cookieoutlen, NULL, NULL, &uri_out, destflags); + if (flags & VIR_MIGRATE_TUNNELLED) { + if (!(st = virStreamNew(dconn, 0))) + goto cleanup; + ret = dconn->driver->domainMigratePrepareTunnel3Params + (dconn, st, params, nparams, cookieout, cookieoutlen, NULL, NULL, destflags); + } else { + ret = dconn->driver->domainMigratePrepare3Params + (dconn, params, nparams, cookieout, cookieoutlen, NULL, NULL, &uri_out, destflags); + } virObjectLock(vm); if (ret == -1) goto cleanup; - if (uri_out) { - if (virTypedParamsReplaceString(&params, &nparams, - VIR_MIGRATE_PARAM_URI, uri_out) < 0) { - orig_err = virSaveLastError(); + if (!(flags & VIR_MIGRATE_TUNNELLED)) { + if (uri_out) { + if (virTypedParamsReplaceString(&params, &nparams, + VIR_MIGRATE_PARAM_URI, uri_out) < 0) { + orig_err = virSaveLastError(); + goto finish; + } + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("domainMigratePrepare3 did not set uri")); goto finish; } - } else { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("domainMigratePrepare3 did not set uri")); - goto finish; } VIR_DEBUG("Perform3 uri=%s", NULLSTR(uri_out)); - ret = libxlDomainMigrationPerform(driver, vm, NULL, NULL, - uri_out, NULL, flags); + if (flags & VIR_MIGRATE_TUNNELLED) { + if (VIR_ALLOC(libxlTunnelMigationThreadPtr) < 0) + goto cleanup; + if (pipe(dataFD) < 0) { + virReportError(errno, "%s", _("Unable to make pipes")); + goto cleanup; + } + /* Read from pipe */ + libxlTunnelMigationThreadPtr->srcFD = dataFD[0]; + /* Write to dest stream */ + libxlTunnelMigationThreadPtr->st = st; + if (virThreadCreate(&libxlTunnelMigationThreadPtr->thread, true, + libxlTunnel3MigrationFunc, + libxlTunnelMigationThreadPtr) < 0) { + virReportError(errno, "%s", + _("Unable to create tunnel migration thread")); + goto cleanup; + } + virObjectUnlock(vm); + /* Send data to pipe */ + ret = libxlDoMigrateSend(driver, vm, flags, dataFD[1]); + virObjectLock(vm); + } else { + ret = libxlDomainMigrationPerform(driver, vm, NULL, NULL, + uri_out, NULL, flags); + } if (ret < 0) orig_err = virSaveLastError(); @@ -824,6 +1027,14 @@ libxlDoMigrateP2P(libxlDriverPrivatePtr driver, vm->def->name); cleanup: + if (libxlTunnelMigationThreadPtr) { + virThreadCancel(&libxlTunnelMigationThreadPtr->thread); + VIR_FREE(libxlTunnelMigationThreadPtr); + } + VIR_FORCE_CLOSE(dataFD[0]); + VIR_FORCE_CLOSE(dataFD[1]); + virObjectUnref(st); + if (ddomain) { virObjectUnref(ddomain); ret = 0; diff --git a/src/libxl/libxl_migration.h b/src/libxl/libxl_migration.h index 8a074a0..fcea558 100644 --- a/src/libxl/libxl_migration.h +++ b/src/libxl/libxl_migration.h @@ -29,6 +29,7 @@ # define LIBXL_MIGRATION_FLAGS \ (VIR_MIGRATE_LIVE | \ VIR_MIGRATE_PEER2PEER | \ + VIR_MIGRATE_TUNNELLED | \ VIR_MIGRATE_PERSIST_DEST | \ VIR_MIGRATE_UNDEFINE_SOURCE | \ VIR_MIGRATE_PAUSED) @@ -53,6 +54,14 @@ libxlDomainMigrationPrepareDef(libxlDriverPrivatePtr driver, const char *dname); int +libxlDomainMigrationPrepareTunnel3(virConnectPtr dconn, + virStreamPtr st, + virDomainDefPtr *def, + const char *cookiein, + int cookieinlen, + unsigned int flags); + +int libxlDomainMigrationPrepare(virConnectPtr dconn, virDomainDefPtr *def, const char *uri_in, -- 2.6.5

2 2

[libvirt] [PATCH 0/6] Address assignment fixes
by Ján Tomko 27 Oct '16

27 Oct '16

Fixes a crash on usb-serial hotplug and missing addresses after libvirtd restart, and makes some code more readable. Ján Tomko (6): Add 'FromCache' to virDomainVirtioSerialAddrAutoAssign Introduce virDomainVirtioSerialAddrAutoAssign again Return directly from qemuDomainAttachChrDeviceAssignAddr Also create the USB address cache for domains with all the USB addresses Fix crash on usb-serial hotplug Do not try to release virtio serial addresses src/conf/domain_addr.c | 41 +++++++++++++++++++++++++++++++++---- src/conf/domain_addr.h | 14 +++++++++++-- src/libvirt_private.syms | 2 ++ src/qemu/qemu_domain_address.c | 22 ++++++++++++++++---- src/qemu/qemu_hotplug.c | 46 +++++++++++++++++------------------------- 5 files changed, 87 insertions(+), 38 deletions(-) -- 2.7.3

2 13

[libvirt] libvirt compiles on RISC-V (RV64G)
by Richard W.M. Jones 27 Oct '16

27 Oct '16

I'm happy to announce that libvirt compiles fine from git on Fedora/RISC-V. This has little or no practical value at all, since RISC-V lacks such essentials such as virtualization, qemu etc. However I suppose you could use it as a remote client. # file src/.libs/libvirt.so.0.2004.0 src/.libs/libvirt.so.0.2004.0: ELF 64-bit LSB shared object, UCB RISC-V, version 1 (SYSV), dynamically linked, BuildID[sha1]=525ed42ce6d1284c6a909bd6f1b0d6181e88af7b, not stripped # file tools/.libs/virsh tools/.libs/virsh: ELF 64-bit LSB shared object, UCB RISC-V, version 1 (SYSV), dynamically linked, interpreter /lib/ld.so.1, for GNU/Linux 2.6.32, BuildID[sha1]=67e2b69f9007c02137545fed1b7fd9b2871740ee, not stripped Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

2 4

[libvirt] [PATCH] Added the description for Qt Virtual machine manager and Qt Remote Viewer as libvirt related applications.
by F1ash 27 Oct '16

27 Oct '16

--- docs/apps.html.in | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/apps.html.in b/docs/apps.html.in index 1a138b3..3098b73 100644 --- a/docs/apps.html.in +++ b/docs/apps.html.in @@ -208,6 +208,17 @@ to remote consoles supporting the VNC protocol. Also provides an optional mozilla browser plugin. </dd> + <dt><a href="http://f1ash.github.io/qt-virt-manager">qt-virt-manager</a></dt> + <dd> + The Qt GUI for create and control VMs and another virtual entities + (aka networks, storages, interfaces, secrets, network filters). + Contains integrated LXC/SPICE/VNC viewer for accessing the graphical or + text console associated with a virtual machine or container. + </dd> + <dt><a href="http://f1ash.github.io/qt-virt-manager">qt-remote-viewer</a></dt> + <dd> + The Qt VNC/SPICE viewer for access to remote desktops or VMs. + </dd> </dl> <h2><a name="iaas">Infrastructure as a Service (IaaS)</a></h2> -- 2.10.1

2 1

[libvirt] [PATCH] qemu: Minimalize global driver accesses
by Michal Privoznik 27 Oct '16

27 Oct '16

Whilst working on another issue, I've noticed that in some functions we have a local @driver variable among with access to global @qemu_driver variable. This makes no sense. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_driver.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index bacabd2..3495cbf 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -3831,7 +3831,7 @@ qemuDomainScreenshot(virDomainPtr dom, } unlink_tmp = true; - virSecurityManagerSetSavedStateLabel(qemu_driver->securityManager, vm->def, tmp); + virSecurityManagerSetSavedStateLabel(driver->securityManager, vm->def, tmp); qemuDomainObjEnterMonitor(driver, vm); if (qemuMonitorScreendump(priv->mon, tmp) < 0) { @@ -11449,7 +11449,7 @@ qemuDomainMemoryPeek(virDomainPtr dom, goto endjob; } - virSecurityManagerSetSavedStateLabel(qemu_driver->securityManager, vm->def, tmp); + virSecurityManagerSetSavedStateLabel(driver->securityManager, vm->def, tmp); priv = vm->privateData; qemuDomainObjEnterMonitor(driver, vm); -- 2.8.4

2 1

[libvirt] [PATCH] virsh: Fix on_shutdown to on_poweroff
by Martin Kletzander 27 Oct '16

27 Oct '16

The 'on_shutdown' was the name in Xen, there was never such option in libvirt's domain XML. Reported-by: Ruben Kerkhof <ruben(a)rubenkerkhof.com> Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- Pushed as trivial. tools/virsh.pod | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/virsh.pod b/tools/virsh.pod index 8236a36fc2af..f278fecafde3 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -2398,7 +2398,7 @@ succeed, and may take a variable length of time depending on what services must be shutdown in the domain. The exact behavior of a domain when it shuts down is set by the -I<on_shutdown> parameter in the domain's XML definition. +I<on_poweroff> parameter in the domain's XML definition. If I<domain> is transient, then the metadata of any snapshots will be lost once the guest stops running, but the snapshot contents still -- 2.10.1

1 0

[libvirt] [PATCH 1/1] qemu: host NUMA hugepage policy without guest NUMA
by Sam Bobroff 27 Oct '16

27 Oct '16

At the moment, guests that are backed by hugepages in the host are only able to use policy to control the placement of those hugepages on a per-(guest-)CPU basis. Policy applied globally is ignored. Such guests would use <memoryBacking><hugepages/></memoryBacking> and a <numatune> block with <memory mode=... nodeset=.../> but no <memnode .../> elements. This patch corrects this by, in this specific case, changing the QEMU command line from "-mem-prealloc -mem-path=..." (which cannot specify NUMA policy) to "-object memory-backend-file ..." (which can). Note: This is not visible to the guest and does not appear to create a migration incompatibility. Signed-off-by: Sam Bobroff <sam.bobroff(a)au1.ibm.com> --- There was some discussion leading up to this patch, here: https://www.redhat.com/archives/libvir-list/2016-October/msg00033.html During that discussion it seemed that fixing this issue would cause migration incompatibility but after some careful testing, it appears that it only does when a memory-backend object is attached to a guest NUMA node and that is not the case here. If only one is created, and used globally (not attached via mem=<id>), the migration data does not seem to be changed and so it seems reasonable to try something like this patch. This patch does work for my test cases but I don't claim a deep understanding of the libvirt code so this is at least partly a RFC. Comments welcome :-) Cheers, Sam. src/qemu/qemu_command.c | 55 ++++++++++++++++++++++++++++++++++++++++--------- src/qemu/qemu_command.h | 2 +- 2 files changed, 46 insertions(+), 11 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 0804133..c28c8f2 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3143,7 +3143,7 @@ qemuBuildMemoryBackendStr(unsigned long long size, int guestNode, virBitmapPtr userNodeset, virBitmapPtr autoNodeset, - virDomainDefPtr def, + const virDomainDefPtr def, virQEMUCapsPtr qemuCaps, virQEMUDriverConfigPtr cfg, const char **backendType, @@ -7129,12 +7129,18 @@ qemuBuildSmpCommandLine(virCommandPtr cmd, static int qemuBuildMemPathStr(virQEMUDriverConfigPtr cfg, - const virDomainDef *def, + const virDomainDefPtr def, virQEMUCapsPtr qemuCaps, - virCommandPtr cmd) + virCommandPtr cmd, + virBitmapPtr auto_nodeset) { const long system_page_size = virGetSystemPageSizeKB(); char *mem_path = NULL; + virBitmapPtr nodemask = NULL; + const char *backendType = NULL; + char *backendStr = NULL; + virJSONValuePtr props; + int rv = -1; /* * No-op if hugepages were not requested. @@ -7159,18 +7165,47 @@ qemuBuildMemPathStr(virQEMUDriverConfigPtr cfg, if (qemuGetHupageMemPath(cfg, def->mem.hugepages[0].size, &mem_path) < 0) return -1; - virCommandAddArgList(cmd, "-mem-prealloc", "-mem-path", mem_path, NULL); + if (virDomainNumatuneMaybeGetNodeset(def->numa, auto_nodeset, + &nodemask, -1) < 0) + return -1; + if (nodemask && virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_MEMORY_FILE)) { + props = virJSONValueNewObject(); + if (qemuBuildMemoryBackendStr(virDomainDefGetMemoryInitial(def), + 0, -1, NULL, auto_nodeset, + def, qemuCaps, cfg, &backendType, + &props, false) < 0) + goto cleanup; + if (!(backendStr = virQEMUBuildObjectCommandlineFromJSON(backendType, + "mem", + props))) + goto cleanup; + virCommandAddArgList(cmd, "-object", backendStr, NULL); + rv = 0; +cleanup: + VIR_FREE(backendStr); + VIR_FREE(props); + } + else { + if (nodemask || 1) + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Memory file backend objects are " + "unsupported by QEMU binary. Global NUMA " + "hugepage policy will be ignored.")); + virCommandAddArgList(cmd, "-mem-prealloc", "-mem-path", mem_path, NULL); + rv = 0; + } VIR_FREE(mem_path); - return 0; + return rv; } static int qemuBuildMemCommandLine(virCommandPtr cmd, virQEMUDriverConfigPtr cfg, - const virDomainDef *def, - virQEMUCapsPtr qemuCaps) + const virDomainDefPtr def, + virQEMUCapsPtr qemuCaps, + virBitmapPtr auto_nodeset) { if (qemuDomainDefValidateMemoryHotplug(def, qemuCaps, NULL) < 0) return -1; @@ -7194,7 +7229,7 @@ qemuBuildMemCommandLine(virCommandPtr cmd, * there is no numa node specified. */ if (!virDomainNumaGetNodeCount(def->numa) && - qemuBuildMemPathStr(cfg, def, qemuCaps, cmd) < 0) + qemuBuildMemPathStr(cfg, def, qemuCaps, cmd, auto_nodeset) < 0) return -1; if (def->mem.locked && !virQEMUCapsGet(qemuCaps, QEMU_CAPS_REALTIME_MLOCK)) { @@ -7331,7 +7366,7 @@ qemuBuildNumaArgStr(virQEMUDriverConfigPtr cfg, } if (!needBackend && - qemuBuildMemPathStr(cfg, def, qemuCaps, cmd) < 0) + qemuBuildMemPathStr(cfg, def, qemuCaps, cmd, auto_nodeset) < 0) goto cleanup; for (i = 0; i < ncells; i++) { @@ -9381,7 +9416,7 @@ qemuBuildCommandLine(virQEMUDriverPtr driver, if (!migrateURI && !snapshot && qemuDomainAlignMemorySizes(def) < 0) goto error; - if (qemuBuildMemCommandLine(cmd, cfg, def, qemuCaps) < 0) + if (qemuBuildMemCommandLine(cmd, cfg, def, qemuCaps, nodeset) < 0) goto error; if (qemuBuildSmpCommandLine(cmd, def) < 0) diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index 2f2a6ff..1e97d76 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -126,7 +126,7 @@ int qemuBuildMemoryBackendStr(unsigned long long size, int guestNode, virBitmapPtr userNodeset, virBitmapPtr autoNodeset, - virDomainDefPtr def, + const virDomainDefPtr def, virQEMUCapsPtr qemuCaps, virQEMUDriverConfigPtr cfg, const char **backendType, -- 2.10.0.297.gf6727b0

3 12

[libvirt] on shutdown
by Ruben Kerkhof 26 Oct '16

26 Oct '16

Hi all, virsh(1) has this to say about virsh shutdown: "The exact behavior of a domain when it shuts down is set by the on_shutdown parameter in the domain's XML definition." on_shutdown isn't documented in https://libvirt.org/formatdomain.html Is the virsh(1) manpage wrong? Kind regards, Ruben Kerkhof

2 4

[libvirt] [PATCH] Resuming running domain raise error message now
by Sławek Kapłoński 26 Oct '16

26 Oct '16

This patch introduce raising XML error when user tries to resume domain (Qemu or LXC) which is already running. Closes bug: https://bugzilla.redhat.com/show_bug.cgi?id=1009008 Sławek Kapłoński (1): Resuming running domain raise error message src/lxc/lxc_driver.c | 8 +++++++- src/qemu/qemu_driver.c | 4 ++++ 2 files changed, 11 insertions(+), 1 deletion(-) -- 2.10.1

2 5

[libvirt] [PATCH 0/5] wireshark: Fix $(prefix) handling
by Andrea Bolognani 26 Oct '16

26 Oct '16

Well, almost :) There are still some cases that are not handled correctly, but at least this will unbreak 'make rpm' while I work on the rest. Tested by running 'make rpm' successfully on Fedora 23, Fedora 24 and Fedora rawhide. Andrea Bolognani (5): wireshark: Introduce $ws_modversion wireshark: Hoist $ws_prefix declaration wireshark: Strip prefix correctly wireshark: Inject $(prefix) at the right time wireshark: Rename plugindir to ws_plugindir m4/virt-wireshark.m4 | 29 ++++++++++++++++++----------- tools/Makefile.am | 2 +- 2 files changed, 19 insertions(+), 12 deletions(-) -- 2.7.4

3 8

[libvirt] [PATCH] virt-wireshark.m4: Defer $(prefix) substitution
by Michal Privoznik 26 Oct '16

26 Oct '16

The problem with evaluating $(prefix) in configure phase is that autoconf does a lot of magic with this variable. Firstly, if the --prefix argument is not set for the configure script, the variable will have value "NONE". Wait, wat? Secondly, even autoconf scripts then have special cases for when variable is "NONE". It's even worse - whenever they need to construct new path based on prefix, they save its current value, substitute it with proper value (like /usr/local/), do what they need and then restore the original value. Le sigh. This mad behaviour is, however, suppressed if there's --prefix on the configure command line. Now, the problem is that we tried to construct path for wireshark plugin in the m4 file. Based on the knowledge described above, we shouldn't do it and just construct the full path in Makefile later where prefix has proper value. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- m4/virt-wireshark.m4 | 2 +- tools/Makefile.am | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/m4/virt-wireshark.m4 b/m4/virt-wireshark.m4 index f383e2b..e1e4a59 100644 --- a/m4/virt-wireshark.m4 +++ b/m4/virt-wireshark.m4 @@ -38,7 +38,7 @@ AC_DEFUN([LIBVIRT_CHECK_WIRESHARK],[ if test "x$ws_prefix" = "x" ; then ws_prefix="/usr"; fi - plugindir="${prefix}${plugindir#$ws_prefix}" + plugindir="${plugindir#$ws_prefix}" fi elif test "x$with_ws_plugindir" = "xno" || test "x$with_ws_plugindir" = "xyes"; then AC_MSG_ERROR([ws-plugindir must be used only with valid path]) diff --git a/tools/Makefile.am b/tools/Makefile.am index e7e42c3..08e1680 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -398,7 +398,7 @@ EXTRA_DIST += \ if WITH_WIRESHARK_DISSECTOR -ws_plugindir = $(plugindir) +ws_plugindir = $(prefix)$(plugindir) ws_plugin_LTLIBRARIES = wireshark/src/libvirt.la wireshark_src_libvirt_la_CPPFLAGS = \ -I wireshark/src $(WIRESHARK_DISSECTOR_CFLAGS) -- 2.8.4

4 6

[libvirt] [PATCH v11 0/5] Add native TLS encrypted chardev TCP support
by John Ferlan 26 Oct '16

26 Oct '16

v10: http://www.redhat.com/archives/libvir-list/2016-October/msg00866.html Changes since v10... This is essentially the adding the secret to/for the TLS chardev TCP. Lots of messy stuff. Patches 1-3 are new... In particular, I kept looking at the RNG removal code and kept thinking, something just doesn't look right when compared to the RemoveVirtioDisk code. If I'm incorrect with order of removal it's an easy enough change. Kept trying to think what depends on what and for the RNG it would seem a need a to unplug the backend before removing the front end object would be more logical. Patches 4-5 are the old patch 3-4. They (hopefully) encompass the previous review plus other things since discovered. John Ferlan (5): qemu: Move TLS object remove from DetachChr to RemoveChr qemu: Swap order of RNG hot unplug removal qemu: Need to remove TLS object in RemoveRNGDevice qemu: Add a secret object to/for a char source dev qemu: Add secret object hotplug for TCP chardev TLS src/qemu/qemu_command.c | 30 +++- src/qemu/qemu_command.h | 1 + src/qemu/qemu_domain.c | 173 +++++++++++++++++- src/qemu/qemu_domain.h | 17 +- src/qemu/qemu_driver.c | 6 +- src/qemu/qemu_hotplug.c | 195 +++++++++++++++++---- src/qemu/qemu_hotplug.h | 9 +- src/qemu/qemu_process.c | 9 +- tests/qemuhotplugtest.c | 2 +- ...xml2argv-serial-tcp-tlsx509-secret-chardev.args | 38 ++++ ...uxml2argv-serial-tcp-tlsx509-secret-chardev.xml | 50 ++++++ tests/qemuxml2argvtest.c | 17 ++ 12 files changed, 495 insertions(+), 52 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-serial-tcp-tlsx509-secret-chardev.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-serial-tcp-tlsx509-secret-chardev.xml -- 2.7.4

2 16