March 2015 - Devel - libvirt List Archives

[libvirt] Plans for next release
by Daniel Veillard 25 Mar '15

25 Mar '15

So we already have around 300 commits made since 1.2.13, and the end of the month is coming, so I guess we should enter freeze soon. I would suggest to do this friday to try to shoot for a 1.2.14 release by April 1st - always a great date to ship a release ! any opposition to this plan ? BTW as was pointed earlier by pkrempa we just went over 19,000 commits for libvirt. At the current rate we should hit 20k around july :-) Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 0

[libvirt] [PATCH] Makefile: fix typo
by Pavel Hrdina 25 Mar '15

25 Mar '15

Commit 95695388 introduced new util/virthreadjob.c/h files but the makefile has type that breaks rpm build. Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com> --- Pushed under build-breaker rule. src/Makefile.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/Makefile.am b/src/Makefile.am index ec62ab4..97e060a 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -160,7 +160,7 @@ UTIL_SOURCES = \ util/virsysinfo.c util/virsysinfo.h \ util/virsystemd.c util/virsystemd.h \ util/virthread.c util/virthread.h \ - util/virthreadjob.c utils/virthreadjob.h \ + util/virthreadjob.c util/virthreadjob.h \ util/virthreadpool.c util/virthreadpool.h \ util/virtime.h util/virtime.c \ util/virtpm.h util/virtpm.c \ -- 2.0.5

1 0

[libvirt] [PATCH] qemu: fix can use setmaxmem to change the maximum memory biger than max memory
by Luyao Huang 25 Mar '15

25 Mar '15

When call qemuDomainSetMemoryFlags() to change maximum memory size, we do not check if the maximum memory is biger than the max memory, this will make vm disappear after libvirtd restart if we set a big maximum memory. Add a check for this, also fix a typos issue. Signed-off-by: Luyao Huang <lhuang(a)redhat.com> --- src/conf/domain_conf.c | 2 +- src/qemu/qemu_driver.c | 7 +++++++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index d633f93..0d4b165 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -16646,7 +16646,7 @@ virDomainDefCheckABIStability(virDomainDefPtr src, if (src->mem.memory_slots != dst->mem.memory_slots) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("Target domain memory slots count '%u' doesn't match source '%u"), + _("Target domain memory slots count '%u' doesn't match source '%u'"), dst->mem.memory_slots, src->mem.memory_slots); goto error; } diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 3518dec..86b87af 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2324,6 +2324,13 @@ static int qemuDomainSetMemoryFlags(virDomainPtr dom, unsigned long newmem, "nodes cannot be modified with this API")); goto endjob; } + if (persistentDef->mem.max_memory && + persistentDef->mem.max_memory < newmem) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("cannot set maximum memory size biger than " + "the max memory size")); + goto endjob; + } virDomainDefSetMemoryInitial(persistentDef, newmem); -- 1.8.3.1

3 2

[libvirt] [PATCH 0/2] fix domain documentation error
by Chen Fan 25 Mar '15

25 Mar '15

Chen Fan (2): docs: no 'via' attribute in route element docs: route element must specify network address docs/formatdomain.html.in | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) -- 1.9.3

2 3

[libvirt] [PATCH 0/8] Make debugging of "cannot acquire state change lock" easier
by Jiri Denemark 25 Mar '15

25 Mar '15

https://bugzilla.redhat.com/show_bug.cgi?id=853839 Jiri Denemark (8): POTFILES.in: Sort Add support for tracking thread jobs Force usage of virThreadCreate virThread: Set thread job virThreadPool: Set thread worker name Set thread job for every RPC call qemu: Track the API which started the current job qemu: Add timing to domain jobs cfg.mk | 9 +++ daemon/remote.c | 1 + include/libvirt/virterror.h | 1 + po/POTFILES.in | 5 +- src/Makefile.am | 2 + src/libvirt_private.syms | 11 +++- src/locking/lock_daemon_dispatch.c | 1 + src/nwfilter/nwfilter_learnipaddr.c | 15 ++--- src/nwfilter/nwfilter_learnipaddr.h | 1 - src/qemu/qemu_domain.c | 60 ++++++++++++++--- src/qemu/qemu_domain.h | 4 ++ src/rpc/gendispatch.pl | 6 +- src/util/virerror.c | 1 + src/util/virthread.c | 25 +++++-- src/util/virthread.h | 13 ++-- src/util/virthreadjob.c | 126 ++++++++++++++++++++++++++++++++++++ src/util/virthreadjob.h | 33 ++++++++++ src/util/virthreadpool.c | 44 ++++++++----- src/util/virthreadpool.h | 14 ++-- 19 files changed, 317 insertions(+), 55 deletions(-) create mode 100644 src/util/virthreadjob.c create mode 100644 src/util/virthreadjob.h -- 2.3.3

2 10

[libvirt] [PATCH] qemu: add a check for node set when build memory device cmd
by Luyao Huang 25 Mar '15

25 Mar '15

When we set a host not exist nodemask in memory device and then start the vm, qemu will report error. # virsh start test3 error: Failed to start domain test3 error: internal error: process exited while connecting to monitor: 2015-03-25T01:12:17.205913Z qemu-kvm: -object memory-backend-ram,id=memdimm0 ,size=536870912,host-nodes=1-3,policy=bind: cannot bind memory to host NUMA nodes: Invalid argument We have some function to check this, and add a check when build memory cmd line will report error more early and clearly. And the check will be done when we start a vm have memory device and hotplug a memory device. The error will be: # virsh start test3 error: Failed to start domain test3 error: internal error: NUMA node 1 is unavailable Signed-off-by: Luyao Huang <lhuang(a)redhat.com> --- src/qemu/qemu_command.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 99a19d6..04c8df7 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -4757,6 +4757,8 @@ qemuBuildMemoryBackendStr(unsigned long long size, } if (nodemask) { + if (!virNumaNodesetIsAvailable(nodemask)) + goto cleanup; if (virJSONValueObjectAdd(props, "m:host-nodes", nodemask, "S:policy", qemuNumaPolicyTypeToString(mode), -- 1.8.3.1

2 1

[libvirt] [PATCH] Document that USB hostdevs do not need nodeDettach
by Ján Tomko 25 Mar '15

25 Mar '15

The virNodeDeviceDettach API only works on PCI devices. Originally added by commit 10d3272e, but the API never supported USB devices. Reported by: Martin Polednik <mpolednik(a)redhat.com> --- docs/formatdomain.html.in | 19 +++++++++---------- tools/virsh.pod | 17 ++++++++--------- 2 files changed, 17 insertions(+), 19 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 82aa14f..d6abe17 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -3114,21 +3114,20 @@ with additional attributes noted. <dl> <dt>usb</dt> - <dd>For USB devices, the user is responsible to call - <code>virNodeDeviceDettach</code> (or - <code>virsh nodedev-detach</code>) before starting the guest - or hot-plugging the device and <code>virNodeDeviceReAttach</code> - (or <code>virsh nodedev-reattach</code>) after hot-unplug or - stopping the guest. + <dd>USB devices are detached from the host on guest startup + and reattached after the guest exits or the device is + hot-unplugged. </dd> <dt>pci</dt> <dd>For PCI devices, when <code>managed</code> is "yes" it is detached from the host before being passed on to the guest and reattached to the host after the guest exits. If - <code>managed</code> is omitted or "no", follow the steps - described for a USB device to detach before starting the - guest or hot-plugging and reattach after stopping the guest - or hot-unplug. + <code>managed</code> is omitted or "no", the user is + responsible to call <code>virNodeDeviceDettach</code> + (or <code>virsh nodedev-detach</code> before starting the guest + or hot-plugging the device and <code>virNodeDeviceReAttach</code> + (or <code>virsh nodedev-reattach</code>) after hot-unplug or + stopping the guest. </dd> <dt>scsi</dt> <dd>For SCSI devices, user is responsible to make sure the device diff --git a/tools/virsh.pod b/tools/virsh.pod index 8262a45..4d825c1 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -2385,7 +2385,7 @@ attach taking effect the next time libvirt starts the domain. For cdrom and floppy devices, this command only replaces the media within an existing device; consider using B<update-device> for this usage. For passthrough host devices, see also B<nodedev-detach>, -needed if the device does not use managed mode. +needed if the PCI device does not use managed mode. If I<--live> is specified, affect a running domain. If I<--config> is specified, affect the next startup of a persistent domain. @@ -2646,15 +2646,14 @@ L<http://libvirt.org/formatnode.html>. Passthrough devices cannot be simultaneously used by the host and its guest domains, nor by multiple active guests at once. If the -<hostdev> description includes the attribute B<managed='yes'>, and the -hypervisor driver supports it, then the device is in managed mode, and +<hostdev> description of a PCI device includes the attribute B<managed='yes'>, +and the hypervisor driver supports it, then the device is in managed mode, and attempts to use that passthrough device in an active guest will automatically behave as if B<nodedev-detach> (guest start, device hot-plug) and B<nodedev-reattach> (guest stop, device hot-unplug) were -called at the right points (currently, qemu does this for PCI devices, -but not USB). If a device is not marked as managed, then it must -manually be detached before guests can use it, and manually reattached -to be returned to the host. Also, if a device is manually detached, +called at the right points. If a PCI device is not marked as managed, +then it must manually be detached before guests can use it, and manually +reattached to be returned to the host. Also, if a device is manually detached, then the host does not regain control of the device without a matching reattach, even if the guests use the device in managed mode. @@ -2712,8 +2711,8 @@ I<cap> and I<--tree> are mutually exclusive. Declare that I<nodedev> is no longer in use by any guests, and that the host can resume normal use of the device. This is done -automatically for devices in managed mode, but must be done explicitly -to match any explicit B<nodedev-detach>. +automatically for PCI devices in managed mode and USB devices, but +must be done explicitly to match any explicit B<nodedev-detach>. =item B<nodedev-reset> I<nodedev> -- 2.0.5

2 2

[libvirt] [PATCH] rpc: serverclient: Clear pointer with NULL instead of 0
by Peter Krempa 25 Mar '15

25 Mar '15

--- Notes: Pushed as trivial. src/rpc/virnetserverclient.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rpc/virnetserverclient.c b/src/rpc/virnetserverclient.c index f5259c2..d36e80c 100644 --- a/src/rpc/virnetserverclient.c +++ b/src/rpc/virnetserverclient.c @@ -721,7 +721,7 @@ virNetServerClientCreateIdentity(virNetServerClientPtr client) error: virObjectUnref(ret); - ret = 0; + ret = NULL; goto cleanup; } -- 2.2.2

1 0

[libvirt] [PATCH 00/10] libxl: switch driver to use a single libxl_ctx
by Jim Fehlig 24 Mar '15

24 Mar '15

This series is a follow up to https://www.redhat.com/archives/libvir-list/2015-February/msg00024.html It goes a step further and changes the libxl driver to use one, driver-wide libxl_ctx. Currently the libxl driver has one driver-wide ctx for operations that are not domain-specific and a ctx for each domain. This approach was necessary back in the old Xen4.1 libxl days, but with the newer libxl it is more of a hinderance than benefit. Ian Jackson suggested moving to a single ctx while discussing some deadlocks and assertions encountered in the libxl driver when under load from tests such as OpenStack Tempest. Making such a change involves quite a bit of code movement. I've tried to split that up into a reviewable series, the result of which are the 9 patches that follow. I've ran this through all of my automated tests as well as some hacky tests I created to reproduce failures revealed by Tempest. One downside of moving to a single ctx is losing the per-domain log files. Currently, a single log stream can be associated with ctx, hence all logging from libxl will go to a single file. Ian is going to investigate possibilities to accommodate per-domain log files in libxl, but in the meantime folks using Xen are accustomed to a single log file from the xend days. I've been testing this series on xen-unstable and Xen 4.4.1 + commits 2ffeb5d7, 4b9143e4, 5a968257, 60ce518a, 66bff9fd, 77a1bf37, f49f9b41, 6b5a5bba, 93699882d, f1335f0d, and 8bc64413. Results are much better than before applying the series, but I do notice a stuck hypercall after many (hundreds) concurrent domain create/destroy operations. The single libxl_ctx is locked in the callpath, essentially deadlocking the driver. Thread 1 (Thread 0x7f0649a198c0 (LWP 2235)): 0 0x00007f0645272397 in ioctl () from /lib64/libc.so.6 1 0x00007f0645d8e353 in linux_privcmd_hypercall (xch=<optimized out>, h=<optimized out>, hypercall=<optimized out>) at xc_linux_osdep.c:134 2 0x00007f0645d854b8 in do_xen_hypercall (xch=xch@entry=0x7f0630039390, hypercall=hypercall@entry=0x7fffd53f80e0) at xc_private.c:249 3 0x00007f0645d86aa4 in do_sysctl (sysctl=sysctl@entry=0x7fffd53f8080, xch=xch@entry=0x7f0630039390) at xc_private.h:281 4 xc_sysctl (xch=xch@entry=0x7f0630039390, sysctl=sysctl@entry=0x7fffd53f8170) at xc_private.c:656 5 0x00007f0645d7bfbf in xc_domain_getinfolist (xch=0x7f0630039390, first_domain=first_domain@entry=119, max_domains=max_domains@entry=1, info=info@entry=0x7fffd53f8260) at xc_domain.c:382 6 0x00007f0645fabca6 in domain_death_xswatch_callback (egc=0x7fffd53f83f0, w=<optimized out>, wpath=<optimized out>, epath=<optimized out>) at libxl.c:1041 7 0x00007f0645fd75a8 in watchfd_callback (egc=0x7fffd53f83f0, ev=<optimized out>, fd=<optimized out>, events=<optimized out>, revents=<optimized out>) at libxl_event.c:515 8 0x00007f0645fd8ac3 in libxl_osevent_occurred_fd (ctx=<optimized out>, for_libxl=<optimized out>, fd=<optimized out>, events_ign=<optimized out>, revents_ign=<optimized out>) at libxl_event.c:1259 9 0x00007f063a23402c in libxlFDEventCallback (watch=454, fd=33, vir_events=1, fd_info=0x7f0608007e70) at libxl/libxl_driver.c:123 There is no hint in any logs or dmesg suggesting a cause for the stuck hypercall. Any suggestions for further debugging tips appreciated. Jim Fehlig (10): libxl: remove redundant calls to libxl_evdisable_domain_death libxl: use libxl_ctx passed to libxlConsoleCallback libxl: use driver-wide ctx in fd and timer event handling libxl: Move setup of child processing code to driver initialization libxl: move event registration to driver initialization libxl: use global libxl_ctx in event handler libxl: remove unnecessary libxlDomainEventsRegister libxl: make libxlDomainFreeMem static libxl: remove per-domain libxl_ctx libxl: change libxl log stream to ERROR log level src/libxl/libxl_conf.c | 2 +- src/libxl/libxl_domain.c | 438 ++++++--------------------------------- src/libxl/libxl_domain.h | 27 +-- src/libxl/libxl_driver.c | 484 +++++++++++++++++++++++++++++++------------- src/libxl/libxl_migration.c | 17 +- 5 files changed, 426 insertions(+), 542 deletions(-) -- 1.8.4.5

2 12

[libvirt] [PATCH] libxl: fix dom0 balloon logic
by Jim Fehlig 24 Mar '15

24 Mar '15

Recent testing on large memory systems revealed a bug in the Xen xl tool's freemem() function. When autoballooning is enabled, freemem() is used to ensure enough memory is available to start a domain, ballooning dom0 if necessary. When ballooning large amounts of memory from dom0, freemem() would exceed its self-imposed wait time and return an error. Meanwhile, dom0 continued to balloon. Starting the domain later, after sufficient memory was ballooned from dom0, would succeed. The libvirt implementation in libxlDomainFreeMem() suffers the same bug since it is modeled after freemem(). In the end, the best place to fix the bug on the Xen side was to slightly change the behavior of libxl_wait_for_memory_target(). Instead of failing after caller-provided wait_sec, the function now blocks as long as dom0 memory ballooning is progressing. It will return failure only when more memory is needed to reach the target and wait_sec have expired with no progress being made. See xen.git commit fd3aa246. There was a dicussion on how this would affect other libxl apps like libvirt http://lists.xen.org/archives/html/xen-devel/2015-03/msg00739.html If libvirt containing this patch was build against a Xen containing the old libxl_wait_for_memory_target() behavior, libxlDomainFreeMem() will fail after 30 sec and domain creation will be terminated. Without this patch and with old libxl_wait_for_memory_target() behavior, libxlDomainFreeMem() does not succeed after 30 sec, but returns success anyway. Domain creation continues resulting in all sorts of fun stuff like cpu soft lockups in the guest OS. It was decided to properly fix libxl_wait_for_memory_target(), and if anything improve the default behavior of apps using the freemem reference impl in xl. xl was patched to accommodate the change in libxl_wait_for_memory_target() with xen.git commit 883b30a0. This patch does the same in the libxl driver. While at it, I changed the logic to essentially match freemem() in $xensrc/tools/libxl/xl_cmdimpl.c. It was a bit cleaner IMO and will make it easier to spot future, potentially interesting divergences. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- src/libxl/libxl_domain.c | 57 ++++++++++++++++++++++++------------------------ 1 file changed, 28 insertions(+), 29 deletions(-) diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 407a9bd..ff78133 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -1121,38 +1121,41 @@ libxlDomainFreeMem(libxlDomainObjPrivatePtr priv, libxl_domain_config *d_config) { uint32_t needed_mem; uint32_t free_mem; - size_t i; - int ret = -1; + int ret; int tries = 3; int wait_secs = 10; - if ((ret = libxl_domain_need_memory(priv->ctx, &d_config->b_info, - &needed_mem)) >= 0) { - for (i = 0; i < tries; ++i) { - if ((ret = libxl_get_free_memory(priv->ctx, &free_mem)) < 0) - break; + ret = libxl_domain_need_memory(priv->ctx, &d_config->b_info, + &needed_mem); + if (ret < 0) + goto error; - if (free_mem >= needed_mem) { - ret = 0; - break; - } + do { + ret = libxl_get_free_memory(priv->ctx, &free_mem); + if (ret < 0) + goto error; - if ((ret = libxl_set_memory_target(priv->ctx, 0, - free_mem - needed_mem, - /* relative */ 1, 0)) < 0) - break; + if (free_mem >= needed_mem) + return 0; - ret = libxl_wait_for_free_memory(priv->ctx, 0, needed_mem, - wait_secs); - if (ret == 0 || ret != ERROR_NOMEM) - break; + ret = libxl_set_memory_target(priv->ctx, 0, + free_mem - needed_mem, + /* relative */ 1, 0); + if (ret < 0) + goto error; - if ((ret = libxl_wait_for_memory_target(priv->ctx, 0, 1)) < 0) - break; - } - } + ret = libxl_wait_for_free_memory(priv->ctx, 0, needed_mem, + wait_secs); + if (ret < 0) + goto error; - return ret; + tries--; + } while (tries > 0); + + error: + virReportSystemError(ret, "%s", + _("Failed to balloon domain0 memory")); + return -1; } static void @@ -1271,12 +1274,8 @@ libxlDomainStart(libxlDriverPrivatePtr driver, virDomainObjPtr vm, priv->ctx, &d_config) < 0) goto endjob; - if (cfg->autoballoon && libxlDomainFreeMem(priv, &d_config) < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("libxenlight failed to get free memory for domain '%s'"), - d_config.c_info.name); + if (cfg->autoballoon && libxlDomainFreeMem(priv, &d_config) < 0) goto endjob; - } if (virHostdevPrepareDomainDevices(hostdev_mgr, LIBXL_DRIVER_NAME, vm->def, VIR_HOSTDEV_SP_PCI) < 0) -- 1.8.4.5

3 3