- Devel - libvirt List Archives

[libvirt] [PATCH libguestfs v3] lib: Handle slow USB devices more gracefully.
by Richard W.M. Jones 19 Jan '16

19 Jan '16

Libvirt has a fixed 15 second timeout for qemu to exit. If qemu is writing to a slow USB key, it can hang (in D state) for much longer than this - many minutes usually. The solution is to check specifically for the libvirt EBUSY error when this happens, and retry the virDomainDestroyFlags operation (indefinitely). See also the description here: https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html Similar to the following OpenStack Nova commit: http://git.openstack.org/cgit/openstack/nova/commit/?id=3907867 Thanks: Kashyap Chamarthy and Daniel Berrange. --- src/launch-libvirt.c | 45 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 35 insertions(+), 10 deletions(-) diff --git a/src/launch-libvirt.c b/src/launch-libvirt.c index 8215e02..90b6c49 100644 --- a/src/launch-libvirt.c +++ b/src/launch-libvirt.c @@ -25,6 +25,7 @@ #include <unistd.h> #include <fcntl.h> #include <grp.h> +#include <errno.h> #include <sys/types.h> #include <sys/stat.h> #include <assert.h> @@ -2015,6 +2016,8 @@ ignore_errors (void *ignore, virErrorPtr ignore2) /* empty */ } +static int destroy_domain (guestfs_h *g, virDomainPtr dom, int check_for_errors); + static int shutdown_libvirt (guestfs_h *g, void *datav, int check_for_errors) { @@ -2023,23 +2026,14 @@ shutdown_libvirt (guestfs_h *g, void *datav, int check_for_errors) virDomainPtr dom = data->dom; size_t i; int ret = 0; - int flags; /* Note that we can be called back very early in launch (specifically * from launch_libvirt itself), when conn and dom might be NULL. */ - if (dom != NULL) { - flags = check_for_errors ? VIR_DOMAIN_DESTROY_GRACEFUL : 0; - debug (g, "calling virDomainDestroy \"%s\" flags=%s", - data->name, check_for_errors ? "VIR_DOMAIN_DESTROY_GRACEFUL" : "0"); - if (virDomainDestroyFlags (dom, flags) == -1) { - libvirt_error (g, _("could not destroy libvirt domain")); - ret = -1; - } + ret = destroy_domain (g, dom, check_for_errors); virDomainFree (dom); } - if (conn != NULL) virConnectClose (conn); @@ -2068,6 +2062,37 @@ shutdown_libvirt (guestfs_h *g, void *datav, int check_for_errors) return ret; } +/* Wrapper around virDomainDestroy which handles errors and retries.. */ +static int +destroy_domain (guestfs_h *g, virDomainPtr dom, int check_for_errors) +{ + const int flags = check_for_errors ? VIR_DOMAIN_DESTROY_GRACEFUL : 0; + virErrorPtr err; + + again: + debug (g, "calling virDomainDestroy flags=%s", + check_for_errors ? "VIR_DOMAIN_DESTROY_GRACEFUL" : "0"); + if (virDomainDestroyFlags (dom, flags) == -1) { + err = virGetLastError (); + + /* Second chance if we're just waiting for qemu to shut down. See: + * https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html + */ + if ((flags & VIR_DOMAIN_DESTROY_GRACEFUL) && + err && err->code == VIR_ERR_SYSTEM_ERROR && err->int1 == EBUSY) + goto again; + + /* "Domain not found" is not treated as an error. */ + if (err && err->code == VIR_ERR_NO_DOMAIN) + return 0; + + libvirt_error (g, _("could not destroy libvirt domain")); + return -1; + } + + return 0; +} + /* Wrapper around error() which produces better errors for * libvirt functions. */ -- 2.5.0

2 3

[libvirt] Failed to terminate process 1275 with SIGTERM: Device or resource busy
by Richard W.M. Jones 19 Jan '16

19 Jan '16

I didn't file a bug about this because it's not really clear what the bug is. Anyway, here goes ... $ virt-builder fedora-23 -o /dev/sdX [ 2.3] Downloading: http://libguestfs.org/download/builder/fedora-23.xz [ 3.0] Planning how to build this image [ 3.0] Uncompressing [ 18.1] Resizing (using virt-resize) to expand the disk to 14.9G virt-resize: error: libguestfs error: could not destroy libvirt domain: Failed to terminate process 1275 with SIGTERM: Device or resource busy [code=38 domain=0] $ ps ax | grep qemu 1275 ? Dl 0:49 /usr/bin/qemu-system-x86_64 -machine accel=kvm [...] What's actually happening here is that /dev/sdX is an incredibly slow[1] USB key. The virt-resize operation completes successfully, but qemu is blocked writing/fsyncing to the USB key, and takes a few minutes to exit. libvirt has a loop (src/util/virprocess.c:virProcessKillPainfully) which waits 15 seconds [comment says 10 seconds], and this wait is not configurable. There seem to be several possible solutions, none of them very nice: - Hard-code a longer wait in libvirt. - Make the wait in libvirt configurable. - If the process is in 'D' state, wait indefinitely. I tried another workaround which was to get virt-resize to fsync the output file before closing the libvirt connection, but that doesn't work for reasons I don't understand so far - still studying this. Rich. [1] "Incredibly slow" ... but it's just a regular USB key, they are all very slow! -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

3 8

[libvirt] [PATCH 00/16] Implement post-copy migration
by Jiri Denemark 19 Jan '16

19 Jan '16

(See "Add public APIs for post-copy migration" patch for more details about post-copy migration.) Post-copy support was originally written by Cristian Klein in 2014, but no one touched the series since then. Some patches in this series are modified versions of the old patches from Cristian, some patches had to be rewritten from scratch since libvirt code changed a lot (we started using migration events), and some patches are completely new. While post-copy migration is included in QEMU 2.5.0, it didn't support everything libvirt needs. Thus you need QEMU from git to use post-copy. Luckily, the QEMU migration capability we need to enable to use post-copy is still prefixed with "x-", which means it's still considered experimental. Once we are sure QEMU gives us all we need, the "x-" prefix will be removed. Seamless SPICE migration doesn't work with post-copy now, but I'll look at that. This series (VIR_MIGRATE_POSTCOPY_AFTER_PRECOPY support) depends on "Introduce migration iteration event" series I sent yesterday. Cristian Klein (6): Add public APIs for post-copy migration qemu: Add QMP functions for post-copy migration qemu: Add support for VIR_MIGRATE_POSTCOPY flag qemu: Implement virDomainMigrateStartPostCopy virsh: Add support for post-copy migration virsh: Add --postcopy-after-precopy option to migrate Jiri Denemark (10): Add event and state details for post-copy qemu: Don't explicitly stop CPUs after migration qemu: Handle postcopy-active migration state qemu: Add flags to qemuMigrationWaitForCompletion qemu: Add support for VIR_MIGRATE_POSTCOPY_AFTER_PRECOPY flag virsh: Configurable migrate --timeout action qemu: Don't kill running migrated domain on daemon restart qemu: Refactor qemuProcessRecoverMigration qemu: Handle post-copy migration failures qemu: Refuse to abort migration in post-copy mode examples/object-events/event-test.c | 9 ++ include/libvirt/libvirt-domain.h | 12 ++ src/conf/domain_conf.c | 7 +- src/driver-hypervisor.h | 5 + src/libvirt-domain.c | 140 +++++++++++++++++ src/libvirt_public.syms | 4 + src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 3 + src/qemu/qemu_driver.c | 72 ++++++++- src/qemu/qemu_migration.c | 299 +++++++++++++++++++++++++++++++----- src/qemu/qemu_migration.h | 7 +- src/qemu/qemu_monitor.c | 16 +- src/qemu/qemu_monitor.h | 4 + src/qemu/qemu_monitor_json.c | 23 +++ src/qemu/qemu_monitor_json.h | 3 + src/qemu/qemu_process.c | 252 +++++++++++++++++++----------- src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 13 +- src/remote_protocol-structs | 5 + tools/virsh-domain-monitor.c | 7 +- tools/virsh-domain.c | 131 ++++++++++++++-- tools/virsh.pod | 31 +++- 22 files changed, 884 insertions(+), 161 deletions(-) -- 2.7.0

2 18

[libvirt] [PATCH] leaseshelper: fix crash when no mac is specified
by Ján Tomko 19 Jan '16

19 Jan '16

If dnsmasq specified DNSMASQ_IAID (so we're dealing with an IPv6 lease) but no DNSMASQ_MAC, we skip creation of the new lease object. Also skip adding it to the leases array. https://bugzilla.redhat.com/show_bug.cgi?id=1202350 https://bugzilla.redhat.com/show_bug.cgi?id=1292941 --- src/network/leaseshelper.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/network/leaseshelper.c b/src/network/leaseshelper.c index 2d528f7..6930310 100644 --- a/src/network/leaseshelper.c +++ b/src/network/leaseshelper.c @@ -439,7 +439,7 @@ main(int argc, char **argv) case VIR_LEASE_ACTION_OLD: case VIR_LEASE_ACTION_ADD: - if (virJSONValueArrayAppend(leases_array_new, lease_new) < 0) { + if (lease_new && virJSONValueArrayAppend(leases_array_new, lease_new) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("failed to create json")); goto cleanup; -- 2.4.6

2 1

[libvirt] [PATCH 0/6] Refactor leaseshelper
by Ján Tomko 19 Jan '16

19 Jan '16

Split out some functions to make main() more readable. May depend on https://www.redhat.com/archives/libvir-list/2016-January/msg00595.html No functional change intented. Ján Tomko (6): leaseshelper: store server_duid as an allocated string leaseshelper: split out custom leases file read leaseshelper: split out virLeasePrintLeases leaseshelper: remove useless comparison leaseshelper: split out virLeaseNewFromArgv leaseshelper: reduce indentation level in virLeaseReadCustomLeaseFile src/network/leaseshelper.c | 500 +++++++++++++++++++++++++-------------------- 1 file changed, 284 insertions(+), 216 deletions(-) -- 2.4.6

2 12

[libvirt] [PATCH] cpu_map.xml: Expose svm flag for nested kvm on AMD CPU model
by Lin Ma 19 Jan '16

19 Jan '16

QEMU commit 75d373e disables svm by default since PC machine 2.2. So with PC machine version > 2.1, When a user wants to expose the svm flag to a guest, says: ...... <feature policy='require' name='svm'/> ...... The svm flag wont be passed to qemu command line, This patch fixes it. Signed-off-by: Lin Ma <lma(a)suse.com> --- src/cpu/cpu_map.xml | 6 ------ 1 file changed, 6 deletions(-) diff --git a/src/cpu/cpu_map.xml b/src/cpu/cpu_map.xml index 0b6d424..8f96843 100644 --- a/src/cpu/cpu_map.xml +++ b/src/cpu/cpu_map.xml @@ -686,7 +686,6 @@ <feature name='sep'/> <feature name='sse'/> <feature name='sse2'/> - <feature name='svm'/> <feature name='syscall'/> <feature name='tsc'/> </model> @@ -1195,7 +1194,6 @@ <feature name='sep'/> <feature name='sse'/> <feature name='sse2'/> - <feature name='svm'/> <feature name='syscall'/> <feature name='tsc'/> </model> @@ -1257,7 +1255,6 @@ <feature name='sep'/> <feature name='sse'/> <feature name='sse2'/> - <feature name='svm'/> <feature name='syscall'/> <feature name='tsc'/> </model> @@ -1295,7 +1292,6 @@ <feature name='sse'/> <feature name='sse2'/> <feature name='sse4a'/> - <feature name='svm'/> <feature name='syscall'/> <feature name='tsc'/> </model> @@ -1341,7 +1337,6 @@ <feature name='sse4.2'/> <feature name='sse4a'/> <feature name='ssse3'/> - <feature name='svm'/> <feature name='syscall'/> <feature name='tsc'/> <feature name='xop'/> @@ -1391,7 +1386,6 @@ <feature name='sse4.2'/> <feature name='sse4a'/> <feature name='ssse3'/> - <feature name='svm'/> <feature name='syscall'/> <feature name='tbm'/> <feature name='tsc'/> -- 2.1.4

2 1

[libvirt] [PATCH v2] resize, builder: fsync the output file before closing the libvirt connection.
by Richard W.M. Jones 18 Jan '16

18 Jan '16

Libvirt has a fixed 15 second timeout for qemu to exit. If qemu is writing to a slow USB key, it can hang (in D state) for much longer than this - many minutes usually. To work around this, fsync the output file before closing the libvirt connection so that qemu shouldn't have anything (much) to write. We have to do this in a few places in the code. This is a hack - it would be better to find a way to fix this in libvirt. See also the description here: https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html --- builder/builder.ml | 8 +++++++- resize/resize.ml | 10 ++++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/builder/builder.ml b/builder/builder.ml index 1f9a472..093982d 100644 --- a/builder/builder.ml +++ b/builder/builder.ml @@ -722,7 +722,13 @@ let main () = (* Unmount everything and we're done! *) message (f_"Finishing off"); - g#umount_all (); + if cmdline.sync then ( + (* Work around libvirt 15 second timeout waiting for qemu + https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html *) + g#sync (); + Fsync.file output_filename + ); + g#shutdown (); g#close (); diff --git a/resize/resize.ml b/resize/resize.ml index d6dd9a5..84e38be 100644 --- a/resize/resize.ml +++ b/resize/resize.ml @@ -1283,6 +1283,11 @@ read the man page virt-resize(1). let g = if to_be_expanded then ( + (* Work around libvirt 15 second timeout waiting for qemu + https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html *) + g#sync (); + Fsync.file outfile; + g#shutdown (); g#close (); @@ -1354,6 +1359,11 @@ read the man page virt-resize(1). ) lvs ); + (* Work around libvirt 15 second timeout waiting for qemu + https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html *) + g#sync (); + Fsync.file outfile; + (* Finished. Unmount disks and exit. *) g#shutdown (); g#close (); -- 2.5.0

1 0

[libvirt] [PATCH 0/2] Fix log related crasher
by Michal Privoznik 18 Jan '16

18 Jan '16

*** BLURB HERE *** Michal Privoznik (2): qemuProcessReadLog: Fix memcpy arguments virLogManagerDomainReadLogFile: Don't do dummy allocs src/logging/log_manager.c | 4 +--- src/qemu/qemu_process.c | 2 +- 2 files changed, 2 insertions(+), 4 deletions(-) -- 2.4.10

3 6

[libvirt] [PATCH v2] qemuTestDriverInit: fill driver with zeroes
by Michal Privoznik 18 Jan '16

18 Jan '16

In the commit aea47e48c473a we have fixed a single pointer within driver structure. Since all callers pass statically allocated driver on stack other pointers within driver may contain random values too. Before touching it lets overwrite it with zeroes and thus fix all dangling pointers. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- diff to v1: -Jirka's review worked in tests/testutilsqemu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index f2eacdd..ae69a18 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -555,11 +555,11 @@ int qemuTestCapsCacheInsert(virQEMUCapsCachePtr cache, const char *binary, int qemuTestDriverInit(virQEMUDriver *driver) { + memset(driver, 0, sizeof(*driver)); + if (virMutexInit(&driver->lock) < 0) return -1; - driver->securityManager = NULL; - driver->config = virQEMUDriverConfigNew(false); if (!driver->config) goto error; -- 2.4.10

3 2

[libvirt] [python PATCH v2] setup: Use cflags and ldflags properly
by Jiri Denemark 18 Jan '16

18 Jan '16

The setup.py script reads cflags and ldflags from pkg-config and uses them when compiling/linking C modules. Since both cflags and ldflags may include multiple compiler arguments we need to split them rather than concatenating them into a single argument. Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- Notes: Version 2: - fix all modules I'm surprised it ever worked. setup.py | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-) diff --git a/setup.py b/setup.py index 9bf583b..c44f84a 100755 --- a/setup.py +++ b/setup.py @@ -88,17 +88,15 @@ def get_module_lists(): c_modules = [] py_modules = [] - ldflags = get_pkgconfig_data(["--libs-only-L"], "libvirt", False) - cflags = get_pkgconfig_data(["--cflags"], "libvirt", False) + ldflags = get_pkgconfig_data(["--libs-only-L"], "libvirt", False).split() + cflags = get_pkgconfig_data(["--cflags"], "libvirt", False).split() module = Extension('libvirtmod', sources = ['libvirt-override.c', 'build/libvirt.c', 'typewrappers.c', 'libvirt-utils.c'], libraries = [ "virt" ], include_dirs = [ "." ]) - if cflags != "": - module.extra_compile_args.append(cflags) - if ldflags != "": - module.extra_link_args.append(ldflags) + module.extra_compile_args.extend(cflags) + module.extra_link_args.extend(ldflags) c_modules.append(module) py_modules.append("libvirt") @@ -107,10 +105,8 @@ def get_module_lists(): sources = ['libvirt-qemu-override.c', 'build/libvirt-qemu.c', 'typewrappers.c', 'libvirt-utils.c'], libraries = [ "virt-qemu" ], include_dirs = [ "." ]) - if cflags != "": - moduleqemu.extra_compile_args.append(cflags) - if ldflags != "": - moduleqemu.extra_link_args.append(ldflags) + moduleqemu.extra_compile_args.extend(cflags) + moduleqemu.extra_link_args.extend(ldflags) c_modules.append(moduleqemu) py_modules.append("libvirt_qemu") @@ -120,10 +116,8 @@ def get_module_lists(): sources = ['libvirt-lxc-override.c', 'build/libvirt-lxc.c', 'typewrappers.c', 'libvirt-utils.c'], libraries = [ "virt-lxc" ], include_dirs = [ "." ]) - if cflags != "": - modulelxc.extra_compile_args.append(cflags) - if ldflags != "": - modulelxc.extra_link_args.append(ldflags) + modulelxc.extra_compile_args.extend(cflags) + modulelxc.extra_link_args.extend(ldflags) c_modules.append(modulelxc) py_modules.append("libvirt_lxc") -- 2.7.0

2 1