[libvirt] [PATCH libguestfs v3] lib: Handle slow USB devices more gracefully.
by Richard W.M. Jones
Libvirt has a fixed 15 second timeout for qemu to exit. If qemu is
writing to a slow USB key, it can hang (in D state) for much longer
than this - many minutes usually.
The solution is to check specifically for the libvirt EBUSY error when
this happens, and retry the virDomainDestroyFlags operation
(indefinitely).
See also the description here:
https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html
Similar to the following OpenStack Nova commit:
http://git.openstack.org/cgit/openstack/nova/commit/?id=3907867
Thanks: Kashyap Chamarthy and Daniel Berrange.
---
src/launch-libvirt.c | 45 +++++++++++++++++++++++++++++++++++----------
1 file changed, 35 insertions(+), 10 deletions(-)
diff --git a/src/launch-libvirt.c b/src/launch-libvirt.c
index 8215e02..90b6c49 100644
--- a/src/launch-libvirt.c
+++ b/src/launch-libvirt.c
@@ -25,6 +25,7 @@
#include <unistd.h>
#include <fcntl.h>
#include <grp.h>
+#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <assert.h>
@@ -2015,6 +2016,8 @@ ignore_errors (void *ignore, virErrorPtr ignore2)
/* empty */
}
+static int destroy_domain (guestfs_h *g, virDomainPtr dom, int check_for_errors);
+
static int
shutdown_libvirt (guestfs_h *g, void *datav, int check_for_errors)
{
@@ -2023,23 +2026,14 @@ shutdown_libvirt (guestfs_h *g, void *datav, int check_for_errors)
virDomainPtr dom = data->dom;
size_t i;
int ret = 0;
- int flags;
/* Note that we can be called back very early in launch (specifically
* from launch_libvirt itself), when conn and dom might be NULL.
*/
-
if (dom != NULL) {
- flags = check_for_errors ? VIR_DOMAIN_DESTROY_GRACEFUL : 0;
- debug (g, "calling virDomainDestroy \"%s\" flags=%s",
- data->name, check_for_errors ? "VIR_DOMAIN_DESTROY_GRACEFUL" : "0");
- if (virDomainDestroyFlags (dom, flags) == -1) {
- libvirt_error (g, _("could not destroy libvirt domain"));
- ret = -1;
- }
+ ret = destroy_domain (g, dom, check_for_errors);
virDomainFree (dom);
}
-
if (conn != NULL)
virConnectClose (conn);
@@ -2068,6 +2062,37 @@ shutdown_libvirt (guestfs_h *g, void *datav, int check_for_errors)
return ret;
}
+/* Wrapper around virDomainDestroy which handles errors and retries.. */
+static int
+destroy_domain (guestfs_h *g, virDomainPtr dom, int check_for_errors)
+{
+ const int flags = check_for_errors ? VIR_DOMAIN_DESTROY_GRACEFUL : 0;
+ virErrorPtr err;
+
+ again:
+ debug (g, "calling virDomainDestroy flags=%s",
+ check_for_errors ? "VIR_DOMAIN_DESTROY_GRACEFUL" : "0");
+ if (virDomainDestroyFlags (dom, flags) == -1) {
+ err = virGetLastError ();
+
+ /* Second chance if we're just waiting for qemu to shut down. See:
+ * https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html
+ */
+ if ((flags & VIR_DOMAIN_DESTROY_GRACEFUL) &&
+ err && err->code == VIR_ERR_SYSTEM_ERROR && err->int1 == EBUSY)
+ goto again;
+
+ /* "Domain not found" is not treated as an error. */
+ if (err && err->code == VIR_ERR_NO_DOMAIN)
+ return 0;
+
+ libvirt_error (g, _("could not destroy libvirt domain"));
+ return -1;
+ }
+
+ return 0;
+}
+
/* Wrapper around error() which produces better errors for
* libvirt functions.
*/
--
2.5.0
8 years, 11 months
[libvirt] Failed to terminate process 1275 with SIGTERM: Device or resource busy
by Richard W.M. Jones
I didn't file a bug about this because it's not really clear what the
bug is. Anyway, here goes ...
$ virt-builder fedora-23 -o /dev/sdX
[ 2.3] Downloading: http://libguestfs.org/download/builder/fedora-23.xz
[ 3.0] Planning how to build this image
[ 3.0] Uncompressing
[ 18.1] Resizing (using virt-resize) to expand the disk to 14.9G
virt-resize: error: libguestfs error: could not destroy libvirt
domain: Failed to terminate process 1275 with SIGTERM: Device or
resource busy [code=38 domain=0]
$ ps ax | grep qemu
1275 ? Dl 0:49 /usr/bin/qemu-system-x86_64 -machine accel=kvm [...]
What's actually happening here is that /dev/sdX is an incredibly
slow[1] USB key. The virt-resize operation completes successfully,
but qemu is blocked writing/fsyncing to the USB key, and takes a few
minutes to exit.
libvirt has a loop (src/util/virprocess.c:virProcessKillPainfully)
which waits 15 seconds [comment says 10 seconds], and this wait is not
configurable.
There seem to be several possible solutions, none of them very nice:
- Hard-code a longer wait in libvirt.
- Make the wait in libvirt configurable.
- If the process is in 'D' state, wait indefinitely.
I tried another workaround which was to get virt-resize to fsync the
output file before closing the libvirt connection, but that doesn't
work for reasons I don't understand so far - still studying this.
Rich.
[1] "Incredibly slow" ... but it's just a regular USB key, they are
all very slow!
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
8 years, 11 months
[libvirt] [PATCH 00/16] Implement post-copy migration
by Jiri Denemark
(See "Add public APIs for post-copy migration" patch for more details
about post-copy migration.)
Post-copy support was originally written by Cristian Klein in 2014, but
no one touched the series since then. Some patches in this series are
modified versions of the old patches from Cristian, some patches had to
be rewritten from scratch since libvirt code changed a lot (we started
using migration events), and some patches are completely new.
While post-copy migration is included in QEMU 2.5.0, it didn't support
everything libvirt needs. Thus you need QEMU from git to use post-copy.
Luckily, the QEMU migration capability we need to enable to use
post-copy is still prefixed with "x-", which means it's still considered
experimental. Once we are sure QEMU gives us all we need, the "x-"
prefix will be removed.
Seamless SPICE migration doesn't work with post-copy now, but I'll look
at that.
This series (VIR_MIGRATE_POSTCOPY_AFTER_PRECOPY support) depends on
"Introduce migration iteration event" series I sent yesterday.
Cristian Klein (6):
Add public APIs for post-copy migration
qemu: Add QMP functions for post-copy migration
qemu: Add support for VIR_MIGRATE_POSTCOPY flag
qemu: Implement virDomainMigrateStartPostCopy
virsh: Add support for post-copy migration
virsh: Add --postcopy-after-precopy option to migrate
Jiri Denemark (10):
Add event and state details for post-copy
qemu: Don't explicitly stop CPUs after migration
qemu: Handle postcopy-active migration state
qemu: Add flags to qemuMigrationWaitForCompletion
qemu: Add support for VIR_MIGRATE_POSTCOPY_AFTER_PRECOPY flag
virsh: Configurable migrate --timeout action
qemu: Don't kill running migrated domain on daemon restart
qemu: Refactor qemuProcessRecoverMigration
qemu: Handle post-copy migration failures
qemu: Refuse to abort migration in post-copy mode
examples/object-events/event-test.c | 9 ++
include/libvirt/libvirt-domain.h | 12 ++
src/conf/domain_conf.c | 7 +-
src/driver-hypervisor.h | 5 +
src/libvirt-domain.c | 140 +++++++++++++++++
src/libvirt_public.syms | 4 +
src/qemu/qemu_domain.c | 1 +
src/qemu/qemu_domain.h | 3 +
src/qemu/qemu_driver.c | 72 ++++++++-
src/qemu/qemu_migration.c | 299 +++++++++++++++++++++++++++++++-----
src/qemu/qemu_migration.h | 7 +-
src/qemu/qemu_monitor.c | 16 +-
src/qemu/qemu_monitor.h | 4 +
src/qemu/qemu_monitor_json.c | 23 +++
src/qemu/qemu_monitor_json.h | 3 +
src/qemu/qemu_process.c | 252 +++++++++++++++++++-----------
src/remote/remote_driver.c | 1 +
src/remote/remote_protocol.x | 13 +-
src/remote_protocol-structs | 5 +
tools/virsh-domain-monitor.c | 7 +-
tools/virsh-domain.c | 131 ++++++++++++++--
tools/virsh.pod | 31 +++-
22 files changed, 884 insertions(+), 161 deletions(-)
--
2.7.0
8 years, 11 months
[libvirt] [PATCH] leaseshelper: fix crash when no mac is specified
by Ján Tomko
If dnsmasq specified DNSMASQ_IAID (so we're dealing with an IPv6
lease) but no DNSMASQ_MAC, we skip creation of the new lease object.
Also skip adding it to the leases array.
https://bugzilla.redhat.com/show_bug.cgi?id=1202350
https://bugzilla.redhat.com/show_bug.cgi?id=1292941
---
src/network/leaseshelper.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/network/leaseshelper.c b/src/network/leaseshelper.c
index 2d528f7..6930310 100644
--- a/src/network/leaseshelper.c
+++ b/src/network/leaseshelper.c
@@ -439,7 +439,7 @@ main(int argc, char **argv)
case VIR_LEASE_ACTION_OLD:
case VIR_LEASE_ACTION_ADD:
- if (virJSONValueArrayAppend(leases_array_new, lease_new) < 0) {
+ if (lease_new && virJSONValueArrayAppend(leases_array_new, lease_new) < 0) {
virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
_("failed to create json"));
goto cleanup;
--
2.4.6
8 years, 11 months
[libvirt] [PATCH 0/6] Refactor leaseshelper
by Ján Tomko
Split out some functions to make main() more readable.
May depend on https://www.redhat.com/archives/libvir-list/2016-January/msg00595.html
No functional change intented.
Ján Tomko (6):
leaseshelper: store server_duid as an allocated string
leaseshelper: split out custom leases file read
leaseshelper: split out virLeasePrintLeases
leaseshelper: remove useless comparison
leaseshelper: split out virLeaseNewFromArgv
leaseshelper: reduce indentation level in virLeaseReadCustomLeaseFile
src/network/leaseshelper.c | 500 +++++++++++++++++++++++++--------------------
1 file changed, 284 insertions(+), 216 deletions(-)
--
2.4.6
8 years, 11 months
[libvirt] [PATCH] cpu_map.xml: Expose svm flag for nested kvm on AMD CPU model
by Lin Ma
QEMU commit 75d373e disables svm by default since PC machine 2.2.
So with PC machine version > 2.1, When a user wants to expose the
svm flag to a guest, says:
......
<feature policy='require' name='svm'/>
......
The svm flag wont be passed to qemu command line, This patch fixes it.
Signed-off-by: Lin Ma <lma(a)suse.com>
---
src/cpu/cpu_map.xml | 6 ------
1 file changed, 6 deletions(-)
diff --git a/src/cpu/cpu_map.xml b/src/cpu/cpu_map.xml
index 0b6d424..8f96843 100644
--- a/src/cpu/cpu_map.xml
+++ b/src/cpu/cpu_map.xml
@@ -686,7 +686,6 @@
<feature name='sep'/>
<feature name='sse'/>
<feature name='sse2'/>
- <feature name='svm'/>
<feature name='syscall'/>
<feature name='tsc'/>
</model>
@@ -1195,7 +1194,6 @@
<feature name='sep'/>
<feature name='sse'/>
<feature name='sse2'/>
- <feature name='svm'/>
<feature name='syscall'/>
<feature name='tsc'/>
</model>
@@ -1257,7 +1255,6 @@
<feature name='sep'/>
<feature name='sse'/>
<feature name='sse2'/>
- <feature name='svm'/>
<feature name='syscall'/>
<feature name='tsc'/>
</model>
@@ -1295,7 +1292,6 @@
<feature name='sse'/>
<feature name='sse2'/>
<feature name='sse4a'/>
- <feature name='svm'/>
<feature name='syscall'/>
<feature name='tsc'/>
</model>
@@ -1341,7 +1337,6 @@
<feature name='sse4.2'/>
<feature name='sse4a'/>
<feature name='ssse3'/>
- <feature name='svm'/>
<feature name='syscall'/>
<feature name='tsc'/>
<feature name='xop'/>
@@ -1391,7 +1386,6 @@
<feature name='sse4.2'/>
<feature name='sse4a'/>
<feature name='ssse3'/>
- <feature name='svm'/>
<feature name='syscall'/>
<feature name='tbm'/>
<feature name='tsc'/>
--
2.1.4
8 years, 11 months
[libvirt] [PATCH v2] resize, builder: fsync the output file before closing the libvirt connection.
by Richard W.M. Jones
Libvirt has a fixed 15 second timeout for qemu to exit. If qemu is
writing to a slow USB key, it can hang (in D state) for much longer
than this - many minutes usually.
To work around this, fsync the output file before closing the libvirt
connection so that qemu shouldn't have anything (much) to write. We
have to do this in a few places in the code.
This is a hack - it would be better to find a way to fix this in
libvirt.
See also the description here:
https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html
---
builder/builder.ml | 8 +++++++-
resize/resize.ml | 10 ++++++++++
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/builder/builder.ml b/builder/builder.ml
index 1f9a472..093982d 100644
--- a/builder/builder.ml
+++ b/builder/builder.ml
@@ -722,7 +722,13 @@ let main () =
(* Unmount everything and we're done! *)
message (f_"Finishing off");
- g#umount_all ();
+ if cmdline.sync then (
+ (* Work around libvirt 15 second timeout waiting for qemu
+ https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html *)
+ g#sync ();
+ Fsync.file output_filename
+ );
+
g#shutdown ();
g#close ();
diff --git a/resize/resize.ml b/resize/resize.ml
index d6dd9a5..84e38be 100644
--- a/resize/resize.ml
+++ b/resize/resize.ml
@@ -1283,6 +1283,11 @@ read the man page virt-resize(1).
let g =
if to_be_expanded then (
+ (* Work around libvirt 15 second timeout waiting for qemu
+ https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html *)
+ g#sync ();
+ Fsync.file outfile;
+
g#shutdown ();
g#close ();
@@ -1354,6 +1359,11 @@ read the man page virt-resize(1).
) lvs
);
+ (* Work around libvirt 15 second timeout waiting for qemu
+ https://www.redhat.com/archives/libvir-list/2016-January/msg00767.html *)
+ g#sync ();
+ Fsync.file outfile;
+
(* Finished. Unmount disks and exit. *)
g#shutdown ();
g#close ();
--
2.5.0
8 years, 11 months
[libvirt] [PATCH 0/2] Fix log related crasher
by Michal Privoznik
*** BLURB HERE ***
Michal Privoznik (2):
qemuProcessReadLog: Fix memcpy arguments
virLogManagerDomainReadLogFile: Don't do dummy allocs
src/logging/log_manager.c | 4 +---
src/qemu/qemu_process.c | 2 +-
2 files changed, 2 insertions(+), 4 deletions(-)
--
2.4.10
8 years, 11 months
[libvirt] [PATCH v2] qemuTestDriverInit: fill driver with zeroes
by Michal Privoznik
In the commit aea47e48c473a we have fixed a single pointer within
driver structure. Since all callers pass statically allocated
driver on stack other pointers within driver may contain random
values too. Before touching it lets overwrite it with zeroes and
thus fix all dangling pointers.
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
diff to v1:
-Jirka's review worked in
tests/testutilsqemu.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c
index f2eacdd..ae69a18 100644
--- a/tests/testutilsqemu.c
+++ b/tests/testutilsqemu.c
@@ -555,11 +555,11 @@ int qemuTestCapsCacheInsert(virQEMUCapsCachePtr cache, const char *binary,
int qemuTestDriverInit(virQEMUDriver *driver)
{
+ memset(driver, 0, sizeof(*driver));
+
if (virMutexInit(&driver->lock) < 0)
return -1;
- driver->securityManager = NULL;
-
driver->config = virQEMUDriverConfigNew(false);
if (!driver->config)
goto error;
--
2.4.10
8 years, 11 months
[libvirt] [python PATCH v2] setup: Use cflags and ldflags properly
by Jiri Denemark
The setup.py script reads cflags and ldflags from pkg-config and uses
them when compiling/linking C modules. Since both cflags and ldflags may
include multiple compiler arguments we need to split them rather than
concatenating them into a single argument.
Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com>
---
Notes:
Version 2:
- fix all modules
I'm surprised it ever worked.
setup.py | 22 ++++++++--------------
1 file changed, 8 insertions(+), 14 deletions(-)
diff --git a/setup.py b/setup.py
index 9bf583b..c44f84a 100755
--- a/setup.py
+++ b/setup.py
@@ -88,17 +88,15 @@ def get_module_lists():
c_modules = []
py_modules = []
- ldflags = get_pkgconfig_data(["--libs-only-L"], "libvirt", False)
- cflags = get_pkgconfig_data(["--cflags"], "libvirt", False)
+ ldflags = get_pkgconfig_data(["--libs-only-L"], "libvirt", False).split()
+ cflags = get_pkgconfig_data(["--cflags"], "libvirt", False).split()
module = Extension('libvirtmod',
sources = ['libvirt-override.c', 'build/libvirt.c', 'typewrappers.c', 'libvirt-utils.c'],
libraries = [ "virt" ],
include_dirs = [ "." ])
- if cflags != "":
- module.extra_compile_args.append(cflags)
- if ldflags != "":
- module.extra_link_args.append(ldflags)
+ module.extra_compile_args.extend(cflags)
+ module.extra_link_args.extend(ldflags)
c_modules.append(module)
py_modules.append("libvirt")
@@ -107,10 +105,8 @@ def get_module_lists():
sources = ['libvirt-qemu-override.c', 'build/libvirt-qemu.c', 'typewrappers.c', 'libvirt-utils.c'],
libraries = [ "virt-qemu" ],
include_dirs = [ "." ])
- if cflags != "":
- moduleqemu.extra_compile_args.append(cflags)
- if ldflags != "":
- moduleqemu.extra_link_args.append(ldflags)
+ moduleqemu.extra_compile_args.extend(cflags)
+ moduleqemu.extra_link_args.extend(ldflags)
c_modules.append(moduleqemu)
py_modules.append("libvirt_qemu")
@@ -120,10 +116,8 @@ def get_module_lists():
sources = ['libvirt-lxc-override.c', 'build/libvirt-lxc.c', 'typewrappers.c', 'libvirt-utils.c'],
libraries = [ "virt-lxc" ],
include_dirs = [ "." ])
- if cflags != "":
- modulelxc.extra_compile_args.append(cflags)
- if ldflags != "":
- modulelxc.extra_link_args.append(ldflags)
+ modulelxc.extra_compile_args.extend(cflags)
+ modulelxc.extra_link_args.extend(ldflags)
c_modules.append(modulelxc)
py_modules.append("libvirt_lxc")
--
2.7.0
8 years, 11 months
