November 2011 - Devel - libvirt List Archives

[libvirt] using php-libvirt libvirt_domain_is_active function
by Eduard B 14 Nov '11

14 Nov '11

Hi All, Im trying to figure how to use libvirt_domain_is_active but I'm unable to find what I am doing wrong. I tested it with php-libvirt-0.4.4 an with 0.4.3 but i have the same problem Im using libvirt-0.9.6-2.fc16.x86_64 with the following code Can someone point me in the right direction with the use of libvirt_domain_is_active ? Thanks ------8<------CODE --------8<------------------------------------ print_r ( libvirt_version() ); $domains=libvirt_list_domains($conn); print_r ($domains); foreach ( $domains as $domain ) { print "$domain\n"; print libvirt_domain_is_active($domain); } ---------8<------- OUTPUT ----------------8<------------------- Array ( [libvirt.release] => 6 [libvirt.minor] => 9 [libvirt.major] => 0 [connector.version] => 0.4.3 [connector.major] => 0 [connector.minor] => 4 [connector.release] => 3 ) Array ( [0] => Domain-0 [1] => vm1 [2] => vm2 [3] => vm3 ) Domain-0 PHP Warning: libvirt_domain_is_active() expects parameter 1 to be resource, integer given in /var/www/html/libvirt/index.php on line 20 PHP Warning: libvirt_domain_is_active(): Invalid arguments in /var/www/html/libvirt/index.php on line 20 vm1 PHP Warning: libvirt_domain_is_active() expects parameter 1 to be resource, integer given in /var/www/html/libvirt/index.php on line 20 PHP Warning: libvirt_domain_is_active(): Invalid arguments in /var/www/html/libvirt/index.php on line 20 vm2 PHP Warning: libvirt_domain_is_active() expects parameter 1 to be resource, integer given in /var/www/html/libvirt/index.php on line 20 PHP Warning: libvirt_domain_is_active(): Invalid arguments in /var/www/html/libvirt/index.php on line 20 vm3 PHP Warning: libvirt_domain_is_active() expects parameter 1 to be resource, integer given in /var/www/html/libvirt/index.php on line 20 PHP Warning: libvirt_domain_is_active(): Invalid arguments in /var/www/html/libvirt/index.php on line 20

2 1

[libvirt] Bridged Networking Wiki Up-to-Date?
by Bob Cochran 14 Nov '11

14 Nov '11

Greetings, I just installed Fedora 16 on a new server and want to create a Fedora 16 virtual machine. I have not created or used VMs on Fedora in quite some time. Now I have several compelling reasons to use Fedora virtualization so I am kind of "coming back" to it. Is the libvirt bridged networking wiki up-to-date? http://wiki.libvirt.org/page/Networking My hope is that the default networking support provided on Fedora 16 will let any virtual guest connect to any other host on my internal LAN (within reason, of course) so that for example one virtual guest can be dedicated as a DNS server and all the other hosts on my network can query it. That is, I hope I won't need to do extensive additional networking support when I create the guest OS, but I am willing to if I have to. Thanks Bob Cochran

2 1

[libvirt] [PATCH] util: Compile error
by taget＠linux.vnet.ibm.com 14 Nov '11

14 Nov '11

From: Eli Qiao <taget(a)linux.vnet.ibm.com> Add virnetdev.h,virnetdevbridge.h,virnetdevtap.h to private symbols Signed-off-by: Eli Qiao <taget(a)linux.vnet.ibm.com> --- src/libvirt_private.syms | 29 +++++++++++++++++++++++++++++ 1 files changed, 29 insertions(+), 0 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 81de6b4..ebe34c9 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1195,6 +1195,35 @@ virFileRewrite; # virnetclient.h virNetClientHasPassFD; +# virnetdev.h +virNetDevClearIPv4Address; +virNetDevExists; +virNetDevGetMAC; +virNetDevGetMTU; +virNetDevIsOnline; +virNetDevSetIPv4Address; +virNetDevSetMAC; +virNetDevSetMTU; +virNetDevSetMTUFromDevice; +virNetDevSetOnline; + + +# virnetdevbridge.h +virNetDevBridgeAddPort; +virNetDevBridgeCreate; +virNetDevBridgeDelete; +virNetDevBridgeGetSTP; +virNetDevBridgeGetSTPDelay; +virNetDevBridgeRemovePort; +virNetDevBridgeSetSTP; +virNetDevBridgeSetSTPDelay; + + +# virnetdevtap.h +virNetDevTapCreate; +virNetDevTapCreateInBridgePort; +virNetDevTapDelete; + # virnetmessage.h virNetMessageClear; -- 1.7.4.4

6 7

[libvirt] [PATCH] qemu: fix domjobabort regression
by Eric Blake 12 Nov '11

12 Nov '11

This reverts commit ef1065cf5ac; see also this bug report: https://bugzilla.redhat.com/show_bug.cgi?id=751900 In qemu 0.15.1 and earlier, during migration to file, the qemu_savevm_state_begin and qemu_savevm_state_iterate methods will both process as much migration data as possible until either 1. The file descriptor returns EAGAIN 2. The bandwidth rate limit is reached If we set the rate limit to ULONG_MAX, test 2 never becomes true. We're passing a plain file descriptor to QEMU and POSIX does not support EAGAIN on regular files / block devices, so test 1 never becomes true either. In the 'virsh save --bypass-cache' case, we pass a pipe instead of a regular fd, but using a pipe adds I/O overhead, so always passing a pipe just so qemu can see EAGAIN doesn't seem nice. The ultimate fix needs to come from qemu - background migration must respect asynchronous abort requests, or else periodically return control to the main handling loop without an EAGAIN and without waiting to hit an insanely large amount of data. But until a version of qemu is fixed to support "unlimited" data rates while still allowing cancellation, the best we can do is avoid the automatic use of unlimited rates from within libvirt (users can still explicitly change the migration rates, if they are aware that they are giving up the ability to cancel a job). Reverting the lone use of QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX is the simplest patch; this slows migration back down to a default 32M/sec cap, but also ensures that the main qemu processing loop will still be responsive to cancellation requests. Hopefully upstream qemu will provide us a means of safely using unlimited speed, including a runtime probe of that capability. * src/qemu/qemu_migration.c (qemuMigrationToFile): Revert attempt to use unlimited migration bandwidth when migrating to file. Signed-off-by: Daniel Veillard <veillard(a)redhat.com> Signed-off-by: Eric Blake <eblake(a)redhat.com> --- src/qemu/qemu_migration.c | 17 ----------------- 1 files changed, 0 insertions(+), 17 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 838a31f..30f805d 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2825,16 +2825,6 @@ qemuMigrationToFile(struct qemud_driver *driver, virDomainObjPtr vm, bool restoreLabel = false; virCommandPtr cmd = NULL; int pipeFD[2] = { -1, -1 }; - unsigned long saveMigBandwidth = priv->migMaxBandwidth; - - /* Increase migration bandwidth to unlimited since target is a file. - * Failure to change migration speed is not fatal. */ - if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) == 0) { - qemuMonitorSetMigrationSpeed(priv->mon, - QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX); - priv->migMaxBandwidth = QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX; - qemuDomainObjExitMonitorWithDriver(driver, vm); - } if (qemuCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATE_QEMU_FD) && (!compressor || pipe(pipeFD) == 0)) { @@ -2946,13 +2936,6 @@ qemuMigrationToFile(struct qemud_driver *driver, virDomainObjPtr vm, ret = 0; cleanup: - /* Restore max migration bandwidth */ - if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) == 0) { - qemuMonitorSetMigrationSpeed(priv->mon, saveMigBandwidth); - priv->migMaxBandwidth = saveMigBandwidth; - qemuDomainObjExitMonitorWithDriver(driver, vm); - } - VIR_FORCE_CLOSE(pipeFD[0]); VIR_FORCE_CLOSE(pipeFD[1]); virCommandFree(cmd); -- 1.7.4.4

3 3

[libvirt] [PATCH] fix two bugs in bridge_driver.c
by Hu Tao 12 Nov '11

12 Nov '11

steps to reproduce: 1. having a network xml file(named default.xml) like this one: <network> <name>default</name> <uuid>c5322c4c-81d0-4985-a363-ad6389780d89</uuid> <bridge name="virbr0" /> <forward/> <ip address="192.168.122.1" netmask="255.255.255.0"> <dhcp> <range start="192.168.122.2" end="192.168.122.254" /> </dhcp> </ip> </network> in /etc/libvirt/qemu/networks/, and mark it as autostart: $ ls -l /etc/libvirt/qemu/networks/autostart total 0 lrwxrwxrwx 1 root root 14 Oct 12 14:02 default.xml -> ../default.xml 2. start libvirtd and the device virbr0 is not automatically up. The reason is that the function virNetDevExists is now returns 1 if the device exists, comparing to the former one returns 0 if the device exists. But with only this fix will cause a segmentation fault(the same steps as above) that is fixed by the second chunk of code. --- src/network/bridge_driver.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/network/bridge_driver.c b/src/network/bridge_driver.c index 32cceb0..951b5aa 100644 --- a/src/network/bridge_driver.c +++ b/src/network/bridge_driver.c @@ -214,7 +214,7 @@ networkFindActiveConfigs(struct network_driver *driver) { /* If bridge exists, then mark it active */ if (obj->def->bridge && - virNetDevExists(obj->def->bridge) == 0) { + virNetDevExists(obj->def->bridge) == 1) { obj->active = 1; /* Try and read dnsmasq/radvd pids if any */ @@ -1815,8 +1815,10 @@ networkStartNetworkVirtual(struct network_driver *driver, if (!save_err) save_err = virSaveLastError(); - ignore_value(virNetDevTapDelete(macTapIfName)); - VIR_FREE(macTapIfName); + if (macTapIfName) { + ignore_value(virNetDevTapDelete(macTapIfName)); + VIR_FREE(macTapIfName); + } err0: if (!save_err) -- 1.7.3.1

3 2

[libvirt] [PATCH] build: drop useless dirent.h includes
by Eric Blake 11 Nov '11

11 Nov '11

* .gnulib: Update to latest, for improved syntax-check. * src/lxc/lxc_container.c (includes): Drop unused include. * src/network/bridge_driver.c: Likewise. * src/node_device/node_device_linux_sysfs.c: Likewise. * src/openvz/openvz_driver.c: Likewise. * src/qemu/qemu_conf.c: Likewise. * src/storage/storage_backend_iscsi.c: Likewise. * src/storage/storage_backend_mpath.c: Likewise. * src/uml/uml_conf.c: Likewise. * src/uml/uml_driver.c: Likewise. --- I'm not sure whether to include this in 0.9.7. On the one hand, this is pretty trivial; on the other hand, I hate making more .gnulib updates than necessary after release candidates have been posted, in case an unintended regression sneaks in from a half-baked gnulib change. But looking at the 22 patches, I'm not seeing anything likely to impact libvirt besides the new syntax check rule. * .gnulib 0031e4f...e56e96f (22): > maint.mk: silence new syntax check > Doc about floating-point and math API. > stdalign tests: Skip the test when compiled by Sun C. > ansi-c++-opt: Complete the 2011-06-05 change. > * doc/posix-headers/stdalign.texi (stdalign.h): Mention Sun review ID. > copysignl: Fix result for zero argument on HP-UX 11 with HP C. > update from texinfo > ldexp, ldexpf, ldexpl: Enhance tests. > math tests: Cosmetics. > fma*: Simplify test. > Tests for module 'fmal'. > New module 'fmal'. > Tests for module 'fmaf'. > New module 'fmaf'. > Tests for module 'fma'. > New module 'fma'. > Extend gl_MATHFUNC. > New modules 'at-internal', 'openat-h', split off from module 'openat'. > maint.mk: also prohibit inclusion of dirent.h without use > ldexpl tests: Avoid test failure on MSVC 9. > New modules 'at-internal', 'openat-h', split off from module 'openat'. > openat: Include <stdbool.h>. .gnulib | 2 +- src/lxc/lxc_container.c | 1 - src/network/bridge_driver.c | 1 - src/node_device/node_device_linux_sysfs.c | 3 +-- src/openvz/openvz_driver.c | 1 - src/qemu/qemu_conf.c | 1 - src/storage/storage_backend_iscsi.c | 1 - src/storage/storage_backend_mpath.c | 3 +-- src/uml/uml_conf.c | 1 - src/uml/uml_driver.c | 1 - 10 files changed, 3 insertions(+), 12 deletions(-) diff --git a/.gnulib b/.gnulib index 0031e4f..e56e96f 160000 --- a/.gnulib +++ b/.gnulib @@ -1 +1 @@ -Subproject commit 0031e4f6353cc7077a9d0dad0c793bd6e3dc7aaa +Subproject commit e56e96fe20f72586e9ec5c528b9a9a06daa2ecc6 diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index c4e5f28..c2484a7 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -35,7 +35,6 @@ #include <sys/stat.h> #include <unistd.h> #include <mntent.h> -#include <dirent.h> /* Yes, we want linux private one, for _syscall2() macro */ #include <linux/unistd.h> diff --git a/src/network/bridge_driver.c b/src/network/bridge_driver.c index 445c3cb..e897b32 100644 --- a/src/network/bridge_driver.c +++ b/src/network/bridge_driver.c @@ -25,7 +25,6 @@ #include <sys/types.h> #include <sys/poll.h> -#include <dirent.h> #include <limits.h> #include <string.h> #include <stdio.h> diff --git a/src/node_device/node_device_linux_sysfs.c b/src/node_device/node_device_linux_sysfs.c index 844231a..d352800 100644 --- a/src/node_device/node_device_linux_sysfs.c +++ b/src/node_device/node_device_linux_sysfs.c @@ -2,7 +2,7 @@ * node_device_hal_linuc.c: Linux specific code to gather device data * not available through HAL. * - * Copyright (C) 2009-2010 Red Hat, Inc. + * Copyright (C) 2009-2011 Red Hat, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -32,7 +32,6 @@ #include "memory.h" #include "logging.h" #include "virfile.h" -#include <dirent.h> #define VIR_FROM_THIS VIR_FROM_NODEDEV diff --git a/src/openvz/openvz_driver.c b/src/openvz/openvz_driver.c index 69ff444..4eed8ad 100644 --- a/src/openvz/openvz_driver.c +++ b/src/openvz/openvz_driver.c @@ -30,7 +30,6 @@ #include <sys/types.h> #include <sys/poll.h> -#include <dirent.h> #include <limits.h> #include <string.h> #include <stdio.h> diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index d1bf075..0cf921f 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -23,7 +23,6 @@ #include <config.h> -#include <dirent.h> #include <string.h> #include <limits.h> #include <sys/types.h> diff --git a/src/storage/storage_backend_iscsi.c b/src/storage/storage_backend_iscsi.c index 99e69c9..354f99b 100644 --- a/src/storage/storage_backend_iscsi.c +++ b/src/storage/storage_backend_iscsi.c @@ -32,7 +32,6 @@ #include <regex.h> #include <fcntl.h> #include <unistd.h> -#include <dirent.h> #include <sys/stat.h> #include "virterror_internal.h" diff --git a/src/storage/storage_backend_mpath.c b/src/storage/storage_backend_mpath.c index 4f3add3..f09ce9b 100644 --- a/src/storage/storage_backend_mpath.c +++ b/src/storage/storage_backend_mpath.c @@ -1,7 +1,7 @@ /* * storage_backend_mpath.c: storage backend for multipath handling * - * Copyright (C) 2009-2010 Red Hat, Inc. + * Copyright (C) 2009-2011 Red Hat, Inc. * Copyright (C) 2009-2008 Dave Allan * * This library is free software; you can redistribute it and/or @@ -25,7 +25,6 @@ #include <unistd.h> #include <stdio.h> -#include <dirent.h> #include <fcntl.h> #include <libdevmapper.h> diff --git a/src/uml/uml_conf.c b/src/uml/uml_conf.c index f2bdd74..e06eb90 100644 --- a/src/uml/uml_conf.c +++ b/src/uml/uml_conf.c @@ -23,7 +23,6 @@ #include <config.h> -#include <dirent.h> #include <string.h> #include <limits.h> #include <sys/types.h> diff --git a/src/uml/uml_driver.c b/src/uml/uml_driver.c index 772e1c6..35b73b6 100644 --- a/src/uml/uml_driver.c +++ b/src/uml/uml_driver.c @@ -25,7 +25,6 @@ #include <sys/types.h> #include <sys/poll.h> -#include <dirent.h> #include <limits.h> #include <string.h> #include <stdio.h> -- 1.7.4.4

1 1

[libvirt] [PATCH] xenapi: remove unused variable
by Eric Blake 11 Nov '11

11 Nov '11

CC libvirt_driver_xenapi_la-xenapi_driver.lo xenapi/xenapi_driver.c: In function 'xenapiDomainGetVcpus': xenapi/xenapi_driver.c:1209:21: error: variable 'cpus' set but not used [-Werror=unused-but-set-variable] * src/xenapi/xenapi_driver.c (xenapiDomainGetVcpus): Silence compiler warning. --- Pushing under the build-breaker rule. src/xenapi/xenapi_driver.c | 6 ++---- 1 files changed, 2 insertions(+), 4 deletions(-) diff --git a/src/xenapi/xenapi_driver.c b/src/xenapi/xenapi_driver.c index d454589..a835b2d 100644 --- a/src/xenapi/xenapi_driver.c +++ b/src/xenapi/xenapi_driver.c @@ -1206,7 +1206,7 @@ xenapiDomainGetVcpus (virDomainPtr dom, xen_vm_set *vms = NULL; xen_vm vm = NULL; xen_string_string_map *vcpu_params = NULL; - int nvcpus = 0, cpus = 0, i; + int nvcpus = 0, i; virDomainInfo domInfo; virNodeInfo nodeInfo; virVcpuInfoPtr ifptr; @@ -1221,9 +1221,7 @@ xenapiDomainGetVcpus (virDomainPtr dom, _("Couldn't fetch Domain Information")); return -1; } - if (xenapiNodeGetInfo(dom->conn, &nodeInfo) == 0) - cpus = nodeInfo.cpus; - else { + if (xenapiNodeGetInfo(dom->conn, &nodeInfo) != 0) { xenapiSessionErrorHandler(dom->conn, VIR_ERR_INTERNAL_ERROR, _("Couldn't fetch Node Information")); return -1; -- 1.7.7.1

1 0

[libvirt] 0.9.7 virsh console regression?
by Albert W. Hopkins 11 Nov '11

11 Nov '11

Hi folks, I have a few "headless" qemu domains that use serial consoles. Typically i connect to the consoles with # virsh console <domain> However with 0.9.7 I get Connected to domain gentoo-base Escape character is ^] error: internal error character device (null) is not using a PTY I am still able to connect to the console if i go into the virt-manager gui and use the "gui" console, but I'd rather not do that as sometimes I don't have access to the gui. Is this a regression or am I expected to do some kind of migration on my domains. These are qemu guests running on Gentoo Linux. An xml dump of a typical domain follows: <domain type='kvm'> <name>base</name> <uuid>b4a528dd-4c31-03ca-bdde-795c4c38592d</uuid> <memory>524288</memory> <currentMemory>61440</currentMemory> <vcpu>1</vcpu> <os> <type arch='x86_64' machine='pc-0.12'>hvm</type> <kernel>/var/lib/libvirt/images/vmlinuz</kernel> <cmdline>root=/dev/vda1 rootfstype=ext4 quiet console=ttyS0 TERM=vt100</cmdline> <boot dev='hd'/> <bootmenu enable='no'/> </os> <features> <acpi/> <pae/> </features> <cpu match='exact'> <model>Nehalem</model> <vendor>Intel</vendor> <feature policy='require' name='tm2'/> <feature policy='require' name='est'/> <feature policy='require' name='monitor'/> <feature policy='require' name='ss'/> <feature policy='require' name='vme'/> <feature policy='require' name='rdtscp'/> <feature policy='require' name='ht'/> <feature policy='require' name='ds'/> <feature policy='require' name='pbe'/> <feature policy='require' name='tm'/> <feature policy='require' name='vmx'/> <feature policy='require' name='ds_cpl'/> <feature policy='require' name='xtpr'/> <feature policy='require' name='acpi'/> <feature policy='require' name='x2apic'/> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/lib/libvirt/images/base.qcow'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <interface type='network'> <mac address='52:54:00:06:84:c0'/> <source network='default'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </memballoon> </devices> </domain> Thanks in advance, -a

3 2

[libvirt] Open Source Virtualization and Cloud at FOSDEM
by Mark McLoughlin 11 Nov '11

11 Nov '11

Hey, Here's one for the calendar - a 2 day track at FOSDEM devoted to Open Source Virtualization and Cloud. Hope to see you there! :) Cheers, Mark. -------- Forwarded Message -------- > From: Thierry Carrez <thierry(a)openstack.org> > To: openstack(a)lists.launchpad.net <openstack(a)lists.launchpad.net> > Subject: [Openstack] OpenStack at FOSDEM > Date: Fri, 11 Nov 2011 11:34:22 +0100 > > Hi everyone, > > Our OpenStack devroom was accepted for FOSDEM, February 4-5, 2012 in > Brussels. It was actually grouped with other "Open Source virtualization > and cloud" projects in a giant devroom (550-seat auditorium) that lasts > the full 2 days ! > > Expect a CFP in the coming weeks... > > See other devrooms at: > http://fosdem.org/2012/devrooms_for_2012 > > -- > Thierry Carrez (ttx) > Release Manager, OpenStack > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp

1 0

[libvirt] Long QEMU main loop pauses during migration (to file) under heavy load
by Daniel P. Berrange 11 Nov '11

11 Nov '11

Libvirt recently introduced a change to the way it does 'save to file' with QEMU. Historically QEMU has a 32MB/s I/O limit on migration by default. When saving to file, we didn't want any artificial limit, but rather to max out the underlying storage. So when doing save to file, we set a large bandwidth limit (INT64_MAX / (1024 * 1024)) so it is effectively unlimited. After doing this, we discovered that the QEMU monitor was becoming entirely blocked. It did not even return from the 'migrate' command until migration was complete despite the 'detach' flag being set. This was a bug in libvirt, because we passed a plain file descriptor which does not support EAGAIN. Thank you POSIX. Libvirt has another mode where it uses an I/O helper command so get O_DIRECT, and in this mode we pass a pipe() FD to QEMU. After ensuring that this pipe FD really does have O_NONBLOCK set, we still saw some odd behaviour. I'm not sure whether what I describe can neccessarily be called a QEMU bug, but I wanted to raise it for discussion anyway.... The sequence of steps is - libvirt sets qemu migration bandwidth to "unlimited" - libvirt opens a pipe() and sets O_NONBLOCK on the write end - libvirt spawns libvirt-iohelper giving it the target file on disk, and the read end of the pipe - libvirt does 'getfd migfile' monitor command to give QEMU the write end of the pipe - libvirt does 'migrate fd:migfile -d' to run migration - In parallel - QEMU is writing to the pipe (which is non-blocking) - libvirt_helper reading the pipe & writing to disk with O_DIRECT The initial 'migrate' command detaches into the background OK, and libvirt can enter its loop doing 'query-migrate' frequently to monitor progress. Initially this works fine, but at some points during the migration, QEMU will get "stuck" for a very long time and not respond to the monitor (or indeed the mainloop at all). These blackouts are anywhere from 10 to 20 seconds long. Using a combination of systemtap, gdb and strace I managed to determine out the following - Most of the qemu_savevm_state_iterate() calls complete in 10-20 ms - Reasonably often a qemu_savevm_state_iterate() call takes 300-400 ms - Fairly rarely a qemu_savevm_state_iterate() call takes 10-20 *seconds* - I can see EAGAIN from the FD QEMU is migrating from - hence most of the iterations are quite short. - In the 10-20 second long calls, no EAGAIN is seen for the entire period. - The host OS in general is fairly "laggy", presumably due to the high rate of direct I/O being performed by the I/O helper, and bad schedular tuning IIUC, there are two things which will cause a qemu_savevm_state_iterate() call to return - Getting EAGAIN on the migration FD - Hitting the max bandwidth limit We have set effectively unlimited bandwidth, so everything relies on the EAGAIN behaviour. If the OS is doing a good job at scheduling processes & I/O, then this seems to work reasonably well. If the OS is highly loaded and becoming less responsive to scheduling apps, then QEMU gets itself into a bad way. What I think is happening is that the OS is giving too much time to the I/O helper process that is reading the other end of the pipe given to QEMU, and then doing the O_DIRECT to disk. Thus in the shorter-than-normal windows of time when QEMU itself is scheduled by the OS, the pipe is fairly empty, so QEMU does not see EAGAIN for a very long period of wallclock time. So we get into a case where QEMU sees 10-20 second gaps betweeen iterations of the main loop. Having a non-inifinite max-bandwidth for the migration, would likely mitigate this to some extent, but I still think it'd be possible to get QEMU into these pathelogical conditions under high load for a host. Is this a scenario we need to worry about for QEMU ? On the one hand it seems like it is a rare edge case in OS behaviour overall. On the other hand, times when a host is highly loaded and non-responsive are exactly the times when a mgmt app might want to save a guest to disk, or migrate it elsewhere. Which would mean we need QEMU to behave as well as possible in these adverse conditions. Thus should we consider having an absolute bound on the execution time of qemu_savevm_state_iterate(), independant of EAGAIN & bandwidth limits, to ensure the main loop doesn't get starved ? Or perhaps moving migration to a separate thread, out of the mainloop is what we need to strive for ? Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

1 1