November 2012 - Devel - libvirt List Archives

[libvirt] [PATCHv2 0/3] Use virNodeGetCPUMap where appropriate
by Viktor Mihajlovski 01 Nov '12

01 Nov '12

This series concludes the introduction of the virNodeGetCPUMap API by replacing calls to virNodeGetInfo used only for the purpose of computing the maximum number of node CPUs (which has the potential to yield the incorrect number). Most prominently, with patch 3/3 the output of virsh vcpuinfo will now be correct for domains on hosts with offline CPUs V2 Changes: Rework based on Eric's feedback: - Use nodeGetCPUCount instead of nodeGetCPUMap - Avoid code bloat by computing node CPU count in a helper function Viktor Mihajlovski (3): qemu, lxc: Change host CPU detection logic. python: Use virNodeGetCPUMap where possible virsh: Use virNodeGetCPUMap if possible python/libvirt-override.c | 87 ++++++++++++++++++++++++++++------------------- src/lxc/lxc_controller.c | 8 ++--- src/qemu/qemu_driver.c | 14 +++----- src/qemu/qemu_process.c | 8 ++--- tools/virsh-domain.c | 32 ++++++++++++----- 5 files changed, 86 insertions(+), 63 deletions(-) -- 1.7.12.4

3 10

[libvirt] dhcp6, radvd, ip6tables, etc.
by Gene Czarcinski 01 Nov '12

01 Nov '12

OK, I have the basic implementation for libvirt support of dhcp6. Let me say again that 98% of the work was already done. There is still a bunch of work today which includes writing some tests, understanding how things such as bootp, dhcp-host, etc. should be supported with dhcp6, as well as the items I discuss below. 1. Right now, the only way that dhcp6 is in effect is if there is no dhcp4 range definition. This will be fixed/expanded so that, at a minimum, you can have both a dhcp4 and dhcp6 on the same interface. However, it appears to be easier to just pass to dnsmasq ANY/EVERY dhcp4 range or dhcp6 range defined in the xml. Comments? Any input on which approach to use or avoid? 2. I have modified radvd so both stateful (dhcp6) and stateless (SLAAC) addressing is supported with radvd for the default route. This is done on an interface basis (that is the way it works). So if any dhcp6 range is specified, then stateful is used. The way this is implemented will make it easy to add some tests verifying that the configuration parameters are working. I intend this to be an expansion to networkxml2argvtest since it has the xml specification files which determine both dnsmasq and radvd configuration parameters. 3. After completing what I thought was code that should result in a guest getting dhcp6 addresses, it was not working. Once more it took me a little time to realize that ip6tables rules were blocking it. [I have been down this path before, you would think I would realize the problem sooner.] 3a. In looking over the ip6tables rules, I saw a whole bunch of additions at the top of the INPUT chain which were accepts for udp/tcp port 53. In looking at the code in bridge_driver.c, I found that, every time a network device was started, 3 FORWARD rules and 2 INPUT rules were added, but, when the network device was destroyed, only the 3 FORWARD rules were removed. I believe this is a bug (but not high priority) and I will be submitting a separate patch to fix this. 3b. There are two different approaches for the rule which allows the dhcp6 server to work. I could add (actually insert) one rule to the INPUT chain which accepted the packet if it is "-d ff02::1:2 "--dport 547". Or, I could add (insert) a rule specifying "-i virbr__" for every IPv6 device which would be removed when the device was destroyed. 4. After getting all of this working to my satisfaction, my next mountain to climb is VM ... it really does not like network xml definitions which include a dhcp-range for an ipv6 definition. Comments? NOTE: I am implementing all of this assuming that my previous patches have been accepted ... the ones for creating a dnsmasq conf-file for parameters rather than using the dnsmasq command-line. I am sure that someone could spend the time refitting the dhcp6 patches to the old code but why get aggravated? If you folks do not want to do things that way, fine, please say so. But if it is going to be accepted, then I would like some indication of this. Gene

5 18

[libvirt] [PATCH repost] daemon: Make the default PolicyKit policy auth_admin_keep.
by Richard W.M. Jones 01 Nov '12

01 Nov '12

Reposted at Cole's request. Previous discussion here: https://www.redhat.com/archives/libvir-list/2012-October/thread.html#00682 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora

3 3

[libvirt] [PATCH v2] iohelper: fsync() at the end
by Michal Privoznik 01 Nov '12

01 Nov '12

Currently, when we are doing (managed) save, we insert the iohelper between the qemu and OS. The pipe is created, the writing end is passed to qemu and the reading end to the iohelper. It reads data and write them into given file. However, with write() being asynchronous data may still be in OS caches and hence in some (corner) cases, all migration data may have been read and written (not physically though). So qemu will report success, as well as iohelper. However, with some non local filesystems, where ENOSPACE is polled every X time units, we may get into situation where all operations succeeded but data hasn't reached the disk. And in fact will never do. Therefore we ought sync caches to make sure data has reached the block device on remote host. --- For more information follow: https://bugzilla.redhat.com/show_bug.cgi?id=866369 src/util/iohelper.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index c6542ed..0d356c2 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -40,6 +40,7 @@ #include "virterror_internal.h" #include "configmake.h" #include "virrandom.h" +#include "storage_file.h" #define VIR_FROM_THIS VIR_FROM_STORAGE @@ -179,6 +180,14 @@ runIO(const char *path, int fd, int oflags, unsigned long long length) } } + /* If we are on shared FS ensure all data is written as some + * FSs may do writeback caching or polling for ENOSPC or any + * other magic that local FS does not.*/ + if (virStorageFileIsSharedFS(fdoutname) && (fdatasync(fdout) < 0)) { + virReportSystemError(errno, _("unable to fsync %s"), fdoutname); + goto cleanup; + } + ret = 0; cleanup: -- 1.7.8.6

3 2

[libvirt] [00/11] Automatically shutdown VMs on session quit
by Daniel P. Berrange 01 Nov '12

01 Nov '12

This is a followup to Alex's original proposal: https://www.redhat.com/archives/libvir-list/2012-October/msg00365.html The core idea & concepts are the same as in Alex's patch, but I realized I could take the opportunity to refactor part of libvirtd to improve life in general. The state drivers currently have an 'active' method which is polled on every iteration of the event loop to determine if any resources (such as VMs) are active. This is the kind of information we need to inhibit host shutdown/suspend, but it is not being made available in an easily consumable way. The key idea I had was to remove the 'active' method from the state drivers and instead pass in a callback to the 'startup' method in the state driver. This so called 'inhibit' callback is used by drivers to signal when they have resources active. This callback is used both to inhibit shutdown of libvirtd (when the --timeout arg is used), and to inhibit shutdown of the host OS itself. The state driver also gains a 'stop' method which is intended to stop any resources which are causing the shutdown inhibition. In other words, to save any VMs to disk. One of the side effects of this change is that the --timeout param to libvirtd actually now works much better. Only running VMs cause libvirtd shutdown to be inhibited. Networks / storage pools no longer inhibit it. This is obviously targetted at post-1.0.0

4 18

[libvirt] [PATCH] qemu: Add controllers in specified order
by Michal Privoznik 01 Nov '12

01 Nov '12

qemu is sensitive to the order of arguments passed. Hence, if a device requires a controller, the controller cmd string must precede device cmd string. The same apply for controllers, when for instance ccid controller requires usb controller. So controllers create partial ordering in which they should be added to qemu cmd line. --- The order is basically random for now, with one constraint: CCID must be preceded with USB. src/qemu/qemu_command.c | 102 +++++++++++++++++++++++++--------------------- 1 files changed, 55 insertions(+), 47 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index fe99f5d..ffeb965 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -4429,7 +4429,7 @@ qemuBuildCommandLine(virConnectPtr conn, virDomainSnapshotObjPtr snapshot, enum virNetDevVPortProfileOp vmop) { - int i; + int i, j; struct utsname ut; int disableKQEMU = 0; int enableKQEMU = 0; @@ -4446,6 +4446,17 @@ qemuBuildCommandLine(virConnectPtr conn, int usbcontroller = 0; bool usblegacy = false; uname_normalize(&ut); + int contOrder[] = { + /* We don't add an explicit IDE or FD controller because the + * provided PIIX4 device already includes one. It isn't possible to + * remove the PIIX4. */ + VIR_DOMAIN_CONTROLLER_TYPE_USB, + VIR_DOMAIN_CONTROLLER_TYPE_SCSI, + VIR_DOMAIN_CONTROLLER_TYPE_SATA, + VIR_DOMAIN_CONTROLLER_TYPE_VIRTIO_SERIAL, + VIR_DOMAIN_CONTROLLER_TYPE_CCID, + }; + VIR_DEBUG("conn=%p driver=%p def=%p mon=%p json=%d " "caps=%p migrateFrom=%s migrateFD=%d " @@ -5024,61 +5035,59 @@ qemuBuildCommandLine(virConnectPtr conn, } if (qemuCapsGet(caps, QEMU_CAPS_DEVICE)) { - for (i = 0 ; i < def->ncontrollers ; i++) { - virDomainControllerDefPtr cont = def->controllers[i]; + for (j = 0; j < ARRAY_CARDINALITY(contOrder); j++) { + for (i = 0; i < def->ncontrollers; i++) { + virDomainControllerDefPtr cont = def->controllers[i]; - /* We don't add an explicit IDE or FD controller because the - * provided PIIX4 device already includes one. It isn't possible to - * remove the PIIX4. */ - if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_IDE || - cont->type == VIR_DOMAIN_CONTROLLER_TYPE_FDC) - continue; + if (cont->type != contOrder[j]) + continue; - /* Also, skip USB controllers with type none.*/ - if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_USB && - cont->model == VIR_DOMAIN_CONTROLLER_MODEL_USB_NONE) { - usbcontroller = -1; /* mark we don't want a controller */ - continue; - } + /* Also, skip USB controllers with type none.*/ + if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_USB && + cont->model == VIR_DOMAIN_CONTROLLER_MODEL_USB_NONE) { + usbcontroller = -1; /* mark we don't want a controller */ + continue; + } - /* Only recent QEMU implements a SATA (AHCI) controller */ - if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_SATA) { - if (!qemuCapsGet(caps, QEMU_CAPS_ICH9_AHCI)) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("SATA is not supported with this " - "QEMU binary")); - goto error; + /* Only recent QEMU implements a SATA (AHCI) controller */ + if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_SATA) { + if (!qemuCapsGet(caps, QEMU_CAPS_ICH9_AHCI)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("SATA is not supported with this " + "QEMU binary")); + goto error; + } else { + char *devstr; + + virCommandAddArg(cmd, "-device"); + if (!(devstr = qemuBuildControllerDevStr(def, cont, + caps, NULL))) + goto error; + + virCommandAddArg(cmd, devstr); + VIR_FREE(devstr); + } + } else if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_USB && + cont->model == -1 && + !qemuCapsGet(caps, QEMU_CAPS_PIIX3_USB_UHCI)) { + if (usblegacy) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Multiple legacy USB controllers are " + "not supported")); + goto error; + } + usblegacy = true; } else { - char *devstr; - virCommandAddArg(cmd, "-device"); - if (!(devstr = qemuBuildControllerDevStr(def, cont, - caps, NULL))) + + char *devstr; + if (!(devstr = qemuBuildControllerDevStr(def, cont, caps, + &usbcontroller))) goto error; virCommandAddArg(cmd, devstr); VIR_FREE(devstr); } - } else if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_USB && - cont->model == -1 && - !qemuCapsGet(caps, QEMU_CAPS_PIIX3_USB_UHCI)) { - if (usblegacy) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("Multiple legacy USB controllers are " - "not supported")); - goto error; - } - usblegacy = true; - } else { - virCommandAddArg(cmd, "-device"); - - char *devstr; - if (!(devstr = qemuBuildControllerDevStr(def, cont, caps, - &usbcontroller))) - goto error; - - virCommandAddArg(cmd, devstr); - VIR_FREE(devstr); } } } @@ -5553,7 +5562,6 @@ qemuBuildCommandLine(virConnectPtr conn, virDomainSmartcardDefPtr smartcard = def->smartcards[0]; char *devstr; virBuffer opt = VIR_BUFFER_INITIALIZER; - int j; const char *database; if (def->nsmartcards > 1 || -- 1.7.8.6

2 1

[libvirt] [PATCH] iohelper: fsync() at the end
by Michal Privoznik 01 Nov '12

01 Nov '12

Currently, when we are doing (managed) save, we insert the iohelper between the qemu and OS. The pipe is created, the writing end is passed to qemu and the reading end to the iohelper. It reads data and write them into given file. However, with write() being asynchronous data may still be in OS caches and hence in some (corner) cases, all migration data may have been read and written (not physically though). So qemu will report success, as well as iohelper. However, with some non local filesystems, where ENOSPACE is polled every X time units, we may get into situation where all operations succeeded but data hasn't reached the disk. And in fact will never do. Therefore we ought sync caches to make sure data has reached the block device on remote host. --- For more information follow: https://bugzilla.redhat.com/show_bug.cgi?id=866369 src/util/iohelper.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index c6542ed..aad5cb8 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -179,6 +179,12 @@ runIO(const char *path, int fd, int oflags, unsigned long long length) } } + /* Ensure all data is written */ + if (fsync(fdout) < 0) { + virReportSystemError(errno, _("unable to fsync %s"), fdoutname); + goto cleanup; + } + ret = 0; cleanup: -- 1.7.8.6

2 2

[libvirt] [PATCH] gitignore: ignore more files
by liguang 01 Nov '12

01 Nov '12

ignore *.patch, cscope.po.out, cscope.in.out, tags Signed-off-by: liguang <lig.fnst(a)cn.fujitsu.com> --- .gitignore | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/.gitignore b/.gitignore index 1cd2d45..018b5d9 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,7 @@ *.pyc *.rej *.s +*.patch *~ .deps .gdb_history @@ -181,10 +182,13 @@ /update.log Makefile Makefile.in +tags TAGS coverage cscope.files cscope.out +cscope.in.out +cscope.po.out results.log stamp-h stamp-h.in -- 1.7.1

3 2

[libvirt] libvirt can not get right stats of a rbd pool
by yue 01 Nov '12

01 Nov '12

Allocation exceed Capacity ,but Available is not 0. #virsh pool-info 2361a6d4-0edc-3534-87ae-e7ee09199921 Name: 2361a6d4-0edc-3534-87ae-e7ee09199921 UUID: 2361a6d4-0edc-3534-87ae-e7ee09199921 State: running Persistent: yes Autostart: no Capacity: 285.57 GiB Allocation: 489.89 GiB Available: 230.59 GiB

4 3

[libvirt] [PATCH] build: Fix RPM build for non-x86 platforms
by Viktor Mihajlovski 01 Nov '12

01 Nov '12

Make the post install script for the lock-sanlock package optional to prevent break on non-x86 platforms. Signed-off-by: Viktor Mihajlovski <mihajlov(a)linux.vnet.ibm.com> --- libvirt.spec.in | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/libvirt.spec.in b/libvirt.spec.in index 41d2628..f8ede3f 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1568,12 +1568,13 @@ fi /bin/systemctl try-restart libvirt-guests.service >/dev/null 2>&1 || : %endif +%if %{with_sanlock} %post lock-sanlock if getent group sanlock > /dev/null ; then chmod 0770 %{_localstatedir}/lib/libvirt/sanlock chown root:sanlock %{_localstatedir}/lib/libvirt/sanlock fi - +%endif %files %defattr(-, root, root) -- 1.7.0.4

2 1