- Devel - libvirt List Archives

[libvirt] [PATCH 0/6 v4] Atomic API to list secrets
by Osier Yang 06 Sep '12

06 Sep '12

v3 - v4: * Just rebase on the top, split the patches from v3's larget set Osier Yang (6): list: Define new API virConnectListAllSecrets list: Implement RPC calls for virConnectListAllSecrets list: Implement listAllSecrets list: Expose virConnectListAllSecrets to Python binding list: Expose virConnectListAllSecrets to Python binding virsh: Remove unused vshNameSorter daemon/remote.c | 54 ++++++++++ include/libvirt/libvirt.h.in | 3 + python/generator.py | 1 + python/libvirt-override-api.xml | 6 + python/libvirt-override-virConnect.py | 12 ++ python/libvirt-override.c | 48 +++++++++ src/driver.h | 5 + src/libvirt.c | 50 +++++++++ src/libvirt_public.syms | 1 + src/remote/remote_driver.c | 64 ++++++++++++ src/remote/remote_protocol.x | 13 ++- src/remote_protocol-structs | 12 ++ src/secret/secret_driver.c | 59 +++++++++++- tools/virsh-secret.c | 182 +++++++++++++++++++++++++++----- tools/virsh.c | 9 -- tools/virsh.h | 1 - 16 files changed, 479 insertions(+), 41 deletions(-) -- 1.7.7.3

3 16

[libvirt] An introduction to the "libvirt-designer" library
by Daniel P. Berrange 06 Sep '12

06 Sep '12

People might have noticed Michal's patches[1] for the "libvirt-designer" and wondered wtf.com that is. This is an attempt to explain it... If you are familiar with the code in virt-install (also used by virt-manager) you'll know it has a bunch of internal classes / APIs for dealing with libvirt configuration, particularly wrt guest domains. If you're also familiar with other apps like GNOME Boxes, VDSM/oVirt, OpenStack, then you'll know that these apps have roughly the same needs as virt-install and virt-manager wrt libvirt configuration. The virt-install internal APIs though cannot practically be used by these apps since they were never really designed with broad re-use in mind. Obviously in such scenarios it is desirable to share code across apps instead of having them all reinvent the wheel. The libvirt-gconfig library was the first part in an attempt to address this problem by providing a formal API for creating & reading libvirt configurations without having to know anything about XML. This library is completely policy-free, restricting itself to direct/explicit manipulation of the configuration attributes. This deals with some of functionality found in the virt-install internal APIs. The libosinfo library [2] provides an API for discovering supportable and optimal hardware for operating systems & hypervisors. This replaces the OS configuration data tables found inside virt-install. The next step is to provide a policy-driven API for configurating virtual machines. When configuring a virtual machine, parameters can be roughly split into two groups - guest hardware config and host interaction config. The idea behind the libvirt-designer API is that when creating or changing a guest config, application developers should only need to care about the host interaction config. The guest hardware config can be figured out automatically on their behalf using data obtained from the libosinfo library. As an example, consider adding a disk to a guest. Traditionally an app developer needs to - Choose what virtual hardware to use - Figure out guest device name - Set the host filename - Choose optimal I/O parameters (cache mode, driver type, etc, etc) With the libvirt-designer APIs, an app developer will need to - Set the host filename The rest of the points will be filled in automatically, though the app developer will still get the opportunity to further customize the defaults via the existing libvirt-gconfig APIs. As mentioned above the libvirt-designer library is binding together the libvirt-gconfig and libosinfo APIs. It explicitly does *not* directly depend on libvirt or libvirt-gobject. This is to try to allow the libvirt-designer library to be used by a wide variety of applications, no matter how they are interacting with libvirt. eg if they use the CIM, SNMP or QMF bindings to libvirt they can still use libvirt-designer to build their XML configurations. The intent is that there is yet another library, so far called libvirt-builder, which binds together the libvirt-designer and libvirt-gobject APIs, to actually automate some of the lifecycle management and VM provisioning tasks. Since it will use libvirt-gobject, this final library would not be accessible to apps using CIM/SNMP/QMF bindings. With this library, we would have more or less complete coverage of all the important tasks found in the virt-install internal APIs, which can be reused by other apps. There is of course still quite a long way to go before we get there, since we're aiming to build this all up incrementally from the bottom up. The hope is that in the near term, GNOME Boxes will be able to start taking advantage of some of the code provided by libvirt-designer, since it already uses libvirt-gconfig & libosinfo. In addition as more functionality appears we'll be able to start changing virt-install to switch it over to these APIs in favour of its internal code. Regards, Daniel [1] https://www.redhat.com/archives/libvir-list/2012-September/msg00214.html [2] https://fedorahosted.org/libosinfo -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

1 0

[libvirt] [RFC] [PATCH v2 1/6] add configure option --with-fuse for libvirt
by Gao feng 06 Sep '12

06 Sep '12

add a configure option --with-fuse to prepare introduction of fuse support for libvirt lxc. With help from Daniel Signed-off-by: Gao feng <gaofeng(a)cn.fujitsu.com> --- configure.ac | 36 ++++++++++++++++++++++++++++++++++++ libvirt.spec.in | 9 +++++++++ 2 files changed, 45 insertions(+), 0 deletions(-) diff --git a/configure.ac b/configure.ac index 3cc7b3c..e6d207e 100644 --- a/configure.ac +++ b/configure.ac @@ -1695,6 +1695,37 @@ AC_SUBST([CAPNG_CFLAGS]) AC_SUBST([CAPNG_LIBS]) +dnl libfuse +AC_ARG_WITH([fuse], + AC_HELP_STRING([--with-fuse], [use libfuse to proivde fuse filesystem support for libvirt lxc]), + [], + [with_fuse=check]) + +dnl +dnl This check looks for 'fuse' +dnl +FUSE_CFLAGS= +FUSE_LIBS= +if test "x$with_fuse" != "xno"; then + PKG_CHECK_MODULES([FUSE], [fuse], + [with_fuse=yes], [ + if test "x$with_fuse" = "xcheck" ; then + with_fuse=no + else + AC_MSG_ERROR( + [You must install fuse-devel to compile libvirt]) + fi + ]) + if test "x$with_fuse" = "xyes" ; then + FUSE_LIBS="-lfuse" + FUSE_CFLAGS="-D_FILE_OFFSET_BITS=64" + AC_DEFINE_UNQUOTED([HAVE_FUSE], 1, [whether fuse is available for libvirt lxc]) + fi +fi +AM_CONDITIONAL([HAVE_FUSE], [test "x$with_fuse" = "xyes"]) +AC_SUBST([FUSE_CFLAGS]) +AC_SUBST([FUSE_LIBS]) + dnl virsh libraries AC_CHECK_HEADERS([readline/readline.h]) @@ -2944,6 +2975,11 @@ AC_MSG_NOTICE([ capng: $CAPNG_CFLAGS $CAPNG_LIBS]) else AC_MSG_NOTICE([ capng: no]) fi +if test "$with_fuse" = "yes" ; then +AC_MSG_NOTICE([ fuse: $FUSE_CFLAGS $FUSE_LIBS]) +else +AC_MSG_NOTICE([ fuse: no]) +fi if test "$with_xen" = "yes" ; then AC_MSG_NOTICE([ xen: $XEN_CFLAGS $XEN_LIBS]) else diff --git a/libvirt.spec.in b/libvirt.spec.in index 140a182..939569e 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -91,6 +91,7 @@ # A few optional bits off by default, we enable later %define with_polkit 0%{!?_without_polkit:0} %define with_capng 0%{!?_without_capng:0} +%define with_fuse 0%{!?_without_fuse:0} %define with_netcf 0%{!?_without_netcf:0} %define with_udev 0%{!?_without_udev:0} %define with_hal 0%{!?_without_hal:0} @@ -466,6 +467,9 @@ BuildRequires: numactl-devel %if %{with_capng} BuildRequires: libcap-ng-devel >= 0.5.0 %endif +%if %{with_fuse} +BuildRequires: fuse-devel +%endif %if %{with_phyp} BuildRequires: libssh2-devel %endif @@ -1116,6 +1120,10 @@ of recent versions of Linux (and other OSes). %define _without_capng --without-capng %endif +%if ! %{with_fuse} +%define _without_fuse --without-fuse +%endif + %if ! %{with_netcf} %define _without_netcf --without-netcf %endif @@ -1206,6 +1214,7 @@ autoreconf -if %{?_without_numactl} \ %{?_without_numad} \ %{?_without_capng} \ + %{?_without_fuse} \ %{?_without_netcf} \ %{?_without_selinux} \ %{?_without_hal} \ -- 1.7.7.6

3 12

[libvirt] Bug:libvirtd boot up blocked in NAT network setup
by Wenchao Xia 06 Sep '12

06 Sep '12

Hello, guys I met a strange problem in libvirt+kvm massive deployment case. What shows is that libvirtd will not respond to any virsh calls, with debug I found that libvirtd was blocked at the step to setup default nat network, that is command "iptables -table nat xxxxx". In terminal, I found command "iptables -L" will also block, and "brctl show" shows that about 160 tap devices "vnetXXX" were attached to virbr0. After I deleted these tap devices, things become normal, But when I re-create them and attach these tap device to virbr0, it can't be reproduced. There are too server met this problem in test, so I think it is not a hardware problem. Does someone have experience this before? information: server: RH6.2 libvirt: libvirt-0.9.4 kernel: 2.6.32 -- Best Regards Wenchao Xia

1 0

[libvirt] [PATCH] pci: Save and restore each devices/functions behind the bus
by Osier Yang 06 Sep '12

06 Sep '12

Previously it refuses to do the secondary bus reset as long as there is(are) devices/functions behind the same bus, regardless of whether the devices/functions are being used or not. And it only save and restore the device itself's PCI config space. But later it was changed to allow the secondary bus reset as long as the devices/functions behind the same bus are not being used. Unfortunately, it still just saves and restores the device itself's PCI config space. It means we will lose the PCI config space for the devices share same bus when doing passthrough. Also, (hope my guess is right) as it assumes the secondary reset is allowed unless the device doesn't have devices/functions behind the same bus, so it only reads the bridge control register from the device, but not the parent. This patch fixes the problem by finding out all the devices/functions behind the same bus of the device to be reset, and save/restore PCI config space for all of them. And read the bridge control register from the device's parent (bridge) before resetting. * src/util/pci.c: - New helper pciSharesBus to check if two devices share same bus. - New helper pciDevicesShareBus to return a list containg all of the devices/functions which share same bus with the device - pciTrySecondaryBusReset: Save and restore PCI config space for all the devices/functions behind the same bus; Read the bridge control register from the device's parent instead before resetting. --- src/util/pci.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 files changed, 75 insertions(+), 5 deletions(-) diff --git a/src/util/pci.c b/src/util/pci.c index 0742d07..1a9777a 100644 --- a/src/util/pci.c +++ b/src/util/pci.c @@ -517,6 +517,39 @@ pciBusContainsActiveDevices(pciDevice *dev, return active; } +/* + * Check if the @dev and @check share bus. + */ +static int +pciSharesBus(pciDevice *dev, pciDevice *check, void *data ATTRIBUTE_UNUSED) +{ + if ((dev->domain == check->domain) && + (dev->bus == check->bus) && + (dev->slot == check->slot)) + return 1; + + return 0; +} + +/* + * Return all the devices/functions share same bus with @dev + * as a list. + */ +static pciDeviceList * +pciDevicesShareBus(pciDevice *dev) +{ + pciDevice *match = NULL; + pciDeviceList *pcis = NULL; + + if (!(pcis = pciDeviceListNew())) + return NULL; + + if (pciIterDevices(pciSharesBus, dev, &match, NULL)) + pciDeviceListAdd(pcis, match); + + return pcis; +} + /* Is @check the parent of @dev ? */ static int pciIsParent(pciDevice *dev, pciDevice *check, void *data) @@ -604,6 +637,9 @@ pciTrySecondaryBusReset(pciDevice *dev, uint8_t config_space[PCI_CONF_LEN]; uint16_t ctl; int ret = -1; + pciDeviceList *list = NULL; + uint8_t (*config_spaces)[PCI_CONF_LEN]; + int i; /* Refuse to do a secondary bus reset if there are other * devices/functions behind the bus are used by the host @@ -628,10 +664,7 @@ pciTrySecondaryBusReset(pciDevice *dev, VIR_DEBUG("%s %s: doing a secondary bus reset", dev->id, dev->name); - /* Save and restore the device's config space; we only do this - * for the supplied device since we refuse to do a reset if there - * are multiple devices/functions - */ + /* Save the device's config space */ if (pciRead(dev, 0, config_space, PCI_CONF_LEN) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Failed to read PCI config space for %s"), @@ -639,10 +672,29 @@ pciTrySecondaryBusReset(pciDevice *dev, goto out; } + /* Save the config space of devices behind the same bus */ + if ((list = pciDevicesShareBus(dev))) { + if (VIR_ALLOC_N(config_spaces, list->count) < 0) { + virReportOOMError(); + goto out; + } + + for (i = 0; i < list->count; i++) { + pciDevice *pci = list->devs[i]; + + if (pciRead(pci, 0, config_spaces[i], PCI_CONF_LEN) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Failed to read PCI config space for %s"), + pci->name); + goto out; + } + } + } + /* Read the control register, set the reset flag, wait 200ms, * unset the reset flag and wait 200ms. */ - ctl = pciRead16(dev, PCI_BRIDGE_CONTROL); + ctl = pciRead16(parent, PCI_BRIDGE_CONTROL); pciWrite16(parent, PCI_BRIDGE_CONTROL, ctl | PCI_BRIDGE_CTL_RESET); @@ -652,14 +704,32 @@ pciTrySecondaryBusReset(pciDevice *dev, usleep(200 * 1000); /* sleep 200ms */ + /* Restore the device's config space */ if (pciWrite(dev, 0, config_space, PCI_CONF_LEN) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Failed to restore PCI config space for %s"), dev->name); goto out; } + + /* Restore the config space of devices behind the same bus */ + if (list) { + for (i = 0; i < list->count; i++) { + pciDevice *pci = list->devs[i]; + + if (pciWrite(pci, 0, config_spaces[i], PCI_CONF_LEN) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Failed to restore PCI config space for %s"), + pci->name); + goto out; + } + } + } + ret = 0; out: + pciDeviceListFree(list); + VIR_FREE(config_spaces); pciFreeDevice(parent); return ret; } -- 1.7.7.3

2 1

[libvirt] [PATCH v2 0/4] Add cpu hotplug support to libvirt.
by Tang Chen 06 Sep '12

06 Sep '12

It seems that libvirt is not cpu hotplug aware. Please refer to the following problem. 1. At first, we have 2 cpus. # cat /cgroup/cpuset/cpuset.cpus 0-1 # cat /cgroup/cpuset/libvirt/qemu/cpuset.cpus 0-1 2. And we have a vm1 with following configuration. <cputune> <vcpupin vcpu='0' cpuset='1'/> <emulatorpin cpuset='1'/> </cputune> 3. Offline cpu1. # echo 0 > /sys/devices/system/cpu/cpu1/online # cat /sys/devices/system/cpu/cpu1/online 0 # cat /cgroup/cpuset/cpuset.cpus 0 # cat /cgroup/cpuset/libvirt/qemu/cpuset.cpus 0 # cat /cgroup/cpuset/libvirt/lxc/cpuset.cpus 0 4. Online cpu1. # echo 1 > /sys/devices/system/cpu/cpu1/online # cat /sys/devices/system/cpu/cpu1/online 1 # cat /cgroup/cpuset/cpuset.cpus 0-1 # cat /cgroup/cpuset/libvirt/cpuset.cpus 0 # cat /cgroup/cpuset/libvirt/qemu/cpuset.cpus 0 # cat /cgroup/cpuset/libvirt/lxc/cpuset.cpus 0 Here，cgroup updated cpuset.cpus，but not for libvirt directory，and also qemu and lxc directory. vm1 cannot be started again. # virsh start vm1 error: Failed to start domain vm1 error: Unable to set cpuset.cpus: Permission denied And libvird gave the following errors. 2012-07-17 07:30:22.478+0000: 3118: error : qemuSetupCgroupVcpuPin:498 : Unable to set cpuset.cpus: Permission denied These patches resolves this problem by listening on the netlink for cpu hotplug event. When the netlink service gets the cpu hotplug event, it will attract the cpuid in the message, and add it into cpuset.cpus in: /cgroup/cpuset/libvirt /cgroup/cpuset/libvirt/qemu /cgroup/cpuset/libvirt/lxc change log of v2: [PATCH v2 1/4]: * Move all the handlers from hotplug.c to modules who want to listen to cpu hotplug events. [PATCH v2 2/4]: * Set /libvirt/cpuset.cpus to the lastest value when libvirtd is being initialized. [PATCH v2 3/4]: * Set /libvirt/qemu/cpuset.cpus to the lastest value when qemu driver is being initialized. [PATCH v2 4/4]: * Set /libvirt/lxc/cpuset.cpus to the lastest value when lxc driver is being initialized. Tang Chen (4): Add helpers to support cpu hotplug in libvirt. Register cpu hotplug netlink handler for libvirtd. Register cpu hotplug netlink handler for qemu driver. Register cpu hotplug netlink handler for lxc driver. daemon/libvirtd.c | 62 +++++++++++ include/libvirt/virterror.h | 2 + src/Makefile.am | 1 + src/libvirt_private.syms | 7 ++ src/lxc/lxc_driver.c | 67 +++++++++++- src/qemu/qemu_driver.c | 64 ++++++++++++ src/util/cgroup.c | 10 +- src/util/cgroup.h | 7 ++ src/util/hotplug.c | 240 +++++++++++++++++++++++++++++++++++++++++++ src/util/hotplug.h | 44 ++++++++ src/util/virterror.c | 3 +- 11 files changed, 499 insertions(+), 8 deletions(-) create mode 100644 src/util/hotplug.c create mode 100644 src/util/hotplug.h -- 1.7.10.1

4 12

[libvirt] [PATCH 0/2] Fix 'make' issues on OpenBSD.
by Eric Blake 06 Sep '12

06 Sep '12

Pushing both under the build-breaker rule. 'make check' still fails some gnulib tests on OpenBSD, but that's not quite as severe. Eric Blake (2): build: avoid warnings from gcc 4.2.1 build: don't fail when xsltproc is missing docs/Makefile.am | 9 +++++---- m4/virt-compile-warnings.m4 | 8 +++++++- src/internal.h | 6 ++++-- 3 files changed, 16 insertions(+), 7 deletions(-) -- 1.7.11.4

1 5

[libvirt] [PATCH] build: require netcf-0.2.2 when installing on Fedora18+
by Laine Stump 06 Sep '12

06 Sep '12

A previous patch forced libnl-3 and netcf-0.2.2 (which itself requires libnl-3) when *building* for Fedora 18+ (and RHEL 7+), but the install-time Requires: for netcf has always been implicit due to libvirtd linking with libnetcf.so. However, the since the API of netcf didn't change when it was rebuilt to use libnl-3, the internal library version didn't change either, making it possible (from rpm's point of view) to upgrade libvirt without upgrading netcf (in reality, that leads to a segfault - see https://bugzilla.redhat.com/show_bug.cgi?id=853381) The solution is to put an explicit Requires: line in libvirt's specfile for fedora >= 18 and rhel >= 7. --- libvirt.spec.in | 3 +++ 1 file changed, 3 insertions(+) diff --git a/libvirt.spec.in b/libvirt.spec.in index 9a1feed..cc5247b 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -589,6 +589,9 @@ Requires: iptables-ipv6 %if %{with_nwfilter} Requires: ebtables %endif +%if %{with_netcf} && (0%{?fedora} >= 18 || 0%{?rhel} >= 7) +Requires: netcf-libs >= 0.2.2 +%endif # needed for device enumeration %if %{with_hal} Requires: hal -- 1.7.11.4

2 2

[libvirt] [PATCH] network: prevent infinite hang if ovs-vswitchd isn't running
by Laine Stump 06 Sep '12

06 Sep '12

This fixes https://bugzilla.redhat.com/show_bug.cgi?id=852984 If a network or interface is configured to use Open vSwitch, but ovs-vswitchd (the Open vSwitch database service) isn't running, the ovs-vsctl add-port/del-port commands will hang indefinitely rather than returning an error. There is a --nowait option, but that appears to have no effect on add-port and del-port commands, so instead we add a --timeout=5 to the commands - they will retry for up to 5 seconds, then fail if there is no response. --- I'm not sure if 5 seconds is a reasonable timeout for this, but it seems to work okay for me (in a very lightly loaded environment, but a successful return is always effectively instantaneous...). Unfortunately there doesn't seem to be another way to bound the wait time of ovs-vsctl. :-( src/util/virnetdevopenvswitch.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/util/virnetdevopenvswitch.c b/src/util/virnetdevopenvswitch.c index 764f478..97b9d52 100644 --- a/src/util/virnetdevopenvswitch.c +++ b/src/util/virnetdevopenvswitch.c @@ -105,7 +105,7 @@ int virNetDevOpenvswitchAddPort(const char *brname, const char *ifname, cmd = virCommandNew(OVSVSCTL); - virCommandAddArgList(cmd, "--", "--may-exist", "add-port", + virCommandAddArgList(cmd, "--timeout=5", "--", "--may-exist", "add-port", brname, ifname, NULL); if (virBufferUse(&buf) != 0) @@ -166,7 +166,7 @@ int virNetDevOpenvswitchRemovePort(const char *brname ATTRIBUTE_UNUSED, const ch virCommandPtr cmd = NULL; cmd = virCommandNew(OVSVSCTL); - virCommandAddArgList(cmd, "--", "--if-exists", "del-port", ifname, NULL); + virCommandAddArgList(cmd, "--timeout=5", "--", "--if-exists", "del-port", ifname, NULL); if (virCommandRun(cmd, NULL) < 0) { virReportSystemError(VIR_ERR_INTERNAL_ERROR, -- 1.7.11.4

2 2

[libvirt] VM Migration with IEEE802.1Qbh and IEEE802.1Qbg
by Jagath Weerasinghe 06 Sep '12

06 Sep '12

Hi All, Can a VM connected as IEEE802.1Qbh be migrated to a destination host as it were connected as IEEE802.1Qbg? What I wanted to know is, is it possible to change the configuration from IEEE802.1Qbh to IEEE802.1Qbg on the fly? Regards Jagath

3 3