- Devel - libvirt List Archives

[libvirt] NL_RECV RETURNED WITH ERROR: NO BUFFER SPACE AVAILABLE ERROR FOR IXGBE LOAD
by vaughan 18 Jun '14

18 Jun '14

Hi experts, Release: OL7 Kernel: 3.10.0-121.el7.x86_64 Noticed below error on OL7 server, while loading Intel 10gigabit nic driver module , ixgbe in syslog -------------------------------------------------------------------- journal: nl_recv returned with error: No buffer space available ------------------------------------------------------------------- Complete syslog content for ixgbe module load : un 16 20:46:10 ca-ostest432 kernel: ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 3.15.1-k Jun 16 20:46:10 ca-ostest432 kernel: ixgbe: Copyright (c) 1999-2013 Intel Corporation. Jun 16 20:46:10 ca-ostest432 kvm: 1 guest now active Jun 16 20:46:10 ca-ostest432 kvm: 0 guests now active Jun 16 20:46:10 ca-ostest432 kernel: ixgbe 0000:13:00.0: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 Jun 16 20:46:10 ca-ostest432 kernel: ixgbe 0000:13:00.0: (PCI Express:5.0GT/s:Width x8) 00:1b:21:c8:24:74 Jun 16 20:46:10 ca-ostest432 kernel: ixgbe 0000:13:00.0: MAC: 2, PHY: 9, SFP+: 3, PBA No: E70856-007 Jun 16 20:46:10 ca-ostest432 kernel: ixgbe 0000:13:00.0: PCI Express bandwidth of 32GT/s available Jun 16 20:46:10 ca-ostest432 kernel: ixgbe 0000:13:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%) Jun 16 20:46:10 ca-ostest432 journal: nl_recv returned with error: No buffer space available Jun 16 20:46:11 ca-ostest432 journal: nl_recv returned with error: No buffer space available Jun 16 20:46:11 ca-ostest432 systemd-udevd[4939]: renamed network interface eth0 to ens2f0 Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.0: Intel(R) 10 Gigabit Network Connection Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: (PCI Express:5.0GT/s:Width x8) 00:1b:21:c8:24:75 Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: MAC: 2, PHY: 9, SFP+: 4, PBA No: E70856-007 Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: PCI Express bandwidth of 32GT/s available Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%) Jun 16 20:46:11 ca-ostest432 systemd-udevd[4984]: renamed network interface eth0 to ens2f1 Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: Intel(R) 10 Gigabit Network Connection Jun 16 20:46:11 ca-ostest432 NetworkManager[1316]: <info> (ens2f0): carrier is OFF (but ignored) Jun 16 20:46:11 ca-ostest432 NetworkManager[1316]: <info> (ens2f0): new Ethernet device (driver: 'ixgbe' ifindex: 14) Jun 16 20:46:11 ca-ostest432 NetworkManager[1316]: <info> (ens2f0): exported as /org/freedesktop/NetworkManager/Devices/13 Jun 16 20:46:11 ca-ostest432 NetworkManager[1316]: <info> (ens2f0): device state change: unmanaged -> unavailable (reason 'managed') [10 20 2] Jun 16 20:46:11 ca-ostest432 journal: nl_recv returned with error: No buffer space available Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.0: registered PHC device on ens2f0 Jun 16 20:46:12 ca-ostest432 kernel: IPv6: ADDRCONF(NETDEV_UP): ens2f0: link is not ready Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f0): preparing device Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1): carrier is OFF (but ignored) Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1): new Ethernet device (driver: 'ixgbe' ifindex: 15) Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1): exported as /org/freedesktop/NetworkManager/Devices/14 Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1): device state change: unmanaged -> unavailable (reason 'managed') [10 20 2] Jun 16 20:46:12 ca-ostest432 journal: nl_recv returned with error: No buffer space available Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1): preparing device Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f0): device state change: unavailable -> disconnected (reason 'none') [20 30 0] Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1): device state change: unavailable -> disconnected (reason 'none') [20 30 0] Jun 16 20:46:12 ca-ostest432 kernel: ixgbe 0000:13:00.0 ens2f0: detected SFP+: 3 Jun 16 20:46:12 ca-ostest432 kernel: ixgbe 0000:13:00.1: registered PHC device on ens2f1 Jun 16 20:46:12 ca-ostest432 kernel: IPv6: ADDRCONF(NETDEV_UP): ens2f1: link is not ready Jun 16 20:46:12 ca-ostest432 kernel: ixgbe 0000:13:00.1 ens2f1: detected SFP+: 4 Jun 16 20:46:12 ca-ostest432 kernel: ixgbe 0000:13:00.1 ens2f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX Jun 16 20:46:12 ca-ostest432 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): ens2f1: link becomes ready Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1): link connected Jun 16 20:46:12 ca-ostest432 kernel: ixgbe 0000:13:00.0 ens2f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX Jun 16 20:46:12 ca-ostest432 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): ens2f0: link becomes ready Jun 16 20:46:13 ca-ostest432 NetworkManager[1316]: <info> (ens2f0): link connected --------------------------------------------------------------------------------------------- Issue is not seen for other nic drivers when loaded , i.e verified with be2net ,e1000e , igb drivers , so it looks like specific to ixgbe driver module. This problem can be addressed by increasing the size of the socket receive buffer via nl_socket_set_buffer_size(). However, since reliable transmissions from kernel to user are impossible in any case, I don't know if we need to ensure enough buffer and how much it will be enough. In my test, 32768*4 bytes of receive buffer size is ok. Below is my patch. >From 4505e0001563bcef814dd2c6dc02b2d83e521867 Mon Sep 17 00:00:00 2001 From: rpm-build <rpm-build> Date: Mon, 16 Jun 2014 20:05:08 +0800 Subject: [PATCH] set nl socket buffer size to 32768*4 Signed-off-by: Vaughan Cao <vaughan.cao(a)oracle.com> --- src/util/virnetlink.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/util/virnetlink.c b/src/util/virnetlink.c index 5849ed7..cf44ad2 100644 --- a/src/util/virnetlink.c +++ b/src/util/virnetlink.c @@ -561,6 +561,15 @@ virNetlinkEventServiceStart(unsigned int protocol, unsigned int groups) goto error_server; } + if (nl_socket_set_buffer_size(srv->netlinknh, (32768*4), 0)) { + virReportSystemError(errno, "%s", + _("cannot set netlink socket buffer size to 32768*4")); + goto error_server; + } else { + virReportSystemError(0, "%s", + _("set netlink socket buffer size to 32768*4")); + } + if ((srv->eventwatch = virEventAddHandle(fd, VIR_EVENT_HANDLE_READABLE, virNetlinkEventCallback, -- 1.9.0 -- Regards, Vaughan

4 3

[libvirt] [PATCH 0/2] fix issues in vbox found by make check on RHEL6
by Pavel Hrdina 18 Jun '14

18 Jun '14

Pavel Hrdina (2): tests: fix vbox snapshot xmls vbox_snapshot_conf: fix wrong use of 'xmlSaveFormatFileEnc' src/vbox/vbox_snapshot_conf.c | 2 +- tests/vboxsnapshotxmldata/2disks-1snap.vbox | 292 +++---- tests/vboxsnapshotxmldata/2disks-2snap.vbox | 584 +++++++------- .../vboxsnapshotxmldata/2disks-3snap-brother.vbox | 292 +++---- tests/vboxsnapshotxmldata/2disks-3snap.vbox | 876 ++++++++++----------- 5 files changed, 1023 insertions(+), 1023 deletions(-) -- 1.8.5.5

2 5

[libvirt] [PATCH 0/2] virsh: man: Tweak some old docs problems
by Peter Krempa 18 Jun '14

18 Jun '14

Peter Krempa (2): virsh: man: Fix examples and docs for virsh version virsh: man: Correctly spell QEMU tools/virsh.pod | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) -- 1.9.3

2 4

[libvirt] [PATCH 00/12] Couple of host info getting improvements
by Michal Privoznik 18 Jun '14

18 Jun '14

The rationale is to fill in the gaps as described here: https://wiki.openstack.org/wiki/VirtDriverGuestCPUMemoryPlacement Note that these are targeted for 1.2.6. Michal Privoznik (12): Introduce virNodeHugeTLB Initial virsh exposure of virNodeHugeTLB nodeinfo: Implement nodeHugeTLB virpci: Introduce virPCIDeviceIsPCIExpress and friends nodedev: Introduce <pci-express/> to PCI devices virInterface: Expose link state & speed interface_backend_udev: Implement link speed & state interface_backend_netcf: Implement link speed & state virhostdev: Move IOMMU and VFIO funcs from qemu virCaps: Introduce IOMMU and VFIO capabilities Expose IOMMU and VFIO capabilities from virCaps nodedev: Export NUMA node locality for PCI devices daemon/remote.c | 54 ++++++ docs/formatcaps.html.in | 8 +- docs/formatnode.html.in | 25 +++ docs/schemas/capability.rng | 12 ++ docs/schemas/interface.rng | 27 +++ docs/schemas/nodedev.rng | 37 ++++ include/libvirt/libvirt.h.in | 26 +++ src/conf/capabilities.c | 50 ++++++ src/conf/capabilities.h | 11 ++ src/conf/interface_conf.c | 39 ++++- src/conf/interface_conf.h | 15 ++ src/conf/node_device_conf.c | 166 +++++++++++++++++- src/conf/node_device_conf.h | 32 +++- src/driver.h | 7 + src/interface/interface_backend_netcf.c | 99 +++++++++++ src/interface/interface_backend_udev.c | 19 ++ src/libvirt.c | 59 +++++++ src/libvirt_private.syms | 13 ++ src/libvirt_public.syms | 5 + src/lxc/lxc_conf.c | 3 + src/node_device/node_device_udev.c | 38 ++++ src/nodeinfo.c | 192 +++++++++++++++++++++ src/nodeinfo.h | 6 + src/parallels/parallels_driver.c | 3 + src/phyp/phyp_driver.c | 3 + src/qemu/qemu_capabilities.c | 3 + src/qemu/qemu_driver.c | 30 +++- src/qemu/qemu_hostdev.c | 93 ++-------- src/qemu/qemu_hostdev.h | 2 - src/remote/remote_driver.c | 48 ++++++ src/remote/remote_protocol.x | 19 +- src/remote_protocol-structs | 13 ++ src/uml/uml_conf.c | 3 + src/util/virhostdev.c | 73 ++++++++ src/util/virhostdev.h | 4 + src/util/virpci.c | 81 ++++++++- src/util/virpci.h | 8 + src/vbox/vbox_tmpl.c | 3 + tests/capabilityschemadata/caps-qemu-kvm.xml | 2 + tests/capabilityschemadata/caps-test.xml | 2 + tests/capabilityschemadata/caps-test2.xml | 2 + tests/capabilityschemadata/caps-test3.xml | 2 + tests/interfaceschemadata/bridge-no-address.xml | 1 + tests/interfaceschemadata/bridge.xml | 1 + tests/interfaceschemadata/ethernet-dhcp.xml | 1 + tests/nodedevschemadata/pci_1002_71c4.xml | 1 + tests/nodedevschemadata/pci_8086_10c9_sriov_pf.xml | 1 + .../pci_8086_4238_pcie_wireless.xml | 26 +++ tests/nodedevxml2xmltest.c | 1 + tests/xencapsdata/xen-i686-pae-hvm.xml | 2 + tests/xencapsdata/xen-i686-pae.xml | 2 + tests/xencapsdata/xen-i686.xml | 2 + tests/xencapsdata/xen-ia64-be-hvm.xml | 2 + tests/xencapsdata/xen-ia64-be.xml | 2 + tests/xencapsdata/xen-ia64-hvm.xml | 2 + tests/xencapsdata/xen-ia64.xml | 2 + tests/xencapsdata/xen-ppc64.xml | 2 + tests/xencapsdata/xen-x86_64-hvm.xml | 2 + tests/xencapsdata/xen-x86_64.xml | 2 + tools/virsh-host.c | 62 +++++++ tools/virsh.pod | 7 + 61 files changed, 1371 insertions(+), 87 deletions(-) create mode 100644 tests/nodedevschemadata/pci_8086_4238_pcie_wireless.xml -- 1.9.3

3 30

Re: [libvirt] [PATCH v2 4/4] libxl: Add a test suite for libxl option generator
by Ian Campbell 18 Jun '14

18 Jun '14

On Mon, 2014-06-16 at 17:11 -0600, Jim Fehlig wrote: > > This function exists in Xen 4.2 as well, in libxl.h. > > > > Any ideas on how to handle this? I'm not aware of an existing macro to > check for func 'foo' defined in header 'bar'. Is writing a custom macro > along these lines a good solution? A bad solution I tried was hacking > the test to check libxl version via libxl_get_version_info(), but that > API does not work if not running Xen. Given that it exists in everything from 4.2 onwards why do you need to check for it? >From the PoV of these tests (or any application generally) I'd have thought you wouldn't really care if you get this function from libxl.h directly or indirectly via libxl.h including _libxl_types.h. Ian.

3 3

[libvirt] [PATCH 1/1] virt-aa-helper: allow access to /dev/vhost-net if needed
by Serge Hallyn 18 Jun '14

18 Jun '14

Only allow the access if it is a KVM domain which has a NIC which wants non-userspace networking. This addresses https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1322568 Signed-off-by: Serge Hallyn <serge.hallyn(a)ubuntu.com> --- src/security/virt-aa-helper.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/src/security/virt-aa-helper.c b/src/security/virt-aa-helper.c index 1d246c7..e54f73f 100644 --- a/src/security/virt-aa-helper.c +++ b/src/security/virt-aa-helper.c @@ -929,7 +929,7 @@ get_files(vahControl * ctl) size_t i; char *uuid; char uuidstr[VIR_UUID_STRING_BUFLEN]; - bool needsVfio = false; + bool needsVfio = false, needsvhost = false; /* verify uuid is same as what we were given on the command line */ virUUIDFormat(ctl->def->uuid, uuidstr); @@ -1105,6 +1105,21 @@ get_files(vahControl * ctl) } } + if (ctl->def->virtType == VIR_DOMAIN_VIRT_KVM) { + for (i = 0; i < ctl->def->nnets; i++) { + virDomainNetDefPtr net = ctl->def->nets[i]; + if (net && net->model) { + if (net->driver.virtio.name == VIR_DOMAIN_NET_BACKEND_TYPE_QEMU) + continue; + if (STRNEQ(net->model, "virtio")) + continue; + } + needsvhost = true; + } + } + if (needsvhost) + virBufferAddLit(&buf, " /dev/vhost-net rw,\n"); + if (needsVfio) { virBufferAddLit(&buf, " /dev/vfio/vfio rw,\n"); virBufferAddLit(&buf, " /dev/vfio/[0-9]* rw,\n"); -- 1.9.1

2 1

[libvirt] [PATCH v3] Added example script on how to convert LXC container config
by Cédric Bosdonnat 18 Jun '14

18 Jun '14

--- Diff to v2: * Added copyright + license header * Fixed the nits Makefile.am | 3 +- configure.ac | 1 + examples/lxcconvert/Makefile.am | 18 +++++ examples/lxcconvert/virt-lxc-convert | 127 +++++++++++++++++++++++++++++++++++ 4 files changed, 148 insertions(+), 1 deletion(-) create mode 100644 examples/lxcconvert/Makefile.am create mode 100644 examples/lxcconvert/virt-lxc-convert diff --git a/Makefile.am b/Makefile.am index b961c0e..abc5a2f 100644 --- a/Makefile.am +++ b/Makefile.am @@ -23,7 +23,8 @@ SUBDIRS = . gnulib/lib include src daemon tools docs gnulib/tests \ tests po examples/object-events examples/hellolibvirt \ examples/dominfo examples/domsuspend examples/apparmor \ examples/xml/nwfilter examples/openauth examples/systemtap \ - tools/wireshark examples/dommigrate + tools/wireshark examples/dommigrate \ + examples/lxcconvert ACLOCAL_AMFLAGS = -I m4 diff --git a/configure.ac b/configure.ac index 368d094..710cb71 100644 --- a/configure.ac +++ b/configure.ac @@ -2742,6 +2742,7 @@ AC_CONFIG_FILES([\ examples/hellolibvirt/Makefile \ examples/systemtap/Makefile \ examples/xml/nwfilter/Makefile \ + examples/lxcconvert/Makefile \ tools/wireshark/Makefile \ tools/wireshark/src/Makefile]) AC_OUTPUT diff --git a/examples/lxcconvert/Makefile.am b/examples/lxcconvert/Makefile.am new file mode 100644 index 0000000..09cf5d9 --- /dev/null +++ b/examples/lxcconvert/Makefile.am @@ -0,0 +1,18 @@ +## Copyright (C) 2014 SUSE LINUX Products GmbH, Nuernberg, Germany. +## +## This library is free software; you can redistribute it and/or +## modify it under the terms of the GNU Lesser General Public +## License as published by the Free Software Foundation; either +## version 2.1 of the License, or (at your option) any later version. +## +## This library is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +## Lesser General Public License for more details. +## +## You should have received a copy of the GNU Lesser General Public +## License along with this library. If not, see +## <http://www.gnu.org/licenses/>. + +EXTRA_DIST= \ + virt-lxc-convert diff --git a/examples/lxcconvert/virt-lxc-convert b/examples/lxcconvert/virt-lxc-convert new file mode 100644 index 0000000..7220153 --- /dev/null +++ b/examples/lxcconvert/virt-lxc-convert @@ -0,0 +1,127 @@ +#!/bin/sh +# lxc_native.c: LXC native configuration import +# +# Copyright (c) 2014 SUSE LINUX Products GmbH, Nuernberg, Germany. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# <http://www.gnu.org/licenses/>. +# +# Author: Cedric Bosdonnat <cbosdonnat(a)suse.com> + +handler_cleanup() +{ + if test "$conf_dir"; then + # Remove the temporary config + rm -r "$conf_dir" + fi +} +trap handler_cleanup INT EXIT + +show_help() +{ + cat << EOF +$0 /path/to/lxc/config/file + +Wrapper around virsh domxml-from-native to ease conversion of LXC +containers configuration to libvirt domain XML. +EOF +} + +if test $# != 1; then + show_help + exit 1 +fi + +if test "$1" = "--help" || test "$1" = "-h"; then + show_help + exit $? +fi + +conf=$1 + +conf_dir=$(mktemp --tmpdir -d virt-lxc-convert-XXX) +conf_new=$conf_dir/config + +cp "$conf" "$conf_new" + +# Do we have lxc.mount, and is it pointing to a readable file? +fstab=$(sed -n '/lxc.mount[[:space:]]*=/ s/[[:space:]]*=[[:space:]]*/=/p' \ + "$conf_new" | cut -f 2 -d '=') +if test -r "$fstab"; then + sed 's/^lxc.mount[[:space:]]*=.*$//' "$conf_new" >"${conf_new}.tmp" + mv "${conf_new}.tmp" "${conf_new}" + sed 's/^$[^#]$/lxc.mount.entry = \1/' "$fstab" >>"${conf_new}" +fi + +memory=$(free | sed -n '/Mem:/s/ \+/ /gp' | cut -f 2 -d ' ') +default_tmpfs="size=$((memory/2))" + +# Do we have tmpfs without size param? +lineno=0 +while read line; do + lineno=$(expr $lineno + 1) + has_rel_size=false + case $line in + lxc.mount.entry[[:space:]]*=[[:space:]]*tmpfs[[:space:]]*) + is_tmpfs=true + ;; + *) + is_tmpfs=false + ;; + esac + + # We only care about tmpfs mount entries here + if ! $is_tmpfs; then + continue + fi + + case $line in + *size=[0-9][0-9]*%*) + has_rel_size=true + has_size=true + ;; + *size=*) + has_size=true + ;; + *) + has_size=false + ;; + esac + + # Add the default size here (50%) if no size is given + if ! $has_size; then + last_option_match="$[[:space:]]*[0-9][[:space:]]*[0-9][::space::]*$$" + sed "${lineno}s/$last_option_match/,$default_tmpfs\1/" \ + "$conf_new" >"${conf_new}.tmp" + mv "${conf_new}.tmp" "${conf_new}" + fi + + # Convert relative sizes + if $has_rel_size; then + percent=$(echo "$line" | sed 's/.*size=$[0-9][0-9]*$%.*/\1/') + size="$((memory*percent/100))" + sed "${lineno}s/size=[0-9]*%/size=${size}/" \ + "$conf_new" >"${conf_new}.tmp" + mv "${conf_new}.tmp" "${conf_new}" + fi +done < "$conf_new" + +# Do we have any memory limit set? +mem_limit=$(grep 'lxc.cgroup.memory.limit_in_bytes[[:space:]]*=' $conf_new) +if test -z "$mem_limit"; then + echo "lxc.cgroup.memory.limit_in_bytes = $memory" >> "$conf_new" +fi + +virsh -c lxc:/// domxml-from-native lxc-tools $conf_new +exit $? -- 1.8.4.5

2 1

[libvirt] [PATCH] vbox: fix a segfault when taking a snapshot
by Yohan BELLEGUIC 18 Jun '14

18 Jun '14

there is a segfault in the vbox driver when taking a snapshot in the following functions: - vboxDomainGetXMLDesc - vboxSnapshotGetReadWriteDisks - vboxSnapshotGetReadOnlyDisks The virStorageSourcePtr in virDomainDiskDef was not correctly allocated. --- src/vbox/vbox_tmpl.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/src/vbox/vbox_tmpl.c b/src/vbox/vbox_tmpl.c index 1ed2729..6365f2a 100644 --- a/src/vbox/vbox_tmpl.c +++ b/src/vbox/vbox_tmpl.c @@ -2872,10 +2872,12 @@ static char *vboxDomainGetXMLDesc(virDomainPtr dom, unsigned int flags) { /* Allocate mem, if fails return error */ if (VIR_ALLOC_N(def->disks, def->ndisks) >= 0) { for (i = 0; i < def->ndisks; i++) { - if (VIR_ALLOC(def->disks[i]) < 0) { + virDomainDiskDefPtr disk = virDomainDiskDefNew(); + if (!disk) { error = true; break; } + def->disks[i] = disk; } } else { error = true; @@ -7175,6 +7177,10 @@ int vboxSnapshotGetReadWriteDisks(virDomainSnapshotDefPtr def, /* Allocate mem, if fails return error */ if (VIR_ALLOC_N(def->disks, def->ndisks) < 0) goto cleanup; + for (i = 0; i < def->ndisks; i++) { + if (VIR_ALLOC(def->disks[i].src) < 0) + goto cleanup; + } if (!vboxGetMaxPortSlotValues(data->vboxObj, maxPortPerInst, maxSlotPerPort)) goto cleanup; @@ -7302,11 +7308,11 @@ int vboxSnapshotGetReadWriteDisks(virDomainSnapshotDefPtr def, ret = 0; cleanup: if (ret < 0) { - for (i = 0; i < def->dom->ndisks; i++) - VIR_FREE(def->dom->disks[i]); - VIR_FREE(def->dom->disks); - def->dom->ndisks = 0; - ret = -1; + for (i = 0; i < def->ndisks; i++) { + VIR_FREE(def->disks[i].src); + } + VIR_FREE(def->disks); + def->ndisks = 0; } VBOX_RELEASE(snap); return ret; @@ -7380,8 +7386,10 @@ int vboxSnapshotGetReadOnlyDisks(virDomainSnapshotPtr snapshot, /* Allocate mem, if fails return error */ if (VIR_ALLOC_N(def->dom->disks, def->dom->ndisks) >= 0) { for (i = 0; i < def->dom->ndisks; i++) { - if (VIR_ALLOC(def->dom->disks[i]) < 0) + virDomainDiskDefPtr diskDef = virDomainDiskDefNew(); + if (!diskDef) goto cleanup; + def->dom->disks[i] = diskDef; } } else { goto cleanup; @@ -7516,7 +7524,7 @@ int vboxSnapshotGetReadOnlyDisks(virDomainSnapshotPtr snapshot, cleanup: if (ret < 0) { for (i = 0; i < def->dom->ndisks; i++) - VIR_FREE(def->dom->disks[i]); + virDomainDiskDefFree(def->dom->disks[i]); VIR_FREE(def->dom->disks); def->dom->ndisks = 0; } -- 1.7.10.4

2 1

[libvirt] [PATCH] [POC] Support cgmanager
by Serge Hallyn 18 Jun '14

18 Jun '14

I'm sending this patch mainly as an FYI. Cgmanager is a cgroup manager which accepts cgroup administration commands over dbus and allows proper delegation in the face of user namespaces. We are using it to support nested lxc containers, and so wanted to port libvirt to it as well to match dependencies. However, until we have a chance to better define cgmanager's future (if any) and relationship to systemd, I'm certainly not asking for this to be carried in libvirt upstream. With this patch, kvm vms and lxc containers work fine with cgmanager installed and cgroups (beside name=systemd) not installed system-wide. I tweaked the container startup process to have the container enter its cgroup after configuring it. Previously it was entering it as soon as it was created, then configuring it. If we pre-compute the per-controller placement then we might be able to avoid this, but requests placed with cgmanager by design are relative to your current cgroup, so this was simpler. TODO: The vircgroup test is disabled with cgmanager support TODO: Libvirt should try to move itself into the root cgroup (using MovePidAbs("/", getpid) in case a user is starting it from inside a non-init cgroup. That is not being done because I hit linker errors and I deemed it non-essential for v1. This patch is done intrusively in a "just make it work" manner, though in my defense basically the same way as the systemd one. If the feature were not rejected outright upstream, then refactoring the cgroup support to support all three (systemd, native-fs, and cgmanager) plus any others (and of course abiding the coding style), would be worthwhile. Signed-off-by: Serge Hallyn <serge.hallyn(a)ubuntu.com> --- configure.ac | 15 ++ daemon/Makefile.am | 7 +- daemon/libvirtd.c | 5 + src/Makefile.am | 10 +- src/lxc/lxc_cgroup.c | 2 +- src/lxc/lxc_controller.c | 3 + src/lxc/lxc_process.c | 28 +-- src/util/cgmanager.c | 337 +++++++++++++++++++++++++++ src/util/cgmanager.h | 51 +++++ src/util/vircgroup.c | 582 ++++++++++++++++++++++++++++++++++++++++++++++- src/util/vircgroup.h | 5 + tests/Makefile.am | 5 +- tests/vircgrouptest.c | 9 + 13 files changed, 1033 insertions(+), 26 deletions(-) create mode 100644 src/util/cgmanager.c create mode 100644 src/util/cgmanager.h diff --git a/configure.ac b/configure.ac index 368d094..66e1005 100644 --- a/configure.ac +++ b/configure.ac @@ -2214,6 +2214,20 @@ if test "${enable_oom}" = yes; then AC_DEFINE([TEST_OOM], 1, [Whether malloc OOM checking is enabled]) fi +AC_ARG_ENABLE([cgmanager], + [AC_HELP_STRING([--enable-cgmanager], [enable cgmanager support [default=auto]])], + [], [enable_cgmanager=auto]) +if test "x$enable_cgmanager" = "xauto" ; then + AC_CHECK_LIB([cgmanager],[cgmanager_create],[enable_cgmanager=yes],[enable_cgmanager=no],[-lnih -lnih-dbus -ldbus-1]) +fi +AM_CONDITIONAL([ENABLE_CGMANAGER], [test "x$enable_cgmanager" = "xyes"]) + +AM_COND_IF([ENABLE_CGMANAGER], + [PKG_CHECK_MODULES([CGMANAGER], [libcgmanager]) + PKG_CHECK_MODULES([NIH], [libnih >= 1.0.2]) + PKG_CHECK_MODULES([NIH_DBUS], [libnih-dbus >= 1.0.0]) + PKG_CHECK_MODULES([DBUS], [dbus-1 >= 1.2.16]) + ]) AC_ARG_ENABLE([test-locking], [AS_HELP_STRING([--enable-test-locking], @@ -2905,6 +2919,7 @@ AC_MSG_NOTICE([ DTrace: $with_dtrace]) AC_MSG_NOTICE([ numad: $with_numad]) AC_MSG_NOTICE([ XML Catalog: $XML_CATALOG_FILE]) AC_MSG_NOTICE([ Init script: $with_init_script]) +AC_MSG_NOTICE([ CGManager: $with_cgmanager]) AC_MSG_NOTICE([Char device locks: $with_chrdev_lock_files]) AC_MSG_NOTICE([]) AC_MSG_NOTICE([Developer Tools]) diff --git a/daemon/Makefile.am b/daemon/Makefile.am index 00221ab..3dcda97 100644 --- a/daemon/Makefile.am +++ b/daemon/Makefile.am @@ -104,14 +104,16 @@ libvirtd_conf_la_CFLAGS = \ $(XDR_CFLAGS) \ $(WARN_CFLAGS) $(PIE_CFLAGS) \ $(COVERAGE_CFLAGS) \ + $(CGMANAGER_CFLAGS) \ $(NULL) libvirtd_conf_la_LDFLAGS = \ $(RELRO_LDFLAGS) \ $(PIE_LDFLAGS) \ $(COVERAGE_LDFLAGS) \ $(NO_INDIRECT_LDFLAGS) \ + $(NO_INDIRECT_LDFLAGS) \ $(NULL) -libvirtd_conf_la_LIBADD = $(LIBXML_LIBS) +libvirtd_conf_la_LIBADD = $(LIBXML_LIBS) $(CGMANAGER_LIBS) man8_MANS = libvirtd.8 @@ -142,7 +144,7 @@ libvirtd_CFLAGS = \ $(LIBXML_CFLAGS) $(GNUTLS_CFLAGS) $(SASL_CFLAGS) \ $(XDR_CFLAGS) $(POLKIT_CFLAGS) $(DBUS_CFLAGS) $(LIBNL_CFLAGS) \ $(WARN_CFLAGS) $(PIE_CFLAGS) \ - $(COVERAGE_CFLAGS) \ + $(COVERAGE_CFLAGS) $(CGMANAGER_CFLAGS) \ -DQEMUD_PID_FILE="\"$(QEMUD_PID_FILE)\"" libvirtd_LDFLAGS = \ @@ -158,6 +160,7 @@ libvirtd_LDADD = \ $(SASL_LIBS) \ $(DBUS_LIBS) \ $(POLKIT_LIBS) \ + $(CGMANAGER_LIBS) \ $(LIBNL_LIBS) if WITH_DTRACE_PROBES diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 4c926b3..d5d3466 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -49,6 +49,7 @@ #include "viralloc.h" #include "virconf.h" #include "virnetlink.h" +#include "vircgroup.h" #include "virnetserver.h" #include "remote.h" #include "virhook.h" @@ -1269,6 +1270,10 @@ int main(int argc, char **argv) { exit(EXIT_FAILURE); } + /* move ourselves to root cgroup if necessary */ + // XXX todo - figure out how to get the fn included + // virCgroupEscape(); + if (daemonSetupLogging(config, privileged, verbose, godaemon) < 0) { VIR_ERROR(_("Can't initialize logging")); exit(EXIT_FAILURE); diff --git a/src/Makefile.am b/src/Makefile.am index 2b9ac61..34f0c6f 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -96,6 +96,7 @@ UTIL_SOURCES = \ util/virbitmap.c util/virbitmap.h \ util/virbuffer.c util/virbuffer.h \ util/vircgroup.c util/vircgroup.h util/vircgrouppriv.h \ + util/cgmanager.c util/cgmanager.h \ util/virclosecallbacks.c util/virclosecallbacks.h \ util/vircommand.c util/vircommand.h util/vircommandpriv.h \ util/virconf.c util/virconf.h \ @@ -979,11 +980,13 @@ libvirt_util_la_SOURCES = \ libvirt_util_la_CFLAGS = $(CAPNG_CFLAGS) $(YAJL_CFLAGS) $(LIBNL_CFLAGS) \ $(AM_CFLAGS) $(AUDIT_CFLAGS) $(DEVMAPPER_CFLAGS) \ $(DBUS_CFLAGS) $(LDEXP_LIBM) $(NUMACTL_CFLAGS) \ - $(SYSTEMD_DAEMON_CFLAGS) -I$(top_srcdir)/src/conf + $(SYSTEMD_DAEMON_CFLAGS) -I$(top_srcdir)/src/conf \ + $(CGMANAGER_CFLAGS) libvirt_util_la_LIBADD = $(CAPNG_LIBS) $(YAJL_LIBS) $(LIBNL_LIBS) \ $(THREAD_LIBS) $(AUDIT_LIBS) $(DEVMAPPER_LIBS) \ $(LIB_CLOCK_GETTIME) $(DBUS_LIBS) $(MSCOM_LIBS) $(LIBXML_LIBS) \ - $(SECDRIVER_LIBS) $(NUMACTL_LIBS) $(SYSTEMD_DAEMON_LIBS) + $(SECDRIVER_LIBS) $(NUMACTL_LIBS) $(SYSTEMD_DAEMON_LIBS) \ + $(CGMANAGER_LIBS) noinst_LTLIBRARIES += libvirt_conf.la @@ -1018,6 +1021,9 @@ libvirt_xenxs_la_CFLAGS = \ libvirt_xenxs_la_SOURCES = $(XENXS_SOURCES) endif WITH_XENXS +if ENABLE_CGMANAGER +AM_CFLAGS += -DHAVE_CGMANAGER +endif noinst_LTLIBRARIES += libvirt_driver.la libvirt_la_BUILT_LIBADD += libvirt_driver.la diff --git a/src/lxc/lxc_cgroup.c b/src/lxc/lxc_cgroup.c index 8dfdc60..f5946a1 100644 --- a/src/lxc/lxc_cgroup.c +++ b/src/lxc/lxc_cgroup.c @@ -479,7 +479,7 @@ virCgroupPtr virLXCCgroupCreate(virDomainDefPtr def) true, def->uuid, NULL, - getpid(), + -1, true, def->resource->partition, -1, diff --git a/src/lxc/lxc_controller.c b/src/lxc/lxc_controller.c index fe2a5dc..158cf17 100644 --- a/src/lxc/lxc_controller.c +++ b/src/lxc/lxc_controller.c @@ -2166,6 +2166,9 @@ virLXCControllerRun(virLXCControllerPtr ctrl) if (virLXCControllerSetupResourceLimits(ctrl) < 0) goto cleanup; + if (virCgroupAddTask(ctrl->cgroup, getpid()) < 0) + goto cleanup; + if (virLXCControllerSetupDevPTS(ctrl) < 0) goto cleanup; diff --git a/src/lxc/lxc_process.c b/src/lxc/lxc_process.c index 0aef13a..a83c0d6 100644 --- a/src/lxc/lxc_process.c +++ b/src/lxc/lxc_process.c @@ -1263,20 +1263,6 @@ int virLXCProcessStart(virConnectPtr conn, goto cleanup; } - if (virCgroupNewDetectMachine(vm->def->name, "lxc", vm->pid, - vm->def->resource ? - vm->def->resource->partition : - NULL, - -1, &priv->cgroup) < 0) - goto error; - - if (!priv->cgroup) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("No valid cgroup for machine %s"), - vm->def->name); - goto error; - } - priv->stopReason = VIR_DOMAIN_EVENT_STOPPED_FAILED; priv->wantReboot = false; vm->def->id = vm->pid; @@ -1297,6 +1283,20 @@ int virLXCProcessStart(virConnectPtr conn, goto error; } + if (virCgroupNewDetectMachine(vm->def->name, "lxc", vm->pid, + vm->def->resource ? + vm->def->resource->partition : + NULL, + -1, &priv->cgroup) < 0) + goto error; + + if (!priv->cgroup) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("No valid cgroup for machine %s"), + vm->def->name); + goto error; + } + if (autoDestroy && virCloseCallbacksSet(driver->closeCallbacks, vm, conn, lxcProcessAutoDestroy) < 0) diff --git a/src/util/cgmanager.c b/src/util/cgmanager.c new file mode 100644 index 0000000..bbbb230 --- /dev/null +++ b/src/util/cgmanager.c @@ -0,0 +1,337 @@ +/* + * lxc: linux Container library + * + * (C) Copyright Canonical Ltd. 2013-2014 + * + * Authors: + * Serge Hallyn <serge.hallyn(a)canonical.com> + * Stéphane Graber <stephane.graber(a)canonical.com> + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifdef HAVE_CGMANAGER + +#include <config.h> + +#include <libgen.h> +#include <stdlib.h> +#include <sys/types.h> +#include <unistd.h> + +#include "virutil.h" +#include "viralloc.h" +#include "virerror.h" +#include "virlog.h" +#include "cgmanager.h" + +static NihDBusProxy *cgroup_manager = NULL; +bool cgm_running = false; + +VIR_LOG_INIT("util.cgmanager"); + +#define CGMANAGER_DBUS_SOCK "unix:path=/sys/fs/cgroup/cgmanager/sock" +bool cgm_dbus_connect(void) +{ + DBusError dbus_error; + DBusConnection *connection; + dbus_error_init(&dbus_error); + + connection = dbus_connection_open_private(CGMANAGER_DBUS_SOCK, &dbus_error); + if (!connection) { + dbus_error_free(&dbus_error); + return false; + } + + dbus_connection_set_exit_on_disconnect(connection, FALSE); + dbus_error_free(&dbus_error); + cgroup_manager = nih_dbus_proxy_new(NULL, connection, + NULL /* p2p */, + "/org/linuxcontainers/cgmanager", NULL, NULL); + dbus_connection_unref(connection); + if (!cgroup_manager) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: Error opening proxy: %s", nerr->message); + nih_free(nerr); + return false; + } + + // force fd passing negotiation + if (cgmanager_ping_sync(NULL, cgroup_manager, 0) != 0) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: Error pinging manager: %s", nerr->message); + nih_free(nerr); + nih_free(cgroup_manager); + cgroup_manager = NULL; + return false; + } + cgm_running = true; + return true; +} + +void cgm_dbus_disconnect(void) +{ + if (cgroup_manager) { + dbus_connection_flush(cgroup_manager->connection); + dbus_connection_close(cgroup_manager->connection); + nih_free(cgroup_manager); + cgroup_manager = NULL; + } +} + +bool cgm_create(const char *controller, const char *cgroup_path, int32_t *existed) +{ + if (cgroup_path[0] == '/') + cgroup_path++; + + if ( cgmanager_create_sync(NULL, cgroup_manager, controller, + cgroup_path, existed) != 0) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: cgm_create for controller=%s, cgroup_path=%s failed: %s", + controller, cgroup_path, nerr->message); + nih_free(nerr); + return false; + } + + return true; +} + +bool cgm_remove(const char *controller, const char *cgroup_path, int recursive) +{ + int existed; + + if (cgroup_path[0] == '/') + cgroup_path++; + + if ( cgmanager_remove_sync(NULL, cgroup_manager, controller, + cgroup_path, recursive, &existed) != 0) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: cgm_remove for controller=%s, cgroup_path=%s, recursive=%d failed: %s", + controller, cgroup_path, recursive, nerr->message); + nih_free(nerr); + return false; + } + + if (existed == -1) { + VIR_ERROR("cgmanager: cgm_remove failed: %s:%s did not exist", controller, cgroup_path); + return false; + } + return true; +} + +char *cgm_get(const char *controller, const char *cgroup_path, const char *key) +{ + char *result = NULL; + + if (cgroup_path[0] == '/') + cgroup_path++; + + if (cgmanager_get_value_sync(NULL, cgroup_manager, controller, cgroup_path, key, &result) != 0) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: cgm_get for controller=%s, cgroup_path=%s failed: %s", + controller, cgroup_path, nerr->message); + nih_free(nerr); + free(result); + return NULL; + } + return result; +} + +bool cgm_set(const char *controller, const char *cgroup_path, const char *key, const char *val) +{ + if (cgroup_path[0] == '/') + cgroup_path++; + + if (cgmanager_set_value_sync(NULL, cgroup_manager, controller, cgroup_path, key, val) != 0) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: cgm_set for controller=%s, cgroup_path=%s failed: %s", + controller, cgroup_path, nerr->message); + nih_free(nerr); + return false; + } + return true; +} + +bool cgm_chmod(const char *controller, const char *cgroup_path, int mode) +{ + nih_local char *path_dirname = NULL; + nih_local char *path_basename = NULL; + + if (cgroup_path[0] == '/') + cgroup_path++; + + path_dirname = NIH_MUST(nih_strdup(NULL, cgroup_path)); + path_basename = NIH_MUST(nih_strdup(NULL, cgroup_path)); + + if (cgmanager_chmod_sync(NULL, cgroup_manager, controller, + dirname(path_dirname), basename(path_basename), mode) != 0) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: cgm_chmod for controller=%s, cgroup_path=%s, mode=%d failed: %s", + controller, cgroup_path, mode, nerr->message); + nih_free(nerr); + return false; + } + + return true; +} + +bool cgm_chown(const char *controller, const char *cgroup_path, int uid, int gid) +{ + if (cgroup_path[0] == '/') + cgroup_path++; + + if (cgmanager_chown_sync(NULL, cgroup_manager, controller, + cgroup_path, uid, gid) != 0) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: cgm_chown for controller=%s, cgroup_path=%s, uid=%d, gid=%d failed: %s", + controller, cgroup_path, uid, gid, nerr->message); + nih_free(nerr); + return false; + } + + return true; +} + +bool cgm_list_children(const char *controller, const char *cgroup_path, char ***children) +{ + if (cgroup_path[0] == '/') + cgroup_path++; + + if ( cgmanager_list_children_sync(NULL, cgroup_manager, controller, + cgroup_path, children) != 0) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: cgm_list_children for controller=%s, cgroup_path=%s failed: %s", + controller, cgroup_path, nerr->message); + nih_free(nerr); + return false; + } + + return true; +} + +/* + * caller must nih_free(pids) when done + */ +bool cgm_get_tasks(const char *controller, const char *cgroup_path, pid_t **pids, size_t *nrpids) +{ + int ret; + if (cgroup_path[0] == '/') + cgroup_path++; + + ret = cgmanager_get_tasks_sync(NULL, cgroup_manager, controller, cgroup_path, pids, nrpids); + if (ret) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: cgm_get_tasks for controller=%s, cgroup_path=%s failed: %s", + controller, cgroup_path, nerr->message); + nih_free(nerr); + return false; + } + + return true; +} + + +/* + * caller of cgm_get_pid_cgroup must nih_free(path) if we + * returned true + */ +bool cgm_get_pid_cgroup(const char *controller, pid_t pid, char **cgpath) +{ + *cgpath = NULL; + if ( cgmanager_get_pid_cgroup_sync(NULL, cgroup_manager, controller, + pid, cgpath) != 0) { + NihError *nerr; + nerr = nih_error_get(); + nih_free(nerr); + return false; + } + + return true; +} + +/* + * caller of cgm_get_pid_cgroup_abs must nih_free(path) if we + * returned true + */ +bool cgm_get_pid_cgroup_abs(const char *controller, pid_t pid, char **cgpath) +{ + *cgpath = NULL; + if ( cgmanager_get_pid_cgroup_abs_sync(NULL, cgroup_manager, controller, + pid, cgpath) != 0) { + NihError *nerr; + nerr = nih_error_get(); + nih_free(nerr); + return false; + } + + return true; +} + +bool cgm_controller_exists(const char *controller) +{ + char *cgroup_name = NULL; + + if ( cgmanager_get_pid_cgroup_sync(NULL, cgroup_manager, controller, + getpid(), &cgroup_name) != 0) { + NihError *nerr; + nerr = nih_error_get(); + nih_free(nerr); + return false; + } + + nih_free(cgroup_name); + return true; +} + +bool cgm_enter(const char *controller, const char *cgroup_path, pid_t pid) +{ + if (cgroup_path[0] == '/') + cgroup_path++; + + if (cgmanager_move_pid_sync(NULL, cgroup_manager, controller, + cgroup_path, pid) != 0) { + NihError *nerr; + nerr = nih_error_get(); + VIR_ERROR("cgmanager: cgm_enter for controller=%s, cgroup_path=%s, pid=%d failed: %s", + controller, cgroup_path, pid, nerr->message); + nih_free(nerr); + return false; + } + return true; +} + +bool cgm_escape(const char *controller) +{ + if (cgmanager_move_pid_sync(NULL, cgroup_manager, controller, "/", + getpid()) != 0) { + NihError *nerr; + nerr = nih_error_get(); + VIR_DEBUG("cgmanager: Failed escaping to root cgroup for controller %s: %s", + controller, nerr->message); + nih_free(nerr); + } + return true; +} +#endif diff --git a/src/util/cgmanager.h b/src/util/cgmanager.h new file mode 100644 index 0000000..d7a83b5 --- /dev/null +++ b/src/util/cgmanager.h @@ -0,0 +1,51 @@ +/* + * lxc: linux Container library + * + * (C) Copyright Canonical Ltd. 2013-2014 + * + * Authors: + * Serge Hallyn <serge.hallyn(a)canonical.com> + * Stéphane Graber <stephane.graber(a)canonical.com> + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifdef HAVE_CGMANAGER + +#include <stdbool.h> +#include <nih-dbus/dbus_connection.h> +#include <cgmanager/cgmanager-client.h> +#include <nih/alloc.h> +#include <nih/error.h> +#include <nih/string.h> + +extern bool cgm_running; + +bool cgm_dbus_connect(void); +void cgm_dbus_disconnect(void); +bool cgm_create(const char *controller, const char *cgroup_path, int32_t *existed); +bool cgm_remove(const char *controller, const char *cgroup_path, int recursive); +char *cgm_get(const char *controller, const char *path, const char *key); +bool cgm_set(const char *controller, const char *cgroup_path, const char *key, const char *val); +bool cgm_chmod(const char *controller, const char *cgroup_path, int mode); +bool cgm_chown(const char *controller, const char *cgroup_path, int uid, int gid); +bool cgm_list_children(const char *controller, const char *cgroup_path, char ***children); +bool cgm_get_tasks(const char *controller, const char *cgroup_path, pid_t **pids, size_t *nrpids); +bool cgm_get_pid_cgroup(const char *controller, pid_t pid, char **cgpath); +bool cgm_get_pid_cgroup_abs(const char *controller, pid_t pid, char **cgpath); +bool cgm_controller_exists(const char *controller); +bool cgm_enter(const char *controller, const char *cgroup_path, pid_t pid); +bool cgm_escape(const char *controller); +#endif diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index c578bd0..b72455c 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -52,6 +52,7 @@ #include "virstring.h" #include "virsystemd.h" #include "virtypedparam.h" +#include "cgmanager.h" #include "nodeinfo.h" @@ -92,6 +93,13 @@ virCgroupAvailable(void) struct mntent entry; char buf[CGROUP_MAX_VAL]; +#ifdef HAVE_CGMANAGER + if (cgm_dbus_connect()) { + cgm_dbus_disconnect(); + return true; + } +#endif + if (!virFileExists("/proc/cgroups")) return false; @@ -229,6 +237,12 @@ virCgroupValidateMachineGroup(virCgroupPtr group, if (virCgroupPartitionEscape(&scopename) < 0) goto cleanup; +#ifdef HAVE_CGMANAGER + if (cgm_running) + goto good; + VIR_ERROR("cgm was NOT running"); +#endif + for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { char *tmp; @@ -266,6 +280,7 @@ virCgroupValidateMachineGroup(virCgroupPtr group, } } +good: valid = true; cleanup: @@ -295,6 +310,54 @@ virCgroupCopyMounts(virCgroupPtr group, return 0; } +#ifdef HAVE_CGMANAGER +static void cg_add_cgroup(virCgroupPtr group, const char *g) +{ + int i = virCgroupControllerTypeFromString(g); + if (i < 0) + return; + if (VIR_STRDUP(group->controllers[i].mountPoint, "/") < 0) { + VIR_WARN("Out of memory copying \"/\". Proceeding without cgroup controll %s.", g); + return; + } + group->controllers[i].linkPoint = NULL; +} + +static bool cg_get_cgroups(virCgroupPtr group) +{ + FILE *fin = NULL; + char *line = NULL; + size_t len = 0; + + if (!cgm_dbus_connect()) { + return false; + } + /* check to see if name=systemd is mounted */ + if (cgm_controller_exists("name=systemd")) + cg_add_cgroup(group, "name=systemd"); + cgm_dbus_disconnect(); + fin = fopen("/proc/cgroups", "r"); + if (fin == NULL) { + virReportSystemError(errno, "%s", + _("Unable to open /proc/cgroups")); + return false; + } + while (getline(&line, &len, fin) > 0) { + char *p; + if (line[0] == '#') + continue; + p = strchr(line, '\t'); + if (p) + *p = '\0'; + cg_add_cgroup(group, line); + } + VIR_FREE(line); + VIR_FORCE_FCLOSE(fin); + return true; +} +#else +#define cg_get_cgroups(x) (false) +#endif /* * Process /proc/mounts figuring out what controllers are @@ -308,6 +371,9 @@ virCgroupDetectMounts(virCgroupPtr group) struct mntent entry; char buf[CGROUP_MAX_VAL]; + if (cg_get_cgroups(group)) + return 0; + mounts = fopen("/proc/mounts", "r"); if (mounts == NULL) { virReportSystemError(errno, "%s", @@ -435,6 +501,55 @@ virCgroupCopyPlacement(virCgroupPtr group, } +#ifdef HAVE_CGMANAGER +static bool +cg_detect_placement(virCgroupPtr group, + pid_t pid, + const char *path) +{ + int i; + bool ret = false; + + VIR_DEBUG("cgm: Detecting placement for pid %lld path %s", + (unsigned long long)pid, path); + if (!cgm_dbus_connect()) + return false; + for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { + const char *typestr = virCgroupControllerTypeToString(i); + char *selfpath; + if (!group->controllers[i].mountPoint) + continue; + if (!cgm_get_pid_cgroup(typestr, pid == -1 ? getpid() : pid, &selfpath)) { + VIR_WARN("Failed to get cgroup path for %s", typestr); + goto out; + } + if (virAsprintf(&group->controllers[i].placement, + "%s%s%s", selfpath, + (STREQ(selfpath, "/") || + STREQ(path, "") ? "" : "/"), + path) < 0) { + VIR_WARN("Failed to save cgroup path"); + nih_free(selfpath); + goto out; + } + nih_free(selfpath); + } + ret = true; + + VIR_DEBUG("cgm: done detecting placement for pid %lld path %s", + (unsigned long long)pid, path); +out: + cgm_dbus_disconnect(); + return ret; +} +#else +static inline bool +cg_detect_placement(virCgroupPtr group, + pid_t pid, + const char *path) +{ return false; } +#endif + /* * virCgroupDetectPlacement: * @group: the group to process @@ -457,7 +572,7 @@ virCgroupCopyPlacement(virCgroupPtr group, * * It then appends @path to each detected path. */ -static int +int virCgroupDetectPlacement(virCgroupPtr group, pid_t pid, const char *path) @@ -468,6 +583,9 @@ virCgroupDetectPlacement(virCgroupPtr group, int ret = -1; char *procfile; + if (cg_detect_placement(group, pid, path)) + return 0; + VIR_DEBUG("Detecting placement for pid %lld path %s", (unsigned long long)pid, path); if (pid == -1) { @@ -597,6 +715,9 @@ virCgroupDetect(virCgroupPtr group, if (!((1 << j) & controllers)) continue; +#ifdef HAVE_CGMANAGER + if (!cgm_running) +#endif if (STREQ_NULLABLE(group->controllers[i].mountPoint, group->controllers[j].mountPoint)) { virReportSystemError(EINVAL, @@ -674,6 +795,14 @@ virCgroupSetValueStr(virCgroupPtr group, char *keypath = NULL; char *tmp = NULL; +#ifdef HAVE_CGMANAGER + if (cgm_dbus_connect()) { + ret = cgm_set(virCgroupControllerTypeToString(controller), group->path, + key, value) ? 0 : -1; + cgm_dbus_disconnect(); + return ret; + } +#endif if (virCgroupPathOfController(group, controller, key, &keypath) < 0) return -1; @@ -710,11 +839,26 @@ virCgroupGetValueStr(virCgroupPtr group, *value = NULL; + VIR_DEBUG("Get value %s", keypath); + +#ifdef HAVE_CGMANAGER + if (cgm_dbus_connect()) { + char *strval = NULL; + strval = cgm_get(virCgroupControllerTypeToString(controller), group->path, key); + cgm_dbus_disconnect(); + if (!strval) { + VIR_ERROR("failed to get %s for %s", key, group->path); + goto cleanup; + } + if (VIR_STRDUP(*value, strval) >=0 ) + ret = 0; + nih_free(strval); + return ret; + } +#endif if (virCgroupPathOfController(group, controller, key, &keypath) < 0) return -1; - VIR_DEBUG("Get value %s", keypath); - if ((rc = virFileReadAll(keypath, 1024*1024, value)) < 0) { virReportSystemError(errno, _("Unable to read from '%s'"), keypath); @@ -745,6 +889,12 @@ virCgroupSetValueU64(virCgroupPtr group, if (virAsprintf(&strval, "%llu", value) < 0) return -1; +#ifdef HAVE_CGMANAGER + if (cgm_dbus_connect()) { + ret = cgm_set(virCgroupControllerTypeToString(controller), group->path, key, strval) ? 0 : -1; + cgm_dbus_disconnect(); + } else +#endif ret = virCgroupSetValueStr(group, controller, key, strval); VIR_FREE(strval); @@ -765,6 +915,12 @@ virCgroupSetValueI64(virCgroupPtr group, if (virAsprintf(&strval, "%lld", value) < 0) return -1; +#ifdef HAVE_CGMANAGER + if (cgm_dbus_connect()) { + ret = cgm_set(virCgroupControllerTypeToString(controller), group->path, key, strval) ? 0 : -1; + cgm_dbus_disconnect(); + } else +#endif ret = virCgroupSetValueStr(group, controller, key, strval); VIR_FREE(strval); @@ -782,6 +938,16 @@ virCgroupGetValueI64(virCgroupPtr group, char *strval = NULL; int ret = -1; +#ifdef HAVE_CGMANAGER + bool usecgm = false; + if (cgm_dbus_connect()) { + strval = cgm_get(virCgroupControllerTypeToString(controller), group->path, key); + cgm_dbus_disconnect(); + usecgm = true; + if (!strval) + goto cleanup; + } else +#endif if (virCgroupGetValueStr(group, controller, key, &strval) < 0) goto cleanup; @@ -795,6 +961,11 @@ virCgroupGetValueI64(virCgroupPtr group, ret = 0; cleanup: +#ifdef HAVE_CGMANAGER + if (usecgm) + nih_free(strval); + else +#endif VIR_FREE(strval); return ret; } @@ -836,6 +1007,12 @@ virCgroupCpuSetInherit(virCgroupPtr parent, virCgroupPtr group) "cpuset.mems", }; +#ifdef HAVE_CGMANAGER + /* cgmanager will have set up inheritence for us */ + if (cgm_running) + return 0; +#endif + VIR_DEBUG("Setting up inheritance %s -> %s", parent->path, group->path); for (i = 0; i < ARRAY_CARDINALITY(inherit_values); i++) { char *value; @@ -868,6 +1045,11 @@ virCgroupSetMemoryUseHierarchy(virCgroupPtr group) unsigned long long value; const char *filename = "memory.use_hierarchy"; +#ifdef HAVE_CGMANAGER + /* cgmanager will have set up inheritence for us */ + if (cgm_running) + return 0; +#endif if (virCgroupGetValueU64(group, VIR_CGROUP_CONTROLLER_MEMORY, filename, &value) < 0) @@ -895,6 +1077,11 @@ virCgroupMakeGroup(virCgroupPtr parent, { size_t i; int ret = -1; +#ifdef HAVE_CGMANAGER + bool usecgm = false; + if (cgm_dbus_connect()) + usecgm = true; +#endif VIR_DEBUG("Make group %s", group->path); for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { @@ -913,6 +1100,15 @@ virCgroupMakeGroup(virCgroupPtr parent, continue; } +#ifdef HAVE_CGMANAGER + if (usecgm) { + int32_t existed; + if (!cgm_create(virCgroupControllerTypeToString(i), + group->path, &existed)) + goto cleanup; + continue; + } +#endif if (virCgroupPathOfController(group, i, "", &path) < 0) return -1; @@ -977,6 +1173,10 @@ virCgroupMakeGroup(virCgroupPtr parent, ret = 0; cleanup: +#ifdef HAVE_CGMANAGER + if (usecgm) + cgm_dbus_disconnect(); +#endif return ret; } @@ -1058,6 +1258,19 @@ virCgroupAddTask(virCgroupPtr group, pid_t pid) if (i == VIR_CGROUP_CONTROLLER_SYSTEMD) continue; +#ifdef HAVE_CGMANAGER + if (cgm_dbus_connect()) { + if (!cgm_enter(virCgroupControllerTypeToString(i), group->path, pid)) { + cgm_dbus_disconnect(); + VIR_ERROR("Failed to move %d to %s:%s", pid, + virCgroupControllerTypeToString(i), group->path); + goto cleanup; + } + cgm_dbus_disconnect(); + VIR_INFO("Moved %d to %s:%s", pid, + virCgroupControllerTypeToString(i), group->path); + } else +#endif if (virCgroupSetValueU64(group, i, "tasks", pid) < 0) goto cleanup; } @@ -1093,6 +1306,20 @@ virCgroupAddTaskController(virCgroupPtr group, pid_t pid, int controller) return -1; } +#ifdef HAVE_CGMANAGER + if (cgm_dbus_connect()) { + if (!cgm_enter(virCgroupControllerTypeToString(controller), group->path, pid)) { + cgm_dbus_disconnect(); + VIR_ERROR("Failed to move %d to %s:%s", pid, + virCgroupControllerTypeToString(controller), group->path); + return -1; + } + cgm_dbus_disconnect(); + VIR_INFO("Moved %d to %s:%s", pid, + virCgroupControllerTypeToString(controller), group->path); + return 0; + } else +#endif return virCgroupSetValueU64(group, controller, "tasks", (unsigned long long)pid); } @@ -1157,8 +1384,15 @@ virCgroupMoveTask(virCgroupPtr src_group, virCgroupPtr dest_group) int ret = -1; char *content = NULL; size_t i; +#ifdef HAVE_CGMANAGER + bool usecgm = false; + + if (cgm_dbus_connect()) + usecgm = true; +#endif for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { + bool firstrun = true; if (!src_group->controllers[i].mountPoint || !dest_group->controllers[i].mountPoint) continue; @@ -1172,6 +1406,35 @@ virCgroupMoveTask(virCgroupPtr src_group, virCgroupPtr dest_group) * aware that it needs to move. Therefore, we must iterate * until content is empty. */ while (1) { +#ifdef HAVE_CGMANAGER + const char *typestr = virCgroupControllerTypeToString(i); + pid_t *pids = NULL; + size_t j, nrpids; + if (usecgm) { + if (!cgm_get_tasks(typestr, src_group->path, &pids, &nrpids)) { + if (firstrun) + goto cleanup; + else + break; + } + firstrun = false; + if (nrpids) { + for (j=0; j<nrpids; j++) { + if (!cgm_enter(typestr, dest_group->path, pids[j])) { + VIR_ERROR("Failed to move pid %d from %s to %s", + pids[j], src_group->path, dest_group->path); + } + } + nih_free(pids); + VIR_INFO("Moved %d pids to %s:%s", (int) nrpids, typestr, + dest_group->path); + } else { + VIR_WARN("No pids found in %s:%s", typestr, src_group->path); + break; + } + continue; + } +#endif VIR_FREE(content); if (virCgroupGetValueStr(src_group, i, "tasks", &content) < 0) return -1; @@ -1187,6 +1450,10 @@ virCgroupMoveTask(virCgroupPtr src_group, virCgroupPtr dest_group) ret = 0; cleanup: VIR_FREE(content); +#ifdef HAVE_CGMANAGER + if (usecgm) + cgm_dbus_disconnect(); +#endif return ret; } @@ -1581,7 +1848,7 @@ virCgroupNewMachineSystemd(const char *name, } } - if (virCgroupAddTask(*group, pidleader) < 0) { + if (pidleader != -1 && virCgroupAddTask(*group, pidleader) < 0) { virErrorPtr saved = virSaveLastError(); virCgroupRemove(*group); virCgroupFree(group); @@ -1628,7 +1895,7 @@ virCgroupNewMachineManual(const char *name, group) < 0) goto cleanup; - if (virCgroupAddTask(*group, pidleader) < 0) { + if (pidleader != -1 && virCgroupAddTask(*group, pidleader) < 0) { virErrorPtr saved = virSaveLastError(); virCgroupRemove(*group); virCgroupFree(group); @@ -3200,6 +3467,12 @@ virCgroupRemove(virCgroupPtr group) int rc = 0; size_t i; char *grppath = NULL; +#ifdef HAVE_CGMANAGER + bool usecgm = false; + + if (cgm_dbus_connect()) + usecgm = true; +#endif VIR_DEBUG("Removing cgroup %s", group->path); for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { @@ -3216,6 +3489,14 @@ virCgroupRemove(virCgroupPtr group) if (STREQ(group->controllers[i].placement, "/")) continue; +#ifdef HAVE_CGMANAGER + if (usecgm) { + if (!cgm_remove(virCgroupControllerTypeToString(i), + group->path, 1)) + VIR_WARN("Error removing %s:%s", virCgroupControllerTypeToString(i), group->path); + continue; + } +#endif if (virCgroupPathOfController(group, i, NULL, @@ -3228,6 +3509,10 @@ virCgroupRemove(virCgroupPtr group) } VIR_DEBUG("Done removing cgroup %s", group->path); +#ifdef HAVE_CGMANAGER + if (usecgm) + cgm_dbus_disconnect(); +#endif return rc; } @@ -3332,6 +3617,181 @@ virCgroupPidCopy(const void *name) return (void*)name; } +#ifdef HAVE_CGMANAGER +/* + * return -1 on error, else the controller index + */ +static int cgm_get_controller_path(virCgroupPtr group, + int controller) +{ + if (controller == -1) { + size_t i; + nih_local char *path = NULL; + + for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { + /* Reject any controller in which our cgroup does + * not begin with the container name */ + VIR_DEBUG("(pid %d) Checking %d ctrl %s mntpt %s placement %s", (int) getpid(), + (int)i, virCgroupControllerTypeToString(i), + group->controllers[i].mountPoint ? group->controllers[i].mountPoint : "null", + group->controllers[i].placement ? group->controllers[i].placement : "null"); + if (group->controllers[i].mountPoint && + group->controllers[i].placement && + STRNEQ(group->controllers[i].placement, "/")) { + controller = i; + VIR_DEBUG("choosing controller %s", virCgroupControllerTypeToString(i)); + break; + } + + VIR_DEBUG("Skipping controller %s where I'm in cgroup %s", + virCgroupControllerTypeToString(i), + group->controllers[i].placement ? group->controllers[i].placement : "null"); + } + } + if (controller == -1) { + virReportSystemError(ENOSYS, "%s", + _("cgm_get_controller_path: No controllers are mounted")); + return -1; + } + + if (group->controllers[controller].mountPoint == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("cgm_get_controller_path: Controller '%s' is not mounted"), + virCgroupControllerTypeToString(controller)); + return -1; + } + + if (group->controllers[controller].placement == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Controller '%s' is not enabled for group"), + virCgroupControllerTypeToString(controller)); + return -1; + } + + return controller; +} + +static int cgm_kill(virCgroupPtr group, int signum, virHashTablePtr pidhash) +{ + bool killedAny = false; + int i; + bool done = false; + const char *controller, *cgpath; + int ret = -1; + + if ((i = cgm_get_controller_path(group, -1)) < 0) { + VIR_ERROR("cgm_kill: Could not get controller path"); + return -1; + } + + controller = virCgroupControllerTypeToString(i); + cgpath = group->controllers[i].placement; + + /* PIDs may be forking as we kill them, so loop + * until there are no new PIDs found + */ + while (!done) { + size_t j, nrpids; + int32_t *pids = NULL; + unsigned long upid; + done = true; + + if (!cgm_get_tasks(controller, cgpath, &pids, &nrpids)) + goto done; + for (j = 0; j < nrpids; j++) { + upid = pids[j]; + if (virHashLookup(pidhash, (void*)upid)) + continue; + if (kill(pids[j], signum) < 0) { + if (errno != ESRCH) { + virReportSystemError(errno, + _("Failed to kill process %d"), + pids[j]); + nih_free(pids); + goto cleanup; + } + } else { + killedAny = true; + done = false; + } + ignore_value(virHashAddEntry(pidhash, (void*)upid, (void*)1)); + } + nih_free(pids); + } + + done: + ret = killedAny ? 1 : 0; + +cleanup: + + VIR_DEBUG("cgm_kill: returning %d, killedAny %d", ret, killedAny); + return ret; +} + +static int cgm_kill_recursive(virCgroupPtr group, int signum, virHashTablePtr pidhash) +{ + int i, j, ret; + bool killedAny = false; + const char *controller, *cgpath; + char **children; + virCgroupPtr subgroup = NULL; + + ret = cgm_kill(group, signum, pidhash); + if (ret < 0) + return ret; + if (ret) + killedAny = true; + + if ((i = cgm_get_controller_path(group, -1)) < 0) + return -1; + controller = virCgroupControllerTypeToString(i); + cgpath = group->controllers[i].placement; + if (!cgm_list_children(controller, cgpath, &children)) + return -1; + for (j = 0; children[j]; j++) { + if (virCgroupNew(-1, children[j], group, -1, &subgroup) < 0) + goto bad; + ret = cgm_kill_recursive(subgroup, signum, pidhash); + if (ret < 0) + goto bad; + if (ret > 0) + killedAny = true; + virCgroupFree(&subgroup); + } + + nih_free(children); + return killedAny ? 1 : 0; + +bad: + virCgroupFree(&subgroup); + nih_free(children); + return -1; +} + +static int cgm_remove_children(virCgroupPtr group) +{ + int i; + int ret = 0; + const char *controller, *cgpath; + + for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { + if (!group->controllers[i].mountPoint) + continue; + if (!group->controllers[i].placement) + continue; + if (!STRNEQ(group->controllers[i].placement, "/")) + continue; + controller = virCgroupControllerTypeToString(i), + cgpath = group->controllers[i].placement; + if (!cgm_remove(controller, cgpath, 1)) { + VIR_ERROR("Failed to remove %s:%s", controller, cgpath); + ret = -1; + } + } + + return ret; +} +#endif /* * Returns 1 if some PIDs are killed, 0 if none are killed, or -1 on error @@ -3341,6 +3801,7 @@ virCgroupKill(virCgroupPtr group, int signum) { VIR_DEBUG("group=%p path=%s signum=%d", group, group->path, signum); int ret; + /* The 'tasks' file in cgroups can contain duplicated * pids, so we use a hash to track which we've already * killed. @@ -3352,6 +3813,12 @@ virCgroupKill(virCgroupPtr group, int signum) virCgroupPidCopy, NULL); +#ifdef HAVE_CGMANAGER + if (cgm_dbus_connect()) { + ret = cgm_kill(group, signum, pids); + cgm_dbus_disconnect(); + } else +#endif ret = virCgroupKillInternal(group, signum, pids); virHashFree(pids); @@ -3442,6 +3909,7 @@ virCgroupKillRecursive(virCgroupPtr group, int signum) { int ret; VIR_DEBUG("group=%p path=%s signum=%d", group, group->path, signum); + virHashTablePtr pids = virHashCreateFull(100, NULL, virCgroupPidCode, @@ -3449,6 +3917,14 @@ virCgroupKillRecursive(virCgroupPtr group, int signum) virCgroupPidCopy, NULL); +#ifdef HAVE_CGMANAGER + if (cgm_dbus_connect()) { + ret = cgm_kill_recursive(group, signum, pids); + if (ret == 0) + ret = cgm_remove_children(group); + cgm_dbus_disconnect(); + } else +#endif ret = virCgroupKillRecursiveInternal(group, signum, pids, false); virHashFree(pids); @@ -3607,6 +4083,50 @@ virCgroupGetFreezerState(virCgroupPtr group, char **state) "freezer.state", state); } +#ifdef HAVE_CGMANAGER +static bool cgm_bind_cgsocket(const char *oldroot) +{ + int ret; + struct stat st; + char path[1024]; + + ret = snprintf(path, 1024, "%s/sys/fs/cgroup/cgmanager/sock", oldroot); + if (ret < 0 || ret >= 1024) + return false; + ret = stat(path, &st); + if (ret != 0) { + VIR_ERROR("%s/sys/fs/cgroup/cgmanager not found", oldroot); + return false; + } + + ret = stat("/sys/fs/cgroup/cgmanager", &st); + if (ret == 0) + goto do_bind; + ret = stat("/sys/fs/cgroup", &st); + if (ret != 0) + return false; + if (mount("cgroup", "/sys/fs/cgroup", "tmpfs", 0, "size=10000,mode=755") < 0) { + VIR_ERROR("Failed mounting tmpfs onto sys/fs/cgroup"); + return false; + } + if (virFileMakePath("/sys/fs/cgroup/cgmanager") < 0) { + if (mkdir("/sys/fs/cgroup/cgmanager", 0755) < 0) { + VIR_ERROR("mkdir by hand failed: %m"); + return false; + } + } + +do_bind: + sprintf(path, "%s/sys/fs/cgroup/cgmanager", oldroot); + if (mount(path, "sys/fs/cgroup/cgmanager", "none", + MS_BIND, 0) < 0) { + VIR_ERROR("Failed bind-mounting the cgmanager socket"); + return false; + } + VIR_INFO("bind-mounted the cgmanager socket"); + return true; +} +#endif int virCgroupIsolateMount(virCgroupPtr group, const char *oldroot, @@ -3617,6 +4137,11 @@ virCgroupIsolateMount(virCgroupPtr group, const char *oldroot, char *opts = NULL; char *root = NULL; +#ifdef HAVE_CGMANAGER + if (cgm_bind_cgsocket(oldroot)) + return 0; +#endif + if (!(root = virCgroupIdentifyRoot(group))) return -1; @@ -3706,6 +4231,11 @@ int virCgroupSetOwner(virCgroupPtr cgroup, size_t i; char *base = NULL, *entry = NULL; DIR *dh = NULL; +#ifdef HAVE_CGMANAGER + bool usecgm = false; + if (cgm_dbus_connect()) + usecgm = true; +#endif int direrr; for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) { @@ -3717,6 +4247,19 @@ int virCgroupSetOwner(virCgroupPtr cgroup, if (!cgroup->controllers[i].mountPoint) continue; +#ifdef HAVE_CGMANAGER + if (usecgm) { + if (!cgm_chown(virCgroupControllerTypeToString(i), + cgroup->path, uid, gid)) { + VIR_ERROR("Failed to chown cgroup %s:%s to %d:%d", + virCgroupControllerTypeToString(i), cgroup->path, + uid, gid); + goto cleanup; + } + continue; + } +#endif + if (virAsprintf(&base, "%s%s", cgroup->controllers[i].mountPoint, cgroup->controllers[i].placement) < 0) goto cleanup; @@ -3766,6 +4309,10 @@ int virCgroupSetOwner(virCgroupPtr cgroup, closedir(dh); VIR_FREE(entry); VIR_FREE(base); +#ifdef HAVE_CGMANAGER + if (usecgm) + cgm_dbus_disconnect(); +#endif return ret; } @@ -3786,6 +4333,19 @@ virCgroupSupportsCpuBW(virCgroupPtr cgroup) if (!cgroup) return false; +#ifdef HAVE_CGMANAGER + if (cgm_dbus_connect()) { + char *str; + str = cgm_get(virCgroupControllerTypeToString(VIR_CGROUP_CONTROLLER_CPU), + cgroup->path, "cpu.cfs_period_us"); + cgm_dbus_disconnect(); + if (str) { + ret = true; + nih_free(str); + } + return ret; + } +#endif if (virCgroupPathOfController(cgroup, VIR_CGROUP_CONTROLLER_CPU, "cpu.cfs_period_us", &path) < 0) { virResetLastError(); @@ -3800,6 +4360,16 @@ virCgroupSupportsCpuBW(virCgroupPtr cgroup) } +void virCgroupEscape(void) { +#ifdef HAVE_CGMANAGER + int i; + if (cgm_dbus_connect()) { + for (i = 0; i < VIR_CGROUP_CONTROLLER_LAST; i++) + cgm_escape(virCgroupControllerTypeToString(i)); + cgm_dbus_disconnect(); + } +#endif +} #else /* !VIR_CGROUP_SUPPORTED */ bool @@ -4497,4 +5067,6 @@ virCgroupSetOwner(virCgroupPtr cgroup ATTRIBUTE_UNUSED, return -1; } +void virCgroupEscape(void) { +} #endif /* !VIR_CGROUP_SUPPORTED */ diff --git a/src/util/vircgroup.h b/src/util/vircgroup.h index 7bb46bf..da763b4 100644 --- a/src/util/vircgroup.h +++ b/src/util/vircgroup.h @@ -255,4 +255,9 @@ int virCgroupSetOwner(virCgroupPtr cgroup, gid_t gid, int controllers); +void virCgroupEscape(void); +int virCgroupDetectPlacement(virCgroupPtr group, + pid_t pid, + const char *path); + #endif /* __VIR_CGROUP_H__ */ diff --git a/tests/Makefile.am b/tests/Makefile.am index c999061..d7e2d90 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -933,8 +933,9 @@ libvirportallocatormock_la_LDFLAGS = -module -avoid-version \ -rpath /evil/libtool/hack/to/force/shared/lib/creation vircgrouptest_SOURCES = \ - vircgrouptest.c testutils.h testutils.c -vircgrouptest_LDADD = $(LDADDS) + vircgrouptest.c testutils.h testutils.c ../src/util/cgmanager.c +vircgrouptest_LDADD = $(LDADDS) $(CGMANAGER_LIBS) +vircgrouptest_CFLAGS = $(AM_CFLAGS) $(CGMANAGER_CFLAGS) -DHAVE_CGMANAGER vircgroupmock_la_SOURCES = \ vircgroupmock.c diff --git a/tests/vircgrouptest.c b/tests/vircgrouptest.c index 35ac0c0..3331d52 100644 --- a/tests/vircgrouptest.c +++ b/tests/vircgrouptest.c @@ -34,6 +34,7 @@ # include "virfile.h" # include "testutilslxc.h" # include "nodeinfo.h" +# include "cgmanager.h" # define VIR_FROM_THIS VIR_FROM_NONE @@ -758,6 +759,14 @@ mymain(void) int ret = 0; char *fakesysfsdir; +#ifdef HAVE_CGMANAGER + /* TODO make tests work with cgmanager */ + if (cgm_dbus_connect()) { + cgm_dbus_disconnect(); + return 0; + } +#endif + if (VIR_STRDUP_QUIET(fakesysfsdir, FAKESYSFSDIRTEMPLATE) < 0) { fprintf(stderr, "Out of memory\n"); abort(); -- 1.9.1

1 0

[libvirt] [PATCH v2] blockjob: use stable disk string in job event
by Eric Blake 17 Jun '14

17 Jun '14

When the block job event was first added, it was for block pull, where the active layer of the disk remains the same name. It was also in a day where we only cared about local files, and so we always had a canonical absolute file name. But two things have changed since then: we now have network disks, where determining a single absolute string does not really make sense; and we have two-phase jobs (copy and active commit) where the name of the active layer changes between the first event (ready, on the old name) and second (complete, on the pivoted name). Adam Litke reported that having an unstable string between events makes life harder for clients. Furthermore, all of our API that operate on a particular disk of a domain accept multiple strings: not only the absolute name of the active layer, but also the destination device name (such as 'vda'). As this latter name is stable, even for network sources, it serves as a better string to supply in block job events. But backwards-compatibility demands that we should not change the name handed to users unless they explicitly request it. Therefore, this patch adds a new event, BLOCK_JOB_2 (alas, I couldn't think of any nicer name), and has to double up on emitting both old-style and new-style events according to what clients have registered for (see also how IOError and IOErrorReason emits double events, but there the difference was a larger struct rather than changed meaning of one of the struct members). Unfortunately, adding a new event isn't something that can easily be broken into pieces, so the commit is rather large. * include/libvirt/libvirt.h.in (virDomainEventID): Add a new id for VIR_DOMAIN_EVENT_ID_BLOCK_JOB_2. (virConnectDomainEventBlockJobCallback): Document new semantics. * src/conf/domain_event.c (_virDomainEventBlockJob): Rename field, to ensure we catch all clients. (virDomainEventBlockJobNew): Add parameter. (virDomainEventBlockJobDispose, virDomainEventBlockJobNew) (virDomainEventBlockJobNewFromObj) (virDomainEventBlockJobNewFromDom) (virDomainEventDispatchDefaultFunc): Adjust clients. (virDomainEventBlockJob2NewFromObj) (virDomainEventBlockJob2NewFromDom): New functions. * src/conf/domain_event.h: Add new prototypes. * src/libvirt_private.syms (domain_event.h): Export new functions. * src/qemu/qemu_driver.c (qemuDomainBlockJobImpl): Generate two different events. * src/qemu/qemu_process.c (qemuProcessHandleBlockJob): Likewise. * src/remote/remote_protocol.x (remote_domain_event_block_job_msg): Adjust client. (remote_domain_event_block_job_2_msg): New struct. (REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB_2): New RPC. * src/remote/remote_driver.c (remoteDomainBuildEventBlockJobHelper): Adjust client. (remoteDomainBuildEventBlockJob2): New handler. (remoteEvents): Register new event. * daemon/remote.c (remoteRelayDomainEventBlockJob): Adjust client. (remoteRelayDomainEventBlockJob2): New handler. (domainEventCallbacks): Register new event. * tools/virsh-domain.c (vshEventCallbacks): Likewise. (vshEventBlockJobPrint): Adjust client. * src/remote_protocol-structs: Regenerate. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- v1: https://www.redhat.com/archives/libvir-list/2014-June/msg00675.html Diff from v1: add a new event type, and keep clients of the old event unchanged in what they get in their callback. daemon/remote.c | 47 +++++++++++++++++++++++++++++++++++---- include/libvirt/libvirt.h.in | 18 ++++++++++++--- src/conf/domain_event.c | 52 +++++++++++++++++++++++++++++++++----------- src/conf/domain_event.h | 15 +++++++++++-- src/libvirt_private.syms | 2 ++ src/qemu/qemu_driver.c | 8 ++++++- src/qemu/qemu_process.c | 7 ++++++ src/remote/remote_driver.c | 33 +++++++++++++++++++++++++++- src/remote/remote_protocol.x | 18 +++++++++++++-- src/remote_protocol-structs | 10 ++++++++- tools/virsh-domain.c | 7 ++++-- 11 files changed, 188 insertions(+), 29 deletions(-) diff --git a/daemon/remote.c b/daemon/remote.c index 34c96c9..56cf84f 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -558,7 +558,7 @@ remoteRelayDomainEventGraphics(virConnectPtr conn, static int remoteRelayDomainEventBlockJob(virConnectPtr conn, virDomainPtr dom, - const char *path, + const char *disk, int type, int status, void *opaque) @@ -571,11 +571,11 @@ remoteRelayDomainEventBlockJob(virConnectPtr conn, return -1; VIR_DEBUG("Relaying domain block job event %s %d %s %i, %i, callback %d", - dom->name, dom->id, path, type, status, callback->callbackID); + dom->name, dom->id, disk, type, status, callback->callbackID); /* build return data */ memset(&data, 0, sizeof(data)); - if (VIR_STRDUP(data.path, path) < 0) + if (VIR_STRDUP(data.disk, disk) < 0) goto error; data.type = type; data.status = status; @@ -596,7 +596,7 @@ remoteRelayDomainEventBlockJob(virConnectPtr conn, return 0; error: - VIR_FREE(data.path); + VIR_FREE(data.disk); return -1; } @@ -931,6 +931,44 @@ remoteRelayDomainEventDeviceRemoved(virConnectPtr conn, } +static int +remoteRelayDomainEventBlockJob2(virConnectPtr conn, + virDomainPtr dom, + const char *disk, + int type, + int status, + void *opaque) +{ + daemonClientEventCallbackPtr callback = opaque; + remote_domain_event_block_job_2_msg data; + + if (callback->callbackID < 0 || + !remoteRelayDomainEventCheckACL(callback->client, conn, dom)) + return -1; + + VIR_DEBUG("Relaying domain block job 2 event %s %d %s %i, %i, callback %d", + dom->name, dom->id, disk, type, status, callback->callbackID); + + /* build return data */ + memset(&data, 0, sizeof(data)); + data.callbackID = callback->callbackID; + if (VIR_STRDUP(data.disk, disk) < 0) + goto error; + data.type = type; + data.status = status; + make_nonnull_domain(&data.dom, dom); + + remoteDispatchObjectEventSend(callback->client, remoteProgram, + REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB_2, + (xdrproc_t)xdr_remote_domain_event_block_job_2_msg, &data); + + return 0; + error: + VIR_FREE(data.disk); + return -1; +} + + static virConnectDomainEventGenericCallback domainEventCallbacks[] = { VIR_DOMAIN_EVENT_CALLBACK(remoteRelayDomainEventLifecycle), VIR_DOMAIN_EVENT_CALLBACK(remoteRelayDomainEventReboot), @@ -948,6 +986,7 @@ static virConnectDomainEventGenericCallback domainEventCallbacks[] = { VIR_DOMAIN_EVENT_CALLBACK(remoteRelayDomainEventBalloonChange), VIR_DOMAIN_EVENT_CALLBACK(remoteRelayDomainEventPMSuspendDisk), VIR_DOMAIN_EVENT_CALLBACK(remoteRelayDomainEventDeviceRemoved), + VIR_DOMAIN_EVENT_CALLBACK(remoteRelayDomainEventBlockJob2), }; verify(ARRAY_CARDINALITY(domainEventCallbacks) == VIR_DOMAIN_EVENT_ID_LAST); diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index dc88c40..c36fec3 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -4852,13 +4852,24 @@ typedef enum { * virConnectDomainEventBlockJobCallback: * @conn: connection object * @dom: domain on which the event occurred - * @disk: fully-qualified filename of the affected disk + * @disk: name associated with the affected disk * @type: type of block job (virDomainBlockJobType) * @status: status of the operation (virConnectDomainEventBlockJobStatus) * @opaque: application specified data * - * The callback signature to use when registering for an event of type - * VIR_DOMAIN_EVENT_ID_BLOCK_JOB with virConnectDomainEventRegisterAny() + * The string returned for @disk can be used in any of the libvirt API + * that operate on a particular disk of the domain, and depends on what + * event type was registered with virConnectDomainEventRegisterAny(). + * If the callback was registered using the older type of + * VIR_DOMAIN_EVENT_ID_BLOCK_JOB, then @disk contains the absolute file + * name of the host resource for the active layer of the disk; however, + * this name is unstable (pivoting via block copy or active block commit + * will change which file is active, giving a different name for the two + * events associated with the same job) and cannot be relied on if the + * active layer is associated with a network resource. If the callback + * was registered using the newer type of VIR_DOMAIN_EVENT_ID_BLOCK_JOB_2, + * then @disk will contain the device target shorthand (the <target + * dev='...'/> sub-element, such as "vda"). */ typedef void (*virConnectDomainEventBlockJobCallback)(virConnectPtr conn, virDomainPtr dom, @@ -5062,6 +5073,7 @@ typedef enum { VIR_DOMAIN_EVENT_ID_BALLOON_CHANGE = 13, /* virConnectDomainEventBalloonChangeCallback */ VIR_DOMAIN_EVENT_ID_PMSUSPEND_DISK = 14, /* virConnectDomainEventPMSuspendDiskCallback */ VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED = 15, /* virConnectDomainEventDeviceRemovedCallback */ + VIR_DOMAIN_EVENT_ID_BLOCK_JOB_2 = 16, /* virConnectDomainEventBlockJobCallback */ #ifdef VIR_ENUM_SENTINELS VIR_DOMAIN_EVENT_ID_LAST diff --git a/src/conf/domain_event.c b/src/conf/domain_event.c index b565732..43794d3 100644 --- a/src/conf/domain_event.c +++ b/src/conf/domain_event.c @@ -128,7 +128,7 @@ typedef virDomainEventIOError *virDomainEventIOErrorPtr; struct _virDomainEventBlockJob { virDomainEvent parent; - char *path; + char *disk; int type; int status; }; @@ -364,7 +364,7 @@ virDomainEventBlockJobDispose(void *obj) virDomainEventBlockJobPtr event = obj; VIR_DEBUG("obj=%p", event); - VIR_FREE(event->path); + VIR_FREE(event->disk); } static void @@ -775,10 +775,11 @@ virDomainEventGraphicsNewFromObj(virDomainObjPtr obj, } static virObjectEventPtr -virDomainEventBlockJobNew(int id, +virDomainEventBlockJobNew(int event, + int id, const char *name, unsigned char *uuid, - const char *path, + const char *disk, int type, int status) { @@ -788,11 +789,11 @@ virDomainEventBlockJobNew(int id, return NULL; if (!(ev = virDomainEventNew(virDomainEventBlockJobClass, - VIR_DOMAIN_EVENT_ID_BLOCK_JOB, + event, id, name, uuid))) return NULL; - if (VIR_STRDUP(ev->path, path) < 0) { + if (VIR_STRDUP(ev->disk, disk) < 0) { virObjectUnref(ev); return NULL; } @@ -804,22 +805,46 @@ virDomainEventBlockJobNew(int id, virObjectEventPtr virDomainEventBlockJobNewFromObj(virDomainObjPtr obj, - const char *path, + const char *disk, int type, int status) { - return virDomainEventBlockJobNew(obj->def->id, obj->def->name, - obj->def->uuid, path, type, status); + return virDomainEventBlockJobNew(VIR_DOMAIN_EVENT_ID_BLOCK_JOB, + obj->def->id, obj->def->name, + obj->def->uuid, disk, type, status); } virObjectEventPtr virDomainEventBlockJobNewFromDom(virDomainPtr dom, - const char *path, + const char *disk, int type, int status) { - return virDomainEventBlockJobNew(dom->id, dom->name, dom->uuid, - path, type, status); + return virDomainEventBlockJobNew(VIR_DOMAIN_EVENT_ID_BLOCK_JOB, + dom->id, dom->name, dom->uuid, + disk, type, status); +} + +virObjectEventPtr +virDomainEventBlockJob2NewFromObj(virDomainObjPtr obj, + const char *disk, + int type, + int status) +{ + return virDomainEventBlockJobNew(VIR_DOMAIN_EVENT_ID_BLOCK_JOB_2, + obj->def->id, obj->def->name, + obj->def->uuid, disk, type, status); +} + +virObjectEventPtr +virDomainEventBlockJob2NewFromDom(virDomainPtr dom, + const char *disk, + int type, + int status) +{ + return virDomainEventBlockJobNew(VIR_DOMAIN_EVENT_ID_BLOCK_JOB_2, + dom->id, dom->name, dom->uuid, + disk, type, status); } virObjectEventPtr @@ -1250,12 +1275,13 @@ virDomainEventDispatchDefaultFunc(virConnectPtr conn, goto cleanup; case VIR_DOMAIN_EVENT_ID_BLOCK_JOB: + case VIR_DOMAIN_EVENT_ID_BLOCK_JOB_2: { virDomainEventBlockJobPtr blockJobEvent; blockJobEvent = (virDomainEventBlockJobPtr)event; ((virConnectDomainEventBlockJobCallback)cb)(conn, dom, - blockJobEvent->path, + blockJobEvent->disk, blockJobEvent->type, blockJobEvent->status, cbopaque); diff --git a/src/conf/domain_event.h b/src/conf/domain_event.h index 9c41090..c7b8761 100644 --- a/src/conf/domain_event.h +++ b/src/conf/domain_event.h @@ -117,16 +117,27 @@ virDomainEventControlErrorNewFromObj(virDomainObjPtr obj); virObjectEventPtr virDomainEventBlockJobNewFromObj(virDomainObjPtr obj, - const char *path, + const char *disk, int type, int status); virObjectEventPtr virDomainEventBlockJobNewFromDom(virDomainPtr dom, - const char *path, + const char *disk, int type, int status); virObjectEventPtr +virDomainEventBlockJob2NewFromObj(virDomainObjPtr obj, + const char *disk, + int type, + int status); +virObjectEventPtr +virDomainEventBlockJob2NewFromDom(virDomainPtr dom, + const char *disk, + int type, + int status); + +virObjectEventPtr virDomainEventDiskChangeNewFromObj(virDomainObjPtr obj, const char *oldSrcPath, const char *newSrcPath, diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 29a9ed1..9e25b8a 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -439,6 +439,8 @@ virDomainXMLOptionNew; # conf/domain_event.h virDomainEventBalloonChangeNewFromDom; virDomainEventBalloonChangeNewFromObj; +virDomainEventBlockJob2NewFromDom; +virDomainEventBlockJob2NewFromObj; virDomainEventBlockJobNewFromDom; virDomainEventBlockJobNewFromObj; virDomainEventControlErrorNewFromDom; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 4ab5a7b..ca58d6b 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -15027,6 +15027,7 @@ qemuDomainBlockJobImpl(virDomainObjPtr vm, int ret = -1; bool async = false; virObjectEventPtr event = NULL; + virObjectEventPtr event2 = NULL; int idx; virDomainDiskDefPtr disk; virStorageSourcePtr baseSource = NULL; @@ -15130,11 +15131,14 @@ qemuDomainBlockJobImpl(virDomainObjPtr vm, if (mode == BLOCK_JOB_ABORT) { if (!async) { /* Older qemu that lacked async reporting also lacked - * active commit, so we can hardcode the event to pull */ + * active commit, so we can hardcode the event to pull. + * We have to generate two variants of the event. */ int type = VIR_DOMAIN_BLOCK_JOB_TYPE_PULL; int status = VIR_DOMAIN_BLOCK_JOB_CANCELED; event = virDomainEventBlockJobNewFromObj(vm, disk->src->path, type, status); + event2 = virDomainEventBlockJob2NewFromObj(vm, disk->dst, type, + status); } else if (!(flags & VIR_DOMAIN_BLOCK_JOB_ABORT_ASYNC)) { while (1) { /* Poll every 50ms */ @@ -15178,6 +15182,8 @@ qemuDomainBlockJobImpl(virDomainObjPtr vm, virObjectUnlock(vm); if (event) qemuDomainEventQueue(driver, event); + if (event2) + qemuDomainEventQueue(driver, event2); return ret; } diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index e4845ba..f1c0041 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -1013,6 +1013,7 @@ qemuProcessHandleBlockJob(qemuMonitorPtr mon ATTRIBUTE_UNUSED, { virQEMUDriverPtr driver = opaque; virObjectEventPtr event = NULL; + virObjectEventPtr event2 = NULL; const char *path; virDomainDiskDefPtr disk; @@ -1020,8 +1021,12 @@ qemuProcessHandleBlockJob(qemuMonitorPtr mon ATTRIBUTE_UNUSED, disk = qemuProcessFindDomainDiskByAlias(vm, diskAlias); if (disk) { + /* Have to generate two variants of the event for old vs. new + * client callbacks */ path = virDomainDiskGetSource(disk); event = virDomainEventBlockJobNewFromObj(vm, path, type, status); + event2 = virDomainEventBlockJob2NewFromObj(vm, disk->dst, type, + status); /* XXX If we completed a block pull or commit, then recompute * the cached backing chain to match. Better would be storing * the chain ourselves rather than reprobing, but this @@ -1048,6 +1053,8 @@ qemuProcessHandleBlockJob(qemuMonitorPtr mon ATTRIBUTE_UNUSED, if (event) qemuDomainEventQueue(driver, event); + if (event2) + qemuDomainEventQueue(driver, event2); return 0; } diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 85fe597..8df35e2 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -321,6 +321,11 @@ remoteDomainBuildEventCallbackDeviceRemoved(virNetClientProgramPtr prog, void *evdata, void *opaque); static void +remoteDomainBuildEventBlockJob2(virNetClientProgramPtr prog, + virNetClientPtr client, + void *evdata, void *opaque); + +static void remoteNetworkBuildEventLifecycle(virNetClientProgramPtr prog ATTRIBUTE_UNUSED, virNetClientPtr client ATTRIBUTE_UNUSED, void *evdata, void *opaque); @@ -467,6 +472,10 @@ static virNetClientProgramEvent remoteEvents[] = { remoteDomainBuildEventCallbackDeviceRemoved, sizeof(remote_domain_event_callback_device_removed_msg), (xdrproc_t)xdr_remote_domain_event_callback_device_removed_msg }, + { REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB_2, + remoteDomainBuildEventBlockJob2, + sizeof(remote_domain_event_block_job_2_msg), + (xdrproc_t)xdr_remote_domain_event_block_job_2_msg }, }; @@ -5023,7 +5032,7 @@ remoteDomainBuildEventBlockJobHelper(virConnectPtr conn, if (!dom) return; - event = virDomainEventBlockJobNewFromDom(dom, msg->path, msg->type, + event = virDomainEventBlockJobNewFromDom(dom, msg->disk, msg->type, msg->status); virDomainFree(dom); @@ -5048,6 +5057,28 @@ remoteDomainBuildEventCallbackBlockJob(virNetClientProgramPtr prog ATTRIBUTE_UNU remote_domain_event_callback_block_job_msg *msg = evdata; remoteDomainBuildEventBlockJobHelper(conn, &msg->msg, msg->callbackID); } +static void +remoteDomainBuildEventBlockJob2(virNetClientProgramPtr prog ATTRIBUTE_UNUSED, + virNetClientPtr client ATTRIBUTE_UNUSED, + void *evdata, void *opaque) +{ + virConnectPtr conn = opaque; + remote_domain_event_block_job_2_msg *msg = evdata; + struct private_data *priv = conn->privateData; + virDomainPtr dom; + virObjectEventPtr event = NULL; + + dom = get_nonnull_domain(conn, msg->dom); + if (!dom) + return; + + event = virDomainEventBlockJob2NewFromDom(dom, msg->disk, msg->type, + msg->status); + + virDomainFree(dom); + + remoteEventQueue(priv, event, msg->callbackID); +} static void remoteDomainBuildEventGraphicsHelper(virConnectPtr conn, diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index ab9b83d..95a1437 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -2352,7 +2352,7 @@ struct remote_domain_event_callback_graphics_msg { struct remote_domain_event_block_job_msg { remote_nonnull_domain dom; - remote_nonnull_string path; + remote_nonnull_string disk; int type; int status; }; @@ -2948,6 +2948,14 @@ struct remote_domain_event_callback_device_removed_msg { remote_domain_event_device_removed_msg msg; }; +struct remote_domain_event_block_job_2_msg { + int callbackID; + remote_nonnull_domain dom; + remote_nonnull_string disk; + int type; + int status; +}; + struct remote_connect_get_cpu_model_names_args { remote_nonnull_string arch; int need_results; @@ -5338,5 +5346,11 @@ enum remote_procedure { * @generate: both * @acl: domain:set_time */ - REMOTE_PROC_DOMAIN_SET_TIME = 338 + REMOTE_PROC_DOMAIN_SET_TIME = 338, + + /** + * @generate: none + * @acl: none + */ + REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB_2 = 339 }; diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index 5b22049..3abc076 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -1797,7 +1797,7 @@ struct remote_domain_event_callback_graphics_msg { }; struct remote_domain_event_block_job_msg { remote_nonnull_domain dom; - remote_nonnull_string path; + remote_nonnull_string disk; int type; int status; }; @@ -2413,6 +2413,13 @@ struct remote_domain_event_callback_device_removed_msg { int callbackID; remote_domain_event_device_removed_msg msg; }; +struct remote_domain_event_block_job_2_msg { + int callbackID; + remote_nonnull_domain dom; + remote_nonnull_string disk; + int type; + int status; +}; struct remote_connect_get_cpu_model_names_args { remote_nonnull_string arch; int need_results; @@ -2802,4 +2809,5 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_FSTHAW = 336, REMOTE_PROC_DOMAIN_GET_TIME = 337, REMOTE_PROC_DOMAIN_SET_TIME = 338, + REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB_2 = 339, }; diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 6b3dd70..d136862 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -10921,8 +10921,9 @@ vshEventBlockJobPrint(virConnectPtr conn ATTRIBUTE_UNUSED, if (!data->loop && *data->count) return; - vshPrint(data->ctl, _("event 'block-job' for domain %s: %s for %s %s\n"), - virDomainGetName(dom), vshDomainBlockJobToString(type), + vshPrint(data->ctl, _("event '%s' for domain %s: %s for %s %s\n"), + data->cb->name, virDomainGetName(dom), + vshDomainBlockJobToString(type), disk, vshDomainBlockJobStatusToString(status)); (*data->count)++; if (!data->loop) @@ -11049,6 +11050,8 @@ static vshEventCallback vshEventCallbacks[] = { VIR_DOMAIN_EVENT_CALLBACK(vshEventPMChangePrint), }, { "device-removed", VIR_DOMAIN_EVENT_CALLBACK(vshEventDeviceRemovedPrint), }, + { "block-job-2", + VIR_DOMAIN_EVENT_CALLBACK(vshEventBlockJobPrint), }, }; verify(VIR_DOMAIN_EVENT_ID_LAST == ARRAY_CARDINALITY(vshEventCallbacks)); -- 1.9.3

2 3