September 2015 - Devel - libvirt List Archives

[libvirt] [PATCH] Fix python error reporting for some storage operations
by Cole Robinson 07 Mar '25

07 Mar '25

In the python bindings, all vir* classes expect to be passed a virConnect object when instantiated. Before the storage stuff, these classes were only instantiated in virConnect methods, so the generator is hardcoded to pass 'self' as the connection instance to these classes. Problem is there are some methods that return pool or vol instances which aren't called from virConnect: you can lookup a storage volume's associated pool, and can lookup volumes from a pool. In these cases passing 'self' doesn't give the vir* instance a connection, so when it comes time to raise an exception crap hits the fan. Rather than rework the generator to accomodate this edge case, I just fixed the init functions for virStorage* to pull the associated connection out of the passed value if it's not a virConnect instance. Thanks, Cole diff --git a/python/generator.py b/python/generator.py index 01a17da..c706b19 100755 --- a/python/generator.py +++ b/python/generator.py @@ -962,8 +962,12 @@ def buildWrappers(): list = reference_keepers[classname] for ref in list: classes.write(" self.%s = None\n" % ref[1]) - if classname in [ "virDomain", "virNetwork", "virStoragePool", "virStorageVol" ]: + if classname in [ "virDomain", "virNetwork" ]: classes.write(" self._conn = conn\n") + elif classname in [ "virStorageVol", "virStoragePool" ]: + classes.write(" self._conn = conn\n" + \ + " if not isinstance(conn, virConnect):\n" + \ + " self._conn = conn._conn\n") classes.write(" if _obj != None:self._o = _obj;return\n") classes.write(" self._o = None\n\n"); destruct=None

4 3

[libvirt] [PATCH] web docs -- macvtap mode explanation
by Stefan Berger 27 Oct '23

27 Oct '23

This adds more information about the different macvtap device modes. Regards, Stefan

3 2

[libvirt] Supporting vhost-net and macvtap in libvirt for QEMU
by Anthony Liguori 27 Oct '23

27 Oct '23

Disclaimer: I am neither an SR-IOV nor a vhost-net expert, but I've CC'd people that are who can throw tomatoes at me for getting bits wrong :-) I wanted to start a discussion about supporting vhost-net in libvirt. vhost-net has not yet been merged into qemu but I expect it will be soon so it's a good time to start this discussion. There are two modes worth supporting for vhost-net in libvirt. The first mode is where vhost-net backs to a tun/tap device. This is behaves in very much the same way that -net tap behaves in qemu today. Basically, the difference is that the virtio backend is in the kernel instead of in qemu so there should be some performance improvement. Current, libvirt invokes qemu with -net tap,fd=X where X is an already open fd to a tun/tap device. I suspect that after we merge vhost-net, libvirt could support vhost-net in this mode by just doing -net vhost,fd=X. I think the only real question for libvirt is whether to provide a user visible switch to use vhost or to just always use vhost when it's available and it makes sense. Personally, I think the later makes sense. The more interesting invocation of vhost-net though is one where the vhost-net device backs directly to a physical network card. In this mode, vhost should get considerably better performance than the current implementation. I don't know the syntax yet, but I think it's reasonable to assume that it will look something like -net tap,dev=eth0. The effect will be that eth0 is dedicated to the guest. On most modern systems, there is a small number of network devices so this model is not all that useful except when dealing with SR-IOV adapters. In that case, each physical device can be exposed as many virtual devices (VFs). There are a few restrictions here though. The biggest is that currently, you can only change the number of VFs by reloading a kernel module so it's really a parameter that must be set at startup time. I think there are a few ways libvirt could support vhost-net in this second mode. The simplest would be to introduce a new tag similar to <source network='br0'>. In fact, if you probed the device type for the network parameter, you could probably do something like <source network='eth0'> and have it Just Work. Another model would be to have libvirt see an SR-IOV adapter as a network pool whereas it handled all of the VF management. Considering how inflexible SR-IOV is today, I'm not sure whether this is the best model. Has anyone put any more thought into this problem or how this should be modeled in libvirt? Michael, could you share your current thinking for -net syntax? -- Regards, Anthony Liguori

8 26

[libvirt] Bump tar format?
by Andrea Bolognani 15 Jun '16

15 Jun '16

Hi everyone, commit cc3d52e[1] has apparently broken 'make dist' for at least some platforms: Fedora 20 is fine[2], but both Fedora 21 and rawhide[3][4] are currently failing. My laptop is running Fedora 22 and can make dist just fine, which is of course very confusing. The issue is caused by the following tar error: tar: libvirt-1.2.18/tests/nodeinfodata/linux-deconfigured-cpus/ \ node/node0/cpu16/topology/core_siblings_list: \ link name is too long; not dumped There seem to be basically two ways to solve the failure: 1. rename the test case, making it shorter; 2. bump tar format from ustar, which has limitations on the length of file names, to posix/pax, which has no such limitations. I'm not a fan of the former option because short names are usually very opaque, see tests/nodeinfodata/linux-test[0-8] for some perfect examples. On the other hand, I'm not sure whether bumping the tar version would affect portability in a significant way: the automake documentation[5] reports that this format is very young and should probably be restricted to packages that target only very modern platforms but the GNU tar documentation[6], while containing some warnings as well, suggest that this format is going to become the default. Any suggestions on how to proceed? Cheers. [1] https://www.redhat.com/archives/libvir-list/2015-July/msg00662.html [2] https://ci.centos.org/view/libvirt-project/job/libvirt-daemon -rpm/systems=libvirt-fedora-20/327/ [3] https://ci.centos.org/view/libvirt-project/job/libvirt-daemon -rpm/systems=libvirt-fedora-21/327/ [4] https://ci.centos.org/view/libvirt-project/job/libvirt-daemon -rpm/systems=libvirt-fedora-rawhide/327/ [5] https://www.gnu.org/software/automake/manual/html_node/List-of -Automake-options.html [6] http://www.gnu.org/software/tar/manual/html_chapter/tar_8.html -- Andrea Bolognani Software Engineer - Virtualization Team

6 13

[libvirt] Libvirt multi queue support
by Naor Shlomo 14 Jun '16

14 Jun '16

Hello experts, Could anyone please tell me if Multi Queue it fully supported in Libvirt and if so what version contains it? Thanks, Naor

3 9

[libvirt] [PATCH/RFC] Add missing delta from Ubuntu to apparmor profiles
by Stefan Bader 25 May '16

25 May '16

This had been on the Debian package list before but its time to take this onwards. So the goal would be to have one set to rule them all (when using apparmor) and drop the seperate set of definitions which exist at least in the Ubuntu packaging. Right now the patch would be at a state which adds all missing files and rules to the current examples in libvirt and installs them when using --with-apparmor-profiles. One problem seems to be that some of the definitions might cause parse failures on certain versions of apparmor. I checked this morning and this looks a bit hairy. So some apparmor 2.8 versions potentially have issues, but not all apparmor 2.8 are the same (gah). I could imagine (but John, we really could use some guidance here ;)) that at least some changes could be related to version 2.8.95~2430: + debian/patches/mediate-signals.patch, debian/patches/change-signal-syntax.patch: Parse signal rules with apparmor_parser. See the apparmor.d(5) man page for syntax details. + debian/patches/change-ptrace-syntax.patch, debian/patches/mediate-ptrace.patch: Parse ptrace rules with apparmor_parser. See the apparmor.d(5) man page for syntax details. But, regardless of the when, the apparmor rules maybe need a way to handle versioned features of the parser. One proposal was to comment out problematic rules and allow the packager to re-enable things. Maybe going one step further and have some pre-processing that handles version based sections (like #if (APPARMOR_VERSION >= xxx)). So that is where we stand. Ideas are very welcome. -Stefan --- >From aec5cf8cc30c80492a37856626264c3d4c27a31f Mon Sep 17 00:00:00 2001 From: Stefan Bader <stefan.bader(a)canonical.com> Date: Thu, 18 Sep 2014 14:15:17 +0200 Subject: [PATCH] Add missing delta from Ubuntu to apparmor profiles This fixes up the upstream profiles and would allow to drop apparmor related delta from the Ubuntu package. Thanks to Serge Hallyn for the Makefile.am install hook that allows to rename the local file. Signed-off-by: Stefan Bader <stefan.bader(a)canonical.com> --- examples/apparmor/Makefile.am | 10 ++++++++ examples/apparmor/libvirt-lxc | 15 +++++++++++- examples/apparmor/libvirt-qemu | 31 +++++++++++++++++++++++- examples/apparmor/local-usr.sbin.libvirtd | 2 ++ examples/apparmor/usr.lib.libvirt.virt-aa-helper | 25 ++++++++++++++++--- examples/apparmor/usr.sbin.libvirtd | 17 ++++++++++++- 6 files changed, 94 insertions(+), 6 deletions(-) create mode 100644 examples/apparmor/local-usr.sbin.libvirtd diff --git a/examples/apparmor/Makefile.am b/examples/apparmor/Makefile.am index 7a20e16..aa46cb9 100644 --- a/examples/apparmor/Makefile.am +++ b/examples/apparmor/Makefile.am @@ -20,6 +20,7 @@ EXTRA_DIST= \ libvirt-qemu \ libvirt-lxc \ usr.lib.libvirt.virt-aa-helper \ + local-usr.sbin.libvirtd \ usr.sbin.libvirtd if WITH_APPARMOR_PROFILES @@ -29,6 +30,15 @@ apparmor_DATA = \ usr.sbin.libvirtd \ $(NULL) +localdir = $(apparmordir)/local +local_DATA = \ + local-usr.sbin.libvirtd \ + $(NULL) + +install-data-hook: + mv $(DESTDIR)$(localdir)/local-usr.sbin.libvirtd \ + $(DESTDIR)$(localdir)/usr.sbin.libvirtd + abstractionsdir = $(apparmordir)/abstractions abstractions_DATA = \ libvirt-qemu \ diff --git a/examples/apparmor/libvirt-lxc b/examples/apparmor/libvirt-lxc index 4bfb503..4705e0a 100644 --- a/examples/apparmor/libvirt-lxc +++ b/examples/apparmor/libvirt-lxc @@ -1,12 +1,18 @@ -# Last Modified: Fri Feb 7 13:01:36 2014 +# Last Modified: Thu, 18 Sep 2014 13:56:49 +0200 #include <abstractions/base> umount, + dbus, + signal, + ptrace, # ignore DENIED message on / remount deny mount options=(ro, remount) -> /, + # support use of cgmanager proxy + mount options=(move) /sys/fs/cgroup/cgmanager/ -> /sys/fs/cgroup/cgmanager.lower/, + # allow tmpfs mounts everywhere mount fstype=tmpfs, @@ -33,8 +39,15 @@ mount fstype=fusectl -> /sys/fs/fuse/connections/, mount fstype=securityfs -> /sys/kernel/security/, mount fstype=debugfs -> /sys/kernel/debug/, + deny mount fstype=debugfs -> /var/lib/ureadahead/debugfs/, mount fstype=proc -> /proc/, mount fstype=sysfs -> /sys/, + + mount options=(rw nosuid nodev noexec remount) -> /sys/, + mount options=(rw remount) -> /sys/kernel/security/, + mount options=(rw remount) -> /sys/fs/pstore/, + mount options=(ro remount) -> /sys/fs/pstore/, + deny /sys/firmware/efi/efivars/** rwklx, deny /sys/kernel/security/** rwklx, diff --git a/examples/apparmor/libvirt-qemu b/examples/apparmor/libvirt-qemu index c6de6dd..b69e64c 100644 --- a/examples/apparmor/libvirt-qemu +++ b/examples/apparmor/libvirt-qemu @@ -1,4 +1,4 @@ -# Last Modified: Wed Sep 3 21:52:03 2014 +# Last Modified: Thu, 18 Sep 2014 16:41:21 +0200 #include <abstractions/base> #include <abstractions/consoles> @@ -13,15 +13,22 @@ capability setgid, capability setuid, + # this is needed with libcap-ng support, however it breaks a lot of things + # atm, so just silence the denial until libcap-ng works right. LP: #522845 + deny capability setpcap, + network inet stream, network inet6 stream, /dev/net/tun rw, + /dev/tap* rw, /dev/kvm rw, /dev/ptmx rw, /dev/kqemu rw, @{PROC}/*/status r, @{PROC}/sys/kernel/cap_last_cap r, + owner @{PROC}/*/auxv r, + @{PROC}/sys/vm/overcommit_memory r, # For hostdev access. The actual devices will be added dynamically /sys/bus/usb/devices/ r, @@ -38,6 +45,9 @@ /dev/snd/* rw, capability ipc_lock, # spice + /usr/bin/qemu-system-i386-spice rmix, + /usr/bin/qemu-system-x86_64-spice rmix, + /{dev,run}/shm/ r, owner /{dev,run}/shm/spice.* rw, # 'kill' is not required for sound and is a security risk. Do not enable # unless you absolutely need it. @@ -73,6 +83,7 @@ # the various binaries /usr/bin/kvm rmix, /usr/bin/qemu rmix, + /usr/bin/qemu-system-aarch64 rmix, /usr/bin/qemu-system-arm rmix, /usr/bin/qemu-system-cris rmix, /usr/bin/qemu-system-i386 rmix, @@ -91,6 +102,7 @@ /usr/bin/qemu-system-sparc rmix, /usr/bin/qemu-system-sparc64 rmix, /usr/bin/qemu-system-x86_64 rmix, + /usr/bin/qemu-system-x86_64-spice rmix, /usr/bin/qemu-alpha rmix, /usr/bin/qemu-arm rmix, /usr/bin/qemu-armeb rmix, @@ -117,6 +129,16 @@ /bin/dash rmix, /bin/dd rmix, /bin/cat rmix, + /etc/pki/CA/ r, + /etc/pki/CA/* r, + /etc/pki/libvirt/ r, + /etc/pki/libvirt/** r, + + # for rbd + /etc/ceph/ceph.conf r, + + # for access to hugepages + owner "/run/hugepages/kvm/libvirt/qemu/**" rw, # for usb access /dev/bus/usb/ r, @@ -124,6 +146,13 @@ /sys/bus/ r, /sys/class/ r, + signal (receive) peer=/usr/sbin/libvirtd, + ptrace (tracedby) peer=/usr/sbin/libvirtd, + + # for ppc device-tree access + @{PROC}/device-tree/ r, + @{PROC}/device-tree/** r, + /usr/{lib,libexec}/qemu-bridge-helper Cx -> qemu_bridge_helper, # child profile for bridge helper process profile qemu_bridge_helper { diff --git a/examples/apparmor/local-usr.sbin.libvirtd b/examples/apparmor/local-usr.sbin.libvirtd new file mode 100644 index 0000000..6e19f20 --- /dev/null +++ b/examples/apparmor/local-usr.sbin.libvirtd @@ -0,0 +1,2 @@ +# Site-specific additions and overrides for usr.sbin.libvirtd. +# For more details, please see /etc/apparmor.d/local/README. diff --git a/examples/apparmor/usr.lib.libvirt.virt-aa-helper b/examples/apparmor/usr.lib.libvirt.virt-aa-helper index bceaaff..4df86b0 100644 --- a/examples/apparmor/usr.lib.libvirt.virt-aa-helper +++ b/examples/apparmor/usr.lib.libvirt.virt-aa-helper @@ -1,8 +1,9 @@ -# Last Modified: Mon Apr 5 15:10:27 2010 +# Last Modified: Thu, 18 Sep 2014 14:05:36 +0200 #include <tunables/global> /usr/lib/libvirt/virt-aa-helper { #include <abstractions/base> + #include <abstractions/user-tmp> # needed for searching directories capability dac_override, @@ -19,6 +20,12 @@ # for hostdev /sys/devices/ r, /sys/devices/** r, + /sys/bus/usb/devices/ r, + /sys/bus/usb/devices/** r, + deny /dev/sd* r, + deny /dev/dm-* r, + deny /dev/mapper/ r, + deny /dev/mapper/* r, /usr/lib/libvirt/virt-aa-helper mr, /sbin/apparmor_parser Ux, @@ -26,8 +33,11 @@ /etc/apparmor.d/libvirt/* r, /etc/apparmor.d/libvirt/libvirt-[0-9a-f]*-[0-9a-f]*-[0-9a-f]*-[0-9a-f]*-[0-9a-f]* rw, - # for backingstore -- allow access to non-hidden files in @{HOME} as well - # as storage pools + # For backingstore, virt-aa-helper needs to peek inside the disk image, so + # allow access to non-hidden files in @{HOME} as well as storage pools, and + # removable media and filesystems, and certain file extentions. A + # virt-aa-helper failure when checking a disk for backinsgstore is non-fatal + # (but obviously the backingstore won't be added). audit deny @{HOME}/.* mrwkl, audit deny @{HOME}/.*/ rw, audit deny @{HOME}/.*/** mrwkl, @@ -35,8 +45,17 @@ audit deny @{HOME}/bin/** mrwkl, @{HOME}/ r, @{HOME}/** r, + @{HOME}/.Private/** mrwlk, + @{HOMEDIRS}/.ecryptfs/*/.Private/** mrwlk, + /var/lib/libvirt/images/ r, /var/lib/libvirt/images/** r, + /var/lib/nova/images/** r, + /var/lib/nova/instances/_base/** r, + /var/lib/nova/instances/snapshots/** r, + /var/lib/eucalyptus/instances/**/disk* r, + /var/lib/eucalyptus/instances/**/loader* r, + /var/lib/uvtool/libvirt/images/** r, /{media,mnt,opt,srv}/** r, /**.img r, diff --git a/examples/apparmor/usr.sbin.libvirtd b/examples/apparmor/usr.sbin.libvirtd index 3011eff..814b4d81 100644 --- a/examples/apparmor/usr.sbin.libvirtd +++ b/examples/apparmor/usr.sbin.libvirtd @@ -1,10 +1,12 @@ -# Last Modified: Mon Apr 5 15:03:58 2010 +# Last Modified: Tue, 23 Sep 2014 09:28:07 +0200 #include <tunables/global> @{LIBVIRT}="libvirt" /usr/sbin/libvirtd { #include <abstractions/base> #include <abstractions/dbus> + # Site-specific additions and overrides. See local/README for details. + #include <local/usr.sbin.libvirtd> capability kill, capability net_admin, @@ -23,6 +25,7 @@ capability setpcap, capability mknod, capability fsetid, + capability ipc_lock, capability audit_write, # Needed for vfio @@ -33,6 +36,12 @@ network inet6 stream, network inet6 dgram, network packet dgram, + network netlink, + + dbus bus=system, + signal, + ptrace, + unix, # Very lenient profile for libvirtd since we want to first focus on confining # the guests. Guests will have a very restricted profile. @@ -45,6 +54,12 @@ /usr/sbin/* PUx, /lib/udev/scsi_id PUx, /usr/lib/xen-common/bin/xen-toolstack PUx, + /usr/lib/xen-*/bin/pygrub PUx, + /usr/lib/xen-*/bin/libxl-save-helper PUx, + + # Required by nwfilter_ebiptables_driver.c:ebiptablesWriteToTempFile() to + # write and run an ebtables script. + /var/lib/libvirt/virtd* ixr, # force the use of virt-aa-helper audit deny /sbin/apparmor_parser rwxl, -- 1.9.1

5 17

[libvirt] [PATCH] qemu: don't refuse to undefine a guest with NVRAM file
by Daniel P. Berrange 23 May '16

23 May '16

The undefine operation should always be allowed to succeed regardless of whether any NVRAM file exists. ie we should not force the application to use the VIR_DOMAIN_UNDEFINE_NVRAM flag. It is valid for the app to decide it wants the NVRAM file left on disk, in the same way that disk images are left on disk at undefine. --- src/qemu/qemu_driver.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index bec05d4..302bf48 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -6985,19 +6985,13 @@ qemuDomainUndefineFlags(virDomainPtr dom, if (!virDomainObjIsActive(vm) && vm->def->os.loader && vm->def->os.loader->nvram && - virFileExists(vm->def->os.loader->nvram)) { - if (!(flags & VIR_DOMAIN_UNDEFINE_NVRAM)) { - virReportError(VIR_ERR_OPERATION_INVALID, "%s", - _("cannot delete inactive domain with nvram")); - goto cleanup; - } - - if (unlink(vm->def->os.loader->nvram) < 0) { - virReportSystemError(errno, - _("failed to remove nvram: %s"), - vm->def->os.loader->nvram); - goto cleanup; - } + virFileExists(vm->def->os.loader->nvram) && + (flags & VIR_DOMAIN_UNDEFINE_NVRAM) && + (unlink(vm->def->os.loader->nvram) < 0)) { + virReportSystemError(errno, + _("failed to remove nvram: %s"), + vm->def->os.loader->nvram); + goto cleanup; } if (virDomainDeleteConfig(cfg->configDir, cfg->autostartDir, vm) < 0) -- 2.1.0

6 17

[libvirt] [PATCH v2 0/8] Add support for fetching statistics of completed jobs
by Jiri Denemark 09 May '16

09 May '16

Using virDomainGetJobStats, we can monitor running jobs but sometimes it may be useful to get statistics about a job that already finished, for example, to get the final amount of data transferred during migration or to get an idea about total downtime. This is what the following patches are about. Version 2: - changed according to John's review (see individual patches for details) Jiri Denemark (8): Refactor job statistics qemu: Avoid incrementing jobs_queued if virTimeMillisNow fails Add support for fetching statistics of completed jobs qemu: Silence coverity on optional migration stats virsh: Add support for completed job stats qemu: Transfer migration statistics to destination qemu: Recompute downtime and total time when migration completes qemu: Transfer recomputed stats back to source include/libvirt/libvirt.h.in | 11 ++ src/libvirt.c | 11 +- src/qemu/qemu_domain.c | 189 ++++++++++++++++++++++++++- src/qemu/qemu_domain.h | 32 ++++- src/qemu/qemu_driver.c | 130 ++++-------------- src/qemu/qemu_migration.c | 304 ++++++++++++++++++++++++++++++++++++------- src/qemu/qemu_monitor_json.c | 10 +- src/qemu/qemu_process.c | 9 +- tools/virsh-domain.c | 27 +++- tools/virsh.pod | 10 +- 10 files changed, 557 insertions(+), 176 deletions(-) -- 2.1.0

3 12

[libvirt] Libvirt bite sized tasks
by Michal Privoznik 20 Mar '16

20 Mar '16

Dear lists, I've just started new wiki page which aim is to summarize small and trivial tasks, that starting contributors can take, investigate and implement. The aim is to give them something easy to start with while not scaring them out about complexity of our code. The page can be found here: http://wiki.libvirt.org/page/BiteSizedTasks The name is copied from qemu wiki: http://wiki.qemu.org/BiteSizedTasks Please, feel free to extend the list. I plan to use it in the future when interviewing GSoC candidates. Michal

3 4

[libvirt] [PATCH V2 0/9] support multi-thread compress migration.
by ShaoHe Feng 18 Jan '16

18 Jan '16

These series patches support multi-thread compress during live migration. Eli Qiao (4): Add test cases for qemuMonitorJSONGetMigrationParameter remote: Add support for set and get multil thread migration parameters qemu_driver: Add support to set/get migration parameters. virsh: Add set and get multi-thread migration parameters commands ShaoHe Feng (5): qemu_migration: Add support for mutil-thread compressed migration enable qemu: Add monitor API for get/set migration parameters set multi-thread compress params for Migrate3 during live migration virsh: add multi-thread migration option for live migrate command Implement the public APIs for multi-thread compress parameters. .gnulib | 2 +- daemon/remote.c | 62 +++++++++++ include/libvirt/libvirt-domain.h | 31 ++++++ src/driver-hypervisor.h | 14 +++ src/libvirt-domain.c | 110 +++++++++++++++++++ src/libvirt_public.syms | 5 + src/qemu/qemu_domain.h | 3 + src/qemu/qemu_driver.c | 186 ++++++++++++++++++++++++++++++++ src/qemu/qemu_migration.c | 105 ++++++++++++++++++ src/qemu/qemu_migration.h | 32 ++++-- src/qemu/qemu_monitor.c | 40 ++++++- src/qemu/qemu_monitor.h | 11 ++ src/qemu/qemu_monitor_json.c | 93 ++++++++++++++++ src/qemu/qemu_monitor_json.h | 9 ++ src/qemu/qemu_monitor_text.c | 95 +++++++++++++++++ src/qemu/qemu_monitor_text.h | 10 ++ src/remote/remote_driver.c | 54 ++++++++++ src/remote/remote_protocol.x | 30 +++++- src/remote_protocol-structs | 26 +++++ tests/qemumonitorjsontest.c | 53 ++++++++++ tools/virsh-domain.c | 223 ++++++++++++++++++++++++++++++++++++++- tools/virsh.pod | 37 +++++-- 22 files changed, 1212 insertions(+), 19 deletions(-) -- 2.1.4

4 19