- Devel - libvirt List Archives

[libvirt] [RFC PATCH 0/5] Summary on QEMU I/O throttling support to libvirt
by Lei Li 12 Oct '11

12 Oct '11

Today libvirt supports the cgroups blkio-controller, which handles proportional shares and throughput/iops limits on host block devices. blkio-controller does not support network file systems (NFS) or other QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are not host block devices. QEMU I/O throttling works with all types of drive and can be applied independently to each drive attached to a guest and supports throughput/iops limits. To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomainBlockIoThrottle(), new command 'blkiothrottle' and Python bindings. Notes: we are sending this series out now(even though they are not completed yet.)because we want to start the review process. These patches implemented the below features by Zhi Yong Wu: 1) Enable the blkio throttling in xml when guest is starting up. Add blkio throttling in xml as follows: <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/kvm-one.img'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> <iothrottle bps='100000' bps_rd='20000'.../> </disk> 2) Enable blkio throttling setting at guest running time. virsh blkiothrottle<domain> <device> [--bps<number>] [--bps_rd<number>] \ [--bps_wr<number>] [--iops<number>] [--iops_rd<number>] [--iops_wr<number>] [PATCH 1/5] Add new API virDomainBlockIoThrottle. [PATCH 2/5] Add virDomainBlockIoThrottle support to the remote driver. [PATCH 3/5] Implement virDomainBlockIoThrottle for the qemu driver. [PATCH 4/5] Enable the virDomainBlockIoThrottle API in virsh. [PATCH 5/5] Enable virDomainBlockIoThrottle in the python API. And I will address feedback and work on the missing features in few days includes: 3) The support to display the setting for block i/o throttling - HMP/QMP 4) Python binding support. 5) --current --live --config options support to unify the libvirt API. -- Lei

2 1

[libvirt] [PATCH 0/4] snapshot: optimize qemu relationship traversal
by Eric Blake 12 Oct '11

12 Oct '11

After working with ESX and VBox, I realized that a tweak to how snapshot relationships are stored would speed things up from O(n^2) to O(n). The hash table must be kept for name lookup, but the speedup comes by tracking relationships with each snapshot rather than crawling the entire hash table rebuilding those relations for every operation that needs particular relations. Eric Blake (4): snapshot: framework for more efficient relation traversal snapshot: track qemu snapshot relations snapshot: take advantage of new relations snapshot: drop dead parameters src/conf/domain_conf.c | 266 +++++++++++++++++++++++++--------------------- src/conf/domain_conf.h | 22 ++-- src/libvirt_private.syms | 3 +- src/qemu/qemu_driver.c | 84 +++++++++++---- 4 files changed, 224 insertions(+), 151 deletions(-) -- 1.7.4.4

2 17

[libvirt] [PATCH] snapshot: use correct qmp monitor command
by Eric Blake 11 Oct '11

11 Oct '11

To date, disk snapshots worked by accident, as they were always using hmp fallback due to a typo in commit e702b5b not picking up on the (intentional) difference in command names between the two monitor protocols. * src/qemu/qemu_monitor_json.c (qemuMonitorJSONDiskSnapshot): Spell QMP command correctly. Reported by Luiz Capitulino. --- How embarrassing. http://wiki.qemu.org/Features/Snapshots even called out the difference in naming. src/qemu/qemu_monitor_json.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index 3d383c8..18d1c9f 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -2960,7 +2960,7 @@ qemuMonitorJSONDiskSnapshot(qemuMonitorPtr mon, const char *device, virJSONValuePtr cmd; virJSONValuePtr reply = NULL; - cmd = qemuMonitorJSONMakeCommand("snapshot-blkdev-sync", + cmd = qemuMonitorJSONMakeCommand("blockdev-snapshot-sync", "s:device", device, "s:snapshot-file", file, NULL); @@ -2972,7 +2972,7 @@ qemuMonitorJSONDiskSnapshot(qemuMonitorPtr mon, const char *device, if (qemuMonitorJSONHasError(reply, "CommandNotFound") && qemuMonitorCheckHMP(mon, "snapshot_blkdev")) { - VIR_DEBUG("snapshot-blkdev-sync command not found, trying HMP"); + VIR_DEBUG("blockdev-snapshot-sync command not found, trying HMP"); ret = qemuMonitorTextDiskSnapshot(mon, device, file); goto cleanup; } -- 1.7.4.4

2 2

[libvirt] [PATCHv2 0/7] snapshot: listing children
by Eric Blake 11 Oct '11

11 Oct '11

Cleaned up the rebase goofs present throughout my v1: https://www.redhat.com/archives/libvir-list/2011-September/msg01270.html Eric Blake (7): snapshot: new virDomainSnapshotListChildrenNames API snapshot: virsh snapshot-list and children snapshot: virsh fallback for snapshot-list --tree --from snapshot: virsh fallback for snapshot-list --from children snapshot: virsh fallback for snapshot-list --descendants --from snapshot: remote protocol for snapshot children snapshot: implement snapshot children listing in qemu include/libvirt/libvirt.h.in | 27 +++++-- python/generator.py | 4 + python/libvirt-override-api.xml | 12 ++- python/libvirt-override.c | 45 +++++++++ src/conf/domain_conf.c | 51 +++++++++++ src/conf/domain_conf.h | 7 ++ src/driver.h | 12 +++ src/libvirt.c | 111 +++++++++++++++++++++++ src/libvirt_private.syms | 2 + src/libvirt_public.syms | 2 + src/qemu/qemu_driver.c | 87 ++++++++++++++++++ src/remote/remote_driver.c | 2 + src/remote/remote_protocol.x | 25 +++++- src/remote_protocol-structs | 20 ++++ tools/virsh.c | 190 ++++++++++++++++++++++++++++++++------ tools/virsh.pod | 9 ++- 16 files changed, 564 insertions(+), 42 deletions(-) -- 1.7.4.4

3 43

[libvirt] [PATCH] build: fix mingw build without sasl
by Eric Blake 11 Oct '11

11 Oct '11

Detected by autogen.sh on a cross-mingw build. * src/libvirt_private.syms (virnetsaslcontext.h): Move symbols... * src/libvirt_sasl.syms: ...to new file. * src/Makefile.am (USED_SYM_FILES) [HAVE_SASL]: Use new file. (EXTRA_DIST): Ship it. --- Pushing under the build-breaker rule. src/Makefile.am | 13 +++++++++---- src/libvirt_private.syms | 15 --------------- src/libvirt_sasl.syms | 20 ++++++++++++++++++++ 3 files changed, 29 insertions(+), 19 deletions(-) create mode 100644 src/libvirt_sasl.syms diff --git a/src/Makefile.am b/src/Makefile.am index 272dfbf..302d395 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -1188,18 +1188,23 @@ if WITH_NETWORK USED_SYM_FILES += libvirt_network.syms endif +if HAVE_SASL +USED_SYM_FILES += libvirt_sasl.syms +endif + EXTRA_DIST += \ libvirt_public.syms \ libvirt_private.syms \ libvirt_driver_modules.syms \ libvirt_bridge.syms \ + libvirt_daemon.syms \ libvirt_linux.syms \ libvirt_macvtap.syms \ - libvirt_daemon.syms \ - libvirt_nwfilter.syms \ + libvirt_network.syms \ + libvirt_nwfilter.syms \ + libvirt_sasl.syms \ libvirt_vmx.syms \ - libvirt_xenxs.syms \ - libvirt_network.syms + libvirt_xenxs.syms GENERATED_SYM_FILES = libvirt.syms libvirt.def libvirt_qemu.def diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index d79f65c..caf9f59 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1178,20 +1178,6 @@ virNetMessageQueueServe; virNetMessageSaveError; -# virnetsaslcontext.h -virNetSASLContextCheckIdentity; -virNetSASLContextNewServer; -virNetSASLSessionExtKeySize; -virNetSASLSessionFree; -virNetSASLSessionGetIdentity; -virNetSASLSessionGetKeySize; -virNetSASLSessionListMechanisms; -virNetSASLSessionNewServer; -virNetSASLSessionSecProps; -virNetSASLSessionServerStart; -virNetSASLSessionServerStep; - - # virnetserver.h virNetServerAddProgram; virNetServerAddService; @@ -1232,7 +1218,6 @@ virNetServerClientSendMessage; virNetServerClientSetCloseHook; virNetServerClientSetIdentity; virNetServerClientSetPrivateData; -virNetServerClientSetSASLSession; # virnetserverprogram.h diff --git a/src/libvirt_sasl.syms b/src/libvirt_sasl.syms new file mode 100644 index 0000000..2c278c8 --- /dev/null +++ b/src/libvirt_sasl.syms @@ -0,0 +1,20 @@ +# +# SASL-specific symbols +# + +# virnetsaslcontext.h +virNetSASLContextCheckIdentity; +virNetSASLContextNewServer; +virNetSASLSessionExtKeySize; +virNetSASLSessionFree; +virNetSASLSessionGetIdentity; +virNetSASLSessionGetKeySize; +virNetSASLSessionListMechanisms; +virNetSASLSessionNewServer; +virNetSASLSessionSecProps; +virNetSASLSessionServerStart; +virNetSASLSessionServerStep; + + +# virnetserverclient.h +virNetServerClientSetSASLSession; -- 1.7.4.4

1 0

[libvirt] [PATCH] build: fix 'make rpm'
by Eric Blake 11 Oct '11

11 Oct '11

Since commit ddf3bd3, 'make rpm' failed with: RPM build errors: File not found: /home/remote/eblake/rpmbuild/BUILDROOT/libvirt-0.9.6-1.fc14.eblake1318366440.x86_64/usr/share/systemtap/tapset/libvirtd.stp * libvirt.spec.in (with_dtrace): Match installed .stp files. --- Pushing under the build-breaker rule. Still one to go before ./autobuild.sh is happy. A mingw cross-build now fails with: Creating library file: .libs/libvirt.dll.a Cannot export virNetSASLContextCheckIdentity: symbol not defined Cannot export virNetSASLContextNewServer: symbol not defined ... libvirt.spec.in | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/libvirt.spec.in b/libvirt.spec.in index 1e76452..7c63710 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -969,7 +969,8 @@ fi rm -f $RPM_BUILD_ROOT%{_sysconfdir}/sysctl.d/libvirtd %endif %if %{with_dtrace} -%{_datadir}/systemtap/tapset/libvirtd.stp +%{_datadir}/systemtap/tapset/libvirt_probes.stp +%{_datadir}/systemtap/tapset/libvirt_functions.stp %endif %dir %attr(0700, root, root) %{_localstatedir}/log/libvirt/qemu/ %dir %attr(0700, root, root) %{_localstatedir}/log/libvirt/lxc/ -- 1.7.4.4

1 0

[libvirt] [PATCH] build: fix 'make distcheck'
by Eric Blake 11 Oct '11

11 Oct '11

I got these distcheck failures with sanlock enabled: ERROR: files left in build directory after distclean: ./tools/virt-sanlock-cleanup ./src/locking/qemu-sanlock.conf * src/Makefile.am (DISTCLEANFILES) [HAVE_SANLOCK]: Clean built file. * tools/Makefile.am (DISTCLEANFILES): Likewise. --- Pushing under the build-breaker rule. src/Makefile.am | 1 + tools/Makefile.am | 5 ++++- 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 793f674..272dfbf 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -1328,6 +1328,7 @@ $(builddir)/locking/%-sanlock.conf: $(srcdir)/locking/sanlock.conf if WITH_QEMU conf_DATA += locking/qemu-sanlock.conf BUILT_SOURCES += locking/qemu-sanlock.conf +DISTCLEANFILES += locking/qemu-sanlock.conf endif else EXTRA_DIST += $(LOCK_DRIVER_SANLOCK_SOURCES) diff --git a/tools/Makefile.am b/tools/Makefile.am index 8a8be1a..c96c7d9 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -28,11 +28,14 @@ EXTRA_DIST = \ libvirt-guests.init.sh \ libvirt-guests.sysconf +DISTCLEANFILES = + bin_SCRIPTS = virt-xml-validate virt-pki-validate bin_PROGRAMS = virsh if HAVE_SANLOCK sbin_SCRIPTS = virt-sanlock-cleanup +DISTCLEANFILES += virt-sanlock-cleanup endif dist_man1_MANS = virt-xml-validate.1 virt-pki-validate.1 virsh.1 @@ -187,4 +190,4 @@ CLEANFILES = $(bin_SCRIPTS) CLEANFILES += *.gcov .libs/*.gcda .libs/*.gcno *.gcno *.gcda *.i *.s MAINTAINERCLEANFILES = $(dist_man1_MANS) -DISTCLEANFILES = $(BUILT_SOURCES) +DISTCLEANFILES += $(BUILT_SOURCES) -- 1.7.4.4

1 0

[libvirt] [PATCH] libvirt-tck: Add testcase to test snapshot functionality.
by Guido Günther 11 Oct '11

11 Oct '11

Hi, I got a bit confused how snapshotting/reverting is supposed to work now so I added some tests. O.k. to apply? This requires Sys::Virt 0.9.5. Cheers, -- Guido --- Build.PL | 2 +- perl-Sys-Virt-TCK.spec.PL | 2 +- scripts/domain/400-snapshots.t | 127 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 129 insertions(+), 2 deletions(-) create mode 100644 scripts/domain/400-snapshots.t diff --git a/Build.PL b/Build.PL index b0c53a2..0b373bf 100644 --- a/Build.PL +++ b/Build.PL @@ -85,7 +85,7 @@ my $b = $class->new( 'Test::Builder' => 0, 'Test::More' => 0, 'Sub::Uplevel' => 0, - 'Sys::Virt' => '0.2.0', + 'Sys::Virt' => '0.9.5', 'XML::Twig' => 0, 'XML::Writer' => 0, 'XML::XPath' => 0, diff --git a/perl-Sys-Virt-TCK.spec.PL b/perl-Sys-Virt-TCK.spec.PL index b6a989e..4ebbcd0 100644 --- a/perl-Sys-Virt-TCK.spec.PL +++ b/perl-Sys-Virt-TCK.spec.PL @@ -63,7 +63,7 @@ BuildRequires: perl(TAP::Harness::Archive) BuildRequires: perl(Test::Builder) BuildRequires: perl(Test::More) BuildRequires: perl(Sub::Uplevel) -BuildRequires: perl(Sys::Virt) >= 0.2.0 +BuildRequires: perl(Sys::Virt) >= 0.9.5 BuildRequires: perl(XML::Twig) BuildRequires: perl(XML::Writer) # RPM autoprovides misses these 3 diff --git a/scripts/domain/400-snapshots.t b/scripts/domain/400-snapshots.t new file mode 100644 index 0000000..4c3a636 --- /dev/null +++ b/scripts/domain/400-snapshots.t @@ -0,0 +1,127 @@ +# -*- perl -*- +# +# Copyright (C) 2011 Univention GmbH +# Author: Guido Guenther <agx(a)sigxcpu.org> +# +# This program is free software; You can redistribute it and/or modify +# it under the GNU General Public License as published by the Free +# Software Foundation; either version 2, or (at your option) any +# later version +# +# The file "LICENSE" distributed along with this file provides full +# details of the terms and conditions +# + +=pod + +=head1 NAME + +domain/400-snapshot.t - Check snapshot operations + +=head1 DESCRIPTION + +Check if snapshots can be created and deleted. Check if a snapshot gets +associated with the right vm metadata when reverting snapshots. + +=cut + +use strict; +use warnings; + +use Test::More tests => 23; + +use Sys::Virt::TCK; +use Test::Exception; + +my $tck = Sys::Virt::TCK->new(); +my $conn = eval { $tck->setup(); }; +BAIL_OUT "failed to setup test harness: $@" if $@; +END { $tck->cleanup if $tck; } + +SKIP: { + my ($pool, $vol, $dom, $snap1, $snap2); + my $minmem = 64 * 1024; + my $maxmem = $minmem * 2; + my $snapshots = 0; + + my $poolxml = $tck->generic_pool("dir") + ->mode("0755")->as_xml; + + diag "Defining transient storage pool $poolxml"; + ok_pool(sub { $pool = $conn->define_storage_pool($poolxml) }, "define transient storage pool"); + lives_ok(sub { $pool->build(0) }, "built storage pool"); + lives_ok(sub { $pool->create }, "started storage pool"); + + my $volxml = $tck->generic_volume("tck.img", "qcow2", 1024*1024*50) + ->allocation(0)->as_xml; + ok_volume(sub { $vol = $pool->create_volume($volxml) }, "creating qcow2 volume $volxml"); + my $pathvol = xpath($vol, "string(/volume/target/path)"); + + my $gd = $tck->generic_domain("tck") + ->memory($minmem); + $gd->rmdisk; + my $domxml = $gd->disk(format => { name => "qemu", type => "qcow2" }, + type => "file", + src => $pathvol, + dst => "vda")->as_xml; + + ok_domain(sub { $dom = $conn->define_domain($domxml) }, "defined persistent domain"); + lives_ok(sub {$dom->create() }, "started domain"); + + my $currmem = int(xpath($dom, "number(/domain/currentMemory)")); + is($currmem, $minmem, "XML description has memory set to $minmem"); + + my $snap1xml = <<EOF; +<domainsnapshot> + <name>Snapshot1</name> + <description>First snapshot</description> +</domainsnapshot> +EOF + diag "Create first snapshot"; + eval { $snap1 = $dom->create_snapshot($snap1xml); }; + skip "Snapshots not implemented", 16 if $@ && err_not_implemented($@); + ok(!$@, "Snapshot created"); + die $@ if $@; + $snapshots++; + + lives_ok( sub { $dom->destroy() }, "Domain destroyed"); + lives_ok(sub { $dom->set_max_memory($maxmem) }, "Doubled memory to $maxmem"); + lives_ok(sub {$dom->create() }, "started domain"); + + $currmem = int(xpath($dom, "number(/domain/currentMemory)")); + is($currmem, $maxmem, "XML description has memory set to $maxmem"); + + my $snap2xml = <<EOF; +<domainsnapshot> + <name>Snapshot2</name> + <description>Second snapshot</description> +</domainsnapshot> +EOF + ok_domain_snapshot( sub { $snap2 = $dom->create_snapshot($snap2xml) }, "create second snapshot $snap2xml"); + $snapshots++; + is($dom->num_of_snapshots(), $snapshots, "We have $snapshots snapshots"); + + # Reverting to the same vm configuration is o.k. + lives_ok( sub { $snap2->revert_to() }, "Reverting to snapshot snapshot 2"); + + # Reverting to another vm configuration would involve killing qemu so this + # is rejected: + ok_error( sub { $snap1->revert_to() }, "Reverting to changed vm configuration not supported", Sys::Virt::Error::ERR_CONFIG_UNSUPPORTED); + + # Reverting to another vm configiguration while shut off is o.k.: + lives_ok( sub { $dom->destroy() }, "Domain destroyed"); + lives_ok( sub { $snap1->revert_to() }, "Reverting to snapshot snapshot 1 with vm shutoff"); + $currmem = int(xpath($dom, "number(/domain/currentMemory)")); + is($currmem, $minmem, "Memory is $minmem"); + + is($dom->num_of_snapshots(), $snapshots, "We have $snapshots snapshots"); + + $snap1->delete(); + $snapshots--; + is($dom->num_of_snapshots(), $snapshots, "We have $snapshots snapshots"); + + $snap2->delete(); + $snapshots--; + is($dom->num_of_snapshots(), $snapshots, "We have $snapshots snapshots"); + is($snapshots, 0, "All snapshots are gone"); +} -- 1.7.6.3

2 2

[libvirt] [PATCH] build: ship helper scripts
by Eric Blake 11 Oct '11

11 Oct '11

Otherwise, 'make rpm' fails with: GEN libvirt_qemu.def make[2]: *** No rule to make target `dtrace2systemtap.pl', needed by `libvirt_probes.stp'. Stop. * src/Makefile.am (EXTRA_DIST): Add recent script additions. --- Pushing under the build-breaker rule. src/Makefile.am | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 6451ad3..dcb4d33 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -1342,9 +1342,11 @@ noinst_LTLIBRARIES += \ libvirt-net-rpc-client.la EXTRA_DIST += \ - rpc/virnetprotocol.x \ - rpc/gendispatch.pl \ - rpc/genprotocol.pl + rpc/dtrace2systemtap.pl \ + rpc/gendispatch.pl \ + rpc/genprotocol.pl \ + rpc/gensystemtap.pl \ + rpc/virnetprotocol.x VIR_NET_RPC_GENERATED = \ $(srcdir)/rpc/virnetprotocol.h \ -- 1.7.4.4

1 1

[libvirt] [PATCH] disable xenlight for non-Xen platforms
by Dan Horák 11 Oct '11

11 Oct '11

Hi, when building libvirt in Fedora/s390x I've found that xenlight needs to be explicitly disabled in the spec file. Configure properly sets the library as non-existent, but the %files section still wants to package the 3 /var/*/libvirt/libxl directories. See also https://bugzilla.redhat.com/show_bug.cgi?id=745020 With regards, Dan

2 1