October 2009 - Devel - libvirt List Archives

[libvirt] [PATCH] Fix QEMU restore from file in raw format
by Daniel P. Berrange 13 Oct '09

13 Oct '09

The logic for running the decompression programs was broken in commit f238709304f9f6c57204cdd943e542cbae38fa5f, so that for non-raw formats the decompression program was never run, and for raw formats, it tried to exec an argv[] with initial NULL in the program name. * src/qemu/qemu_driver.c: Fix logic in runing decompression program --- src/qemu/qemu_driver.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 1e23a9b..556984a 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -3781,9 +3781,8 @@ static int qemudDomainRestore(virConnectPtr conn, goto cleanup; } - if (header.compressed != QEMUD_SAVE_FORMAT_RAW) + if (header.compressed != QEMUD_SAVE_FORMAT_RAW) { intermediate_argv[0] = prog; - else { intermediatefd = fd; fd = -1; if (virExec(conn, intermediate_argv, NULL, NULL, -- 1.6.2.5

3 2

[libvirt] [PATCH 0/9] Support disk-hotremove and controller hotplugging
by Wolfgang Mauerer 13 Oct '09

13 Oct '09

Hi, this patch reworks libvirt's disk-hotadd support and introduces support for disk controller hotplugging and disk-hotremove (see http://thread.gmane.org/gmane.comp.emulators.libvirt/15860 for more details). Currently, it targets only qemu and also requires some additions to qemu that have only recently been submitted, which is why this is cross-posted to qemu-devel. Hopefully the lack of support in qemu does not prevent the community from reviewing the code, though ;-) Notice that no documentation and testcases are included yet; I'll supply those once the interface is fixed and possible issues are settled. Thanks for your comments, Wolfgang docs/schemas/domain.rng | 145 ++++++++++---- src/domain_conf.c | 208 ++++++++++++++++++++- src/domain_conf.h | 39 ++++- src/libvirt_private.syms | 4 +- src/qemu_driver.c | 479 +++++++++++++++++++++++++++++++++++++++++++--- 5 files changed, 802 insertions(+), 73 deletions(-) -- Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux

5 26

[libvirt] [PATCH] Fix virFileReadLimFD/virFileReadAll to handle EINTR
by Daniel P. Berrange 13 Oct '09

13 Oct '09

The fread_file_lim() function uses fread() but never handles EINTR results, causing unexpected failures when reading QEMU help arg info. It was unneccessarily using FILE * instead of plain UNIX file handles, which prevented use of saferead() * src/util/util.c: Switch fread_file_lim over to use saferead instead of fread, remove FILE * use, and rename --- src/util/util.c | 45 ++++++++++++--------------------------------- 1 files changed, 12 insertions(+), 33 deletions(-) diff --git a/src/util/util.c b/src/util/util.c index e5135fc..98f8a14 100644 --- a/src/util/util.c +++ b/src/util/util.c @@ -898,7 +898,7 @@ virExec(virConnectPtr conn, number of bytes. If the length of the input is <= max_len, and upon error while reading that data, it works just like fread_file. */ static char * -fread_file_lim (FILE *stream, size_t max_len, size_t *length) +saferead_lim (int fd, size_t max_len, size_t *length) { char *buf = NULL; size_t alloc = 0; @@ -906,8 +906,8 @@ fread_file_lim (FILE *stream, size_t max_len, size_t *length) int save_errno; for (;;) { - size_t count; - size_t requested; + int count; + int requested; if (size + BUFSIZ + 1 > alloc) { alloc += alloc / 2; @@ -923,12 +923,12 @@ fread_file_lim (FILE *stream, size_t max_len, size_t *length) /* Ensure that (size + requested <= max_len); */ requested = MIN (size < max_len ? max_len - size : 0, alloc - size - 1); - count = fread (buf + size, 1, requested, stream); + count = saferead (fd, buf + size, requested); size += count; if (count != requested || requested == 0) { save_errno = errno; - if (ferror (stream)) + if (count < 0) break; buf[size] = '\0'; *length = size; @@ -941,12 +941,12 @@ fread_file_lim (FILE *stream, size_t max_len, size_t *length) return NULL; } -/* A wrapper around fread_file_lim that maps a failure due to +/* A wrapper around saferead_lim that maps a failure due to exceeding the maximum size limitation to EOVERFLOW. */ -static int virFileReadLimFP(FILE *fp, int maxlen, char **buf) +int virFileReadLimFD(int fd, int maxlen, char **buf) { size_t len; - char *s = fread_file_lim (fp, maxlen+1, &len); + char *s = saferead_lim (fd, maxlen+1, &len); if (s == NULL) return -1; if (len > maxlen || (int)len != len) { @@ -960,37 +960,16 @@ static int virFileReadLimFP(FILE *fp, int maxlen, char **buf) return len; } -/* Like virFileReadLimFP, but use a file descriptor rather than a FILE*. */ -int virFileReadLimFD(int fd_arg, int maxlen, char **buf) -{ - int fd = dup (fd_arg); - if (fd >= 0) { - FILE *fp = fdopen (fd, "r"); - if (fp) { - int len = virFileReadLimFP (fp, maxlen, buf); - int saved_errno = errno; - fclose (fp); - errno = saved_errno; - return len; - } else { - int saved_errno = errno; - close (fd); - errno = saved_errno; - } - } - return -1; -} - int virFileReadAll(const char *path, int maxlen, char **buf) { - FILE *fh = fopen(path, "r"); - if (fh == NULL) { + int fd = open(path, O_RDONLY); + if (fd < 0) { virReportSystemError(NULL, errno, _("Failed to open file '%s'"), path); return -1; } - int len = virFileReadLimFP (fh, maxlen, buf); - fclose(fh); + int len = virFileReadLimFD(fd, maxlen, buf); + close(fd); if (len < 0) { virReportSystemError(NULL, errno, _("Failed to read file '%s'"), path); return -1; -- 1.6.2.5

2 2

[libvirt] Tentative schedule for next release
by Daniel Veillard 09 Oct '09

09 Oct '09

I'm suggesting to try to push 0.7.2 around Fri 16 Oct, we already have a number of cleanup patches in and serious improvements for ESX driver. If we go with that plan I think patches and big changes should be pushed in the tree by Fri 9, this includes among other things the AppArmor security driver. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/

2 3

[libvirt] dynamic scsi disk attach seems to be broken in qemu(-kvm)-0.11, libvirt-0.7.1
by Dustin Kirkland 09 Oct '09

09 Oct '09

We're experiencing what we believe to be a regression in qemu-kvm-0.11 and libvirt-0.7.0, from previous versions of kvm-84 and libvirt-0.6.1: Dynamically attaching sdb (a second scsi disk) fails. (Note that I have tested this against libvirt-0.7.1 built from source as well--same results.) Here are some instructions to reproduce this issue: Create some small auxiliary storage to dynamically add to a running VM. (host) $ dd if=/dev/zero of=/tmp/foo bs=1M count=64 Boot a vm in under qemu-kvm-0.11.0 like this: (host) $ kvm -drive file=server.img,if=scsi,boot=on -boot c To reproduce the problem, you must use if=scsi. This problem manifests itself when subsequent scsi storage devices are added. If the device is the first scsi device on the system, then it would succeed. It's the 2nd scsi device that causes the problem. Once the system is booted, login and in the vm, load this module: (vm) $ sudo modprobe acpiphp Check that the acpiphp slots are loaded in dmesg. And note that there is only one /dev/sd[a-z] device. Now, drop to the qemu console with ctrl-alt-2, and add the storage: (qemu) pci_add auto storage file=/tmp/foo,if=scsi OK domain 0, bus 0, slot 6, function 0 Switch back to the vm linux shell with ctrl-alt-1, and look at the dmesg output. (vm) $ dmesg | tail -n 12 [ 44.033397] pci 0000:00:06.0: reg 10 io port: [0x00-0xff] [ 44.033443] pci 0000:00:06.0: reg 14 32bit mmio: [0x000000-0x0003ff] [ 44.033486] pci 0000:00:06.0: reg 18 32bit mmio: [0x000000-0x001fff] [ 44.033899] pci 0000:00:02.0: BAR 6: bogus alignment [0x0-0x0] flags 0x2 [ 44.033975] decode_hpp: Could not get hotplug parameters. Use defaults [ 44.042277] sym53c8xx 0000:00:06.0: enabling device (0000 -> 0003) [ 44.043230] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 11 [ 44.043247] sym53c8xx 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 11 (level, high) -> IRQ 11 [ 44.045237] sym1: <895a> rev 0x0 at pci 0000:00:06.0 irq 11 [ 44.047586] sym1: No NVRAM, ID 7, Fast-40, LVD, parity checking [ 44.055399] sym1: SCSI BUS has been reset. [ 44.063329] scsi3 : sym-2.2.3 More importantly, note that no /dev/sd[b-z] device shows up. If you now drop to the qemu console with ctrl-alt-2, and do this: (qemu) pci_add auto storage file=/tmp/foo,if=virtio OK domain 0, bus 0, slot 7, function 0 Going back to the vm with ctrl-alt-1, you can now see a new /dev/vda device registered. Now, all of that said, it is actually possible to hot-add a second scsi device. However, as far as I can tell, this method is not yet supported by libvirt. It looks to me that with modern qemu, you have to do it this way: Drop to a qemu console with ctrl-alt-2. Get the address of the current scsi bus: (qemu) info pci Look for "SCSI Controller". In my case, it's on Bus 0, device 4, function 0 Now instead of pci_add, use drive_add (qemu) drive_add 0:4 file=/tmp/foo,if=scsi OK bus 0, unit 1 This is not ideal, however, as I tried re-scan the scsi bus with rescan-scsi-bus.sh (from scsitools) without luck. It did not pick up the new sdb device. However, I did reboot the vm, and voila, I now have /dev/sdb. I would like some advice on how to proceed with this bug, and where the solution lies...in qemu or in libvirt. Ultimately, I would like the behavior we had in our previous release with kvm-84 and libvirt-0.6.1, where we could dynamically add scsi devices without a problem, using: pci_add 1 storage file=/tmp/foo,if=scsi Can anyone else reproduce this? Is this considered a regression by anyone else? Where should I look to solve this, in libvirt, or in qemu? -- :-Dustin

3 4

[libvirt] Help needed: simple python api to obtain events
by Dan Kenigsberg 09 Oct '09

09 Oct '09

Would someone help me have a shrink-wrapped solution for obtaining libvirt events in python? I'm looking for a python module with no API calls (or maybe just a few). I'd like to register a callback functions with virConnect.domainEventRegister(), and want them to be called in the context of a specialized (automatically-created?) thread. I don't mind if the module has two API calls instead: register() and startPollThread(). I learned that it takes me far too long to do it on my own, and that just modifying events-python.py (attached) reaches miserable outcomes ("libvir: Domain error : invalid domain pointer in virDomainFree", and deadlocked python: #0 0x00000037fe40d174 in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00000037fe408aca in _L_lock_1034 () from /lib64/libpthread.so.0 #2 0x00000037fe40898c in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x0000003cce04db84 in remoteDomainEventFired () #4 0x00002b4cc96e1612 in libvirt_virEventInvokeHandleCallback () ) Help most welcome, Dan.

4 10

[libvirt] [PATCH 0/9] Support new peer-2-peer migration mode & public API
by Daniel P. Berrange 09 Oct '09

09 Oct '09

This series is an update of http://www.redhat.com/archives/libvir-list/2009-September/msg00540.html There isn't as much functional change here as you might presume from the number of patches. Since Chris' tunnelled migration code is added, I thought it better to do alot of small refactoring steps rather than munge it all in one patch. One particular notable change since last time is that the new virDomainMigrateToURI method does *not* mandate use of the new "VIR_MIGRATE_PEER2PEER" flag anymore. I will now explain why... Traditionally migration has been a 3 step process 1. client -> dest (prepare) 2. client -> source (perform) 3. client -> dest (finish) This is why virDomainMigrate requires the extra 'dconn' parameter from the client app With the tunnelled migration / peer-to-peer migration, the extra 'dconn' parameter is no longer required from the client, because the source libvirt driver directly opens a connection to the destination libvirtd daemon on the remote host. ie libvirtd is talking to another libvirtd peer-2-peer. In considering the implementation of the PEER_TO_PEER flag for the Xen and VMWare drivers though I realized that even this is not technically neccessary, and in fact detrimental to use of libvirt for migrating with these drivers. Both Xen and VMWare have their own management daemon which is perfectly capable of handling the entire process without any need for libvirtd to be involved on the destination host. libvirt need only initiate the migration op on the source host. If you look at the impl of Prepare/Finish ops in the Xen driver this should be obvious, since they are pretty much no-ops Thus we in fact have 3 possible migration processes * Traditional libvirt client managed client -> dest (prepare) client -> source (perform) client -> dest (finish) * New peer-to-peer migration (on which tunnelled mig is built) client -> source (perform) source -> dest (open) source -> dest (prepareTunnel) source -> dest (streamSend) source -> dest (finish) source -> dest (close) * New direct migration (hypervisor managed) client -> source (perform) source -> dest (whatever the HV's native migration does) In terms of libvirt public APIs this works out as follows * Traditional libvirt client managed A hypervisor specific URI style, passed to virDomainMigrate() with no flags set * New peer-to-peer migration (on which tunnelled mig is built) The standard libvirt URI style, passed to virDomainMigrate() or virDomainMigrateToURI() with VIR_DOMAIN_MIGRATE_PEER2PEER flag set. The virDomainMigrateToURI() method is better since it avoids the unneccessary burden on the client of connecting to the remote libvirtd. * New direct migration (hypervisor managed) A hypervisor specific URI style, passed to virDomainMigrateToURI() with no flags set The QEMU driver can't support the latter, since it has no native management daemon - it requires libvirtd's help. Xen and VMWare, and probably VirtualBox can support this just fine. This is good for flexibility of usage in terms of authentication too, since each of these 3 modes has different characteristics * Traditional libvirt client managed client -> source libvirtd auth client -> dest libvirtd auth possible hypervisor -> hypervisor auth * New peer-to-peer migration, without tunnelling client -> source libvirtd auth source -> dest libvirtd auth possible hypervisor -> hypervisor auth * New peer-to-peer migration, without tunnelling client -> source libvirtd auth source -> dest libvirtd auth * New direct migration (hypervisor managed) client -> source libvirtd auth possible hypervisor -> hypervisor auth NB, 'client -> source libvirtd auth' is essentially no-op if initiating migration from the source host directly in all these cases. With with the combination of the two virDomainMigrate and virDomainMigrateToURI methods, and the VIR_MIGRATE_PEER2PEER flag, I believe we're well placed to cover all the main deployment/auth scenarios for migration in all hypervisors without imposing extra auth burden ourselves as we did in the past Further things that need to be done though: - Xen could easily support PEER2PEER and TUNNELLED flags - Add a VIR_MIGRATE_SECURE flag, to indicate that the app only wants the migration to be performed if the HV can guarentee the data channel is secure. - Document the scenarios I just outlined and give some mre guidance to app developers/administrators about the tradeoff between each scenario Regards, Daniel

3 15

[libvirt] [PATCH] Remove some auto-generated files
by Daniel P. Berrange 09 Oct '09

09 Oct '09

The original mail i tried to send was far too large, so here's it edited to remove the boring bits. Daniel > From: "Daniel P. Berrange" <berrange(a)redhat.com> > To: libvir-list(a)redhat.com > Cc: "Daniel P. Berrange" <berrange(a)redhat.com> > Subject: [PATCH] Remove some auto-generated files > Date: Thu, 8 Oct 2009 17:03:07 +0100 > > Removes some auto-generated files still under version control. > It also moves the rule for generating NEWS into the Makefile.am > that's in the same directory as the output file to avoid confusion > > * docs/libvirt-api.xml, docs/libvirt-refs.xml, NEWS: Remove > auto-generated files from source control > * Makefile.am: Add rule for generating NEWS file > * docs/Makefile.am: Remove rule for generating NEWS file > --- > Makefile.am | 9 + > NEWS | 1620 ---------- > docs/Makefile.am | 10 +- > docs/libvirt-api.xml | 2986 ------------------ > docs/libvirt-refs.xml | 8313 ------------------------------------------------- > 5 files changed, 10 insertions(+), 12928 deletions(-) > delete mode 100644 NEWS > delete mode 100644 docs/libvirt-api.xml > delete mode 100644 docs/libvirt-refs.xml > > diff --git a/Makefile.am b/Makefile.am > index add7e2a..977ad0c 100644 > --- a/Makefile.am > +++ b/Makefile.am > @@ -31,6 +31,15 @@ EXTRA_DIST = \ > pkgconfigdir = $(libdir)/pkgconfig > pkgconfig_DATA = libvirt.pc > > +NEWS: $(top_srcdir)/docs/news.xsl $(top_srcdir)/docs/news.html.in > + -@(if [ -x $(XSLTPROC) ] ; then \ > + $(XSLTPROC) --nonet $(top_srcdir)/docs/news.xsl \ > + $(top_srcdir)/docs/news.html.in \ > + | perl -0777 -pe 's/\n\n+$$/\n/' \ > + | perl -pe 's/[ \t]+$$//' \ > + > $@-t && mv $@-t $@ ; fi ); > + > + > rpm: clean > @(unset CDPATH ; $(MAKE) dist && rpmbuild -ta $(distdir).tar.gz) > > diff --git a/docs/Makefile.am b/docs/Makefile.am > index 8832b72..0c206c3 100644 > --- a/docs/Makefile.am > +++ b/docs/Makefile.am > @@ -80,7 +80,7 @@ EXTRA_DIST= \ > > CLEANFILES = $(dot_html) $(apihtml) $(devhelphtml) > > -all: web $(top_builddir)/NEWS > +all: web > > api: libvirt-api.xml libvirt-refs.xml > > @@ -133,14 +133,6 @@ libvirt-api.xml libvirt-refs.xml: apibuild.py \ > $(srcdir)/../src/libvirt.c $(srcdir)/../src/util/virterror.c > -(./apibuild.py) > > -$(top_builddir)/NEWS: $(top_srcdir)/docs/news.xsl $(top_srcdir)/docs/news.html.in > - -@(if [ -x $(XSLTPROC) ] ; then \ > - $(XSLTPROC) --nonet $(top_srcdir)/docs/news.xsl \ > - $(top_srcdir)/docs/news.html.in \ > - | perl -0777 -pe 's/\n\n+$$/\n/' \ > - | perl -pe 's/[ \t]+$$//' \ > - > $@-t && mv $@-t $@ ; fi ); > - > clean-local: > rm -f *~ *.bak *.hierarchy *.signals *-unused.txt > ----- End forwarded message ----- -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

2 1

[libvirt] Xen: Domain.migrate() causes invalid listDomains() output
by Thomas Treutner 09 Oct '09

09 Oct '09

Hi, I'm using Xen 3.2 (Debian Lenny) and libvirt 0.7.1. When I use Domain.migrate() from libvirt-java 0.3.0, a following Connect.listDomains() gives wrong DomainIDs (they seem to be the ones before the migration) and therefore DomainLookupByID() fails because of a nonexistent ID. Restarting libvirtd and/or xend doesn't help. xm list etc. output is still correct, but anything that uses libvirt (like virt-top) is having troubles. The migration itself is working without glitches, the domains are up&running and responsive. But at this point, libvirt is completely mistaken about the running domains and their IDs. There are some quite similar bug reports and mails here, but they are rather old. Shall I provide more debug output? tia&kr, thomas

2 4

[libvirt] libvirt/kvm config for routed networking
by William Wagner 08 Oct '09

08 Oct '09

Hello, I hope this is the right list I should post to as my question is not development related. I am trying to setup a kvm/libvirt VM on my host (Ubuntu jaunty). My host has a public static IP and my VM also has a public static IP. Unfortunately I can not use bridged networking as the hosting provider has configured their switch to only accept packets from the MAC address of the host. I want to be able to setup my VM so it has the public static IP and it appears to be directly connected to the net. I believe I can do this with routed networking. I have created a new routed network: <network> <name>routed-net</name> <bridge name="routed%d" /> <forward mode="route" dev="eth0"/> <ip address="10.255.255.2" netmask="255.255.255.255"> </ip> </network> and in my vm's config I have: <interface type='user'> <source network='routed-net'/> <mac address='54:52:00:47:a8:38'/> <model type='virtio'/> </interface> Will this mean that the VM is placed on the routed network? Then I just need to add suitable routing rules on the host and everything should work? Is there a way to get libvirt to add the rules automatically when the VM starts. I have previously used Xen where you are able to specify what the IP address of the VM is and entries are automatically added to iptables. Is there similar syntax for libvirt and if so what is it? If not how do you recommend adding the routing rules? Thanks. Will. -- ------------------------------------------------------------------------ Will Wagner will_wagner(a)carallon.com Development Manager Office Tel: +44 (0)20 7371 2032 Carallon Ltd, Studio G20, Shepherds Building, Rockley Rd, London W14 0DA ------------------------------------------------------------------------

2 1