- Devel - libvirt List Archives

[Libvir] Release of libvirt-0.1.4
by Daniel Veillard 16 Aug '06

16 Aug '06

Okay, there was a lot of new stuff which accumulated in the last month, it was urgent to push a new release. Available at ftp://libvirt.org/libvirt/ and http://libvirt.org/sources/ * bug fixes: - spec file fix (Mark McLoughlin) - error report problem (with Hugh Brock) - long integer in Python bindings (with Daniel Berrange) - XML generation bug for CDRom (Daniel Berrange) - bug when using number() XPath function (Mark McLoughlin) - fix python detection code - remove duplicate initialization errors (Daniel Berrange) * improvements: - UUID in XML description (Peter Vetere) - proxy code cleanup - virtual CPU and affinity support + virsh support (Michel Ponceau, Philippe Berthault, Daniel Berrange) - port and tty informations for console in XML (Daniel Berrange) - added XML dump to driver and proxy support (Daniel Berrange) - extention of boot options with support for floppy and cdrom (Daniel Berrange) - features block in XML to report/ask PAE, ACPI, APIC for HVM domains (Daniel Berrange) - fail side-effect operations when using read-only connection - large improvements to test driver (Daniel Berrange) * documentation: - spelling (Daniel Berrange) - test driver examples. Thanks to everybody who helped for this release ! Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

1 0

[Libvir] Re-factor sexpr_uuid method
by Daniel P. Berrange 16 Aug '06

16 Aug '06

The sexpr_uuid method is used to convert from the printable representation of a UUID into the binary format (unsigned char[16]). This patch refactors the main body of the method so that it is usable by other non-XenD driver backends for parsing UUIDs. This new re-factored method will be used by a patch I'll shortly submit for the test driver backend. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 2

[Libvir] Record an error when refusing a read-only operation
by Daniel P. Berrange 16 Aug '06

16 Aug '06

The libvirt.c front-end can be compiled in 'pedantic' mode in which case it explicitly refuses operations with side-effects in read-only connections. When doing so, however, it was not recording any error message so feedback to the user was misleading / obsure. This patch adds an extra error code VIR_ERR_READ_ONLY which is used when an operation is denied due to a read only connection. This patch also passes 'VIR_DRV_OPEN_QUIET' to the 'open' method so tell the XenD driver not complain if it fails to open a HV connection. This is because it was complaining whenever trying to open a test:///default connection. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 2

[Libvir] Major extensio of test backend driver functionality
by Daniel P. Berrange 16 Aug '06

16 Aug '06

Ok, here comes the huge patch for today - a major extension of the test driver backend... As well as the interal hardcoded 'test:///default' hypervisor config you can now provide a path to an arbitrary XML file defining your HV and its domains. An example file to define a moster 128x6GHz CPU box with 128 GB of RAM and two guest domains looks like: <node> <domain file="testdomfv0.xml"/> <domain file="testdomfc4.xml"/> <cpu> <mhz>6000</mhz> <model>i986</model> <active>50</active> <nodes>4</nodes> <sockets>4</sockets> <cores>4</cores> <threads>2</threads> </cpu> <memory>137438953472</memory> </node> The individual domains 'testdomfv0.xml' and 'testdomfc4.xml' are defined using the regular libvirt XML format. The only exception is that the 'type' attribute on the top level '<domain>' tag should be 'test' instead of 'xen' So for example: $ virsh --connect test:///home/berrange/src/xen/libvirt/docs/testnode.xml virsh > nodeinfo CPU model: i986 CPU(s): 50 CPU frequency: 6000 MHz CPU socket(s): 4 Core(s) per socket: 4 Thread(s) per core: 2 NUMA cell(s): 4 Memory size: 137438953472 kB virsh > list Id Name State ---------------------------------- 0 fv0 running 1 fc4 running virsh > dominfo fc4 Id: 1 Name: fc4 UUID: ef861801-45b9-11cb-88e3-afbfe5370493 OS Type: linux State: running CPU(s): 1 CPU time: 1155740835.5s Max memory: 131072 kB Used memory: 131072 kB virsh > Next up, I've written implementations of the set memory, set max memory, set vcpus, dump xml, create linux APIs for the test driver. THe only major areas of functionality lacking in the test driver now are handling of disk and network devices when creating domains, and the VCPU pinning methods. This test backend makes testing the virt-manager application much more reliable - although we still do need some level of testing against a real Xen backend (for example domain creation). Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 2

[Libvir] Add 'setvcpus', 'setmem', 'setmaxmem' to virsh
by Daniel P. Berrange 16 Aug '06

16 Aug '06

The attached patch adds three new commands to virsh: setmem change memory allocation setmaxmem change maximum memory limit setvcpus change number of virtual CPUs It also ensures then when using the 'test' hypervisor driver, it always opens a read-write connection even as non-root. The test hypervisor does not have any distinction between privileged / unprivileged users, so this test for 'uuid == 0' was bogus. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 1

[Libvir] Flags for CPU / system 'features'
by Daniel P. Berrange 15 Aug '06

15 Aug '06

In the XenD config there are various flags which allow CPU / system features to be enabled / disabled for the guest domain. For example, HVM domains can be created with, or without, PAE, ACPI, APIC. The XenD sexpr contains simple boolean flags for these features within the (image (hvm ....)) block. IMHO, although these 3 particular features may be only be relevant for HVM, it does not make sense to restrict the expression of features to only HVM domains. Thus I propose a top level '<features>' block within the '<domain>' XML doc. This block would contain tags named after each feature to be enabled. Presence of the named tag enables the feature, ommision disables it (thus all features are disabled by default - matching current behaviour). As an example, to enable ACPI, PAE & APIC: <domain type='xen' id='55'> <name>too</name> <uuid>b5d70dd275cdaca517769660b059d8bc</uuid> <os> <type>hvm</type> <loader>/usr/lib/xen/boot/hvmloader</loader> <boot dev='hd'/> </os> <memory>409600</memory> <vcpu>1</vcpu> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <features> <acpi/> <apic/> <pae/> </features> <devices> ...snipped from example.. </devices> </domain> More flags could trivially be added over time. The attached patch implements this proposal. My only thought, is 'features' a good name for this ? Only other name I came up with was 'capabilities' but this is harder to spell :-) Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 2

[Libvir] Fix handling of HVM boot parameters
by Daniel P. Berrange 15 Aug '06

15 Aug '06

Jeremy mentioned that it looked like libvirt wasn't able to create an HVM domain configured to boot off cdrom, so I took a closer look at the code and indeed the code dealing with <boot> section was both incomplete, and just plain broken. Incomplete in so much as it never included details of the ISO file backing the CDROM, and broken in so much as it was doing string comparisons against the wrong variables. Digging further found lots more work relating to creation of HVM domains so I've had a go at writing a patch to resolve matters. * Parsing of UUID from SEXPR assumed that the UUID fetched would always have four '-' in the usual places. Well, when you run 'dumpxml' from virsh the <uuid> element has the UUID encoded without any '-' chars. If you feed this back into 'virsh create' and then once again run 'dumpxml' parsing will fail & libvirt throws errors. So this patch allows for parsing of UUID's without '-'. * The XML document size was limited to 1k - we just 'malloc(1000)'. While this was enough for common cases, if someone creates lots of disk or network devices this would overflow and libvirt return an error. So I increased it to 4k which ought to be enough for forseeable future - in any case my previous patch to fetch XML via the proxy is limited to 4k in size too. Now onto the fine details... When converting SEXPR into XML current code is doing the following: (boot a) -> <boot dev='/dev/fd0'/> (boot c) -> <boot dev='hda'/> (boot d) -> <boot dev='/dev/cdrom'/> This is rather inconsistent - the 'hda' is intended to map to an entry in <devices> block. The '/dev/fd0' and '/dev/cdrom' entries did not map to anything. Meanwhile, when converting from XML to SXEXPR it is doing the following <boot dev='hda'/> -> (boot a) <boot dev='hdd'/> -> (boot d) <boot dev='*'/> -> (boot c) Obviously this sucks because these processes should be matching each other. Secondly, the (image (hvm....)) SEXPR has three entries for defining the ISO / disk image file backing the CDROM / Floppy devices. These were just being ignored, rather than turned into <disk> entries within the <devices> block. Similarly there was no way to express a <disk> entry for CDROM/Floppy in the XML when creating a domain. The upshot of all this is that although the last release of libvirt included HVM support it was basically unusable for domain creation unless you were using HD to boot. The XML returned was also incorrect. Now the good news. Since it was sooo broken, we can fix without worrying about XML compatability since there is no way any application could be relying on it in its current state. Now before I discuss the solution, one final point. The Xen IOMMU model allows specifiying the boot device in terms of 'a', 'c', 'd' which has the meaning: a - first connected floppy device c - first connected harddrive d - first connected cdrom In particular it does *not* allow for expressing boot device in terms of an explicit disk device - you can't say boot off 'hdb' or 'hdc'. So the current libvirt code which tries to fake such semantics is doomed to failure. This isn't really too bad IMHO - VMware has these same semantics, so does QEMU and so do normal bare metal BIOS This in the patch I have attached I have implemented the following mapping: (boot a) <-> <boot dev='fd'/> (boot c) <-> <boot dev='hd'/> (boot d) <-> <boot dev='cdrom'/> The other part of the patch is to deal with definition of the floppy and cdrom device backing files. For this I have done the following: (image (hvm (fda /root/diskboot.img))) <devices> <disk type='file'> <source file='/root/diskboot.img'/> <target dev='fda'/> </disk> </devices> And similar for 'fdb'. Then for cdroms: (image (hvm (cdrom /root/image.iso))) <devices> <disk type='file'> <source file='/root/image.iso'/> <target dev='cdrom'/> </disk> </devices> The patch has a little bit of logic such that when converting the <devices> block backinto an SEXPR it filters out the disk entries with a dev of 'fda', 'fdb' and 'cdrom' since they need to end up in a different part of the SEXPR. Finally, although PV guests automatically get a serial console created for them (and i don't think you can turn it off), HVM guests need to have a serial console explicitly requested. Since I recently added support in the DumpXML method for including details of serial console like this: <devices> <console tty="/dev/pts/5"/> </devices> I decided to leverage this same structure when creating HVM domains. So if you have: <devices> <console/> </devices> Then the SEXPR sent to XenD will include (image (hvm (serial pty))) Which enables allocation of PseudoTTY for the HVM's serial console. I have tested that with this patch I can successfully create a HVM domain which boots off a floppy, harddrive or cdrom. Furthermore if you then dump the XML of this domain,the XML you get back will match the XML you fed in (with the obvious exception of domain ID, and the Pseudo TTY path). If you have been monitoring xen-devel mailing lists you'll be aware that in 3.0.3 the way CDROM devices are configured is changing: http://lists.xensource.com/archives/html/xen-devel/2006-08/msg00369.html Although the patch attached does not support the config outlined in that mail, I'm pretty confident that a small incremental patch will be able to support it without breaking compatability with the changes I've outlined in this mail. The only tricky bit will be that we need to detect whether libvirt is running against a 3.0.2 or 3.0.3 version of XenD to decide how to convert XML -> SEXPR & vica verca. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

4 16

[Libvir] Guest Identification
by Juan Walker 11 Aug '06

11 Aug '06

Hi, I'm trying to find a good way to correlate a domain to the machine it's running on. Can I use libvir or some other method on a domU to get some information that uniquely identifies it? Perhaps something that I can get from the XML running on dom0? I'm thinking of using the MAC address because I imagine that those things should be unique at least per hypervisor, but I'm not sure if that would work across physical boxes. The solution would be to use libvir to get the list of domains and then get the xml which includes the mac address. Then every domain on my network would hit a server and report its mac address, and then I could make a tree that links the dom0's to the domU's. I'm wondering if there's a better way to do this. Any suggestions would be greatly appreciated. Thanks for your help.

3 4

[Libvir] Patch to fix duplicate pause/destroy/shutdown calls
by Daniel P. Berrange 10 Aug '06

10 Aug '06

I spent a while chasing an issue where running 'virsh destroy foo' on an HVM domain would result in an error being returned by the HV ioctl. What I eventually discovered is that the code was doing a destroy op via XenD and then also doing it via the hypervisor backend. It didn't typically show as a problem for PV domains because it seems they took longer to destroy - so were still around by the time the HV destroy was was called. I tracked this issue down to the main dispatch methods in src/libvirt.c which would indeed call both the XenD and HV driver backends for pause destroy & shutdown methods. The patch attached simply returns as soon as one of the driver methods returns a success return code. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

1 0

[Libvir] Add port number to <graphics> tag
by Daniel P. Berrange 09 Aug '06

09 Aug '06

We recently added the '<graphics>' tag to the XML description for HVM machines. Any apps using this info though would have to hardcode the assumption that Xen's QEMU listens on Domain-ID + 5900. Hardcoding the port numbers in the first place is already a really questionable decision, so we should avoid that propagating out to application code. Thus, the attached patch changes: <graphics type="vnc"/> So that it now looks like <graphics type="vnc" port="5905"/> The hardcoded 'Domain-ID + 5900' craziness is now at least isolated in libvirt, so provided apps take the port number from the XML they will be immune from changes in port numbering scheme. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 15