- Devel - libvirt List Archives

[Libvir] Adding test suite for sexpr<->xml conversions
by Daniel P. Berrange 24 Aug '06

24 Aug '06

Quite a critical bit of the libvirt code is that which converts between SEXPR and XML, and vica-verca. I've broken this code several times when making changes, so it is way overdue to get some unit test coverage in this area. The attached patch adds such coverage - defining two tests xml2sexprtest and sexpr2xmltest. The tests are pretty simple, in the tests/ directory I just have a bunch of xml & sexpr files - one is the fixed input, the other is the expected output. The test just runs the conversion and compares the actual output with the expected output. Currently I've added two example data files - one for paravirt & one for fully-virt. With this patch you can run tests: $ make check make check-TESTS make[1]: Entering directory `/home/berrange/src/xen/libvirt/tests' XML-2-SEXPR PV config ... OK XML-2-SEXPR FV config ... OK PASS: xml2sexprtest SEXPR-2-XML PV config ... OK SEXPR-2-XML FV config ... OK PASS: sexpr2xmltest ================== All 2 tests passed ================== make[1]: Leaving directory `/home/berrange/src/xen/libvirt/tests' I'm shortly about to start on updating the code to deal with new CDROM configuration format for HVM guests in Xen 3.0.3, and so intend to add further test files during this process (and of course make sure I don't break the existing ones :-) Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 8

[Libvir] [Libvirt] Proposal to add 3 functions for VBDs (Virtual Block Devices)
by michel.ponceau＠bull.net 24 Aug '06

24 Aug '06

After the new VCPU functions, we also need Virtual Block Device management equivalent to Xen commands: 'xm block-list', 'xm block-attach', 'xm block-detach'. I propose to add in Libvirt the following 3 functions: /** * virDomainGetVbdevs: * @domain: pointer to domain object, or NULL for Domain0 * @info: pointer to an array of virVbdevInfo structures * @maxinfo: number of structures in info array * * Extract information about virtual block devices of domain, store it in info. * * Returns the number of info filled in case of success, -1 in case of failure. */ int virDomainGetVbdevs(virDomainPtr domain, virVbdevInfoPtr info, int maxinfo) /** * virDomainCreateVbdev: * @domain: pointer to domain object, or NULL for Domain0 * @number: virtual device number * @mode: read-only (VIR_VBDMD_RO), or read/write (VIR_VBDMD_RW) * @bkendID: ID of domain hosting the backend device (usually 0 for domain0) * @bkend: pointer to backend device path * * Create a virtual block device. * * Returns 0 in case of success, -1 in case of failure. */ int virDomainCreateVbdev(virDomainPtr domain, unsigned int number, int mode, int bkendID, char *bkend) /** * virDomainDestroyVbdev: * @domain: pointer to domain object, or NULL for Domain0 * @number: virtual device number * * Destroy a virtual block device. * * Returns 0 in case of success, -1 in case of failure. */ int virDomainDestroyVbdev(virDomainPtr domain, unsigned int number) Structure in libvirt.h: /** * virVbdevInfo: information structure for a Virtual Block Device in a domain */ typedef struct _virVbdevInfo virVbdevInfo; struct _virVbdevInfo { unsigned int number; /* virtual device number */ int state; int evtChn; int ringRef; int bkendID; /* ID of domain hosting the backend device */ char bkend[128]; }; typedef virVbdevInfo *virVbdevInfoPtr; /* Virtual Block Device access modes: read-only or read/write */ #define VIR_VBDMD_RO 0 #define VIR_VBDMD_RW 1

4 5

Re: [Libvir] Semantics for ListDomains/ ListDefinedDomains
by Jim Fehlig 24 Aug '06

24 Aug '06

>>> Gareth S Bestor <bestor(a)us.ibm.com> 08/23/06 8:04 PM >>> > At the *very* worst, is there a way you can pipe an arbitrary xm config > file thru xm and have it spit out what it would have done, without actually > doing it? That could be a reliable fallback migration path perhaps. xm create /path/to/config --dryrun Spits out s-expr last time I checked. Jim

1 0

[Libvir] Release of libvirt-0.1.4
by Daniel Veillard 16 Aug '06

16 Aug '06

Okay, there was a lot of new stuff which accumulated in the last month, it was urgent to push a new release. Available at ftp://libvirt.org/libvirt/ and http://libvirt.org/sources/ * bug fixes: - spec file fix (Mark McLoughlin) - error report problem (with Hugh Brock) - long integer in Python bindings (with Daniel Berrange) - XML generation bug for CDRom (Daniel Berrange) - bug when using number() XPath function (Mark McLoughlin) - fix python detection code - remove duplicate initialization errors (Daniel Berrange) * improvements: - UUID in XML description (Peter Vetere) - proxy code cleanup - virtual CPU and affinity support + virsh support (Michel Ponceau, Philippe Berthault, Daniel Berrange) - port and tty informations for console in XML (Daniel Berrange) - added XML dump to driver and proxy support (Daniel Berrange) - extention of boot options with support for floppy and cdrom (Daniel Berrange) - features block in XML to report/ask PAE, ACPI, APIC for HVM domains (Daniel Berrange) - fail side-effect operations when using read-only connection - large improvements to test driver (Daniel Berrange) * documentation: - spelling (Daniel Berrange) - test driver examples. Thanks to everybody who helped for this release ! Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

1 0

[Libvir] Re-factor sexpr_uuid method
by Daniel P. Berrange 16 Aug '06

16 Aug '06

The sexpr_uuid method is used to convert from the printable representation of a UUID into the binary format (unsigned char[16]). This patch refactors the main body of the method so that it is usable by other non-XenD driver backends for parsing UUIDs. This new re-factored method will be used by a patch I'll shortly submit for the test driver backend. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 2

[Libvir] Record an error when refusing a read-only operation
by Daniel P. Berrange 16 Aug '06

16 Aug '06

The libvirt.c front-end can be compiled in 'pedantic' mode in which case it explicitly refuses operations with side-effects in read-only connections. When doing so, however, it was not recording any error message so feedback to the user was misleading / obsure. This patch adds an extra error code VIR_ERR_READ_ONLY which is used when an operation is denied due to a read only connection. This patch also passes 'VIR_DRV_OPEN_QUIET' to the 'open' method so tell the XenD driver not complain if it fails to open a HV connection. This is because it was complaining whenever trying to open a test:///default connection. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 2

[Libvir] Major extensio of test backend driver functionality
by Daniel P. Berrange 16 Aug '06

16 Aug '06

Ok, here comes the huge patch for today - a major extension of the test driver backend... As well as the interal hardcoded 'test:///default' hypervisor config you can now provide a path to an arbitrary XML file defining your HV and its domains. An example file to define a moster 128x6GHz CPU box with 128 GB of RAM and two guest domains looks like: <node> <domain file="testdomfv0.xml"/> <domain file="testdomfc4.xml"/> <cpu> <mhz>6000</mhz> <model>i986</model> <active>50</active> <nodes>4</nodes> <sockets>4</sockets> <cores>4</cores> <threads>2</threads> </cpu> <memory>137438953472</memory> </node> The individual domains 'testdomfv0.xml' and 'testdomfc4.xml' are defined using the regular libvirt XML format. The only exception is that the 'type' attribute on the top level '<domain>' tag should be 'test' instead of 'xen' So for example: $ virsh --connect test:///home/berrange/src/xen/libvirt/docs/testnode.xml virsh > nodeinfo CPU model: i986 CPU(s): 50 CPU frequency: 6000 MHz CPU socket(s): 4 Core(s) per socket: 4 Thread(s) per core: 2 NUMA cell(s): 4 Memory size: 137438953472 kB virsh > list Id Name State ---------------------------------- 0 fv0 running 1 fc4 running virsh > dominfo fc4 Id: 1 Name: fc4 UUID: ef861801-45b9-11cb-88e3-afbfe5370493 OS Type: linux State: running CPU(s): 1 CPU time: 1155740835.5s Max memory: 131072 kB Used memory: 131072 kB virsh > Next up, I've written implementations of the set memory, set max memory, set vcpus, dump xml, create linux APIs for the test driver. THe only major areas of functionality lacking in the test driver now are handling of disk and network devices when creating domains, and the VCPU pinning methods. This test backend makes testing the virt-manager application much more reliable - although we still do need some level of testing against a real Xen backend (for example domain creation). Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 2

[Libvir] Add 'setvcpus', 'setmem', 'setmaxmem' to virsh
by Daniel P. Berrange 16 Aug '06

16 Aug '06

The attached patch adds three new commands to virsh: setmem change memory allocation setmaxmem change maximum memory limit setvcpus change number of virtual CPUs It also ensures then when using the 'test' hypervisor driver, it always opens a read-write connection even as non-root. The test hypervisor does not have any distinction between privileged / unprivileged users, so this test for 'uuid == 0' was bogus. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 1

[Libvir] Flags for CPU / system 'features'
by Daniel P. Berrange 15 Aug '06

15 Aug '06

In the XenD config there are various flags which allow CPU / system features to be enabled / disabled for the guest domain. For example, HVM domains can be created with, or without, PAE, ACPI, APIC. The XenD sexpr contains simple boolean flags for these features within the (image (hvm ....)) block. IMHO, although these 3 particular features may be only be relevant for HVM, it does not make sense to restrict the expression of features to only HVM domains. Thus I propose a top level '<features>' block within the '<domain>' XML doc. This block would contain tags named after each feature to be enabled. Presence of the named tag enables the feature, ommision disables it (thus all features are disabled by default - matching current behaviour). As an example, to enable ACPI, PAE & APIC: <domain type='xen' id='55'> <name>too</name> <uuid>b5d70dd275cdaca517769660b059d8bc</uuid> <os> <type>hvm</type> <loader>/usr/lib/xen/boot/hvmloader</loader> <boot dev='hd'/> </os> <memory>409600</memory> <vcpu>1</vcpu> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <features> <acpi/> <apic/> <pae/> </features> <devices> ...snipped from example.. </devices> </domain> More flags could trivially be added over time. The attached patch implements this proposal. My only thought, is 'features' a good name for this ? Only other name I came up with was 'capabilities' but this is harder to spell :-) Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 2

[Libvir] Fix handling of HVM boot parameters
by Daniel P. Berrange 15 Aug '06

15 Aug '06

Jeremy mentioned that it looked like libvirt wasn't able to create an HVM domain configured to boot off cdrom, so I took a closer look at the code and indeed the code dealing with <boot> section was both incomplete, and just plain broken. Incomplete in so much as it never included details of the ISO file backing the CDROM, and broken in so much as it was doing string comparisons against the wrong variables. Digging further found lots more work relating to creation of HVM domains so I've had a go at writing a patch to resolve matters. * Parsing of UUID from SEXPR assumed that the UUID fetched would always have four '-' in the usual places. Well, when you run 'dumpxml' from virsh the <uuid> element has the UUID encoded without any '-' chars. If you feed this back into 'virsh create' and then once again run 'dumpxml' parsing will fail & libvirt throws errors. So this patch allows for parsing of UUID's without '-'. * The XML document size was limited to 1k - we just 'malloc(1000)'. While this was enough for common cases, if someone creates lots of disk or network devices this would overflow and libvirt return an error. So I increased it to 4k which ought to be enough for forseeable future - in any case my previous patch to fetch XML via the proxy is limited to 4k in size too. Now onto the fine details... When converting SEXPR into XML current code is doing the following: (boot a) -> <boot dev='/dev/fd0'/> (boot c) -> <boot dev='hda'/> (boot d) -> <boot dev='/dev/cdrom'/> This is rather inconsistent - the 'hda' is intended to map to an entry in <devices> block. The '/dev/fd0' and '/dev/cdrom' entries did not map to anything. Meanwhile, when converting from XML to SXEXPR it is doing the following <boot dev='hda'/> -> (boot a) <boot dev='hdd'/> -> (boot d) <boot dev='*'/> -> (boot c) Obviously this sucks because these processes should be matching each other. Secondly, the (image (hvm....)) SEXPR has three entries for defining the ISO / disk image file backing the CDROM / Floppy devices. These were just being ignored, rather than turned into <disk> entries within the <devices> block. Similarly there was no way to express a <disk> entry for CDROM/Floppy in the XML when creating a domain. The upshot of all this is that although the last release of libvirt included HVM support it was basically unusable for domain creation unless you were using HD to boot. The XML returned was also incorrect. Now the good news. Since it was sooo broken, we can fix without worrying about XML compatability since there is no way any application could be relying on it in its current state. Now before I discuss the solution, one final point. The Xen IOMMU model allows specifiying the boot device in terms of 'a', 'c', 'd' which has the meaning: a - first connected floppy device c - first connected harddrive d - first connected cdrom In particular it does *not* allow for expressing boot device in terms of an explicit disk device - you can't say boot off 'hdb' or 'hdc'. So the current libvirt code which tries to fake such semantics is doomed to failure. This isn't really too bad IMHO - VMware has these same semantics, so does QEMU and so do normal bare metal BIOS This in the patch I have attached I have implemented the following mapping: (boot a) <-> <boot dev='fd'/> (boot c) <-> <boot dev='hd'/> (boot d) <-> <boot dev='cdrom'/> The other part of the patch is to deal with definition of the floppy and cdrom device backing files. For this I have done the following: (image (hvm (fda /root/diskboot.img))) <devices> <disk type='file'> <source file='/root/diskboot.img'/> <target dev='fda'/> </disk> </devices> And similar for 'fdb'. Then for cdroms: (image (hvm (cdrom /root/image.iso))) <devices> <disk type='file'> <source file='/root/image.iso'/> <target dev='cdrom'/> </disk> </devices> The patch has a little bit of logic such that when converting the <devices> block backinto an SEXPR it filters out the disk entries with a dev of 'fda', 'fdb' and 'cdrom' since they need to end up in a different part of the SEXPR. Finally, although PV guests automatically get a serial console created for them (and i don't think you can turn it off), HVM guests need to have a serial console explicitly requested. Since I recently added support in the DumpXML method for including details of serial console like this: <devices> <console tty="/dev/pts/5"/> </devices> I decided to leverage this same structure when creating HVM domains. So if you have: <devices> <console/> </devices> Then the SEXPR sent to XenD will include (image (hvm (serial pty))) Which enables allocation of PseudoTTY for the HVM's serial console. I have tested that with this patch I can successfully create a HVM domain which boots off a floppy, harddrive or cdrom. Furthermore if you then dump the XML of this domain,the XML you get back will match the XML you fed in (with the obvious exception of domain ID, and the Pseudo TTY path). If you have been monitoring xen-devel mailing lists you'll be aware that in 3.0.3 the way CDROM devices are configured is changing: http://lists.xensource.com/archives/html/xen-devel/2006-08/msg00369.html Although the patch attached does not support the config outlined in that mail, I'm pretty confident that a small incremental patch will be able to support it without breaking compatability with the changes I've outlined in this mail. The only tricky bit will be that we need to detect whether libvirt is running against a 3.0.2 or 3.0.3 version of XenD to decide how to convert XML -> SEXPR & vica verca. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

4 16