March 2007 - Devel - libvirt List Archives

[Libvir] Storage manager initial requirements and thoughts
by Hugh Brock 15 Jun '07

15 Jun '07

Hi all. At the Red Hat virtualization team meeting last week we spent some time talking about the problem of remote storage management, which is a requirement for at the very least creating remote guests. Remote storage management also delivers a lot of benefits for managing guest storage, provisioning, and a host of other issues. Note that we don't (I think) believe that storage management should be part of libvirt. However we might want to be able to grab storage information with libvirt when creating a guest, for example. Here's the tentative requirements we came up with, along with several questions that came up. We have two important use cases for remote storage management: --Create a new guest against an existing physical device --Create a new guest against a file on an existing physical device (SELinux context must match up on both) Requirements: 2 key requirements: enumerate devices, create files * Enumerate devices (unallocated disks) * storage pool * Unallocated space * Allocated volumes -- free -- in use * Read only / Read-write * Host availability * Volume -- Global unique name -- Local device name -- Usage -- Mounted locally -- Assigned to guest read only shared exclusive -- Inactive guest * Create a backing store for a guest * need to know what storage is available * Can be plain file * Can be a physical partition (local scsi/IDE, SAN) * Can be a logical volume * Can be network - iscsi/nbd Cases: Physical device --> create partitions Volume group --> create logical volumes Directory --> create files Todos: Investigate gparted, one of the partition management tools we already have (apis? remote accessibility?) (I believe Jim Meyering volunteered to take a look at this?) Identify what other scenarios we need to address. Please comment; I'm hoping we can get these into a more definite form sooner rather than later. -- Red Hat Virtualization Group http://redhat.com/virtualization Hugh Brock | virt-manager http://virt-manager.org hbrock(a)redhat.com | virtualization library http://libvirt.org

10 43

[Libvir] [RFC]OpenVZ XML def
by Shuveb Hussain 22 May '07

22 May '07

Hi, I am adding OpenVZ support to Libvirt and work is progressing well. I'm able to list VM instances and I'm slowly trying to cover the basic API functions one by one. That brings us to the creation of OpenVZ based VMs. Just wanted to discuss the basic XML definition format so that I can get comments and improve if need be: <domain type='openvz'> <name>openvzdemo</name> <uuid>55a2441d-e012-40fe-a4f7-78e176015d40</uuid> <vpsid>101</vpsid> <template>vps.basic</template> <onboot>true</onboot> <os template='slackware-10.2-i386-minimal'/> <network> <hostname>openvzhost</hostname> <ip address='192.168.1.101' netmask='255.255.255.0' defgw='192.168.1.1' /> <nameserver>192.168.1.1</nameserver> <nameserver>202.56.250.5</nameserver> </network> </domain> Does this look OK? Regards, -- Shuveb Hussain. EasyVZ - OpenVZ management GUI: http://easyvz.sourceforge.net I blog at http://binarykarma.org Spread the Karma.

4 15

[Libvir] [PATCH][RFC] shows scheduler information
by Atsushi SAKAI 12 Apr '07

12 Apr '07

Hi, This patch intends add a showing function of scheduler information. If apply this patch to current revision 1.490 and type virsh dominfo 0 then shows following scheduler information. Currently I assume credit scheduler only. getting function is implemented but setting function is not yet. I just waiting for comments. Id: 0 Name: Domain-0 UUID: 00000000-0000-0000-0000-000000000000 OS Type: linux State: running CPU(s): 1 CPU time: 432.7s Max memory: no limit Used memory: 1933312 kB scheduler info 5 256 0 SchedulerID: 5 Weight: 256 Cap: 0 Thanks Atsushi SAKAI

4 26

[Libvir] RFC [0/3]: Re-factor QEMU daemon protocol to use XDR
by Daniel P. Berrange 11 Apr '07

11 Apr '07

I've been thinking about a way to move forward on the remote management capabilities in libvirt since our last list discussion didn't come to any clear agreement. The current situation - QEMUD daemon uses a hand-crafted protocol basically just dumping structs on the wire. Key points: - Fairly simple to understand code for marshalling/demarshalling - Very low overhead on wire - Not safe endianess - Not safe wrt to architecture data alignment rules - No authentication/encryption - Server driver specific to QEMU - The generic libvirt daemon uses SunRPC as the underlying protocol. - Protocol versioning - Safe wrt to endianess - Safe wrt to architecture data alignment rules - Added TLS encryption / SSH tunneling - Generic for any libvirt backend - Doesn't work with non-blocking sockets due to SunRPC limitations - SunRPC api is not well known / 'wierd' - Call + reply only; no async signals from server Looking at the current impl of the generic libvirt daemon there are some things which strike me: 1. We have an extra layer of marshalling/demarshalling calls. ie the methods in the libvirt driver, call to the SunRPC marshalling methods, which populate an XDR generated struct, which call the XDR routines to write the data. 2. The main loop doesn't allow us to listen for other non-SunRPC file handles (eg the QEMU/dnsmasq processes). 3. We've basically re-written the the SunRPC svc*_create methods to get support for TLS / SSH We are going to need to address 2 by creating our own mainloop impl. For point 1 we can have the libvirt driver populate the XDR struct directly and eliminate the SunRPC generated marshalling stubs completely. If we did that, and given points 2 & 3, we're basically left using very little of the SunRPC capabilities at all. Pretty much only thing it is doing for us at that point is the protocol versioning, and giving us a horrific API which doesn't support non-blocking IO so forces us to be multi-threads or multi-process. So, I'm thinking why don't we just use XDR directly & be done with it. As an experiment I took the existing QEMU daemon & driver and re-factored the hand written binary protocol to use XDR instead. Looking at the diffstat - there was basically no appreciable difference in codesize after the re-factoring, and we get the added benefit of being endianness & alignment safe. qemud/Makefile.am | 15 + qemud/conf.c | 1 qemud/dispatch.c | 538 ++++++++++++++++------------------------------ qemud/dispatch.h | 2 qemud/internal.h | 21 + qemud/protocol.h | 329 ---------------------------- qemud/protocol.x | 608 ++++++++++++++++++++++++++++++++++++++++++++++++++++ qemud/qemud.c | 189 +++++++++++----- qemud/uuid.c | 1 src/Makefile.am | 11 src/internal.h | 2 src/qemu_internal.c | 593 ++++++++++++++++++++++++++------------------------ 12 files changed, 1288 insertions(+), 1022 deletions(-) The particular changes I'll explain in the next 3 mails where I attach the patches. Meanwhile though, I'm thinking we could take approach of incremental code refactoring, pulling in all the various capabilities from the remote patch (except for the SunRPC demarshalling itself) to the QEMU daemon. At each stage having a working daemon & client but with ever growing capabilities, until it provides full remote management. 1. Add an extra initial message type to do a major/minor version number exchange. Based on the negotiated versions, the server can activate or deactivate particular requests it allows from the client. 2. Add in the URI parsing routines to the qemud_interal.c driver, but only allow use of QEMU initially. 3. Add in the TLS + SSH client & server socket setup code to give an authenticated & secured remote channel 4. Re-factor qemud/dispatch.c so that it calls into the regular libvirt API for the Xen case, but keeps calling qemud/driver.c for the QEMU case. BTW, these patches are not intended to be applied to CVS - they're for discussion purposes at this stage. That said, they are fully functional so you can apply them to your checkout & you should be able run all the existing virsh commands against the libvirt_qemud. Just make sure you patch & rebuild both the client & server ;-) Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 12

[Libvir] Unified Xen patch (incomplete, for discussion)
by Richard W.M. Jones 02 Apr '07

02 Apr '07

I've spent the last day or so producing a unified Xen patch. In such a model a connection contains only a single underlying driver, and it is the responsibility of the (unified) Xen driver to try all the different methods it knows. Attached is an incomplete patch for this, for discussion, plus the two extra files of the unified driver itself (for some reason CVS won't give me a diff including these added files). Some points to note: The new structure of the drivers is: <pre> libvirt.c | +------- xen_unified.c | | | +--- xen_internal.c (hypervisor) | | | +--- proxy_internal.c * (proxy) | | | +--- xend_internal.c * (XenD) | | | +--- xs_internal.c (XenStore) | | | +--- xm_internal.c * (inactive domains) | +-------- qemu_internal.c * | +-------- test.c * = not updated yet, so these don't compile </pre> I haven't renamed the Xen sub-drivers. That's really to make the patch easier to read. There is definitely a case for renaming the drivers more logically to reflect the structure above. All Xen-specific hacks in libvirt.c have been moved to xen_unified.c Error handling in the case where a driver doesn't support a libvirt function is now considerably better. Each driver keeps its private data private. At the moment xen_unified pretty much does the "try the drivers in a loop until one succeeds" strategy which used to be in libvirt.c. There is a case for making it do direct calls to the "right" driver, but for simplicity I haven't gone that far. Again for simplicity the Xen sub-drivers still use struct virDriver. They should use their own custom 'struct virXenDriver' or whatever. The effects of this are slight but noticable - some parameters are now no longer used in the sub-drivers, so marked ATTRIBUTE_UNUSED. There is also a single networkDriver pointer in the connect struct. We should modify the functions to handle the case where this is NULL because we couldn't bring up the network functions (all network functions in this case would just return an error). libvirt in CVS fails to run if libvirtd isn't up. I won't be able to get back to this before tomorrow afternoon, so plenty of time for discussion! Rich. -- Emerging Technologies, Red Hat http://et.redhat.com/~rjones/ 64 Baker Street, London, W1U 7DF Mobile: +44 7866 314 421 Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in UK and Wales under Company Registration No. 3798903 Directors: Michael Cunningham (USA), Charlie Peters (USA) and David Owens (Ireland)

3 8

Re: [Libvir] [patch 0/5] Misc. fixes to iptables support
by Mark McLoughlin 30 Mar '07

30 Mar '07

On Wed, 2007-03-21 at 12:47 +0000, Mark McLoughlin wrote: > Hey, > What follows is a few misc patches for qemud's iptables > support. The main point is to add reload-on-HUP as suggested by > danpb. > > Comments welcome .. Hmm, I nearly forgot about these patches ... no objections, then? Cheers, Mark.

3 2

[Libvir] Migration support
by Simon Boulet 30 Mar '07

30 Mar '07

Hi, What is the status of migration support for Libvirt, either offline or live migration? My understanding of Libvirt is that it does not support connecting to remote nodes yet. Thank you Simon

3 2

[Libvir] [PATCH] XPath accessors cleanup
by Daniel Veillard 30 Mar '07

30 Mar '07

It a long time cleanup TODO, the XML parsing code of xml.c is invaded with libxml2 specific XPath lookup code, this patch defines 5 clearly isolated accessor functions and then fixes all the src/xml.c code to use those. IMHO this clearly improve readability and maintanability and I think I also found a couple of bugs in the process. The patch enclosed only fixes xml.c and xml.h, the next step is to chase other modules to apply the same treatment if needed. Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

2 2

[Libvir] [PATCH] Enable USB device setting information handling on virsh.
by Masayuki Sunou 29 Mar '07

29 Mar '07

Hi When domHVM started by virsh create, the information of USB setting is not saved by vish dumpxml. The reason is that USB setting is defined by Xen itself, not virsh. This patch enables USB device setting information handling on virsh create/virsh dumpxml. Signed-off-by: Masayuki Sunou <fj1826dm(a)aa.jp.fujitsu.com> Thanks, Masayuki Sunou Index: libvirt/src/xend_internal.c =================================================================== RCS file: /data/cvs/libvirt/src/xend_internal.c,v retrieving revision 1.99 diff -u -p -r1.99 xend_internal.c --- libvirt/src/xend_internal.c 6 Mar 2007 21:55:44 -0000 1.99 +++ libvirt/src/xend_internal.c 7 Mar 2007 08:28:55 -0000 @@ -1666,6 +1666,13 @@ xend_parse_sexp_desc(virConnectPtr conn, free(tty); } + tmp = sexpr_node(root, "domain/image/hvm/usbdevice"); + if ((tmp != NULL) && (tmp[0] != 0)) { + virBufferVSprintf(&buf, " <usb usbdevice='%s'/>\n", tmp); + } else if (sexpr_int(root, "domain/image/hvm/usb")) { + virBufferAdd(&buf, " <usb/>\n", 11 ); + } + virBufferAdd(&buf, " </devices>\n", 13); virBufferAdd(&buf, "</domain>\n", 10); Index: libvirt/src/xml.c =================================================================== RCS file: /data/cvs/libvirt/src/xml.c,v retrieving revision 1.62 diff -u -p -r1.62 xml.c --- libvirt/src/xml.c 6 Mar 2007 20:00:17 -0000 1.62 +++ libvirt/src/xml.c 7 Mar 2007 08:28:55 -0000 @@ -524,6 +524,20 @@ virDomainParseXMLOSDescHVM(virConnectPtr } xmlXPathFreeObject(obj); + obj = xmlXPathEval(BAD_CAST "/domain/devices/usb", ctxt); + if ((obj != NULL) && (obj->type == XPATH_NODESET) && + (obj->nodesetval != NULL) && (obj->nodesetval->nodeNr == 1)) { + virBufferAdd(buf, "(usb 1)", 7); + xmlXPathFreeObject(obj); + obj = xmlXPathEval(BAD_CAST "string(/domain/devices/usb/@usbdevice)", ctxt); + if ((obj != NULL) && (obj->type == XPATH_STRING) && + (obj->stringval != NULL) && (obj->stringval[0] != 0)) { + virBufferVSprintf(buf, "(usbdevice '%s')", obj->stringval); + } + } + if (obj) + xmlXPathFreeObject(obj); + virBufferAdd(buf, "))", 2); if (boot_dev)

4 10

[Libvir] [PATCH] Python bindings now generate exceptions for libvirt errors
by Richard W.M. Jones 28 Mar '07

28 Mar '07

Following on from this thread: https://www.redhat.com/archives/libvir-list/2007-March/thread.html#00341 The first attachment is a patch which changes the generated Python bindings so that they always raise an exception when an underlying error occurs in the C libvirt library. The second attachment is for information only (not to be applied) - it shows the differences in the generated file libvirtclass.py. I have checked each of these changes by hand against both the libvirt.c interface description and the C implementation of the Python bindings (libvir.c and libvirt-py.c), and each change seems correct. I have _not_ tested this against virt-manager. It is quite possible that virt-manager will cease to work if it wasn't checking the return values from affected functions carefully. I'll check this later today and supply a patch (to virt-manager) if necessary. I have some additional questions about the Python bindings, but I'll put those in a separate thread. Rich. -- Emerging Technologies, Red Hat http://et.redhat.com/~rjones/ 64 Baker Street, London, W1U 7DF Mobile: +44 7866 314 421 "[Negative numbers] darken the very whole doctrines of the equations and make dark of the things which are in their nature excessively obvious and simple" (Francis Maseres FRS, mathematician, 1759)

4 15