- Devel - libvirt List Archives

[Libvir] State of driver backend infrastructure
by Daniel P. Berrange 14 Jun '06

14 Jun '06

I'm looking to get more of the test driver backend working, and so decided it was time to audit the state of the driver backend infrastructure. There are a couple of areas which could do with some work: * Driver specific data. The virConnect & virDomain structs have explicit fields for Xen related data: virConnect: int handle; /* internal handle used for hypercall */ struct xs_handle *xshandle; /* handle to talk to the xenstore */ virDomain: int handle; /* internal handle for the domnain ID */ One idea for resolving this is to have an array of slots, one per drivers in which drivers can store data they need. void **handles Or perhaps, to allow simple int type too void union { int num; void *ptr } *handles * Connect URIs. The 'virDomainOpen' and 'virDomainOpenReadOnly' methods have an optional parameter 'name'. The Xen drivers currently ignore this, or expect it to be NULL / the string "Xen". For sake of back-compat we'll ned to preserve existing behaviour for NULL, but we should hook up URI parsing for the XenD backend to allow non local connections to work. This gives the question of what format should the URI be for Xen domains ? Although 'http://' is the protocol XenD uses, this isn't likely a good idea, because a later VMWare driver might also use a http based protocol. So, perhaps we should use generic URIs:? xend://somehost:port/ * Driver method hookup. The libvirt.c file has alot of methods which are either not hooked up to the driver backend, or hooked up, but still have duplicated (redundant?) Xen specific calls. Here is the complete status: virGetVersion: - Hardcodes support for Xen HV. No virConnectPtr handle, so how would we call out to driver backends ? virConnectOpen - OK virConnectOpenReadOnly - OK virConnectClose: - Does not call out to driver backends, simply free's struct virConnectGetType: - Does not call out to driver backends, hardcodes Xen HV virConnectGetVersion - Does not call out to driver backends, hardcodes Xen HV virConnectListDomains - Calls out to drivers, but also has duplicated code for XenD & XenStore virConnectNumOfDomains - Calls out to drivers, but also has duplicated code for XenD & XenStore virDomainCreateLinux - Does not call out to driver backends, hardcodes XenD virDomainLookupByID - Calls out to drivers, but also has duplicated code for XenD & XenStore virDomainLookupByUUID - Calls out to drivers, but also has duplicated code for XenD & XenStore virDomainLookupByUUIDString - OK (but delegates to virDomainLookupByUUID) virDomainLookupByName - Calls out to drivers, but also has duplicated code for XenD & XenStore virDomainDestroy - Does not call out to driver backends, hardcodes XenD & Xen HV virDomainFree - Does not call out to driver backends, simply free's struct virDomainSuspend - Does not call out to driver backends, hardcodes XenD & Xen HV virDomainResume - Does not call out to driver backends, hardcodes XenD & Xen HV virDomainSave - Does not call out to driver backends, hardcodes XenD virDomainRestore - Does not call out to driver backends, hardcodes XenD virDomainShutdown - Does not call out to driver backends, hardcodes XenD virDomainReboot - Does not call out to driver backends, hardcodes XenD virDomainGetName - OK virDomainGetUUID - Does not call out to driver backends, hardcodes XenD virDomainGetUUIDString - OK (but delegates to virDomainGetUUID) virDomainGetOSType - Does not call out to driver backends, calls function in XenStore driver virDomainGetMaxMemory - Does not call out to driver backends, hardcodes XenD, Xen HV, XenStore virDomainSetMaxMemory - OK virDomainSetMemory - OK virDomainGetInfo - Calls out to drivers, but also has duplicated code for XenD & XenStore virDomainGetXMLDesc - Does not call out to driver backends, hardcodes XenD virNodeGetInfo - OK virDomainDefineXML - OK virDomainUndefineXML - OK virDomainListDefinedDomains - No implemented virDomainCreate - No implemented My immediate needs for testing, are to hook up more of the methods which aren't connected to the driver backends. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 6

[Libvir] A question about libvirt usage
by Philippe Berthault 13 Jun '06

13 Jun '06

Is there a notion of session in libvirt ? In other words, is it possible, between virConnectOpen() and virConnectClose(), that some Xen domains will be started, stopped, destroyed, etc.. by another operator using the xm command ou by another binary command based on libvirt. If the response is yes, is it possible to introduce a global lock in Xen (or in libvirt ?) to avoid conflict between libvirt and xm command or between several libvirt based commands. Thanks, Philippe Berthault.

2 1

[Libvir] Patch to 'ping' XenD when opening connection
by Daniel P. Berrange 13 Jun '06

13 Jun '06

Currently the XenD driver's implementation of the 'open' method will return success, regardless of whether XenD is even present. The attached patch makes the open method do a 'ping' to see if XenD is actually there, returning failure if it is not. This ensures that the XenD driver backend doesn't get activated when connecting to alternate non-Xen backends, such as the test backend I committed last week. The current 'ping' is simply to call the xenDaemonGetVersion() method since that's a pretty simle & low-overhead way to testing livliness of XenD. Any suggestions for a better ping - if not I'll go ahead & commit this change Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 3

[Libvir] A 'mock' driver enabling application unit testing
by Daniel P. Berrange 06 Jun '06

06 Jun '06

The attached patch adds a new driver which provides a 'mock' hypervisor connection. The idea behind this is to provide a completely predictable and isolated hypervisor implementation, which will facilitate creation of unit tests in applications using libvirt. The code attached is not a complete implementation, providing support only for connecting, listing domains, getting a domain by id/uuid/name, getting domain info, getting node info. Further work will hook up methods for creating domains, exporting XML & the other various libvirt APIs. The driver is intended to operate as a black box, closed system - the only interaction an application will have is via the libVirt entry points. It starts off with a single domain present, and fixed node info, and updates domain CPU time based on elapsed wall clock time. When the domain create API is hooked up this will let apps create new dummy domains within the context of the process. As I said, I delibrately implemented this driver to only have process local state. When the process shuts down all state is discarded. This keeps application usage very simple. Longer term I think it would be useful to have a 2nd, dynamic, which would allow fault injection, cross-process interaction & more to allow advanced scripting of a test scenario. I don't have time to create such a driver myself though, so I stuck with the simplest possible impl which will allow 95% of application unit testing needs to be satisfied. To use this driver, simply pass 'TestSimple' as the name parameter to virConnectOpen / virConnectOpenReadonly instead of NULL. If you are using 'virsh' then my previous patch will let you call 'virsh --connect TestSimple' I've also tested this with the 'gnome-vm-applet' panel applet. The only problem I have found is that the 'xend_internal.c' driver will always return success from its 'xenDaemonOpen' method, regardless of whether there is actually a Xen Daemon present. So when using the test driver, every method will first try to ue the XenD driver impl, fail (printing an error message) and then go onto use the test driver. The xenDaemonOpen method really needs to be fixed to only succeed when Xen is actually present. Or perhaps it should only try to run when the 'name' passed to virConnectOpen is NULL or 'Xen' - ie be a no op if the name is 'TestSimple' / QEMU / any other driver implementation. I'm not sure whether we want to commit this to the libvirt codebae just, since there is a fair bit more work to be done to hook up additional methods. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 4

[Libvir] Added 'domuuid' to virsh & --connect command lin arg
by Daniel P. Berrange 31 May '06

31 May '06

The attached patch makes two changes to the virsh command: * Adds a 'domuuid' command to display the printable UUID of a domain * Adds a '--connect NAME' argument allowing the name of a hypervisor to be passed through to 'virConnect' / 'virConnectReadOnly' Although libvirt only has a Xen backend at the moment, I'm working on a 'mock' hypervisor backend to enable unit testing of applications built on libvirt without requiring the test suite to interface with the real Xen hypervisor. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 2

[Libvir] Connect up more internal driver methods
by Daniel P. Berrange 29 May '06

29 May '06

The attached patch connects up more of the internal driver methods, as the final step preparing for a minimally operational test driver. * src/hash.c, src/internal.h: Switch the uuid parameter in virGetDomain to be of type 'unsigned char' since its a raw UUID we're passing in, not a printable one. * src/libvirt.c: Remove bogus "unsigned char" -> "char" type casts. Hook up the "domainLookupByID", "domainLookupByUUID", "domainLookupByName" and "domainGetInfo" driver backend functions. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 1

[Libvir] A few bug fix patches
by Daniel P. Berrange 29 May '06

29 May '06

Attached are two small bug fix patches: - libvirt-vpath.patch - fix test suite include path for VPATH builds - libvirt-symbols.patch - add some missing symbols to the libraries exported symbol table to make the python bindings work again Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

2 1

[Libvir] uuid string format
by Karel Zak 23 May '06

23 May '06

Hello, I've found that we use in our XML dump: <uuid>%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x</uuid> it's strange format of the UUID string representation. Is there any reason for this format? I think normal (rfc4122) is format: "%02x%02x%02x%02x-%02x%02x-%02x%02x-%02x%02x-%02x%02x%02x%02x%02x%02x" it means with '-'. Karel -- Karel Zak <kzak(a)redhat.com>

3 3

[Libvir] BUG: virDomainLookupByID fails in libvirt-0.1.0
by Michel Gauthier 05 May '06

05 May '06

Hi, I tried to execute the info1 C code example on my redhat distribution with Xen installed. It fails due to the virDomainLookupByID API wich returns NULL. The problem seems to be in the internal function xenDaemonListDomains called after xs_get_domain_path inside the API. For the domain 0, xs_get_domain_path returns path=/local/domain/0. Then, the call to xenDaemonListDomains function logs the error message: "libvir: Xen Daemon error : could not connect to Xen Daemon" and returns names=NULL. A bug present in the release 0.0.6 has been fixed for the release 0.1.0. But this bug seems still present! Regards.

3 2

[Libvir] Re: support for hvm guests
by Jim Fehlig 04 May '06

04 May '06

>>>> Anthony Liguori <aliguori(a)us.ibm.com> 05/03/06 6:47 PM >>> > >>>> Currently I can get info on hvm domains but unable to create them. >>>> The create request is sent to xend but then blocks reading from xend >>>> in xend_req(). No problem creating the domain using xm. I have >>>> looked at the config coming into xend and it is identical between xm >>>> and virsh/libvirt. Need to poke around xend and see why there is no >>>> response to the virsh/libvirt create request for hvm domains. >>>> >>> >>> Hum, basically to debug this kind of things I spend my time in >>> /var/log/xend.log especially since it prints what it received at >>> creation >>> time, you can at least check the informations coming from libvirt and >>> from xm look alike. >>> Unfortunately I won't be able to help you, first because I'm about to >>> take vacations for 3 weeks :-), second because I don't have one of those >>> new Intel CPUs, but I am sure others will give an hand if needed. >>> >>> >> >> Well, I have found the problem but not a fix :-). When posting the >> create op to xend, the libvirt code currently waits for a response >> from xend (xend_req() is called after posting in xend_post()). For >> hvm guests, xend does not respond and libvirt blocks indefinitely on >> read. If I skip over the call to xend_req() for the create operation, >> wait_for_devices() and subsequently unpause() are called and the hvm >> guest is launched. > >What do you mean by Xend does not respond? I post a request such as: op=create&config=(vm%20(name%20'j_sles10b9_full')(memory%20512)(vcpus%201)(on_poweroff%20'destroy')(on_reboot%20'destroy')(on_crash%20'destroy')(image%20(hvm%20(kernel%20'/usr/lib/xen/boot/hvmloader')(device_model%20'/usr/lib/xen/bin/qemu-dm')(boot%20c)(sdl%201)(stdvga%201)(vcpus%201)))(device%20(vbd%20(dev%20'ioemu:hda')(uname%20'file:/var/lib/xen/images/j_sles10b9_full/hda')(mode%20'w')))(device%20(vif%20(mac%20'00:16:3e:b2:68:1d')(bridge%20'xenbr0')(script%20'vif-bridge')(type%20'ioemu')))) to xend from xend_post() in libvirt's xend_internal.c, which then expects to read a response from xend but it never arrives and blocks the read in libvirt. >Can you give me a very concrete test case for this? The exact >parameters being passed to domain_create() (it should just be an >S-Expression config). The sexpr is: (vm (name 'j_sles10b9_full')(memory 512)(vcpus 1)(on_poweroff 'destroy')(on_reboot 'destroy')(on_crash 'destroy')(image (hvm (kernel '/usr/lib/xen/boot/hvmloader')(device_model '/usr/lib/xen/bin/qemu-dm')(boot c)(sdl 1)(stdvga 1)(vcpus 1)))(device (vbd (dev 'ioemu:hda')(uname 'file:/var/lib/xen/images/j_sles10b9_full/hda')(mode 'w')))(device (vif (mac '00:16:3e:b2:68:1d')(bridge 'xenbr0')(script 'vif-bridge')(type 'ioemu')))) The url-encoded request sent from libvirt is also provided above. Below is the relevent content in xend.log. Thanks, Jim [2006-05-02 10:58:13 xend.XendDomainInfo] DEBUG (XendDomainInfo:180) XendDomainInfo.create(['vm', ['name', 'j_sles10b9_full'], ['memory', '512'], ['maxmem', '512'], ['vcpus', '1'], ['on_poweroff', 'destroy'], ['on_reboot', 'destroy'], ['on_crash', 'destroy'], ['image', ['hvm', ['kernel', '/usr/lib/xen/boot/hvmloader'], ['device_model', '/usr/lib/xen/bin/qemu-dm'], ['boot', 'c'], ['cdrom', '/tests/iso/SLES-10-i386-Beta9.iso'], ['sdl', '1'], ['stdvga', '1'], ['vcpus', '1']]], ['device', ['vbd', ['dev', 'ioemu:hda'], ['uname', 'file:/var/lib/xen/images/j_sles10b9_full/hda'], ['mode', 'w']]], ['device', ['vif', ['mac', '00:16:3e:b2:68:1d'], ['bridge', 'xenbr0'], ['script', 'vif-bridge'], ['type', 'ioemu']]]]) [2006-05-02 10:58:13 xend.XendDomainInfo] DEBUG (XendDomainInfo:286) parseConfig: config is ['vm', ['name', 'j_sles10b9_full'], ['memory', '512'], ['maxmem', '512'], ['vcpus', '1'], ['on_poweroff', 'destroy'], ['on_reboot', 'destroy'], ['on_crash', 'destroy'], ['image', ['hvm', ['kernel', '/usr/lib/xen/boot/hvmloader'], ['device_model', '/usr/lib/xen/bin/qemu-dm'], ['boot', 'c'], ['cdrom', '/tests/iso/SLES-10-i386-Beta9.iso'], ['sdl', '1'], ['stdvga', '1'], ['vcpus', '1']]], ['device', ['vbd', ['dev', 'ioemu:hda'], ['uname', 'file:/var/lib/xen/images/j_sles10b9_full/hda'], ['mode', 'w']]], ['device', ['vif', ['mac', '00:16:3e:b2:68:1d'], ['bridge', 'xenbr0'], ['script', 'vif-bridge'], ['type', 'ioemu']]]] [2006-05-02 10:58:13 xend.XendDomainInfo] DEBUG (XendDomainInfo:382) parseConfig: result is {'ssidref': None, 'uuid': None, 'on_crash': 'destroy', 'on_reboot': 'destroy', 'localtime': None, 'image': ['hvm', ['kernel', '/usr/lib/xen/boot/hvmloader'], ['device_model', '/usr/lib/xen/bin/qemu-dm'], ['boot', 'c'], ['cdrom', '/tests/iso/SLES-10-i386-Beta9.iso'], ['sdl', '1'], ['stdvga', '1'], ['vcpus', '1']], 'on_poweroff': 'destroy', 'cpus': None, 'name': 'j_sles10b9_full', 'backend': [], 'cpu': None, 'vcpus': 1, 'cpu_weight': None, 'vcpu_avail': None, 'memory': 512, 'device': [('vbd', ['vbd', ['dev', 'ioemu:hda'], ['uname', 'file:/var/lib/xen/images/j_sles10b9_full/hda'], ['mode', 'w']]), ('vif', ['vif', ['mac', '00:16:3e:b2:68:1d'], ['bridge', 'xenbr0'], ['script', 'vif-bridge'], ['type', 'ioemu']])], 'bootloader': None, 'root': None, 'bootentry': None, 'maxmem': 512} [2006-05-02 10:58:13 xend.XendDomainInfo] DEBUG (XendDomainInfo:1189) XendDomainInfo.construct: None 0 [2006-05-02 10:58:13 xend.XendDomainInfo] DEBUG (XendDomainInfo:1221) XendDomainInfo.initDomain: 29 1.0 [2006-05-02 10:58:13 xend] DEBUG (image:266) args: cdrom, val: /tests/iso/SLES-10-i386-Beta9.iso [2006-05-02 10:58:13 xend] DEBUG (image:266) args: boot, val: c [2006-05-02 10:58:13 xend] DEBUG (image:266) args: fda, val: None [2006-05-02 10:58:13 xend] DEBUG (image:266) args: fdb, val: None [2006-05-02 10:58:13 xend] DEBUG (image:266) args: nic-ne2000, val: None [2006-05-02 10:58:13 xend] DEBUG (image:266) args: enable-audio, val: None [2006-05-02 10:58:13 xend] DEBUG (image:266) args: localtime, val: None [2006-05-02 10:58:13 xend] DEBUG (image:266) args: serial, val: None [2006-05-02 10:58:13 xend] DEBUG (image:266) args: std-vga, val: 1 [2006-05-02 10:58:13 xend] DEBUG (image:266) args: isa, val: None [2006-05-02 10:58:13 xend] DEBUG (image:266) args: vcpus, val: 1 [2006-05-02 10:58:13 xend] DEBUG (balloon:177) Balloon: free 2267; need 518; done. [2006-05-02 10:58:13 xend] INFO (image:135) buildDomain os=hvm dom=29 vcpus=1 [2006-05-02 10:58:13 xend] DEBUG (image:224) dom = 29 [2006-05-02 10:58:13 xend] DEBUG (image:225) image = /usr/lib/xen/boot/hvmloader [2006-05-02 10:58:13 xend] DEBUG (image:226) store_evtchn = 1 [2006-05-02 10:58:13 xend] DEBUG (image:227) memsize = 512 [2006-05-02 10:58:13 xend] DEBUG (image:228) vcpus = 1 [2006-05-02 10:58:13 xend] DEBUG (image:229) pae = 0 [2006-05-02 10:58:13 xend] DEBUG (image:230) acpi = 0 [2006-05-02 10:58:13 xend] DEBUG (image:231) apic = 0 [2006-05-02 10:58:13 xend] DEBUG (image:409) hvm shutdown watch registered [2006-05-02 10:58:13 xend] DEBUG (blkif:24) exception looking up device number for hda: [Errno 2] No such file or directory: '/dev/hda' [2006-05-02 10:58:13 xend] DEBUG (DevController:103) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vbd/29/768'} to /local/domain/29/device/vbd/768. [2006-05-02 10:58:13 xend] DEBUG (DevController:105) DevController: writing {'domain': 'j_sles10b9_full', 'frontend': '/local/domain/29/device/vbd/768', 'dev': 'ioemu:hda', 'state': '1', 'params': '/var/lib/xen/images/j_sles10b9_full/hda', 'mode': 'w', 'frontend-id': '29', 'type': 'file'} to /local/domain/0/backend/vbd/29/768. [2006-05-02 10:58:13 xend] DEBUG (DevController:103) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vif/29/0'} to /local/domain/29/device/vif/0. [2006-05-02 10:58:13 xend] DEBUG (DevController:105) DevController: writing {'bridge': 'xenbr0', 'domain': 'j_sles10b9_full', 'handle': '0', 'script': '/etc/xen/scripts/vif-bridge', 'state': '1', 'frontend': '/local/domain/29/device/vif/0', 'mac': '00:16:3e:b2:68:1d', 'frontend-id': '29', 'type': 'ioemu'} to /local/domain/0/backend/vif/29/0. [2006-05-02 10:58:13 xend] INFO (image:355) spawning device models: /usr/lib/xen/bin/qemu-dm ['/usr/lib/xen/bin/qemu-dm', '-d', '29', '-m', '512', '-cdrom', '/tests/iso/SLES-10-i386-Beta9.iso', '-boot', 'c', '-std-vga', '-vcpus', '1', '-domain-name', 'j_sles10b9_full', '-hda', '/var/lib/xen/images/j_sles10b9_full/hda', '-macaddr', '00:16:3e:b2:68:1d', '-bridge', 'xenbr0', '-nics', '1'] [2006-05-02 10:58:13 xend] INFO (image:357) device model pid: 3496 [2006-05-02 10:58:13 xend.XendDomainInfo] DEBUG (XendDomainInfo:675) Storing VM details: {'ssidref': '0', 'uuid': 'a3fc6694-3959-b4bf-85b3-0364c231c504', 'on_reboot': 'destroy', 'start_time': '1146589093.41', 'on_poweroff': 'destroy', 'name': 'j_sles10b9_full', 'vcpus': '1', 'vcpu_avail': '1', 'memory': '512', 'on_crash': 'destroy', 'image': '(hvm (kernel /usr/lib/xen/boot/hvmloader) (device_model /usr/lib/xen/bin/qemu-dm) (boot c) (cdrom /tests/iso/SLES-10-i386-Beta9.iso) (sdl 1) (stdvga 1) (vcpus 1))', 'maxmem': '512'} [2006-05-02 10:58:13 xend.XendDomainInfo] DEBUG (XendDomainInfo:700) Storing domain details: {'console/port': '2', 'name': 'j_sles10b9_full', 'console/limit': '1048576', 'vm': '/vm/a3fc6694-3959-b4bf-85b3-0364c231c504', 'domid': '29', 'cpu/0/availability': 'online', 'memory/target': '524288', 'store/ring-ref': '682307', 'store/port': '1'} [2006-05-02 10:58:13 xend] DEBUG (image:432) hvm_shutdown fired, shutdown reason=None [2006-05-02 10:58:13 xend.XendDomainInfo] DEBUG (XendDomainInfo:881) XendDomainInfo.handleShutdownWatch

1 0