October 2009 - Devel - libvirt List Archives

[libvirt] 0.7.1 compile fails on opensuse 11.1
by Dominik Klein 30 Oct '09

30 Oct '09

Hi it seems openSuSE 11.1 does not come with a pkgconfig for the device-mapper-devel package. I created a bug [1] for opensuse but was also told to mention it here so configure.in could be patched once the bug was fixed. Regards Dominik 1 https://bugzilla.novell.com/show_bug.cgi?id=54094

4 9

[libvirt] [PATCH] Priority of vzctl create should be lower
by Yuji NISHIDA 30 Oct '09

30 Oct '09

Hi all About OpenVZ, it uses vzctl command that produces tar/gzip commands. I think we should lower these commands for running other procedures prior to them. This is the patch to achieve it with nice/ionice commands. From ff3cf022995ef541136bd40579af2eeb20aeafdb Mon Sep 17 00:00:00 2001 From: root <root(a)node13.nvlab.org> Date: Mon, 26 Oct 2009 02:44:43 +0000 Subject: [PATCH] lower priority for running tar and gzip during vzctl create --- src/openvz/openvz_driver.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/src/openvz/openvz_driver.c b/src/openvz/openvz_driver.c index f64ad1e..59d38ae 100644 --- a/src/openvz/openvz_driver.c +++ b/src/openvz/openvz_driver.c @@ -130,6 +130,14 @@ openvzDomainDefineCmd(virConnectPtr conn, } while (0) narg = 0; + + /* lower priority for running tar and gzip */ + ADD_ARG_LIT("nice"); + ADD_ARG_LIT("-n"); + ADD_ARG_LIT("19"); + ADD_ARG_LIT("ionice"); + ADD_ARG_LIT("-c3"); + ADD_ARG_LIT(VZCTL); ADD_ARG_LIT("--quiet"); ADD_ARG_LIT("create"); -- 1.5.3.4 ----- Yuji Nishida nishidy(a)nict.go.jp

3 3

[libvirt] QEMU driver thread safety rules
by Daniel P. Berrange 30 Oct '09

30 Oct '09

The current QEMU driver makes use of 2 locks - The driver lock - The virDomainObjPtr lock The idea is the driver lock is not held for long periods of time. Unfortunately we don't always deal with this very well - some code needs todo quite alot with the driver - particularly starting and stopping of guests. The bigger problem is that the virDomainObjPtr lock is often held for long periods, specifically whenever we invoke a monitor command. Some of these commands can take a very long time (even infinite if someone has send SIGSTOP to QEMU). This very quickly blocks the whole driver. I've realized that even with the series of monitor patches I sent out, changing the driver mutex to a RWLock, and adding a separate lock on the qemuMonitorPtr object iself, there's still a major concurrency problem: the virDomainObjPtr lock is held for too long. I propose to drop the RWLock patch, and do something totally different instead.... We fundamentally need to drop the virDomainObjPtr lock whenever we invoke a monitor command. Unfortunately, merely dropping the virDomainObjPtr and acquiring the qemuMonitorPtr is not safe. An API call which changes the VM state typically has 3 phases 1. Check what state/config the VM is in 2. Invoke the monitor command 3. Update the state/config of the VM If we release the virDomainObjPtr, and acquire qemuMonitorPtr at step 2, then other APIs calls will be able to complete their own step 1 checks, and get blocked at step 2. This is not safe, because when the original call moves onto step 3 and changes the state, this will have invalidated the checks the other sleeping API calls made in step 1. We need to prevent any API call starting step 1, for as long as there is a monitor command being run, even if the lock on virDomainObjPtr is not held. The only way I see todo this, is to introduce a condition variable indicating that a state change is to be made. Any API call which intends to make a state change must acquire this condition prior to step 1. They can thus safely do their checks, and move onto step 2, releasing the virDomainObj lock whle the monitor command is running, and reacquiring it after. All other API calls making changes get safely queued up at step 1, but API calls which simply wish to query information can run without being blocked at all. This fixes the major concurrency problem with running monitor commands. The use of a condition variable at the start of step 1, also allows us to time out API calls, if some other thread get stuck in the monitor for too long. I think this also makes the use of a RWLock on the QEMU driver unneccessary, since no code will ever be holding a mutex in any place that sleeps/wait. Only the condition variable will be held during sleeps/waits. Since we'll now effectively have 3 locks, and 1 condition variable this is getting kind of complex. So the rest of this mail is a file I propose to put in src/qemu/THREADS.txt describing what is going on, and showing the recommended design patterns to use. Daniel QEMU Driver Threading: The Rules ================================= This document describes how thread safety is ensured throughout the QEMU driver. The criteria for this model are: - Objects must never be exclusively locked for any pro-longed time - Code which sleeps must be able to time out after suitable period - Must be safe against dispatch asynchronous events from monitor Basic locking primitives ------------------------ There are a number of locks on various objects * struct qemud_driver: RWLock This is the top level lock on the entire driver. Every API call in the QEMU driver is blocked while this is held, though some internal callbacks may still run asynchronously. This lock must never be held for anything which sleeps/waits (ie monitor commands) When obtaining the driver lock, under *NO* circumstances must any lock be held on a virDomainObjPtr. This *WILL* result in deadlock. * virDomainObjPtr: Mutex Will be locked after calling any of the virDomainFindBy{ID,Name,UUID} methods. Lock must be held when changing/reading any variable in the virDomainObjPtr Once the lock is held, you must *NOT* try to lock the driver. You must release all virDomainObjPtr locks before locking the driver, or deadlock *WILL* occurr. If the lock needs to be dropped & then re-acquired for a short period of time, the reference count must be incremented first using virDomainObjRef(). If the reference count is incremented in this way, it is not neccessary to have the driver locked when re-acquiring the dropped locked, since the reference count prevents it being freed by another thread. This lock must not be held for anything which sleeps/waits (ie monitor commands). * qemuMonitorPrivatePtr: Job condition Since virDomainObjPtr lock must not be held during sleeps, the job condition provides additional protection for code making updates. Immediately after acquiring the virDomainObjPtr lock, any method which intends to update state, must acquire the job condition. The virDomainObjPtr lock is released while blocking on this condition variable. Once the job condition is acquired a method can safely release the virDomainObjPtr lock whenever it hits a piece of code which may sleep/wait, and re-acquire it after the sleep/ wait. * qemuMonitorPtr: Mutex Lock to be used when invoking any monitor command to ensure safety wrt any asynchronous events that may be dispatched from the monitor. It should be acquired before running a command. The job condition *MUST* be held before acquiring the monitor lock The virDomainObjPtr lock *MUST* be held before acquiring the monitor lock. The virDomainObjPtr lock *MUST* then be released when invoking the monitor command. The driver lock *MUST* be released when invoking the monitor commands. This ensures that the virDomainObjPtr & driver are both unlocked while sleeping/waiting for the monitor response. Helper methods -------------- To lock the driver qemuDriverLock() - Acquires the driver lock qemuDriverUnlock() - Releases the driver lock To lock the virDomainObjPtr virDomainObjLock() - Acquires the virDomainObjPtr lock virDomainObjUnlock() - Releases the virDomainObjPtr lock To acquire the job condition variable (int jobActive) qemuDomainObjBeginJob() (if driver is unlocked) - Increments ref count on virDomainObjPtr - Wait qemuDomainObjPrivate condition 'jobActive != 0' using virDomainObjPtr mutex - Sets jobActive to 1 qemuDomainObjBeginJobWithDriver() (if driver needs to be locked) - Unlocks driver - Increments ref count on virDomainObjPtr - Wait qemuDomainObjPrivate condition 'jobActive != 0' using virDomainObjPtr mutex - Sets jobActive to 1 - Unlocks virDomainObjPtr - Locks driver - Locks virDomainObjPtr NB: this variant is required in order to comply with lock ordering rules for virDomainObjPtr vs driver qemuDomainObjEndJob() - Set jobActive to 0 - Signal on qemuDomainObjPrivate condition - Decrements ref count on virDomainObjPtr To acquire the QEMU monitor lock qemuDomainObjEnterMonitor() - Acquires the qemuMonitorObjPtr lock - Releases the virDomainObjPtr lock qemuDomainObjExitMonitor() - Acquires the virDomainObjPtr lock - Releases the qemuMonitorObjPtr lock NB: caller must take care to drop the driver lock if neccessary Design patterns --------------- All driver methods must follow one of these design patterns to ensure thread safety and lock correctness. * Accessing or updating something with just the driver qemuDriverLock(driver); ...do work... qemuDriverUnlock(driver); * Accessing something directly todo with a virDomainObjPtr virDomainObjPtr obj; qemuDriverLock(driver); obj = virDomainFindByUUID(driver->domains, dom->uuid); qemuDriverUnlock(driver); ...do work... virDomainObjUnlock(obj); * Accessing something directly todo with a virDomainObjPtr and driver virDomainObjPtr obj; qemuDriverLock(driver); obj = virDomainFindByUUID(driver->domains, dom->uuid); ...do work... virDomainObjUnlock(obj); qemuDriverUnlock(driver); * Updating something directly todo with a virDomainObjPtr virDomainObjPtr obj; qemuDriverLock(driver); obj = virDomainFindByUUID(driver->domains, dom->uuid); qemuDriverUnlock(driver); qemuDomainObjBeginJob(obj); ...do work... qemuDomainObjEndJob(obj); virDomainObjUnlock(obj); * Invoking a monitor command on a virDomainObjPtr virDomainObjPtr obj; qemuDomainObjPrivatePtr priv; qemuDriverLockRO(driver); obj = virDomainFindByUUID(driver->domains, dom->uuid); qemuDriverUnlock(driver); qemuDomainObjBeginJob(obj); ...do prep work... qemuDomainObjEnterMonitor(obj); qemuMonitorXXXX(priv->mon); qemuDomainObjExitMonitor(obj); ...do final work... qemuDomainObjEndJob(obj); virDomainObjUnlock(obj); * Invoking a monitor command on a virDomainObjPtr with driver locked too virDomainObjPtr obj; qemuDomainObjPrivatePtr priv; qemuDriverLock(driver); obj = virDomainFindByUUID(driver->domains, dom->uuid); qemuDomainObjBeginJobWithDriver(obj); ...do prep work... qemuDomainObjEnterMonitor(obj); qemuDriverUnlock(driver); qemuMonitorXXXX(priv->mon); qemuDriverLock(driver); qemuDomainObjExitMonitor(obj); ...do final work... qemuDomainObjEndJob(obj); virDomainObjUnlock(obj); qemuDriverUnlock(driver); Summary ------- * Respect lock ordering rules: never lock driver if anything else is already locked * Don't hold locks in code which sleeps: unlock driver & virDomainObjPtr when using monitor -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

2 4

[libvirt] remote clients are non-interruptible
by John Levon 30 Oct '09

30 Oct '09

remoteIOEventLoop() has this: 6974 repoll: 6975 ret = poll(fds, ARRAY_CARDINALITY(fds), -1); 6976 if (ret < 0 && errno == EINTR) 6977 goto repoll; with the result that all clients using the remote driver cannot be control-C'd. Dan, you added this code it seems, why? regards john

4 3

[libvirt] Re: [virt-tools-list] Remote management over SSH fails to connect
by Cole Robinson 30 Oct '09

30 Oct '09

On 10/25/2009 10:25 AM, Jon Nordby wrote: > I get the following errors (skipping "system" yields the same): > [root@jon-laptop ~]# virsh -c qemu+ssh://10.0.0.2/system list > error: server closed connection > error: failed to connect to the hypervisor > > However, SSH and libvirtd both work on their own: > [root@jon-laptop ~]# ssh 10.0.0.2 > Last login: Sun Oct 25 14:06:07 2009 from 10.0.0.143 > [root@server0 ~]# virsh list > Id Name State > ---------------------------------- > 21 family-server running > > The connection is allowed in hosts.allow > libvirt 0.7.2 in both ends, kvm backend, Arch Linux > > What could be the source of this problem? And how can I find it? > ccing libvirt-list This could be an issue of 'nc' incompatibility. Does the 'nc' command on the remote system support the -U option? If you run the virsh command with LIBVIRT_DEBUG=1 virsh ... you should see the ssh command being run. See if running the command by hand produces an error. - Cole

3 2

[libvirt] Release 0.3.1 of libvirt-java
by Bryan Kearney 29 Oct '09

29 Oct '09

I have released version 0.3.1 of libvirt-java. Thanks to Thomas Treutner for his debugging help. You can get the latest from: Latest source tarball, rpm, and srpm: http://libvirt.org/sources/java/ Latest source code: git://libvirt.org/git/libvirt-java.git Maven: http://www.libvirt.org/maven2/ Latest rpms were submitted to Fedora Rawhide and F-11. iBiblio should update soon. -- bk

1 0

[libvirt] Got Windows guests?
by Richard W.M. Jones 29 Oct '09

29 Oct '09

Hello fellow Fedora, libvirt and libguestfs users, If you have any Windows guests, then you can help Fedora to support Windows guests better by spending a few minutes testing the Windows Registry feature we just added to libguestfs 1.0.75. You will need: - A Windows NT/200x/XP/Vista/7/... guest - Fedora 12 or Fedora Rawhide host - libguestfs-tools >= 1.0.75 (from updates or http://koji.fedoraproject.org/koji/packageinfo?packageID=8391 ) - a few minutes of your time The tests: (1) Run the virt-win-reg commands shown in the following web page, where "MyWinGuest" should be replaced with the name of the Windows guest as known to libvirt: http://libguestfs.org/virt-win-reg.1.html#examples Do the commands run without any errors? Does the output look sensible? If you have several Windows guests, please try as many different sorts as possible! (2) Download the registry binary files and try to convert them to XML: guestfish -i MyWinGuest --ro <<'EOF' download win:\windows\system32\config\software software download win:\windows\system32\config\system system download win:\windows\system32\config\sam sam download win:\windows\system32\config\security security EOF hivexml software > software.xml hivexml system > system.xml hivexml sam > sam.xml hivexml security > security.xml Do those commands run without error? If there's an error, try adding the hivexml -k option. Does the XML look complete? (Try running the XML through tidy -xml -indent -quiet < foo.xml | less ) I hope you don't find any bugs, but if you do: Send a reply to this message, or report a bug in Bugzilla: https://bugzilla.redhat.com/enter_bug.cgi?component=libguestfs&product=Virt… It's useful to include the following details: HIVEX_DEBUG=1 hivexml regfile 2>&1 > log.out The Registry file itself that is failing (but note that Registry files can contain sensitive data). It's also useful to have positive feedback ("it worked!"). Thanks for any testing you can give, and if you have any other suggestions for handling Windows guests from Fedora, please let me know. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/

1 0

[libvirt] Application using libvirt crashes when having concurrent TLS connections (gnutls problem)
by Thomas Treutner 29 Oct '09

29 Oct '09

Hi list, I was wondering about the status of this bug: https://bugzilla.redhat.com/show_bug.cgi?id=512367 Is it correct that this is a bug in the libvirt client? I ran into this today, as I've written a kind of a VM scheduler (ressource requirements, placement etc.) in Java (libvirt-java 0.3.0, libvirt 0.6.5) and this is a show-stopper for me right now. I had some troubles with the newest version of libvirt (it couldn't connect to Xen IIRC), so I don't want to mess my setup again for nothing. What is actually causing this problem resp. in which situation is libvirt broken? When a client uses more than one connection at the same time? Or when a client uses more than one connection to the same server at the same time? Are there any recommeded workarounds? TIA & kr, thomas

3 8

[libvirt] [PATCH] ESX: Change disk selection for datastore detection.
by Matthias Bolte 29 Oct '09

29 Oct '09

In order to register a new virtual machine the ESX driver needs to upload a VMX file to a datastore. Try to put this file beside the main VMDK file of the virtual machine. Change the disk selection for datastore detection to choose the first file-based harddisk instead of just the first disk. The first disk may be a CDROM disk and ISO images are normaly not located in the virtual machine's directory. * src/esx/esx_driver.c: change disk selection for datastore detection --- src/esx/esx_driver.c | 44 +++++++++++++++++++++++++++++++++----------- 1 files changed, 33 insertions(+), 11 deletions(-) diff --git a/src/esx/esx_driver.c b/src/esx/esx_driver.c index e063b46..a0efa5d 100644 --- a/src/esx/esx_driver.c +++ b/src/esx/esx_driver.c @@ -2410,6 +2410,8 @@ esxDomainDefineXML(virConnectPtr conn, const char *xml ATTRIBUTE_UNUSED) esxPrivate *priv = (esxPrivate *)conn->privateData; virDomainDefPtr def = NULL; char *vmx = NULL; + int i; + virDomainDiskDefPtr disk = NULL; esxVI_ObjectContent *virtualMachine = NULL; char *datastoreName = NULL; char *directoryName = NULL; @@ -2458,31 +2460,51 @@ esxDomainDefineXML(virConnectPtr conn, const char *xml ATTRIBUTE_UNUSED) goto failure; } - /* Build VMX datastore URL */ + /* + * Build VMX datastore URL. Use the source of the first file-based harddisk + * to deduce the datastore and path for the VMX file. Don't just use the + * first disk, because it may be CDROM disk and ISO images are normaly not + * located in the virtual machine's directory. This approach to deduce the + * datastore isn't perfect but should work in the majority of cases. + */ if (def->ndisks < 1) { ESX_ERROR(conn, VIR_ERR_INTERNAL_ERROR, - "Domain XML doesn't contain a disk, cannot deduce datastore " - "and path for VMX file"); + "Domain XML doesn't contain any disks, cannot deduce " + "datastore and path for VMX file"); goto failure; } - if (def->disks[0]->src == NULL) { + for (i = 0; i < def->ndisks; ++i) { + if (def->disks[i]->device == VIR_DOMAIN_DISK_DEVICE_DISK && + def->disks[i]->type == VIR_DOMAIN_DISK_TYPE_FILE) { + disk = def->disks[i]; + break; + } + } + + if (disk == NULL) { ESX_ERROR(conn, VIR_ERR_INTERNAL_ERROR, - "First disk has no source, cannot deduce datastore and path " - "for VMX file"); + "Domain XML doesn't contain any file-based harddisks, " + "cannot deduce datastore and path for VMX file"); goto failure; } - if (esxUtil_ParseDatastoreRelatedPath(conn, def->disks[0]->src, - &datastoreName, &directoryName, - &fileName) < 0) { + if (disk->src == NULL) { + ESX_ERROR(conn, VIR_ERR_INTERNAL_ERROR, + "First file-based harddisk has no source, cannot deduce " + "datastore and path for VMX file"); + goto failure; + } + + if (esxUtil_ParseDatastoreRelatedPath(conn, disk->src, &datastoreName, + &directoryName, &fileName) < 0) { goto failure; } if (! virFileHasSuffix(fileName, ".vmdk")) { ESX_ERROR(conn, VIR_ERR_INTERNAL_ERROR, - "Expecting source of first disk '%s' to be a VMDK image", - def->disks[0]->src); + "Expecting source '%s' of first file-based harddisk to be a " + "VMDK image", disk->src); goto failure; } -- 1.6.0.4

2 2

[libvirt] [PATCH] ESX: Fallback to the preliminary name if the datastore cannot be found.
by Matthias Bolte 28 Oct '09

28 Oct '09

This allows to use domain-xml-from-native with VMX files that reference unavailable datastores. * src/esx/esx_vmx.c: fallback to the preliminary name if the datastore cannot be found --- src/esx/esx_vmx.c | 43 +++++++++++++++++++++++-------------------- 1 files changed, 23 insertions(+), 20 deletions(-) diff --git a/src/esx/esx_vmx.c b/src/esx/esx_vmx.c index 8e2bdec..a9ff1b4 100644 --- a/src/esx/esx_vmx.c +++ b/src/esx/esx_vmx.c @@ -606,34 +606,37 @@ esxVMX_AbsolutePathToDatastoreRelatedPath(virConnectPtr conn, if (ctx != NULL) { if (esxVI_LookupDatastoreByName(conn, ctx, preliminaryDatastoreName, NULL, &datastore, - esxVI_Occurence_RequiredItem) < 0) { + esxVI_Occurence_OptionalItem) < 0) { goto failure; } - for (dynamicProperty = datastore->propSet; dynamicProperty != NULL; - dynamicProperty = dynamicProperty->_next) { - if (STREQ(dynamicProperty->name, "summary.accessible")) { - /* Ignore it */ - } else if (STREQ(dynamicProperty->name, "summary.name")) { - if (esxVI_AnyType_ExpectType(conn, dynamicProperty->val, - esxVI_Type_String) < 0) { - goto failure; + if (datastore != NULL) { + for (dynamicProperty = datastore->propSet; dynamicProperty != NULL; + dynamicProperty = dynamicProperty->_next) { + if (STREQ(dynamicProperty->name, "summary.accessible")) { + /* Ignore it */ + } else if (STREQ(dynamicProperty->name, "summary.name")) { + if (esxVI_AnyType_ExpectType(conn, dynamicProperty->val, + esxVI_Type_String) < 0) { + goto failure; + } + + datastoreName = dynamicProperty->val->string; + break; + } else if (STREQ(dynamicProperty->name, "summary.url")) { + /* Ignore it */ + } else { + VIR_WARN("Unexpected '%s' property", dynamicProperty->name); } - - datastoreName = dynamicProperty->val->string; - break; - } else if (STREQ(dynamicProperty->name, "summary.url")) { - /* Ignore it */ - } else { - VIR_WARN("Unexpected '%s' property", dynamicProperty->name); } } if (datastoreName == NULL) { - ESX_ERROR(conn, VIR_ERR_INTERNAL_ERROR, - "Could not retrieve datastore name for absolute path '%s'", - absolutePath); - goto failure; + VIR_WARN("Could not retrieve datastore name for absolute " + "path '%s', falling back to preliminary name '%s'", + absolutePath, preliminaryDatastoreName); + + datastoreName = preliminaryDatastoreName; } } else { datastoreName = preliminaryDatastoreName; -- 1.6.0.4

2 1