May 2009 - Devel - libvirt List Archives

[libvirt] [RFC][PATCH] lxc: fix for ns cgroups subsystem
by Ryota Ozaki 20 May '09

20 May '09

Hi, lxc does not work if ns cgroups subsystem is enabled because of two factors; one is that ns has a special rule to create a group[*] unlike other subsystems and the other is lxc controller creates a new namespace for /dev/pts prior to create a new group for a domain. Unfortunately the new namespace breaks the rule of ns and that prevents a lxc controller from creating a new group. This patch addresses the problem by creating a new group before creating a new namespace (i.e. call unshare syscall). Note that this patch is only for the case ns is enabled and current code works well if it disabled. However, I think this patch makes sense because not just a few users know much about cgroups and likely to enable all of subsystems without notions (i.e. mount cgroups without any options). [*] http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=k… Thanks, ozaki-r Signed-off-by: Ryota Ozaki <ozaki.ryota(a)gmail.com> >From 46531182708dc3eb132b14ce2f23fbc639430176 Mon Sep 17 00:00:00 2001 From: Ryota Ozaki <ozaki.ryota(a)gmail.com> Date: Fri, 8 May 2009 05:31:03 +0900 Subject: [PATCH] lxc: fix for ns cgroups subsystem lxc does not work if ns cgroups subsystem is enabled because of two factors; one is that ns has a special rule to create a group[*] unlike other subsystems and the other is lxc controller creates a new namespace for /dev/pts prior to create a new group for a domain. Unfortunately the new namespace breaks the rule of ns and that prevents a lxc controller from creating a new group. This patch addresses the problem by creating a new group before creating a new namespace (i.e. call unshare syscall). Note that this patch is only for the case ns is enabled and current code works well if it disabled. However, I think this patch makes sense because not just a few users know much about cgroups and likely to enable all of subsystems without notions (i.e. mount cgroups without any options). [*] http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=k… --- src/lxc_controller.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/lxc_controller.c b/src/lxc_controller.c index e0fb05d..1231817 100644 --- a/src/lxc_controller.c +++ b/src/lxc_controller.c @@ -458,6 +458,9 @@ lxcControllerRun(virDomainDefPtr def, goto cleanup; } + if (lxcSetContainerResources(def) < 0) + goto cleanup; + root = virDomainGetRootFilesystem(def); /* @@ -543,9 +546,6 @@ lxcControllerRun(virDomainDefPtr def, } - if (lxcSetContainerResources(def) < 0) - goto cleanup; - if ((container = lxcContainerStart(def, nveths, veths, -- 1.6.0.6

4 12

[libvirt] PATCH: 0/5: API for converting domain XML to/from native config formats
by Daniel P. Berrange 20 May '09

20 May '09

This is a long overdue update to the idea I described here http://www.redhat.com/archives/libvir-list/2009-April/msg00266.html And the initial patch posted here http://www.redhat.com/archives/libvir-list/2009-April/msg00401.html Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

3 17

[libvirt] [PATCH] lxc: use new error code in lxcDomainUndefine
by Amy Griffis 20 May '09

20 May '09

Update lxcDomainUndefine() to use VIR_ERR_INVALID_OPERATION instead of VIR_ERR_INTERNAL_ERROR. Signed-off-by: Amy Griffis <amy.griffis(a)hp.com> diff --git a/src/lxc_driver.c b/src/lxc_driver.c index 5392af1..64c735d 100644 --- a/src/lxc_driver.c +++ b/src/lxc_driver.c @@ -329,13 +329,13 @@ static int lxcDomainUndefine(virDomainPtr dom) } if (virDomainIsActive(vm)) { - lxcError(dom->conn, dom, VIR_ERR_INTERNAL_ERROR, + lxcError(dom->conn, dom, VIR_ERR_OPERATION_INVALID, "%s", _("cannot delete active domain")); goto cleanup; } if (!vm->persistent) { - lxcError(dom->conn, dom, VIR_ERR_INTERNAL_ERROR, + lxcError(dom->conn, dom, VIR_ERR_OPERATION_INVALID, "%s", _("cannot undefine transient domain")); goto cleanup; }

2 1

[libvirt] virStorageVolLookupByPath fails with "libvir: Storage error : invalid storage volume pointer in no storage vol with matching path"
by Francesco Latino 19 May '09

19 May '09

Hello, I have a running KVM virtual guest "demo1" with a defined disk at path /guests/mydisk.img. But when I run conn = virConnectOpenReadOnly(NULL); path = "/guests/mydisk.img"; virStorageVolLookupByPath(conn,path); I got the error " ibvir: Storage error : invalid storage volume pointer in no storage vol with matching path " what am I missing ? Thanks, Francesco

2 2

[libvirt] [PATCH] python: Better error reporting if 'import libvirtmod' fails
by Cole Robinson 19 May '09

19 May '09

Don't squash a possibly legitimate libvirtmod error (e.g. some from clashing libvirt.so versions) with 'Cannot import cygvirtmod' --- python/libvir.py | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/python/libvir.py b/python/libvir.py index 8a16dd0..3cda8dc 100644 --- a/python/libvir.py +++ b/python/libvir.py @@ -8,8 +8,12 @@ # On cygwin, the DLL is called cygvirtmod.dll try: import libvirtmod -except: - import cygvirtmod as libvirtmod +except ImportError, lib_e: + try: + import cygvirtmod as libvirtmod + except ImportError, cyg_e: + if str(cyg_e).count("No module named"): + raise lib_e import types -- 1.6.2.2

3 23

[libvirt] [PATCH v2 0/2] FS backend VolCreateXMLFrom implementation.
by Cole Robinson 19 May '09

19 May '09

The following 2 patches implement virStorageVolCreateXMLFrom for the storage driver and the FS storage backend. Changes from v1: - Mark origvol as building in driver implementation - Make sure driver is locked when relocking pool object - Simplify sparse file detection with memcmp - Allocate read buffer on the heap. Thanks, Cole Cole Robinson (2): Storage driver implementation for CreateXMLFrom VolumeCreateXMLFrom FS storage backend implementation. src/storage_backend.h | 2 + src/storage_backend_fs.c | 247 +++++++++++++++++++++++++++++++++++++++------- src/storage_driver.c | 159 +++++++++++++++++++++++++++++- 3 files changed, 369 insertions(+), 39 deletions(-)

3 6

[libvirt] libvirt API to write sectors on a device
by Francesco Latino 19 May '09

19 May '09

Hello,***I am able to compile libvirt samples on my KVM hypervisor. (I edited and recompiled info.c code) *I am creating a sample code to read/write disk devices. With method *virDomainBlockPeek I can** read sectors from a given device attached to a KVM virtual guest.**I am not able to* find an API to write sectors on a disk: am I missing something ?Is there such method to write sectors from a given virtual disk ? Thanks,Francesco

2 2

[libvirt] [PATCH 0/5] Interface Config Public APIs and remote stubs, with MAC addresses in string format
by Laine Stump 19 May '09

19 May '09

These patches are based on those I sent last week (the public virInterface*() API definition, the local plumbing, and the RPC glue), but with suggestions incorporated: 1) MAC address is always used in null-terminated ASCII string format. This eliminates any potential problems with extra long addresses. 2) no comments in libvirt.h 3) flags arguments are all unsigned. Item (1) caused changes in most of the patches, so I just regenerated the entire set. Is this the proper/expected thing to do, or should I be making patches of patches?

3 9

[libvirt] PATCH: Enable migration with QEMU >= 0.10.0
by Daniel P. Berrange 19 May '09

19 May '09

The KVM migration code was added to QEMU for the 0.10.0 release, so we should enable this in libvirt now. Daniel diff -r be7993675e07 src/qemu_conf.c --- a/src/qemu_conf.c Thu Apr 30 14:49:27 2009 +0100 +++ b/src/qemu_conf.c Thu Apr 30 15:08:45 2009 +0100 @@ -473,16 +473,13 @@ int qemudExtractVersionInfo(const char * /* * Handling of -incoming arg with varying features - * -incoming tcp (kvm >= 79) - * -incoming exec (kvm >= 80) + * -incoming tcp (kvm >= 79, qemu >= 0.10.0) + * -incoming exec (kvm >= 80, qemu >= 0.10.0) * -incoming stdio (all earlier kvm) * * NB, there was a pre-kvm-79 'tcp' support, but it * was broken, because it blocked the monitor console * while waiting for data, so pretend it doesn't exist - * - * XXX when next QEMU release after 0.9.1 arrives, - * we'll need to add MIGRATE_QEMU_TCP/EXEC here too */ if (kvm_version >= 79) { flags |= QEMUD_CMD_FLAG_MIGRATE_QEMU_TCP; @@ -490,6 +487,9 @@ int qemudExtractVersionInfo(const char * flags |= QEMUD_CMD_FLAG_MIGRATE_QEMU_EXEC; } else if (kvm_version > 0) { flags |= QEMUD_CMD_FLAG_MIGRATE_KVM_STDIO; + } else if (version >= 10000) { + flags |= QEMUD_CMD_FLAG_MIGRATE_QEMU_TCP; + flags |= QEMUD_CMD_FLAG_MIGRATE_QEMU_EXEC; } if (retversion) -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

3 8

[libvirt] PATCH: Declare QEMU migration capabilities
by Daniel P. Berrange 18 May '09

18 May '09

This patches adds the missing data in the capabilities XML about the QEMU driver's migration support. Specifically it declares support for both offline and live migrate, and registers the 'tcp' URI scheme. Daniel diff -r 00f2e113eca8 src/qemu_conf.c --- a/src/qemu_conf.c Mon May 18 12:38:18 2009 +0100 +++ b/src/qemu_conf.c Mon May 18 13:15:33 2009 +0100 @@ -371,7 +371,7 @@ virCapsPtr qemudCapsInit(void) { uname (&utsname); if ((caps = virCapabilitiesNew(utsname.machine, - 0, 0)) == NULL) + 1, 1)) == NULL) goto no_memory; /* Using KVM's mac prefix for QEMU too */ @@ -380,6 +380,9 @@ virCapsPtr qemudCapsInit(void) { if (virCapsInitNUMA(caps) < 0) goto no_memory; + virCapabilitiesAddHostMigrateTransport(caps, + "tcp"); + /* First the pure HVM guests */ for (i = 0 ; i < ARRAY_CARDINALITY(arch_info_hvm) ; i++) if (qemudCapsInitGuest(caps, -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

2 1

[libvirt] Domain Event Handlers
by Noah Zoschke 18 May '09

18 May '09

Hello, I have been playing around with the libvirt domain event handlers in Python and am seeing something surprising. I am using trunk libvirt and `event-test.py` with some more of the debugging strings uncommented. When I trigger my first event: virsh # define xppro-sp2-0.img.xml Domain xppro-sp2-0.img defined from xppro-sp2-0.img.xml The timeout and handle callbacks are called, but not my domainEventRegister handler: Invoking Timeout CB Invoking Handle CB When I trigger a second event: virsh # create xppro-sp2-0.img.xml Domain xppro-sp2-0.img created from xppro-sp2-0.img.xml my event handler for the first event is finally called: Invoking Timeout CB domainEventHandler EVENT: Domain xppro-sp2-0.img(-1) Added 1 Invoking Handle CB Likewise for a third event: virsh # destroy xppro-sp2-0.img Domain xppro-sp2-0.img destroyed Invoking Timeout CB domainEventHandler EVENT: Domain xppro-sp2-0.img(1) Started 0 Invoking Handle CB So my handler is always one event behind. This behavior is not intended is it? Am I missing either something in `event-test.py` or in how I am using virsh that would correct this behavior? Thank you, Noah

3 3

[libvirt] Usage of UUID
by Eduardo Otubo 18 May '09

18 May '09

Hello all, When I started to implement phyp driver, I use to handle the UUID as a normal 'unsigned char *' with a LPAR value called "logical_serial_num" that is a string in the format "067BE8B2", an identifier. In the last comments I got from danpb and DV, they called my attention to const string at UUID handling and some funtcions that must be implemented: domainDumpXML, listDefinedDomains and numOfDefinedDomains. My question is: Once I don't have information from my hypervisor about UUID, do I have to handle UUID at "list" or "list --all" commands? Or I should keep and handle this information only when defining a domaing at libvirt? I mean, in my point of view, there is no need to generate and store a UUID while just listing LPARs in the remote server. What I should do is to generate a random UUID (and store it) just when the user decides to define it at libvirt. Am I thinking right? Thanks for the help. []'s -- Eduardo Otubo Software Engineer Linux Technology Center IBM Systems & Technology Group Mobile: +55 19 8135 0885 otubo(a)linux.vnet.ibm.com

2 1

[libvirt] is qemu really needed for make rpm or is kvm sufficient?
by Gerrit Slomma 18 May '09

18 May '09

Hello I am curious if qemu is really needed to do "make rpm". As far as i have seen only the src.rpm has this dependency but neither of the install-rpms. What is this needed for in the src.rpm-package that do lack the install-packages? I haven't installed qemu on my hosts, only kvm and everything works fine, even live-migration works now without the extra qemu installed on the host i build the libvirt. But now "make rpm" fails due to the dependency for qemu. Couldn't either: A) the qemu-requirement be patched out of the requirements for the src.rpm (if possible) or B) the kvm-package has qemu and qemu-img added as "provides" (okay - this would be another mailing list) Gerrit

2 1

[libvirt] patch for virsh list
by Gerrit Slomma 16 May '09

16 May '09

Hello I thought it would be cool if virsh list behaves a bit more like virt-manager or xm list because as a system-administrator i want to know with one look what VMs are active, how much memory they commit and what their runtime is. I implemented this with the following patch which give such an sample output. What is your opinion and/or have i missed something? [root@rr016 ~]# virsh list --all Id Name Status Mem(kB) VCPUs Time(s) --------------------------------------------------------------- 1 rr019v3 laufend 1572864 2 15,3 2 rr019v4 laufend 1572864 2 11,7 - OpenSolaris08.11 ausschalten - Win7b_64 ausschalten --- a/src/virsh.c 2009-04-15 21:43:26.000000000 +0200 +++ b/src/virsh.c 2009-05-15 21:35:53.000000000 +0200 @@ -658,14 +658,16 @@ qsort(&names[0], maxname, sizeof(char*), namesorter); } } - vshPrintExtra(ctl, "%3s %-20s %s\n", _("Id"), _("Name"), _("State")); - vshPrintExtra(ctl, "----------------------------------\n"); + vshPrintExtra(ctl, "%3s %-20s %-15s %-8s %-5s %-7s\n", _("Id"), _("Name"), _("State"), + _("Mem(kB)"), _("VCPUs"), _("Time(s)")); + vshPrintExtra(ctl, "---------------------------------------------------------------\n"); for (i = 0; i < maxid; i++) { virDomainInfo info; virDomainPtr dom = virDomainLookupByID(ctl->conn, ids[i]); const char *state; - + double cpuUsed = 0; + /* this kind of work with domains is not atomic operation */ if (!dom) continue; @@ -675,10 +677,19 @@ else state = N_(vshDomainStateToString(info.state)); - vshPrint(ctl, "%3d %-20s %s\n", + if (info.cpuTime != 0) { + cpuUsed = info.cpuTime; + + cpuUsed /= 1000000000.0; + } + + vshPrint(ctl, "%3d %-20s %-15s %-8d %-5d %7.1f\n", virDomainGetID(dom), virDomainGetName(dom), - state); + state, + virDomainGetMaxMemory(dom), + info.nrVirtCpu, + cpuUsed); virDomainFree(dom); } for (i = 0; i < maxname; i++) {

2 2

[libvirt] USB device with product id 0x0000
by André Welter 16 May '09

16 May '09

Hi everyone! I'm new to this list and I joined because I stumbled upon a problem I could not find mentioned anywhere. I have a usb device (a bluetooth dongle) that reports a product id of 0x0000. When I try to attach this to a domain I get a "missing product" error. I checked the sources (0.6.3) and found in domain_conf.c lines 1645 following a check for a product id different from 0. If I remove this check and recompile everything works fine. I can attach the device to the virtual machine and it is recognized. I don't know the background of this check and if disabling it has any negative consequences so I would be happy about any comments. Regards André

2 1

[libvirt] (no subject)
by Eduardo Otubo 15 May '09

15 May '09

Hello all, I've been working on a libvirt extension to manage IBM's Power VMs (LPARs). The Power systems are managed through management console referred to as the HMC or using a management partition (IVM). Both HMC and IVM runs an SSH, then you can reach it via command line, and an HTTP server, then you can reach it via web browser. The protocol between the console and the partition (LPAR) is not disclosed, therefore I propose the driver to execute commands remoetly over an SSH connection to the consoles to manage IBM LPARs. The patch attached is the first scratch of the driver that will interact with HMC over a SSH connection. The URI model that is used in virsh command line is: virsh --conect phyp://$user@$server Some known issues are: * Next step is to make the URI like this: phyp://$user@ $HMC/@managed_system. Almost finished. What it takes now is $server = $HMC = $managed_system. * Next features in my TODO list are "resume", "stop" and "reboot" the LPAR. Any comments are welcome. Thanks, -- Eduardo Otubo Software Engineer Linux Technology Center IBM Systems & Technology Group Mobile: +55 19 8135 0885 otubo(a)linux.vnet.ibm.com

4 18

[libvirt] [PATCH 3/3] networking API for hostonly networks in VirtualBox driver in libvirt
by Pritesh Kothari 15 May '09

15 May '09

Hi All, As discussed on the list resending the networking patch's. the patch's are as below: [PATCH 1/3]: contains support for "Internal" network in libvirt [PATCH 2/3]: contains support for "Host only" and "Internal" networks in VirtualBox driver [PATCH 3/3]: contains networking API for hostonly networks in VirtualBox driver in libvirt (it contains all the fix's proposed on list along with two extra *DefinedNetworks functions) The patches work as per cvs checkin today. (git SHA1 ID: 9136ae2056b45ea83854d3fe31d860f645b8c883) Regards, Pritesh

3 8

[libvirt] [PATCH 1/1] Support for rdp/desktop types in graphics tag
by Pritesh Kothari 15 May '09

15 May '09

Hi All, As per the discussion earlier on the list I have modified the rdp type and added a new desktop type and posting the patch for same. Regards, Pritesh

3 14

[libvirt] [PATCH 0/2] Implement nodedev destroy API
by David Allan 15 May '09

15 May '09

Here is a patch that implements the node device destroy API call, as well as the suggestions that Dan made in his feedback on the patch implementing create. The patch also makes it not necessary for the caller of the node device create function to specify a name for the device and a host number for the adapter in the XML. The change to do so is kind of kludgy, though, IMO, as it involved changing the function signature of chain of internal function calls to differentiate a device that is being created from an existing device. I'm torn as to whether I think that's a better solution than requiring the caller of the nodedev create API to specify a name that will be ignored. One benefit of requiring the user to specify the name is that it is used in some error messages, so I can argue it both ways. Opinions very welcome. I'd particularly like someone in the Solaris world to make sure that the code builds as there is some #ifdef __linux__ in the patch. I tried to spin up Opensolaris to try it out but ran up against bug 6784591 and a lack of time. I've also tried to make it relatively easy for someone to add support for non-Linux OSes, so a patch there would be appreciated. Dave

3 4

[libvirt] how to configure keymap & keyboard layout with virt-manager
by libvirt＠pivert.org 14 May '09

14 May '09

Hi, I'm fighting to configure the keyboard on libvirt (kvm) (0.6.1-0ubuntu5). I'm using a belgian keyboard, so, my question is : - Which keyboard layout should I place in the xml file (or through the vm display adapter configuration). - Which keyboard layout should I use in the vm ? PS: When I use 'be' as keymap on the vnc line, the libvirtd just segfault at startup. Regards, -- François Delpierre

2 2

[libvirt] PATCH: Fix misc locking bugs, and add lock checker
by Daniel P. Berrange 14 May '09

14 May '09

This patch fixes a number of locking bugs, some serious, some not. It also adds the CIL lock checker I previously wrote. It is improved since last time, because it explicitly looks for the virDriver static variables and uses that to extract list of functions to be checked. This removes a huge number of false positives. It also now checks for a couple of dead lock scenarios, in addition to previous checks for lock ordering correctness. The serious locking bugs - qemudDomainMigratePrepare2: didn't unlock VM causing deadlock - storagePoolRefresh: unlocked the driver too early - storageVolumeCreateXML: locked the pool without having locked driver - umlDomainGetAutostart: missing unlock call with later deadlock The lock checker requires that you can ocaml and ocaml-cil installed. You then need to run configure using '--enable-test-locking' and do a full make clean, followed by make. This is because it has to set some special CFLAGS to disable booleans, and keep temporary files around. Once built, you can run 'tests/object-locking 2>/dev/null' to validate It should not report any errors, with this patch now applied .hgignore | 6 b/tests/object-locking.ml | 828 ++++++++++++++++++++++++++++++++++++++++++++++ configure.in | 16 src/.cvsignore | 2 src/.gitignore | 2 src/Makefile.am | 5 src/network_driver.c | 6 src/qemu_driver.c | 39 -- src/storage_driver.c | 7 src/test.c | 3 src/uml_driver.c | 4 tests/.cvsignore | 4 tests/.gitignore | 4 tests/Makefile.am | 27 + 14 files changed, 920 insertions(+), 33 deletions(-) Daniel diff -r 71a7d1d0ad9b .hgignore --- a/.hgignore Thu May 14 17:38:11 2009 +0100 +++ b/.hgignore Thu May 14 17:47:01 2009 +0100 @@ -235,9 +235,11 @@ src/*.exe src/*.gcda src/*.gcno src/*.gcov +src/*.i src/*.la src/*.lo src/*.loT +src/*.s src/.deps src/.libs src/Makefile @@ -264,6 +266,10 @@ tests/conftest tests/eventtest tests/nodedevxml2xmltest tests/nodeinfotest +tests/object-locking +tests/object-locking-files.txt +tests/object-locking.cmi +tests/object-locking.cmx tests/qemuxml2argvtest tests/qemuxml2xmltest tests/qparamtest diff -r 71a7d1d0ad9b configure.in --- a/configure.in Thu May 14 17:38:11 2009 +0100 +++ b/configure.in Thu May 14 17:47:01 2009 +0100 @@ -1132,6 +1132,22 @@ if test "${enable_oom}" = yes; then AC_DEFINE([TEST_OOM], 1, [Whether malloc OOM checking is enabled]) fi + +AC_ARG_ENABLE([test-locking], +[ --enable-test-locking thread locking tests using CIL], +[case "${enableval}" in + yes|no) ;; + *) AC_MSG_ERROR([bad value ${enableval} for test-locking option]) ;; + esac], + [enableval=no]) +enable_locking=$enableval + +if test "$enable_locking" = "yes"; then + LOCK_CHECKING_CFLAGS="-Dbool=char -D_Bool=char -save-temps" + AC_SUBST([LOCK_CHECKING_CFLAGS]) +fi +AM_CONDITIONAL([WITH_CIL],[test "$enable_locking" = "yes"]) + dnl Enable building the proxy? AC_ARG_WITH([xen-proxy], diff -r 71a7d1d0ad9b src/.cvsignore --- a/src/.cvsignore Thu May 14 17:38:11 2009 +0100 +++ b/src/.cvsignore Thu May 14 17:47:01 2009 +0100 @@ -16,3 +16,5 @@ libvirt_lxc virsh-net-edit.c virsh-pool-edit.c libvirt.syms +*.i +*.s diff -r 71a7d1d0ad9b src/.gitignore --- a/src/.gitignore Thu May 14 17:38:11 2009 +0100 +++ b/src/.gitignore Thu May 14 17:47:01 2009 +0100 @@ -16,3 +16,5 @@ libvirt_lxc virsh-net-edit.c virsh-pool-edit.c libvirt.syms +*.i +*.s diff -r 71a7d1d0ad9b src/Makefile.am --- a/src/Makefile.am Thu May 14 17:38:11 2009 +0100 +++ b/src/Makefile.am Thu May 14 17:47:01 2009 +0100 @@ -16,7 +16,8 @@ INCLUDES = \ -DLOCALEBASEDIR=\""$(datadir)/locale"\" \ -DLOCAL_STATE_DIR=\""$(localstatedir)"\" \ -DGETTEXT_PACKAGE=\"$(PACKAGE)\" \ - $(WARN_CFLAGS) + $(WARN_CFLAGS) \ + $(LOCK_CHECKING_CFLAGS) confdir = $(sysconfdir)/libvirt/ conf_DATA = qemu.conf @@ -662,5 +663,5 @@ if WITH_NETWORK endif -CLEANFILES = *.gcov .libs/*.gcda .libs/*.gcno *.gcno *.gcda +CLEANFILES = *.gcov .libs/*.gcda .libs/*.gcno *.gcno *.gcda *.i *.s DISTCLEANFILES = $(BUILT_SOURCES) diff -r 71a7d1d0ad9b src/network_driver.c --- a/src/network_driver.c Thu May 14 17:38:11 2009 +0100 +++ b/src/network_driver.c Thu May 14 17:47:01 2009 +0100 @@ -1217,7 +1217,6 @@ static int networkStart(virNetworkPtr ne networkDriverLock(driver); network = virNetworkFindByUUID(&driver->networks, net->uuid); - networkDriverUnlock(driver); if (!network) { networkReportError(net->conn, NULL, net, VIR_ERR_INVALID_NETWORK, @@ -1230,6 +1229,7 @@ static int networkStart(virNetworkPtr ne cleanup: if (network) virNetworkObjUnlock(network); + networkDriverUnlock(driver); return ret; } @@ -1240,7 +1240,6 @@ static int networkDestroy(virNetworkPtr networkDriverLock(driver); network = virNetworkFindByUUID(&driver->networks, net->uuid); - networkDriverUnlock(driver); if (!network) { networkReportError(net->conn, NULL, net, VIR_ERR_INVALID_NETWORK, @@ -1258,6 +1257,7 @@ static int networkDestroy(virNetworkPtr cleanup: if (network) virNetworkObjUnlock(network); + networkDriverUnlock(driver); return ret; } @@ -1349,7 +1349,6 @@ static int networkSetAutostart(virNetwor networkDriverLock(driver); network = virNetworkFindByUUID(&driver->networks, net->uuid); - networkDriverUnlock(driver); if (!network) { networkReportError(net->conn, NULL, net, VIR_ERR_INVALID_NETWORK, @@ -1399,6 +1398,7 @@ cleanup: VIR_FREE(autostartLink); if (network) virNetworkObjUnlock(network); + networkDriverUnlock(driver); return ret; } diff -r 71a7d1d0ad9b src/qemu_driver.c --- a/src/qemu_driver.c Thu May 14 17:38:11 2009 +0100 +++ b/src/qemu_driver.c Thu May 14 17:47:01 2009 +0100 @@ -1811,9 +1811,10 @@ static virDrvOpenStatus qemudOpen(virCon static int qemudClose(virConnectPtr conn) { struct qemud_driver *driver = conn->privateData; + qemuDriverLock(driver); /* Get rid of callbacks registered for this conn */ virDomainEventCallbackListRemoveConn(conn, driver->domainEventCallbacks); - + qemuDriverUnlock(driver); conn->privateData = NULL; return 0; @@ -2229,7 +2230,6 @@ static int qemudDomainSuspend(virDomainP qemuDriverLock(driver); vm = virDomainFindByUUID(&driver->domains, dom->uuid); - qemuDriverUnlock(driver); if (!vm) { char uuidstr[VIR_UUID_STRING_BUFLEN]; @@ -2264,11 +2264,9 @@ cleanup: if (vm) virDomainObjUnlock(vm); - if (event) { - qemuDriverLock(driver); - qemuDomainEventQueue(driver, event); - qemuDriverUnlock(driver); - } + if (event) + qemuDomainEventQueue(driver, event); + qemuDriverUnlock(driver); return ret; } @@ -2282,7 +2280,6 @@ static int qemudDomainResume(virDomainPt qemuDriverLock(driver); vm = virDomainFindByUUID(&driver->domains, dom->uuid); - qemuDriverUnlock(driver); if (!vm) { char uuidstr[VIR_UUID_STRING_BUFLEN]; @@ -2316,11 +2313,9 @@ static int qemudDomainResume(virDomainPt cleanup: if (vm) virDomainObjUnlock(vm); - if (event) { - qemuDriverLock(driver); - qemuDomainEventQueue(driver, event); - qemuDriverUnlock(driver); - } + if (event) + qemuDomainEventQueue(driver, event); + qemuDriverUnlock(driver); return ret; } @@ -3153,7 +3148,6 @@ static int qemudDomainGetSecurityLabel(v qemuDriverLock(driver); vm = virDomainFindByUUID(&driver->domains, dom->uuid); - qemuDriverUnlock(driver); if (!vm) { char uuidstr[VIR_UUID_STRING_BUFLEN]; @@ -3199,6 +3193,7 @@ static int qemudDomainGetSecurityLabel(v cleanup: if (vm) virDomainObjUnlock(vm); + qemuDriverUnlock(driver); return ret; } @@ -3617,7 +3612,6 @@ static int qemudDomainStart(virDomainPtr qemuDriverLock(driver); vm = virDomainFindByUUID(&driver->domains, dom->uuid); - qemuDriverUnlock(driver); if (!vm) { char uuidstr[VIR_UUID_STRING_BUFLEN]; @@ -3636,11 +3630,9 @@ static int qemudDomainStart(virDomainPtr cleanup: if (vm) virDomainObjUnlock(vm); - if (event) { - qemuDriverLock(driver); - qemuDomainEventQueue(driver, event); - qemuDriverUnlock(driver); - } + if (event) + qemuDomainEventQueue(driver, event); + qemuDriverUnlock(driver); return ret; } @@ -4144,7 +4136,6 @@ static int qemudDomainAttachDevice(virDo dev = virDomainDeviceDefParse(dom->conn, driver->caps, vm->def, xml, VIR_DOMAIN_XML_INACTIVE); - qemuDriverUnlock(driver); if (dev == NULL) goto cleanup; @@ -4197,6 +4188,7 @@ cleanup: virDomainDeviceDefFree(dev); if (vm) virDomainObjUnlock(vm); + qemuDriverUnlock(driver); return ret; } @@ -4296,7 +4288,6 @@ static int qemudDomainDetachDevice(virDo dev = virDomainDeviceDefParse(dom->conn, driver->caps, vm->def, xml, VIR_DOMAIN_XML_INACTIVE); - qemuDriverUnlock(driver); if (dev == NULL) goto cleanup; @@ -4320,6 +4311,7 @@ cleanup: virDomainDeviceDefFree(dev); if (vm) virDomainObjUnlock(vm); + qemuDriverUnlock(driver); return ret; } @@ -4359,7 +4351,6 @@ static int qemudDomainSetAutostart(virDo qemuDriverLock(driver); vm = virDomainFindByUUID(&driver->domains, dom->uuid); - qemuDriverUnlock(driver); if (!vm) { char uuidstr[VIR_UUID_STRING_BUFLEN]; @@ -4417,6 +4408,7 @@ cleanup: VIR_FREE(autostartLink); if (vm) virDomainObjUnlock(vm); + qemuDriverUnlock(driver); return ret; } @@ -4985,6 +4977,7 @@ qemudDomainMigratePrepare2 (virConnectPt vm->def->name); goto cleanup; } + virDomainObjUnlock(vm); } if (!(vm = virDomainAssignDef(dconn, diff -r 71a7d1d0ad9b src/storage_driver.c --- a/src/storage_driver.c Thu May 14 17:38:11 2009 +0100 +++ b/src/storage_driver.c Thu May 14 17:47:01 2009 +0100 @@ -792,7 +792,6 @@ storagePoolRefresh(virStoragePoolPtr obj storageDriverLock(driver); pool = virStoragePoolObjFindByUUID(&driver->pools, obj->uuid); - storageDriverUnlock(driver); if (!pool) { virStorageReportError(obj->conn, VIR_ERR_INVALID_STORAGE_POOL, @@ -834,6 +833,7 @@ storagePoolRefresh(virStoragePoolPtr obj cleanup: if (pool) virStoragePoolObjUnlock(pool); + storageDriverUnlock(driver); return ret; } @@ -939,7 +939,6 @@ storagePoolSetAutostart(virStoragePoolPt storageDriverLock(driver); pool = virStoragePoolObjFindByUUID(&driver->pools, obj->uuid); - storageDriverUnlock(driver); if (!pool) { virStorageReportError(obj->conn, VIR_ERR_INVALID_STORAGE_POOL, @@ -988,6 +987,7 @@ storagePoolSetAutostart(virStoragePoolPt cleanup: if (pool) virStoragePoolObjUnlock(pool); + storageDriverUnlock(driver); return ret; } @@ -1274,7 +1274,10 @@ storageVolumeCreateXML(virStoragePoolPtr buildret = backend->buildVol(obj->conn, buildvoldef); + storageDriverLock(driver); virStoragePoolObjLock(pool); + storageDriverUnlock(driver); + voldef->building = 0; pool->asyncjobs--; diff -r 71a7d1d0ad9b src/test.c --- a/src/test.c Thu May 14 17:38:11 2009 +0100 +++ b/src/test.c Thu May 14 17:47:01 2009 +0100 @@ -659,10 +659,12 @@ static virDrvOpenStatus testOpen(virConn if (ret == VIR_DRV_OPEN_SUCCESS) { testConnPtr privconn = conn->privateData; + testDriverLock(privconn); /* Init callback list */ if (VIR_ALLOC(privconn->domainEventCallbacks) < 0 || !(privconn->domainEventQueue = virDomainEventQueueNew())) { virReportOOMError(NULL); + testDriverUnlock(privconn); testClose(conn); return VIR_DRV_OPEN_ERROR; } @@ -671,6 +673,7 @@ static virDrvOpenStatus testOpen(virConn virEventAddTimeout(-1, testDomainEventFlush, privconn, NULL)) < 0) DEBUG0("virEventAddTimeout failed: No addTimeoutImpl defined. " "continuing without events."); + testDriverUnlock(privconn); } return (ret); diff -r 71a7d1d0ad9b src/uml_driver.c --- a/src/uml_driver.c Thu May 14 17:38:11 2009 +0100 +++ b/src/uml_driver.c Thu May 14 17:47:01 2009 +0100 @@ -1553,7 +1553,6 @@ static int umlDomainStart(virDomainPtr d umlDriverLock(driver); vm = virDomainFindByUUID(&driver->domains, dom->uuid); - umlDriverUnlock(driver); if (!vm) { umlReportError(dom->conn, dom, NULL, VIR_ERR_INVALID_DOMAIN, @@ -1672,6 +1671,7 @@ static int umlDomainGetAutostart(virDoma cleanup: if (vm) virDomainObjUnlock(vm); + umlDriverUnlock(driver); return ret; } @@ -1684,7 +1684,6 @@ static int umlDomainSetAutostart(virDoma umlDriverLock(driver); vm = virDomainFindByUUID(&driver->domains, dom->uuid); - umlDriverUnlock(driver); if (!vm) { umlReportError(dom->conn, dom, NULL, VIR_ERR_INVALID_DOMAIN, @@ -1740,6 +1739,7 @@ cleanup: VIR_FREE(autostartLink); if (vm) virDomainObjUnlock(vm); + umlDriverUnlock(driver); return ret; } diff -r 71a7d1d0ad9b tests/.cvsignore --- a/tests/.cvsignore Thu May 14 17:38:11 2009 +0100 +++ b/tests/.cvsignore Thu May 14 17:47:01 2009 +0100 @@ -20,3 +20,7 @@ eventtest *.gcda *.gcno *.exe +object-locking +object-locking.cmi +object-locking.cmx +object-locking-files.txt diff -r 71a7d1d0ad9b tests/.gitignore --- a/tests/.gitignore Thu May 14 17:38:11 2009 +0100 +++ b/tests/.gitignore Thu May 14 17:47:01 2009 +0100 @@ -20,3 +20,7 @@ eventtest *.gcda *.gcno *.exe +object-locking +object-locking.cmi +object-locking.cmx +object-locking-files.txt diff -r 71a7d1d0ad9b tests/Makefile.am --- a/tests/Makefile.am Thu May 14 17:38:11 2009 +0100 +++ b/tests/Makefile.am Thu May 14 17:47:01 2009 +0100 @@ -68,6 +68,10 @@ if WITH_SECDRIVER_SELINUX noinst_PROGRAMS += seclabeltest endif +if WITH_CIL +noinst_PROGRAMS += object-locking +endif + noinst_PROGRAMS += nodedevxml2xmltest test_scripts = \ @@ -234,4 +238,25 @@ eventtest_SOURCES = \ eventtest_LDADD = -lrt $(LDADDS) endif -CLEANFILES = *.cov *.gcov .libs/*.gcda .libs/*.gcno *.gcno *.gcda +if WITH_CIL +CILOPTFLAGS = +CILOPTINCS = +CILOPTPACKAGES = -package unix,str,cil +CILOPTLIBS = -linkpkg + +object_locking_SOURCES = object-locking.ml + +%.cmx: %.ml + ocamlfind ocamlopt $(CILOPTFLAGS) $(CILOPTINCS) $(CILOPTPACKAGES) -c $< + +object-locking: object-locking.cmx object-locking-files.txt + ocamlfind ocamlopt $(CILOPTFLAGS) $(CILOPTINCS) $(CILOPTPACKAGES) $(CILOPTLIBS) $< -o $@ + +object-locking-files.txt: + find $(top_builddir)/src/ -name '*.i' > $@ + +else +EXTRA_DIST += object-locking.ml +endif + +CLEANFILES = *.cov *.gcov .libs/*.gcda .libs/*.gcno *.gcno *.gcda object-locking-files.txt diff -r 71a7d1d0ad9b tests/object-locking.ml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tests/object-locking.ml Thu May 14 17:47:01 2009 +0100 @@ -0,0 +1,828 @@ +(* + * Analyse libvirt driver API methods for mutex locking mistakes + * + * Copyright 2008-2009 Red Hat, Inc + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Author: Daniel P. Berrange <berrange(a)redhat.com> + *) + +open Pretty +open Cil + +(* + * Convenient routine to load the contents of a file into + * a list of strings + *) +let input_file filename = + let chan = open_in filename in + let lines = ref [] in + try while true; do lines := input_line chan :: !lines done; [] + with + End_of_file -> close_in chan; List.rev !lines + +module DF = Dataflow +module UD = Usedef +module IH = Inthash +module E = Errormsg +module VS = UD.VS + +let debug = ref false + + +let driverTables = [ + "virDriver"; + "virNetworkDriver"; + "virStorageDriver"; + "virDeviceMonitor"; +(* "virStateDriver"; Disable for now, since shutdown/startup have wierd locking rules *) +] + +(* + * This is the list of all libvirt methods which return + * pointers to locked objects + *) +let lockedObjMethods = [ + "virDomainFindByID"; + "virDomainFindByUUID"; + "virDomainFindByName"; + "virDomainAssignDef"; + + "virNetworkFindByUUID"; + "virNetworkFindByName"; + "virNetworkAssignDef"; + + "virNodeDeviceFindByName"; + "virNodeDeviceAssignDef"; + + "virStoragePoolObjFindByUUID"; + "virStoragePoolObjFindByName"; + "virStoragePoolObjAssignDef" +] + + +(* + * This is the list of all libvirt methods which + * can release an object lock. Technically we + * ought to pair them up correctly with previous + * ones, but the compiler can already complain + * about passing a virNetworkObjPtr to a virDomainObjUnlock + * method so lets be lazy + *) +let objectLockMethods = [ + "virDomainObjLock"; + "virNetworkObjLock"; + "virStoragePoolObjLock"; + "virNodeDevObjLock" +] + +(* + * This is the list of all libvirt methods which + * can release an object lock. Technically we + * ought to pair them up correctly with previous + * ones, but the compiler can already complain + * about passing a virNetworkObjPtr to a virDomainObjUnlock + * method so lets be lazy + *) +let objectUnlockMethods = [ + "virDomainObjUnlock"; + "virNetworkObjUnlock"; + "virStoragePoolObjUnlock"; + "virNodeDevObjUnlock" +] + +(* + * The data types that the previous two sets of + * methods operate on + *) +let lockableObjects = [ + "virDomainObjPtr"; + "virNetworkObjPtr"; + "virStoragePoolObjPtr"; + "virNodeDevObjPtr" +] + + + +(* + * The methods which globally lock an entire driver + *) +let driverLockMethods = [ + "qemuDriverLock"; + "openvzDriverLock"; + "testDriverLock"; + "lxcDriverLock"; + "umlDriverLock"; + "nodedevDriverLock"; + "networkDriverLock"; + "storageDriverLock" +] + +(* + * The methods which globally unlock an entire driver + *) +let driverUnlockMethods = [ + "qemuDriverUnlock"; + "openvzDriverUnlock"; + "testDriverUnlock"; + "lxcDriverUnlock"; + "umlDriverUnlock"; + "nodedevDriverUnlock"; + "networkDriverUnlock"; + "storageDriverUnlock" +] + +(* + * The data types that the previous two sets of + * methods operate on. These may be structs or + * typedefs, we don't care + *) +let lockableDrivers = [ + "qemud_driver"; + "openvz_driver"; + "testConnPtr"; + "lxc_driver_t"; + "uml_driver"; + "virStorageDriverStatePtr"; + "network_driver"; + "virDeviceMonitorState"; +] + + +let isFuncCallLval lval methodList = + match lval with + Var vi, o -> + List.mem vi.vname methodList + | _ -> false + +let isFuncCallExp exp methodList = + match exp with + Lval lval -> + isFuncCallLval lval methodList + | _ -> false + +let isFuncCallInstr instr methodList = + match instr with + Call (retval,exp,explist,srcloc) -> + isFuncCallExp exp methodList + | _ -> false + + + +let findDriverFunc init = + match init with + SingleInit (exp) -> ( + match exp with + AddrOf (lval) -> ( + match lval with + Var vi, o -> + true + | _ -> false + ) + | _ -> false + ) + | _ ->false + +let findDriverFuncs init = + match init with + CompoundInit (typ, list) -> + List.filter ( + fun l -> + match l with + (offset, init) -> + findDriverFunc init + + ) list; + | _ -> ([]) + + +let getDriverFuncs initinfo = + match initinfo.init with + Some (i) -> + let ls = findDriverFuncs i in + ls + | _ -> [] + +let getDriverFuncName init = + match init with + SingleInit (exp) -> ( + match exp with + AddrOf (lval) -> ( + match lval with + Var vi, o -> + + vi.vname + | _ -> "unknown" + ) + | _ -> "unknown" + ) + | _ -> "unknown" + + +let getDriverFuncNames initinfo = + List.map ( + fun l -> + match l with + (offset, init) -> + getDriverFuncName init + ) (getDriverFuncs initinfo) + + +(* + * Convenience methods which take a Cil.Instr object + * and ask whether its associated with one of the + * method sets defined earlier + *) +let isObjectFetchCall instr = + isFuncCallInstr instr lockedObjMethods + +let isObjectLockCall instr = + isFuncCallInstr instr objectLockMethods + +let isObjectUnlockCall instr = + isFuncCallInstr instr objectUnlockMethods + +let isDriverLockCall instr = + isFuncCallInstr instr driverLockMethods + +let isDriverUnlockCall instr = + isFuncCallInstr instr driverUnlockMethods + + +let isWantedType typ typeList = + match typ with + TNamed (tinfo, attrs) -> + List.mem tinfo.tname typeList + | TPtr (ptrtyp, attrs) -> + let f = match ptrtyp with + TNamed (tinfo2, attrs) -> + List.mem tinfo2.tname typeList + | TComp (cinfo, attrs) -> + List.mem cinfo.cname typeList + | _ -> + false in + f + | _ -> false + +(* + * Convenience methods which take a Cil.Varinfo object + * and ask whether it matches a variable datatype that + * we're interested in tracking for locking purposes + *) +let isLockableObjectVar varinfo = + isWantedType varinfo.vtype lockableObjects + +let isLockableDriverVar varinfo = + isWantedType varinfo.vtype lockableDrivers + +let isDriverTable varinfo = + isWantedType varinfo.vtype driverTables + + +(* + * Take a Cil.Exp object (ie an expression) and see whether + * the expression corresponds to a check for NULL against + * one of our interesting objects + * eg + * + * if (!vm) ... + * + * For a variable 'virDomainObjPtr vm' + *) +let isLockableThingNull exp funcheck = + match exp with + | UnOp (op,exp,typ) -> ( + match op with + LNot -> ( + match exp with + Lval (lhost, off) -> ( + match lhost with + Var vi -> + funcheck vi + | _ -> false + ) + | _ -> false + ) + | _ -> false + ) + | _ -> + false + +let isLockableObjectNull exp = + isLockableThingNull exp isLockableObjectVar + +let isLockableDriverNull exp = + isLockableThingNull exp isLockableDriverVar + + +(* + * Prior to validating a function, intialize these + * to VS.empty + * + * They contain the list of driver and object variables + * objects declared as local variables + * + *) +let lockableObjs: VS.t ref = ref VS.empty +let lockableDriver: VS.t ref = ref VS.empty + +(* + * Given a Cil.Instr object (ie a single instruction), get + * the list of all used & defined variables associated with + * it. Then caculate intersection with the driver and object + * variables we're interested in tracking and return four sets + * + * List of used driver variables + * List of defined driver variables + * List of used object variables + * List of defined object variables + *) +let computeUseDefState i = + let u, d = UD.computeUseDefInstr i in + let useo = VS.inter u !lockableObjs in + let defo = VS.inter d !lockableObjs in + let used = VS.inter u !lockableDriver in + let defd = VS.inter d !lockableDriver in + (used, defd, useo, defo) + + +(* Some crude helpers for debugging this horrible code *) +let printVI vi = + ignore(printf " | %a %s\n" d_type vi.vtype vi.vname) + +let printVS vs = + VS.iter printVI vs + + +let prettyprint2 stmdat () (_, ld, ud, lo, ui, uud, uuo, loud, ldlo, dead) = + text "" + + +type ilist = Cil.instr list + +(* + * This module implements the Cil.DataFlow.ForwardsTransfer + * interface. This is what 'does the interesting stuff' + * when walking over a function's code paths + *) +module Locking = struct + let name = "Locking" + let debug = debug + + (* + * Our state currently consists of + * + * The set of driver variables that are locked + * The set of driver variables that are unlocked + * The set of object variables that are locked + * The set of object variables that are unlocked + * + * Lists of Cil.Instr for: + * + * Instrs using an unlocked driver variable + * Instrs using an unlocked object variable + * Instrs locking a object variable while not holding a locked driver variable + * Instrs locking a driver variable while holding a locked object variable + * Instrs causing deadlock by fetching a lock object, while an object is already locked + * + *) + type t = (unit * VS.t * VS.t * VS.t * VS.t * ilist * ilist * ilist * ilist * ilist) + + (* This holds an instance of our state data, per statement *) + let stmtStartData = IH.create 32 + + let pretty = + prettyprint2 stmtStartData + + let copy (_, ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) = + ((), ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) + + let computeFirstPredecessor stm (_, ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) = + ((), ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) + + + (* + * Merge existing state for a statement, with new state + * + * If new and old state is the same, this returns None, + * If they are different, then returns the union. + *) + let combinePredecessors (stm:stmt) ~(old:t) ((_, ldn, udn, lon, uon, uudn, uuon, loudn, ldlon, deadn):t) = + match old with (_, ldo, udo, loo,uoo, uudo, uuoo, loudo, ldloo, deado)-> begin + let lde= (VS.equal ldo ldn) || ((VS.is_empty ldo) && (VS.is_empty ldn)) in + let ude= VS.equal udo udn || ((VS.is_empty udo) && (VS.is_empty udn)) in + let loe= VS.equal loo lon || ((VS.is_empty loo) && (VS.is_empty lon)) in + let uoe= VS.equal uoo uon || ((VS.is_empty uoo) && (VS.is_empty uon)) in + + if lde && ude && loe && uoe then + None + else ( + let ldret = VS.union ldo ldn in + let udret = VS.union udo udn in + let loret = VS.union loo lon in + let uoret = VS.union uoo uon in + Some ((), ldret, udret, loret, uoret, uudn, uuon, loudn, ldlon, deadn) + ) + end + + + (* + * This handles a Cil.Instr object. This is sortof a C level statement. + *) + let doInstr i (_, ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) = + let transform (_, ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) = + let used, defd, useo, defo = computeUseDefState i in + + + if isDriverLockCall i then ( + (* + * A driver was locked, so add to the list of locked + * driver variables, and remove from the unlocked list + *) + let retld = VS.union ld used in + let retud = VS.diff ud used in + + (* + * Report if any objects are locked already since + * thats a deadlock risk + *) + if VS.is_empty lo then + ((), retld, retud, lo, uo, uud, uuo, loud, ldlo, dead) + else + ((), retld, retud, lo, uo, uud, uuo, loud, List.append ldlo [i], dead) + ) else if isDriverUnlockCall i then ( + (* + * A driver was unlocked, so add to the list of unlocked + * driver variables, and remove from the locked list + *) + let retld = VS.diff ld used in + let retud = VS.union ud used in + + ((), retld, retud, lo, uo, uud, uuo, loud, ldlo, dead); + ) else if isObjectFetchCall i then ( + (* + * A object was fetched & locked, so add to the list of + * locked driver variables. Nothing to remove from unlocked + * list here. + * + * XXX, not entirely true. We should check if they're + * blowing away an existing non-NULL value in the lval + * really. + *) + let retlo = VS.union lo defo in + + (* + * Report if driver is not locked, since that's a safety + * risk + *) + if VS.is_empty ld then ( + if VS.is_empty lo then ( + ((), ld, ud, retlo, uo, uud, uuo, List.append loud [i], ldlo, dead) + ) else ( + ((), ld, ud, retlo, uo, uud, uuo, List.append loud [i], ldlo, List.append dead [i]) + ) + ) else ( + if VS.is_empty lo then ( + ((), ld, ud, retlo, uo, uud, uuo, loud, ldlo, dead) + ) else ( + ((), ld, ud, retlo, uo, uud, uuo, loud, ldlo, List.append dead [i]) + ) + ) + ) else if isObjectLockCall i then ( + (* + * A driver was locked, so add to the list of locked + * driver variables, and remove from the unlocked list + *) + let retlo = VS.union lo useo in + let retuo = VS.diff uo useo in + + (* + * Report if driver is not locked, since that's a safety + * risk + *) + if VS.is_empty ld then + ((), ld, ud, retlo, retuo, uud, uuo, List.append loud [i], ldlo, dead) + else + ((), ld, ud, retlo, retuo, uud, uuo, loud, ldlo, dead) + ) else if isObjectUnlockCall i then ( + (* + * A object was unlocked, so add to the list of unlocked + * driver variables, and remove from the locked list + *) + let retlo = VS.diff lo useo in + let retuo = VS.union uo useo in + ((), ld, ud, retlo, retuo, uud, uuo, loud, ldlo, dead); + ) else ( + (* + * Nothing special happened, at best an assignment. + * So add any defined variables to the list of unlocked + * object or driver variables. + * XXX same edge case as isObjectFetchCall about possible + * overwriting + *) + let retud = VS.union ud defd in + let retuo = VS.union uo defo in + + (* + * Report is a driver is used while unlocked + *) + let retuud = + if not (VS.is_empty used) && (VS.is_empty ld) then + List.append uud [i] + else + uud in + (* + * Report is a object is used while unlocked + *) + let retuuo = + if not (VS.is_empty useo) && (VS.is_empty lo) then + List.append uuo [i] + else + uuo in + + ((), ld, retud, lo, retuo, retuud, retuuo, loud, ldlo, dead) + ); + in + + DF.Post transform + + (* + * This handles a Cil.Stmt object. This is sortof a C code block + *) + let doStmt stm (_, ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) = + DF.SUse ((), ld, ud, lo, uo, [], [], [], [], []) + + + (* + * This handles decision making for a conditional statement, + * ie an if (foo). It is called twice for each conditional + * ie, once per possible choice. + *) + let doGuard exp (_, ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) = + (* + * If we're going down a branch where our object variable + * is set to NULL, then we must remove it from the + * list of locked objects. This handles the case of... + * + * vm = virDomainFindByUUID(..) + * if (!vm) { + * .... this code branch .... + * } else { + * .... leaves default handling for this branch ... + * } + *) + let lonull = UD.computeUseExp exp in + + let loret = + if isLockableObjectNull exp then + VS.diff lo lonull + else + lo in + let uoret = + if isLockableObjectNull exp then + VS.union uo lonull + else + uo in + let ldret = + if isLockableDriverNull exp then + VS.diff ld lonull + else + ld in + let udret = + if isLockableDriverNull exp then + VS.union ud lonull + else + ud in + + DF.GUse ((), ldret, udret, loret, uoret, uud, uuo, loud, ldlo, dead) + + (* + * We're not filtering out any statements + *) + let filterStmt stm = true + +end + +module L = DF.ForwardsDataFlow(Locking) + +let () = + (* Read the list of files from "libvirt-files". *) + let files = input_file "object-locking-files.txt" in + let f3iles = [ "../src/qemu_driver.i" ] in + + (* Load & parse each input file. *) + let files = + List.map ( + fun filename -> + (* Why does parse return a continuation? *) + let f = Frontc.parse filename in + f () + ) files in + + (* Merge them. *) + let file = Mergecil.merge files "test" in + + (* Do control-flow-graph analysis. *) + Cfg.computeFileCFG file; + + print_endline ""; + + let driverVars = List.filter ( + function + | GVar (varinfo, initinfo, loc) -> (* global variable *) + let name = varinfo.vname in + if isDriverTable varinfo then + true + else + false + | _ -> false + ) file.globals in + + let driverVarFuncs = List.map ( + function + | GVar (varinfo, initinfo, loc) -> (* global variable *) + let name = varinfo.vname in + if isDriverTable varinfo then + getDriverFuncNames initinfo + else + [] + | _ -> [] + ) driverVars in + + let driverFuncsAll = List.flatten driverVarFuncs in + let driverFuncsSkip = [ + "testClose"; + "openvzClose"; + ] in + let driverFuncs = List.filter ( + fun st -> + if List.mem st driverFuncsSkip then + false + else + true + ) driverFuncsAll in + + (* + * Now comes our fun.... iterate over every global symbol + * definition Cfg found..... but... + *) + List.iter ( + function + (* ....only care about functions *) + | GFun (fundec, loc) -> (* function definition *) + let name = fundec.svar.vname in + + if List.mem name driverFuncs then ( + (* Initialize list of driver & object variables to be empty *) + ignore (lockableDriver = ref VS.empty); + ignore (lockableObjs = ref VS.empty); + + (* + * Query all local variables, and figure out which correspond + * to interesting driver & object variables we track + *) + List.iter ( + fun var -> + if isLockableDriverVar var then + lockableDriver := VS.add var !lockableDriver + else if isLockableObjectVar var then + lockableObjs := VS.add var !lockableObjs; + ) fundec.slocals; + + List.iter ( + fun gl -> + match gl with + GVar (vi, ii, loc) -> + if isLockableDriverVar vi then + lockableDriver := VS.add vi !lockableDriver + | _ -> () + ) file.globals; + + (* + * Initialize the state for each statement (ie C code block) + * to be empty + *) + List.iter ( + fun st -> + IH.add Locking.stmtStartData st.sid ((), + VS.empty, VS.empty, VS.empty, VS.empty, + [], [], [], [], []) + ) fundec.sallstmts; + + (* + * This walks all the code paths in the function building + * up the state for each statement (ie C code block) + * ie, this is invoking the "Locking" module we created + * earlier + *) + L.compute fundec.sallstmts; + + (* + * Find all statements (ie C code blocks) which have no + * successor statements. This means they are exit points + * in the function + *) + let exitPoints = List.filter ( + fun st -> + List.length st.succs = 0 + ) fundec.sallstmts in + + (* + * For each of the exit points, check to see if there are + * any with locked driver or object variables & grab them + *) + let leaks = List.filter ( + fun st -> + let (_, ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) = + IH.find Locking.stmtStartData st.sid in + let leakDrivers = not (VS.is_empty ld) in + let leakObjects = not (VS.is_empty lo) in + leakDrivers or leakObjects + ) exitPoints in + + let mistakes = List.filter ( + fun st -> + let (_, ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) = + IH.find Locking.stmtStartData st.sid in + let lockDriverOrdering = (List.length ldlo) > 0 in + let lockObjectOrdering = (List.length loud) > 0 in + + let useDriverUnlocked = (List.length uud) > 0 in + let useObjectUnlocked = (List.length uuo) > 0 in + + let deadLocked = (List.length dead) > 0 in + + lockDriverOrdering or lockObjectOrdering or useDriverUnlocked or useObjectUnlocked or deadLocked + ) fundec.sallstmts in + + if (List.length leaks) > 0 || (List.length mistakes) > 0 then ( + print_endline "================================================================"; + ignore (printf "Function: %s\n" name); + print_endline "----------------------------------------------------------------"; + ignore (printf " - Total exit points with locked vars: %d\n" (List.length leaks)); + + (* + * Finally tell the user which exit points had locked varaibles + * And show them the line number and code snippet for easy fixing + *) + List.iter ( + fun st -> + ignore (Pretty.printf " - At exit on %a\n^^^^^^^^^\n" d_stmt st); + let (_, ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) = + IH.find Locking.stmtStartData st.sid in + print_endline " variables still locked are"; + printVS ld; + printVS lo + ) leaks; + + + ignore (printf " - Total blocks with lock ordering mistakes: %d\n" (List.length mistakes)); + List.iter ( + fun st -> + let (_, ld, ud, lo, uo, uud, uuo, loud, ldlo, dead) = + IH.find Locking.stmtStartData st.sid in + List.iter ( + fun i -> + ignore (Pretty.printf " - Driver locked while object is locked on %a\n" d_instr i); + ) ldlo; + List.iter ( + fun i -> + ignore (Pretty.printf " - Object locked while driver is unlocked on %a\n" d_instr i); + ) loud; + List.iter ( + fun i -> + ignore (Pretty.printf " - Driver used while unlocked on %a\n" d_instr i); + ) uud; + List.iter ( + fun i -> + ignore (Pretty.printf " - Object used while unlocked on %a\n" d_instr i); + ) uuo; + List.iter ( + fun i -> + ignore (Pretty.printf " - Object fetched while locked objects exist %a\n" d_instr i); + ) dead; + ) mistakes; + print_endline "================================================================"; + print_endline ""; + print_endline ""; + ); + + () + ) + | _ -> () + ) file.globals; + + -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

2 2

[libvirt] Remote virConnect.listDomainsID() returns invalid IDs (Python, Xen)
by Russell Haering 14 May '09

14 May '09

I'm working with the Libvirt Python bindings and have discovered the following: If I run virConnect.listDomainsID() locally I get a list of valid IDs. For example: >>> import libvirt >>> conn = libvirt.open('xen://') >>> conn.listDomainsID() [0, 35, 36, 37, 38, 39, 40, 61] If I run the exact same thing remotely I get a list including numerous invalid IDs. This example shows the same host as shown above, about 20 seconds earlier (everything essentially identical): >>> import libvirt >>> conn = libvirt.open('xen+tls://xxxx.xxxxx') >>> conn.listDomainsID() [0, 35, 36, 37, 38, 39, 40, 51, 52, 53, 54, 60, 61] IDs 51, 52, 53, 54 and 60 are invalid. I first noticed this problem working with libvirt 0.6.0 on the client and 0.6.2 on the server, and noticed that with those versions the problem also occurred with virsh. Locally(0.6.0): local ~ # virsh list Id Name State ---------------------------------- 0 Domain-0 running 35 xxxxxxx idle 36 xxxxx idle 37 xxxxxx idle 38 xxxxxxxxx idle 39 xxxxxx idle 40 xxxxx idle 61 xxxxxxxxxx idle Whereas remotely(0.6.0): remote # virsh -c xen+tls://xxxxx.xxxxx list Id Name State ---------------------------------- 0 Domain-0 running 35 xxxxxxx idle 36 xxxxx idle 37 xxxxxx idle 38 xxxxxxxxx idle 39 xxxxxx idle 40 xxxxx idle libvir: Xen Daemon error : GET operation failed: xend_get: error from xen daemon: libvir: Xen Daemon error : GET operation failed: xend_get: error from xen daemon: libvir: Xen Daemon error : GET operation failed: xend_get: error from xen daemon: libvir: Xen Daemon error : GET operation failed: xend_get: error from xen daemon: libvir: Xen Daemon error : GET operation failed: xend_get: error from xen daemon: 61 xxxxxxxxxx idle However with virsh 0.6.2 on the client this appears to be fixed. Thanks, -- Russell Haering Systems Administrator OSU Open Source Lab

1 0

[libvirt] Has anyone ever gotten the client or full to compile on OS X?
by Schley Andrew Kutz 14 May '09

14 May '09

It's me. The original instigator of the long Leopard thread from early 2008. I once again need to do libvirt development from my Mac, and I still cannot seem to get it to compile :( It is currently bugging out at: gcc -DHAVE_CONFIG_H -I. -I.. -I../gnulib/lib -I../gnulib/lib -I../ include -I../include -I../qemud -I/usr/include/libxml2 -DLIBDIR=\"/opt/ libvirt//lib\" -DBINDIR=\"/opt/libvirt//libexec\" -DSBINDIR=\"/opt/ libvirt//sbin\" -DSYSCONF_DIR=\"/opt/libvirt//etc\" -DLOCALEBASEDIR=\"/ opt/libvirt//share/locale\" -DLOCAL_STATE_DIR=\"/opt/libvirt//var\" - DGETTEXT_PACKAGE=\"libvirt\" -Wall -Wformat -Wmissing-prototypes - Wnested-externs -Wpointer-arith -Wextra -Wshadow -Wcast-align -Wwrite- strings -Waggregate-return -Wstrict-prototypes -Winline -Wredundant- decls -Wno-sign-compare -Wp,-D_FORTIFY_SOURCE=2 -fexceptions - fasynchronous-unwind-tables -I/opt/local/include -g -O2 -MT pci.lo -MD -MP -MF .deps/pci.Tpo -c pci.c -fno-common -DPIC -o .libs/pci.o pci.c: In function 'pciFindStubDriver': pci.c:652: error: 'MODPROBE' undeclared (first use in this function) pci.c:652: error: (Each undeclared identifier is reported only once pci.c:652: error: for each function it appears in.) make[3]: *** [pci.lo] Error 1 make[2]: *** [all] Error 2 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 Any help is appreciated. -- -a "Ideally, a code library must be immediately usable by naive developers, easily customized by more sophisticated developers, and readily extensible by experts." -- L. Stein

4 16

[libvirt] [PATCH] Fix the style of argument("cpumap") in op_pincpu()
by Tatsuro Enokura 14 May '09

14 May '09

Hi all, The form of cpumap argument in op_pincpu method on xend change by following a patch. http://xenbits.xensource.com/xen-unstable.hg?rev/a63d20d7a941 xenDaemonDomainPinCpu() sends the string for cpumap argument like python's list style now. This patch is following the change of cpumap argument. "[0,1,2]" ---> "0,1,2" Signed-off-by: Tatsuro Enokura <fj2026af(a)aa.jp.fujitsu.com> # my e-mail address has changed. Thanks Tatsuro Enokura

3 9

[libvirt] LibvirtException: socket closed unexpectedly
by Marcel Michelmann 14 May '09

14 May '09

Hi *, I wrote a program in Java communicating with libvirt/XEN over SSH, with libvirt-java-0.2.1. I poll the CPU and RAM usage quite often (about 5 times a second). My program runs well about between 5-8 minutes, but then I get LibvirtException: socket closed unexpectedly. Although I try to catch the LibvirtExceptions with special System.out.println messages I have no idea where this is coming from. Now my question: what leads to this exception? exception caught: org.libvirt.LibvirtException: socket closed unexpectedly level:VIR_ERR_ERROR code:VIR_ERR_RPC domain:VIR_FROM_REMOTE hasConn:false hasDom:false hasNet:false message:socket closed unexpectedly str1:%s str2:socket closed unexpectedly str3:null int1:0 int2:0 After that I can start my program again, and it is the same again (runs some minutes and then throws the exception). Perhaps someone knows? I would be very thankful for any help! Best regards, Marcel

3 4