I tested these changes, and seem to be having some issues opening the xen driver in a post 3.0.3 codepath: DEBUG: xen_unified.c: xenUnifiedOpen (Trying hypervisor sub-driver) DEBUG: xen_unified.c: xenUnifiedOpen (Activated hypervisor sub-driver) libvir: Xen Daemon error : internal error failed to create a socket libvir: Xen error : out of memory DEBUG: xen_unified.c: xenUnifiedOpen (Failed to make capabilities) DEBUG: xen_unified.c: xenUnifiedOpen (Failed to activate a mandatory sub-driver) DEBUG: libvirt.c: do_open (driver 1 Xen returned ERROR) I haven't chased this down fully yet, but something is interfering... Daniel P. Berrange wrote on 11/21/2008 11:13 AM:

On Thu, Nov 20, 2008 at 03:49:07PM -0500, Ben Guthro wrote:

...

New xen events patch attached. This removes a couple unnecessary changes from my prior patch, but remains functionally the same as the last version.

This will emit the following events for Xen:

STARTED STOPPED ADDED REMOVED

I hit a few problems with the old Xen 3.0.3 codepath for /etc/xen files in this patch. What follows is the patch I needed to make it work reliably on RHEL-5 Xen.

The changes are

- Remove hardcoded qemu:///system to let libvirt probe HV - Add a cast to workaround lack of const-ness in RHEL5 python - Add an explicit xenXMConfigRemoveFile() to deal with files going away - Remove use of IN_MODIFY - it fired when the config was still incompletely written resulting in wierd errors - Add use of IN_CLOSE_WRITE so we're notified when the file is finished writing - Ignoring IN_CREATE if stat() shows zero size, because this fires the moment the name is allocated in the FS, but before content is created. We can't ignore it completely, because its needed if someone creates the file via hard-linking, when the entire content appears attomically & no IN_CLOSED_WRITE is fired. - Allocate capabilities info before initializing inotify driver because loading XM config files /etc/xen requires this - Fix removal of file handles for inotify & xenstore, since we need to remove based on watch number, not FD number

Regards, Daniel

diff -r fe87b41b48e3 examples/domain-events/events-c/event-test.c --- a/examples/domain-events/events-c/event-test.c Fri Nov 21 08:02:15 2008 -0500 +++ b/examples/domain-events/events-c/event-test.c Fri Nov 21 08:40:58 2008 -0500 @@ -308,7 +308,7 @@ int main(int argc, char **argv) myEventRemoveTimeoutFunc);

virConnectPtr dconn = NULL; - dconn = virConnectOpen (argv[1] ? argv[1] : "qemu:///system"); + dconn = virConnectOpen (argv[1] ? argv[1] : NULL); if (!dconn) { printf("error opening\n"); return -1; diff -r fe87b41b48e3 include/libvirt/virterror.h --- a/include/libvirt/virterror.h Fri Nov 21 08:02:15 2008 -0500 +++ b/include/libvirt/virterror.h Fri Nov 21 08:02:47 2008 -0500 @@ -60,6 +60,7 @@ typedef enum { VIR_FROM_DOMAIN, /* Error from domain config */ VIR_FROM_UML, /* Error at the UML driver */ VIR_FROM_NODEDEV, /* Error from node device monitor */ + VIR_FROM_XEN_INOTIFY, /* Error from xen inotify layer */ } virErrorDomain;

diff -r fe87b41b48e3 python/libvir.c --- a/python/libvir.c Fri Nov 21 08:02:15 2008 -0500 +++ b/python/libvir.c Fri Nov 21 08:36:40 2008 -0500 @@ -1564,7 +1564,8 @@ getLibvirtModuleObject (void) { return libvirt_module;

// PyImport_ImportModule returns a new reference - libvirt_module = PyImport_ImportModule("libvirt"); + /* Bogus (char *) cast for RHEL-5 python API brokenness */ + libvirt_module = PyImport_ImportModule((char *)"libvirt"); if(!libvirt_module) { #if DEBUG_ERROR printf("%s Error importing libvirt module\n", __FUNCTION__); diff -r fe87b41b48e3 src/util.c --- a/src/util.c Fri Nov 21 08:02:15 2008 -0500 +++ b/src/util.c Fri Nov 21 11:07:03 2008 -0500 @@ -613,6 +613,10 @@ virRun(virConnectPtr conn, VIR_FREE(outbuf); VIR_FREE(errbuf); VIR_FREE(argv_str); + if (outfd != -1) + close(outfd); + if (errfd != -1) + close(errfd); return ret; }

diff -r fe87b41b48e3 src/xen_inotify.c --- a/src/xen_inotify.c Fri Nov 21 08:02:15 2008 -0500 +++ b/src/xen_inotify.c Fri Nov 21 11:07:22 2008 -0500 @@ -49,12 +49,6 @@ static const char *configDir = NU static const char *configDir = NULL; static int useXenConfigCache = 0; static xenUnifiedDomainInfoListPtr configInfoList = NULL; - -/* declared in xm_internal.c */ -virHashTablePtr xenXMGetConfigCache(void); -char *xenXMGetConfigDir(void); -int xenXMConfigCacheRefresh (virConnectPtr conn); -int xenXMConfigCacheAddRemoveFile(virConnectPtr conn, const char *filename);

struct xenUnifiedDriver xenInotifyDriver = { xenInotifyOpen, /* open */ @@ -121,7 +115,7 @@ xenInotifyXendDomainsDirLookup(virConnec int i; virDomainPtr dom; const char *uuid_str; - const unsigned char uuid[VIR_UUID_BUFLEN]; + unsigned char uuid[VIR_UUID_BUFLEN];

/* xend is managing domains. we will get * a filename in the manner: @@ -145,7 +139,7 @@ xenInotifyXendDomainsDirLookup(virConnec for (i=0; i<configInfoList->count; i++) { if (STREQ(uuid_str, configInfoList->doms[i]->uuid)) { if(!(dom = virGetDomain(conn, configInfoList->doms[i]->name, - configInfoList->doms[i]->uuid))) { + (unsigned char *)configInfoList->doms[i]->uuid))) { virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, "finding dom for %s", uuid_str); return NULL; @@ -238,14 +232,14 @@ static int static int xenInotifyRemoveDomainConfigInfo(virConnectPtr conn, const char *fname) { - return useXenConfigCache ? xenXMConfigCacheAddRemoveFile(conn, fname) : + return useXenConfigCache ? xenXMConfigCacheRemoveFile(conn, fname) : xenInotifyXendDomainsDirRemoveEntry(conn, fname); }

static int xenInotifyAddDomainConfigInfo(virConnectPtr conn, const char *fname) { - return useXenConfigCache ? xenXMConfigCacheAddRemoveFile(conn, fname) : + return useXenConfigCache ? xenXMConfigCacheAddFile(conn, fname) : xenInotifyXendDomainsDirAddEntry(conn, fname); }

@@ -318,7 +312,7 @@ reread: "%s", _("Error adding file to config cache")); return; } - } else if (e->mask & ( IN_MODIFY | IN_CREATE) ) { + } else if (e->mask & ( IN_CREATE | IN_CLOSE_WRITE) ) { if (xenInotifyAddDomainConfigInfo(conn, fname) < 0 ) { virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, "%s", _("Error adding file to config cache")); @@ -409,7 +403,8 @@ xenInotifyOpen(virConnectPtr conn ATTRIB DEBUG("Adding a watch on %s", configDir); if (inotify_add_watch(priv->inotifyFD, configDir, - IN_CREATE | IN_MODIFY | IN_DELETE) < 0) { + IN_CREATE | + IN_CLOSE_WRITE | IN_DELETE) < 0) { virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, "adding watch on %s", _(configDir)); return -1; @@ -417,15 +412,16 @@ xenInotifyOpen(virConnectPtr conn ATTRIB

DEBUG0("Building initial config cache"); if (useXenConfigCache && - xenXMConfigCacheRefresh (conn) < 0) - return -1; - + xenXMConfigCacheRefresh (conn) < 0) { + DEBUG("Failed to enable XM config cache %s", conn->err.message); + return -1; + } + + DEBUG0("Registering with event loop"); /* Add the handle for monitoring */ - if (virEventAddHandle(priv->inotifyFD, VIR_EVENT_HANDLE_READABLE, - xenInotifyEvent, conn, NULL) < 0) { - virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, - "%s", _("Failed to add inotify event handle")); - return -1; + if ((priv->inotifyWatch = virEventAddHandle(priv->inotifyFD, VIR_EVENT_HANDLE_READABLE, + xenInotifyEvent, conn, NULL)) < 0) { + DEBUG0("Failed to add inotify handle, disabling events"); }

conn->refs++; @@ -448,7 +444,8 @@ xenInotifyClose(virConnectPtr conn) if(configInfoList) xenUnifiedDomainInfoListFree(configInfoList);

- virEventRemoveHandle(priv->inotifyFD); + if (priv->inotifyWatch != -1) + virEventRemoveHandle(priv->inotifyWatch); close(priv->inotifyFD); virUnrefConnect(conn);

diff -r fe87b41b48e3 src/xen_unified.c --- a/src/xen_unified.c Fri Nov 21 08:02:15 2008 -0500 +++ b/src/xen_unified.c Fri Nov 21 09:25:22 2008 -0500 @@ -291,6 +291,11 @@ xenUnifiedOpen (virConnectPtr conn, virC DEBUG0("Activated hypervisor sub-driver"); priv->opened[XEN_UNIFIED_HYPERVISOR_OFFSET] = 1; } + } + + if (!(priv->caps = xenHypervisorMakeCapabilities(conn))) { + DEBUG0("Failed to make capabilities"); + goto fail; }

/* XenD is required to suceed if root. @@ -351,11 +356,6 @@ xenUnifiedOpen (virConnectPtr conn, virC } #endif

- if (!(priv->caps = xenHypervisorMakeCapabilities(conn))) { - DEBUG0("Failed to make capabilities"); - goto fail; - } - return VIR_DRV_OPEN_SUCCESS;

fail: @@ -1324,6 +1324,11 @@ xenUnifiedDomainEventRegister (virConnec void (*freefunc)(void *)) { GET_PRIVATE (conn); + if (priv->xsWatch == -1) { + xenUnifiedError (conn, VIR_ERR_NO_SUPPORT, __FUNCTION__); + return -1; + } + conn->refs++; return virDomainEventCallbackListAdd(conn, priv->domainEventCallbacks, callback, opaque, freefunc); @@ -1335,6 +1340,11 @@ xenUnifiedDomainEventDeregister (virConn { int ret; GET_PRIVATE (conn); + if (priv->xsWatch == -1) { + xenUnifiedError (conn, VIR_ERR_NO_SUPPORT, __FUNCTION__); + return -1; + } + ret = virDomainEventCallbackListRemove(conn, priv->domainEventCallbacks, callback); virUnrefConnect(conn); diff -r fe87b41b48e3 src/xen_unified.h --- a/src/xen_unified.h Fri Nov 21 08:02:15 2008 -0500 +++ b/src/xen_unified.h Fri Nov 21 09:16:19 2008 -0500 @@ -155,6 +155,7 @@ struct _xenUnifiedPrivate {

/* A list of xenstore watches */ xenStoreWatchListPtr xsWatchList; + int xsWatch;

/* An list of callbacks */ virDomainEventCallbackListPtr domainEventCallbacks; @@ -162,6 +163,7 @@ struct _xenUnifiedPrivate { #if WITH_XEN_INOTIFY /* The inotify fd */ int inotifyFD; + int inotifyWatch; #endif };

diff -r fe87b41b48e3 src/xm_internal.c --- a/src/xm_internal.c Fri Nov 21 08:02:15 2008 -0500 +++ b/src/xm_internal.c Fri Nov 21 10:51:01 2008 -0500 @@ -45,6 +45,8 @@ #include "uuid.h" #include "util.h" #include "memory.h" +#include "logging.h" +

/* The true Xen limit varies but so far is always way less than 1024, which is the Linux kernel limit according @@ -65,10 +67,6 @@ char * xenXMAutoAssignMac(void); char * xenXMAutoAssignMac(void); static int xenXMDomainAttachDevice(virDomainPtr domain, const char *xml); static int xenXMDomainDetachDevice(virDomainPtr domain, const char *xml); -virHashTablePtr xenXMGetConfigCache (void); -char *xenXMGetConfigDir (void); -int xenXMConfigCacheRefresh (virConnectPtr conn); -int xenXMConfigCacheAddRemoveFile(virConnectPtr conn, const char *filename);

#define XM_REFRESH_INTERVAL 10

@@ -377,15 +375,45 @@ xenXMConfigSaveFile(virConnectPtr conn, }

int -xenXMConfigCacheAddRemoveFile(virConnectPtr conn, const char *filename) +xenXMConfigCacheRemoveFile(virConnectPtr conn ATTRIBUTE_UNUSED, + const char *filename) +{ + xenXMConfCachePtr entry; + + entry = virHashLookup(configCache, filename); + if (!entry) { + DEBUG("No config entry for %s", filename); + return 0; + } + + virHashRemoveEntry(nameConfigMap, entry->def->name, NULL); + virHashRemoveEntry(configCache, filename, xenXMConfigFree); + DEBUG("Removed %s %s", entry->def->name, filename); + return 0; +} + + +int +xenXMConfigCacheAddFile(virConnectPtr conn, const char *filename) { xenXMConfCachePtr entry; struct stat st; int newborn = 0; time_t now = time(NULL);

+ DEBUG("Adding file %s", filename); + /* Get modified time */ if ((stat(filename, &st) < 0)) { + xenXMError (conn, VIR_ERR_INTERNAL_ERROR, + "cannot stat %s: %s", filename, strerror(errno)); + return -1; + } + + /* Ignore zero length files, because inotify fires before + any content has actually been created */ + if (st.st_size == 0) { + DEBUG("Ignoring zero length file %s", filename); return -1; }

@@ -421,6 +449,7 @@ xenXMConfigCacheAddRemoveFile(virConnect entry->refreshedAt = now;

if (!(entry->def = xenXMConfigReadFile(conn, entry->filename))) { + DEBUG("Failed to read %s", entry->filename); if (!newborn) virHashRemoveEntry(configCache, filename, NULL); VIR_FREE(entry); @@ -449,6 +478,7 @@ xenXMConfigCacheAddRemoveFile(virConnect VIR_FREE(entry); } } + DEBUG("Added config %s %s", entry->def->name, filename);

return 0; } @@ -526,8 +556,8 @@ int xenXMConfigCacheRefresh (virConnectP

/* If we already have a matching entry and it is not modified, then carry on to next one*/ - if (xenXMConfigCacheAddRemoveFile(conn, path) < 0) { - goto cleanup; + if (xenXMConfigCacheAddFile(conn, path) < 0) { + /* Ignoring errors, since alot of stuff goes wrong in /etc/xen */ } }

@@ -538,7 +568,6 @@ int xenXMConfigCacheRefresh (virConnectP virHashRemoveSet(configCache, xenXMConfigReaper, xenXMConfigFree, (const void*) &now); ret = 0;

- cleanup: if (dh) closedir(dh);

diff -r fe87b41b48e3 src/xm_internal.h --- a/src/xm_internal.h Fri Nov 21 08:02:15 2008 -0500 +++ b/src/xm_internal.h Fri Nov 21 10:32:03 2008 -0500 @@ -32,6 +32,12 @@ extern struct xenUnifiedDriver xenXMDriv extern struct xenUnifiedDriver xenXMDriver; int xenXMInit (void);

+virHashTablePtr xenXMGetConfigCache(void); +char *xenXMGetConfigDir(void); +int xenXMConfigCacheRefresh (virConnectPtr conn); +int xenXMConfigCacheAddFile(virConnectPtr conn, const char *filename); +int xenXMConfigCacheRemoveFile(virConnectPtr conn, const char *filename); + int xenXMOpen(virConnectPtr conn, virConnectAuthPtr auth, int flags); int xenXMClose(virConnectPtr conn); const char *xenXMGetType(virConnectPtr conn); diff -r fe87b41b48e3 src/xs_internal.c --- a/src/xs_internal.c Fri Nov 21 08:02:15 2008 -0500 +++ b/src/xs_internal.c Fri Nov 21 09:25:57 2008 -0500 @@ -44,10 +44,10 @@ #error "unsupported platform" #endif

+#ifndef PROXY /* A list of active domain name/uuids */ static xenUnifiedDomainInfoListPtr activeDomainList = NULL;

-#ifndef PROXY static char *xenStoreDomainGetOSType(virDomainPtr domain);

struct xenUnifiedDriver xenStoreDriver = { @@ -345,16 +345,12 @@ xenStoreOpen(virConnectPtr conn, }

/* Add an event handle */ - if (virEventAddHandle(xs_fileno(priv->xshandle), - VIR_EVENT_HANDLE_READABLE, - xenStoreWatchEvent, - conn, - NULL) < 0) { - virXenStoreError(NULL, VIR_ERR_INTERNAL_ERROR, - "%s", _("failed to handle")); - - return -1; - } + if ((priv->xsWatch = virEventAddHandle(xs_fileno(priv->xshandle), + VIR_EVENT_HANDLE_READABLE, + xenStoreWatchEvent, + conn, + NULL)) < 0) + DEBUG0("Failed to add event handle, disabling events\n");

#endif //PROXY return 0; @@ -371,7 +367,6 @@ int int xenStoreClose(virConnectPtr conn) { - int fd; xenUnifiedPrivatePtr priv;

if (conn == NULL) { @@ -398,9 +393,8 @@ xenStoreClose(virConnectPtr conn) if (priv->xshandle == NULL) return(-1);

- fd = xs_fileno(priv->xshandle); - virEventRemoveHandle(fd); - close(fd); + if (priv->xsWatch != -1) + virEventRemoveHandle(priv->xsWatch); xs_daemon_close(priv->xshandle); priv->xshandle = NULL;

On Fri, Nov 21, 2008 at 02:11:39PM -0500, Ben Guthro wrote:

I have integrated your changes, and moved this bit referenced below, so it works with my setup. Could you please test with yours, and make sure I didn't break anything?

It didn't break anything, but I found more edge cases & a couple of bugs and optimizations. Changes in this version - We now also monitor IN_MOVED_TO and IN_MOVED_FROM, so that we catch 'mv foo /etc/xen' and 'mv /etc/xen/foo /tmp' as define & undefine events - On the @introduced watch handler we'll retry if we don't find the UUID / name first time around, to deal with inevitable race while XenD populates this info in xenstore. - In the @introduced & @removed handlers, simply to just check the domain ID, not name/uuid, since running VMs are unique on ID. - Fix UUID stored to be the raw unsigned char, not NULL terminated string form, so we're passing correct data into virGetDomain. This is now working reliably with my RHEL-5 Xen for all events, and should be a little more resilent on new Xen too. b/src/xen_inotify.c | 458 +++++++++++++++++++++++++++ b/src/xen_inotify.h | 31 + configure.in | 13 examples/domain-events/events-c/event-test.c | 2 include/libvirt/virterror.h | 1 po/POTFILES.in | 1 python/libvir.c | 3 src/Makefile.am | 5 src/util.c | 4 src/virterror.c | 3 src/xen_unified.c | 197 +++++++++++ src/xen_unified.h | 63 +++ src/xm_internal.c | 198 +++++++---- src/xm_internal.h | 6 src/xs_internal.c | 419 ++++++++++++++++++++++++ src/xs_internal.h | 52 +++ 16 files changed, 1383 insertions(+), 73 deletions(-) Daniel diff -r f10add69a53c configure.in --- a/configure.in Mon Nov 24 09:08:00 2008 -0500 +++ b/configure.in Mon Nov 24 09:08:01 2008 -0500 @@ -147,6 +147,8 @@ dnl Allow to build without Xen, QEMU/KVM dnl Allow to build without Xen, QEMU/KVM, test or remote driver AC_ARG_WITH([xen], [ --with-xen add XEN support (on)],[],[with_xen=yes]) +AC_ARG_WITH([xen-inotify], +[ --with-xen-inotify add XEN inotify support (on)],[],[with_xen_inotify=yes]) AC_ARG_WITH([qemu], [ --with-qemu add QEMU/KVM support (on)],[],[with_qemu=yes]) AC_ARG_WITH([uml], @@ -333,6 +335,17 @@ AM_CONDITIONAL([WITH_XEN], [test "$with_ AM_CONDITIONAL([WITH_XEN], [test "$with_xen" = "yes"]) AC_SUBST([XEN_CFLAGS]) AC_SUBST([XEN_LIBS]) + +dnl +dnl check for kernel headers required by xen_inotify +dnl +if test "$with_xen_inotify" != "no"; then + AC_CHECK_HEADER([linux/inotify.h],[],[with_xen_inotify=no]) +fi +if test "$with_xen_inotify" = "yes"; then + AC_DEFINE_UNQUOTED([WITH_XEN_INOTIFY], 1,[whether Xen inotify sub-driver is enabled]) +fi +AM_CONDITIONAL([WITH_XEN_INOTIFY], [test "$with_xen_inotify" = "yes"]) dnl dnl check for kernel headers required by src/bridge.c diff -r f10add69a53c examples/domain-events/events-c/event-test.c --- a/examples/domain-events/events-c/event-test.c Mon Nov 24 09:08:00 2008 -0500 +++ b/examples/domain-events/events-c/event-test.c Mon Nov 24 09:08:01 2008 -0500 @@ -308,7 +308,7 @@ int main(int argc, char **argv) myEventRemoveTimeoutFunc); virConnectPtr dconn = NULL; - dconn = virConnectOpen (argv[1] ? argv[1] : "qemu:///system"); + dconn = virConnectOpen (argv[1] ? argv[1] : NULL); if (!dconn) { printf("error opening\n"); return -1; diff -r f10add69a53c include/libvirt/virterror.h --- a/include/libvirt/virterror.h Mon Nov 24 09:08:00 2008 -0500 +++ b/include/libvirt/virterror.h Mon Nov 24 09:08:01 2008 -0500 @@ -60,6 +60,7 @@ typedef enum { VIR_FROM_DOMAIN, /* Error from domain config */ VIR_FROM_UML, /* Error at the UML driver */ VIR_FROM_NODEDEV, /* Error from node device monitor */ + VIR_FROM_XEN_INOTIFY, /* Error from xen inotify layer */ } virErrorDomain; diff -r f10add69a53c po/POTFILES.in --- a/po/POTFILES.in Mon Nov 24 09:08:00 2008 -0500 +++ b/po/POTFILES.in Mon Nov 24 09:08:01 2008 -0500 @@ -35,6 +35,7 @@ src/uuid.c src/uuid.c src/virsh.c src/virterror.c +src/xen_inotify.c src/xen_internal.c src/xend_internal.c src/xm_internal.c diff -r f10add69a53c python/libvir.c --- a/python/libvir.c Mon Nov 24 09:08:00 2008 -0500 +++ b/python/libvir.c Mon Nov 24 09:08:01 2008 -0500 @@ -1564,7 +1564,8 @@ getLibvirtModuleObject (void) { return libvirt_module; // PyImport_ImportModule returns a new reference - libvirt_module = PyImport_ImportModule("libvirt"); + /* Bogus (char *) cast for RHEL-5 python API brokenness */ + libvirt_module = PyImport_ImportModule((char *)"libvirt"); if(!libvirt_module) { #if DEBUG_ERROR printf("%s Error importing libvirt module\n", __FUNCTION__); diff -r f10add69a53c src/Makefile.am --- a/src/Makefile.am Mon Nov 24 09:08:00 2008 -0500 +++ b/src/Makefile.am Mon Nov 24 09:08:01 2008 -0500 @@ -101,6 +101,9 @@ XEN_DRIVER_SOURCES = \ xend_internal.c xend_internal.h \ xm_internal.c xm_internal.h \ xs_internal.c xs_internal.h +if WITH_XEN_INOTIFY +XEN_DRIVER_SOURCES += xen_inotify.c xen_inotify.h +endif LXC_DRIVER_SOURCES = \ lxc_conf.c lxc_conf.h \ @@ -188,6 +191,8 @@ libvirt_driver_la_SOURCES = \ libvirt_driver_la_CFLAGS = $(XEN_CFLAGS) libvirt_driver_la_LDFLAGS = $(XEN_LIBS) + +libvirt_driver_la_CFLAGS = $(XEN_CFLAGS) if WITH_TEST if WITH_DRIVER_MODULES diff -r f10add69a53c src/util.c --- a/src/util.c Mon Nov 24 09:08:00 2008 -0500 +++ b/src/util.c Mon Nov 24 09:08:01 2008 -0500 @@ -613,6 +613,10 @@ virRun(virConnectPtr conn, VIR_FREE(outbuf); VIR_FREE(errbuf); VIR_FREE(argv_str); + if (outfd != -1) + close(outfd); + if (errfd != -1) + close(errfd); return ret; } diff -r f10add69a53c src/virterror.c --- a/src/virterror.c Mon Nov 24 09:08:00 2008 -0500 +++ b/src/virterror.c Mon Nov 24 09:08:01 2008 -0500 @@ -262,6 +262,9 @@ virDefaultErrorFunc(virErrorPtr err) case VIR_FROM_XENSTORE: dom = "Xen Store "; break; + case VIR_FROM_XEN_INOTIFY: + dom = "Xen Inotify "; + break; case VIR_FROM_DOM: dom = "Domain "; break; diff -r f10add69a53c src/xen_inotify.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/xen_inotify.c Mon Nov 24 10:29:13 2008 -0500 @@ -0,0 +1,458 @@ +/* + * xen_inofify.c: Xen notification of xml file activity in the + * following dirs: + * /etc/xen + * /var/lib/xend/domains + * + * Copyright (C) 2008 VirtualIron + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Author: Ben Guthro + */ +#include <config.h> +#include <dirent.h> +#include <sys/inotify.h> + +#include "virterror_internal.h" +#include "datatypes.h" +#include "driver.h" +#include "memory.h" +#include "event.h" +#include "xen_unified.h" +#include "conf.h" +#include "domain_conf.h" +#include "xen_inotify.h" +#include "xend_internal.h" +#include "logging.h" +#include "uuid.h" + +#include "xm_internal.h" /* for xenXMDomainConfigParse */ + +#define virXenInotifyError(conn, code, fmt...) \ + virReportErrorHelper(NULL, VIR_FROM_XEN_INOTIFY, code, __FILE__, \ + __FUNCTION__, __LINE__, fmt) + +#define LIBVIRTD_DOMAINS_DIR "/var/lib/xend/domains" +static const char *configDir = NULL; +static int useXenConfigCache = 0; +static xenUnifiedDomainInfoListPtr configInfoList = NULL; + +struct xenUnifiedDriver xenInotifyDriver = { + xenInotifyOpen, /* open */ + xenInotifyClose, /* close */ + NULL, /* version */ + NULL, /* hostname */ + NULL, /* URI */ + NULL, /* nodeGetInfo */ + NULL, /* getCapabilities */ + NULL, /* listDomains */ + NULL, /* numOfDomains */ + NULL, /* domainCreateLinux */ + NULL, /* domainSuspend */ + NULL, /* domainResume */ + NULL, /* domainShutdown */ + NULL, /* domainReboot */ + NULL, /* domainDestroy */ + NULL, /* domainGetOSType */ + NULL, /* domainGetMaxMemory */ + NULL, /* domainSetMaxMemory */ + NULL, /* domainSetMemory */ + NULL, /* domainGetInfo */ + NULL, /* domainSave */ + NULL, /* domainRestore */ + NULL, /* domainCoreDump */ + NULL, /* domainSetVcpus */ + NULL, /* domainPinVcpu */ + NULL, /* domainGetVcpus */ + NULL, /* domainGetMaxVcpus */ + NULL, /* listDefinedDomains */ + NULL, /* numOfDefinedDomains */ + NULL, /* domainCreate */ + NULL, /* domainDefineXML */ + NULL, /* domainUndefine */ + NULL, /* domainAttachDevice */ + NULL, /* domainDetachDevice */ + NULL, /* domainGetAutostart */ + NULL, /* domainSetAutostart */ + NULL, /* domainGetSchedulerType */ + NULL, /* domainGetSchedulerParameters */ + NULL, /* domainSetSchedulerParameters */ +}; + +static virDomainPtr +xenInotifyXenCacheLookup(virConnectPtr conn, const char *filename) { + xenXMConfCachePtr entry; + virDomainPtr dom; + + if (!(entry = virHashLookup(xenXMGetConfigCache(), filename))) { + DEBUG("No config found for %s", filename); + return NULL; + } + + if(!(dom = virGetDomain(conn, entry->def->name, + (unsigned char*)entry->def->uuid))) { + DEBUG0("Error getting dom from def"); + return NULL; + } + return dom; +} + +static virDomainPtr +xenInotifyXendDomainsDirLookup(virConnectPtr conn, const char *filename) { + int i; + virDomainPtr dom; + const char *uuid_str; + unsigned char uuid[VIR_UUID_BUFLEN]; + + /* xend is managing domains. we will get + * a filename in the manner: + * /var/lib/xend/domains/<uuid>/ + */ + uuid_str = filename + strlen(LIBVIRTD_DOMAINS_DIR) + 1; + + if (virUUIDParse(uuid_str, uuid) < 0) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "parsing uuid %s", uuid_str); + return (NULL); + } + /* call directly into xend here, as driver may not yet + be set during open while we are building our + initial list of domains */ + DEBUG("Looking for dom with uuid: %s", uuid_str); + if(!(dom = xenDaemonLookupByUUID(conn, uuid))) { + /* If we are here, the domain has gone away. + search for, and create a domain from the stored + list info */ + for (i=0; i<configInfoList->count; i++) { + if (!memcmp(uuid, configInfoList->doms[i]->uuid, VIR_UUID_BUFLEN)) { + if(!(dom = virGetDomain(conn, configInfoList->doms[i]->name, + configInfoList->doms[i]->uuid))) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "finding dom for %s", uuid_str); + return NULL; + } + DEBUG0("Found dom on list"); + return dom; + } + } + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("finding dom on config list")); + return NULL; + } + + /* succeeded too find domain by uuid */ + return dom; +} + +static virDomainPtr +xenInotifyDomainLookup(virConnectPtr conn, const char *filename) { + virDomainPtr dom; + virDomainInfo info; + + dom = useXenConfigCache ? xenInotifyXenCacheLookup(conn, filename) : + xenInotifyXendDomainsDirLookup(conn, filename); + + if(dom) { + if ( (useXenConfigCache ? xenXMDomainGetInfo(dom, &info) : + xenDaemonDomainGetInfo(dom, &info)) < 0) + dom->id = -1; + else + dom->id = (info.state == VIR_DOMAIN_SHUTOFF) ? -1 : dom->id; + return dom; + } + return NULL; +} + +static int +xenInotifyXendDomainsDirRemoveEntry(virConnectPtr conn ATTRIBUTE_UNUSED, + const char *fname) { + const char *uuidstr = fname + strlen(LIBVIRTD_DOMAINS_DIR) + 1; + unsigned char uuid[VIR_UUID_BUFLEN]; + int i; + + if (virUUIDParse(uuidstr, uuid) < 0) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "parsing uuid %s", uuidstr); + return -1; + } + + /* match and remove on uuid */ + for (i=0; i<configInfoList->count; i++) { + if (!memcmp(uuid, configInfoList->doms[i]->uuid, VIR_UUID_BUFLEN)) { + VIR_FREE(configInfoList->doms[i]->name); + VIR_FREE(configInfoList->doms[i]); + + if (i < (configInfoList->count - 1)) + memmove(configInfoList->doms + i, + configInfoList->doms + i + 1, + sizeof(*(configInfoList->doms)) * + (configInfoList->count - (i + 1))); + + if (VIR_REALLOC_N(configInfoList->doms, + configInfoList->count - 1) < 0) { + ; /* Failure to reduce memory allocation isn't fatal */ + } + configInfoList->count--; + return 0; + } + } + return -1; +} + +static int +xenInotifyXendDomainsDirAddEntry(virConnectPtr conn, + const char *fname) { + virDomainPtr dom = xenInotifyDomainLookup(conn, fname); + if(!dom) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("Error looking up domain")); + return -1; + } + + if( xenUnifiedAddDomainInfo(configInfoList, + dom->id, dom->name, dom->uuid) < 0) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("Error adding file to config cache")); + virUnrefDomain(dom); + return -1; + } + virUnrefDomain(dom); + return 0; +} + +static int +xenInotifyRemoveDomainConfigInfo(virConnectPtr conn, + const char *fname) { + return useXenConfigCache ? xenXMConfigCacheRemoveFile(conn, fname) : + xenInotifyXendDomainsDirRemoveEntry(conn, fname); +} + +static int +xenInotifyAddDomainConfigInfo(virConnectPtr conn, + const char *fname) { + return useXenConfigCache ? xenXMConfigCacheAddFile(conn, fname) : + xenInotifyXendDomainsDirAddEntry(conn, fname); +} + +static void +xenInotifyEvent(int watch ATTRIBUTE_UNUSED, + int fd, + int events ATTRIBUTE_UNUSED, + void *data) +{ + char buf[1024]; + char fname[1024]; + struct inotify_event *e; + int got; + char *tmp, *name; + virConnectPtr conn = (virConnectPtr) data; + xenUnifiedPrivatePtr priv = NULL; + virDomainPtr dom = NULL; + + DEBUG0("got inotify event"); + + if( conn && conn->privateData ) { + priv = conn->privateData; + } else { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("conn, or private data is NULL")); + return; + } + +reread: + got = read(fd, buf, sizeof(buf)); + if (got == -1) { + if (errno == EINTR) + goto reread; + return; + } + + tmp = buf; + while (got) { + if (got < sizeof(struct inotify_event)) + return; /* bad */ + + e = (struct inotify_event *)tmp; + tmp += sizeof(struct inotify_event); + got -= sizeof(struct inotify_event); + + if (got < e->len) + return; + + tmp += e->len; + got -= e->len; + + name = (char *)&(e->name); + + snprintf(fname, 1024, "%s/%s", configDir, name); + + if (e->mask & (IN_DELETE | IN_MOVED_FROM)) { + if (!(dom = xenInotifyDomainLookup(conn, fname))) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("looking up dom")); + continue; + } + + xenUnifiedDomainEventDispatch(conn->privateData, dom, + VIR_DOMAIN_EVENT_UNDEFINED, + VIR_DOMAIN_EVENT_UNDEFINED_REMOVED); + + + if (xenInotifyRemoveDomainConfigInfo(conn, fname) < 0 ) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("Error adding file to config cache")); + return; + } + } else if (e->mask & ( IN_CREATE | IN_CLOSE_WRITE | IN_MOVED_TO) ) { + if (xenInotifyAddDomainConfigInfo(conn, fname) < 0 ) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("Error adding file to config cache")); + return; + } + + if (!(dom = xenInotifyDomainLookup(conn, fname))) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("looking up dom")); + continue; + } + + xenUnifiedDomainEventDispatch(conn->privateData, dom, + VIR_DOMAIN_EVENT_DEFINED, + VIR_DOMAIN_EVENT_DEFINED_ADDED); + } + + } +} + +/** + * xenInotifyOpen: + * @conn: pointer to the connection block + * @name: URL for the target, NULL for local + * @flags: combination of virDrvOpenFlag(s) + * + * Connects and starts listening for inotify events + * + * Returns 0 or -1 in case of error. + */ +int +xenInotifyOpen(virConnectPtr conn ATTRIBUTE_UNUSED, + virConnectAuthPtr auth ATTRIBUTE_UNUSED, + int flags ATTRIBUTE_UNUSED) +{ + DIR *dh; + struct dirent *ent; + char path[PATH_MAX]; + xenUnifiedPrivatePtr priv = (xenUnifiedPrivatePtr) conn->privateData; + + if(priv->xendConfigVersion <= 2) { + /* /etc/xen */ + configDir = xenXMGetConfigDir(); + useXenConfigCache = 1; + } else { + /* /var/lib/xend/domains/<uuid>/config.sxp */ + configDir = LIBVIRTD_DOMAINS_DIR; + useXenConfigCache = 0; + + if ( VIR_ALLOC(configInfoList ) < 0) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("failed to allocate configInfoList")); + return -1; + } + + /* populate initial list */ + if (!(dh = opendir(configDir))) { + virXenInotifyError (NULL, VIR_ERR_INTERNAL_ERROR, + "%s", strerror(errno)); + return -1; + } + while ((ent = readdir(dh))) { + if (STRPREFIX(ent->d_name, ".")) + continue; + + /* Build the full file path */ + if ((strlen(configDir) + 1 + strlen(ent->d_name) + 1) > PATH_MAX) + continue; + strcpy(path, configDir); + strcat(path, "/"); + strcat(path, ent->d_name); + + if (xenInotifyAddDomainConfigInfo(conn, path) < 0 ) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("Error adding file to config list")); + return -1; + } + } + + } + + if ((priv->inotifyFD = inotify_init()) < 0) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("initializing inotify")); + return -1; + } + + DEBUG("Adding a watch on %s", configDir); + if (inotify_add_watch(priv->inotifyFD, + configDir, + IN_CREATE | + IN_CLOSE_WRITE | IN_DELETE | + IN_MOVED_TO | IN_MOVED_FROM) < 0) { + virXenInotifyError(NULL, VIR_ERR_INTERNAL_ERROR, + "adding watch on %s", _(configDir)); + return -1; + } + + DEBUG0("Building initial config cache"); + if (useXenConfigCache && + xenXMConfigCacheRefresh (conn) < 0) { + DEBUG("Failed to enable XM config cache %s", conn->err.message); + return -1; + } + + DEBUG0("Registering with event loop"); + /* Add the handle for monitoring */ + if ((priv->inotifyWatch = virEventAddHandle(priv->inotifyFD, VIR_EVENT_HANDLE_READABLE, + xenInotifyEvent, conn, NULL)) < 0) { + DEBUG0("Failed to add inotify handle, disabling events"); + } + + conn->refs++; + return 0; +} + +/** + * xenInotifyClose: + * @conn: pointer to the connection block + * + * Close and stop listening for inotify events + * + * Returns 0 in case of success or -1 in case of error. + */ +int +xenInotifyClose(virConnectPtr conn) +{ + xenUnifiedPrivatePtr priv = (xenUnifiedPrivatePtr) conn->privateData; + + if(configInfoList) + xenUnifiedDomainInfoListFree(configInfoList); + + if (priv->inotifyWatch != -1) + virEventRemoveHandle(priv->inotifyWatch); + close(priv->inotifyFD); + virUnrefConnect(conn); + + return 0; +} diff -r f10add69a53c src/xen_inotify.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/xen_inotify.h Mon Nov 24 09:08:01 2008 -0500 @@ -0,0 +1,31 @@ +/* + * xen_inofify.h: Xen notification of xml files + * + * Copyright (C) 2008 VirtualIron + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Author: Ben Guthro + */ +#ifndef __VIR_XEN_INOTIFY_H__ +#define __VIR_XEN_INOTIFY_H__ +#include "internal.h" +extern struct xenUnifiedDriver xenInotifyDriver; + +int xenInotifyOpen (virConnectPtr conn, + virConnectAuthPtr auth, + int flags); +int xenInotifyClose (virConnectPtr conn); +#endif diff -r f10add69a53c src/xen_unified.c --- a/src/xen_unified.c Mon Nov 24 09:08:00 2008 -0500 +++ b/src/xen_unified.c Mon Nov 24 09:46:55 2008 -0500 @@ -37,6 +37,9 @@ #include "xend_internal.h" #include "xs_internal.h" #include "xm_internal.h" +#if WITH_XEN_INOTIFY +#include "xen_inotify.h" +#endif #include "xml.h" #include "util.h" #include "memory.h" @@ -57,6 +60,9 @@ static struct xenUnifiedDriver *drivers[ [XEN_UNIFIED_XEND_OFFSET] = &xenDaemonDriver, [XEN_UNIFIED_XS_OFFSET] = &xenStoreDriver, [XEN_UNIFIED_XM_OFFSET] = &xenXMDriver, +#if WITH_XEN_INOTIFY + [XEN_UNIFIED_INOTIFY_OFFSET] = &xenInotifyDriver, +#endif }; #define xenUnifiedError(conn, code, fmt...) \ @@ -223,6 +229,7 @@ xenUnifiedOpen (virConnectPtr conn, virC { int i, ret = VIR_DRV_OPEN_DECLINED; xenUnifiedPrivatePtr priv; + virDomainEventCallbackListPtr cbList; if (conn->uri == NULL) { if (!xenUnifiedProbe()) @@ -259,6 +266,13 @@ xenUnifiedOpen (virConnectPtr conn, virC return VIR_DRV_OPEN_ERROR; } conn->privateData = priv; + + /* Allocate callback list */ + if (VIR_ALLOC(cbList) < 0) { + xenUnifiedError (NULL, VIR_ERR_NO_MEMORY, "allocating callback list"); + return VIR_DRV_OPEN_ERROR; + } + priv->domainEventCallbacks = cbList; priv->handle = -1; priv->xendConfigVersion = -1; @@ -333,6 +347,15 @@ xenUnifiedOpen (virConnectPtr conn, virC goto fail; } +#if WITH_XEN_INOTIFY + DEBUG0("Trying Xen inotify sub-driver"); + if (drivers[XEN_UNIFIED_INOTIFY_OFFSET]->open(conn, auth, flags) == + VIR_DRV_OPEN_SUCCESS) { + DEBUG0("Activated Xen inotify sub-driver"); + priv->opened[XEN_UNIFIED_INOTIFY_OFFSET] = 1; + } +#endif + return VIR_DRV_OPEN_SUCCESS; fail: @@ -357,6 +380,8 @@ xenUnifiedClose (virConnectPtr conn) int i; virCapabilitiesFree(priv->caps); + virDomainEventCallbackListFree(priv->domainEventCallbacks); + for (i = 0; i < XEN_UNIFIED_NR_DRIVERS; ++i) if (priv->opened[i] && drivers[i]->close) (void) drivers[i]->close (conn); @@ -1290,6 +1315,40 @@ xenUnifiedNodeGetFreeMemory (virConnectP xenUnifiedError (conn, VIR_ERR_NO_SUPPORT, __FUNCTION__); return(0); +} + +static int +xenUnifiedDomainEventRegister (virConnectPtr conn, + void *callback, + void *opaque, + void (*freefunc)(void *)) +{ + GET_PRIVATE (conn); + if (priv->xsWatch == -1) { + xenUnifiedError (conn, VIR_ERR_NO_SUPPORT, __FUNCTION__); + return -1; + } + + conn->refs++; + return virDomainEventCallbackListAdd(conn, priv->domainEventCallbacks, + callback, opaque, freefunc); +} + +static int +xenUnifiedDomainEventDeregister (virConnectPtr conn, + void *callback) +{ + int ret; + GET_PRIVATE (conn); + if (priv->xsWatch == -1) { + xenUnifiedError (conn, VIR_ERR_NO_SUPPORT, __FUNCTION__); + return -1; + } + + ret = virDomainEventCallbackListRemove(conn, priv->domainEventCallbacks, + callback); + virUnrefConnect(conn); + return ret; } /*----- Register with libvirt.c, and initialise Xen drivers. -----*/ @@ -1356,6 +1415,8 @@ static virDriver xenUnifiedDriver = { .domainBlockPeek = xenUnifiedDomainBlockPeek, .nodeGetCellsFreeMemory = xenUnifiedNodeGetCellsFreeMemory, .getFreeMemory = xenUnifiedNodeGetFreeMemory, + .domainEventRegister = xenUnifiedDomainEventRegister, + .domainEventDeregister = xenUnifiedDomainEventDeregister, }; /** @@ -1375,3 +1436,139 @@ xenRegister (void) return virRegisterDriver (&xenUnifiedDriver); } +/** + * xenUnifiedDomainInfoListFree: + * + * Free the Domain Info List + */ +void +xenUnifiedDomainInfoListFree(xenUnifiedDomainInfoListPtr list) +{ + int i; + for (i=0; i<list->count; i++) { + VIR_FREE(list->doms[i]->name); + VIR_FREE(list->doms[i]); + } + VIR_FREE(list); +} + +/** + * xenUnifiedAddDomainInfo: + * + * Add name and uuid to the domain info list + * + * Returns: 0 on success, -1 on failure + */ +int +xenUnifiedAddDomainInfo(xenUnifiedDomainInfoListPtr list, + int id, char *name, + unsigned char *uuid) +{ + xenUnifiedDomainInfoPtr info; + int n; + + /* check if we already have this callback on our list */ + for (n=0; n < list->count; n++) { + if (STREQ(list->doms[n]->name, name) && + !memcmp(list->doms[n]->uuid, uuid, VIR_UUID_BUFLEN)) { + DEBUG0("WARNING: dom already tracked"); + return -1; + } + } + + if (VIR_ALLOC(info) < 0) + goto memory_error; + if (!(info->name = strdup(name))) + goto memory_error; + + memcpy(info->uuid, uuid, VIR_UUID_BUFLEN); + info->id = id; + + /* Make space on list */ + n = list->count; + if (VIR_REALLOC_N(list->doms, n + 1) < 0) { + goto memory_error; + } + + list->doms[n] = info; + list->count++; + return 0; +memory_error: + xenUnifiedError (NULL, VIR_ERR_NO_MEMORY, "allocating domain info"); + if (info) + VIR_FREE(info->name); + VIR_FREE(info); + return -1; +} + +/** + * xenUnifiedRemoveDomainInfo: + * + * Removes name and uuid to the domain info list + * + * Returns: 0 on success, -1 on failure + */ +int +xenUnifiedRemoveDomainInfo(xenUnifiedDomainInfoListPtr list, + int id, char *name, + unsigned char *uuid) +{ + int i; + for (i = 0 ; i < list->count ; i++) { + if( list->doms[i]->id == id && + STREQ(list->doms[i]->name, name) && + !memcmp(list->doms[i]->uuid, uuid, VIR_UUID_BUFLEN)) { + + VIR_FREE(list->doms[i]->name); + VIR_FREE(list->doms[i]); + + if (i < (list->count - 1)) + memmove(list->doms + i, + list->doms + i + 1, + sizeof(*(list->doms)) * + (list->count - (i + 1))); + + if (VIR_REALLOC_N(list->doms, + list->count - 1) < 0) { + ; /* Failure to reduce memory allocation isn't fatal */ + } + list->count--; + + return 0; + } + } + return -1; +} + +/** + * xenUnifiedDomainEventDispatch: + * + * Dispatch domain events to registered callbacks + * + */ +void xenUnifiedDomainEventDispatch (xenUnifiedPrivatePtr priv, + virDomainPtr dom, + int event, + int detail) +{ + int i; + virDomainEventCallbackListPtr cbList; + + if(!priv) return; + + cbList = priv->domainEventCallbacks; + if(!cbList) return; + + for(i=0 ; i < cbList->count ; i++) { + if(cbList->callbacks[i] && cbList->callbacks[i]->cb) { + if (dom) { + DEBUG("Dispatching callback %p %p event %d", + cbList->callbacks[i], + cbList->callbacks[i]->cb, event); + cbList->callbacks[i]->cb(cbList->callbacks[i]->conn, + dom, event, detail, + cbList->callbacks[i]->opaque); + } + } + } +} diff -r f10add69a53c src/xen_unified.h --- a/src/xen_unified.h Mon Nov 24 09:08:00 2008 -0500 +++ b/src/xen_unified.h Mon Nov 24 10:18:38 2008 -0500 @@ -14,6 +14,12 @@ #include "internal.h" #include "capabilities.h" #include "driver.h" +#include "domain_conf.h" +#include "xs_internal.h" +#if WITH_XEN_INOTIFY +#include "xen_inotify.h" +#endif +#include "domain_event.h" #ifndef HAVE_WINSOCK2_H #include <sys/un.h> @@ -29,7 +35,13 @@ extern int xenRegister (void); #define XEN_UNIFIED_XEND_OFFSET 2 #define XEN_UNIFIED_XS_OFFSET 3 #define XEN_UNIFIED_XM_OFFSET 4 + +#if WITH_XEN_INOTIFY +#define XEN_UNIFIED_INOTIFY_OFFSET 5 +#define XEN_UNIFIED_NR_DRIVERS 6 +#else #define XEN_UNIFIED_NR_DRIVERS 5 +#endif #define MIN_XEN_GUEST_SIZE 64 /* 64 megabytes */ @@ -85,6 +97,33 @@ struct xenUnifiedDriver { virDrvDomainSetSchedulerParameters domainSetSchedulerParameters; }; +typedef struct xenXMConfCache *xenXMConfCachePtr; +typedef struct xenXMConfCache { + time_t refreshedAt; + char filename[PATH_MAX]; + virDomainDefPtr def; +} xenXMConfCache; + +/* xenUnifiedDomainInfoPtr: + * The minimal state we have about active domains + * This is the minmal info necessary to still get a + * virDomainPtr when the domain goes away + */ +struct _xenUnifiedDomainInfo { + int id; + char *name; + unsigned char uuid[VIR_UUID_BUFLEN]; +}; +typedef struct _xenUnifiedDomainInfo xenUnifiedDomainInfo; +typedef xenUnifiedDomainInfo *xenUnifiedDomainInfoPtr; + +struct _xenUnifiedDomainInfoList { + unsigned int count; + xenUnifiedDomainInfoPtr *doms; +}; +typedef struct _xenUnifiedDomainInfoList xenUnifiedDomainInfoList; +typedef xenUnifiedDomainInfoList *xenUnifiedDomainInfoListPtr; + /* xenUnifiedPrivatePtr: * * Per-connection private data, stored in conn->privateData. All Xen @@ -113,6 +152,19 @@ struct _xenUnifiedPrivate { * xen_unified.c. */ int opened[XEN_UNIFIED_NR_DRIVERS]; + + /* A list of xenstore watches */ + xenStoreWatchListPtr xsWatchList; + int xsWatch; + + /* An list of callbacks */ + virDomainEventCallbackListPtr domainEventCallbacks; + +#if WITH_XEN_INOTIFY + /* The inotify fd */ + int inotifyFD; + int inotifyWatch; +#endif }; typedef struct _xenUnifiedPrivate *xenUnifiedPrivatePtr; @@ -122,4 +174,15 @@ int xenNbCpus(virConnectPtr conn); int xenNbCpus(virConnectPtr conn); char *xenDomainUsedCpus(virDomainPtr dom); +void xenUnifiedDomainInfoListFree(xenUnifiedDomainInfoListPtr info); +int xenUnifiedAddDomainInfo(xenUnifiedDomainInfoListPtr info, + int id, char *name, + unsigned char *uuid); +int xenUnifiedRemoveDomainInfo(xenUnifiedDomainInfoListPtr info, + int id, char *name, + unsigned char *uuid); +void xenUnifiedDomainEventDispatch (xenUnifiedPrivatePtr priv, + virDomainPtr dom, + int event, + int detail); #endif /* __VIR_XEN_UNIFIED_H__ */ diff -r f10add69a53c src/xm_internal.c --- a/src/xm_internal.c Mon Nov 24 09:08:00 2008 -0500 +++ b/src/xm_internal.c Mon Nov 24 09:08:01 2008 -0500 @@ -45,6 +45,8 @@ #include "uuid.h" #include "util.h" #include "memory.h" +#include "logging.h" + /* The true Xen limit varies but so far is always way less than 1024, which is the Linux kernel limit according @@ -53,13 +55,6 @@ static int xenXMConfigSetString(virConfPtr conf, const char *setting, const char *str); - -typedef struct xenXMConfCache *xenXMConfCachePtr; -typedef struct xenXMConfCache { - time_t refreshedAt; - char filename[PATH_MAX]; - virDomainDefPtr def; -} xenXMConfCache; static char configDir[PATH_MAX]; /* Config file name to config object */ @@ -124,6 +119,14 @@ struct xenUnifiedDriver xenXMDriver = { NULL, /* domainSetSchedulerParameters */ }; +virHashTablePtr xenXMGetConfigCache (void) { + return configCache; +} + +char *xenXMGetConfigDir (void) { + return configDir; +} + #define xenXMError(conn, code, fmt...) \ virReportErrorHelper(conn, VIR_FROM_XENXM, code, __FILE__, \ __FUNCTION__, __LINE__, fmt) @@ -371,13 +374,121 @@ xenXMConfigSaveFile(virConnectPtr conn, return ret; } +int +xenXMConfigCacheRemoveFile(virConnectPtr conn ATTRIBUTE_UNUSED, + const char *filename) +{ + xenXMConfCachePtr entry; + + entry = virHashLookup(configCache, filename); + if (!entry) { + DEBUG("No config entry for %s", filename); + return 0; + } + + virHashRemoveEntry(nameConfigMap, entry->def->name, NULL); + virHashRemoveEntry(configCache, filename, xenXMConfigFree); + DEBUG("Removed %s %s", entry->def->name, filename); + return 0; +} + + +int +xenXMConfigCacheAddFile(virConnectPtr conn, const char *filename) +{ + xenXMConfCachePtr entry; + struct stat st; + int newborn = 0; + time_t now = time(NULL); + + DEBUG("Adding file %s", filename); + + /* Get modified time */ + if ((stat(filename, &st) < 0)) { + xenXMError (conn, VIR_ERR_INTERNAL_ERROR, + "cannot stat %s: %s", filename, strerror(errno)); + return -1; + } + + /* Ignore zero length files, because inotify fires before + any content has actually been created */ + if (st.st_size == 0) { + DEBUG("Ignoring zero length file %s", filename); + return -1; + } + + /* If we already have a matching entry and it is not + modified, then carry on to next one*/ + if ((entry = virHashLookup(configCache, filename))) { + char *nameowner; + + if (entry->refreshedAt >= st.st_mtime) { + entry->refreshedAt = now; + /* return success if up-to-date */ + return 0; + } + + /* If we currently own the name, then release it and + re-acquire it later - just in case it was renamed */ + nameowner = (char *)virHashLookup(nameConfigMap, entry->def->name); + if (nameowner && STREQ(nameowner, filename)) { + virHashRemoveEntry(nameConfigMap, entry->def->name, NULL); + } + + /* Clear existing config entry which needs refresh */ + virDomainDefFree(entry->def); + entry->def = NULL; + } else { /* Completely new entry */ + newborn = 1; + if (VIR_ALLOC(entry) < 0) { + xenXMError (conn, VIR_ERR_NO_MEMORY, "%s", strerror(errno)); + return -1; + } + memcpy(entry->filename, filename, PATH_MAX); + } + entry->refreshedAt = now; + + if (!(entry->def = xenXMConfigReadFile(conn, entry->filename))) { + DEBUG("Failed to read %s", entry->filename); + if (!newborn) + virHashRemoveEntry(configCache, filename, NULL); + VIR_FREE(entry); + return -1; + } + + /* If its a completely new entry, it must be stuck into + the cache (refresh'd entries are already registered) */ + if (newborn) { + if (virHashAddEntry(configCache, entry->filename, entry) < 0) { + virDomainDefFree(entry->def); + VIR_FREE(entry); + xenXMError (conn, VIR_ERR_INTERNAL_ERROR, + "%s", _("xenXMConfigCacheRefresh: virHashAddEntry")); + return -1; + } + } + + /* See if we need to map this config file in as the primary owner + * of the domain in question + */ + if (!virHashLookup(nameConfigMap, entry->def->name)) { + if (virHashAddEntry(nameConfigMap, entry->def->name, entry->filename) < 0) { + virHashRemoveEntry(configCache, filename, NULL); + virDomainDefFree(entry->def); + VIR_FREE(entry); + } + } + DEBUG("Added config %s %s", entry->def->name, filename); + + return 0; +} /* This method is called by various methods to scan /etc/xen (or whatever directory was set by LIBVIRT_XM_CONFIG_DIR environment variable) and process any domain configs. It has rate-limited so never rescans more frequently than once every X seconds */ -static int xenXMConfigCacheRefresh (virConnectPtr conn) { +int xenXMConfigCacheRefresh (virConnectPtr conn) { DIR *dh; struct dirent *ent; time_t now = time(NULL); @@ -401,9 +512,7 @@ static int xenXMConfigCacheRefresh (virC } while ((ent = readdir(dh))) { - xenXMConfCachePtr entry; struct stat st; - int newborn = 0; char path[PATH_MAX]; /* @@ -447,62 +556,8 @@ static int xenXMConfigCacheRefresh (virC /* If we already have a matching entry and it is not modified, then carry on to next one*/ - if ((entry = virHashLookup(configCache, path))) { - char *nameowner; - - if (entry->refreshedAt >= st.st_mtime) { - entry->refreshedAt = now; - continue; - } - - /* If we currently own the name, then release it and - re-acquire it later - just in case it was renamed */ - nameowner = (char *)virHashLookup(nameConfigMap, entry->def->name); - if (nameowner && STREQ(nameowner, path)) { - virHashRemoveEntry(nameConfigMap, entry->def->name, NULL); - } - - /* Clear existing config entry which needs refresh */ - virDomainDefFree(entry->def); - entry->def = NULL; - } else { /* Completely new entry */ - newborn = 1; - if (VIR_ALLOC(entry) < 0) { - xenXMError (conn, VIR_ERR_NO_MEMORY, "%s", strerror(errno)); - goto cleanup; - } - memcpy(entry->filename, path, PATH_MAX); - } - entry->refreshedAt = now; - - if (!(entry->def = xenXMConfigReadFile(conn, entry->filename))) { - if (!newborn) - virHashRemoveEntry(configCache, path, NULL); - VIR_FREE(entry); - continue; - } - - /* If its a completely new entry, it must be stuck into - the cache (refresh'd entries are already registered) */ - if (newborn) { - if (virHashAddEntry(configCache, entry->filename, entry) < 0) { - virDomainDefFree(entry->def); - VIR_FREE(entry); - xenXMError (conn, VIR_ERR_INTERNAL_ERROR, - "%s", _("xenXMConfigCacheRefresh: virHashAddEntry")); - goto cleanup; - } - } - - /* See if we need to map this config file in as the primary owner - * of the domain in question - */ - if (!virHashLookup(nameConfigMap, entry->def->name)) { - if (virHashAddEntry(nameConfigMap, entry->def->name, entry->filename) < 0) { - virHashRemoveEntry(configCache, ent->d_name, NULL); - virDomainDefFree(entry->def); - VIR_FREE(entry); - } + if (xenXMConfigCacheAddFile(conn, path) < 0) { + /* Ignoring errors, since alot of stuff goes wrong in /etc/xen */ } } @@ -513,7 +568,6 @@ static int xenXMConfigCacheRefresh (virC virHashRemoveSet(configCache, xenXMConfigReaper, xenXMConfigFree, (const void*) &now); ret = 0; - cleanup: if (dh) closedir(dh); @@ -1503,8 +1557,10 @@ virDomainPtr xenXMDomainLookupByName(vir return (NULL); } +#ifndef WITH_XEN_INOTIFY if (xenXMConfigCacheRefresh (conn) < 0) return (NULL); +#endif if (!(filename = virHashLookup(nameConfigMap, domname))) return (NULL); @@ -1555,8 +1611,10 @@ virDomainPtr xenXMDomainLookupByUUID(vir return (NULL); } +#ifndef WITH_XEN_INOTIFY if (xenXMConfigCacheRefresh (conn) < 0) return (NULL); +#endif if (!(entry = virHashSearch(configCache, xenXMDomainSearchForUUID, (const void *)uuid))) { return (NULL); @@ -2208,8 +2266,10 @@ virDomainPtr xenXMDomainDefineXML(virCon if (conn->flags & VIR_CONNECT_RO) return (NULL); +#ifndef WITH_XEN_INOTIFY if (xenXMConfigCacheRefresh (conn) < 0) return (NULL); +#endif if (!(def = virDomainDefParseString(conn, priv->caps, xml))) return (NULL); @@ -2376,8 +2436,10 @@ int xenXMListDefinedDomains(virConnectPt return (-1); } +#ifndef WITH_XEN_INOTIFY if (xenXMConfigCacheRefresh (conn) < 0) return (-1); +#endif if (maxnames > virHashSize(configCache)) maxnames = virHashSize(configCache); @@ -2401,8 +2463,10 @@ int xenXMNumOfDefinedDomains(virConnectP return (-1); } +#ifndef WITH_XEN_INOTIFY if (xenXMConfigCacheRefresh (conn) < 0) return (-1); +#endif return virHashSize(nameConfigMap); } diff -r f10add69a53c src/xm_internal.h --- a/src/xm_internal.h Mon Nov 24 09:08:00 2008 -0500 +++ b/src/xm_internal.h Mon Nov 24 09:08:01 2008 -0500 @@ -32,6 +32,12 @@ extern struct xenUnifiedDriver xenXMDriv extern struct xenUnifiedDriver xenXMDriver; int xenXMInit (void); +virHashTablePtr xenXMGetConfigCache(void); +char *xenXMGetConfigDir(void); +int xenXMConfigCacheRefresh (virConnectPtr conn); +int xenXMConfigCacheAddFile(virConnectPtr conn, const char *filename); +int xenXMConfigCacheRemoveFile(virConnectPtr conn, const char *filename); + int xenXMOpen(virConnectPtr conn, virConnectAuthPtr auth, int flags); int xenXMClose(virConnectPtr conn); const char *xenXMGetType(virConnectPtr conn); diff -r f10add69a53c src/xs_internal.c --- a/src/xs_internal.c Mon Nov 24 09:08:00 2008 -0500 +++ b/src/xs_internal.c Mon Nov 24 10:19:11 2008 -0500 @@ -29,6 +29,10 @@ #include "virterror_internal.h" #include "datatypes.h" #include "driver.h" +#include "memory.h" +#include "event.h" +#include "logging.h" +#include "uuid.h" #include "xen_unified.h" #include "xs_internal.h" #include "xen_internal.h" /* for xenHypervisorCheckID */ @@ -42,6 +46,9 @@ #endif #ifndef PROXY +/* A list of active domain name/uuids */ +static xenUnifiedDomainInfoListPtr activeDomainList = NULL; + static char *xenStoreDomainGetOSType(virDomainPtr domain); struct xenUnifiedDriver xenStoreDriver = { @@ -302,7 +309,52 @@ xenStoreOpen(virConnectPtr conn, } return (-1); } - return (0); + +#ifndef PROXY + /* Init activeDomainList */ + if ( VIR_ALLOC(activeDomainList) < 0) { + virXenStoreError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("failed to allocate activeDomainList")); + return -1; + } + + /* Init watch list before filling in domInfoList, + so we can know if it is the first time through + when the callback fires */ + if ( VIR_ALLOC(priv->xsWatchList) < 0 ) { + virXenStoreError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("failed to allocate xsWatchList")); + return -1; + } + + /* This will get called once at start */ + if ( xenStoreAddWatch(conn, "@releaseDomain", + "releaseDomain", xenStoreDomainReleased, priv) < 0 ) + { + virXenStoreError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("adding watch @releaseDomain")); + return -1; + } + + /* The initial call of this will fill domInfoList */ + if( xenStoreAddWatch(conn, "@introduceDomain", + "introduceDomain", xenStoreDomainIntroduced, priv) < 0 ) + { + virXenStoreError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("adding watch @introduceDomain")); + return -1; + } + + /* Add an event handle */ + if ((priv->xsWatch = virEventAddHandle(xs_fileno(priv->xshandle), + VIR_EVENT_HANDLE_READABLE, + xenStoreWatchEvent, + conn, + NULL)) < 0) + DEBUG0("Failed to add event handle, disabling events\n"); + +#endif //PROXY + return 0; } /** @@ -324,10 +376,29 @@ xenStoreClose(virConnectPtr conn) } priv = (xenUnifiedPrivatePtr) conn->privateData; - if (priv->xshandle == NULL) - return(-1); - + + if (xenStoreRemoveWatch(conn, "@introduceDomain", "introduceDomain") < 0) { + DEBUG0("Warning, could not remove @introduceDomain watch"); + /* not fatal */ + } + + if (xenStoreRemoveWatch(conn, "@releaseDomain", "releaseDomain") < 0) { + DEBUG0("Warning, could not remove @releaseDomain watch"); + /* not fatal */ + } + + xenStoreWatchListFree(priv->xsWatchList); +#ifndef PROXY + xenUnifiedDomainInfoListFree(activeDomainList); +#endif + if (priv->xshandle == NULL) + return(-1); + + if (priv->xsWatch != -1) + virEventRemoveHandle(priv->xsWatch); xs_daemon_close(priv->xshandle); + priv->xshandle = NULL; + return (0); } @@ -920,3 +991,343 @@ char *xenStoreDomainGetName(virConnectPt return xs_read(priv->xshandle, 0, prop, &len); } +#ifndef PROXY +int xenStoreDomainGetUUID(virConnectPtr conn, + int id, + unsigned char *uuid) { + char prop[200]; + xenUnifiedPrivatePtr priv; + unsigned int len; + char *uuidstr; + int ret = 0; + + priv = (xenUnifiedPrivatePtr) conn->privateData; + if (priv->xshandle == NULL) + return -1; + + snprintf(prop, 199, "/local/domain/%d/vm", id); + prop[199] = 0; + // This will return something like + // /vm/00000000-0000-0000-0000-000000000000 + uuidstr = xs_read(priv->xshandle, 0, prop, &len); + + // remove "/vm/" + ret = virUUIDParse(uuidstr + 4, uuid); + + VIR_FREE(uuidstr); + + return ret; +} +#endif //PROXY + +void xenStoreWatchListFree(xenStoreWatchListPtr list) +{ + int i; + for (i=0; i<list->count; i++) { + VIR_FREE(list->watches[i]->path); + VIR_FREE(list->watches[i]->token); + VIR_FREE(list->watches[i]); + } + VIR_FREE(list); +} + +int xenStoreAddWatch(virConnectPtr conn, + const char *path, + const char *token, + xenStoreWatchCallback cb, + void *opaque) +{ + xenStoreWatchPtr watch; + int n; + xenStoreWatchListPtr list; + xenUnifiedPrivatePtr priv = (xenUnifiedPrivatePtr) conn->privateData; + + if (priv->xshandle == NULL) + return -1; + + list = priv->xsWatchList; + if(!list) + return -1; + + /* check if we already have this callback on our list */ + for (n=0; n < list->count; n++) { + if( STREQ(list->watches[n]->path, path) && + STREQ(list->watches[n]->token, token)) { + virXenStoreError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("watch already tracked")); + return -1; + } + } + + if (VIR_ALLOC(watch) < 0) + return -1; + watch->path = strdup(path); + watch->token = strdup(token); + watch->cb = cb; + watch->opaque = opaque; + + /* Make space on list */ + n = list->count; + if (VIR_REALLOC_N(list->watches, n + 1) < 0) { + virXenStoreError(NULL, VIR_ERR_INTERNAL_ERROR, + "%s", _("reallocating list")); + VIR_FREE(watch); + return -1; + } + + list->watches[n] = watch; + list->count++; + + conn->refs++; + + return xs_watch(priv->xshandle, watch->path, watch->token); +} + +int xenStoreRemoveWatch(virConnectPtr conn, + const char *path, + const char *token) +{ + int i; + xenStoreWatchListPtr list; + xenUnifiedPrivatePtr priv = (xenUnifiedPrivatePtr) conn->privateData; + + if (priv->xshandle == NULL) + return -1; + + list = priv->xsWatchList; + if(!list) + return -1; + + for (i = 0 ; i < list->count ; i++) { + if( STREQ(list->watches[i]->path, path) && + STREQ(list->watches[i]->token, token)) { + + if (!xs_unwatch(priv->xshandle, + list->watches[i]->path, + list->watches[i]->path)) + { + DEBUG0("WARNING: Could not remove watch"); + /* Not fatal, continue */ + } + + VIR_FREE(list->watches[i]->path); + VIR_FREE(list->watches[i]->token); + VIR_FREE(list->watches[i]); + + if (i < (list->count - 1)) + memmove(list->watches + i, + list->watches + i + 1, + sizeof(*(list->watches)) * + (list->count - (i + 1))); + + if (VIR_REALLOC_N(list->watches, + list->count - 1) < 0) { + ; /* Failure to reduce memory allocation isn't fatal */ + } + list->count--; +#ifndef PROXY + virUnrefConnect(conn); +#endif + return 0; + } + } + return -1; +} + +xenStoreWatchPtr xenStoreFindWatch(xenStoreWatchListPtr list, + const char *path, + const char *token) +{ + int i; + for (i = 0 ; i < list->count ; i++) + if( STREQ(path, list->watches[i]->path) && + STREQ(token, list->watches[i]->token) ) + return list->watches[i]; + + return NULL; +} + +void xenStoreWatchEvent(int watch ATTRIBUTE_UNUSED, + int fd ATTRIBUTE_UNUSED, + int events ATTRIBUTE_UNUSED, + void *data) +{ + char **event; + char *path; + char *token; + unsigned int stringCount; + xenStoreWatchPtr sw; + + virConnectPtr conn = data; + xenUnifiedPrivatePtr priv = (xenUnifiedPrivatePtr) conn->privateData; + if(!priv) return; + if(!priv->xshandle) return; + + event = xs_read_watch(priv->xshandle, &stringCount); + if (!event) + return; + + path = event[XS_WATCH_PATH]; + token = event[XS_WATCH_TOKEN]; + + sw = xenStoreFindWatch(priv->xsWatchList, path, token); + if( sw ) + sw->cb(conn, path, token, sw->opaque); + VIR_FREE(event); +} + +#ifndef PROXY +/* The domain callback for the @introduceDomain watch */ +int xenStoreDomainIntroduced(virConnectPtr conn, + const char *path ATTRIBUTE_UNUSED, + const char *token ATTRIBUTE_UNUSED, + void *opaque) +{ + int i, j, found, missing = 0, retries = 20; + int new_domain_cnt; + int *new_domids; + int nread; + + xenUnifiedPrivatePtr priv = (xenUnifiedPrivatePtr) opaque; + +retry: + new_domain_cnt = xenStoreNumOfDomains(conn); + if( VIR_ALLOC_N(new_domids,new_domain_cnt) < 0 ) { + virXenStoreError(NULL, VIR_ERR_NO_MEMORY, + "%s", _("failed to allocate domids")); + return -1; + } + nread = xenStoreListDomains(conn, new_domids, new_domain_cnt); + if (nread != new_domain_cnt) { + // mismatch. retry this read + VIR_FREE(new_domids); + goto retry; + } + + missing = 0; + for (i=0 ; i < new_domain_cnt ; i++) { + found = 0; + for (j = 0 ; j < activeDomainList->count ; j++) { + if (activeDomainList->doms[j]->id == new_domids[i]) { + found = 1; + break; + } + } + + if (!found) { + virDomainPtr dom; + char *name; + unsigned char uuid[VIR_UUID_BUFLEN]; + + if (!(name = xenStoreDomainGetName(conn, new_domids[i]))) { + missing = 1; + continue; + } + if (xenStoreDomainGetUUID(conn, new_domids[i], uuid) < 0) { + missing = 1; + VIR_FREE(name); + continue; + } + + dom = virGetDomain(conn, name, uuid); + if (dom) { + dom->id = new_domids[i]; + + /* This domain was not in the old list. Emit an event */ + xenUnifiedDomainEventDispatch(priv, dom, + VIR_DOMAIN_EVENT_STARTED, + VIR_DOMAIN_EVENT_STARTED_BOOTED); + + /* Add to the list */ + xenUnifiedAddDomainInfo(activeDomainList, + new_domids[i], name, uuid); + + virUnrefDomain(dom); + } + + VIR_FREE(name); + } + } + VIR_FREE(new_domids); + + if (missing && retries--) { + DEBUG0("Some domains were missing, trying again"); + usleep(100 * 1000); + goto retry; + } + return 0; +} + +/* The domain callback for the @destroyDomain watch */ +int xenStoreDomainReleased(virConnectPtr conn, + const char *path ATTRIBUTE_UNUSED, + const char *token ATTRIBUTE_UNUSED, + void *opaque) +{ + int i, j, found, removed, retries = 20; + int new_domain_cnt; + int *new_domids; + int nread; + + xenUnifiedPrivatePtr priv = (xenUnifiedPrivatePtr) opaque; + + if(!activeDomainList->count) return 0; + +retry: + new_domain_cnt = xenStoreNumOfDomains(conn); + + if( VIR_ALLOC_N(new_domids,new_domain_cnt) < 0 ) { + virXenStoreError(NULL, VIR_ERR_NO_MEMORY, + "%s", _("failed to allocate domids")); + return -1; + } + nread = xenStoreListDomains(conn, new_domids, new_domain_cnt); + if (nread != new_domain_cnt) { + // mismatch. retry this read + VIR_FREE(new_domids); + goto retry; + } + + removed = 0; + for (j=0 ; j < activeDomainList->count ; j++) { + found = 0; + for (i=0 ; i < new_domain_cnt ; i++) { + if (activeDomainList->doms[j]->id == new_domids[i]) { + found = 1; + break; + } + } + + if (!found) { + virDomainPtr dom = virGetDomain(conn, + activeDomainList->doms[j]->name, + activeDomainList->doms[j]->uuid); + if(dom) { + dom->id = -1; + /* This domain was not in the new list. Emit an event */ + xenUnifiedDomainEventDispatch(priv, dom, + VIR_DOMAIN_EVENT_STOPPED, + VIR_DOMAIN_EVENT_STOPPED_SHUTDOWN); + /* Remove from the list */ + xenUnifiedRemoveDomainInfo(activeDomainList, + activeDomainList->doms[j]->id, + activeDomainList->doms[j]->name, + activeDomainList->doms[j]->uuid); + + virUnrefDomain(dom); + removed = 1; + } + } + } + + VIR_FREE(new_domids); + + if (!removed && retries--) { + DEBUG0("No domains removed, retrying"); + usleep(100 * 1000); + goto retry; + } + return 0; +} + +#endif //PROXY diff -r f10add69a53c src/xs_internal.h --- a/src/xs_internal.h Mon Nov 24 09:08:00 2008 -0500 +++ b/src/xs_internal.h Mon Nov 24 10:18:59 2008 -0500 @@ -51,5 +51,57 @@ char * xenStoreDomainGetDiskID(virConne const char *dev); char * xenStoreDomainGetName(virConnectPtr conn, int id); +int xenStoreDomainGetUUID(virConnectPtr conn, + int id, + unsigned char *uuid); +typedef int (*xenStoreWatchCallback)(virConnectPtr conn, + const char *path, + const char *token, + void *opaque); + +struct _xenStoreWatch { + char *path; + char *token; + xenStoreWatchCallback cb; + void *opaque; +}; +typedef struct _xenStoreWatch xenStoreWatch; +typedef xenStoreWatch *xenStoreWatchPtr; + +struct _xenStoreWatchList { + unsigned int count; + xenStoreWatchPtr *watches; +}; +typedef struct _xenStoreWatchList xenStoreWatchList; +typedef xenStoreWatchList *xenStoreWatchListPtr; + + +void xenStoreWatchListFree(xenStoreWatchListPtr head); + +int xenStoreAddWatch(virConnectPtr conn, + const char *path, + const char *token, + xenStoreWatchCallback cb, + void *opaque); +int xenStoreRemoveWatch(virConnectPtr conn, + const char *path, + const char *token); +xenStoreWatchPtr xenStoreFindWatch(xenStoreWatchListPtr list, + const char *path, + const char *token); + +void xenStoreWatchEvent(int watch, int fd, int events, void *data); + +/* domain events */ +int xenStoreDomainIntroduced(virConnectPtr conn, + const char *path, + const char *token, + void *opaque); +int xenStoreDomainReleased(virConnectPtr conn, + const char *path, + const char *token, + void *opaque); + +int xenStoreDomainEventEmitted(virDomainEventType evt); #endif /* __VIR_XS_INTERNAL_H__ */ -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

On Mon, Nov 24, 2008 at 03:44:28PM +0000, Daniel P. Berrange wrote:

On Fri, Nov 21, 2008 at 02:11:39PM -0500, Ben Guthro wrote:

...

I have integrated your changes, and moved this bit referenced below, so it works with my setup. Could you please test with yours, and make sure I didn't break anything?

It didn't break anything, but I found more edge cases & a couple of bugs and optimizations.

Changes in this version

- We now also monitor IN_MOVED_TO and IN_MOVED_FROM, so that we catch 'mv foo /etc/xen' and 'mv /etc/xen/foo /tmp' as define & undefine events - On the @introduced watch handler we'll retry if we don't find the UUID / name first time around, to deal with inevitable race while XenD populates this info in xenstore. - In the @introduced & @removed handlers, simply to just check the domain ID, not name/uuid, since running VMs are unique on ID. - Fix UUID stored to be the raw unsigned char, not NULL terminated string form, so we're passing correct data into virGetDomain.

This is now working reliably with my RHEL-5 Xen for all events, and should be a little more resilent on new Xen too.

Okay, based on previous reviews and this thread, I just suggest

+++ b/configure.in Mon Nov 24 09:08:01 2008 -0500 @@ -147,6 +147,8 @@ dnl Allow to build without Xen, QEMU/KVM dnl Allow to build without Xen, QEMU/KVM, test or remote driver AC_ARG_WITH([xen], [ --with-xen add XEN support (on)],[],[with_xen=yes]) +AC_ARG_WITH([xen-inotify], +[ --with-xen-inotify add XEN inotify support (on)],[],[with_xen_inotify=yes]) AC_ARG_WITH([qemu], [ --with-qemu add QEMU/KVM support (on)],[],[with_qemu=yes]) AC_ARG_WITH([uml], @@ -333,6 +335,17 @@ AM_CONDITIONAL([WITH_XEN], [test "$with_ AM_CONDITIONAL([WITH_XEN], [test "$with_xen" = "yes"]) AC_SUBST([XEN_CFLAGS]) AC_SUBST([XEN_LIBS]) + +dnl +dnl check for kernel headers required by xen_inotify +dnl +if test "$with_xen_inotify" != "no"; then + AC_CHECK_HEADER([linux/inotify.h],[],[with_xen_inotify=no]) +fi +if test "$with_xen_inotify" = "yes"; then + AC_DEFINE_UNQUOTED([WITH_XEN_INOTIFY], 1,[whether Xen inotify sub-driver is enabled]) +fi +AM_CONDITIONAL([WITH_XEN_INOTIFY], [test "$with_xen_inotify" = "yes"])

I would suggest we also test there that xen is enable before adding xen_inotify support. I don't think it makes sense to activate the later if the former is not selected or detected. +1 Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/