[libvirt] [PATCH 00/14] Add a virtlockd lock manager daemon
The lock manager infrastructure we recently added to QEMU only has two possible drivers at this time, 'nop' and 'sanlock'. The former does absolutely nothing, while the latter requires a 3rd party package installed and is a little heavy on disk I/O and storage requirements. This series adds a new daemon 'virtlockd' which is intended to be enabled by default on all hosts running 'libvirtd'. This daemon provides a service for disk locking based on the traditional fcntl() lock primitives. There is a new libvirt manager plugin which talks to this daemon over RPC. The reason for doing the locks in a separate process is that we want the locks to remain active, even if libvirtd crashes, or is restarted. The virtlockd daemon has this one single job so should be pretty reliable and selfcontained. This patch series really benefits from the new RPC APIs, requiring minimal code for the new daemon / client At this time, virtlockd does not lock the actual disk files, but instead creates a lockspace & leases under /var/lib/libvirt/lockd. The lockspace we use for disks is named org.libvirt.lockd.files, and lease names are based on a SHA256 checksum of the fully qualified disk name. eg /var/lib/libvirt/lockd/org.libvirt.lockd.files/adf94fc33a24da1abff7dd7374a9919bb51efee646da8c3ac464c10cd59750bd These leases are all zero-bytes long and no I/O is ever performed on them, only fcntl() is used. So there is material overhead. Whenever creating or deleting leases, we first acquire a lock on /var/lib/libvirt/lockd/org.libvirt.lockd.files/org.libvirt.lockd.index A non-root virtlockd will instead use $HOME/.libvirt/lockd By default we gain protection out of the box against - Starting two guests on the same host with the same disk image not marked with <shareable/> - libvirtd getting confused and forgetting a guest, allowing it to be started for a 2nd time If the admin mounts a shared filesytem (eg NFS) on /var/lib/libvirt/lockd then this protection is extended across all hosts sharing that mount volume. As part of this series, I also introduce support for systemd services for libvirtd and libvir-guests.
From: "Daniel P. Berrange" <berrange@redhat.com> * src/rpc/virnetserverprogram.h: Remove unused typedef for virNetServerProgramErrorHander function callback * daemon/remote.h: Remove decl for non-existant variables --- daemon/remote.h | 2 -- src/rpc/virnetserverprogram.h | 3 --- 2 files changed, 0 insertions(+), 5 deletions(-) diff --git a/daemon/remote.h b/daemon/remote.h index c9bf5d7..5444e47 100644 --- a/daemon/remote.h +++ b/daemon/remote.h @@ -31,11 +31,9 @@ extern virNetServerProgramProc remoteProcs[]; extern size_t remoteNProcs; -extern virNetServerProgramErrorHander remoteErr; extern virNetServerProgramProc qemuProcs[]; extern size_t qemuNProcs; -extern virNetServerProgramErrorHander qemuErr; int remoteClientInitHook(virNetServerPtr srv, virNetServerClientPtr client); diff --git a/src/rpc/virnetserverprogram.h b/src/rpc/virnetserverprogram.h index 98071a2..ca31b7e 100644 --- a/src/rpc/virnetserverprogram.h +++ b/src/rpc/virnetserverprogram.h @@ -39,9 +39,6 @@ typedef virNetServerProgram *virNetServerProgramPtr; typedef struct _virNetServerProgramProc virNetServerProgramProc; typedef virNetServerProgramProc *virNetServerProgramProcPtr; -typedef struct _virNetServerProgramErrorHandler virNetServerProgramErrorHander; -typedef virNetServerProgramErrorHander *virNetServerProgramErrorHanderPtr; - typedef int (*virNetServerProgramDispatchFunc)(virNetServerPtr server, virNetServerClientPtr client, virNetMessageHeaderPtr hdr, -- 1.7.6
On 07/07/2011 08:17 AM, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
* src/rpc/virnetserverprogram.h: Remove unused typedef for virNetServerProgramErrorHander function callback * daemon/remote.h: Remove decl for non-existant variables --- daemon/remote.h | 2 -- src/rpc/virnetserverprogram.h | 3 --- 2 files changed, 0 insertions(+), 5 deletions(-)
ACK. Pure deletion patches are always fun :) -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
From: "Daniel P. Berrange" <berrange@redhat.com> Add some simple wrappers around the fcntl() discretionary file locking capability. * src/util/util.c, src/util/util.h, src/libvirt_private.syms: Add virFileLock and virFileUnlock APIs --- src/libvirt_private.syms | 2 ++ src/util/util.c | 33 +++++++++++++++++++++++++++++++++ src/util/util.h | 3 +++ 3 files changed, 38 insertions(+), 0 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index ae0d199..0746dee 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -993,6 +993,7 @@ virFileFindMountPoint; virFileHasSuffix; virFileIsExecutable; virFileLinkPointsTo; +virFileLock; virFileMakePath; virFileMatchesNameSuffix; virFileOpenAs; @@ -1004,6 +1005,7 @@ virFileReadPid; virFileResolveLink; virFileSanitizePath; virFileStripSuffix; +virFileUnlock; virFileWaitForDevices; virFileWriteStr; virFindFileInPath; diff --git a/src/util/util.c b/src/util/util.c index 4710fc5..1d8083f 100644 --- a/src/util/util.c +++ b/src/util/util.c @@ -1175,6 +1175,39 @@ int virFileOpenTtyAt(const char *ptmx ATTRIBUTE_UNUSED, } #endif + +int virFileLock(int fd, bool shared, off_t start, off_t len) +{ + struct flock fl = { + .l_type = shared ? F_RDLCK : F_WRLCK, + .l_whence = SEEK_SET, + .l_start = start, + .l_len = len, + }; + + if (fcntl(fd, F_SETLK, &fl) < 0) + return -errno; + + return 0; +} + + +int virFileUnlock(int fd, off_t start, off_t len) +{ + struct flock fl = { + .l_type = F_UNLCK, + .l_whence = SEEK_SET, + .l_start = start, + .l_len = len, + }; + + if (fcntl(fd, F_SETLK, &fl) < 0) + return -errno; + + return 0; +} + + char* virFilePid(const char *dir, const char* name) { char *pidfile; diff --git a/src/util/util.h b/src/util/util.h index 1555653..820c9ff 100644 --- a/src/util/util.h +++ b/src/util/util.h @@ -117,6 +117,9 @@ int virFileOpenTtyAt(const char *ptmx, char **ttyName, int rawmode); +int virFileLock(int fd, bool shared, off_t start, off_t len); +int virFileUnlock(int fd, off_t start, off_t len); + char* virFilePid(const char *dir, const char *name); -- 1.7.6
On 07/07/2011 08:17 AM, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
Add some simple wrappers around the fcntl() discretionary file locking capability.
* src/util/util.c, src/util/util.h, src/libvirt_private.syms: Add virFileLock and virFileUnlock APIs --- src/libvirt_private.syms | 2 ++ src/util/util.c | 33 +++++++++++++++++++++++++++++++++ src/util/util.h | 3 +++ 3 files changed, 38 insertions(+), 0 deletions(-)
+ +int virFileLock(int fd, bool shared, off_t start, off_t len) +{ + struct flock fl = { + .l_type = shared ? F_RDLCK : F_WRLCK, + .l_whence = SEEK_SET, + .l_start = start, + .l_len = len, + };
This won't compile on mingw. We need to either make a no-op stub, or figure out how to do rudimentary file locking on mingw, or get gnulib to figure it out (probably the first option is easiest for now). -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
From: "Daniel P. Berrange" <berrange@redhat.com> * src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add virNetSocketDupFD() * src/rpc/virnetclient.c, src/rpc/virnetclient.h: Add virNetClientDupFD() and virNetClientGetFD() --- src/rpc/virnetclient.c | 20 ++++++++++++++++++++ src/rpc/virnetclient.h | 3 +++ src/rpc/virnetsocket.c | 12 ++++++++++++ src/rpc/virnetsocket.h | 2 ++ 4 files changed, 37 insertions(+), 0 deletions(-) diff --git a/src/rpc/virnetclient.c b/src/rpc/virnetclient.c index 615de6c..ecc5c37 100644 --- a/src/rpc/virnetclient.c +++ b/src/rpc/virnetclient.c @@ -217,6 +217,26 @@ void virNetClientRef(virNetClientPtr client) } +int virNetClientGetFD(virNetClientPtr client) +{ + int fd; + virNetClientLock(client); + fd = virNetSocketGetFD(client->sock); + virNetClientUnlock(client); + return fd; +} + + +int virNetClientDupFD(virNetClientPtr client) +{ + int fd; + virNetClientLock(client); + fd = virNetSocketDupFD(client->sock); + virNetClientUnlock(client); + return fd; +} + + void virNetClientFree(virNetClientPtr client) { int i; diff --git a/src/rpc/virnetclient.h b/src/rpc/virnetclient.h index de0782c..381d647 100644 --- a/src/rpc/virnetclient.h +++ b/src/rpc/virnetclient.h @@ -51,6 +51,9 @@ virNetClientPtr virNetClientNewExternal(const char **cmdargv); void virNetClientRef(virNetClientPtr client); +int virNetClientGetFD(virNetClientPtr client); +int virNetClientDupFD(virNetClientPtr client); + int virNetClientAddProgram(virNetClientPtr client, virNetClientProgramPtr prog); diff --git a/src/rpc/virnetsocket.c b/src/rpc/virnetsocket.c index 4b0c2ee..aff68bc 100644 --- a/src/rpc/virnetsocket.c +++ b/src/rpc/virnetsocket.c @@ -675,6 +675,18 @@ int virNetSocketGetFD(virNetSocketPtr sock) } +int virNetSocketDupFD(virNetSocketPtr sock) +{ + int fd; + if ((fd = dup(sock->fd)) < 0) { + virReportSystemError(errno, "%s", + _("Unable to copy socket file handle")); + return -1; + } + return fd; +} + + bool virNetSocketIsLocal(virNetSocketPtr sock) { if (sock->localAddr.data.sa.sa_family == AF_UNIX) diff --git a/src/rpc/virnetsocket.h b/src/rpc/virnetsocket.h index 356d6c6..8053213 100644 --- a/src/rpc/virnetsocket.h +++ b/src/rpc/virnetsocket.h @@ -75,6 +75,8 @@ int virNetSocketNewConnectExternal(const char **cmdargv, virNetSocketPtr *addr); int virNetSocketGetFD(virNetSocketPtr sock); +int virNetSocketDupFD(virNetSocketPtr sock); + bool virNetSocketIsLocal(virNetSocketPtr sock); int virNetSocketGetPort(virNetSocketPtr sock); -- 1.7.6
On 07/07/2011 08:17 AM, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add virNetSocketDupFD() * src/rpc/virnetclient.c, src/rpc/virnetclient.h: Add virNetClientDupFD() and virNetClientGetFD() --- src/rpc/virnetclient.c | 20 ++++++++++++++++++++ src/rpc/virnetclient.h | 3 +++ src/rpc/virnetsocket.c | 12 ++++++++++++ src/rpc/virnetsocket.h | 2 ++ 4 files changed, 37 insertions(+), 0 deletions(-)
In the interest of using atomic FD_CLOEXEC in more places, I would suggest making a change to this API:
+ +int virNetClientDupFD(virNetClientPtr client)
This should be virNetClientDupFD(virNetClientPtr client, bool cloexec)
+int virNetSocketDupFD(virNetSocketPtr sock) +{ + int fd; + if ((fd = dup(sock->fd)) < 0) {
And this should be: int virNetSocketDupFD(virNetSocketPtr sock, bool cloexec) { int fd; if (cloexec) fd = fcntl(F_DUPFD_CLOEXEC, sock->fd); else fd = dup(sock->fd); if (fd < 0) { ... and rely on gnulib's guarantee that fcntl(F_DUPFD_CLOEXEC) works. (Oh, that means we need to import the fcntl module in bootstrap.conf). -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
From: "Daniel P. Berrange" <berrange@redhat.com> When replacing the default SEGV/ABORT/BUS signal handlers you can't rely on the process being terminated after your custom handler runs. It is neccessary to manually restore the default handler and then re-raise the signal * src/rpc/virnetserver.c: Restore default handler and raise signal --- src/rpc/virnetserver.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 5e1719b..d5e9d0a 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -257,8 +257,9 @@ static void virNetServerFatalSignal(int sig, siginfo_t *siginfo ATTRIBUTE_UNUSED #ifdef SIGUSR2 if (sig != SIGUSR2) { #endif - sig_action.sa_handler = SIG_IGN; + sig_action.sa_handler = SIG_DFL; sigaction(sig, &sig_action, NULL); + raise(sig); #ifdef SIGUSR2 } #endif -- 1.7.6
On 07/07/2011 08:17 AM, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
When replacing the default SEGV/ABORT/BUS signal handlers you can't rely on the process being terminated after your custom handler runs. It is neccessary to manually restore the default
s/neccessary/necessary/
handler and then re-raise the signal
* src/rpc/virnetserver.c: Restore default handler and raise signal --- src/rpc/virnetserver.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
ACK. There's a gnulib module for raise() which we are not using, but that module says it is only needed for _really_ old platforms (even mingw already has raise), so I don't think we need to worry about using it. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
From: "Daniel P. Berrange" <berrange@redhat.com> If libvirtd crashes then the pidfile may not be cleaned up, making a later restart fail, even though the original process no longer exists. Instead of simply using file creation as the test for successful pidfile acquisition, actually take out a lock on the pidfile. To avoid races, after locking the pidfile, verify that the inode hasn't changed. Also make sure the unprivileged libvirtd now acquires the pidfile, instead of relying on the UNIX socket to ensure mutual exclusion * daemon/libvirtd.c: Take a fcntl() based lock on the pidfile --- daemon/libvirtd.c | 93 +++++++++++++++++++++++++++++++++------------------- 1 files changed, 59 insertions(+), 34 deletions(-) diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 06d2077..bb388d4 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -53,6 +53,7 @@ #include "hooks.h" #include "uuid.h" #include "virtaudit.h" +#include "intprops.h" #ifdef WITH_DRIVER_MODULES # include "driver.h" @@ -258,42 +259,65 @@ static int daemonForkIntoBackground(const char *argv0) } } -static int daemonWritePidFile(const char *pidFile, const char *argv0) -{ +static int +daemonAcquirePidFile(const char *argv0, const char *pidFile) { int fd; - FILE *fh; char ebuf[1024]; + unsigned long long pid = (unsigned long long)getpid(); + char pidstr[INT_BUFSIZE_BOUND(pid)]; if (pidFile[0] == '\0') return 0; - if ((fd = open(pidFile, O_WRONLY|O_CREAT|O_EXCL, 0644)) < 0) { - VIR_ERROR(_("Failed to open pid file '%s' : %s"), - pidFile, virStrerror(errno, ebuf, sizeof ebuf)); - return -1; - } + while (1) { + struct stat a, b; + if ((fd = open(pidFile, O_WRONLY|O_CREAT, 0644)) < 0) { + VIR_ERROR(_("%s: Failed to open pid file '%s' : %s"), + argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + return -1; + } + + if (fstat(fd, &b) < 0) { + VIR_ERROR(_("%s: Pid file '%s' disappeared: %s"), + argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + VIR_FORCE_CLOSE(fd); + return -1; + } + + if (virFileLock(fd, false, 0, 1) < 0) { + VIR_ERROR(_("%s: Failed to acquire pid file '%s' : %s"), + argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + VIR_FORCE_CLOSE(fd); + return -1; + } + + /* Now make sure the pidfile we locked is the same + * one that now exists on the filesystem + */ + + if (stat(pidFile, &a) < 0) { + VIR_DEBUG("%s: Pid file '%s' disappeared: %s", + argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + VIR_FORCE_CLOSE(fd); + continue; + } - if (!(fh = VIR_FDOPEN(fd, "w"))) { - VIR_ERROR(_("Failed to fdopen pid file '%s' : %s"), - pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + if (a.st_ino == b.st_ino) + break; + + VIR_DEBUG("%s: Pid file '%s' was recreated", + argv0, pidFile); VIR_FORCE_CLOSE(fd); - return -1; } - if (fprintf(fh, "%lu\n", (unsigned long)getpid()) < 0) { - VIR_ERROR(_("%s: Failed to write to pid file '%s' : %s"), - argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); - VIR_FORCE_FCLOSE(fh); - return -1; - } + snprintf(pidstr, sizeof(pidstr), "%llu", pid); - if (VIR_FCLOSE(fh) == EOF) { - VIR_ERROR(_("%s: Failed to close pid file '%s' : %s"), - argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); - return -1; + if (safewrite(fd, pidstr, strlen(pidstr)) < 0) { + VIR_ERROR(_("Failed to write to pid file '%s' : %s"), + pidFile, virStrerror(errno, ebuf, sizeof ebuf)); } - return 0; + return fd; } @@ -1269,6 +1293,7 @@ int main(int argc, char **argv) { char *remote_config_file = NULL; int statuswrite = -1; int ret = 1; + int pid_file_fd = -1; char *pid_file = NULL; char *sock_file = NULL; char *sock_file_ro = NULL; @@ -1382,7 +1407,7 @@ int main(int argc, char **argv) { if (daemonSetupLogging(config, privileged, verbose, godaemon) < 0) exit(EXIT_FAILURE); - if (!pid_file && privileged && + if (!pid_file && daemonPidFilePath(privileged, &pid_file) < 0) exit(EXIT_FAILURE); @@ -1409,14 +1434,6 @@ int main(int argc, char **argv) { } } - /* If we have a pidfile set, claim it now, exiting if already taken */ - if (pid_file != NULL && - daemonWritePidFile(pid_file, argv[0]) < 0) { - VIR_FREE(pid_file); /* Prevent unlinking of someone else's pid ! */ - ret = VIR_DAEMON_ERR_PIDFILE; - goto cleanup; - } - /* Ensure the rundir exists (on tmpfs on some systems) */ if (privileged) { const char *rundir = LOCALSTATEDIR "/run/libvirt"; @@ -1436,6 +1453,12 @@ int main(int argc, char **argv) { umask(old_umask); } + /* Try to claim the pidfile, existing if we can't */ + if ((pid_file_fd = daemonAcquirePidFile(argv[0], pid_file)) < 0) { + ret = VIR_DAEMON_ERR_PIDFILE; + goto cleanup; + } + if (!(srv = virNetServerNew(config->min_workers, config->max_workers, config->max_clients, @@ -1570,8 +1593,10 @@ cleanup: } VIR_FORCE_CLOSE(statuswrite); } - if (pid_file) - unlink (pid_file); + if (pid_file_fd != -1) { + unlink(pid_file); + VIR_FORCE_CLOSE(pid_file_fd); + } VIR_FREE(sock_file); VIR_FREE(sock_file_ro); -- 1.7.6
On 07/07/2011 08:17 AM, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
If libvirtd crashes then the pidfile may not be cleaned up, making a later restart fail, even though the original process no longer exists.
Instead of simply using file creation as the test for successful pidfile acquisition, actually take out a lock on the pidfile.
To avoid races, after locking the pidfile, verify that the inode hasn't changed.
Also make sure the unprivileged libvirtd now acquires the pidfile, instead of relying on the UNIX socket to ensure mutual exclusion
Cool idea.
-static int daemonWritePidFile(const char *pidFile, const char *argv0) -{ +static int +daemonAcquirePidFile(const char *argv0, const char *pidFile) {
Why'd you hoist the { to the previous line? Our convention has been (slowly) leaning towards a function body starting on column 1.
int fd; - FILE *fh; char ebuf[1024]; + unsigned long long pid = (unsigned long long)getpid();
See my comments in an earlier thread about our existing assumptions that pid_t fits in int. In fact, I would be okay with: verify(sizeof(pid_t) <= sizeof(int)); int pid = getpid(); char pidstr[INT_BUFSIZE_BOUND(pid)]; ... snprintf(pidstr, sizeof(pidstr), "%u", pid);
+ char pidstr[INT_BUFSIZE_BOUND(pid)];
if (pidFile[0] == '\0') return 0;
- if ((fd = open(pidFile, O_WRONLY|O_CREAT|O_EXCL, 0644)) < 0) { - VIR_ERROR(_("Failed to open pid file '%s' : %s"), - pidFile, virStrerror(errno, ebuf, sizeof ebuf)); - return -1; - } + while (1) { + struct stat a, b; + if ((fd = open(pidFile, O_WRONLY|O_CREAT, 0644)) < 0) { + VIR_ERROR(_("%s: Failed to open pid file '%s' : %s"), + argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + return -1; + } + + if (fstat(fd, &b) < 0) { + VIR_ERROR(_("%s: Pid file '%s' disappeared: %s"),
Misleading error message. fstat can indeed fail (although such failures are rare), but not because a file disappeared - after all, you have an fd open to the file.
- if (fprintf(fh, "%lu\n", (unsigned long)getpid()) < 0) { - VIR_ERROR(_("%s: Failed to write to pid file '%s' : %s"), - argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); - VIR_FORCE_FCLOSE(fh); - return -1; - } + snprintf(pidstr, sizeof(pidstr), "%llu", pid);
- if (VIR_FCLOSE(fh) == EOF) { - VIR_ERROR(_("%s: Failed to close pid file '%s' : %s"), - argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); - return -1; + if (safewrite(fd, pidstr, strlen(pidstr)) < 0) {
Nice conversion from FILE* to write().
@@ -1436,6 +1453,12 @@ int main(int argc, char **argv) { umask(old_umask); }
+ /* Try to claim the pidfile, existing if we can't */
s/existing/exiting/
+ if ((pid_file_fd = daemonAcquirePidFile(argv[0], pid_file)) < 0) { + ret = VIR_DAEMON_ERR_PIDFILE; + goto cleanup; + } + if (!(srv = virNetServerNew(config->min_workers, config->max_workers, config->max_clients, @@ -1570,8 +1593,10 @@ cleanup: } VIR_FORCE_CLOSE(statuswrite); } - if (pid_file) - unlink (pid_file); + if (pid_file_fd != -1) { + unlink(pid_file); + VIR_FORCE_CLOSE(pid_file_fd);
Swap these two lines. Not that flock (or even libvirtd) works on mingw, but mingw doesn't like unlink() on an open fd. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On Fri, Jul 08, 2011 at 04:33:28PM -0600, Eric Blake wrote:
On 07/07/2011 08:17 AM, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
If libvirtd crashes then the pidfile may not be cleaned up, making a later restart fail, even though the original process no longer exists.
Instead of simply using file creation as the test for successful pidfile acquisition, actually take out a lock on the pidfile.
To avoid races, after locking the pidfile, verify that the inode hasn't changed.
Also make sure the unprivileged libvirtd now acquires the pidfile, instead of relying on the UNIX socket to ensure mutual exclusion
Cool idea.
-static int daemonWritePidFile(const char *pidFile, const char *argv0) -{ +static int +daemonAcquirePidFile(const char *argv0, const char *pidFile) {
Why'd you hoist the { to the previous line? Our convention has been (slowly) leaning towards a function body starting on column 1.
int fd; - FILE *fh; char ebuf[1024]; + unsigned long long pid = (unsigned long long)getpid();
See my comments in an earlier thread about our existing assumptions that pid_t fits in int. In fact, I would be okay with:
verify(sizeof(pid_t) <= sizeof(int));
int pid = getpid(); char pidstr[INT_BUFSIZE_BOUND(pid)];
... snprintf(pidstr, sizeof(pidstr), "%u", pid);
+ char pidstr[INT_BUFSIZE_BOUND(pid)];
if (pidFile[0] == '\0') return 0;
- if ((fd = open(pidFile, O_WRONLY|O_CREAT|O_EXCL, 0644)) < 0) { - VIR_ERROR(_("Failed to open pid file '%s' : %s"), - pidFile, virStrerror(errno, ebuf, sizeof ebuf)); - return -1; - } + while (1) { + struct stat a, b; + if ((fd = open(pidFile, O_WRONLY|O_CREAT, 0644)) < 0) { + VIR_ERROR(_("%s: Failed to open pid file '%s' : %s"), + argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + return -1; + } + + if (fstat(fd, &b) < 0) { + VIR_ERROR(_("%s: Pid file '%s' disappeared: %s"),
Misleading error message. fstat can indeed fail (although such failures are rare), but not because a file disappeared - after all, you have an fd open to the file.
- if (fprintf(fh, "%lu\n", (unsigned long)getpid()) < 0) { - VIR_ERROR(_("%s: Failed to write to pid file '%s' : %s"), - argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); - VIR_FORCE_FCLOSE(fh); - return -1; - } + snprintf(pidstr, sizeof(pidstr), "%llu", pid);
- if (VIR_FCLOSE(fh) == EOF) { - VIR_ERROR(_("%s: Failed to close pid file '%s' : %s"), - argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); - return -1; + if (safewrite(fd, pidstr, strlen(pidstr)) < 0) {
Nice conversion from FILE* to write().
@@ -1436,6 +1453,12 @@ int main(int argc, char **argv) { umask(old_umask); }
+ /* Try to claim the pidfile, existing if we can't */
s/existing/exiting/
+ if ((pid_file_fd = daemonAcquirePidFile(argv[0], pid_file)) < 0) { + ret = VIR_DAEMON_ERR_PIDFILE; + goto cleanup; + } + if (!(srv = virNetServerNew(config->min_workers, config->max_workers, config->max_clients, @@ -1570,8 +1593,10 @@ cleanup: } VIR_FORCE_CLOSE(statuswrite); } - if (pid_file) - unlink (pid_file); + if (pid_file_fd != -1) { + unlink(pid_file); + VIR_FORCE_CLOSE(pid_file_fd);
Swap these two lines. Not that flock (or even libvirtd) works on mingw, but mingw doesn't like unlink() on an open fd.
If you swap those two lines you open up a (small) race condition in the locking scheme where you could unlink a pidfile that has now been claimed by someone else. By unlinking while we still have it open, we know we still hold the fcntl() lock. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
From: "Daniel P. Berrange" <berrange@redhat.com> This patch adds support for a systemd init service for libvirtd and libvirt-guests. The libvirtd.service is *not* written to use socket activation, since we want libvirtd to start on boot so it can do guest auto-start. The libvirt-guests.service is pretty lame, just exec'ing the original init script for now. Ideally we would factor out the functionality, into some shared tool. Instead of ./configure --with-init-script=redhat You can now do ./configure --with-init-script=systemd Or better still: ./configure --with-init-script=systemd+redhat * configure.ac: Add systemd, and systemd+redhat options to --with-init-script option * daemon/Makefile.am: Install systemd services * daemon/libvirtd.sysconf: Add note about unused env variable with systemd * daemon/libvirtd.service.in: libvirtd systemd service unit * libvirt.spec.in: Add scripts to installing systemd services and migrating from legacy init scripts * tools/Makefile.am: Install systemd services * tools/libvirt-guests.init.sh: Rename to tools/libvirt-guests.init.in * tools/libvirt-guests.service.in: systemd service unit --- configure.ac | 32 +++++-- daemon/.gitignore | 1 + daemon/Makefile.am | 62 ++++++++++---- daemon/libvirtd.service.in | 20 ++++ daemon/libvirtd.sysconf | 3 + libvirt.spec.in | 92 +++++++++++++++++++- tools/Makefile.am | 63 +++++++++++--- ...bvirt-guests.init.sh => libvirt-guests.init.in} | 0 tools/libvirt-guests.service.in | 13 +++ 9 files changed, 245 insertions(+), 41 deletions(-) create mode 100644 daemon/libvirtd.service.in rename tools/{libvirt-guests.init.sh => libvirt-guests.init.in} (100%) create mode 100644 tools/libvirt-guests.service.in diff --git a/configure.ac b/configure.ac index 13b4afb..d70e574 100644 --- a/configure.ac +++ b/configure.ac @@ -305,16 +305,30 @@ dnl init script flavor dnl AC_MSG_CHECKING([for init script flavor]) AC_ARG_WITH([init-script], - [AC_HELP_STRING([--with-init-script=@<:@redhat|auto|none@:>@], + [AC_HELP_STRING([--with-init-script=@<:@redhat|systemd|systemd+redhat|upstart|auto|none@:>@], [Style of init script to install @<:@default=auto@:>@])]) -if test "x$with_init_script" = "x" || test "x$with_init_script" = "xauto"; then - if test "$cross_compiling" = yes || test ! -f /etc/redhat-release; then - with_init_script=none - else - with_init_script=redhat - fi -fi -AM_CONDITIONAL([LIBVIRT_INIT_SCRIPT_RED_HAT], test x$with_init_script = xredhat) +init_redhat=no +init_systemd=no +case "$with_init_script" in + systemd+redhat) + init_redhat=yes + init_systemd=yes + ;; + systemd) + init_systemd=yes + ;; + redhat) + init_redhat=yes + ;; + *) + if test "$cross_compiling" != yes && test -f /etc/redhat-release; then + init_redhat=yes + with_init_script=redhat + fi + ;; +esac +AM_CONDITIONAL([LIBVIRT_INIT_SCRIPT_RED_HAT], test "$init_redhat" = "yes") +AM_CONDITIONAL([LIBVIRT_INIT_SCRIPT_SYSTEMD], test "$init_systemd" = "yes") AC_MSG_RESULT($with_init_script) dnl RHEL-5 has a peculiar version of Xen, which requires some special casing diff --git a/daemon/.gitignore b/daemon/.gitignore index ab3d093..2873143 100644 --- a/daemon/.gitignore +++ b/daemon/.gitignore @@ -7,6 +7,7 @@ Makefile.in libvirt_qemud libvirtd libvirtd.init +libvirtd.service libvirtd*.logrotate libvirtd.8 libvirtd.8.in diff --git a/daemon/Makefile.am b/daemon/Makefile.am index 8ed29b8..abb5e1f 100644 --- a/daemon/Makefile.am +++ b/daemon/Makefile.am @@ -180,13 +180,13 @@ probes.o: probes.d CLEANFILES += probes.h probes.o endif -install-data-local: install-init install-data-sasl install-data-polkit \ +install-data-local: install-init install-systemd install-data-sasl install-data-polkit \ install-logrotate mkdir -p $(DESTDIR)$(localstatedir)/log/libvirt mkdir -p $(DESTDIR)$(localstatedir)/run/libvirt mkdir -p $(DESTDIR)$(localstatedir)/lib/libvirt -uninstall-local:: uninstall-init uninstall-data-sasl uninstall-data-polkit +uninstall-local:: uninstall-init uninstall-systemd uninstall-data-sasl uninstall-data-polkit rmdir $(DESTDIR)$(localstatedir)/log/libvirt || : rmdir $(DESTDIR)$(localstatedir)/run/libvirt || : rmdir $(DESTDIR)$(localstatedir)/lib/libvirt || : @@ -244,20 +244,47 @@ install-logrotate: $(LOGROTATE_CONFS) $(INSTALL_DATA) libvirtd.lxc.logrotate $(DESTDIR)$(sysconfdir)/logrotate.d/libvirtd.lxc $(INSTALL_DATA) libvirtd.uml.logrotate $(DESTDIR)$(sysconfdir)/logrotate.d/libvirtd.uml +install-sysconfig: + mkdir -p $(DESTDIR)$(sysconfdir)/sysconfig + $(INSTALL_DATA) $(srcdir)/libvirtd.sysconf \ + $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd +uninstall-sysconfig: + rm -f $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd + if LIBVIRT_INIT_SCRIPT_RED_HAT -install-init: libvirtd.init + +BUILT_SOURCES += libvirtd.init + +install-init: install-sysconfig libvirtd.init mkdir -p $(DESTDIR)$(sysconfdir)/rc.d/init.d $(INSTALL_SCRIPT) libvirtd.init \ $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirtd - mkdir -p $(DESTDIR)$(sysconfdir)/sysconfig - $(INSTALL_DATA) $(srcdir)/libvirtd.sysconf \ - $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd +uninstall-init: uninstall-sysconfig + rm -f $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirtd +else +install-init: uninstall-init: - rm -f $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirtd \ - $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd +endif # LIBVIRT_INIT_SCRIPT_RED_HAT -BUILT_SOURCES += libvirtd.init + +EXTRA_DIST += libvirtd.service.in +if LIBVIRT_INIT_SCRIPT_SYSTEMD + +SYSTEMD_UNIT_DIR = /lib/systemd/system +BUILT_SOURCES += libvirtd.service + +install-systemd: install-sysconfig libvirtd.service + mkdir -p $(DESTDIR)$(SYSTEMD_UNIT_DIR) + $(INSTALL_SCRIPT) libvirtd.service \ + $(DESTDIR)$(SYSTEMD_UNIT_DIR)/libvirtd.service + +uninstall-systemd: uninstall-sysconfig + rm -f $(DESTDIR)$(SYSTEMD_UNIT_DIR)/libvirtd.service +else +install-systemd: +uninstall-systemd: +endif # LIBVIRT_INIT_SCRIPT_SYSTEMD libvirtd.init: libvirtd.init.in $(top_builddir)/config.status $(AM_V_GEN)sed \ @@ -268,18 +295,21 @@ libvirtd.init: libvirtd.init.in $(top_builddir)/config.status chmod a+x $@-t && \ mv $@-t $@ +libvirtd.service: libvirtd.service.in $(top_builddir)/config.status + $(AM_V_GEN)sed \ + -e s!\@localstatedir\@!@localstatedir@!g \ + -e s!\@sbindir\@!@sbindir@!g \ + -e s!\@sysconfdir\@!@sysconfdir@!g \ + < $< > $@-t && \ + chmod a+x $@-t && \ + mv $@-t $@ + + check-local: $(AM_V_GEN)if test -x '$(AUGPARSE)'; then \ '$(AUGPARSE)' -I $(srcdir) $(srcdir)/test_libvirtd.aug; \ fi -else - -install-init: -uninstall-init: -libvirtd.init: - -endif # LIBVIRT_INIT_SCRIPT_RED_HAT # This must be added last, since functions it provides/replaces # are used by nearly every other library. diff --git a/daemon/libvirtd.service.in b/daemon/libvirtd.service.in new file mode 100644 index 0000000..9661428 --- /dev/null +++ b/daemon/libvirtd.service.in @@ -0,0 +1,20 @@ +# NB we don't use socket activation. When libvirtd starts it will +# spawn any virtual machines registered for autostart. We want this +# to occur on every boot, regardless of whether any client connects +# to a socket. Thus socket activation doesn't have any benefit + +[Unit] +Description=Virtualization daemon +After=syslog.target +After=udev.target +After=avahi.target +After=dbus.target +Before=libvirt-guests.service + +[Service] +EnvironmentFile=-/etc/sysconfig/libvirtd +ExecStart=@sbindir@/libvirtd $LIBVIRTD_ARGS +ExecReload=/bin/kill -HUP $MAINPID + +[Install] +WantedBy=multi-user.target diff --git a/daemon/libvirtd.sysconf b/daemon/libvirtd.sysconf index b730c5e..ab273c8 100644 --- a/daemon/libvirtd.sysconf +++ b/daemon/libvirtd.sysconf @@ -1,4 +1,7 @@ # Override the default config file +# NOTE: This setting is no longer honoured if using +# systemd. Set '--config /etc/libvirt/libvirtd.conf' +# in LIBVIRTD_ARGS instead. #LIBVIRTD_CONFIG=/etc/libvirt/libvirtd.conf # Listen for TCP/IP connections diff --git a/libvirt.spec.in b/libvirt.spec.in index bf220f3..650f0bc 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -77,6 +77,7 @@ %define with_dtrace 0%{!?_without_dtrace:0} %define with_cgconfig 0%{!?_without_cgconfig:0} %define with_sanlock 0%{!?_without_sanlock:0} +%define with_systemd 0%{!?_without_systemd:0} # Non-server/HV driver defaults which are always enabled %define with_python 0%{!?_without_python:1} @@ -107,6 +108,10 @@ %define with_libxl 0 %endif +%if 0%{?fedora} >= 15 +%define with_systemd 1 +%endif + # RHEL-5 has restricted QEMU to x86_64 only and is too old for LXC %if 0%{?rhel} == 5 %ifnarch x86_64 @@ -313,7 +318,9 @@ Requires: libcgroup # All build-time requirements BuildRequires: python-devel - +%if %{with_systemd} +BuildRequires: systemd-units +%endif %if %{with_xen} BuildRequires: xen-devel %endif @@ -455,6 +462,13 @@ BuildRequires: nfs-utils # Fedora build root suckage BuildRequires: gawk +%if %{with_systemd} +Requires(post): systemd-units +Requires(post): systemd-sysv +Requires(preun): systemd-units +Requires(postun): systemd-units +%endif + %description Libvirt is a C toolkit to interact with the virtualization capabilities of recent versions of Linux (and other OSes). The main package includes @@ -671,6 +685,13 @@ of recent versions of Linux (and other OSes). %define with_packager --with-packager="%{who}, %{when}, %{where}" %define with_packager_version --with-packager-version="%{release}" +%if %{with_systemd} +# We use 'systemd+redhat', so if someone installs upstart or +# legacy init scripts, they can still start libvirtd, etc +%define init_scripts --with-init_script=systemd+redhat +%else +%define init_scripts --with-init_script=redhat +%endif %configure %{?_without_xen} \ %{?_without_qemu} \ @@ -711,7 +732,7 @@ of recent versions of Linux (and other OSes). %{with_packager_version} \ --with-qemu-user=%{qemu_user} \ --with-qemu-group=%{qemu_group} \ - --with-init-script=redhat \ + %{init_scripts} \ --with-remote-pid-file=%{_localstatedir}/run/libvirtd.pid make %{?_smp_mflags} gzip -9 ChangeLog @@ -719,7 +740,7 @@ gzip -9 ChangeLog %install rm -fr %{buildroot} -%makeinstall +%makeinstall SYSTEMD_UNIT_DIR=%{?buildroot:%{buildroot}}%{_unitdir} for i in domain-events/events-c dominfo domsuspend hellolibvirt openauth python xml/nwfilter systemtap do (cd examples/$i ; make clean ; rm -rf .deps .libs Makefile Makefile.in) @@ -868,6 +889,13 @@ do done %endif +%if %{with_systemd} +if [ $1 -eq 1 ] ; then + # Initial installation + /bin/systemctl enable libvirtd.service >/dev/null 2>&1 || : + /bin/systemctl enable cgconfig.service >/dev/null 2>&1 || : +fi +%else %if %{with_cgconfig} if [ "$1" -eq "1" ]; then /sbin/chkconfig cgconfig on @@ -879,25 +907,64 @@ if [ "$1" -ge "1" ]; then /sbin/service libvirtd condrestart > /dev/null 2>&1 fi %endif +%endif %preun %if %{with_libvirtd} +%if %{with_systemd} +if [ $1 -eq 0 ] ; then + # Package removal, not upgrade + /bin/systemctl --no-reload disable libvirtd.service > /dev/null 2>&1 || : + /bin/systemctl stop libvirtd.service > /dev/null 2>&1 || : +fi +%else if [ $1 = 0 ]; then /sbin/service libvirtd stop 1>/dev/null 2>&1 /sbin/chkconfig --del libvirtd fi %endif +%endif + +%postun +%if %{with_libvirtd} +%if %{with_systemd} +/bin/systemctl daemon-reload >/dev/null 2>&1 || : +if [ $1 -ge 1 ] ; then + # Package upgrade, not uninstall + /bin/systemctl try-restart libvirtd.service >/dev/null 2>&1 || : +fi +%endif +%endif + +%if %{with_libvirtd} +%if %{with_systemd} +%triggerun -- libvirt < 0.9.4 +%{_bindir}/systemd-sysv-convert --save libvirtd >/dev/null 2>&1 ||: + +# If the package is allowed to autostart: +/bin/systemctl --no-reload enable libvirtd.service >/dev/null 2>&1 ||: + +# Run these because the SysV package being removed won't do them +/sbin/chkconfig --del libvirtd >/dev/null 2>&1 || : +/bin/systemctl try-restart libvirtd.service >/dev/null 2>&1 || : +%endif +%endif %preun client +%if %{with_systemd} +%else if [ $1 = 0 ]; then /sbin/chkconfig --del libvirt-guests rm -f /var/lib/libvirt/libvirt-guests fi +%endif %post client /sbin/ldconfig +%if %{with_systemd} +%else /sbin/chkconfig --add libvirt-guests if [ $1 -ge 1 ]; then level=$(/sbin/runlevel | /bin/cut -d ' ' -f 2) @@ -907,9 +974,22 @@ if [ $1 -ge 1 ]; then /sbin/service libvirt-guests start > /dev/null 2>&1 || true fi fi +%endif %postun client -p /sbin/ldconfig +%if %{with_systemd} +%triggerun client -- libvirt < 0.9.4 +%{_bindir}/systemd-sysv-convert --save libvirt-guests >/dev/null 2>&1 ||: + +# If the package is allowed to autostart: +/bin/systemctl --no-reload enable libvirt-guests.service >/dev/null 2>&1 ||: + +# Run these because the SysV package being removed won't do them +/sbin/chkconfig --del libvirt-guests >/dev/null 2>&1 || : +/bin/systemctl try-restart libvirt-guests.service >/dev/null 2>&1 || : +%endif + %if %{with_libvirtd} %files %defattr(-, root, root) @@ -927,6 +1007,9 @@ fi %{_sysconfdir}/libvirt/nwfilter/*.xml %{_sysconfdir}/rc.d/init.d/libvirtd +%if %{with_systemd} +%{_unitdir}/libvirtd.service +%endif %doc daemon/libvirtd.upstart %config(noreplace) %{_sysconfdir}/sysconfig/libvirtd %config(noreplace) %{_sysconfdir}/libvirt/libvirtd.conf @@ -1070,6 +1153,9 @@ fi %{_datadir}/libvirt/cpu_map.xml %{_sysconfdir}/rc.d/init.d/libvirt-guests +%if %{with_systemd} +%{_unitdir}/libvirt-guests.service +%endif %config(noreplace) %{_sysconfdir}/sysconfig/libvirt-guests %dir %attr(0755, root, root) %{_localstatedir}/lib/libvirt/ diff --git a/tools/Makefile.am b/tools/Makefile.am index 62c275e..7537f81 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -15,7 +15,6 @@ EXTRA_DIST = \ virt-sanlock-cleanup.in \ virt-sanlock-cleanup.8 \ virsh.pod \ - libvirt-guests.init.sh \ libvirt-guests.sysconf bin_SCRIPTS = virt-xml-validate virt-pki-validate @@ -145,22 +144,33 @@ install-data-local: install-init uninstall-local: uninstall-init +install-sysconfig: + mkdir -p $(DESTDIR)$(sysconfdir)/sysconfig + $(INSTALL_DATA) $(srcdir)/libvirt-guests.sysconf \ + $(DESTDIR)$(sysconfdir)/sysconfig/libvirt-guests + +uninstall-sysconfig: + rm -f $(DESTDIR)$(sysconfdir)/sysconfig/libvirt-guests + +EXTRA_DIST += libvirt-guests.init.in + if LIBVIRT_INIT_SCRIPT_RED_HAT -install-init: libvirt-guests.init +install-init: libvirt-guests.init install-sysconfig mkdir -p $(DESTDIR)$(sysconfdir)/rc.d/init.d $(INSTALL_SCRIPT) libvirt-guests.init \ $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirt-guests - mkdir -p $(DESTDIR)$(sysconfdir)/sysconfig - $(INSTALL_DATA) $(srcdir)/libvirt-guests.sysconf \ - $(DESTDIR)$(sysconfdir)/sysconfig/libvirt-guests -uninstall-init: - rm -f $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirt-guests \ - $(DESTDIR)$(sysconfdir)/sysconfig/libvirt-guests +uninstall-init: install-sysconfig + rm -f $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirt-guests BUILT_SOURCES += libvirt-guests.init -libvirt-guests.init: libvirt-guests.init.sh $(top_builddir)/config.status +else +install-init: +uninstall-init: +endif # LIBVIRT_INIT_SCRIPT_RED_HAT + +libvirt-guests.init: libvirt-guests.init.in $(top_builddir)/config.status $(AM_V_GEN)sed \ -e 's!\@PACKAGE\@!$(PACKAGE)!g' \ -e 's!\@bindir\@!$(bindir)!g' \ @@ -171,11 +181,38 @@ libvirt-guests.init: libvirt-guests.init.sh $(top_builddir)/config.status < $< > $@-t && \ chmod a+x $@-t && \ mv $@-t $@ + + + +EXTRA_DIST += libvirt-guests.service.in + +if LIBVIRT_INIT_SCRIPT_SYSTEMD +install-systemd: libvirt-guests.service install-sysconfig + mkdir -p $(DESTDIR)$(sysconfdir)/rc.d/systemd.d + $(INSTALL_SCRIPT) libvirt-guests.service \ + $(DESTDIR)$(sysconfdir)/rc.d/systemd.d/libvirt-guests + +uninstall-systemd: install-sysconfig + rm -f $(DESTDIR)$(sysconfdir)/rc.d/systemd.d/libvirt-guests + +BUILT_SOURCES += libvirt-guests.service + else -install-init: -uninstall-init: -libvirt-guests.init: -endif # LIBVIRT_INIT_SCRIPT_RED_HAT +install-systemd: +uninstall-systemd: +endif # LIBVIRT_INIT_SCRIPT_SYSTEMD + +libvirt-guests.service: libvirt-guests.service.in $(top_builddir)/config.status + $(AM_V_GEN)sed \ + -e 's!\@PACKAGE\@!$(PACKAGE)!g' \ + -e 's!\@bindir\@!$(bindir)!g' \ + -e 's!\@localedir\@!$(localedir)!g' \ + -e 's!\@localstatedir\@!$(localstatedir)!g' \ + -e 's!\@sbindir\@!$(sbindir)!g' \ + -e 's!\@sysconfdir\@!$(sysconfdir)!g' \ + < $< > $@-t && \ + chmod a+x $@-t && \ + mv $@-t $@ CLEANFILES = $(bin_SCRIPTS) diff --git a/tools/libvirt-guests.init.sh b/tools/libvirt-guests.init.in similarity index 100% rename from tools/libvirt-guests.init.sh rename to tools/libvirt-guests.init.in diff --git a/tools/libvirt-guests.service.in b/tools/libvirt-guests.service.in new file mode 100644 index 0000000..123aecc --- /dev/null +++ b/tools/libvirt-guests.service.in @@ -0,0 +1,13 @@ +[Unit] +Description=Suspend Active Libvirt Guests +After=syslog.target network.target + +[Service] +EnvironmentFile=-/etc/sysconfig/libvirt-guests +# Hack just call traditional service until we factor +# out the code +ExecStart=/etc/init.d/libvirt-guests start +ExecStop=/etc/init.d/libvirt-guests stop + +[Install] +WantedBy=multi-user.target \ No newline at end of file -- 1.7.6
Hi Daniel, On Thu, Jul 07, 2011 at 03:17:24PM +0100, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
This patch adds support for a systemd init service for libvirtd and libvirt-guests. The libvirtd.service is *not* written to use socket activation, since we want libvirtd to start on boot so it can do guest auto-start.
It seems this nevert got pushed. I't be great to have systemd support upstream so we don't reinvent the wheel on every distro that uses systemd. Cheers, -- Guido
The libvirt-guests.service is pretty lame, just exec'ing the original init script for now. Ideally we would factor out the functionality, into some shared tool.
Instead of
./configure --with-init-script=redhat
You can now do
./configure --with-init-script=systemd
Or better still:
./configure --with-init-script=systemd+redhat
* configure.ac: Add systemd, and systemd+redhat options to --with-init-script option * daemon/Makefile.am: Install systemd services * daemon/libvirtd.sysconf: Add note about unused env variable with systemd * daemon/libvirtd.service.in: libvirtd systemd service unit * libvirt.spec.in: Add scripts to installing systemd services and migrating from legacy init scripts * tools/Makefile.am: Install systemd services * tools/libvirt-guests.init.sh: Rename to tools/libvirt-guests.init.in * tools/libvirt-guests.service.in: systemd service unit --- configure.ac | 32 +++++-- daemon/.gitignore | 1 + daemon/Makefile.am | 62 ++++++++++---- daemon/libvirtd.service.in | 20 ++++ daemon/libvirtd.sysconf | 3 + libvirt.spec.in | 92 +++++++++++++++++++- tools/Makefile.am | 63 +++++++++++--- ...bvirt-guests.init.sh => libvirt-guests.init.in} | 0 tools/libvirt-guests.service.in | 13 +++ 9 files changed, 245 insertions(+), 41 deletions(-) create mode 100644 daemon/libvirtd.service.in rename tools/{libvirt-guests.init.sh => libvirt-guests.init.in} (100%) create mode 100644 tools/libvirt-guests.service.in
diff --git a/configure.ac b/configure.ac index 13b4afb..d70e574 100644 --- a/configure.ac +++ b/configure.ac @@ -305,16 +305,30 @@ dnl init script flavor dnl AC_MSG_CHECKING([for init script flavor]) AC_ARG_WITH([init-script], - [AC_HELP_STRING([--with-init-script=@<:@redhat|auto|none@:>@], + [AC_HELP_STRING([--with-init-script=@<:@redhat|systemd|systemd+redhat|upstart|auto|none@:>@], [Style of init script to install @<:@default=auto@:>@])]) -if test "x$with_init_script" = "x" || test "x$with_init_script" = "xauto"; then - if test "$cross_compiling" = yes || test ! -f /etc/redhat-release; then - with_init_script=none - else - with_init_script=redhat - fi -fi -AM_CONDITIONAL([LIBVIRT_INIT_SCRIPT_RED_HAT], test x$with_init_script = xredhat) +init_redhat=no +init_systemd=no +case "$with_init_script" in + systemd+redhat) + init_redhat=yes + init_systemd=yes + ;; + systemd) + init_systemd=yes + ;; + redhat) + init_redhat=yes + ;; + *) + if test "$cross_compiling" != yes && test -f /etc/redhat-release; then + init_redhat=yes + with_init_script=redhat + fi + ;; +esac +AM_CONDITIONAL([LIBVIRT_INIT_SCRIPT_RED_HAT], test "$init_redhat" = "yes") +AM_CONDITIONAL([LIBVIRT_INIT_SCRIPT_SYSTEMD], test "$init_systemd" = "yes") AC_MSG_RESULT($with_init_script)
dnl RHEL-5 has a peculiar version of Xen, which requires some special casing diff --git a/daemon/.gitignore b/daemon/.gitignore index ab3d093..2873143 100644 --- a/daemon/.gitignore +++ b/daemon/.gitignore @@ -7,6 +7,7 @@ Makefile.in libvirt_qemud libvirtd libvirtd.init +libvirtd.service libvirtd*.logrotate libvirtd.8 libvirtd.8.in diff --git a/daemon/Makefile.am b/daemon/Makefile.am index 8ed29b8..abb5e1f 100644 --- a/daemon/Makefile.am +++ b/daemon/Makefile.am @@ -180,13 +180,13 @@ probes.o: probes.d CLEANFILES += probes.h probes.o endif
-install-data-local: install-init install-data-sasl install-data-polkit \ +install-data-local: install-init install-systemd install-data-sasl install-data-polkit \ install-logrotate mkdir -p $(DESTDIR)$(localstatedir)/log/libvirt mkdir -p $(DESTDIR)$(localstatedir)/run/libvirt mkdir -p $(DESTDIR)$(localstatedir)/lib/libvirt
-uninstall-local:: uninstall-init uninstall-data-sasl uninstall-data-polkit +uninstall-local:: uninstall-init uninstall-systemd uninstall-data-sasl uninstall-data-polkit rmdir $(DESTDIR)$(localstatedir)/log/libvirt || : rmdir $(DESTDIR)$(localstatedir)/run/libvirt || : rmdir $(DESTDIR)$(localstatedir)/lib/libvirt || : @@ -244,20 +244,47 @@ install-logrotate: $(LOGROTATE_CONFS) $(INSTALL_DATA) libvirtd.lxc.logrotate $(DESTDIR)$(sysconfdir)/logrotate.d/libvirtd.lxc $(INSTALL_DATA) libvirtd.uml.logrotate $(DESTDIR)$(sysconfdir)/logrotate.d/libvirtd.uml
+install-sysconfig: + mkdir -p $(DESTDIR)$(sysconfdir)/sysconfig + $(INSTALL_DATA) $(srcdir)/libvirtd.sysconf \ + $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd +uninstall-sysconfig: + rm -f $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd + if LIBVIRT_INIT_SCRIPT_RED_HAT -install-init: libvirtd.init + +BUILT_SOURCES += libvirtd.init + +install-init: install-sysconfig libvirtd.init mkdir -p $(DESTDIR)$(sysconfdir)/rc.d/init.d $(INSTALL_SCRIPT) libvirtd.init \ $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirtd - mkdir -p $(DESTDIR)$(sysconfdir)/sysconfig - $(INSTALL_DATA) $(srcdir)/libvirtd.sysconf \ - $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd
+uninstall-init: uninstall-sysconfig + rm -f $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirtd +else +install-init: uninstall-init: - rm -f $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirtd \ - $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd +endif # LIBVIRT_INIT_SCRIPT_RED_HAT
-BUILT_SOURCES += libvirtd.init + +EXTRA_DIST += libvirtd.service.in +if LIBVIRT_INIT_SCRIPT_SYSTEMD + +SYSTEMD_UNIT_DIR = /lib/systemd/system +BUILT_SOURCES += libvirtd.service + +install-systemd: install-sysconfig libvirtd.service + mkdir -p $(DESTDIR)$(SYSTEMD_UNIT_DIR) + $(INSTALL_SCRIPT) libvirtd.service \ + $(DESTDIR)$(SYSTEMD_UNIT_DIR)/libvirtd.service + +uninstall-systemd: uninstall-sysconfig + rm -f $(DESTDIR)$(SYSTEMD_UNIT_DIR)/libvirtd.service +else +install-systemd: +uninstall-systemd: +endif # LIBVIRT_INIT_SCRIPT_SYSTEMD
libvirtd.init: libvirtd.init.in $(top_builddir)/config.status $(AM_V_GEN)sed \ @@ -268,18 +295,21 @@ libvirtd.init: libvirtd.init.in $(top_builddir)/config.status chmod a+x $@-t && \ mv $@-t $@
+libvirtd.service: libvirtd.service.in $(top_builddir)/config.status + $(AM_V_GEN)sed \ + -e s!\@localstatedir\@!@localstatedir@!g \ + -e s!\@sbindir\@!@sbindir@!g \ + -e s!\@sysconfdir\@!@sysconfdir@!g \ + < $< > $@-t && \ + chmod a+x $@-t && \ + mv $@-t $@ + + check-local: $(AM_V_GEN)if test -x '$(AUGPARSE)'; then \ '$(AUGPARSE)' -I $(srcdir) $(srcdir)/test_libvirtd.aug; \ fi
-else - -install-init: -uninstall-init: -libvirtd.init: - -endif # LIBVIRT_INIT_SCRIPT_RED_HAT
# This must be added last, since functions it provides/replaces # are used by nearly every other library. diff --git a/daemon/libvirtd.service.in b/daemon/libvirtd.service.in new file mode 100644 index 0000000..9661428 --- /dev/null +++ b/daemon/libvirtd.service.in @@ -0,0 +1,20 @@ +# NB we don't use socket activation. When libvirtd starts it will +# spawn any virtual machines registered for autostart. We want this +# to occur on every boot, regardless of whether any client connects +# to a socket. Thus socket activation doesn't have any benefit + +[Unit] +Description=Virtualization daemon +After=syslog.target +After=udev.target +After=avahi.target +After=dbus.target +Before=libvirt-guests.service + +[Service] +EnvironmentFile=-/etc/sysconfig/libvirtd +ExecStart=@sbindir@/libvirtd $LIBVIRTD_ARGS +ExecReload=/bin/kill -HUP $MAINPID + +[Install] +WantedBy=multi-user.target diff --git a/daemon/libvirtd.sysconf b/daemon/libvirtd.sysconf index b730c5e..ab273c8 100644 --- a/daemon/libvirtd.sysconf +++ b/daemon/libvirtd.sysconf @@ -1,4 +1,7 @@ # Override the default config file +# NOTE: This setting is no longer honoured if using +# systemd. Set '--config /etc/libvirt/libvirtd.conf' +# in LIBVIRTD_ARGS instead. #LIBVIRTD_CONFIG=/etc/libvirt/libvirtd.conf
# Listen for TCP/IP connections diff --git a/libvirt.spec.in b/libvirt.spec.in index bf220f3..650f0bc 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -77,6 +77,7 @@ %define with_dtrace 0%{!?_without_dtrace:0} %define with_cgconfig 0%{!?_without_cgconfig:0} %define with_sanlock 0%{!?_without_sanlock:0} +%define with_systemd 0%{!?_without_systemd:0}
# Non-server/HV driver defaults which are always enabled %define with_python 0%{!?_without_python:1} @@ -107,6 +108,10 @@ %define with_libxl 0 %endif
+%if 0%{?fedora} >= 15 +%define with_systemd 1 +%endif + # RHEL-5 has restricted QEMU to x86_64 only and is too old for LXC %if 0%{?rhel} == 5 %ifnarch x86_64 @@ -313,7 +318,9 @@ Requires: libcgroup
# All build-time requirements BuildRequires: python-devel - +%if %{with_systemd} +BuildRequires: systemd-units +%endif %if %{with_xen} BuildRequires: xen-devel %endif @@ -455,6 +462,13 @@ BuildRequires: nfs-utils # Fedora build root suckage BuildRequires: gawk
+%if %{with_systemd} +Requires(post): systemd-units +Requires(post): systemd-sysv +Requires(preun): systemd-units +Requires(postun): systemd-units +%endif + %description Libvirt is a C toolkit to interact with the virtualization capabilities of recent versions of Linux (and other OSes). The main package includes @@ -671,6 +685,13 @@ of recent versions of Linux (and other OSes). %define with_packager --with-packager="%{who}, %{when}, %{where}" %define with_packager_version --with-packager-version="%{release}"
+%if %{with_systemd} +# We use 'systemd+redhat', so if someone installs upstart or +# legacy init scripts, they can still start libvirtd, etc +%define init_scripts --with-init_script=systemd+redhat +%else +%define init_scripts --with-init_script=redhat +%endif
%configure %{?_without_xen} \ %{?_without_qemu} \ @@ -711,7 +732,7 @@ of recent versions of Linux (and other OSes). %{with_packager_version} \ --with-qemu-user=%{qemu_user} \ --with-qemu-group=%{qemu_group} \ - --with-init-script=redhat \ + %{init_scripts} \ --with-remote-pid-file=%{_localstatedir}/run/libvirtd.pid make %{?_smp_mflags} gzip -9 ChangeLog @@ -719,7 +740,7 @@ gzip -9 ChangeLog %install rm -fr %{buildroot}
-%makeinstall +%makeinstall SYSTEMD_UNIT_DIR=%{?buildroot:%{buildroot}}%{_unitdir} for i in domain-events/events-c dominfo domsuspend hellolibvirt openauth python xml/nwfilter systemtap do (cd examples/$i ; make clean ; rm -rf .deps .libs Makefile Makefile.in) @@ -868,6 +889,13 @@ do done %endif
+%if %{with_systemd} +if [ $1 -eq 1 ] ; then + # Initial installation + /bin/systemctl enable libvirtd.service >/dev/null 2>&1 || : + /bin/systemctl enable cgconfig.service >/dev/null 2>&1 || : +fi +%else %if %{with_cgconfig} if [ "$1" -eq "1" ]; then /sbin/chkconfig cgconfig on @@ -879,25 +907,64 @@ if [ "$1" -ge "1" ]; then /sbin/service libvirtd condrestart > /dev/null 2>&1 fi %endif +%endif
%preun %if %{with_libvirtd} +%if %{with_systemd} +if [ $1 -eq 0 ] ; then + # Package removal, not upgrade + /bin/systemctl --no-reload disable libvirtd.service > /dev/null 2>&1 || : + /bin/systemctl stop libvirtd.service > /dev/null 2>&1 || : +fi +%else if [ $1 = 0 ]; then /sbin/service libvirtd stop 1>/dev/null 2>&1 /sbin/chkconfig --del libvirtd fi %endif +%endif + +%postun +%if %{with_libvirtd} +%if %{with_systemd} +/bin/systemctl daemon-reload >/dev/null 2>&1 || : +if [ $1 -ge 1 ] ; then + # Package upgrade, not uninstall + /bin/systemctl try-restart libvirtd.service >/dev/null 2>&1 || : +fi +%endif +%endif + +%if %{with_libvirtd} +%if %{with_systemd} +%triggerun -- libvirt < 0.9.4 +%{_bindir}/systemd-sysv-convert --save libvirtd >/dev/null 2>&1 ||: + +# If the package is allowed to autostart: +/bin/systemctl --no-reload enable libvirtd.service >/dev/null 2>&1 ||: + +# Run these because the SysV package being removed won't do them +/sbin/chkconfig --del libvirtd >/dev/null 2>&1 || : +/bin/systemctl try-restart libvirtd.service >/dev/null 2>&1 || : +%endif +%endif
%preun client
+%if %{with_systemd} +%else if [ $1 = 0 ]; then /sbin/chkconfig --del libvirt-guests rm -f /var/lib/libvirt/libvirt-guests fi +%endif
%post client
/sbin/ldconfig +%if %{with_systemd} +%else /sbin/chkconfig --add libvirt-guests if [ $1 -ge 1 ]; then level=$(/sbin/runlevel | /bin/cut -d ' ' -f 2) @@ -907,9 +974,22 @@ if [ $1 -ge 1 ]; then /sbin/service libvirt-guests start > /dev/null 2>&1 || true fi fi +%endif
%postun client -p /sbin/ldconfig
+%if %{with_systemd} +%triggerun client -- libvirt < 0.9.4 +%{_bindir}/systemd-sysv-convert --save libvirt-guests >/dev/null 2>&1 ||: + +# If the package is allowed to autostart: +/bin/systemctl --no-reload enable libvirt-guests.service >/dev/null 2>&1 ||: + +# Run these because the SysV package being removed won't do them +/sbin/chkconfig --del libvirt-guests >/dev/null 2>&1 || : +/bin/systemctl try-restart libvirt-guests.service >/dev/null 2>&1 || : +%endif + %if %{with_libvirtd} %files %defattr(-, root, root) @@ -927,6 +1007,9 @@ fi %{_sysconfdir}/libvirt/nwfilter/*.xml
%{_sysconfdir}/rc.d/init.d/libvirtd +%if %{with_systemd} +%{_unitdir}/libvirtd.service +%endif %doc daemon/libvirtd.upstart %config(noreplace) %{_sysconfdir}/sysconfig/libvirtd %config(noreplace) %{_sysconfdir}/libvirt/libvirtd.conf @@ -1070,6 +1153,9 @@ fi %{_datadir}/libvirt/cpu_map.xml
%{_sysconfdir}/rc.d/init.d/libvirt-guests +%if %{with_systemd} +%{_unitdir}/libvirt-guests.service +%endif %config(noreplace) %{_sysconfdir}/sysconfig/libvirt-guests %dir %attr(0755, root, root) %{_localstatedir}/lib/libvirt/
diff --git a/tools/Makefile.am b/tools/Makefile.am index 62c275e..7537f81 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -15,7 +15,6 @@ EXTRA_DIST = \ virt-sanlock-cleanup.in \ virt-sanlock-cleanup.8 \ virsh.pod \ - libvirt-guests.init.sh \ libvirt-guests.sysconf
bin_SCRIPTS = virt-xml-validate virt-pki-validate @@ -145,22 +144,33 @@ install-data-local: install-init
uninstall-local: uninstall-init
+install-sysconfig: + mkdir -p $(DESTDIR)$(sysconfdir)/sysconfig + $(INSTALL_DATA) $(srcdir)/libvirt-guests.sysconf \ + $(DESTDIR)$(sysconfdir)/sysconfig/libvirt-guests + +uninstall-sysconfig: + rm -f $(DESTDIR)$(sysconfdir)/sysconfig/libvirt-guests + +EXTRA_DIST += libvirt-guests.init.in + if LIBVIRT_INIT_SCRIPT_RED_HAT -install-init: libvirt-guests.init +install-init: libvirt-guests.init install-sysconfig mkdir -p $(DESTDIR)$(sysconfdir)/rc.d/init.d $(INSTALL_SCRIPT) libvirt-guests.init \ $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirt-guests - mkdir -p $(DESTDIR)$(sysconfdir)/sysconfig - $(INSTALL_DATA) $(srcdir)/libvirt-guests.sysconf \ - $(DESTDIR)$(sysconfdir)/sysconfig/libvirt-guests
-uninstall-init: - rm -f $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirt-guests \ - $(DESTDIR)$(sysconfdir)/sysconfig/libvirt-guests +uninstall-init: install-sysconfig + rm -f $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirt-guests
BUILT_SOURCES += libvirt-guests.init
-libvirt-guests.init: libvirt-guests.init.sh $(top_builddir)/config.status +else +install-init: +uninstall-init: +endif # LIBVIRT_INIT_SCRIPT_RED_HAT + +libvirt-guests.init: libvirt-guests.init.in $(top_builddir)/config.status $(AM_V_GEN)sed \ -e 's!\@PACKAGE\@!$(PACKAGE)!g' \ -e 's!\@bindir\@!$(bindir)!g' \ @@ -171,11 +181,38 @@ libvirt-guests.init: libvirt-guests.init.sh $(top_builddir)/config.status < $< > $@-t && \ chmod a+x $@-t && \ mv $@-t $@ + + + +EXTRA_DIST += libvirt-guests.service.in + +if LIBVIRT_INIT_SCRIPT_SYSTEMD +install-systemd: libvirt-guests.service install-sysconfig + mkdir -p $(DESTDIR)$(sysconfdir)/rc.d/systemd.d + $(INSTALL_SCRIPT) libvirt-guests.service \ + $(DESTDIR)$(sysconfdir)/rc.d/systemd.d/libvirt-guests + +uninstall-systemd: install-sysconfig + rm -f $(DESTDIR)$(sysconfdir)/rc.d/systemd.d/libvirt-guests + +BUILT_SOURCES += libvirt-guests.service + else -install-init: -uninstall-init: -libvirt-guests.init: -endif # LIBVIRT_INIT_SCRIPT_RED_HAT +install-systemd: +uninstall-systemd: +endif # LIBVIRT_INIT_SCRIPT_SYSTEMD + +libvirt-guests.service: libvirt-guests.service.in $(top_builddir)/config.status + $(AM_V_GEN)sed \ + -e 's!\@PACKAGE\@!$(PACKAGE)!g' \ + -e 's!\@bindir\@!$(bindir)!g' \ + -e 's!\@localedir\@!$(localedir)!g' \ + -e 's!\@localstatedir\@!$(localstatedir)!g' \ + -e 's!\@sbindir\@!$(sbindir)!g' \ + -e 's!\@sysconfdir\@!$(sysconfdir)!g' \ + < $< > $@-t && \ + chmod a+x $@-t && \ + mv $@-t $@
CLEANFILES = $(bin_SCRIPTS) diff --git a/tools/libvirt-guests.init.sh b/tools/libvirt-guests.init.in similarity index 100% rename from tools/libvirt-guests.init.sh rename to tools/libvirt-guests.init.in diff --git a/tools/libvirt-guests.service.in b/tools/libvirt-guests.service.in new file mode 100644 index 0000000..123aecc --- /dev/null +++ b/tools/libvirt-guests.service.in @@ -0,0 +1,13 @@ +[Unit] +Description=Suspend Active Libvirt Guests +After=syslog.target network.target + +[Service] +EnvironmentFile=-/etc/sysconfig/libvirt-guests +# Hack just call traditional service until we factor +# out the code +ExecStart=/etc/init.d/libvirt-guests start +ExecStop=/etc/init.d/libvirt-guests stop + +[Install] +WantedBy=multi-user.target \ No newline at end of file -- 1.7.6
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Nov 09, 2011 at 09:27:20AM +0100, Guido Günther wrote:
Hi Daniel,
On Thu, Jul 07, 2011 at 03:17:24PM +0100, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
This patch adds support for a systemd init service for libvirtd and libvirt-guests. The libvirtd.service is *not* written to use socket activation, since we want libvirtd to start on boot so it can do guest auto-start.
It seems this nevert got pushed. I't be great to have systemd support upstream so we don't reinvent the wheel on every distro that uses systemd.
It never got ACKed either. I've pulled it out of the big series and re-posted. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
From: "Daniel P. Berrange" <berrange@redhat.com> To allow a virLockManagerPtr to be created directly from a driver table struct, replace the virLockManagerPluginPtr parameter with a virLockDriverPtr parameter. * src/locking/domain_lock.c, src/locking/lock_manager.c, src/locking/lock_manager.h: Replace plugin param with a driver in virLockManagerNew --- src/locking/domain_lock.c | 2 +- src/locking/lock_manager.c | 25 ++++++++++++++++--------- src/locking/lock_manager.h | 3 ++- 3 files changed, 19 insertions(+), 11 deletions(-) diff --git a/src/locking/domain_lock.c b/src/locking/domain_lock.c index de1937c..0ce74ff 100644 --- a/src/locking/domain_lock.c +++ b/src/locking/domain_lock.c @@ -124,7 +124,7 @@ static virLockManagerPtr virDomainLockManagerNew(virLockManagerPluginPtr plugin, memcpy(params[0].value.uuid, dom->def->uuid, VIR_UUID_BUFLEN); - if (!(lock = virLockManagerNew(plugin, + if (!(lock = virLockManagerNew(virLockManagerPluginGetDriver(plugin), VIR_LOCK_MANAGER_OBJECT_TYPE_DOMAIN, ARRAY_CARDINALITY(params), params, diff --git a/src/locking/lock_manager.c b/src/locking/lock_manager.c index 4ee7f44..f257707 100644 --- a/src/locking/lock_manager.c +++ b/src/locking/lock_manager.c @@ -43,8 +43,8 @@ virReportErrorHelper(VIR_FROM_THIS, code, __FILE__, \ __FUNCTION__, __LINE__, __VA_ARGS__) -#define CHECK_PLUGIN(field, errret) \ - if (!plugin->driver->field) { \ +#define CHECK_DRIVER(field, errret) \ + if (!driver->field) { \ virLockError(VIR_ERR_INTERNAL_ERROR, \ _("Missing '%s' field in lock manager driver"), \ #field); \ @@ -268,9 +268,16 @@ bool virLockManagerPluginUsesState(virLockManagerPluginPtr plugin) } +virLockDriverPtr virLockManagerPluginGetDriver(virLockManagerPluginPtr plugin) +{ + VIR_DEBUG("plugin=%p", plugin); + + return plugin->driver; +} + /** * virLockManagerNew: - * @plugin: the plugin implementation to use + * @driver: the lock manager implementation to use * @type: the type of process to be supervised * @flags: optional flags, currently unused * @@ -279,27 +286,27 @@ bool virLockManagerPluginUsesState(virLockManagerPluginPtr plugin) * * Returns a new lock manager context */ -virLockManagerPtr virLockManagerNew(virLockManagerPluginPtr plugin, +virLockManagerPtr virLockManagerNew(virLockDriverPtr driver, unsigned int type, size_t nparams, virLockManagerParamPtr params, unsigned int flags) { virLockManagerPtr lock; - VIR_DEBUG("plugin=%p type=%u nparams=%zu params=%p flags=%u", - plugin, type, nparams, params, flags); + VIR_DEBUG("driver=%p type=%u nparams=%zu params=%p flags=%u", + driver, type, nparams, params, flags); virLockManagerLogParams(nparams, params); - CHECK_PLUGIN(drvNew, NULL); + CHECK_DRIVER(drvNew, NULL); if (VIR_ALLOC(lock) < 0) { virReportOOMError(); return NULL; } - lock->driver = plugin->driver; + lock->driver = driver; - if (plugin->driver->drvNew(lock, type, nparams, params, flags) < 0) { + if (driver->drvNew(lock, type, nparams, params, flags) < 0) { VIR_FREE(lock); return NULL; } diff --git a/src/locking/lock_manager.h b/src/locking/lock_manager.h index 0fb3bb7..5afb0d0 100644 --- a/src/locking/lock_manager.h +++ b/src/locking/lock_manager.h @@ -37,8 +37,9 @@ void virLockManagerPluginUnref(virLockManagerPluginPtr plugin); const char *virLockManagerPluginGetName(virLockManagerPluginPtr plugin); bool virLockManagerPluginUsesState(virLockManagerPluginPtr plugin); +virLockDriverPtr virLockManagerPluginGetDriver(virLockManagerPluginPtr plugin); -virLockManagerPtr virLockManagerNew(virLockManagerPluginPtr plugin, +virLockManagerPtr virLockManagerNew(virLockDriverPtr driver, unsigned int type, size_t nparams, virLockManagerParamPtr params, -- 1.7.6
From: "Daniel P. Berrange" <berrange@redhat.com> The virtlockd daemon will be responsible for managing locks on virtual machines. Communication will be via the standard RPC infrastructure. This provides the XDR protocol definition * src/locking/lock_protocol.x: Wire protocol for virtlockd --- .gitignore | 1 + src/locking/lock_protocol.x | 98 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 99 insertions(+), 0 deletions(-) create mode 100644 src/locking/lock_protocol.x diff --git a/.gitignore b/.gitignore index 1bfb7b9..e6ab70b 100644 --- a/.gitignore +++ b/.gitignore @@ -51,6 +51,7 @@ /po/* /proxy/ /src/libvirt_iohelper +/src/locking/lock_protocol.[ch] /src/remote/*_client_bodies.h /src/remote/*_protocol.[ch] /src/rpc/virnetprotocol.[ch] diff --git a/src/locking/lock_protocol.x b/src/locking/lock_protocol.x new file mode 100644 index 0000000..ece1652 --- /dev/null +++ b/src/locking/lock_protocol.x @@ -0,0 +1,98 @@ +/* -*- c -*- + */ + +%#include "locking/lock_driver.h" + +typedef opaque lock_uuid[VIR_UUID_BUFLEN]; + +/* Length of long, but not unbounded, strings. + * This is an arbitrary limit designed to stop the decoder from trying + * to allocate unbounded amounts of memory when fed with a bad message. + */ +const LOCK_STRING_MAX = 65536; + +const LOCK_PARAMETERS_MAX = 20; + +/* A long string, which may NOT be NULL. */ +typedef string lock_nonnull_string<LOCK_STRING_MAX>; + +/* A long string, which may be NULL. */ +typedef lock_nonnull_string *lock_string; + +union lock_param_value switch (int type) { +case VIR_LOCK_MANAGER_PARAM_TYPE_STRING: + lock_nonnull_string s; +case VIR_LOCK_MANAGER_PARAM_TYPE_INT: + int i; +case VIR_LOCK_MANAGER_PARAM_TYPE_LONG: + hyper l; +case VIR_LOCK_MANAGER_PARAM_TYPE_UINT: + unsigned int ui; +case VIR_LOCK_MANAGER_PARAM_TYPE_ULONG: + unsigned hyper ul; +case VIR_LOCK_MANAGER_PARAM_TYPE_DOUBLE: + double d; +case VIR_LOCK_MANAGER_PARAM_TYPE_UUID: + lock_uuid u; +}; + +struct lock_param { + lock_nonnull_string key; + lock_param_value value; +}; + +struct lock_register_args { + unsigned int type; + lock_param params<LOCK_PARAMETERS_MAX>; + unsigned int flags; + bool restrictAccess; +}; + + +struct lock_add_resource_args { + unsigned int type; + lock_nonnull_string name; + lock_param params<LOCK_PARAMETERS_MAX>; + unsigned int flags; +}; + +struct lock_acquire_args { + unsigned int flags; + lock_string state; +}; + + +struct lock_release_args { + unsigned int flags; +}; + +struct lock_release_ret { + lock_string state; +}; + + +struct lock_inquire_args { + unsigned int flags; +}; + +struct lock_inquire_ret { + lock_string state; +}; + + +/* Define the program number, protocol version and procedure numbers here. */ +const LOCK_PROGRAM = 0xEA7BEEF; +const LOCK_PROTOCOL_VERSION = 1; + +enum lock_procedure { + /* Each function must have a two-word comment. The first word is + * whether remote_generator.pl handles daemon, the second whether + * it handles src/remote. Additional flags can be specified after a + * pipe. + */ + LOCK_PROC_REGISTER = 1, /* skipgen skipgen */ + LOCK_PROC_ADD_RESOURCE = 2, /* skipgen skipgen */ + LOCK_PROC_ACQUIRE = 3, /* skipgen skipgen */ + LOCK_PROC_RELEASE = 4, /* skipgen skipgen */ + LOCK_PROC_INQUIRE = 5 /* skipgen skipgen */ +}; -- 1.7.6
On 07/07/2011 08:17 AM, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
The virtlockd daemon will be responsible for managing locks on virtual machines. Communication will be via the standard RPC infrastructure. This provides the XDR protocol definition
* src/locking/lock_protocol.x: Wire protocol for virtlockd --- .gitignore | 1 + src/locking/lock_protocol.x | 98 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 99 insertions(+), 0 deletions(-) create mode 100644 src/locking/lock_protocol.x
See my patch for qemu_protocol-structs for validating on-the-wire compatibility. We ought to expand that to include this. https://www.redhat.com/archives/libvir-list/2011-July/msg00463.html -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
From: "Daniel P. Berrange" <berrange@redhat.com> This adds a simple module for managing lockspaces. A lockspace has a path to a directory, which will usually be kep on shared storage like NFS/GFS, etc Inside the lockspace leases are created as plain files. Locks on leases are taken using the virFileLock APIs (fcntl based). Leases can be created ahead of time, or auto-created/deleted at time of lock acquisition/release. Before creating/deleting a lock, a lock must be held on the "index" file for the lockspace * src/locking/lock_database.c, src/locking/lock_database.h: Simple internal API for lockspaces * include/libvirt/virterror.h, src/util/virterror.c: Add VIR_ERR_RESOURCE_BUSY --- include/libvirt/virterror.h | 1 + src/locking/lock_database.c | 915 +++++++++++++++++++++++++++++++++++++++++++ src/locking/lock_database.h | 43 ++ src/util/virterror.c | 6 + 4 files changed, 965 insertions(+), 0 deletions(-) create mode 100644 src/locking/lock_database.c create mode 100644 src/locking/lock_database.h diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h index efa4796..84226ac 100644 --- a/include/libvirt/virterror.h +++ b/include/libvirt/virterror.h @@ -232,6 +232,7 @@ typedef enum { VIR_ERR_INVALID_DOMAIN_SNAPSHOT = 71,/* invalid domain snapshot */ VIR_ERR_NO_DOMAIN_SNAPSHOT = 72, /* domain snapshot not found */ VIR_ERR_INVALID_STREAM = 73, /* stream pointer not valid */ + VIR_ERR_RESOURCE_BUSY = 74, /* resource is already in use */ } virErrorNumber; /** diff --git a/src/locking/lock_database.c b/src/locking/lock_database.c new file mode 100644 index 0000000..4e51548 --- /dev/null +++ b/src/locking/lock_database.c @@ -0,0 +1,915 @@ + +#include <config.h> + +#include <unistd.h> +#include <fcntl.h> +#include <sys/stat.h> + +#include "uuid.h" +#include "lock_database.h" +#include "virterror_internal.h" +#include "logging.h" +#include "memory.h" +#include "files.h" +#include "hash.h" +#include "util.h" + +#define VIR_FROM_THIS VIR_FROM_LOCKING + +#define virLockError(code, ...) \ + virReportErrorHelper(VIR_FROM_THIS, code, __FILE__, \ + __FUNCTION__, __LINE__, __VA_ARGS__) + +/* + * Approach to handling locking: + * + * - Lock storage area, identified by a path to a directory + * + * eg /var/lib/libvirt/lock + * + * The storage area should generally be on shared storage (NFS, + * GFS, etc), unless protection is only desired within the + * scope of the local machine + * + * - Lockspaces within a storage area, identified by an + * arbitrary name, which must not contain '/'. Recommended + * to use domain name style lockspace names, eg org.libvirt.lockd.files + * + * /var/lib/libvirt/lockd/org.libvirt.lockd.files + * + * - Lockspaces contain a single 'index' file which contains + * the original unhashed lockspace name. + * + * - Lease key within a lockspace. Identified by a arbitrary + * string, which must not contain '/', or be the word + * 'org.livirt.lockd.index'. + * + * /var/lib/libvirt/lockd/org.libvirt.lockd.files/wibble + * + * - When creating or deleting a lease must be held on the + * 'org.libvirt.lockd.index' lease of the lockspace + * + */ + +typedef struct _virLockDatabaseLease virLockDatabaseLease; +typedef virLockDatabaseLease *virLockDatabaseLeasePtr; + +typedef struct _virLockDatabaseLockspace virLockDatabaseLockspace; +typedef virLockDatabaseLockspace *virLockDatabaseLockspacePtr; + +typedef struct _virLockDatabaseFile virLockDatabaseFile; +typedef virLockDatabaseFile *virLockDatabaseFilePtr; + +struct _virLockDatabaseFile { + bool shared; + char *path; + unsigned int refs; /* # of attached users for shared locks */ + int fd; +}; + +struct _virLockDatabaseLease { + char *key; + + virLockDatabaseFile file; +}; + +struct _virLockDatabaseLockspace { + char *path; + char *name; + + bool autoLease; + + virLockDatabaseLeasePtr idx; + + size_t nleases; + virLockDatabaseLeasePtr *leases; +}; + +struct _virLockDatabase { + unsigned char hostuuid[VIR_UUID_BUFLEN]; + char *hostname; + + size_t nlockspaces; + virLockDatabaseLockspacePtr *lockspaces; + + virHashTablePtr files; +}; + + +static void virLockDatabaseFileClear(virLockDatabaseFilePtr file) +{ + if (!file) + return; + + VIR_FORCE_CLOSE(file->fd); + VIR_FREE(file->path); +} + + +static void virLockDatabaseLeaseFree(virLockDatabaseLeasePtr lease) +{ + if (!lease) + return; + + virLockDatabaseFileClear(&lease->file); + + VIR_FREE(lease->key); + VIR_FREE(lease); +} + + +static void virLockDatabaseLockspaceFree(virLockDatabaseLockspacePtr lkspc) +{ + size_t i; + + if (!lkspc) + return; + + for (i = 0 ; i < lkspc->nleases ; i++) + virLockDatabaseLeaseFree(lkspc->leases[i]); + VIR_FREE(lkspc->leases); + + virLockDatabaseLeaseFree(lkspc->idx); + + VIR_FREE(lkspc->path); + VIR_FREE(lkspc->name); + VIR_FREE(lkspc); +} + + +void virLockDatabaseFree(virLockDatabasePtr db) +{ + size_t i; + + if (!db) + return; + + virHashFree(db->files); + + for (i = 0 ; i < db->nlockspaces ; i++) + virLockDatabaseLockspaceFree(db->lockspaces[i]); + VIR_FREE(db->lockspaces); + + VIR_FREE(db->hostname); + VIR_FREE(db); +} + + +static char *virLockDatabaseBuildLeasePath(const char *path, + const char *lockspace, + const char *key) +{ + char *res; + + if (virAsprintf(&res, "%s/%s/%s", path, lockspace, key) < 0) + goto no_memory; + + return res; + +no_memory: + virReportOOMError(); + return NULL; +} + + +static char *virLockDatabaseBuildLockspacePath(const char *path, + const char *lockspace) +{ + char *res; + + if (virAsprintf(&res, "%s/%s", path, lockspace) < 0) + goto no_memory; + + return res; + +no_memory: + virReportOOMError(); + return NULL; +} + + +static virLockDatabaseLeasePtr virLockDatabaseLeaseNew(const char *path, + const char *key) +{ + virLockDatabaseLeasePtr lease; + + if (VIR_ALLOC(lease) < 0) + goto no_memory; + + if (!(lease->key = strdup(key))) + goto no_memory; + + if (!(lease->file.path = strdup(path))) + goto no_memory; + lease->file.fd = -1; + + return lease; + +no_memory: + virReportOOMError(); + virLockDatabaseLeaseFree(lease); + return NULL; +} + + +static virLockDatabaseLockspacePtr virLockDatabaseLockspaceNew(const char *path, + const char *name, + bool autoLease) +{ + virLockDatabaseLockspacePtr lkspc = NULL; + char *idxpath = virLockDatabaseBuildLeasePath(path, name, "org.libvirt.lockd.index"); + + if (!idxpath) + goto error; + + if (VIR_ALLOC(lkspc) < 0) + goto no_memory; + + if (!(lkspc->name = strdup(name))) + goto no_memory; + + if (!(lkspc->path = strdup(path))) + goto no_memory; + + lkspc->autoLease = autoLease; + + if (!(lkspc->idx = virLockDatabaseLeaseNew(idxpath, "org.libvirt.lockd.index"))) + goto error; + + VIR_FREE(idxpath); + + return lkspc; + +no_memory: + virReportOOMError(); +error: + VIR_FREE(idxpath); + virLockDatabaseLockspaceFree(lkspc); + return NULL; +} + + +virLockDatabasePtr virLockDatabaseNew(const unsigned char *hostuuid, + const char *hostname) +{ + virLockDatabasePtr db; + + if (VIR_ALLOC(db) < 0) { + virReportOOMError(); + return NULL; + } + + memcpy(db->hostuuid, hostuuid, VIR_UUID_BUFLEN); + if (!(db->hostname = strdup(hostname))) + goto no_memory; + + if (!(db->files = virHashCreate(10, NULL))) + goto error; + + return db; + +no_memory: + virReportOOMError(); +error: + virLockDatabaseFree(db); + return NULL; +} + + +static int virLockDatabaseLockspaceInitialize(virLockDatabaseLockspacePtr lkspc) +{ + int fd = -1; + char *lkspcpath = virLockDatabaseBuildLockspacePath(lkspc->path, lkspc->name); + + if (!lkspcpath) + return -1; + + VIR_DEBUG("Lockspace path %s", lkspcpath); + + if (mkdir(lkspcpath, 0700) < 0) { + if (errno != EEXIST) { + virReportSystemError(errno, + _("Unable to create lockspace index %s"), + lkspc->idx->file.path); + return -1; + } + } + + if ((fd = open(lkspc->idx->file.path, O_RDWR|O_CREAT|O_EXCL, 0600)) < 0) { + if (errno != EEXIST) { + virReportSystemError(errno, + _("Unable to create lockspace index %s"), + lkspc->idx->file.path); + goto error; + } + } + + if (VIR_CLOSE(fd) < 0) { + virReportSystemError(errno, + _("Unable to save lockspace index %s"), + lkspc->idx->file.path); + goto error; + } + + return 0; + +error: + VIR_FORCE_CLOSE(fd); + unlink(lkspc->idx->file.path); + rmdir(lkspcpath); + return -1; +} + +static virLockDatabaseLockspacePtr +virLockDatabaseFindLockspace(virLockDatabasePtr db, + const char *name) +{ + virLockDatabaseLockspacePtr lkspc; + size_t i; + + for (i = 0 ; i < db->nlockspaces ; i++) { + lkspc = db->lockspaces[i]; + if (STREQ(lkspc->name, name)) + return lkspc; + } + + return NULL; +} + +static virLockDatabaseLeasePtr +virLockDatabaseLockspaceFindLease(virLockDatabaseLockspacePtr lkspc, + const char *key) +{ + virLockDatabaseLeasePtr lease; + size_t i; + + for (i = 0 ; i < lkspc->nleases ; i++) { + lease = lkspc->leases[i]; + if (STREQ(lease->key, key)) + return lease; + } + + return NULL; +} + + +static int +virLockDatabaseRemoveLease(virLockDatabaseLockspacePtr lkspc, + virLockDatabaseLeasePtr lease) +{ + size_t i; + + for (i = 0 ; i < lkspc->nleases ; i++) { + if (lkspc->leases[i] == lease) + break; + } + + if (i == lkspc->nleases) + return -1; + + if (lkspc->nleases > 1) { + memmove(lkspc->leases + i, + lkspc->leases + i + 1, + sizeof(*lkspc->leases) * + (lkspc->nleases - (i + 1))); + VIR_SHRINK_N(lkspc->leases, lkspc->nleases, 1); + } else { + VIR_FREE(lkspc->leases); + lkspc->nleases = 0; + } + return 0; +} + + +static void +virLockDatabaseRemoveLockspace(virLockDatabasePtr db, + virLockDatabaseLockspacePtr lkspc) +{ + size_t i; + bool found = false; + + for (i = 0 ; i < db->nlockspaces ; i++) { + if (db->lockspaces[i] == lkspc) { + found = true; + break; + } + } + + if (!found) + return; + + if (db->nlockspaces > 1) { + memmove(db->lockspaces + i, + db->lockspaces + i + 1, + sizeof(*db->lockspaces) * + (db->nlockspaces - (i + 1))); + VIR_SHRINK_N(db->lockspaces, db->nlockspaces, 1); + } else { + VIR_FREE(db->lockspaces); + db->nlockspaces = 0; + } +} + + +int virLockDatabaseCreateLockspace(virLockDatabasePtr db, + const char *path, + const char *name, + bool autoLease) +{ + int ret = -1; + virLockDatabaseLockspacePtr lkspc; + + VIR_DEBUG("db=%p path=%s name=%s autoLease=%d", + db, path, name, autoLease); + + if ((lkspc = virLockDatabaseFindLockspace(db, name)) != NULL) { + if (STRNEQ(path, lkspc->path)) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Incorrect path %s for lockspace %s at %s"), + path, name, lkspc->path); + return -1; + } + + return 0; + } + + if ((lkspc = virLockDatabaseLockspaceNew(path, name, autoLease)) == NULL) + return -1; + + if (VIR_EXPAND_N(db->lockspaces, db->nlockspaces, 1) < 0) { + virReportOOMError(); + goto cleanup; + } + + if (virLockDatabaseLockspaceInitialize(lkspc) < 0) { + VIR_SHRINK_N(db->lockspaces, db->nlockspaces, 1); + goto cleanup; + } + + db->lockspaces[db->nlockspaces-1] = lkspc; + + ret = 0; + +cleanup: + if (ret != 0) + virLockDatabaseLockspaceFree(lkspc); + return ret; +} + + +int virLockDatabaseDeleteLockspace(virLockDatabasePtr db, + const char *path, + const char *name) +{ + char *lkspcpath = NULL; + virLockDatabaseLockspacePtr lkspc; + + VIR_DEBUG("db=%p path=%s name=%s", + db, path, name); + + if (!(lkspc = virLockDatabaseFindLockspace(db, name))) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lockspace %s at %s does not exist"), + name, path); + return -1; + } + + if (STRNEQ(path, lkspc->path)) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Incorrect path %s for lockspace %s at %s"), + path, name, lkspc->path); + return -1; + } + + if (!(lkspcpath = virLockDatabaseBuildLockspacePath(lkspc->path, lkspc->name))) + return -1; + + if (unlink(lkspc->idx->file.path) < 0) { + if (errno != ENOENT) { + virReportSystemError(errno, + _("Unable to delete lockspace index %s"), + lkspc->idx->file.path); + return -1; + } + } + + VIR_DEBUG("Lockspace path %s", lkspcpath); + + if (rmdir(lkspcpath) < 0) { + if (errno != ENOENT) { + virReportSystemError(errno, + _("Unable to delete lockspace %s"), + lkspcpath); + return -1; + } + } + + virLockDatabaseRemoveLockspace(db, lkspc); + virLockDatabaseLockspaceFree(lkspc); + + return 0; +} + + +static virLockDatabaseLeasePtr +virLockDatabaseLockspaceCreateLease(virLockDatabaseLockspacePtr lkspc, + const char *key) +{ + virLockDatabaseLeasePtr lease = NULL; + char *path = NULL; + int fd = -1; + int indexfd = -1; + + VIR_DEBUG("lockspace=%p key=%s", lkspc, key); + + if ((lease = virLockDatabaseLockspaceFindLease(lkspc, key)) != NULL) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lease %s in lockspace %s at %s already exists"), + key, lkspc->name, lkspc->path); + return NULL; + } + + if (!(path = virLockDatabaseBuildLeasePath(lkspc->path, lkspc->name, key))) + return NULL; + + if ((indexfd = open(lkspc->idx->file.path, O_RDWR)) < 0) { + virReportSystemError(errno, + _("Unable to open lease %s lockspace %s at %s"), + lkspc->idx->key, lkspc->name, lkspc->path); + return NULL; + } + + if (virFileLock(indexfd, false, 0, 1) < 0) { + virReportSystemError(errno, + _("Unable to lock lease %s i lockspace %s at %s"), + lkspc->idx->key, lkspc->name, lkspc->path); + goto error; + } + + if (VIR_EXPAND_N(lkspc->leases, lkspc->nleases, 1) < 0) { + virReportOOMError(); + goto error; + } + + if (!(lease = virLockDatabaseLeaseNew(path, key))) { + VIR_SHRINK_N(lkspc->leases, lkspc->nleases, 1); + goto error; + } + + VIR_DEBUG("Lease path %s", lease->file.path); + + if ((fd = open(lease->file.path, O_RDWR|O_CREAT, 0600)) < 0) { + VIR_SHRINK_N(lkspc->leases, lkspc->nleases, 1); + virReportSystemError(errno, + _("Unable to create lease %s lockspace %s at %s"), + key, lkspc->name, lkspc->path); + goto error; + } + if (VIR_CLOSE(fd) < 0) { + VIR_SHRINK_N(lkspc->leases, lkspc->nleases, 1); + virReportSystemError(errno, + _("Unable to save lease %s lockspace %s at %s"), + key, lkspc->name, lkspc->path); + goto error; + } + + if (VIR_CLOSE(indexfd) < 0) { + VIR_SHRINK_N(lkspc->leases, lkspc->nleases, 1); + virReportSystemError(errno, + _("Unable to close lease %s lockspace %s at %s"), + lkspc->idx->key, lkspc->name, lkspc->path); + goto error; + } + + lkspc->leases[lkspc->nleases-1] = lease; + + return lease; + +error: + virLockDatabaseLeaseFree(lease); + if (fd != -1) + unlink(path); + VIR_FORCE_CLOSE(fd); + VIR_FORCE_CLOSE(indexfd); + VIR_FREE(path); + return NULL; +} + + +static int +virLockDatabaseLockspaceDeleteLease(virLockDatabaseLockspacePtr lkspc, + const char *key) +{ + virLockDatabaseLeasePtr lease; + int fd = -1; + int indexfd = -1; + + VIR_DEBUG("lockspace=%p key=%s", lkspc, key); + + if (!(lease = virLockDatabaseLockspaceFindLease(lkspc, key))) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lease %s in lockspace %s at %s does not exist"), + key, lkspc->name, lkspc->path); + return -1; + } + + if ((indexfd = open(lkspc->idx->file.path, O_RDWR)) < 0) { + virReportSystemError(errno, + _("Unable to open lease %s lockspace %s at %s"), + lkspc->idx->key, lkspc->name, lkspc->path); + return -1; + } + + if (virFileLock(indexfd, false, 0, 1) < 0) { + virReportSystemError(errno, + _("Unable to lock lease %s i lockspace %s at %s"), + lkspc->idx->key, lkspc->name, lkspc->path); + goto error; + } + + VIR_DEBUG("Lease path %s", lease->file.path); + + if ((fd = open(lease->file.path, O_RDWR, 0600)) < 0) { + if (errno == ENOENT) + goto cleanup; + + virReportSystemError(errno, + _("Unable to create lease %s lockspace %s at %s"), + key, lkspc->name, lkspc->path); + goto error; + } + + if (unlink(lease->file.path) < 0) { + if (errno == ENOENT) + goto cleanup; + + virReportSystemError(errno, + _("Unable to delete lease %s lockspace %s at %s"), + key, lkspc->name, lkspc->path); + return -1; + } + +cleanup: + if (VIR_CLOSE(fd) < 0) { + virReportSystemError(errno, + _("Unable to closee lease %s lockspace %s at %s"), + key, lkspc->name, lkspc->path); + goto error; + } + + if (VIR_CLOSE(indexfd) < 0) { + virReportSystemError(errno, + _("Unable to close lease %s lockspace %s at %s"), + lkspc->idx->key, lkspc->name, lkspc->path); + goto error; + } + + ignore_value(virLockDatabaseRemoveLease(lkspc, lease)); + + virLockDatabaseLeaseFree(lease); + return 0; + +error: + VIR_FORCE_CLOSE(fd); + VIR_FORCE_CLOSE(indexfd); + return -1; +} + + +int virLockDatabaseCreateLease(virLockDatabasePtr db, + const char *path, + const char *lockspace, + const char *key) +{ + virLockDatabaseLockspacePtr lkspc; + virLockDatabaseLeasePtr lease; + + VIR_DEBUG("db=%p path=%s lockspace=%s key=%s", db, path, lockspace, key); + + if (!(lkspc = virLockDatabaseFindLockspace(db, lockspace))) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lockspace %s at %s does not exist"), + lockspace, path); + return -1; + } + + if (STRNEQ(path, lkspc->path)) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Incorrect path %s for lockspace %s at %s"), + path, lockspace, lkspc->path); + return -1; + } + + if (!(lease = virLockDatabaseLockspaceCreateLease(lkspc, key))) + return -1; + + return 0; +} + + +int virLockDatabaseDeleteLease(virLockDatabasePtr db, + const char *path, + const char *lockspace, + const char *key) +{ + virLockDatabaseLockspacePtr lkspc; + + VIR_DEBUG("db=%p path=%s lockspace=%s key=%s", db, path, lockspace, key); + + if (!(lkspc = virLockDatabaseFindLockspace(db, lockspace))) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lockspace %s at %s does not exist"), + lockspace, path); + return -1; + } + + + if (STRNEQ(path, lkspc->path)) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Incorrect path %s for lockspace %s at %s"), + path, lockspace, lkspc->path); + return -1; + } + + return virLockDatabaseLockspaceDeleteLease(lkspc, key); +} + + +int virLockDatabaseAcquireLease(virLockDatabasePtr db, + const char *path, + const char *lockspace, + const char *key, + bool shared) +{ + int ret = -1; + virLockDatabaseLockspacePtr lkspc = NULL; + virLockDatabaseLeasePtr lease = NULL; + VIR_DEBUG("db=%p path=%s lockspace=%s key=%s shared=%d", + db, path, lockspace, key, shared); + + if (!(lkspc = virLockDatabaseFindLockspace(db, lockspace))) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lockspace %s at %s does not exist"), + lockspace, path); + return -1; + } + + if (STRNEQ(lkspc->path, path)) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Incorrect path %s for lockspace %s at %s"), + path, lockspace, lkspc->path); + return -1; + } + + if (!(lease = virLockDatabaseLockspaceFindLease(lkspc, key))) { + if (!lkspc->autoLease) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lease %s in lockspace %s at %s does not exist"), + key, lockspace, path); + return -1; + } + + if (!(lease = virLockDatabaseLockspaceCreateLease(lkspc, key))) + return -1; + } + + VIR_DEBUG("Lease key=%s fd=%d shared=%d refs=%d", + lease->key, lease->file.fd, lease->file.shared, lease->file.refs); + + + if (lease->file.fd == -1) { + /* No one holds it locally. Try and grab the fcntl lock + * to stop other hosts + */ + if ((lease->file.fd = open(lease->file.path, O_RDWR)) < 0) { + virReportSystemError(errno, + _("Unable to open lease %s i lockspace %s at %s"), + key, lockspace, path); + goto cleanup; + } + + if (virFileLock(lease->file.fd, shared, 0, 1) < 0) { + virReportSystemError(errno, + _("Unable to lock lease %s i lockspace %s at %s"), + key, lockspace, path); + VIR_FORCE_CLOSE(lease->file.fd); + goto cleanup; + } + + lease->file.shared = shared; + lease->file.refs = 1; + } else { + /* Someone on this host already holds the lock, so + * check if sharing mode is compatible */ + if (!lease->file.shared || !shared) { + virLockError(VIR_ERR_RESOURCE_BUSY, + _("Lease %s in lockspace %s at %s is already held"), + key, lockspace, path); + goto cleanup; + } + + /* Both shared, so claim another ref */ + lease->file.refs++; + } + + ret = 0; + +cleanup: + if (ret != 0) { + if (lease && + lease->file.fd == -1 && + lkspc->autoLease) { + virErrorPtr orig_error = virSaveLastError(); + + virLockDatabaseLockspaceDeleteLease(lkspc, key); + + if (orig_error) { + virSetError(orig_error); + virFreeError(orig_error); + } + } + + } + return ret; +} + + +int virLockDatabaseReleaseLease(virLockDatabasePtr db, + const char *path, + const char *lockspace, + const char *key, + bool shared) +{ + int ret = -1; + virLockDatabaseLockspacePtr lkspc = NULL; + virLockDatabaseLeasePtr lease = NULL; + + VIR_DEBUG("db=%p path=%s lockspace=%s key=%s shared=%d", + db, path, lockspace, key, shared); + + if (!(lkspc = virLockDatabaseFindLockspace(db, lockspace))) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lockspace %s at %s does not exist"), + lockspace, path); + return -1; + } + + if (STRNEQ(lkspc->path, path)) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Incorrect path %s for lockspace %s at %s"), + path, lockspace, lkspc->path); + return -1; + } + + if (!(lease = virLockDatabaseLockspaceFindLease(lkspc, key))) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lease %s in lockspace %s at %s does not exist"), + key, lockspace, path); + return -1; + } + + VIR_DEBUG("Lease key=%s fd=%d shared=%d refs=%d", + lease->key, lease->file.fd, lease->file.shared, lease->file.refs); + + if (lease->file.fd == -1) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lease %s in lockspace %s at %s is not held"), + key, lockspace, path); + return -1; + } + + /* Someone on this host already holds the lock, so + * check if sharing mode is compatible */ + if (lease->file.shared) { + if (!shared) { + virLockError(VIR_ERR_RESOURCE_BUSY, + _("Lease %s in lockspace %s at %s is shared, not exclusive"), + key, lockspace, path); + return -1; + } + lease->file.refs--; + } else { + if (shared) { + virLockError(VIR_ERR_RESOURCE_BUSY, + _("Lease %s in lockspace %s at %s is exclusive, not shared"), + key, lockspace, path); + return -1; + } + } + + if (!lease->file.shared || (lease->file.refs == 0)) { + if (virFileUnlock(lease->file.fd, 0, 1) < 0) { + char ebuf[1024]; + VIR_ERROR(_("Unable to unlock lease %s i lockspace %s at %s: %s"), + key, lockspace, path, + virStrerror(errno, ebuf, sizeof(ebuf))); + } + VIR_FORCE_CLOSE(lease->file.fd); + virLockDatabaseRemoveLease(lkspc, lease); + virLockDatabaseLeaseFree(lease); + } + + ret = 0; + + return ret; +} diff --git a/src/locking/lock_database.h b/src/locking/lock_database.h new file mode 100644 index 0000000..3480383 --- /dev/null +++ b/src/locking/lock_database.h @@ -0,0 +1,43 @@ + + +#include "internal.h" + +typedef struct _virLockDatabase virLockDatabase; +typedef virLockDatabase *virLockDatabasePtr; + + +virLockDatabasePtr virLockDatabaseNew(const unsigned char *hostuuid, + const char *hostname); + +void virLockDatabaseFree(virLockDatabasePtr db); + +int virLockDatabaseCreateLockspace(virLockDatabasePtr db, + const char *path, + const char *name, + bool autoLease); + +int virLockDatabaseDeleteLockspace(virLockDatabasePtr db, + const char *path, + const char *name); + +int virLockDatabaseCreateLease(virLockDatabasePtr db, + const char *path, + const char *lockspace, + const char *key); + +int virLockDatabaseDeleteLease(virLockDatabasePtr db, + const char *path, + const char *lockspace, + const char *key); + +int virLockDatabaseAcquireLease(virLockDatabasePtr db, + const char *path, + const char *lockspace, + const char *key, + bool shared); + +int virLockDatabaseReleaseLease(virLockDatabasePtr db, + const char *path, + const char *lockspace, + const char *key, + bool shared); diff --git a/src/util/virterror.c b/src/util/virterror.c index 75058f3..ff93109 100644 --- a/src/util/virterror.c +++ b/src/util/virterror.c @@ -1174,6 +1174,12 @@ virErrorMsg(virErrorNumber error, const char *info) else errmsg = _("invalid stream pointer in %s"); break; + case VIR_ERR_RESOURCE_BUSY: + if (info == NULL) + errmsg = _("resource busy"); + else + errmsg = _("resource busy %s"); + break; } return (errmsg); } -- 1.7.6
From: "Daniel P. Berrange" <berrange@redhat.com> The 'fcntl' lock driver is a implementation of the lock manager driver API, which will be linked directly into the virtlockd process. It will use the simple lock database added in the previous commit to acquire/release locks for leases or disks associated with virtual machines. * src/locking/lock_driver_fcntl.c, src/locking/lock_driver_fcntl.h: Add fcntl lock driver implementation --- bootstrap.conf | 1 + src/locking/lock_driver_fcntl.c | 929 +++++++++++++++++++++++++++++++++++++++ src/locking/lock_driver_fcntl.h | 4 + 3 files changed, 934 insertions(+), 0 deletions(-) create mode 100644 src/locking/lock_driver_fcntl.c create mode 100644 src/locking/lock_driver_fcntl.h diff --git a/bootstrap.conf b/bootstrap.conf index 581d60b..f78d167 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -34,6 +34,7 @@ connect configmake count-one-bits crypto/md5 +crypto/sha256 dirname-lgpl fcntl-h fnmatch diff --git a/src/locking/lock_driver_fcntl.c b/src/locking/lock_driver_fcntl.c new file mode 100644 index 0000000..904be45 --- /dev/null +++ b/src/locking/lock_driver_fcntl.c @@ -0,0 +1,929 @@ +/* + * lock_driver_fcntl.c: A lock driver using fcntl() + * + * Copyright (C) 2010-2011 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + */ + +#include <config.h> + +#include <stdbool.h> +#include <fcntl.h> +#include <signal.h> + +#include "lock_driver_fcntl.h" +#include "lock_database.h" +#include "memory.h" +#include "logging.h" +#include "uuid.h" +#include "util.h" +#include "files.h" +#include "threads.h" +#include "virterror_internal.h" +#include "conf.h" +#include "configmake.h" +#include "sha256.h" +#include "hash.h" + +#define VIR_FROM_THIS VIR_FROM_LOCKING + +#define virLockError(code, ...) \ + virReportErrorHelper(VIR_FROM_THIS, code, __FILE__, \ + __FUNCTION__, __LINE__, __VA_ARGS__) + +typedef struct _virLockManagerFcntlResource virLockManagerFcntlResource; +typedef virLockManagerFcntlResource *virLockManagerFcntlResourcePtr; + +typedef struct _virLockManagerFcntlObject virLockManagerFcntlObject; +typedef virLockManagerFcntlObject *virLockManagerFcntlObjectPtr; + +typedef struct _virLockManagerFcntlPrivate virLockManagerFcntlPrivate; +typedef virLockManagerFcntlPrivate *virLockManagerFcntlPrivatePtr; + +#define VIR_LOCK_FCNTL_LOCKSPACE_FILES_NAME \ + "org.libvirt.lockd.files" + +struct _virLockManagerFcntlResource { + char *path; + char *lockspace; + char *key; + bool shared; + bool active; +}; + +struct _virLockManagerFcntlObject { + char *hashKey; + + int refs; + bool dead; /* True if the primary client has gone */ + + char *name; + unsigned char uuid[VIR_UUID_BUFLEN]; + pid_t pid; + + /* Resources currently held by the object */ + size_t nresources; + virLockManagerFcntlResourcePtr *resources; +}; + +struct _virLockManagerFcntlPrivate { + pid_t pid; + virLockManagerFcntlObjectPtr object; + + /* Resources added, but yet to be acquired/released */ + size_t nresources; + virLockManagerFcntlResourcePtr *resources; +}; + +struct _virLockManagerFcntlDriver { + virMutex lock; + char *autoDiskLeasePath; + virLockDatabasePtr lockDB; + virHashTablePtr objects; +} *driver; + + +static char *virLockManagerFcntlNewHash(const char *key) +{ + char buf[SHA256_DIGEST_SIZE]; + char *sum; + const char hex[16] = { '0', '1', '2', '3', '4', '5', '6', '7', + '8', '9', 'a', 'b', 'c', 'd', 'e', 'f' }; + size_t i; + + if (!(sha256_buffer(key, strlen(key), buf))) + return NULL; + + if (VIR_ALLOC_N(sum, (SHA256_DIGEST_SIZE * 2) + 1) < 0) { + virReportOOMError(); + return NULL; + } + + for (i = 0 ; i < SHA256_DIGEST_SIZE ; i++) { + sum[i*2] = hex[(buf[i] >> 4) & 0xf]; + sum[(i*2)+1] = hex[buf[i] & 0xf]; + } + sum[(SHA256_DIGEST_SIZE*2)] = '\0'; + + return sum; +} + + + +static void virLockManagerFcntlResourceFree(virLockManagerFcntlResourcePtr res) +{ + if (!res) + return; + + VIR_FREE(res->path); + VIR_FREE(res->lockspace); + VIR_FREE(res->key); + + VIR_FREE(res); +} + + +static void virLockManagerFcntlObjectFree(virLockManagerFcntlObjectPtr object) +{ + size_t i; + + if (!object) + return; + + object->refs--; + if (object->refs) + return; + + VIR_FREE(object->hashKey); + VIR_FREE(object->name); + if (object->resources) { + for (i = 0 ; i < object->nresources ; i++) + virLockManagerFcntlResourceFree(object->resources[i]); + VIR_FREE(object->resources); + } + + VIR_FREE(object); +} + + +static void virLockManagerFcntlPrivateFree(virLockManagerFcntlPrivatePtr priv) +{ + size_t i; + + if (!priv) + return; + + virLockManagerFcntlObjectFree(priv->object); + + if (priv->resources) { + for (i = 0 ; i < priv->nresources ; i++) + virLockManagerFcntlResourceFree(priv->resources[i]); + VIR_FREE(priv->resources); + } + + VIR_FREE(priv); +} + + +static virLockManagerFcntlObjectPtr +virLockManagerFcntlObjectNew(const char *name, + unsigned char *uuid, + pid_t pid) +{ + virLockManagerFcntlObjectPtr obj; + + if (VIR_ALLOC(obj) < 0) + goto no_memory; + + if (!(obj->name = strdup(name))) + goto no_memory; + memcpy(obj->uuid, uuid, VIR_UUID_BUFLEN); + obj->pid = pid; + obj->refs = 1; + + return obj; + +no_memory: + virReportOOMError(); + virLockManagerFcntlObjectFree(obj); + return NULL; +} + +static virLockManagerFcntlPrivatePtr +virLockManagerFcntlPrivateNewAttach(virLockManagerFcntlObjectPtr object, + pid_t clientPid) +{ + virLockManagerFcntlPrivatePtr priv; + + if (VIR_ALLOC(priv) < 0) + goto no_memory; + + object->refs++; + priv->object = object; + priv->pid = clientPid; + + return priv; + +no_memory: + virReportOOMError(); + return NULL; +} + + +static virLockManagerFcntlPrivatePtr +virLockManagerFcntlPrivateNew(const char *name, + unsigned char *uuid, + pid_t pid, + pid_t clientPid) +{ + virLockManagerFcntlPrivatePtr priv; + + if (VIR_ALLOC(priv) < 0) + goto no_memory; + + if (!(priv->object = virLockManagerFcntlObjectNew(name, uuid, pid))) + goto error; + priv->pid = clientPid; + + return priv; + +no_memory: + virReportOOMError(); +error: + VIR_FREE(priv); + return NULL; +} + + +static virLockManagerFcntlResourcePtr +virLockManagerFcntlResourceNew(const char *path, + const char *lockspace, + const char *key, + bool hashKey, + bool shared) +{ + virLockManagerFcntlResourcePtr res; + + if (VIR_ALLOC(res) < 0) + goto no_memory; + + if (!(res->path = strdup(path))) + goto no_memory; + if (!(res->lockspace = strdup(lockspace))) + goto no_memory; + if (hashKey) { + if (!(res->key = virLockManagerFcntlNewHash(key))) + goto error; + } else { + if (!(res->key = strdup(key))) + goto no_memory; + } + + res->shared = shared; + + return res; + +no_memory: + virReportOOMError(); +error: + virLockManagerFcntlResourceFree(res); + return NULL; +} + + +#if 0 +/* + * sanlock plugin for the libvirt virLockManager API + */ +static int virLockManagerFcntlLoadConfig(const char *configFile) +{ + virConfPtr conf; + virConfValuePtr p; + + if (access(configFile, R_OK) == -1) { + if (errno != ENOENT) { + virReportSystemError(errno, + _("Unable to access config file %s"), + configFile); + return -1; + } + return 0; + } + + if (!(conf = virConfReadFile(configFile, 0))) + return -1; + +# define CHECK_TYPE(name,typ) if (p && p->type != (typ)) { \ + virLockError(VIR_ERR_INTERNAL_ERROR, \ + "%s: %s: expected type " #typ, \ + configFile, (name)); \ + virConfFree(conf); \ + return -1; \ + } + + p = virConfGetValue(conf, "disk_lease_dir"); + CHECK_TYPE("disk_lease_dir", VIR_CONF_STRING); + if (p && p->str) { + VIR_FREE(driver.autoDiskLeasePath); + if (!(driver.autoDiskLeasePath = strdup(p->str))) { + virReportOOMError(); + virConfFree(conf); + return -1; + } + } + + virConfFree(conf); + return 0; +} +#endif + +static char *virLockManagerFcntlPath(bool privileged) +{ + char *path; + if (privileged) { + if (!(path = strdup(LOCALSTATEDIR "/lib/libvirt/lockd"))) { + virReportOOMError(); + return NULL; + } + } else { + char *userdir; + if (!(userdir = virGetUserDirectory(geteuid()))) + return NULL; + + if (virAsprintf(&path, "%s/.libvirt/lockd", userdir) < 0) { + virReportOOMError(); + } + VIR_FREE(userdir); + } + return path; +} + + + +static int virLockManagerFcntlInit(unsigned int version ATTRIBUTE_UNUSED, + const char *configFile ATTRIBUTE_UNUSED, + unsigned int flags ATTRIBUTE_UNUSED) +{ + unsigned char hostuuid[VIR_UUID_BUFLEN]; + char *hostname = virGetHostname(NULL); + if (!hostname) + return -1; + + if (driver) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Driver already initialized")); + return -1; + } + + if (VIR_ALLOC(driver) < 0) { + virReportOOMError(); + return -1; + } + + if (virMutexInit(&driver->lock) < 0) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Unable to initialize driver mutex")); + VIR_FREE(driver); + return -1; + } + + if (!(driver->lockDB = virLockDatabaseNew(hostuuid, hostname))) + goto error; + + if (!(driver->autoDiskLeasePath = virLockManagerFcntlPath(getuid() == 0))) { + virReportOOMError(); + goto error; + } + + if (!(driver->objects = virHashCreate(10, NULL))) + goto error; + +#if 0 + if (virLockManagerFcntlLoadConfig(configFile) < 0) + goto error; +#endif + + /* A standard lockspace to which we map virtual disk files */ + if (virLockDatabaseCreateLockspace(driver->lockDB, + driver->autoDiskLeasePath, + VIR_LOCK_FCNTL_LOCKSPACE_FILES_NAME, + true) < 0) + goto error; + + VIR_FREE(hostname); + return 0; + +error: + virLockDatabaseFree(driver->lockDB); + driver->lockDB = NULL; + VIR_FREE(driver->autoDiskLeasePath); + VIR_FREE(hostname); + return -1; +} + +static int virLockManagerFcntlDeinit(void) +{ + virLockDatabaseFree(driver->lockDB); + VIR_FREE(driver->autoDiskLeasePath); + virMutexDestroy(&driver->lock); + VIR_FREE(driver); + return 0; +} + + +static char *virLockManagerFcntlObjectName(unsigned char *uuid, + pid_t pid) +{ + char *ret; + char uuidstr[VIR_UUID_STRING_BUFLEN]; + + virUUIDFormat(uuid, uuidstr); + + if (virAsprintf(&ret, "%s-%llu", uuidstr, (long long unsigned)pid) < 0) + return NULL; + return ret; +} + + +static int virLockManagerFcntlNew(virLockManagerPtr lock, + unsigned int type, + size_t nparams, + virLockManagerParamPtr params, + unsigned int flags) +{ + virLockManagerFcntlPrivatePtr priv = NULL; + char *name = NULL; + pid_t pid = 0; + pid_t clientPid = 0; + unsigned char *uuid = NULL; + size_t i; + char *objectName = NULL; + int ret = -1; + + virCheckFlags(0, -1); + + virMutexLock(&driver->lock); + + if (type != VIR_LOCK_MANAGER_OBJECT_TYPE_DOMAIN) + goto cleanup; + + for (i = 0 ; i < nparams ; i++) { + virLockManagerParamPtr param = ¶ms[i]; + if (STREQ(param->key, "uuid")) { + uuid = param->value.uuid; + } else if (STREQ(param->key, "name")) { + name = param->value.str; + } else if (STREQ(param->key, "pid")) { + pid = param->value.ui; + } else if (STREQ(param->key, "client-pid")) { + clientPid = param->value.ui; + } + } + + if (!name) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Missing 'name' parameter for object lock")); + goto cleanup; + } + + if (!pid) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Missing 'pid' parameter for object lock")); + goto cleanup; + } + + if (!clientPid) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Missing 'client-pid' parameter for object lock")); + goto cleanup; + } + + if (!uuid) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Missing 'uuid' parameter for object lock")); + goto cleanup; + } + + if (!virUUIDIsValid(uuid)) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Malformed 'uuid' parameter for object lock")); + goto cleanup; + } + + objectName = virLockManagerFcntlObjectName(uuid, pid); + if (clientPid == pid) { + if (!(priv = virLockManagerFcntlPrivateNew(name, uuid, pid, clientPid))) + goto cleanup; + + if (virHashAddEntry(driver->objects, objectName, priv->object) < 0) + goto cleanup; + + priv->object->hashKey = objectName; + objectName = NULL; + } else { + virLockManagerFcntlObjectPtr object = virHashLookup(driver->objects, objectName); + + if (!object) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("No active connection for object %s (%s)"), + objectName, name); + goto cleanup; + } + + if (!(priv = virLockManagerFcntlPrivateNewAttach(object, clientPid))) + goto cleanup; + } + + lock->privateData = priv; + + ret = 0; + +cleanup: + if (ret != 0) + virLockManagerFcntlPrivateFree(priv); + VIR_FREE(objectName); + virMutexUnlock(&driver->lock); + return ret; +} + + +static int virLockManagerFcntlAddResource(virLockManagerPtr lock, + unsigned int type, + const char *name, + size_t nparams, + virLockManagerParamPtr params, + unsigned int flags) +{ + virLockManagerFcntlPrivatePtr priv = lock->privateData; + virLockManagerFcntlResourcePtr res; + const char *lockspace = NULL; + const char *path = NULL; + bool hashName = false; + size_t i; + int ret = 0; + + virCheckFlags(VIR_LOCK_MANAGER_RESOURCE_SHARED | + VIR_LOCK_MANAGER_RESOURCE_READONLY, -1); + + if ((flags & VIR_LOCK_MANAGER_RESOURCE_READONLY) && + (flags & VIR_LOCK_MANAGER_RESOURCE_SHARED)) { + virLockError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Lease flags cannot request 'shared' and 'readonly' at the same time")); + return -1; + } + + if (flags & VIR_LOCK_MANAGER_RESOURCE_READONLY) + return 0; + + virMutexLock(&driver->lock); + + switch (type) { + case VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK: + hashName = true; + path = driver->autoDiskLeasePath; + lockspace = VIR_LOCK_FCNTL_LOCKSPACE_FILES_NAME; + break; + + case VIR_LOCK_MANAGER_RESOURCE_TYPE_LEASE: + for (i = 0 ; i < nparams ; i++) { + virLockManagerParamPtr param = ¶ms[i]; + if (STREQ(param->key, "path")) { + path = param->value.str; + } else if (STREQ(param->key, "lockspace")) { + lockspace = param->value.str; + } else if (STREQ(param->key, "offset")) { + if (param->value.ul) { + virLockError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("lease offset must be zero")); + goto cleanup; + } + } else { + virLockError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Unexpected parameter %s"), + param->key); + goto cleanup; + } + } + break; + + default: + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Unsupported resource type %d"), + type); + goto cleanup; + } + + if (!lockspace) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Missing lockspace for resource %s"), name); + goto cleanup; + } + + if (!path) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Missing lock path for resource %s"), name); + goto cleanup; + } + + if (priv->object->dead) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("The object connection for %s has died"), + priv->object->name); + goto cleanup; + } + + if (!(res = virLockManagerFcntlResourceNew(path, lockspace, name, hashName, + flags & VIR_LOCK_MANAGER_RESOURCE_SHARED))) + goto cleanup; + + if (VIR_EXPAND_N(priv->resources, priv->nresources, 1) < 0) { + virLockManagerFcntlResourceFree(res); + goto cleanup; + } + priv->resources[priv->nresources-1] = res; + + ret = 0; + +cleanup: + virMutexUnlock(&driver->lock); + return ret; +} + + +static int virLockManagerFcntlAcquire(virLockManagerPtr lock, + const char *state ATTRIBUTE_UNUSED, + unsigned int flags, + int *fd ATTRIBUTE_UNUSED) +{ + virLockManagerFcntlPrivatePtr priv = lock->privateData; + size_t i = 0; + int ret = -1; + + virCheckFlags(VIR_LOCK_MANAGER_ACQUIRE_RESTRICT | + VIR_LOCK_MANAGER_ACQUIRE_REGISTER_ONLY, -1); + + if (flags & VIR_LOCK_MANAGER_ACQUIRE_REGISTER_ONLY) + return 0; + + virMutexLock(&driver->lock); + + if (priv->object->dead) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("The object connection for %s has died"), + priv->object->name); + goto cleanup; + } + + if (VIR_EXPAND_N(priv->object->resources, + priv->object->nresources, + priv->nresources) < 0) { + virReportOOMError(); + goto cleanup; + } + + for (i = 0 ; i < priv->nresources ; i++) { + if (virLockDatabaseAcquireLease(driver->lockDB, + priv->resources[i]->path, + priv->resources[i]->lockspace, + priv->resources[i]->key, + priv->resources[i]->shared) < 0) { + i--; + goto error; + } + priv->resources[i]->active = true; + } + + for (i = 0 ; i < priv->nresources ; i++) { + priv->object->resources[priv->object->nresources-priv->nresources+i] + = priv->resources[i]; + } + VIR_FREE(priv->resources); + priv->nresources = 0; + + ret = 0; + +cleanup: + virMutexUnlock(&driver->lock); + return ret; + +error: + for (i = 0 ; i < priv->nresources ; i++) { + if (priv->resources[i]->active && + virLockDatabaseReleaseLease(driver->lockDB, + priv->resources[i]->path, + priv->resources[i]->lockspace, + priv->resources[i]->key, + priv->resources[i]->shared) < 0) + VIR_WARN("Cannot release resource after failed lock"); + priv->resources[i]->active = false; + } + VIR_SHRINK_N(priv->object->resources, + priv->object->nresources, + priv->nresources); + goto cleanup; +} + + +static int virLockManagerFcntlRelease(virLockManagerPtr lock, + char **state, + unsigned int flags) +{ + virLockManagerFcntlPrivatePtr priv = lock->privateData; + size_t i; + size_t j; + int ret = -1; + bool failed = false; + int *indexes; + + virCheckFlags(0, -1); + + *state = NULL; + + virMutexLock(&driver->lock); + + if (priv->object->dead) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("The object connection for %s has died"), + priv->object->name); + goto cleanup; + } + + if (VIR_ALLOC_N(indexes, priv->nresources) < 0) { + virReportOOMError(); + goto cleanup; + } + + /* Make sure the resources requested for release are in fact + * currently held by this object + */ + for (i = 0 ; i < priv->nresources ; i++) { + bool match = false; + for (j = 0 ; j < priv->object->nresources ; j++) { + if (STREQ(priv->resources[i]->path, + priv->object->resources[j]->path) && + STREQ(priv->resources[i]->lockspace, + priv->object->resources[j]->lockspace) && + STREQ(priv->resources[i]->key, + priv->object->resources[j]->key)) { + match = true; + indexes[i] = j; + break; + } + } + + if (!match) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Lease %s in lockspace %s at %s is not held by this object"), + priv->resources[i]->key, + priv->resources[i]->lockspace, + priv->resources[i]->path); + VIR_FREE(indexes); + return -1; + } + } + + + /* + * Acquire all the new leases + */ + for (i = 0 ; i < priv->nresources ; i++) { + if (virLockDatabaseReleaseLease(driver->lockDB, + priv->resources[i]->path, + priv->resources[i]->lockspace, + priv->resources[i]->key, + priv->resources[i]->shared) < 0) { + failed = true; + } else { + priv->object->resources[indexes[i]]->active = false; + } + } + + /* + * Move the newly held resources to the object resource list + */ + for (i = 0, j = 0 ; i < priv->object->nresources ; i++) { + if (priv->object->resources[i]->active) { + priv->object->resources[j] = priv->object->resources[i]; + j++; + } else { + virLockManagerFcntlResourceFree(priv->object->resources[i]); + } + } + VIR_SHRINK_N(priv->object->resources, + priv->object->nresources, + j); + + VIR_FREE(indexes); + + ret = failed ? -1 : 0; + +cleanup: + virMutexUnlock(&driver->lock); + return ret; +} + + +static int virLockManagerFcntlInquire(virLockManagerPtr lock ATTRIBUTE_UNUSED, + char **state, + unsigned int flags) +{ + virLockManagerFcntlPrivatePtr priv = lock->privateData; + int ret = -1; + + virCheckFlags(0, -1); + + *state = NULL; + + virMutexLock(&driver->lock); + + if (priv->object->dead) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("The object connection for %s has died"), + priv->object->name); + goto cleanup; + } + + ret = 0; + +cleanup: + virMutexUnlock(&driver->lock); + return ret; +} + + +static void virLockManagerFcntlFree(virLockManagerPtr lock) +{ + virLockManagerFcntlPrivatePtr priv = lock->privateData; + + /* + * If the primary object PID closed its connection, + * we need to make sure it is really dead and release + * its resources + */ + if (priv->pid == priv->object->pid) { + size_t i; + + VIR_DEBUG("Killing off PID %llu hash key %s", + (unsigned long long)priv->pid, priv->object->hashKey); + + virHashRemoveEntry(driver->objects, priv->object->hashKey); + + /* + * Kill off all held leases + */ + for (i = 0 ; i < priv->object->nresources ; i++) { + if (virLockDatabaseReleaseLease(driver->lockDB, + priv->object->resources[i]->path, + priv->object->resources[i]->lockspace, + priv->object->resources[i]->key, + priv->object->resources[i]->shared) < 0) { + VIR_WARN("Unable to release resource %s in %s at %s", + priv->object->resources[i]->key, + priv->object->resources[i]->lockspace, + priv->object->resources[i]->path); + } + } + + /* This loop sends SIGTERM, then waits a few iterations + * (1.6 seconds) to see if it dies. If still alive then + * it does SIGKILL, and waits a few more iterations (1.6 + * seconds more) to confirm that it has really gone. + */ + for (i = 0 ; i < 15 ; i++) { + int signum; + if (i == 0) + signum = SIGTERM; + else if (i == 8) + signum = SIGKILL; + else + signum = 0; /* Just check for existence */ + + if (virKillProcess(priv->object->pid, signum) < 0) { + if (errno != ESRCH) { + char ebuf[1024]; + VIR_WARN("Failed to kill process %d %s", + priv->object->pid, virStrerror(errno, ebuf, sizeof ebuf)); + } + break; + } + + usleep(200 * 1000); + } + + priv->object->dead = true; + } + + virLockManagerFcntlPrivateFree(priv); +} + +virLockDriver virLockDriverFcntl = +{ + .version = VIR_LOCK_MANAGER_VERSION, + + .drvInit = virLockManagerFcntlInit, + .drvDeinit = virLockManagerFcntlDeinit, + + .drvNew = virLockManagerFcntlNew, + .drvFree = virLockManagerFcntlFree, + + .drvAddResource = virLockManagerFcntlAddResource, + + .drvAcquire = virLockManagerFcntlAcquire, + .drvRelease = virLockManagerFcntlRelease, + + .drvInquire = virLockManagerFcntlInquire, +}; diff --git a/src/locking/lock_driver_fcntl.h b/src/locking/lock_driver_fcntl.h new file mode 100644 index 0000000..c539fe1 --- /dev/null +++ b/src/locking/lock_driver_fcntl.h @@ -0,0 +1,4 @@ + +#include "lock_driver.h" + +extern virLockDriver virLockDriverFcntl; -- 1.7.6
From: "Daniel P. Berrange" <berrange@redhat.com> This adds a new daemon 'virtlockd' which will manage the locks for all running virtual machines. Communication is via a new RPC lock program. The intent is that virtlockd runs forever and if it ever crashes, the entire host should be restarted (ideally via a hardware fencing agent) to ensure maximum safety of guest disks. * cfg.mk: Ignore the lockd files * src/locking/lock_daemon.c, src/locking/lock_daemon.h: Main daemon startup code * src/locking/lock_daemon_dispatch.c, src/locking/lock_daemon_dispatch.h: RPC dispatcher for lock program * src/locking/virtlockd.init.in: Init script * src/locking/virtlockd.sysconf: Init script configuration * libvirt.spec.in: Add initscript --- cfg.mk | 4 +- libvirt.spec.in | 8 + po/POTFILES.in | 5 + src/.gitignore | 4 + src/Makefile.am | 106 +++++- src/libvirt_private.syms | 2 + src/locking/lock_daemon.c | 776 ++++++++++++++++++++++++++++++++++++ src/locking/lock_daemon_dispatch.c | 334 ++++++++++++++++ src/locking/lock_daemon_dispatch.h | 46 +++ src/locking/virtlockd.init.in | 93 +++++ src/locking/virtlockd.sysconf | 3 + 11 files changed, 1374 insertions(+), 7 deletions(-) create mode 100644 src/locking/lock_daemon.c create mode 100644 src/locking/lock_daemon.h create mode 100644 src/locking/lock_daemon_dispatch.c create mode 100644 src/locking/lock_daemon_dispatch.h create mode 100644 src/locking/virtlockd.init.in create mode 100644 src/locking/virtlockd.sysconf diff --git a/cfg.mk b/cfg.mk index b00cda3..8074b59 100644 --- a/cfg.mk +++ b/cfg.mk @@ -616,7 +616,7 @@ $(srcdir)/src/remote/remote_client_bodies.h: # List all syntax-check exemptions: exclude_file_name_regexp--sc_avoid_strcase = ^tools/virsh\.c$$ -_src1=libvirt|fdstream|qemu/qemu_monitor|util/(command|util)|xen/xend_internal|rpc/virnetsocket +_src1=libvirt|fdstream|qemu/qemu_monitor|util/(command|util)|xen/xend_internal|rpc/virnetsocket|locking/lock_daemon exclude_file_name_regexp--sc_avoid_write = \ ^(src/($(_src1))|daemon/libvirtd|tools/console)\.c$$ @@ -646,7 +646,7 @@ exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF = \ _src2=src/(util/command|libvirt|lxc/lxc_controller) exclude_file_name_regexp--sc_prohibit_fork_wrappers = \ - (^docs|^($(_src2)|tests/testutils|daemon/libvirtd)\.c$$) + (^docs|^($(_src2)|tests/testutils|daemon/libvirtd|src/locking/lock_daemon)\.c$$) exclude_file_name_regexp--sc_prohibit_gethostname = ^src/util/util\.c$$ diff --git a/libvirt.spec.in b/libvirt.spec.in index 650f0bc..7f6fd13 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1007,11 +1007,13 @@ fi %{_sysconfdir}/libvirt/nwfilter/*.xml %{_sysconfdir}/rc.d/init.d/libvirtd +%{_sysconfdir}/rc.d/init.d/virtlockd %if %{with_systemd} %{_unitdir}/libvirtd.service %endif %doc daemon/libvirtd.upstart %config(noreplace) %{_sysconfdir}/sysconfig/libvirtd +%config(noreplace) %{_sysconfdir}/sysconfig/virtlockd %config(noreplace) %{_sysconfdir}/libvirt/libvirtd.conf %if %{with_dtrace} %{_datadir}/systemtap/tapset/libvirtd.stp @@ -1072,6 +1074,11 @@ fi %dir %attr(0755, root, root) %{_localstatedir}/lib/libvirt/dnsmasq/ %endif +%if %{with_libvirtd} +%dir %attr(0755, root, root) %{_libdir}/libvirt/lock-driver +%attr(0755, root, root) %{_libdir}/libvirt/lock-driver/lockd.so +%endif + %if %{with_qemu} %{_datadir}/augeas/lenses/libvirtd_qemu.aug %{_datadir}/augeas/lenses/tests/test_libvirtd_qemu.aug @@ -1105,6 +1112,7 @@ fi %attr(0755, root, root) %{_libexecdir}/libvirt_iohelper %attr(0755, root, root) %{_sbindir}/libvirtd +%attr(0755, root, root) %{_sbindir}/virtlockd %{_mandir}/man8/libvirtd.8* diff --git a/po/POTFILES.in b/po/POTFILES.in index 32eaa2d..bb94d51 100644 --- a/po/POTFILES.in +++ b/po/POTFILES.in @@ -30,8 +30,13 @@ src/fdstream.c src/interface/netcf_driver.c src/internal.h src/libvirt.c +src/locking/lock_database.c +src/locking/lock_driver_fcntl.c +src/locking/lock_driver_lockd.c src/locking/lock_driver_sanlock.c src/locking/lock_manager.c +src/locking/lock_daemon.c +src/locking/lock_daemon_dispatch.c src/lxc/lxc_container.c src/lxc/lxc_conf.c src/lxc/lxc_controller.c diff --git a/src/.gitignore b/src/.gitignore index a619643..c499b47 100644 --- a/src/.gitignore +++ b/src/.gitignore @@ -16,3 +16,7 @@ libvirt_qemu.def *.s remote_protocol-structs-t virt-aa-helper +virtlockd +virtlockd.socket +locking/lock_daemon_dispatch_stubs.h +locking/qemu-sanlock.conf diff --git a/src/Makefile.am b/src/Makefile.am index cd8a7e9..3e2c376 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -96,8 +96,34 @@ DRIVER_SOURCES = \ libvirt.c libvirt_internal.h \ locking/lock_manager.c locking/lock_manager.h \ locking/lock_driver.h \ + locking/domain_lock.h locking/domain_lock.c \ + locking/lock_driver_nop.h locking/lock_driver_nop.c + +LOCK_PROTOCOL_GENERATED = \ + locking/lock_protocol.h \ + locking/lock_protocol.c + +EXTRA_DIST += locking/lock_protocol.x + +LOCKD_GENERATED = \ + $(LOCK_PROTOCOL_GENERATED) \ + locking/lock_daemon_dispatch_stubs.h + +BUILT_SOURCES += $(LOCKD_GENERATED) + +LOCK_DAEMON_SOURCES = \ + locking/lock_daemon.c \ + locking/lock_daemon_dispatch.h \ + locking/lock_daemon_dispatch.c \ + locking/lock_database.c locking/lock_database.h \ + locking/lock_manager.c locking/lock_manager.h \ locking/lock_driver_nop.h locking/lock_driver_nop.c \ - locking/domain_lock.h locking/domain_lock.c + locking/lock_driver_fcntl.h locking/lock_driver_fcntl.c \ + $(LOCKD_GENERATED) + +$(srcdir)/locking/lock_daemon_dispatch_stubs.h: locking/lock_protocol.x $(srcdir)/rpc/gendispatch.pl + $(AM_V_GEN)perl -w $(srcdir)/rpc/gendispatch.pl -b lock $< > $@ + LOCK_DRIVER_SANLOCK_SOURCES = \ locking/lock_driver_sanlock.c @@ -1184,12 +1210,73 @@ libvirt_qemu_la_CFLAGS = $(AM_CFLAGS) libvirt_qemu_la_LIBADD = libvirt.la $(CYGWIN_EXTRA_LIBADD) EXTRA_DIST += $(LIBVIRT_QEMU_SYMBOL_FILE) +if WITH_LIBVIRTD +sbin_PROGRAMS = virtlockd + +virtlockd_SOURCES = $(LOCK_DAEMON_SOURCES) +virtlockd_CFLAGS = \ + $(AM_CFLAGS) +virtlockd_LDFLAGS = \ + $(AM_LDFLAGS) \ + $(CYGWIN_EXTRA_LDFLAGS) \ + $(MINGW_EXTRA_LDFLAGS) +virtlockd_LDADD = \ + libvirt-net-rpc-server.la \ + libvirt-net-rpc.la \ + libvirt_util.la \ + ../gnulib/lib/libgnu.la \ + $(CYGWIN_EXTRA_LIBADD) +endif + +EXTRA_DIST += locking/virtlockd.sysconf + +install-sysconfig: + mkdir -p $(DESTDIR)$(sysconfdir)/sysconfig + $(INSTALL_DATA) $(srcdir)/locking/virtlockd.sysconf \ + $(DESTDIR)$(sysconfdir)/sysconfig/virtlockd + +uninstall-sysconfig: + rm -f $(DESTDIR)$(sysconfdir)/sysconfig/virtlockd + +EXTRA_DIST += locking/virtlockd.init.in + +if WITH_LIBVIRTD +if LIBVIRT_INIT_SCRIPT_RED_HAT +install-init:: virtlockd.init install-sysconfig + mkdir -p $(DESTDIR)$(sysconfdir)/rc.d/init.d + $(INSTALL_SCRIPT) virtlockd.init \ + $(DESTDIR)$(sysconfdir)/rc.d/init.d/virtlockd + +uninstall-init:: uninstall-sysconfig + rm -f $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirtd + +BUILT_SOURCES += virtlockd.init +else +install-init:: +uninstall-init:: +endif +else +install-init:: +uninstall-init:: +endif + +virtlockd.init: locking/virtlockd.init.in $(top_builddir)/config.status + $(AM_V_GEN)sed \ + -e s!\@localstatedir\@!@localstatedir@!g \ + -e s!\@sbindir\@!@sbindir@!g \ + -e s!\@sysconfdir\@!@sysconfdir@!g \ + < $< > $@-t && \ + chmod a+x $@-t && \ + mv $@-t $@ + + + if HAVE_SANLOCK lockdriverdir = $(libdir)/libvirt/lock-driver lockdriver_LTLIBRARIES = sanlock.la sanlock_la_SOURCES = $(LOCK_DRIVER_SANLOCK_SOURCES) -sanlock_la_CFLAGS = $(AM_CLFAGS) +sanlock_la_CFLAGS = $(AM_CFLAGS) sanlock_la_LDFLAGS = -module -avoid-version sanlock_la_LIBADD = -lsanlock \ ../gnulib/lib/libgnu.la @@ -1369,7 +1456,11 @@ endif endif EXTRA_DIST += $(SECURITY_DRIVER_APPARMOR_HELPER_SOURCES) -install-data-local: +install-data-local: install-init +if WITH_LIBVIRTD + $(MKDIR_P) "$(DESTDIR)$(localstatedir)/lib/libvirt/lockd" + $(MKDIR_P) "$(DESTDIR)$(localstatedir)/run/libvirt/lockd" +endif $(MKDIR_P) "$(DESTDIR)$(localstatedir)/cache/libvirt" $(MKDIR_P) "$(DESTDIR)$(localstatedir)/lib/libvirt/images" $(MKDIR_P) "$(DESTDIR)$(localstatedir)/lib/libvirt/boot" @@ -1411,7 +1502,11 @@ if WITH_NETWORK $(DESTDIR)$(sysconfdir)/libvirt/qemu/networks/autostart/default.xml endif -uninstall-local:: +uninstall-local:: uninstall-init +if WITH_LIBVIRTD + rmdir "$(DESTDIR)$(localstatedir)/lib/libvirt/lockd" ||: + rmdir "$(DESTDIR)$(localstatedir)/run/libvirt/lockd" ||: +endif rmdir "$(DESTDIR)$(localstatedir)/cache/libvirt" ||: rmdir "$(DESTDIR)$(localstatedir)/lib/libvirt/images" ||: rmdir "$(DESTDIR)$(localstatedir)/lib/libvirt/boot" ||: @@ -1446,4 +1541,5 @@ endif CLEANFILES = *.gcov .libs/*.gcda .libs/*.gcno *.gcno *.gcda *.i *.s DISTCLEANFILES = $(GENERATED_SYM_FILES) -MAINTAINERCLEANFILES = $(REMOTE_DRIVER_GENERATED) $(VIR_NET_RPC_GENERATED) $(ESX_DRIVER_GENERATED) +MAINTAINERCLEANFILES = $(REMOTE_DRIVER_GENERATED) $(VIR_NET_RPC_GENERATED) \ + $(ESX_DRIVER_GENERATED) $(LOCKD_GENERATED) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 0746dee..ac1d3d2 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -957,6 +957,7 @@ virMutexUnlock; virOnce; virThreadCreate; virThreadID; +virThreadInitialize; virThreadIsSelf; virThreadJoin; virThreadSelf; @@ -1065,6 +1066,7 @@ virAuditSend; # virterror_internal.h virDispatchError; +virErrorInitialize; virErrorMsg; virRaiseErrorFull; virReportErrorHelper; diff --git a/src/locking/lock_daemon.c b/src/locking/lock_daemon.c new file mode 100644 index 0000000..1c20715 --- /dev/null +++ b/src/locking/lock_daemon.c @@ -0,0 +1,776 @@ +/* + * lock_daemon.c: lock management daemon + * + * Copyright (C) 2006-2011 Red Hat, Inc. + * Copyright (C) 2006 Daniel P. Berrange + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Author: Daniel P. Berrange <berrange@redhat.com> + */ + +#include <config.h> + +#include <unistd.h> +#include <fcntl.h> +#include <sys/wait.h> +#include <sys/stat.h> +#include <getopt.h> +#include <stdlib.h> +#include <locale.h> + + +#include "util.h" +#include "files.h" +#include "virterror_internal.h" +#include "logging.h" +#include "memory.h" +#include "conf.h" +#include "rpc/virnetserver.h" +#include "threads.h" +#include "lock_daemon_dispatch.h" +#include "lock_protocol.h" +#include "lock_driver_fcntl.h" +#include "intprops.h" + +#include "configmake.h" + +#define VIR_FROM_THIS VIR_FROM_LOCKING + +#define virLockError(code, ...) \ + virReportErrorHelper(VIR_FROM_THIS, code, __FILE__, \ + __FUNCTION__, __LINE__, __VA_ARGS__) + + +static virNetServerProgramPtr lockProgram = NULL; +static bool privileged; + +enum { + VIR_LOCK_DAEMON_ERR_NONE = 0, + VIR_LOCK_DAEMON_ERR_PIDFILE, + VIR_LOCK_DAEMON_ERR_RUNDIR, + VIR_LOCK_DAEMON_ERR_INIT, + VIR_LOCK_DAEMON_ERR_SIGNAL, + VIR_LOCK_DAEMON_ERR_PRIVS, + VIR_LOCK_DAEMON_ERR_NETWORK, + VIR_LOCK_DAEMON_ERR_CONFIG, + VIR_LOCK_DAEMON_ERR_HOOKS, + + VIR_LOCK_DAEMON_ERR_LAST +}; + +VIR_ENUM_DECL(virDaemonErr) +VIR_ENUM_IMPL(virDaemonErr, VIR_LOCK_DAEMON_ERR_LAST, + "Initialization successful", + "Unable to obtain pidfile", + "Unable to create rundir", + "Unable to initialize libvirt", + "Unable to setup signal handlers", + "Unable to drop privileges", + "Unable to initialize network sockets", + "Unable to load configuration file", + "Unable to look for hook scripts"); + + +static int +virLockDaemonForkIntoBackground(const char *argv0) +{ + int statuspipe[2]; + if (pipe(statuspipe) < 0) + return -1; + + int pid = fork(); + switch (pid) { + case 0: + { + int stdinfd = -1; + int stdoutfd = -1; + int nextpid; + + VIR_FORCE_CLOSE(statuspipe[0]); + + if ((stdinfd = open("/dev/null", O_RDONLY)) < 0) + goto cleanup; + if ((stdoutfd = open("/dev/null", O_WRONLY)) < 0) + goto cleanup; + if (dup2(stdinfd, STDIN_FILENO) != STDIN_FILENO) + goto cleanup; + if (dup2(stdoutfd, STDOUT_FILENO) != STDOUT_FILENO) + goto cleanup; + if (dup2(stdoutfd, STDERR_FILENO) != STDERR_FILENO) + goto cleanup; + if (VIR_CLOSE(stdinfd) < 0) + goto cleanup; + if (VIR_CLOSE(stdoutfd) < 0) + goto cleanup; + + if (setsid() < 0) + goto cleanup; + + nextpid = fork(); + switch (nextpid) { + case 0: + return statuspipe[1]; + case -1: + return -1; + default: + _exit(0); + } + + cleanup: + VIR_FORCE_CLOSE(stdoutfd); + VIR_FORCE_CLOSE(stdinfd); + return -1; + + } + + case -1: + return -1; + + default: + { + int got, exitstatus = 0; + int ret; + char status; + + VIR_FORCE_CLOSE(statuspipe[1]); + + /* We wait to make sure the first child forked successfully */ + if ((got = waitpid(pid, &exitstatus, 0)) < 0 || + got != pid || + exitstatus != 0) { + return -1; + } + + /* Now block until the second child initializes successfully */ + again: + ret = read(statuspipe[0], &status, 1); + if (ret == -1 && errno == EINTR) + goto again; + + if (ret == 1 && status != 0) { + fprintf(stderr, + _("%s: error: %s. Check /var/log/messages or run without " + "--daemon for more info.\n"), argv0, + virDaemonErrTypeToString(status)); + } + _exit(ret == 1 && status == 0 ? 0 : 1); + } + } +} + + +static int +virLockDaemonAcquirePidFile(const char *argv0, const char *pidFile) { + int fd; + char ebuf[1024]; + unsigned long long pid = (unsigned long long)getpid(); + char pidstr[INT_BUFSIZE_BOUND(pid)]; + + if (pidFile[0] == '\0') + return 0; + + while (1) { + struct stat a, b; + if ((fd = open(pidFile, O_WRONLY|O_CREAT, 0644)) < 0) { + VIR_ERROR(_("%s: Failed to open pid file '%s' : %s"), + argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + return -1; + } + + if (fstat(fd, &b) < 0) { + VIR_ERROR(_("%s: Pid file '%s' disappeared: %s"), + argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + VIR_FORCE_CLOSE(fd); + return -1; + } + + if (virFileLock(fd, false, 0, 1) < 0) { + VIR_ERROR(_("%s: Failed to acquire pid file '%s' : %s"), + argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + VIR_FORCE_CLOSE(fd); + return -1; + } + + /* Now make sure the pidfile we locked is the same + * one that now exists on the filesystem + */ + + if (stat(pidFile, &a) < 0) { + VIR_DEBUG("%s: Pid file '%s' disappeared: %s", + argv0, pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + VIR_FORCE_CLOSE(fd); + continue; + } + + if (a.st_ino == b.st_ino) + break; + + VIR_DEBUG("%s: Pid file '%s' was recreated", + argv0, pidFile); + VIR_FORCE_CLOSE(fd); + } + + snprintf(pidstr, sizeof(pidstr), "%llu", pid); + + if (safewrite(fd, pidstr, strlen(pidstr)) < 0) { + VIR_ERROR(_("Failed to write to pid file '%s' : %s"), + pidFile, virStrerror(errno, ebuf, sizeof ebuf)); + } + + return fd; +} + + +static int +virLockDaemonMakePaths(char **basedir, + char **statedir, + char **pidfile, + char **sockfile) +{ + char *userdir = NULL; + uid_t uid = geteuid(); + + *basedir = *statedir = *pidfile = *sockfile = NULL; + + if (privileged) { + if (!(*basedir = strdup(LOCALSTATEDIR "/run/libvirt"))) + goto no_memory; + } else { + if (!(userdir = virGetUserDirectory(uid))) + goto error; + + if (virAsprintf(basedir, "%s/.libvirt", userdir) < 0) + goto no_memory; + } + + if (virAsprintf(statedir, "%s/lockd", *basedir) < 0) + goto no_memory; + if (virAsprintf(pidfile, "%s/lockd.pid", *statedir) < 0) + goto no_memory; + if (virAsprintf(sockfile, "%s/lockd.sock", *statedir) < 0) + goto no_memory; + + VIR_FREE(userdir); + return 0; + +no_memory: + VIR_FREE(*basedir); + VIR_FREE(*statedir); + VIR_FREE(*pidfile); + VIR_FREE(*sockfile); +error: + VIR_FREE(userdir); + return -1; +} + +static void +virLockDaemonErrorHandler(void *opaque ATTRIBUTE_UNUSED, + virErrorPtr err ATTRIBUTE_UNUSED) +{ + /* Don't do anything, since logging infrastructure already + * took care of reporting the error */ +} + + +/* + * Set up the logging environment + * By default if daemonized all errors go to syslog and the logging + * is also saved onto the logfile libvird.log, but if verbose or error + * debugging is asked for then output informations or debug. + */ +static int +virLockDaemonSetLogging(virConfPtr conf ATTRIBUTE_UNUSED, + const char *filename ATTRIBUTE_UNUSED, + int godaemon, int verbose) +{ + //int log_level = 0; + char *log_filters = NULL; + char *log_outputs = NULL; + int ret = -1; + + virLogReset(); +#if 0 + /* + * Libvirtd's order of precedence is: + * cmdline > environment > config + * + * In order to achieve this, we must process configuration in + * different order for the log level versus the filters and + * outputs. Because filters and outputs append, we have to look at + * the environment first and then only check the config file if + * there was no result from the environment. The default output is + * then applied only if there was no setting from either of the + * first two. Because we don't have a way to determine if the log + * level has been set, we must process variables in the opposite + * order, each one overriding the previous. + */ + GET_CONF_INT (conf, filename, log_level); + if (log_level != 0) + virLogSetDefaultPriority(log_level); + + if (virLogGetNbFilters() == 0) { + GET_CONF_STR (conf, filename, log_filters); + virLogParseFilters(log_filters); + } + + if (virLogGetNbOutputs() == 0) { + GET_CONF_STR (conf, filename, log_outputs); + virLogParseOutputs(log_outputs); + } +#endif + + virLogSetFromEnv(); + + /* + * If no defined outputs, then direct to syslog when running + * as daemon. Otherwise the default output is stderr. + */ + if (virLogGetNbOutputs() == 0) { + char *tmp = NULL; + if (godaemon) { + if (virAsprintf (&tmp, "%d:syslog:libvirtd", + virLogGetDefaultPriority()) < 0) + goto free_and_fail; + } else { + if (virAsprintf (&tmp, "%d:stderr", + virLogGetDefaultPriority()) < 0) + goto free_and_fail; + } + virLogParseOutputs(tmp); + VIR_FREE(tmp); + } + + /* + * Command line override for --verbose + */ + if ((verbose) && (virLogGetDefaultPriority() > VIR_LOG_INFO)) + virLogSetDefaultPriority(VIR_LOG_INFO); + + ret = 0; + +free_and_fail: + VIR_FREE(log_filters); + VIR_FREE(log_outputs); + return(ret); +} + +/* Read the config file if it exists. + * Only used in the remote case, hence the name. + */ +static int +virLockDaemonReadConfigFile(const char *filename, + int godaemon, int verbose) +{ + virConfPtr conf; + + if (!(conf = virConfReadFile (filename, 0))) + goto error; + + if (virLockDaemonSetLogging(conf, filename, godaemon, verbose) < 0) + goto error; + + virConfFree (conf); + return 0; + +error: + virConfFree (conf); + + return -1; +} + +/* Display version information. */ +static void + virLockDaemonVersion(const char *argv0) +{ + printf ("%s (%s) %s\n", argv0, PACKAGE_NAME, PACKAGE_VERSION); +} + +static void +virLockDaemonShutdownHandler(virNetServerPtr srv, + siginfo_t *sig ATTRIBUTE_UNUSED, + void *opaque ATTRIBUTE_UNUSED) +{ + virNetServerQuit(srv); +} + +static int +virLockDaemonSetupSignals(virNetServerPtr srv) +{ + if (virNetServerAddSignalHandler(srv, SIGINT, virLockDaemonShutdownHandler, NULL) < 0) + return -1; + if (virNetServerAddSignalHandler(srv, SIGQUIT, virLockDaemonShutdownHandler, NULL) < 0) + return -1; + if (virNetServerAddSignalHandler(srv, SIGTERM, virLockDaemonShutdownHandler, NULL) < 0) + return -1; + return 0; +} + +static int +virLockDaemonSetupNetworking(virNetServerPtr srv, const char *sock_path) +{ + virNetServerServicePtr svc; + + if (!(svc = virNetServerServiceNewUNIX(sock_path, 0700, 0, 0, false, 1, NULL))) + return -1; + + if (virNetServerAddService(srv, svc, NULL) < 0) { + virNetServerServiceFree(svc); + return -1; + } + return 0; +} + + +static void +virLockDaemonClientFreeFunc(void *data) +{ + virLockDaemonClientPrivatePtr priv = data; + + VIR_DEBUG("Client %p dead, freeing manager %p", + priv, priv->manager); + + virMutexLock(&priv->lock); + virLockManagerFree(priv->manager); + virMutexUnlock(&priv->lock); + virMutexDestroy(&priv->lock); + VIR_FREE(priv); +} + +static int +virLockDaemonClientInit(virNetServerPtr srv ATTRIBUTE_UNUSED, + virNetServerClientPtr client) +{ + virLockDaemonClientPrivate *priv; + + if (VIR_ALLOC(priv) < 0) { + virReportOOMError(); + return -1; + } + + if (virMutexInit(&priv->lock) < 0) { + VIR_FREE(priv); + virReportOOMError(); + return -1; + } + + if (virNetServerClientGetLocalIdentity(client, + &priv->uid, + &priv->pid) < 0) + goto error; + + VIR_DEBUG("New client %p pid %llu uid %llu", + priv, (unsigned long long)priv->pid, + (unsigned long long)priv->uid); + + if (!privileged) { + if (geteuid() != priv->uid) { + virLockError(VIR_ERR_OPERATION_DENIED, + _("Disallowing client %llu with uid %llu"), + (unsigned long long)priv->pid, + (unsigned long long)priv->uid); + + goto error; + } + } else { + if (priv->uid != 0) { + virLockError(VIR_ERR_OPERATION_DENIED, + _("Disallowing client %llu with uid %llu"), + (unsigned long long)priv->pid, + (unsigned long long)priv->uid); + goto error; + } + } + + virNetServerClientSetPrivateData(client, priv, virLockDaemonClientFreeFunc); + return 0; + +error: + virMutexDestroy(&priv->lock); + VIR_FREE(priv); + return -1; +} + + +static void +virLockDaemonUsage(const char *argv0) +{ + fprintf (stderr, + _("\n\ +Usage:\n\ + %s [options]\n\ +\n\ +Options:\n\ + -v | --verbose Verbose messages.\n\ + -d | --daemon Run as a daemon & write PID file.\n\ + -t | --timeout <secs> Exit after timeout period.\n\ + -f | --config <file> Configuration file.\n\ + | --version Display version information.\n\ + -p | --pid-file <file> Change name of PID file.\n\ +\n\ +libvirt lock management daemon:\n\ +\n\ + Default paths:\n\ +\n\ + Configuration file (unless overridden by -f):\n\ + %s/libvirt/libvirtd.conf\n\ +\n\ + Sockets (as root):\n\ + %s/run/virtlockd.sock\n\ +\n\ + Sockets (as non-root):\n\ + $HOME/.libvirt/virtlockd.sock (in UNIX abstract namespace)\n\ +\n\ + Default PID file (as root):\ + %s/run/virtlockd.pid\n\ +\n\ + Default PID file (as non-root):\ + $HOME/.libvirt/virtlockd.pid\n\ +\n"), + argv0, + SYSCONFDIR, + LOCALSTATEDIR, + LOCALSTATEDIR); +} + +enum { + OPT_VERSION = 129 +}; + +#define MAX_LISTEN 5 +int main(int argc, char **argv) { + virNetServerPtr srv = NULL; + const char *remote_config_file = NULL; + int statuswrite = -1; + int ret = 1; + int verbose = 0; + int godaemon = 0; + int timeout = 0; + char *base_dir = NULL; + char *state_dir = NULL; + char *pid_file = NULL; + int pid_file_fd = -1; + char *sock_file = NULL; + + struct option opts[] = { + { "verbose", no_argument, &verbose, 1}, + { "daemon", no_argument, &godaemon, 1}, + { "config", required_argument, NULL, 'f'}, + { "timeout", required_argument, NULL, 't'}, + { "pid-file", required_argument, NULL, 'p'}, + { "version", no_argument, NULL, OPT_VERSION }, + { "help", no_argument, NULL, '?' }, + {0, 0, 0, 0} + }; + + privileged = getuid() == 0; + + if (setlocale (LC_ALL, "") == NULL || + bindtextdomain (PACKAGE, LOCALEDIR) == NULL || + textdomain(PACKAGE) == NULL || + virThreadInitialize() < 0 || + virErrorInitialize() < 0 || + virRandomInitialize(time(NULL) ^ getpid())) { + fprintf(stderr, _("%s: initialization failed\n"), argv[0]); + exit(EXIT_FAILURE); + } + + if (virLockDaemonMakePaths(&base_dir, &state_dir, + &pid_file, &sock_file) < 0) + exit(EXIT_FAILURE); + + while (1) { + int optidx = 0; + int c; + char *tmp; + + c = getopt_long(argc, argv, "ldf:p:t:v", opts, &optidx); + + if (c == -1) { + break; + } + + switch (c) { + case 0: + /* Got one of the flags */ + break; + case 'v': + verbose = 1; + break; + case 'd': + godaemon = 1; + break; + + case 't': + if (virStrToLong_i(optarg, &tmp, 10, &timeout) != 0 + || timeout <= 0 + /* Ensure that we can multiply by 1000 without overflowing. */ + || timeout > INT_MAX / 1000) + timeout = -1; + break; + + case 'p': + VIR_FREE(pid_file); + if (!(pid_file = strdup(optarg))) + exit(EXIT_FAILURE); + break; + + case 'f': + remote_config_file = optarg; + break; + + case OPT_VERSION: + virLockDaemonVersion(argv[0]); + return 0; + + case '?': + virLockDaemonUsage(argv[0]); + return 2; + + default: + fprintf (stderr, _("%s: internal error: unknown flag: %c\n"), + argv[0], c); + exit (EXIT_FAILURE); + } + } + + if (remote_config_file == NULL) { + static const char *default_config_file + = SYSCONFDIR "/libvirt/virtlockd.conf"; + remote_config_file = + (access(default_config_file, R_OK) == 0 + ? default_config_file + : "/dev/null"); + } + + if (godaemon) { + char ebuf[1024]; + if ((statuswrite = virLockDaemonForkIntoBackground(argv[0])) < 0) { + VIR_ERROR(_("Failed to fork as daemon: %s"), + virStrerror(errno, ebuf, sizeof ebuf)); + goto cleanup; + } + } + + if (!(srv = virNetServerNew(1, 1, 20, NULL, + virLockDaemonClientInit))) { + ret = VIR_LOCK_DAEMON_ERR_INIT; + goto cleanup; + } + + if ((virLockDaemonSetupSignals(srv)) < 0) { + ret = VIR_LOCK_DAEMON_ERR_SIGNAL; + goto cleanup; + } + + /* Ensure the rundir exists (on tmpfs on some systems) */ + if (mkdir(base_dir, 0755)) { + if (errno != EEXIST) { + char ebuf[1024]; + VIR_ERROR(_("unable to create rundir %s: %s"), base_dir, + virStrerror(errno, ebuf, sizeof(ebuf))); + ret = VIR_LOCK_DAEMON_ERR_RUNDIR; + goto cleanup; + } + } + if (mkdir(state_dir, 0700)) { + if (errno != EEXIST) { + char ebuf[1024]; + VIR_ERROR(_("unable to create rundir %s: %s"), state_dir, + virStrerror(errno, ebuf, sizeof(ebuf))); + ret = VIR_LOCK_DAEMON_ERR_RUNDIR; + goto cleanup; + } + } + + /* If we have a pidfile set, claim it now, exiting if already taken */ + if ((pid_file_fd = virLockDaemonAcquirePidFile(argv[0], pid_file)) < 0) { + ret = VIR_LOCK_DAEMON_ERR_PIDFILE; + goto cleanup; + } + + /* Read the config file (if it exists). */ + if (virLockDaemonReadConfigFile(remote_config_file, godaemon, verbose) < 0) { + ret = VIR_LOCK_DAEMON_ERR_CONFIG; + goto cleanup; + } + + if (virLockDriverFcntl.drvInit(VIR_LOCK_MANAGER_VERSION, NULL, 0) < 0) { + ret = VIR_LOCK_DAEMON_ERR_INIT; + goto cleanup; + } + + if (!(lockProgram = virNetServerProgramNew(LOCK_PROGRAM, + LOCK_PROTOCOL_VERSION, + lockProcs, lockNProcs))) { + ret = VIR_LOCK_DAEMON_ERR_INIT; + goto cleanup; + } + if (virNetServerAddProgram(srv, lockProgram) < 0) { + ret = VIR_LOCK_DAEMON_ERR_INIT; + goto cleanup; + } + + + if (virLockDaemonSetupNetworking(srv, sock_file) < 0) { + ret = VIR_LOCK_DAEMON_ERR_NETWORK; + goto cleanup; + } + + /* Disable error func, now logging is setup */ + virSetErrorFunc(NULL, virLockDaemonErrorHandler); + + + /* Tell parent of daemon that basic initialization is complete + * In particular we're ready to accept net connections & have + * written the pidfile + */ + if (statuswrite != -1) { + char status = 0; + while (write(statuswrite, &status, 1) == -1 && + errno == EINTR) + ; + VIR_FORCE_CLOSE(statuswrite); + } + + /* Start accepting new clients from network */ + + virNetServerUpdateServices(srv, true); + virNetServerRun(srv); + ret = 0; + +cleanup: + virNetServerProgramFree(lockProgram); + virNetServerFree(srv); + if (statuswrite != -1) { + if (ret != 0) { + /* Tell parent of daemon what failed */ + char status = ret; + while (write(statuswrite, &status, 1) == -1 && + errno == EINTR) + ; + } + VIR_FORCE_CLOSE(statuswrite); + } + if (pid_file_fd != -1) { + unlink(pid_file); + VIR_FORCE_CLOSE(pid_file_fd); + } + virLogShutdown(); + VIR_FREE(pid_file); + VIR_FREE(sock_file); + VIR_FREE(state_dir); + return ret; +} diff --git a/src/locking/lock_daemon.h b/src/locking/lock_daemon.h new file mode 100644 index 0000000..e69de29 diff --git a/src/locking/lock_daemon_dispatch.c b/src/locking/lock_daemon_dispatch.c new file mode 100644 index 0000000..1909b9c --- /dev/null +++ b/src/locking/lock_daemon_dispatch.c @@ -0,0 +1,334 @@ +/* + * lock_daemon_dispatch.c: lock management daemon + * + * Copyright (C) 2006-2011 Red Hat, Inc. + * Copyright (C) 2006 Daniel P. Berrange + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Author: Daniel P. Berrange <berrange@redhat.com> + */ + +#include <config.h> + +#include "logging.h" +#include "virterror_internal.h" +#include "rpc/virnetserver.h" +#include "rpc/virnetserverclient.h" +#include "memory.h" +#include "lock_protocol.h" +#include "lock_daemon_dispatch.h" +#include "lock_daemon_dispatch_stubs.h" +#include "lock_driver_fcntl.h" + + +#define VIR_FROM_THIS VIR_FROM_LOCKING + +#define virLockError(code, ...) \ + virReportErrorHelper(VIR_FROM_THIS, code, __FILE__, \ + __FUNCTION__, __LINE__, __VA_ARGS__) + +static virLockManagerParamPtr lockDispatchParameters(size_t nparams, + lock_param *params) +{ + virLockManagerParamPtr ret = NULL; + int i; + + if (VIR_ALLOC_N(ret, nparams) < 0) { + virReportOOMError(); + return NULL; + } + + for (i = 0 ; i < nparams ; i++) { + ret[i].key = params[i].key; + ret[i].type = params[i].value.type; + + switch (ret[i].type) { + case VIR_LOCK_MANAGER_PARAM_TYPE_STRING: + ret[i].value.str = params[i].value.lock_param_value_u.s; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_INT: + ret[i].value.i = params[i].value.lock_param_value_u.i; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_LONG: + ret[i].value.l = params[i].value.lock_param_value_u.l; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_UINT: + ret[i].value.ui = params[i].value.lock_param_value_u.ui; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_ULONG: + ret[i].value.ul = params[i].value.lock_param_value_u.ul; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_DOUBLE: + ret[i].value.d = params[i].value.lock_param_value_u.d; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_UUID: + memcpy(ret[i].value.uuid, params[i].value.lock_param_value_u.u, VIR_UUID_BUFLEN); + break; + } + } + + return ret; +} + +static int lockDispatchRegister(virNetServerPtr server ATTRIBUTE_UNUSED, + virNetServerClientPtr client, + virNetMessageHeaderPtr hdr ATTRIBUTE_UNUSED, + virNetMessageErrorPtr rerr, + lock_register_args *args) +{ + int rv = -1; + virLockDaemonClientPrivate *priv = + virNetServerClientGetPrivateData(client); + virLockManagerParamPtr params = NULL; + size_t nparams; + + virMutexLock(&priv->lock); + + if (priv->manager) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", _("lock manager already open")); + goto cleanup; + } + + if (priv->restrictAccess) { + virLockError(VIR_ERR_OPERATION_DENIED, "%s", + _("lock manager has been restricted")); + goto cleanup; + } + + if (args->params.params_len > LOCK_PARAMETERS_MAX) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Too many parameters %u for lock program, max %u"), + args->params.params_len, LOCK_PARAMETERS_MAX); + goto cleanup; + } + + if (!(params = lockDispatchParameters(args->params.params_len, + args->params.params_val))) + goto cleanup; + + nparams = args->params.params_len; + if (VIR_EXPAND_N(params, nparams, 1) < 0) { + virReportOOMError(); + goto cleanup; + } + + params[nparams-1].type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT; + params[nparams-1].key = "client-pid"; + params[nparams-1].value.i = priv->pid; + + if (!(priv->manager = virLockManagerNew(&virLockDriverFcntl, + args->type, + nparams, + params, + args->flags))) + goto cleanup; + + priv->restrictAccess = args->restrictAccess; + + rv = 0; + +cleanup: + if (rv < 0) + virNetMessageSaveError(rerr); + VIR_FREE(params); + virMutexUnlock(&priv->lock); + return rv; +} + + +static int lockDispatchAddResource(virNetServerPtr server ATTRIBUTE_UNUSED, + virNetServerClientPtr client, + virNetMessageHeaderPtr hdr ATTRIBUTE_UNUSED, + virNetMessageErrorPtr rerr, + lock_add_resource_args *args) +{ + int rv = -1; + virLockDaemonClientPrivate *priv = + virNetServerClientGetPrivateData(client); + virLockManagerParamPtr params = NULL; + + virMutexLock(&priv->lock); + + if (!priv->manager) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", _("lock manager not open")); + goto cleanup; + } + + if (priv->restrictAccess) { + virLockError(VIR_ERR_OPERATION_DENIED, "%s", + _("lock manager has been restricted")); + goto cleanup; + } + + if (args->params.params_len > LOCK_PARAMETERS_MAX) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Too many parameters %u for lock program, max %u"), + args->params.params_len, LOCK_PARAMETERS_MAX); + goto cleanup; + } + + if (!(params = lockDispatchParameters(args->params.params_len, + args->params.params_val))) + goto cleanup; + + if (virLockManagerAddResource(priv->manager, + args->type, + args->name, + args->params.params_len, + params, + args->flags) < 0) + goto cleanup; + + rv = 0; + +cleanup: + if (rv < 0) + virNetMessageSaveError(rerr); + VIR_FREE(params); + virMutexUnlock(&priv->lock); + return rv; +} + + +static int lockDispatchAcquire(virNetServerPtr server ATTRIBUTE_UNUSED, + virNetServerClientPtr client, + virNetMessageHeaderPtr hdr ATTRIBUTE_UNUSED, + virNetMessageErrorPtr rerr, + lock_acquire_args *args) +{ + int rv = -1; + virLockDaemonClientPrivate *priv = + virNetServerClientGetPrivateData(client); + + virMutexLock(&priv->lock); + + if (!priv->manager) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", _("lock manager not open")); + goto cleanup; + } + + if (priv->restrictAccess) { + virLockError(VIR_ERR_OPERATION_DENIED, "%s", + _("lock manager has been restricted")); + goto cleanup; + } + + if (virLockManagerAcquire(priv->manager, + args->state ? *args->state : NULL, + args->flags, + NULL) < 0) + goto cleanup; + + rv = 0; + +cleanup: + if (rv < 0) + virNetMessageSaveError(rerr); + virMutexUnlock(&priv->lock); + return rv; +} + + +static int lockDispatchRelease(virNetServerPtr server ATTRIBUTE_UNUSED, + virNetServerClientPtr client, + virNetMessageHeaderPtr hdr ATTRIBUTE_UNUSED, + virNetMessageErrorPtr rerr, + lock_release_args *args, + lock_release_ret *ret) +{ + int rv = -1; + virLockDaemonClientPrivate *priv = + virNetServerClientGetPrivateData(client); + + virMutexLock(&priv->lock); + + if (!priv->manager) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", _("lock manager not open")); + goto cleanup; + } + + if (priv->restrictAccess) { + virLockError(VIR_ERR_OPERATION_DENIED, "%s", + _("lock manager has been restricted")); + goto cleanup; + } + + if (VIR_ALLOC(ret->state) < 0) { + virReportOOMError(); + goto cleanup; + } + + if (virLockManagerRelease(priv->manager, + ret->state, + args->flags) < 0) + goto cleanup; + + rv = 0; + +cleanup: + if (rv < 0) { + virNetMessageSaveError(rerr); + VIR_FREE(ret->state); + } + virMutexUnlock(&priv->lock); + return rv; +} + + +static int lockDispatchInquire(virNetServerPtr server ATTRIBUTE_UNUSED, + virNetServerClientPtr client, + virNetMessageHeaderPtr hdr ATTRIBUTE_UNUSED, + virNetMessageErrorPtr rerr, + lock_inquire_args *args, + lock_inquire_ret *ret) +{ + int rv = -1; + virLockDaemonClientPrivate *priv = + virNetServerClientGetPrivateData(client); + + virMutexLock(&priv->lock); + + if (!priv->manager) { + virLockError(VIR_ERR_INTERNAL_ERROR, "%s", _("lock manager not open")); + goto cleanup; + } + + if (priv->restrictAccess) { + virLockError(VIR_ERR_OPERATION_DENIED, "%s", + _("lock manager has been restricted")); + goto cleanup; + } + + if (VIR_ALLOC(ret->state) < 0) { + virReportOOMError(); + goto cleanup; + } + + if (virLockManagerInquire(priv->manager, + ret->state, + args->flags) < 0) + goto cleanup; + + rv = 0; + +cleanup: + if (rv < 0) { + virNetMessageSaveError(rerr); + VIR_FREE(ret->state); + } + virMutexUnlock(&priv->lock); + return rv; +} diff --git a/src/locking/lock_daemon_dispatch.h b/src/locking/lock_daemon_dispatch.h new file mode 100644 index 0000000..601e094 --- /dev/null +++ b/src/locking/lock_daemon_dispatch.h @@ -0,0 +1,46 @@ +/* + * lock_daemon_dispatch.h: lock management daemon + * + * Copyright (C) 2006-2011 Red Hat, Inc. + * Copyright (C) 2006 Daniel P. Berrange + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Author: Daniel P. Berrange <berrange@redhat.com> + */ + +#ifndef __VIR_LOCK_DAEMON_DISPATCH_H__ +# define __VIR_LOCK_DAEMON_DISPATCH_H__ + +# include "threads.h" +# include "lock_manager.h" + +extern virNetServerProgramProc lockProcs[]; +extern size_t lockNProcs; + + +typedef struct _virLockDaemonClientPrivate virLockDaemonClientPrivate; +typedef virLockDaemonClientPrivate *virLockDaemonClientPrivatePtr; + +struct _virLockDaemonClientPrivate { + virMutex lock; + virLockManagerPtr manager; + bool restrictAccess; + uid_t uid; + pid_t pid; +}; + + +#endif /* __VIR_LOCK_DAEMON_DISPATCH_H__ */ diff --git a/src/locking/virtlockd.init.in b/src/locking/virtlockd.init.in new file mode 100644 index 0000000..1adea07 --- /dev/null +++ b/src/locking/virtlockd.init.in @@ -0,0 +1,93 @@ +#!/bin/sh + +# the following is the LSB init header see +# http://www.linux-foundation.org/spec//booksets/LSB-Core-generic/LSB-Core-gen... +# +### BEGIN INIT INFO +# Provides: virtlockd +# Default-Start: 3 4 5 +# Short-Description: virtual machine lock manager +# Description: This is a daemon for managing locks +# on virtual machine disk images +### END INIT INFO + +# the following is chkconfig init header +# +# virtlockd: virtual machine lock manager +# +# chkconfig: 345 97 03 +# description: This is a daemon for managing locks \ +# on virtual machine disk images +# +# processname: virtlockd +# pidfile: @localstatedir@/run/libvirt/virtlockd.pid +# + +# Source function library. +. @sysconfdir@/rc.d/init.d/functions + +SERVICE=virtlockd +PROCESS=virtlockd +PIDFILE=@localstatedir@/run/libvirt/lockd/$SERVICE.pid + +VIRTLOCKD_ARGS= + +test -f @sysconfdir@/sysconfig/virtlockd && . @sysconfdir@/sysconfig/virtlockd + +RETVAL=0 + +start() { + echo -n $"Starting $SERVICE daemon: " + daemon --pidfile $PIDFILE --check $SERVICE $PROCESS --daemon $VIRTLOCKD_ARGS + RETVAL=$? + echo + [ $RETVAL -eq 0 ] && touch @localstatedir@/lock/subsys/$SERVICE +} + +stop() { + echo -n $"Stopping $SERVICE daemon: " + + killproc -p $PIDFILE $PROCESS + RETVAL=$? + echo + if [ $RETVAL -eq 0 ]; then + rm -f @localstatedir@/lock/subsys/$SERVICE + rm -f $PIDFILE + fi +} + +restart() { + stop + start +} + +reload() { + echo -n $"Reloading $SERVICE configuration: " + + killproc -p $PIDFILE $PROCESS -HUP + RETVAL=$? + echo + return $RETVAL +} + +# See how we were called. +case "$1" in + start|stop|restart|reload) + $1 + ;; + status) + status -p $PIDFILE $PROCESS + RETVAL=$? + ;; + force-reload) + reload + ;; + condrestart|try-restart) + [ -f @localstatedir@/lock/subsys/$SERVICE ] && restart || : + ;; + *) + echo $"Usage: $0 {start|stop|status|restart|condrestart|reload|force-reload|try-restart}" + exit 2 + ;; +esac +exit $RETVAL diff --git a/src/locking/virtlockd.sysconf b/src/locking/virtlockd.sysconf new file mode 100644 index 0000000..d44dc46 --- /dev/null +++ b/src/locking/virtlockd.sysconf @@ -0,0 +1,3 @@ +# +# Pass extra arguments to virtlockd +#VIRTLOCKD_ARGS= -- 1.7.6
From: "Daniel P. Berrange" <berrange@redhat.com> This adds a 'lockd' lock driver which is just a client which talks to the lockd daemon to perform all locking. This will be the default lock driver for any hypervisor which needs one. * src/Makefile.am: Add lockd.so plugin * src/locking/lock_driver_lockd.c: Lockd driver impl --- src/Makefile.am | 16 +- src/locking/lock_driver_lockd.c | 600 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 613 insertions(+), 3 deletions(-) create mode 100644 src/locking/lock_driver_lockd.c diff --git a/src/Makefile.am b/src/Makefile.am index 3e2c376..9a9deab 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -111,6 +111,10 @@ LOCKD_GENERATED = \ BUILT_SOURCES += $(LOCKD_GENERATED) +LOCK_DRIVER_LOCKD_SOURCES = \ + locking/lock_driver_lockd.c \ + $(LOCK_PROTOCOL_GENERATED) + LOCK_DAEMON_SOURCES = \ locking/lock_daemon.c \ locking/lock_daemon_dispatch.h \ @@ -1211,6 +1215,14 @@ libvirt_qemu_la_LIBADD = libvirt.la $(CYGWIN_EXTRA_LIBADD) EXTRA_DIST += $(LIBVIRT_QEMU_SYMBOL_FILE) if WITH_LIBVIRTD +lockdriverdir = $(libdir)/libvirt/lock-driver +lockdriver_LTLIBRARIES = lockd.la + +lockd_la_SOURCES = $(LOCK_DRIVER_LOCKD_SOURCES) +lockd_la_CFLAGS = $(AM_CFLAGS) +lockd_la_LDFLAGS = -module -avoid-version +lockd_la_LIBADD = ../gnulib/lib/libgnu.la libvirt-net-rpc.la libvirt-net-rpc-client.la + sbin_PROGRAMS = virtlockd virtlockd_SOURCES = $(LOCK_DAEMON_SOURCES) @@ -1272,9 +1284,7 @@ virtlockd.init: locking/virtlockd.init.in $(top_builddir)/config.status if HAVE_SANLOCK -lockdriverdir = $(libdir)/libvirt/lock-driver -lockdriver_LTLIBRARIES = sanlock.la - +lockdriver_LTLIBRARIES += sanlock.la sanlock_la_SOURCES = $(LOCK_DRIVER_SANLOCK_SOURCES) sanlock_la_CFLAGS = $(AM_CFLAGS) sanlock_la_LDFLAGS = -module -avoid-version diff --git a/src/locking/lock_driver_lockd.c b/src/locking/lock_driver_lockd.c new file mode 100644 index 0000000..b7ac0f7 --- /dev/null +++ b/src/locking/lock_driver_lockd.c @@ -0,0 +1,600 @@ +/* + * lock_driver_lockd.c: A lock driver which locks nothing + * + * Copyright (C) 2010-2011 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + */ + +#include <config.h> + +#include "lock_driver.h" +#include "memory.h" +#include "logging.h" +#include "uuid.h" +#include "util.h" +#include "files.h" +#include "virterror_internal.h" +#include "rpc/virnetclient.h" +#include "lock_protocol.h" +#include "configmake.h" + +#define VIR_FROM_THIS VIR_FROM_LOCKING + +#define virLockError(code, ...) \ + virReportErrorHelper(VIR_FROM_THIS, code, __FILE__, \ + __FUNCTION__, __LINE__, __VA_ARGS__) + +typedef struct _virLockManagerLockDPrivate virLockManagerLockDPrivate; +typedef virLockManagerLockDPrivate *virLockManagerLockDPrivatePtr; + +typedef struct _virLockManagerLockDResource virLockManagerLockDResource; +typedef virLockManagerLockDResource *virLockManagerLockDResourcePtr; + +struct _virLockManagerLockDResource { + int type; + char *name; + unsigned int flags; + size_t nparams; + virLockManagerParamPtr params; +}; + +struct _virLockManagerLockDPrivate { + int type; + unsigned int flags; + size_t nparams; + virLockManagerParamPtr params; + size_t nresources; + virLockManagerLockDResourcePtr resources; +}; + + +static virLockManagerParamPtr +virLockManagerLockDCopyParams(size_t nParams, + virLockManagerParamPtr params) +{ + virLockManagerParamPtr newParams; + size_t i, j; + + if (VIR_ALLOC_N(newParams, nParams) < 0) { + virReportOOMError(); + return NULL; + } + + memcpy(newParams, params, sizeof(*params)*nParams); + + for (i = 0 ; i < nParams ; i++) { + if (params[i].type == VIR_LOCK_MANAGER_PARAM_TYPE_STRING) { + if (!(newParams[i].value.str = strdup(params[i].value.str))) { + virReportOOMError(); + goto error; + } + } + } + + return newParams; + +error: + for (j = 0 ; j < i ; j++) { + VIR_FREE(newParams[i].value.str); + } + VIR_FREE(newParams); + return NULL; +} + +static int virLockManagerLockDInit(unsigned int version, + const char *configFile, + unsigned int flags) +{ + VIR_DEBUG("version=%u configFile=%s flags=%u", version, NULLSTR(configFile), flags); + + return 0; +} + +static int virLockManagerLockDDeinit(void) +{ + VIR_DEBUG(" "); + + return 0; +} + + +static void virLockManagerLockDFreeParams(size_t nparams, + virLockManagerParamPtr params) +{ + size_t i; + + if (!params) + return; + + for (i = 0 ; i < nparams ; i++) { + if (params[i].type == VIR_LOCK_MANAGER_PARAM_TYPE_STRING) + VIR_FREE(params[i].value.str); + } + + VIR_FREE(params); +} + + +static void virLockManagerLockDFree(virLockManagerPtr lock) +{ + virLockManagerLockDPrivatePtr priv = lock->privateData; + size_t i; + + if (!priv) + return; + + lock->privateData = NULL; + + for (i = 0 ; i < priv->nresources ; i++) + virLockManagerLockDFreeParams(priv->resources[i].nparams, + priv->resources[i].params); + VIR_FREE(priv->resources); + + virLockManagerLockDFreeParams(priv->nparams, priv->params); + VIR_FREE(priv); +} + + +static lock_param *virLockManagerLockDAddParams(size_t nparams, + virLockManagerParamPtr params) +{ + size_t i; + lock_param *vals = NULL; + + if (VIR_ALLOC_N(vals, nparams) < 0) { + virReportOOMError(); + return NULL; + } + + for (i = 0 ; i < nparams ; i++) { + vals[i].key = (char*)params[i].key; + vals[i].value.type = params[i].type; + switch (params[i].type) { + case VIR_LOCK_MANAGER_PARAM_TYPE_STRING: + vals[i].value.lock_param_value_u.s = params[i].value.str; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_INT: + vals[i].value.lock_param_value_u.i = params[i].value.i; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_LONG: + vals[i].value.lock_param_value_u.l = params[i].value.l; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_UINT: + vals[i].value.lock_param_value_u.ui = params[i].value.ui; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_ULONG: + vals[i].value.lock_param_value_u.ul = params[i].value.ul; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_DOUBLE: + vals[i].value.lock_param_value_u.d = params[i].value.d; + break; + case VIR_LOCK_MANAGER_PARAM_TYPE_UUID: + memcpy(vals[i].value.lock_param_value_u.u, params[i].value.uuid, VIR_UUID_BUFLEN); + break; + } + } + + return vals; +} + + +static char *virLockManagerLockDPath(bool privileged) +{ + char *path; + if (privileged) { + if (!(path = strdup(LOCALSTATEDIR "/run/libvirt/lockd/lockd.sock"))) { + virReportOOMError(); + return NULL; + } + } else { + char *userdir; + if (!(userdir = virGetUserDirectory(geteuid()))) + return NULL; + + if (virAsprintf(&path, "%s/.libvirt/lockd/lockd.sock", userdir) < 0) { + virReportOOMError(); + } + VIR_FREE(userdir); + } + return path; +} + + +static int virLockManagerLockDNew(virLockManagerPtr lock, + unsigned int type, + size_t nparams, + virLockManagerParamPtr params, + unsigned int flags) +{ + virLockManagerLockDPrivatePtr priv; + + if (VIR_ALLOC(priv) < 0) { + virReportOOMError(); + return -1; + } + lock->privateData = priv; + + if (nparams > LOCK_PARAMETERS_MAX) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Too many parameters %zu for lock program, max %d"), + nparams, LOCK_PARAMETERS_MAX); + goto error; + } + + priv->type = type; + priv->flags = flags; + priv->nparams = nparams; + if (!(priv->params = virLockManagerLockDCopyParams(nparams, params))) + goto error; + + return 0; + +error: + virLockManagerLockDFree(lock); + return -1; +} + + +static int virLockManagerLockDAddResource(virLockManagerPtr lock, + unsigned int type, + const char *name, + size_t nparams, + virLockManagerParamPtr params, + unsigned int flags) +{ + virLockManagerLockDPrivatePtr priv = lock->privateData; + lock_add_resource_args args; + char *newName; + virLockManagerParamPtr newParams = NULL; + + memset(&args, 0, sizeof(args)); + + if (nparams > LOCK_PARAMETERS_MAX) { + virLockError(VIR_ERR_INTERNAL_ERROR, + _("Too many parameters %zu for lock program, max %d"), + nparams, LOCK_PARAMETERS_MAX); + goto error; + } + + if (!(newName = strdup(name))) + goto no_memory; + + if (!(newParams = virLockManagerLockDCopyParams(nparams, params))) + goto no_memory; + + if (VIR_EXPAND_N(priv->resources, priv->nresources, 1) < 0) + goto no_memory; + + priv->resources[priv->nresources-1].type = type; + priv->resources[priv->nresources-1].flags = flags; + priv->resources[priv->nresources-1].name = newName; + priv->resources[priv->nresources-1].nparams = nparams; + priv->resources[priv->nresources-1].params = newParams; + + return 0; + +no_memory: + virReportOOMError(); +error: + virLockManagerLockDFreeParams(nparams, newParams); + VIR_FREE(newName); + return -1; +} + + +static virNetClientPtr virLockManagerLockDConnectionNew(bool privileged, + virNetClientProgramPtr *prog) +{ + virNetClientPtr client = NULL; + char *lockdpath; + + *prog = NULL; + + if (!(lockdpath = virLockManagerLockDPath(privileged))) + goto error; + + if (!(client = virNetClientNewUNIX(lockdpath, + false, NULL))) + goto error; + + if (!(*prog = virNetClientProgramNew(LOCK_PROGRAM, + LOCK_PROTOCOL_VERSION, + NULL, + 0, + NULL))) + goto error; + + if (virNetClientAddProgram(client, *prog) < 0) + goto error; + + VIR_FREE(lockdpath); + + return client; + +error: + VIR_FREE(lockdpath); + virNetClientFree(client); + virNetClientProgramFree(*prog); + return NULL; +} + + +static int virLockManagerLockDConnectionRegister(virLockManagerPtr lock, + virNetClientPtr client, + virNetClientProgramPtr program, + int *counter, + bool restrictAccess) +{ + virLockManagerLockDPrivatePtr priv = lock->privateData; + lock_register_args args; + int rv = -1; + + memset(&args, 0, sizeof(args)); + + args.type = priv->type; + args.flags = priv->flags; + args.restrictAccess = restrictAccess; + args.params.params_len = priv->nparams; + if (!(args.params.params_val = virLockManagerLockDAddParams(priv->nparams, priv->params))) + goto cleanup; + + if (virNetClientProgramCall(program, + client, + (*counter)++, + LOCK_PROC_REGISTER, + (xdrproc_t)xdr_lock_register_args, (char*)&args, + (xdrproc_t)xdr_void, NULL) < 0) + goto cleanup; + + rv = 0; + +cleanup: + VIR_FREE(args.params.params_val); + return rv; +} + + +static int virLockManagerLockDConnectionAddResource(virLockManagerPtr lock, + virNetClientPtr client, + virNetClientProgramPtr program, + int *counter, + size_t res) +{ + virLockManagerLockDPrivatePtr priv = lock->privateData; + lock_add_resource_args args; + int rv = -1; + + memset(&args, 0, sizeof(args)); + + args.type = priv->resources[res].type; + args.flags = priv->resources[res].flags; + args.name = priv->resources[res].name; + args.params.params_len = priv->resources[res].nparams; + if (!(args.params.params_val = virLockManagerLockDAddParams(priv->resources[res].nparams, + priv->resources[res].params))) + goto cleanup; + + if (virNetClientProgramCall(program, + client, + (*counter)++, + LOCK_PROC_ADD_RESOURCE, + (xdrproc_t)xdr_lock_add_resource_args, &args, + (xdrproc_t)xdr_void, NULL) < 0) + goto cleanup; + + rv = 0; + +cleanup: + VIR_FREE(args.params.params_val); + return rv; +} + +static int virLockManagerLockDConnectionSetup(virLockManagerPtr lock, + virNetClientPtr client, + virNetClientProgramPtr program, + int *counter, + bool registerOnly, + bool restrictAccess) +{ + virLockManagerLockDPrivatePtr priv = lock->privateData; + size_t i; + + if (virLockManagerLockDConnectionRegister(lock, client, program, + counter, restrictAccess) < 0) + return -1; + + if (registerOnly) + return 0; + + for (i = 0 ; i < priv->nresources ; i++) { + if (virLockManagerLockDConnectionAddResource(lock, client, program, + counter, i) < 0) + return -1; + } + + return 0; +} + +static int virLockManagerLockDAcquire(virLockManagerPtr lock, + const char *state, + unsigned int flags, + int *fd) +{ + virNetClientPtr client = NULL; + virNetClientProgramPtr program = NULL; + int counter = 0; + lock_acquire_args args; + int rv = -1; + + memset(&args, 0, sizeof(args)); + + if (!(client = virLockManagerLockDConnectionNew(getuid() == 0, &program))) + return -1; + + if (fd && + (*fd = virNetClientDupFD(client)) < 0) + goto cleanup; + + if (virLockManagerLockDConnectionSetup(lock, client, program, &counter, + flags & VIR_LOCK_MANAGER_ACQUIRE_REGISTER_ONLY, + flags & VIR_LOCK_MANAGER_ACQUIRE_RESTRICT) < 0) + goto cleanup; + + if (!(flags & VIR_LOCK_MANAGER_ACQUIRE_REGISTER_ONLY)) { + args.flags = flags; + if (state) { + if (VIR_ALLOC(args.state) < 0) { + virReportOOMError(); + goto cleanup; + } + *args.state = (char*)state; + } + + if (virNetClientProgramCall(program, + client, + counter++, + LOCK_PROC_ACQUIRE, + (xdrproc_t)xdr_lock_acquire_args, &args, + (xdrproc_t)xdr_void, NULL) < 0) + goto cleanup; + } + + rv = 0; + +cleanup: + if (rv != 0 && fd) + VIR_FORCE_CLOSE(*fd); + VIR_FREE(args.state); + virNetClientFree(client); + virNetClientProgramFree(program); + + return rv; +} + +static int virLockManagerLockDRelease(virLockManagerPtr lock, + char **state, + unsigned int flags) +{ + virNetClientPtr client = NULL; + virNetClientProgramPtr program = NULL; + int counter = 0; + lock_release_args args; + lock_release_ret ret; + int rv = -1; + + memset(&args, 0, sizeof(args)); + + if (!(client = virLockManagerLockDConnectionNew(getuid() == 0, &program))) + return -1; + + if (virLockManagerLockDConnectionSetup(lock, client, program, + &counter, false, false) < 0) + goto cleanup; + + args.flags = flags; + + if (virNetClientProgramCall(program, + client, + counter++, + LOCK_PROC_RELEASE, + (xdrproc_t)xdr_lock_release_args, &args, + (xdrproc_t)xdr_lock_release_ret, &ret) < 0) + goto cleanup; + + if (ret.state) { + if (state) + *state = *ret.state; + else + VIR_FREE(*ret.state); + VIR_FREE(ret.state); + } + + rv = 0; + +cleanup: + virNetClientFree(client); + virNetClientProgramFree(program); + + return rv; +} + + +static int virLockManagerLockDInquire(virLockManagerPtr lock, + char **state, + unsigned int flags) +{ + virNetClientPtr client = NULL; + virNetClientProgramPtr program = NULL; + int counter = 0; + lock_inquire_args args; + lock_inquire_ret ret; + int rv = -1; + + memset(&args, 0, sizeof(args)); + + if (!(client = virLockManagerLockDConnectionNew(getuid() == 0, &program))) + return -1; + + if (virLockManagerLockDConnectionSetup(lock, client, program, + &counter, false, false) < 0) + goto cleanup; + + args.flags = flags; + + if (virNetClientProgramCall(program, + client, + counter++, + LOCK_PROC_INQUIRE, + (xdrproc_t)xdr_lock_inquire_args, &args, + (xdrproc_t)xdr_lock_inquire_ret, &ret) < 0) + goto cleanup; + + if (ret.state) { + if (state) + *state = *ret.state; + else + VIR_FREE(*ret.state); + VIR_FREE(ret.state); + } + + rv = 0; + +cleanup: + virNetClientFree(client); + virNetClientProgramFree(program); + + return rv; +} + +virLockDriver virLockDriverImpl = +{ + .version = VIR_LOCK_MANAGER_VERSION, + .flags = 0, + + .drvInit = virLockManagerLockDInit, + .drvDeinit = virLockManagerLockDDeinit, + + .drvNew = virLockManagerLockDNew, + .drvFree = virLockManagerLockDFree, + + .drvAddResource = virLockManagerLockDAddResource, + + .drvAcquire = virLockManagerLockDAcquire, + .drvRelease = virLockManagerLockDRelease, + + .drvInquire = virLockManagerLockDInquire, +}; -- 1.7.6
From: "Daniel P. Berrange" <berrange@redhat.com> The current default QEMU lock manager is the 'nop' lock manager, which obviously does not perform any locking. The new virtlockd daemon is able to work out of the box with zero configuration in single-host only mode. Enable this as the default lock manager for all QEMU guests * src/qemu/qemu.conf: Update docs for lock_driver parameter * src/qemu/qemu_conf.c: Change default lock manager to 'lockd' --- src/qemu/qemu.conf | 16 +++++++++++----- src/qemu/qemu_conf.c | 2 +- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index 2c50d9d..a8c1c87 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -281,9 +281,15 @@ # # max_processes = 0 -# To enable strict 'fcntl' based locking of the file -# content (to prevent two VMs writing to the same -# disk), start the 'virtlockd' service, and uncomment -# this # -# lock_manager = "fcntl" +# By default the QEMU driver talks to the 'virtlockd' +# daemon to acquire exclusive locks for all guest disk +# images associated with a running VM. +# +# To revert to behaviour of previous releases which did +# not acquire any locks, set this to 'nop'. +# +# To enable use of the external 'sanlock' locking +# daemon, change this to 'sanlock'. +# +# lock_manager = "lockd" diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 3d8aba4..1baa37e 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -116,7 +116,7 @@ int qemudLoadDriverConfig(struct qemud_driver *driver, #endif if (!(driver->lockManager = - virLockManagerPluginNew("nop", NULL, 0))) + virLockManagerPluginNew("lockd", NULL, 0))) return -1; /* Just check the file is readable before opening it, otherwise -- 1.7.6
From: "Daniel P. Berrange" <berrange@redhat.com> This enhancement virtlockd so that it can receive a pre-opened UNIX domain socket from systemd at launch time, and adds the systemd service/socket unit files * daemon/libvirtd.service.in: Require virtlockd to be running * libvirt.spec.in: Add virtlockd systemd files * src/Makefile.am: Install systemd files * src/locking/lock_daemon.c: Support socket activation * src/locking/virtlockd.service.in, src/locking/virtlockd.socket.in: systemd unit files * src/rpc/virnetserverservice.c, src/rpc/virnetserverservice.h: Add virNetServerServiceNewFD() method * src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add virNetSocketNewListenFD method --- daemon/libvirtd.service.in | 2 + libvirt.spec.in | 6 +++ src/.gitignore | 2 + src/Makefile.am | 51 +++++++++++++++++++++++++- src/locking/lock_daemon.c | 73 ++++++++++++++++++++++++++++++++++++- src/locking/virtlockd.service.in | 16 ++++++++ src/locking/virtlockd.socket.in | 5 +++ src/rpc/virnetserverservice.c | 48 +++++++++++++++++++++++++ src/rpc/virnetserverservice.h | 5 +++ src/rpc/virnetsocket.c | 20 ++++++++++ src/rpc/virnetsocket.h | 3 ++ 11 files changed, 227 insertions(+), 4 deletions(-) create mode 100644 src/locking/virtlockd.service.in create mode 100644 src/locking/virtlockd.socket.in diff --git a/daemon/libvirtd.service.in b/daemon/libvirtd.service.in index 9661428..1cc0ae8 100644 --- a/daemon/libvirtd.service.in +++ b/daemon/libvirtd.service.in @@ -9,7 +9,9 @@ After=syslog.target After=udev.target After=avahi.target After=dbus.target +After=virtlockd.service Before=libvirt-guests.service +Requires=virtlockd.service [Service] EnvironmentFile=-/etc/sysconfig/libvirtd diff --git a/libvirt.spec.in b/libvirt.spec.in index 7f6fd13..ccc3db3 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -892,6 +892,7 @@ done %if %{with_systemd} if [ $1 -eq 1 ] ; then # Initial installation + /bin/systemctl enable virtlockd.service >/dev/null 2>&1 || : /bin/systemctl enable libvirtd.service >/dev/null 2>&1 || : /bin/systemctl enable cgconfig.service >/dev/null 2>&1 || : fi @@ -914,8 +915,10 @@ fi %if %{with_systemd} if [ $1 -eq 0 ] ; then # Package removal, not upgrade + /bin/systemctl --no-reload disable virtlockd.service > /dev/null 2>&1 || : /bin/systemctl --no-reload disable libvirtd.service > /dev/null 2>&1 || : /bin/systemctl stop libvirtd.service > /dev/null 2>&1 || : + /bin/systemctl stop virtlockd.service > /dev/null 2>&1 || : fi %else if [ $1 = 0 ]; then @@ -931,6 +934,7 @@ fi /bin/systemctl daemon-reload >/dev/null 2>&1 || : if [ $1 -ge 1 ] ; then # Package upgrade, not uninstall + /bin/systemctl try-restart virtlockd.service >/dev/null 2>&1 || : /bin/systemctl try-restart libvirtd.service >/dev/null 2>&1 || : fi %endif @@ -1010,6 +1014,8 @@ fi %{_sysconfdir}/rc.d/init.d/virtlockd %if %{with_systemd} %{_unitdir}/libvirtd.service +%{_unitdir}/virtlockd.service +%{_unitdir}/virtlockd.socket %endif %doc daemon/libvirtd.upstart %config(noreplace) %{_sysconfdir}/sysconfig/libvirtd diff --git a/src/.gitignore b/src/.gitignore index c499b47..58729ca 100644 --- a/src/.gitignore +++ b/src/.gitignore @@ -17,6 +17,8 @@ libvirt_qemu.def remote_protocol-structs-t virt-aa-helper virtlockd +virtlockd.service +virtlockd.init virtlockd.socket locking/lock_daemon_dispatch_stubs.h locking/qemu-sanlock.conf diff --git a/src/Makefile.am b/src/Makefile.am index 9a9deab..b14381b 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -1283,6 +1283,53 @@ virtlockd.init: locking/virtlockd.init.in $(top_builddir)/config.status +EXTRA_DIST += locking/virtlockd.service.in locking/virtlockd.socket.in + +if WITH_LIBVIRTD +if LIBVIRT_INIT_SCRIPT_SYSTEMD + +SYSTEMD_UNIT_DIR = /lib/systemd/system + +BUILT_SOURCES += virtlockd.service virtlockd.socket + +install-systemd: virtlockd.init install-sysconfig + mkdir -p $(DESTDIR)$(SYSTEMD_UNIT_DIR) + $(INSTALL_SCRIPT) virtlockd.service \ + $(DESTDIR)$(SYSTEMD_UNIT_DIR)/ + $(INSTALL_SCRIPT) virtlockd.socket \ + $(DESTDIR)$(SYSTEMD_UNIT_DIR)/ + +uninstall-systemd: uninstall-sysconfig + rm -f $(DESTDIR)$(SYSTEMD_UNIT_DIR)/virtlockd.service + rm -f $(DESTDIR)$(SYSTEMD_UNIT_DIR)/virtlockd.socket +else +install-systemd: +uninstall-systemd: +endif +else +install-systemd: +uninstall-systemd: +endif + +virtlockd.service: locking/virtlockd.service.in $(top_builddir)/config.status + $(AM_V_GEN)sed \ + -e s!\@localstatedir\@!@localstatedir@!g \ + -e s!\@sbindir\@!@sbindir@!g \ + -e s!\@sysconfdir\@!@sysconfdir@!g \ + < $< > $@-t && \ + chmod a+x $@-t && \ + mv $@-t $@ + +virtlockd.socket: locking/virtlockd.socket.in $(top_builddir)/config.status + $(AM_V_GEN)sed \ + -e s!\@localstatedir\@!@localstatedir@!g \ + -e s!\@sbindir\@!@sbindir@!g \ + -e s!\@sysconfdir\@!@sysconfdir@!g \ + < $< > $@-t && \ + chmod a+x $@-t && \ + mv $@-t $@ + + if HAVE_SANLOCK lockdriver_LTLIBRARIES += sanlock.la sanlock_la_SOURCES = $(LOCK_DRIVER_SANLOCK_SOURCES) @@ -1466,7 +1513,7 @@ endif endif EXTRA_DIST += $(SECURITY_DRIVER_APPARMOR_HELPER_SOURCES) -install-data-local: install-init +install-data-local: install-init install-systemd if WITH_LIBVIRTD $(MKDIR_P) "$(DESTDIR)$(localstatedir)/lib/libvirt/lockd" $(MKDIR_P) "$(DESTDIR)$(localstatedir)/run/libvirt/lockd" @@ -1512,7 +1559,7 @@ if WITH_NETWORK $(DESTDIR)$(sysconfdir)/libvirt/qemu/networks/autostart/default.xml endif -uninstall-local:: uninstall-init +uninstall-local:: uninstall-init uninstall-systemd if WITH_LIBVIRTD rmdir "$(DESTDIR)$(localstatedir)/lib/libvirt/lockd" ||: rmdir "$(DESTDIR)$(localstatedir)/run/libvirt/lockd" ||: diff --git a/src/locking/lock_daemon.c b/src/locking/lock_daemon.c index 1c20715..797cbe8 100644 --- a/src/locking/lock_daemon.c +++ b/src/locking/lock_daemon.c @@ -418,8 +418,69 @@ virLockDaemonSetupSignals(virNetServerPtr srv) return 0; } + +static int +virLockDaemonSetupNetworkingSystemD(virNetServerPtr srv) +{ + virNetServerServicePtr svc; + const char *pidstr; + const char *fdstr; + unsigned long long pid; + unsigned int nfds; + + if (!(pidstr = getenv("LISTEN_PID"))) { + VIR_DEBUG("No LISTEN_FDS from systemd"); + return 0; + } + + if (virStrToLong_ull(pidstr, NULL, 10, &pid) < 0) { + VIR_DEBUG("Malformed LISTEN_PID from systemd %s", pidstr); + return 0; + } + + if ((pid_t)pid != getpid()) { + VIR_DEBUG("LISTEN_PID %s is not for us %llu", + pidstr, (unsigned long long)getpid()); + return 0; + } + + if (!(fdstr = getenv("LISTEN_FDS"))) { + VIR_DEBUG("No LISTEN_FDS from systemd"); + return 0; + } + + if (virStrToLong_ui(fdstr, NULL, 10, &nfds) < 0) { + VIR_DEBUG("Malformed LISTEN_FDS from systemd %s", fdstr); + return 0; + } + + if (nfds > 1) { + VIR_DEBUG("Too many (%d) file descriptors from systemd", + nfds); + nfds = 1; + } + + unsetenv("LISTEN_PID"); + unsetenv("LISTEN_FDS"); + + if (nfds == 0) + return 0; + + /* Systemd passes FDs, starting immediately after stderr, + * so the first FD we'll get is '3'. */ + if (!(svc = virNetServerServiceNewFD(3, 0, false, 1, NULL))) + return -1; + + if (virNetServerAddService(srv, svc, NULL) < 0) { + virNetServerServiceFree(svc); + return -1; + } + return 1; +} + + static int -virLockDaemonSetupNetworking(virNetServerPtr srv, const char *sock_path) +virLockDaemonSetupNetworkingNative(virNetServerPtr srv, const char *sock_path) { virNetServerServicePtr svc; @@ -563,6 +624,7 @@ int main(int argc, char **argv) { char *pid_file = NULL; int pid_file_fd = -1; char *sock_file = NULL; + int rv; struct option opts[] = { { "verbose", no_argument, &verbose, 1}, @@ -724,7 +786,14 @@ int main(int argc, char **argv) { } - if (virLockDaemonSetupNetworking(srv, sock_file) < 0) { + if ((rv = virLockDaemonSetupNetworkingSystemD(srv)) < 0) { + ret = VIR_LOCK_DAEMON_ERR_NETWORK; + goto cleanup; + } + + /* Only do this, if systemd did not pass a FD */ + if (rv == 0 && + virLockDaemonSetupNetworkingNative(srv, sock_file) < 0) { ret = VIR_LOCK_DAEMON_ERR_NETWORK; goto cleanup; } diff --git a/src/locking/virtlockd.service.in b/src/locking/virtlockd.service.in new file mode 100644 index 0000000..5a74545 --- /dev/null +++ b/src/locking/virtlockd.service.in @@ -0,0 +1,16 @@ +[Unit] +Description=Virtual machine lock manager +Requires=virtlockd.socket +After=syslog.target + +[Service] +EnvironmentFile=-/etc/sysconfig/virtlockd +ExecStart=@sbindir@/virtlockd +ExecReload=/bin/kill -HUP $MAINPID +# Loosing the locks is a really bad thing that will +# cause the machine to be fenced (rebooted), so make +# sure we discourage OOM killer +OOMScoreAdjust=-900 + +[Install] +WantedBy=multi-user.target diff --git a/src/locking/virtlockd.socket.in b/src/locking/virtlockd.socket.in new file mode 100644 index 0000000..5d57aed --- /dev/null +++ b/src/locking/virtlockd.socket.in @@ -0,0 +1,5 @@ +[Unit] +Description=Virtual machine lock manager socket + +[Socket] +ListenStream=/var/run/libvirt/lockd/lockd.sock diff --git a/src/rpc/virnetserverservice.c b/src/rpc/virnetserverservice.c index 8c250e2..560f996 100644 --- a/src/rpc/virnetserverservice.c +++ b/src/rpc/virnetserverservice.c @@ -197,6 +197,54 @@ error: return NULL; } +virNetServerServicePtr virNetServerServiceNewFD(int fd, + int auth, + bool readonly, + size_t nrequests_client_max, + virNetTLSContextPtr tls) +{ + virNetServerServicePtr svc; + int i; + + if (VIR_ALLOC(svc) < 0) + goto no_memory; + + svc->refs = 1; + svc->auth = auth; + svc->readonly = readonly; + svc->nrequests_client_max = nrequests_client_max; + svc->tls = tls; + if (tls) + virNetTLSContextRef(tls); + + svc->nsocks = 1; + if (VIR_ALLOC_N(svc->socks, svc->nsocks) < 0) + goto no_memory; + + if (virNetSocketNewListenFD(fd, + &svc->socks[0]) < 0) + goto error; + + for (i = 0 ; i < svc->nsocks ; i++) { + /* IO callback is initially disabled, until we're ready + * to deal with incoming clients */ + if (virNetSocketAddIOCallback(svc->socks[i], + 0, + virNetServerServiceAccept, + svc) < 0) + goto error; + } + + + return svc; + +no_memory: + virReportOOMError(); +error: + virNetServerServiceFree(svc); + return NULL; +} + int virNetServerServiceGetPort(virNetServerServicePtr svc) { diff --git a/src/rpc/virnetserverservice.h b/src/rpc/virnetserverservice.h index 9357598..fa5d71c 100644 --- a/src/rpc/virnetserverservice.h +++ b/src/rpc/virnetserverservice.h @@ -49,6 +49,11 @@ virNetServerServicePtr virNetServerServiceNewUNIX(const char *path, bool readonly, size_t nrequests_client_max, virNetTLSContextPtr tls); +virNetServerServicePtr virNetServerServiceNewFD(int fd, + int auth, + bool readonly, + size_t nrequests_client_max, + virNetTLSContextPtr tls); int virNetServerServiceGetPort(virNetServerServicePtr svc); diff --git a/src/rpc/virnetsocket.c b/src/rpc/virnetsocket.c index aff68bc..b5d66f5 100644 --- a/src/rpc/virnetsocket.c +++ b/src/rpc/virnetsocket.c @@ -348,6 +348,26 @@ int virNetSocketNewListenUNIX(const char *path ATTRIBUTE_UNUSED, } #endif +int virNetSocketNewListenFD(int fd, + virNetSocketPtr *retsock) +{ + virSocketAddr addr; + *retsock = NULL; + + memset(&addr, 0, sizeof(addr)); + + addr.len = sizeof(addr.data); + if (getsockname(fd, &addr.data.sa, &addr.len) < 0) { + virReportSystemError(errno, "%s", _("Unable to get local socket name")); + return -1; + } + + if (!(*retsock = virNetSocketNew(&addr, NULL, false, fd, -1, 0))) + return -1; + + return 0; +} + int virNetSocketNewConnectTCP(const char *nodename, const char *service, diff --git a/src/rpc/virnetsocket.h b/src/rpc/virnetsocket.h index 8053213..f0e6a37 100644 --- a/src/rpc/virnetsocket.h +++ b/src/rpc/virnetsocket.h @@ -50,6 +50,9 @@ int virNetSocketNewListenUNIX(const char *path, gid_t grp, virNetSocketPtr *addr); +int virNetSocketNewListenFD(int fd, + virNetSocketPtr *addr); + int virNetSocketNewConnectTCP(const char *nodename, const char *service, virNetSocketPtr *addr); -- 1.7.6
On 07/07/2011 04:17 PM, Daniel P. Berrange wrote:
if [ $1 -eq 1 ] ; then # Initial installation + /bin/systemctl enable virtlockd.service>/dev/null 2>&1 || : /bin/systemctl enable libvirtd.service>/dev/null 2>&1 || : /bin/systemctl enable cgconfig.service>/dev/null 2>&1 || : fi @@ -914,8 +915,10 @@ fi %if %{with_systemd} if [ $1 -eq 0 ] ; then # Package removal, not upgrade + /bin/systemctl --no-reload disable virtlockd.service> /dev/null 2>&1 || : /bin/systemctl --no-reload disable libvirtd.service> /dev/null 2>&1 || : /bin/systemctl stop libvirtd.service> /dev/null 2>&1 || : + /bin/systemctl stop virtlockd.service> /dev/null 2>&1 || : fi
Do you really want to enable the .service by default rather than the .socket? IIUC, in the common case of no autostart guests enabling the .socket will cut on boot-time operations. systemd is new to me though, so I may well have misunderstood something... Paolo
On Thu, Jul 07, 2011 at 05:42:01PM +0200, Paolo Bonzini wrote:
On 07/07/2011 04:17 PM, Daniel P. Berrange wrote:
if [ $1 -eq 1 ] ; then # Initial installation + /bin/systemctl enable virtlockd.service>/dev/null 2>&1 || : /bin/systemctl enable libvirtd.service>/dev/null 2>&1 || : /bin/systemctl enable cgconfig.service>/dev/null 2>&1 || : fi @@ -914,8 +915,10 @@ fi %if %{with_systemd} if [ $1 -eq 0 ] ; then # Package removal, not upgrade + /bin/systemctl --no-reload disable virtlockd.service> /dev/null 2>&1 || : /bin/systemctl --no-reload disable libvirtd.service> /dev/null 2>&1 || : /bin/systemctl stop libvirtd.service> /dev/null 2>&1 || : + /bin/systemctl stop virtlockd.service> /dev/null 2>&1 || : fi
Do you really want to enable the .service by default rather than the .socket? IIUC, in the common case of no autostart guests enabling the .socket will cut on boot-time operations.
Yeah, I think you might be right. I'll investigate it further... Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Thu, Jul 07, 2011 at 03:17:18PM +0100, Daniel P. Berrange wrote:
The lock manager infrastructure we recently added to QEMU only has two possible drivers at this time, 'nop' and 'sanlock'. The former does absolutely nothing, while the latter requires a 3rd party package installed and is a little heavy on disk I/O and storage requirements.
This series adds a new daemon 'virtlockd' which is intended to be enabled by default on all hosts running 'libvirtd'. This daemon provides a service for disk locking based on the traditional fcntl() lock primitives. There is a new libvirt manager plugin which talks to this daemon over RPC. The reason for doing the locks in a separate process is that we want the locks to remain active, even if libvirtd crashes, or is restarted. The virtlockd daemon has this one single job so should be pretty reliable and selfcontained. This patch series really benefits from the new RPC APIs, requiring minimal code for the new daemon / client
It seems this nice feature never got in. Are you plannign a respin? Cheers, -- Guido
On Mon, Dec 05, 2011 at 06:54:08PM +0100, Guido Günther wrote:
On Thu, Jul 07, 2011 at 03:17:18PM +0100, Daniel P. Berrange wrote:
The lock manager infrastructure we recently added to QEMU only has two possible drivers at this time, 'nop' and 'sanlock'. The former does absolutely nothing, while the latter requires a 3rd party package installed and is a little heavy on disk I/O and storage requirements.
This series adds a new daemon 'virtlockd' which is intended to be enabled by default on all hosts running 'libvirtd'. This daemon provides a service for disk locking based on the traditional fcntl() lock primitives. There is a new libvirt manager plugin which talks to this daemon over RPC. The reason for doing the locks in a separate process is that we want the locks to remain active, even if libvirtd crashes, or is restarted. The virtlockd daemon has this one single job so should be pretty reliable and selfcontained. This patch series really benefits from the new RPC APIs, requiring minimal code for the new daemon / client
It seems this nice feature never got in. Are you plannign a respin?
Yeah, I did quite a big re-design of the internal code impl, but I have not finished it off yet. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (4)
-
Daniel P. Berrange -
Eric Blake -
Guido Günther -
Paolo Bonzini