[libvirt] [PATCH] RFC: Introduce new domain create APIs to pass pre-opened FDs to LXC
From: "Daniel P. Berrange" <berrange@redhat.com> With container based virt, it is useful to be able to pass pre-opened file descriptors to the container init process. This allows for containers to be auto-activated from incoming socket connections, passing the active socket into the container. To do this, introduce a pair of new APIs, virDomainCreateXMLWithFiles and virDomainCreateWithFiles, which accept an array of file descriptors. For the LXC driver, UNIX file descriptor passing will be used to send them to libvirtd, which will them pass them down to libvirt_lxc, which will then pass them to the container init process. This will only be implemented for LXC right now, but the design is generic enough it could work with other hypervisors, hence I suggest adding this to libvirt.so, rather than libvirt-lxc.so Signed-off-by: Daniel P. Berrange <berrange@redhat.com> --- include/libvirt/libvirt.h.in | 10 +++ src/driver.h | 13 ++++ src/libvirt.c | 150 +++++++++++++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 6 ++ 4 files changed, 179 insertions(+) diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index b87255a..150a231 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -1575,6 +1575,11 @@ virConnectPtr virDomainGetConnect (virDomainPtr domain); virDomainPtr virDomainCreateXML (virConnectPtr conn, const char *xmlDesc, unsigned int flags); +virDomainPtr virDomainCreateXMLWithFiles(virConnectPtr conn, + const char *xmlDesc, + unsigned int nfiles, + int *files, + unsigned int flags); virDomainPtr virDomainLookupByName (virConnectPtr conn, const char *name); virDomainPtr virDomainLookupByID (virConnectPtr conn, @@ -2175,6 +2180,11 @@ int virDomainCreate (virDomainPtr domain); int virDomainCreateWithFlags (virDomainPtr domain, unsigned int flags); +int virDomainCreateWithFiles (virDomainPtr domain, + unsigned int nfiles, + int *files, + unsigned int flags); + int virDomainGetAutostart (virDomainPtr domain, int *autostart); int virDomainSetAutostart (virDomainPtr domain, diff --git a/src/driver.h b/src/driver.h index 31851cb..f4f5873 100644 --- a/src/driver.h +++ b/src/driver.h @@ -136,6 +136,12 @@ typedef virDomainPtr (*virDrvDomainCreateXML)(virConnectPtr conn, const char *xmlDesc, unsigned int flags); +typedef virDomainPtr +(*virDrvDomainCreateXMLWithFiles)(virConnectPtr conn, + const char *xmlDesc, + unsigned int nfiles, + int *files, + unsigned int flags); typedef virDomainPtr (*virDrvDomainLookupByID)(virConnectPtr conn, @@ -334,6 +340,11 @@ typedef int typedef int (*virDrvDomainCreateWithFlags)(virDomainPtr dom, unsigned int flags); +typedef int +(*virDrvDomainCreateWithFiles)(virDomainPtr dom, + unsigned int nfiles, + int *files, + unsigned int flags); typedef virDomainPtr (*virDrvDomainDefineXML)(virConnectPtr conn, @@ -1139,6 +1150,7 @@ struct _virDriver { virDrvConnectNumOfDomains connectNumOfDomains; virDrvConnectListAllDomains connectListAllDomains; virDrvDomainCreateXML domainCreateXML; + virDrvDomainCreateXMLWithFiles domainCreateXMLWithFiles; virDrvDomainLookupByID domainLookupByID; virDrvDomainLookupByUUID domainLookupByUUID; virDrvDomainLookupByName domainLookupByName; @@ -1195,6 +1207,7 @@ struct _virDriver { virDrvConnectNumOfDefinedDomains connectNumOfDefinedDomains; virDrvDomainCreate domainCreate; virDrvDomainCreateWithFlags domainCreateWithFlags; + virDrvDomainCreateWithFiles domainCreateWithFiles; virDrvDomainDefineXML domainDefineXML; virDrvDomainUndefine domainUndefine; virDrvDomainUndefineFlags domainUndefineFlags; diff --git a/src/libvirt.c b/src/libvirt.c index 8e19c64..a673981 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -2005,6 +2005,77 @@ error: } /** + * virDomainCreateXMLWithFiles: + * @conn: pointer to the hypervisor connection + * @xmlDesc: string containing an XML description of the domain + * @nfiles: number of file descriptors passed + * @files: list of file descriptors passed + * @flags: bitwise-OR of supported virDomainCreateFlags + * + * Launch a new guest domain, based on an XML description similar + * to the one returned by virDomainGetXMLDesc() + * This function may require privileged access to the hypervisor. + * The domain is not persistent, so its definition will disappear when it + * is destroyed, or if the host is restarted (see virDomainDefineXML() to + * define persistent domains). + * + * @files provides an array of file descriptors which will be + * made available to the 'init' process of the guest. This is + * only supported for guests which use container based virtualization + * technology. + * + * If the VIR_DOMAIN_START_PAUSED flag is set, the guest domain + * will be started, but its CPUs will remain paused. The CPUs + * can later be manually started using virDomainResume. + * + * If the VIR_DOMAIN_START_AUTODESTROY flag is set, the guest + * domain will be automatically destroyed when the virConnectPtr + * object is finally released. This will also happen if the + * client application crashes / loses its connection to the + * libvirtd daemon. Any domains marked for auto destroy will + * block attempts at migration, save-to-file, or snapshots. + * + * Returns a new domain object or NULL in case of failure + */ +virDomainPtr +virDomainCreateXMLWithFiles(virConnectPtr conn, const char *xmlDesc, + unsigned int nfiles, + int *files, + unsigned int flags) +{ + VIR_DEBUG("conn=%p, xmlDesc=%s, nfiles=%u, files=%p, flags=%x", + conn, xmlDesc, nfiles, files, flags); + + virResetLastError(); + + if (!VIR_IS_CONNECT(conn)) { + virLibConnError(VIR_ERR_INVALID_CONN, __FUNCTION__); + virDispatchError(NULL); + return NULL; + } + virCheckNonNullArgGoto(xmlDesc, error); + if (conn->flags & VIR_CONNECT_RO) { + virLibConnError(VIR_ERR_OPERATION_DENIED, __FUNCTION__); + goto error; + } + + if (conn->driver->domainCreateXMLWithFiles) { + virDomainPtr ret; + ret = conn->driver->domainCreateXMLWithFiles(conn, xmlDesc, + nfiles, files, + flags); + if (!ret) + goto error; + return ret; + } + + virLibConnError(VIR_ERR_NO_SUPPORT, __FUNCTION__); +error: + virDispatchError(conn); + return NULL; +} + +/** * virDomainCreateLinux: * @conn: pointer to the hypervisor connection * @xmlDesc: string containing an XML description of the domain @@ -9362,6 +9433,85 @@ error: } /** + * virDomainCreateWithFiles: + * @domain: pointer to a defined domain + * @nfiles: number of file descriptors passed + * @files: list of file descriptors passed + * @flags: bitwise-OR of supported virDomainCreateFlags + * + * Launch a defined domain. If the call succeeds the domain moves from the + * defined to the running domains pools. + * + * @files provides an array of file descriptors which will be + * made available to the 'init' process of the guest. This is + * only supported for guests which use container based virtualization + * technology. + * + * If the VIR_DOMAIN_START_PAUSED flag is set, or if the guest domain + * has a managed save image that requested paused state (see + * virDomainManagedSave()) the guest domain will be started, but its + * CPUs will remain paused. The CPUs can later be manually started + * using virDomainResume(). In all other cases, the guest domain will + * be running. + * + * If the VIR_DOMAIN_START_AUTODESTROY flag is set, the guest + * domain will be automatically destroyed when the virConnectPtr + * object is finally released. This will also happen if the + * client application crashes / loses its connection to the + * libvirtd daemon. Any domains marked for auto destroy will + * block attempts at migration, save-to-file, or snapshots. + * + * If the VIR_DOMAIN_START_BYPASS_CACHE flag is set, and there is a + * managed save file for this domain (created by virDomainManagedSave()), + * then libvirt will attempt to bypass the file system cache while restoring + * the file, or fail if it cannot do so for the given system; this can allow + * less pressure on file system cache, but also risks slowing loads from NFS. + * + * If the VIR_DOMAIN_START_FORCE_BOOT flag is set, then any managed save + * file for this domain is discarded, and the domain boots from scratch. + * + * Returns 0 in case of success, -1 in case of error + */ +int +virDomainCreateWithFiles(virDomainPtr domain, unsigned int nfiles, + int *files, unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "nfiles=%u, files=%p, flags=%x", + nfiles, files, flags); + + virResetLastError(); + + if (!VIR_IS_CONNECTED_DOMAIN(domain)) { + virLibDomainError(VIR_ERR_INVALID_DOMAIN, __FUNCTION__); + virDispatchError(NULL); + return -1; + } + conn = domain->conn; + if (conn->flags & VIR_CONNECT_RO) { + virLibDomainError(VIR_ERR_OPERATION_DENIED, __FUNCTION__); + goto error; + } + + if (conn->driver->domainCreateWithFiles) { + int ret; + ret = conn->driver->domainCreateWithFiles(domain, + nfiles, files, + flags); + if (ret < 0) + goto error; + return ret; + } + + virLibConnError(VIR_ERR_NO_SUPPORT, __FUNCTION__); + +error: + virDispatchError(domain->conn); + return -1; +} + +/** * virDomainGetAutostart: * @domain: a domain object * @autostart: the value returned diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 7c6edf6..20ac87f 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -627,4 +627,10 @@ LIBVIRT_1.1.0 { virDomainMigrateToURI3; } LIBVIRT_1.0.6; +LIBVIRT_1.1.1 { + global: + virDomainCreateWithFiles; + virDomainCreateXMLWithFiles; +} LIBVIRT_1.1.0; + # .... define new API here using predicted next version number .... -- 1.8.1.4
On Tue, Jul 09, 2013 at 05:15:55PM +0100, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
With container based virt, it is useful to be able to pass pre-opened file descriptors to the container init process. This allows for containers to be auto-activated from incoming socket connections, passing the active socket into the container.
To do this, introduce a pair of new APIs, virDomainCreateXMLWithFiles and virDomainCreateWithFiles, which accept an array of file descriptors. For the LXC driver, UNIX file descriptor passing will be used to send them to libvirtd, which will them pass them down to libvirt_lxc, which will then pass them to the container init process.
This will only be implemented for LXC right now, but the design is generic enough it could work with other hypervisors, hence I suggest adding this to libvirt.so, rather than libvirt-lxc.so
No problem with the principle, but a bit confused by the arguments.
/** + * virDomainCreateXMLWithFiles: + * @conn: pointer to the hypervisor connection + * @xmlDesc: string containing an XML description of the domain + * @nfiles: number of file descriptors passed + * @files: list of file descriptors passed + * @flags: bitwise-OR of supported virDomainCreateFlags + * + * Launch a new guest domain, based on an XML description similar + * to the one returned by virDomainGetXMLDesc() + * This function may require privileged access to the hypervisor. + * The domain is not persistent, so its definition will disappear when it + * is destroyed, or if the host is restarted (see virDomainDefineXML() to + * define persistent domains). + * + * @files provides an array of file descriptors which will be + * made available to the 'init' process of the guest. This is + * only supported for guests which use container based virtualization + * technology.
Hum, say you want to pas 4 fd, one for 0 stdin, one for 1 stdout one for 2 stderr, and somehow one for 4 a preopened fd, what would the files and nfiles contain ? 5 , [stdin, stdou, stderr, -1, fd4] ? basically is -1 used for holes in the list, shouldn't we instead pair (index for fd, fd) and provide an array of them ? Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/
On Wed, Jul 10, 2013 at 01:28:41PM +0800, Daniel Veillard wrote:
On Tue, Jul 09, 2013 at 05:15:55PM +0100, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
With container based virt, it is useful to be able to pass pre-opened file descriptors to the container init process. This allows for containers to be auto-activated from incoming socket connections, passing the active socket into the container.
To do this, introduce a pair of new APIs, virDomainCreateXMLWithFiles and virDomainCreateWithFiles, which accept an array of file descriptors. For the LXC driver, UNIX file descriptor passing will be used to send them to libvirtd, which will them pass them down to libvirt_lxc, which will then pass them to the container init process.
This will only be implemented for LXC right now, but the design is generic enough it could work with other hypervisors, hence I suggest adding this to libvirt.so, rather than libvirt-lxc.so
No problem with the principle, but a bit confused by the arguments.
/** + * virDomainCreateXMLWithFiles: + * @conn: pointer to the hypervisor connection + * @xmlDesc: string containing an XML description of the domain + * @nfiles: number of file descriptors passed + * @files: list of file descriptors passed + * @flags: bitwise-OR of supported virDomainCreateFlags + * + * Launch a new guest domain, based on an XML description similar + * to the one returned by virDomainGetXMLDesc() + * This function may require privileged access to the hypervisor. + * The domain is not persistent, so its definition will disappear when it + * is destroyed, or if the host is restarted (see virDomainDefineXML() to + * define persistent domains). + * + * @files provides an array of file descriptors which will be + * made available to the 'init' process of the guest. This is + * only supported for guests which use container based virtualization + * technology.
Hum, say you want to pas 4 fd, one for 0 stdin, one for 1 stdout one for 2 stderr, and somehow one for 4 a preopened fd, what would the files and nfiles contain ?
5 , [stdin, stdou, stderr, -1, fd4] ?
basically is -1 used for holes in the list, shouldn't we instead pair (index for fd, fd) and provide an array of them ?
The container already has stdin/out/err preset by libvirt and won't be changed here. This is about passing in additional FDs beyond that, so fd >= 3. I'm not intending to support arbitrary mapping of FD numbers, these are just exposed to the guest linearly from FD==3 onwards. So if you pass an array containing 5 FDs, they'll get exposed to the guest as FD numbers 3, 4, 5, 6, 7. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Wed, Jul 10, 2013 at 09:31:52AM +0100, Daniel P. Berrange wrote:
On Wed, Jul 10, 2013 at 01:28:41PM +0800, Daniel Veillard wrote:
On Tue, Jul 09, 2013 at 05:15:55PM +0100, Daniel P. Berrange wrote:
From: "Daniel P. Berrange" <berrange@redhat.com>
With container based virt, it is useful to be able to pass pre-opened file descriptors to the container init process. This allows for containers to be auto-activated from incoming socket connections, passing the active socket into the container.
To do this, introduce a pair of new APIs, virDomainCreateXMLWithFiles and virDomainCreateWithFiles, which accept an array of file descriptors. For the LXC driver, UNIX file descriptor passing will be used to send them to libvirtd, which will them pass them down to libvirt_lxc, which will then pass them to the container init process.
This will only be implemented for LXC right now, but the design is generic enough it could work with other hypervisors, hence I suggest adding this to libvirt.so, rather than libvirt-lxc.so
No problem with the principle, but a bit confused by the arguments.
/** + * virDomainCreateXMLWithFiles: + * @conn: pointer to the hypervisor connection + * @xmlDesc: string containing an XML description of the domain + * @nfiles: number of file descriptors passed + * @files: list of file descriptors passed + * @flags: bitwise-OR of supported virDomainCreateFlags + * + * Launch a new guest domain, based on an XML description similar + * to the one returned by virDomainGetXMLDesc() + * This function may require privileged access to the hypervisor. + * The domain is not persistent, so its definition will disappear when it + * is destroyed, or if the host is restarted (see virDomainDefineXML() to + * define persistent domains). + * + * @files provides an array of file descriptors which will be + * made available to the 'init' process of the guest. This is + * only supported for guests which use container based virtualization + * technology.
Hum, say you want to pas 4 fd, one for 0 stdin, one for 1 stdout one for 2 stderr, and somehow one for 4 a preopened fd, what would the files and nfiles contain ?
5 , [stdin, stdou, stderr, -1, fd4] ?
basically is -1 used for holes in the list, shouldn't we instead pair (index for fd, fd) and provide an array of them ?
The container already has stdin/out/err preset by libvirt and won't be changed here. This is about passing in additional FDs beyond that, so fd >= 3. I'm not intending to support arbitrary mapping of FD numbers, these are just exposed to the guest linearly from FD==3 onwards. So if you pass an array containing 5 FDs, they'll get exposed to the guest as FD numbers 3, 4, 5, 6, 7.
Okay, let's just make that clear :-) Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/
participants (2)
-
Daniel P. Berrange -
Daniel Veillard