On 12/17/19 6:41 PM, Cole Robinson wrote:
On 12/2/19 10:03 AM, Daniel P. Berrangé wrote:
The driver URI scheme:
"$drivername:///embed?root=/some/path"
enables a new way to use the drivers by embedding them directly in the calling process. To use this the process must have a thread running the libvirt event loop. This URI will then cause libvirt to dynamically load the driver module and call its global initialization function. This syntax is applicable to any driver, but only those will have been modified to support a custom root directory and embed URI path will successfully open.
The application can now make normal libvirt API calls which are all serviced in-process with no RPC layer involved.
It is required to specify an explicit root directory, and locks will be acquired on this directory to avoid conflicting with another app that might accidentally pick the same directory.
Use of '/' is not explicitly forbidden, but note that the file layout used underneath the embedded driver root does not match the file layout used by system/session mode drivers. So this cannot be used as a backdoor to interact with, or fake, the system/session mode drivers.
Libvirt will create arbitrary files underneath this root directory. The root directory can be kept untouched across connection open attempts if the application needs persistence. The application is responsible for purging everything underneath this root directory when finally no longer required.
Even when a virt driver is used in embedded mode, it is still possible for it to in turn use functionality that calls out to other secondary drivers in libvirtd. For example an embedded instance of QEMU can open the network, secret or storage drivers in the system libvirtd.
That said, the application would typically want to at least open an embedded secret driver ("secret:///embed?root=/some/path"). Note that multiple different embedded drivers can use the same root prefix and co-operate just as they would inside a normal libvirtd daemon.
A key thing to note is that for this to work, the application that links to libvirt *MUST* be built with -Wl,--export-dynamic to ensure that symbols from libvirt.so are exported & thus available to the dynamically loaded driver module. If libvirt.so itself was dynamically loaded then RTLD_GLOBAL must be passed to dlopen().
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/driver-state.h | 1 + src/driver.h | 2 ++ src/libvirt.c | 72 ++++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 73 insertions(+), 2 deletions(-)
diff --git a/src/driver-state.h b/src/driver-state.h index 1e2f6ed247..6b3f501e05 100644 --- a/src/driver-state.h +++ b/src/driver-state.h @@ -50,6 +50,7 @@ typedef virStateDriver *virStateDriverPtr;
struct _virStateDriver { const char *name; + bool initialized; virDrvStateInitialize stateInitialize; virDrvStateCleanup stateCleanup; virDrvStateReload stateReload; diff --git a/src/driver.h b/src/driver.h index ca82ac974b..6278aa05b3 100644 --- a/src/driver.h +++ b/src/driver.h @@ -82,6 +82,8 @@ struct _virConnectDriver { bool localOnly; /* Whether driver needs a server in the URI */ bool remoteOnly; + /* Whether driver can be used in embedded mode */ + bool embeddable; /* * NULL terminated list of supported URI schemes. * - Single element { NULL } list indicates no supported schemes diff --git a/src/libvirt.c b/src/libvirt.c index bd2952d036..17b6506faa 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -52,6 +52,7 @@ # include "rpc/virnettlscontext.h" #endif #include "vircommand.h" +#include "virevent.h" #include "virfile.h" #include "virrandom.h" #include "viruri.h" @@ -84,6 +85,7 @@ #ifdef WITH_BHYVE # include "bhyve/bhyve_driver.h" #endif +#include "access/viraccessmanager.h"
#define VIR_FROM_THIS VIR_FROM_NONE
@@ -676,10 +678,12 @@ virStateInitialize(bool privileged, return -1;
for (i = 0; i < virStateDriverTabCount; i++) { - if (virStateDriverTab[i]->stateInitialize) { + if (virStateDriverTab[i]->stateInitialize && + !virStateDriverTab[i]->initialized) { virDrvStateInitResult ret; VIR_DEBUG("Running global init for %s state driver", virStateDriverTab[i]->name); + virStateDriverTab[i]->initialized = true; ret = virStateDriverTab[i]->stateInitialize(privileged, root, callback, @@ -872,6 +876,7 @@ virConnectOpenInternal(const char *name, virConnectPtr ret; g_autoptr(virConf) conf = NULL; char *uristr = NULL; + bool embed = false;
ret = virGetConnect(); if (ret == NULL) @@ -962,6 +967,52 @@ virConnectOpenInternal(const char *name, ret->uri) < 0) { goto failed; } + + if (STREQ(ret->uri->path, "/embed")) { + const char *root = NULL; + g_autofree char *regMethod = NULL; + VIR_DEBUG("URI path requests %s driver embedded mode", + ret->uri->scheme); + if (strspn(ret->uri->scheme, "abcdefghijklmnopqrstuvwxyz") != + strlen(ret->uri->scheme)) { + virReportError(VIR_ERR_NO_CONNECT, + _("URI scheme '%s' for embedded driver is not valid"), + ret->uri->scheme); + goto failed; + } + + for (i = 0; i < ret->uri->paramsCount; i++) { + virURIParamPtr var = &ret->uri->params[i]; + if (STREQ(var->name, "root")) + root = var->value; + } + + if (!root) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("root parameter required for embedded driver")); + goto failed; + } + + if (virEventRequireImpl() < 0) + goto failed; + + regMethod = g_strdup_printf("%sRegister", ret->uri->scheme); + + if (virDriverLoadModule(ret->uri->scheme, regMethod, false) < 0) + goto failed; + + if (virAccessManagerGetDefault() == NULL) { + virAccessManagerPtr acl = virAccessManagerNew("none"); + if (!acl) + goto failed; + virAccessManagerSetDefault(acl); + } + + if (virStateInitialize(geteuid() == 0, true, root, NULL, NULL) < 0) + goto failed; + + embed = true; + }
It would be nice if this logic was moved to a separate function
I've hit a couple issues in testing, not sure if/where the fixes will live, so I'll just mention them here. Also the reviewed patches are pushable IMO
I tried this code:
from gi.repository import LibvirtGLib
import libvirt
LibvirtGLib.init(None)
LibvirtGLib.event_register()
conn1 = libvirt.open("qemu:///embed?root=/tmp/foo")
conn2 = libvirt.open("qemu:///embed?root=/tmp/bar")
print(conn1.listAllDomains())
print(conn2.listAllDomains())
I didn't get this far, when I wanted to run virt-install all I can see is the following error: [Tue, 17 Dec 2019 20:22:39 virt-install 129850] ERROR (cli:259) this function is not supported by the connection driver: An event loop implementation must be registered which I tracked to here: <no frame> <no attribute num on current thread> $ r /home/zippy/work/virt-manager.git/virt-install --connect "qemu:///embed?root=/tmp/a" --name f31-uefi --ram 2048 --disk none --boot uefi --import --debug Starting program: /usr/bin/python /home/zippy/work/virt-manager.git/virt-install --connect "qemu:///embed?root=/tmp/a" --name f31-uefi --ram 2048 --disk none --boot uefi --import --debug process 129850 is executing new program: /usr/bin/python3.6m [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". [Tue, 17 Dec 2019 20:21:34 virt-install 129850] DEBUG (cli:203) Launched with command line: /home/zippy/work/virt-manager.git/virt-install --connect qemu:///embed?root=/tmp/a --name f31-uefi --ram 2048 --disk none --boot uefi --import --debug [Tue, 17 Dec 2019 20:21:34 virt-install 129850] DEBUG (virt-install:208) Distilled --network options: ['default'] [Tue, 17 Dec 2019 20:21:34 virt-install 129850] DEBUG (virt-install:140) Distilled --disk options: ['none'] [Tue, 17 Dec 2019 20:21:34 virt-install 129850] DEBUG (cli:219) Requesting libvirt URI qemu:///embed?root=/tmp/a Breakpoint 1, virEventRequireImpl () at ../../src/util/virevent.c:268 268 if (!addHandleImpl || !addTimeoutImpl) { virEventRequireImpl 1 $ bt #0 0x00007ffff693ca53 in virEventRequireImpl () at ../../src/util/virevent.c:268 #1 0x00007ffff6bd2c33 in virConnectOpenInternal (name=0x7ffff710bdd0 "qemu:///embed?root=/tmp/a", auth=0x7fffffffc8a0, flags=0) at ../../src/libvirt.c:996 #2 0x00007ffff6bd38ab in virConnectOpenAuth (name=0x7ffff710bdd0 "qemu:///embed?root=/tmp/a", auth=0x7fffffffc8a0, flags=0) at ../../src/libvirt.c:1272 #3 0x00007ffff6e17b55 in libvirt_virConnectOpenAuth () at /usr/lib64/python3.6/site-packages/libvirtmod.cpython-36m-x86_64-linux-gnu.so But this looks weird, isn't virt-install registerin an event loop? How else does it get events? Michal