Re: [libvirt] [PATCH 4/7] libvirt: support an "embed" URI path selector for opening drivers

On 12/2/19 10:03 AM, Daniel P. Berrangé wrote: > The driver URI scheme: > > "$drivername:///embed?root=/some/path" > > enables a new way to use the drivers by embedding them directly in the > calling process. To use this the process must have a thread running the > libvirt event loop. This URI will then cause libvirt to dynamically load > the driver module and call its global initialization function. This > syntax is applicable to any driver, but only those will have been > modified to support a custom root directory and embed URI path will > successfully open. > > The application can now make normal libvirt API calls which are all > serviced in-process with no RPC layer involved. > > It is required to specify an explicit root directory, and locks will be > acquired on this directory to avoid conflicting with another app that > might accidentally pick the same directory. > > Use of '/' is not explicitly forbidden, but note that the file layout > used underneath the embedded driver root does not match the file > layout used by system/session mode drivers. So this cannot be used as > a backdoor to interact with, or fake, the system/session mode drivers. > > Libvirt will create arbitrary files underneath this root directory. The > root directory can be kept untouched across connection open attempts if > the application needs persistence. The application is responsible for > purging everything underneath this root directory when finally no longer > required. > > Even when a virt driver is used in embedded mode, it is still possible > for it to in turn use functionality that calls out to other secondary > drivers in libvirtd. For example an embedded instance of QEMU can open > the network, secret or storage drivers in the system libvirtd. > > That said, the application would typically want to at least open an > embedded secret driver ("secret:///embed?root=/some/path"). Note that > multiple different embedded drivers can use the same root prefix and > co-operate just as they would inside a normal libvirtd daemon. > > A key thing to note is that for this to work, the application that links > to libvirt *MUST* be built with -Wl,--export-dynamic to ensure that > symbols from libvirt.so are exported & thus available to the dynamically > loaded driver module. If libvirt.so itself was dynamically loaded then > RTLD_GLOBAL must be passed to dlopen(). > > Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; > --- > src/driver-state.h | 1 + > src/driver.h | 2 ++ > src/libvirt.c | 72 ++++++++++++++++++++++++++++++++++++++++++++-- > 3 files changed, 73 insertions(+), 2 deletions(-)

It would be nice if this logic was moved to a separate function I've hit a couple issues in testing, not sure if/where the fixes will live, so I'll just mention them here. Also the reviewed patches are pushable IMO I tried this code: from gi.repository import LibvirtGLib import libvirt LibvirtGLib.init(None) LibvirtGLib.event_register() conn1 = libvirt.open("qemu:///embed?root=/tmp/foo") conn2 = libvirt.open("qemu:///embed?root=/tmp/bar") print(conn1.listAllDomains()) print(conn2.listAllDomains()) With /tmp/foo populated with a VM: both connections see the same values. So this should be rejected. Even trying to close conn1 fully and open a new embed root will only see the old root. So maybe this needs knowledge in the driver lookup.

The second issue: testing with virt-manager, everything locks up with OpenGraphicsFD: #0 0x00007ffff7a7f07a in pthread_cond_timedwait@(a)GLIBC_2.3.2 () at /lib64/libpthread.so.0 #1 0x00007fffe94be113 in virCondWaitUntil (c=c@entry=0x7fffc8071f98, m=m@entry=0x7fffc8071ed0, whenms=whenms@entry=1576602286073) at /home/crobinso/src/libvirt/src/util/virthread.c:159 #2 0x00007fffe44fc549 in qemuDomainObjBeginJobInternal (driver=driver@entry=0x7fffc8004f30, obj=0x7fffc8071ec0, job=job@entry=QEMU_JOB_MODIFY, agentJob=agentJob@entry=QEMU_AGENT_JOB_NONE, asyncJob=asyncJob@entry=QEMU_ASYNC_JOB_NONE, nowait=nowait@entry=false) at /home/crobinso/src/libvirt/src/qemu/qemu_domain.c:9357 #3 0x00007fffe4500aa1 in qemuDomainObjBeginJob (driver=driver@entry=0x7fffc8004f30, obj=<optimized out>, job=job@entry=QEMU_JOB_MODIFY) at /home/crobinso/src/libvirt/src/qemu/qemu_domain.c:9521 #4 0x00007fffe4582572 in qemuDomainOpenGraphicsFD (dom=<optimized out>, idx=<optimized out>, flags=0) at /home/crobinso/src/libvirt/src/qemu/qemu_driver.c:18990 #5 0x00007fffe968699c in virDomainOpenGraphicsFD (dom=0x7fffd0005830, idx=0, flags=0) at /home/crobinso/src/libvirt/src/libvirt-domain.c:10664 #6 0x00007fffe98cc6b1 in libvirt_virDomainOpenGraphicsFD () at /usr/lib64/python3.7/site-packages/libvirtmod.cpython-37m-x86_64-linux-gnu.so I didn't dig into it any more than that. Otherwise in some mild testing I'm surprised how much things seemed to 'just work' :)