On Thu, Aug 21, 2008 at 10:42:51AM +0100, Daniel P. Berrange wrote:
Well, ok so let me step back a bit because there's actually several layers to this issue...
- The libvirtd is single threaded. This was reasonable at first, but some of our APIs take a long time to complete, so we need to have libvirtd parallelized.
- The QEMU/LXC drivers have state stored in the daemon, so even if you merely allow libvirtd to parallelize by putting long running connections in the 'background' you ned to have locking in the QEMU/LXC drivers.
- Given that we need to have locking anyway, might as well just make the libvirtd daemon properly multi-threaded, rather than trying to have a multi-request queuing system.
- Once you've done all that, then solving the virError thread-local issue is all that remains to allow virConnect to be accessed by multiple threads
We need to do 1->3 in order to make libvirtd more scalable. At which point 4 becomes 'not very hard' :-)
That said, there is one other option I've thought about instead of doing step 4, we could add a 'virConnectDup' method. This would take an existing connection object, and create an independant copy of it that another thread could use. The key would be that the internal driver would automagically pass authentication credentials somehow, so you wouldn't have to re-authenticate the 2nd connection. For the remote driver, this could be as simple as invoking an RPC call to fetch a one-time key, and passing that to the 2nd connection to use as their authentication credential.
Actually the one-time key idea wouldn't work because that wouldn't let us negotiate the session encryption, which is done by SASL at the time we authenticate the client. Rich suggested perhaps sharing a single TCP connection and multiplexing messages on it - we have a serial number with every message/reply so this is doable in theory. It'd require some fun code to hand-off replies between the virConnect objects sharing the connection, but might be more viable in general. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|