Re: [libvirt] Using virtEvents
On Thu, Dec 29, 2011 at 12:03:06PM +0100, D. Herrendoerfer wrote:
Sorry to come back on this.
How are the different threads of the libvirt daemon separated ?
I understand that there are 2 separate threads running that are
forked. There are also 4 more that appear to be threads.
Is that correct ?
There are quite a few threads in libvirtd, so lets go from
the beginning.
When libvirtd starts up, it is obviously single threaded.
One of the first things we do is to create the server state
object - virNetServerPtr. This object is what processes for
incoming client connections, creating virNetServerClientPtr
instances. The virNetServerPtr is also responsible for
dispatching RPC messages. To allow maximum concurrency,
the virNetServerPtr creates two pools of worker threads
for handling RPC. One pool is the so called "high priority"
threads pool, which handles RPC messages which are very
fast and never block. The other thread pool handles all
other RPC messages, including those which may take a long
time. Each of these pools has an initial 5 threads, so we
have 10 threads in total for RPC workers, but this number
may grow over time - the thread limits are controlled
in libvirtd.conf
During initialization, we need to run the global driver
init functions (eg qemudStartup and friends). This is done
by spawning a new thread, which invokes virStateInit(),
waits for it to complete and then exits.
The initial process leader thread, invokes virNetServerRun()
which runs the main event loop.
So most of the time there the 10+ worker threads, plus the
event loop thread.
At times other threads may be invoked, eg during tunnelled
migration the QEMU driver spawns an extra thread to handle
I/O. The NWfilter code spawns other threads to handle
certain operations. Also core dumps on crashing QEMU will
spawn threads.
Now, the virEvents main polling loop is running in a forked thread,
it has no access to worker threads that where created through
virsh for example
virsh is a completely separate process from libvirtd.
Last, if a VM needs something to watch over the state of its
network devices, a separate thread needs to be created to do
that because the VMs libvirt daemon thread can't make use
of the virEvent functions without creating a new poll-loop thread ?
Any event loop callback must be sure to complete in very
fast time, since all I/O processsing is blocked while an
event callback is running. So typically callbacks would
restrict themselvs to reading/writing I/O on the triggering
file descriptor, or some other small non-blocking piece of
worker.
If your event loop callback intends to invoke complex
APis, which may in turn need to do I/O, then you must
spawn a new thread so that you don't block the whole
event loop.
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|