11:45 a.m.
Hi -
We have some folks looking into the implementation of events (just VM
state transition events, for now) in libvirtd. I've been assuming that
events will be XML strings, something like:
<event type="domain-state-transition" timestamp="xxxxx">
<domain-id>22</domain-id>
<old-state>nostate</old-state>
<new-state>running</new-state>
</event>
where the contents of the <event> element are determined by the event
type, and the attributes type and timestamp are required.
Originally, I was thinking one would listen for events by registering
a callback, something like:
int virConnectAddEventHandler(virConnPtr conn, char **filter,
virEventHandler handler, void *arg)
where filter is either NULL, indicating interest in all events, or else
a NULL-terminated vector of event type names, allowing filtering by
event type. void handler(conn, const char *eventXML, arg) would be
called for each matching event.
I'm a little concerned that a vector of event type names isn't really
adequate for specifying a filter. Does this need to be more general
(XPathString exprs??)
But my larger concern is that an asynchronous callback mechanism (as
proposed above) assumes the presence of some thread / process from which
to make the callbacks. This works fine in the libvirtd context, but
not outside of it. For instance, we build a "client only" version of
libvirt with ONLY the remote driver, which currently doesn't require
pthreads at all. Introducing asynchronous callbacks into the API means
pthreads is now required for this.
I'm not sure how much requiring this extra thread matters. If it does,
we could always define a synchronous delivery mechanism instead. For
instance, we could have a virDeliverEvents(conn) call to make the
callbacks for any outstanding events. Or we could just dispense with
callbacks altogether, and return a readable (pipe) fd from which the
client can read events.
Comments??
Dave
P.S. We'll also need an internal API for sending events (most likely, an
extension of the existing virEvent stuff), but let's get the basic shape
of the external interface agreed upon first.
4:54 a.m.
On Thu, Sep 18, 2008 at 12:45:07PM -0400, David Lively wrote:
Hi -
We have some folks looking into the implementation of events (just VM
state transition events, for now) in libvirtd. I've been assuming that
events will be XML strings, something like:
<event type="domain-state-transition" timestamp="xxxxx">
<domain-id>22</domain-id>
<old-state>nostate</old-state>
<new-state>running</new-state>
</event>
where the contents of the <event> element are determined by the event
type, and the attributes type and timestamp are required.
Originally, I was thinking one would listen for events by registering
a callback, something like:
int virConnectAddEventHandler(virConnPtr conn, char **filter,
virEventHandler handler, void *arg)
where filter is either NULL, indicating interest in all events, or else
a NULL-terminated vector of event type names, allowing filtering by
event type. void handler(conn, const char *eventXML, arg) would be
called for each matching event.
I think I'd prefer to have properly typed event callbacks for each
type of event we're dealing with, and to register callbacks against
the object being tracked. We also don't want to do transitions
on all the states - eg transitions between nostate & running are
happening many times a second - its not useful to track that IMHO.
Against a virConnectPtr object I'd expect to be able to register
to get an event upon
- A new domain object coming into existance
- A existing domain object going out of existance
So, you could register a callback, call Rich's virConnectListAllDomains()
once, and then rely on the callbacks from that point onwards to keep
your list of domains up2date. So in case of listening for domains:
enum {
VIR_CONNECT_DOMAIN_EVENT_ADDED,
VIR_CONNECT_DOMAIN_EVENT_REMOVED,
};
typedef int (*virConnectDomainEventCallback)(virConnectPtr conn,
virDomainPtr dom,
int event,
void *opaque);
int virCOnnectDomainEventRegister(virConnectPtr conn,
virConnectDomainEventCallback cb,
void *opaque);
There would eventually be equivalent API for virNetworkPtr objects
and virStoragePoolPtr objects, to track addition & removal of them.
Against a virDomainPtr object, I'd expect to be able to register
to get an event upon the significant state transitions. If we
exclude the transition between nonstate & running which just happens
far to often to be practical to track, I don't see a need to filter
the events further - just have the callback get all events against
that domain object.
enum {
VIR_DOMAIN_EVENT_STARTED
VIR_DOMAIN_EVENT_SUSPENDED
VIR_DOMAIN_EVENT_RESUMED
VIR_DOMAIN_EVENT_STOPPED
VIR_DOMAIN_EVENT_SAVED,
VIR_DOMAIN_EVENT_RESTORED,
};
typedef int (*virDomainLifecycleEventCallback)(virDomainPtr dom,
int event,
void *opaque);
virDomainLifecycleEventRegister(virDomainPtr dom,
virDomainLifecycleEventCallback cb,
void *opaque);
I'm a little concerned that a vector of event type names
isn't really
adequate for specifying a filter. Does this need to be more general
(XPathString exprs??)
IMHO, XML / xpath is rather overkill for getting lifecycle events.
But my larger concern is that an asynchronous callback mechanism
(as
proposed above) assumes the presence of some thread / process from which
to make the callbacks. ???This works fine in the libvirtd context, but
not outside of it. For instance, we build a "client only" version of
libvirt with ONLY the remote driver, which currently doesn't require
pthreads at all. Introducing asynchronous callbacks into the API means
pthreads is now required for this.
I wouldn't want to use threads for this - any application which is
structured in such a way as to be able to make use of async events
will have some kind of event loop implementation. We merely need to
provoide a way to hook libvirt into that event loop. We already have
the API defined for this - src/event.h, and have an demonstration
impl that the daemon uses qemud/event.c. So we'd want to validate
this src/event.h API contract by doing a proof-of concept impl with
an external Glib event loop and if it proves sane, then make the
event.h file part of the public API.
Applications could either craft their event loop impl themselves, or
we can provide some add-on pre-built helper libraries with common
impls. eg, an optional libvirt-glib library which comes with a
pre-built event impl for applications which use glib. Likewise a
libvirt-qt helper. That'd cover pretty much all common GUI apps.
This approach has been used very successfully by DBus to avoid tieing
dbus to a specific event loop impl.
I'm not sure how much requiring this extra thread matters. If
it does,
we could always define a synchronous delivery mechanism instead. For
instance, we could have a virDeliverEvents(conn) call to make the
callbacks for any outstanding events. Or we could just dispense with
callbacks altogether, and return a readable (pipe) fd from which the
client can read events.
I don't like the idea of exposing an FD as I consider this to be
internal impl details. Allowing apps to see it will constrain our
abilty to change internals. So I prefer to have the applicaiton
provide an event loop impl, which we can call into to register FDs
or timeout as & when libvirt deems neccessary.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
6:14 a.m.
Daniel P. Berrange wrote:
Applications could either craft their event loop impl themselves, or
we can provide some add-on pre-built helper libraries with common
impls. eg, an optional libvirt-glib library which comes with a
pre-built event impl for applications which use glib. Likewise a
libvirt-qt helper. That'd cover pretty much all common GUI apps.
Sounds sane. avahi does the same btw, and I like that approach.
cheers,
Gerd
6:19 a.m.
On Fri, Sep 19, 2008 at 01:14:14PM +0200, Gerd Hoffmann wrote:
Daniel P. Berrange wrote:
> Applications could either craft their event loop impl themselves, or
> we can provide some add-on pre-built helper libraries with common
> impls. eg, an optional libvirt-glib library which comes with a
> pre-built event impl for applications which use glib. Likewise a
> libvirt-qt helper. That'd cover pretty much all common GUI apps.
Sounds sane. avahi does the same btw, and I like that approach.
Avahi's interface is a thin wrapper arounds dbus's interface :-) And
indeed the libvirtd's event.c impl fits into Avahi's interface nicely
which demonstrates we're consistent in our thinking.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
3:54 p.m.
On Fri, Sep 19, 2008 at 10:54:39AM +0100, Daniel P. Berrange wrote:
Against a virConnectPtr object I'd expect to be able to register
to get an event upon
- A new domain object coming into existance
- A existing domain object going out of existance
So, you could register a callback, call Rich's virConnectListAllDomains()
once, and then rely on the callbacks from that point onwards to keep
your list of domains up2date. So in case of listening for domains:
Just a remark but unfortunately that scheme forces a race between
the start of the event flow and the return of the list. The way used in
the file monitoring API (FAM which I dislike but at least fixed that problem)
is that when you register you get a flow of initial events allowing
to setup your list of object. Certainly less efficient than single
synchronous call but avoid the race. The user code is also simpler
because you only use the events to maintain your state.
Performance vs. accuracy , the balance is still open for long lived
objects like domains though, but as virtualization gets integrated and
efficient maybe it's better to play safe.
Daniel
--
Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library http://libvirt.org/
3:19 a.m.
On Mon, Sep 22, 2008 at 10:54:58PM +0200, Daniel Veillard wrote:
On Fri, Sep 19, 2008 at 10:54:39AM +0100, Daniel P. Berrange wrote:
> Against a virConnectPtr object I'd expect to be able to register
> to get an event upon
>
> - A new domain object coming into existance
> - A existing domain object going out of existance
>
> So, you could register a callback, call Rich's virConnectListAllDomains()
> once, and then rely on the callbacks from that point onwards to keep
> your list of domains up2date. So in case of listening for domains:
Just a remark but unfortunately that scheme forces a race between
the start of the event flow and the return of the list. The way used in
the file monitoring API (FAM which I dislike but at least fixed that problem)
is that when you register you get a flow of initial events allowing
to setup your list of object. Certainly less efficient than single
synchronous call but avoid the race. The user code is also simpler
because you only use the events to maintain your state.
Performance vs. accuracy , the balance is still open for long lived
objects like domains though, but as virtualization gets integrated and
efficient maybe it's better to play safe.
I'm not sure I understand what you mean by the race - if you call the
virConnectListAllDomains() first, and then register for events, then
there is a window where you may have missed some domains starting. If
you register for events first and then call virConnectListAllDomains()
then worst case you see domains that you already know about - you ought
not to miss any events.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
3:37 a.m.
On Tue, Sep 23, 2008 at 09:19:28AM +0100, Daniel P. Berrange wrote:
On Mon, Sep 22, 2008 at 10:54:58PM +0200, Daniel Veillard wrote:
> On Fri, Sep 19, 2008 at 10:54:39AM +0100, Daniel P. Berrange wrote:
> > Against a virConnectPtr object I'd expect to be able to register
> > to get an event upon
> >
> > - A new domain object coming into existance
> > - A existing domain object going out of existance
> >
> > So, you could register a callback, call Rich's virConnectListAllDomains()
> > once, and then rely on the callbacks from that point onwards to keep
> > your list of domains up2date. So in case of listening for domains:
>
> Just a remark but unfortunately that scheme forces a race between
> the start of the event flow and the return of the list. The way used in
> the file monitoring API (FAM which I dislike but at least fixed that problem)
> is that when you register you get a flow of initial events allowing
> to setup your list of object. Certainly less efficient than single
> synchronous call but avoid the race. The user code is also simpler
> because you only use the events to maintain your state.
> Performance vs. accuracy , the balance is still open for long lived
> objects like domains though, but as virtualization gets integrated and
> efficient maybe it's better to play safe.
I'm not sure I understand what you mean by the race - if you call the
virConnectListAllDomains() first, and then register for events, then
there is a window where you may have missed some domains starting. If
you register for events first and then call virConnectListAllDomains()
then worst case you see domains that you already know about - you ought
not to miss any events.
The problem is that you cannot order the events w.r.t. the time where
the list was actually made. If you get a foo domain stopped event just
after the virConnectListAllDomains() returns with foo running in it,
you can't tell if the domain was just stopped and restarted before the
list or if the domain is really stopped. Maybe there is another event
of foo being started pending but basically you have some uncertainty
about the state and the event. You have 2 unsynchronized information
flow and no causal dependancies garanteed between the two :-)
Keeping a single flow is the only way I think to garantee you always
have a coherent state representation on the client considering the
kind of API we have.
Daniel
--
Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library http://libvirt.org/
10:54 a.m.
Daniel Veillard wrote:
On Tue, Sep 23, 2008 at 09:19:28AM +0100, Daniel P. Berrange wrote:
> On Mon, Sep 22, 2008 at 10:54:58PM +0200, Daniel Veillard wrote:
>> On Fri, Sep 19, 2008 at 10:54:39AM +0100, Daniel P. Berrange wrote:
>>> Against a virConnectPtr object I'd expect to be able to register
>>> to get an event upon
>>>
>>> - A new domain object coming into existance
>>> - A existing domain object going out of existance
>>>
>>> So, you could register a callback, call Rich's
virConnectListAllDomains()
>>> once, and then rely on the callbacks from that point onwards to keep
>>> your list of domains up2date. So in case of listening for domains:
>> Just a remark but unfortunately that scheme forces a race between
>> the start of the event flow and the return of the list. The way used in
>> the file monitoring API (FAM which I dislike but at least fixed that problem)
>> is that when you register you get a flow of initial events allowing
>> to setup your list of object. Certainly less efficient than single
>> synchronous call but avoid the race. The user code is also simpler
>> because you only use the events to maintain your state.
>> Performance vs. accuracy , the balance is still open for long lived
>> objects like domains though, but as virtualization gets integrated and
>> efficient maybe it's better to play safe.
> I'm not sure I understand what you mean by the race - if you call the
> virConnectListAllDomains() first, and then register for events, then
> there is a window where you may have missed some domains starting. If
> you register for events first and then call virConnectListAllDomains()
> then worst case you see domains that you already know about - you ought
> not to miss any events.
The problem is that you cannot order the events w.r.t. the time where
the list was actually made. If you get a foo domain stopped event just
after the virConnectListAllDomains() returns with foo running in it,
you can't tell if the domain was just stopped and restarted before the
list or if the domain is really stopped. Maybe there is another event
of foo being started pending but basically you have some uncertainty
about the state and the event. You have 2 unsynchronized information
flow and no causal dependancies garanteed between the two :-)
Keeping a single flow is the only way I think to garantee you always
have a coherent state representation on the client considering the
kind of API we have.
But the event flow is ordered and complete right? The domain can't stop and restart
without both events being generated. In your example, you would know that the domain list
is stale with respect to the stopped domain. If the start message is pending, the
'right' decision is still to ignore the domain list entry for the stopped domain.
The domain list can only supplement knowledge for domains that have not been seen in the
event stream.
Steve
Daniel
5:14 a.m.
On Thu, Sep 18, 2008 at 12:45:07PM -0400, David Lively wrote:
I'm a little concerned that a vector of event type names
isn't really
adequate for specifying a filter. Does this need to be more general
(XPathString exprs??)
I think I'm with Dan on this one. I would start small -- just domains
coming & going (unless VirtualIron needs other events). Since there
is no limit to the number of API calls we can have in libvirt, add an
API call just for registering for these domain events. Instead of
trying to overload untyped strings with complicated meanings.
But my larger concern is that an asynchronous callback mechanism
(as
proposed above) assumes the presence of some thread / process from which
to make the callbacks. This works fine in the libvirtd context, but
not outside of it. For instance, we build a "client only" version of
libvirt with ONLY the remote driver, which currently doesn't require
pthreads at all. Introducing asynchronous callbacks into the API means
pthreads is now required for this.
I'm not quite sure I follow this -- you mean it introduces pthreads
into libvirt or into the caller? As far as I can see, nothing about
this would require threads in either.
The remote protocol allows event messages to be passed back
asynchronously, although the current remote driver wouldn't expect to
receive them which might be a problem for backwards/forwards
compatibility. (Therefore the remote client must tell the remote
server that it can handle asynchronous events, and the remote client
must be prepared to talk to a server which cannot understand
asynchronous events -- there is an internal feature API you can use
for this). Notwithstanding that, you would have to add a client call
to poll for events or (better) to expose the file descriptor so that
callers can use it in select(2)/poll(2).
Actually there are a number of ways to do this -- IIRC dbus does
something semi-elegant.
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
Read my OCaml programming blog: http://camltastic.blogspot.com/
Fedora now supports 68 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
5:22 a.m.
On Fri, Sep 19, 2008 at 11:14:07AM +0100, Richard W.M. Jones wrote:
On Thu, Sep 18, 2008 at 12:45:07PM -0400, David Lively wrote:
> I'm a little concerned that a vector of event type names isn't really
> adequate for specifying a filter. Does this need to be more general
> (XPathString exprs??)
I think I'm with Dan on this one. I would start small -- just domains
coming & going (unless VirtualIron needs other events). Since there
is no limit to the number of API calls we can have in libvirt, add an
API call just for registering for these domain events. Instead of
trying to overload untyped strings with complicated meanings.
> But my larger concern is that an asynchronous callback mechanism (as
> proposed above) assumes the presence of some thread / process from which
> to make the callbacks. ???This works fine in the libvirtd context, but
> not outside of it. For instance, we build a "client only" version of
> libvirt with ONLY the remote driver, which currently doesn't require
> pthreads at all. Introducing asynchronous callbacks into the API means
> pthreads is now required for this.
I'm not quite sure I follow this -- you mean it introduces pthreads
into libvirt or into the caller? As far as I can see, nothing about
this would require threads in either.
The remote protocol allows event messages to be passed back
asynchronously, although the current remote driver wouldn't expect to
receive them which might be a problem for backwards/forwards
compatibility. (Therefore the remote client must tell the remote
server that it can handle asynchronous events, and the remote client
must be prepared to talk to a server which cannot understand
asynchronous events -- there is an internal feature API you can use
for this). Notwithstanding that, you would have to add a client call
to poll for events or (better) to expose the file descriptor so that
callers can use it in select(2)/poll(2).
Making a client poll for events is evil because it prevents the CPU
going into low power mode when nothing is happening. Exposing a FD
severely limits your flexibility, because libvirt then has no control
over when the FD is being monitored - you may well not want it being
select'd on all the time - and/or want to change the events being
monitored for at various times.
Actually there are a number of ways to do this -- IIRC dbus does
something semi-elegant.
Yes, the DBus approach (which is what I followed for the src/events.h
API) is the way I like best. It doesn't require libvirt to do tricks
internally with threads, and doesn't force a particular event loop
impl onto applications, and still allows libvirt full control over
just when & how FD is addded to the event loop for monitoring.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
5:30 a.m.
On Fri, Sep 19, 2008 at 11:22:24AM +0100, Daniel P. Berrange wrote:
Making a client poll for events is evil because it prevents the CPU
going into low power mode when nothing is happening.
Didn't mean literally poll of course :-)
Exposing a FD severely limits your flexibility, because libvirt then
has no control over when the FD is being monitored - you may well
not want it being select'd on all the time - and/or want to change
the events being monitored for at various times.
Yup, didn't get down to the bottom of your other email. Exposing an
FD is also tricky on Windows too (where you have the joys of
"WSAWaitForMultipleEvents"), so best to avoid it if possible. If
someone (dbus? glib?) has solved this in a platform-independent way
already then so much the better.
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/
5:37 a.m.
On Fri, Sep 19, 2008 at 11:30:01AM +0100, Richard W.M. Jones wrote:
On Fri, Sep 19, 2008 at 11:22:24AM +0100, Daniel P. Berrange wrote:
> Making a client poll for events is evil because it prevents the CPU
> going into low power mode when nothing is happening.
Didn't mean literally poll of course :-)
> Exposing a FD severely limits your flexibility, because libvirt then
> has no control over when the FD is being monitored - you may well
> not want it being select'd on all the time - and/or want to change
> the events being monitored for at various times.
Yup, didn't get down to the bottom of your other email. Exposing an
FD is also tricky on Windows too (where you have the joys of
"WSAWaitForMultipleEvents"), so best to avoid it if possible. If
someone (dbus? glib?) has solved this in a platform-independent way
already then so much the better.
The glib event loop implementation on Windows is built using the
WSAWaitForMultipleEvents API, so any libvirt-glib.so impl of the
events.h API would 'just work' in this respect
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
10:08 a.m.
On Fri, 2008-09-19 at 11:14 +0100, Richard W.M. Jones wrote:
On Thu, Sep 18, 2008 at 12:45:07PM -0400, David Lively wrote:
> I'm a little concerned that a vector of event type names isn't really
> adequate for specifying a filter. Does this need to be more general
> (XPathString exprs??)
I think I'm with Dan on this one. I would start small -- just domains
coming & going (unless VirtualIron needs other events). Since there
is no limit to the number of API calls we can have in libvirt, add an
API call just for registering for these domain events. Instead of
trying to overload untyped strings with complicated meanings.
Okay. I'm fine with a more strongly-typed event protocol. While it's
more work to add new classes of events (as compared with extending event
XML), that's probably a Good Thing :-)
> But my larger concern is that an asynchronous callback mechanism (as
> proposed above) assumes the presence of some thread / process from which
> to make the callbacks. This works fine in the libvirtd context, but
> not outside of it. For instance, we build a "client only" version of
> libvirt with ONLY the remote driver, which currently doesn't require
> pthreads at all. Introducing asynchronous callbacks into the API means
> pthreads is now required for this.
I'm not quite sure I follow this -- you mean it introduces pthreads
into libvirt or into the caller? As far as I can see, nothing about
this would require threads in either.
I meant that if we expected the callbacks to "just happen", libvirt (at
least, the libvirtd-less version) would need to spawn a thread to make
the callbacks.
But this is quite easily avoided by providing a hook that clients are
responsible for calling for event delivery, as suggested. I had
considered this as an alternative to the file-descriptor interface (but
thought the fd interface was convenient with to use w/select() &
poll()). After considering the problems with fds and power-saving and
windows compatibility, I agree an event-delivery hook sounds like the
best idea, perhaps:
int virDeliverEvents(int timeout)
where timeout is interpreted as in poll() (i.e., max millisecs to block,
0 means don't block, negative means block forever).
The remote protocol allows event messages to be passed back
asynchronously, although the current remote driver wouldn't expect to
receive them which might be a problem for backwards/forwards
compatibility. (Therefore the remote client must tell the remote
server that it can handle asynchronous events, and the remote client
must be prepared to talk to a server which cannot understand
asynchronous events -- there is an internal feature API you can use
for this). Notwithstanding that, you would have to add a client call
to poll for events or (better) to expose the file descriptor so that
callers can use it in select(2)/poll(2).
Yeah, I expect the remote implementation will be the worst part! Thanks
for the pointers.
Thanks,
Dave
2:08 p.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
David Lively schreef:
Yeah, I expect the remote implementation will be the worst part!
Thanks
for the pointers.
I already created a complete avahi messaging implementation around
libvirt. And to be honest, it is trivial to do so, but who is going to
fetch events from the different backends in a better way than polling
every second?
Stefan
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEAREKAAYFAkjT+JwACgkQYH1+F2Rqwn2EUACgiAdnJRA/wkSywJIG6poo1BVZ
iOAAnjyYuXYQ0Hou/J312ZF3+meHXK2B
=pmSr
-----END PGP SIGNATURE-----
1:23 a.m.
On Fri, Sep 19, 2008 at 11:08:37AM -0400, David Lively wrote:
int virDeliverEvents(int timeout)
This isn't really much use because it doesn't integrate with common
main loops. I suggest you look at glib to see how it handles main
loops and how to integrate with it.
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
Read my OCaml programming blog: http://camltastic.blogspot.com/
Fedora now supports 68 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
10:11 a.m.
Okay - I'm looking at glib docs. You guys are suggesting I create
something that's easy to plug into a glib main loop, right?
So ... something that's easy to "wrap" with a GSource via
g_source_new()? (Stop me now if I'm on the wrong track!) So ...
something that plugs into GSourceFuncs elements (in particular
prepare/check/dispatch/finalize) easily??
int virEventsNeedDelivering(virConnectionPtr conn)
int virEventsDispatch(virConnectionPtr conn)
int virEventsCleanup(virConnectionPtr conn)
and perhaps:
int virEventsRunOnce(virConnectionPtr conn)
to wrap them all for non-glib-ish implementations?
Sorry if I'm being dense. I've looked at the qemud events interface,
but I still feel like I'm missing something here.
Dave
On Mon, 2008-09-22 at 07:23 +0100, Richard W.M. Jones wrote:
On Fri, Sep 19, 2008 at 11:08:37AM -0400, David Lively wrote:
> int virDeliverEvents(int timeout)
This isn't really much use because it doesn't integrate with common
main loops. I suggest you look at glib to see how it handles main
loops and how to integrate with it.
Rich.
10:22 a.m.
On Mon, Sep 22, 2008 at 11:11:17AM -0400, David Lively wrote:
Okay - I'm looking at glib docs. You guys are suggesting I
create
something that's easy to plug into a glib main loop, right?
So ... something that's easy to "wrap" with a GSource via
g_source_new()? (Stop me now if I'm on the wrong track!) So ...
something that plugs into GSourceFuncs elements (in particular
prepare/check/dispatch/finalize) easily??
int virEventsNeedDelivering(virConnectionPtr conn)
int virEventsDispatch(virConnectionPtr conn)
int virEventsCleanup(virConnectionPtr conn)
and perhaps:
int virEventsRunOnce(virConnectionPtr conn)
to wrap them all for non-glib-ish implementations?
No, this the wrong approach. This is defining an event loop impl - we
don't want todo that. We need to define an API to let an application
provide a set of callback for libvirt to talk to an existing event
loop impl. ie a way for libvirt to register FD watches and timeouts,
not a way for apps to manually process libvirt events in this way this
example shows.
The public API for this is along the lines of that currently defined
in the src/events.h file.
There are a set of functions libvirt needs in order to register actions
for FDs, and/or timeouts. So the public API should consist of a way
to register impls of these APIs
typedef int (*virEventAddHandleFunc)(int, int, virEventHandleCallback, void *);
typedef void (*virEventUpdateHandleFunc)(int, int);
typedef int (*virEventRemoveHandleFunc)(int);
typedef int (*virEventAddTimeoutFunc)(int, virEventTimeoutCallback, void *);
typedef void (*virEventUpdateTimeoutFunc)(int, int);
typedef int (*virEventRemoveTimeoutFunc)(int);
void virEventRegisterImpl(virEventAddHandleFunc addHandle,
virEventUpdateHandleFunc updateHandle,
virEventRemoveHandleFunc removeHandle,
virEventAddTimeoutFunc addTimeout,
virEventUpdateTimeoutFunc updateTimeout,
virEventRemoveTimeoutFunc removeTimeout);
A separate libvirt-glib.so, would provide a API call
virEventRegisterGLib()
which calls virEventRegisterImpl() with a suitable implementation for
glib. An application like virt-manager which uses glib and wants events
would then calll virEventRegisterGLib(). If it had a custom event loop
of its own, then it could call virEventRegisterImpl() directly with its
special impl.
It may be worth making our public API even more closely aligned with
dbus - see dbus-connection.h and dbus-server.h - so people writing
glue functions for it could just reuse what they've already written
for dbus.
typedef dbus_bool_t (* DBusAddWatchFunction) (DBusWatch *watch,
void *data);
typedef void (* DBusWatchToggledFunction) (DBusWatch *watch,
void *data);
typedef void (* DBusRemoveWatchFunction) (DBusWatch *watch,
void *data);
typedef dbus_bool_t (* DBusAddTimeoutFunction) (DBusTimeout *timeout,
void *data);
typedef void (* DBusTimeoutToggledFunction) (DBusTimeout *timeout,
void *data);
typedef void (* DBusRemoveTimeoutFunction) (DBusTimeout *timeout,
void *data);
dbus_bool_t dbus_server_set_watch_functions (DBusServer *server,
DBusAddWatchFunction
add_function,
DBusRemoveWatchFunction
remove_function,
DBusWatchToggledFunction
toggled_function,
void *data,
DBusFreeFunction
free_data_function);
dbus_bool_t dbus_server_set_timeout_functions (DBusServer *server,
DBusAddTimeoutFunction
add_function,
DBusRemoveTimeoutFunction
remove_function,
DBusTimeoutToggledFunction
toggled_function,
void *data,
DBusFreeFunction
free_data_function);
A 'watch' in DBus terminology is a file descriptor monitor.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
2:15 p.m.
Okay, at long last, I see what you mean (I think). Apps using libvirt
events must register an EventImpl via virRegisterEventImpl. Internally,
suppose we implement events via an anonymous pipe. libvirt would call
EventImpl.addHandle(pipe_read_fd, POLLIN ..., __virHandleEvents, conn)
so the app's main loop would monitor fd (in whatever manner it chooses),
then call __virHandleEvents(fd, conn) when it detected activity.
__virHandleEvents would pull the event from the pipe and dispatch
handlers as appropriate. We'd call eventImpl.removeHandle(pipe_read_fd)
when the domain goes away. Am I finally on the right track??
If so, are you proposing that we simply make the existing src/events.h
interface public? At this point, I don't see the need for anything but
addHandle and removeHandle (but I can see how the others might be useful
to libvirt eventually).
Dave
On Mon, 2008-09-22 at 16:22 +0100, Daniel P. Berrange wrote:
No, this the wrong approach. This is defining an event loop impl - we
don't want todo that. We need to define an API to let an application
provide a set of callback for libvirt to talk to an existing event
loop impl. ie a way for libvirt to register FD watches and timeouts,
not a way for apps to manually process libvirt events in this way this
example shows.
The public API for this is along the lines of that currently defined
in the src/events.h file.
There are a set of functions libvirt needs in order to register actions
for FDs, and/or timeouts. So the public API should consist of a way
to register impls of these APIs
typedef int (*virEventAddHandleFunc)(int, int, virEventHandleCallback, void *);
typedef void (*virEventUpdateHandleFunc)(int, int);
typedef int (*virEventRemoveHandleFunc)(int);
typedef int (*virEventAddTimeoutFunc)(int, virEventTimeoutCallback, void *);
typedef void (*virEventUpdateTimeoutFunc)(int, int);
typedef int (*virEventRemoveTimeoutFunc)(int);
void virEventRegisterImpl(virEventAddHandleFunc addHandle,
virEventUpdateHandleFunc updateHandle,
virEventRemoveHandleFunc removeHandle,
virEventAddTimeoutFunc addTimeout,
virEventUpdateTimeoutFunc updateTimeout,
virEventRemoveTimeoutFunc removeTimeout);
A separate libvirt-glib.so, would provide a API call
virEventRegisterGLib()
which calls virEventRegisterImpl() with a suitable implementation for
glib. An application like virt-manager which uses glib and wants events
would then calll virEventRegisterGLib(). If it had a custom event loop
of its own, then it could call virEventRegisterImpl() directly with its
special impl.
It may be worth making our public API even more closely aligned with
dbus - see dbus-connection.h and dbus-server.h - so people writing
glue functions for it could just reuse what they've already written
for dbus.
typedef dbus_bool_t (* DBusAddWatchFunction) (DBusWatch *watch,
void *data);
typedef void (* DBusWatchToggledFunction) (DBusWatch *watch,
void *data);
typedef void (* DBusRemoveWatchFunction) (DBusWatch *watch,
void *data);
typedef dbus_bool_t (* DBusAddTimeoutFunction) (DBusTimeout *timeout,
void *data);
typedef void (* DBusTimeoutToggledFunction) (DBusTimeout *timeout,
void *data);
typedef void (* DBusRemoveTimeoutFunction) (DBusTimeout *timeout,
void *data);
dbus_bool_t dbus_server_set_watch_functions (DBusServer *server,
DBusAddWatchFunction
add_function,
DBusRemoveWatchFunction
remove_function,
DBusWatchToggledFunction
toggled_function,
void *data,
DBusFreeFunction
free_data_function);
dbus_bool_t dbus_server_set_timeout_functions (DBusServer *server,
DBusAddTimeoutFunction
add_function,
DBusRemoveTimeoutFunction
remove_function,
DBusTimeoutToggledFunction
toggled_function,
void *data,
DBusFreeFunction
free_data_function);
A 'watch' in DBus terminology is a file descriptor monitor.
Daniel
3:17 a.m.
On Mon, Sep 22, 2008 at 03:15:25PM -0400, David Lively wrote:
Okay, at long last, I see what you mean (I think). Apps using
libvirt
events must register an EventImpl via virRegisterEventImpl. Internally,
suppose we implement events via an anonymous pipe. libvirt would call
EventImpl.addHandle(pipe_read_fd, POLLIN ..., __virHandleEvents, conn)
so the app's main loop would monitor fd (in whatever manner it chooses),
then call __virHandleEvents(fd, conn) when it detected activity.
__virHandleEvents would pull the event from the pipe and dispatch
handlers as appropriate. We'd call eventImpl.removeHandle(pipe_read_fd)
when the domain goes away. Am I finally on the right track??
Well the answer will depend on the particular virtualization driver. For
the remote driver (which includes QEMU + LXC + any others running inside
the daemon) you already have a file descriptor that can be monitored - the
UNIX/TCP socket to the daemon.
For the Xen driver we would grab the xenstored UNIX socket file descriptor
and register xenstore watches - the FD would become readable when the
watch fires.
So, yes, the remote driver would register its FD with EventImpl.addHandle.
In the GLib<->libvirt event loop binding, this would in turn register the
FD with the GLib event loop and a callback wrapper. The GLib<->libvirt
event loop binding would detect the neccessary poll condition, invok the
the callback associated with that FD, which would then invoke the callback
actually regisstered by the remote driver
virHandleEvents() is not something that would directly play a part - that's
a specific part of the libvirtd daemon's event loop impl.
If so, are you proposing that we simply make the existing
src/events.h
interface public? At this point, I don't see the need for anything but
addHandle and removeHandle (but I can see how the others might be useful
to libvirt eventually).
The 'updateHandle' method was essentially an efficiency method - it
allows us to change the poll events that a FD is being monitored for
without having to register & unregister. This both helps efficiency
of the event loop impl, and means the impl can guarentee this operation
will never encounter OOM.
If we split the virEventRegisterImpl() function into two, in the same
way that dbus does - virEventRegisterWatchImpl, virEventRegisterTimeoutImpl
then we could just make the watch impl public for now and worry about the
timeout impl later if there turns out to be a need for it.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
3:47 p.m.
On Fri, Sep 19, 2008 at 11:08:37AM -0400, David Lively wrote:
On Fri, 2008-09-19 at 11:14 +0100, Richard W.M. Jones wrote:
> On Thu, Sep 18, 2008 at 12:45:07PM -0400, David Lively wrote:
> > I'm a little concerned that a vector of event type names isn't
really
> > adequate for specifying a filter. Does this need to be more general
> > (XPathString exprs??)
>
> I think I'm with Dan on this one. I would start small -- just domains
> coming & going (unless VirtualIron needs other events). Since there
> is no limit to the number of API calls we can have in libvirt, add an
> API call just for registering for these domain events. Instead of
> trying to overload untyped strings with complicated meanings.
Okay. I'm fine with a more strongly-typed event protocol. While it's
more work to add new classes of events (as compared with extending event
XML), that's probably a Good Thing :-)
it makes processing event simpler/cheaper, and if you want to scale
that's not neglectible :-0
>
> > But my larger concern is that an asynchronous callback mechanism (as
> > proposed above) assumes the presence of some thread / process from which
> > to make the callbacks. This works fine in the libvirtd context, but
> > not outside of it. For instance, we build a "client only" version
of
> > libvirt with ONLY the remote driver, which currently doesn't require
> > pthreads at all. Introducing asynchronous callbacks into the API means
> > pthreads is now required for this.
>
> I'm not quite sure I follow this -- you mean it introduces pthreads
> into libvirt or into the caller? As far as I can see, nothing about
> this would require threads in either.
I meant that if we expected the callbacks to "just happen", libvirt (at
least, the libvirtd-less version) would need to spawn a thread to make
the callbacks.
But this is quite easily avoided by providing a hook that clients are
responsible for calling for event delivery, as suggested. I had
considered this as an alternative to the file-descriptor interface (but
thought the fd interface was convenient with to use w/select() &
poll()).
the fd registration is really not ideal, as Dan and Rich suggested
the hook based solution allows cleaner code, and more portability.
After considering the problems with fds and power-saving and
windows compatibility, I agree an event-delivery hook sounds like the
best idea, perhaps:
int virDeliverEvents(int timeout)
where timeout is interpreted as in poll() (i.e., max millisecs to block,
0 means don't block, negative means block forever).
let's copy the kind of interface DBus provided, at least for
consistency :-)
>
> The remote protocol allows event messages to be passed back
> asynchronously, although the current remote driver wouldn't expect to
> receive them which might be a problem for backwards/forwards
> compatibility. (Therefore the remote client must tell the remote
> server that it can handle asynchronous events, and the remote client
> must be prepared to talk to a server which cannot understand
> asynchronous events -- there is an internal feature API you can use
> for this). Notwithstanding that, you would have to add a client call
> to poll for events or (better) to expose the file descriptor so that
> callers can use it in select(2)/poll(2).
Yeah, I expect the remote implementation will be the worst part! Thanks
for the pointers.
One thing i would like to make sure for remote is that we can reuse
the existing connection, as opening a second connection initiated by
the server may give troubles with firewalls.
Daniel
--
Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library http://libvirt.org/
5905
days inactive
5910
days old
19 comments
7 participants
participants (7)
-
Daniel P. Berrange -
Daniel Veillard -
David Lively -
Gerd Hoffmann -
Richard W.M. Jones -
Stefan de Konink -
Steve Ofsthun