[Libvir] Asynchronous notifications of domain start/stop
I had some private discussions with a developer who wants to start using libvirt, but requires asynch notifications of domain start/stop (and other stuff, but this will do for the moment). Right now he's using XenStore watches. The alternative of polling didn't look very appealing for him. A rough API for this might look like this: typedef int virEventHandle; /* Register a callback to catch domain start events. */ virEventHandle virEventDomainStart (virConnectPtr, void (*f) (virConnectPtr, int domid)); /* Register a callback to catch domain stop events. */ virEventHandle virEventDomainStop (virConnectPtr, void (*f) (virConnectPtr, int domid)); /* Unregister any type of callback. */ int virEventCancel (virConnectPtr, virEventHandle handle); Implementation -------------- In the Xen case, we can use XenStore watches[1]. Callbacks from XenStore can be translated directly into event callbacks. In the qemu case, the daemon knows when the qemu instances that it manages come and go. In the qemu/remote case, asynch notifications are nominally supported by the protocol, but I haven't really looked at all the implications. Seems like we'd need to use MSG_OOB to signal the client process, and translate the SIGURG signal received into a callback. This is all tricky and Unix-specific though so if you have any better ideas ... We would also need to avoid mixing ordinary request/response with the notifications, which OOB takes care of. Rich. [1] http://wiki.xensource.com/xenwiki/XenBusDriver -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
Hi,
libvirt, but requires asynch notifications of domain start/stop (and
Notification. Not async notification. Point is I want avoid waking up every second or so to poll for rare events. There is no need for the events being truly async.
typedef int virEventHandle;
/* Register a callback to catch domain start events. */ virEventHandle virEventDomainStart (virConnectPtr, void (*f) (virConnectPtr, int domid));
/* Register a callback to catch domain stop events. */ virEventHandle virEventDomainStop (virConnectPtr, void (*f) (virConnectPtr, int domid));
/* Unregister any type of callback. */ int virEventCancel (virConnectPtr, virEventHandle handle);
Doesn't need to be a callback model.
In the Xen case, we can use XenStore watches[1]. Callbacks from XenStore can be translated directly into event callbacks.
xenstore doesn't calls you back. xenstore gives you a file handle you can select() on and a function you can call when data is available to figure which watch did fire. You can register a token together with the watch, which in turn you can use to implement callbacks. But libxenstore itself doesn't callback you.
In the qemu/remote case, asynch notifications are nominally supported by the protocol, but I haven't really looked at all the implications. Seems like we'd need to use MSG_OOB to signal the client process, and translate the SIGURG signal received into a callback.
I'd like to veto that idea. Calling a callback from a signal handler is just asking for trouble. It isn't safe to call libc functions from a signal handler for example. Also libvirt hijacking the SIGURG signal is not very nice.
We would also need to avoid mixing ordinary request/response with the notifications
Why? Just disallowing notifications while a request is in flight should be enougth I think. I'd like to have something which integrates nicely with the usual select() based event loops, like the xenstore model. Sort of "here is a file handle, and if data comes in call this function." The function in turn could either call callbacks or return some event struct. cheers, Gerd
On Wed, Jul 18, 2007 at 12:30:03PM +0200, Gerd Hoffmann wrote:
Hi,
libvirt, but requires asynch notifications of domain start/stop (and
Notification. Not async notification.
Point is I want avoid waking up every second or so to poll for rare events. There is no need for the events being truly async.
typedef int virEventHandle;
/* Register a callback to catch domain start events. */ virEventHandle virEventDomainStart (virConnectPtr, void (*f) (virConnectPtr, int domid));
/* Register a callback to catch domain stop events. */ virEventHandle virEventDomainStop (virConnectPtr, void (*f) (virConnectPtr, int domid));
/* Unregister any type of callback. */ int virEventCancel (virConnectPtr, virEventHandle handle);
Doesn't need to be a callback model.
In general events are a pain because the integration in the application is never that clean or simple. For the callback model to work you need the application to call the library in some way (to then check the condition and do the callback), to me threading in the library is not a good working model because then you callback from a different thread and lot of confusion/races ensue.
In the Xen case, we can use XenStore watches[1]. Callbacks from XenStore can be translated directly into event callbacks.
xenstore doesn't calls you back. xenstore gives you a file handle you can select() on and a function you can call when data is available to figure which watch did fire.
You can register a token together with the watch, which in turn you can use to implement callbacks. But libxenstore itself doesn't callback you.
In the qemu/remote case, asynch notifications are nominally supported by the protocol, but I haven't really looked at all the implications. Seems like we'd need to use MSG_OOB to signal the client process, and translate the SIGURG signal received into a callback.
I'd like to veto that idea. Calling a callback from a signal handler is just asking for trouble. It isn't safe to call libc functions from a signal handler for example. Also libvirt hijacking the SIGURG signal is not very nice.
We would also need to avoid mixing ordinary request/response with the notifications
Why? Just disallowing notifications while a request is in flight should be enougth I think.
I'd like to have something which integrates nicely with the usual select() based event loops, like the xenstore model. Sort of "here is a file handle, and if data comes in call this function." The function in turn could either call callbacks or return some event struct.
I did that trick in gamin to write to the fd within the signal to pass the information in a safe way. Still I would really like to avoid getting down to the same kind of crazyness. The problem is that not everything is data down a file/pipe but that's the only way to integrate decently in an Unix app programming model :-( I resisted so far adding events in libvirt API precisely to keep the level of complexity of the API down. If we go there, then an fd based mechanism with a function called by the user to get the information has my preference. That way except for the fd everything is synchronous and purely within the application flow of control. The problem is how to then feed the fd when "something occurs" especially since the something is happening in a different process or even a different domain. I have no clear answer to this, and until I have (and this is still sensible for the variuous hypervisor) I would rather postpone getting there. Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Daniel Veillard wrote:
For the callback model to work you need the application to call the library in some way (to then check the condition and do the callback), to me threading in the library is not a good working model because then you callback from a different thread and lot of confusion/races ensue.
Yep, in that point threads equally evil like callbacks from signal handlers. Well, slightly better as you can at least use pthread locking primitives then. But forcing apps into using threads isn't very nice. I avoid threads when possible exactly to get around the locking hassle.
I resisted so far adding events in libvirt API precisely to keep the level of complexity of the API down. If we go there, then an fd based mechanism with a function called by the user to get the information has my preference.
/me too ;)
That way except for the fd everything is synchronous and purely within the application flow of control. The problem is how to then feed the fd when "something occurs" especially since the something is happening in a different process or even a different domain.
What troubling events do you have in mind? For remote you'll have the socket to select() on. As I far I know the very same protocol is used for qemu, so you probably have a socket too, right? For xen you can just use the xenstore fd. Dunno on the network stuff. Who manages the network? Probably the libvirtd daemon too? Then the notifications can go through the libvirt <-> libvirtd socket connection, so you have a fd too. What might be needed is allowing multiple file handles (one socket to xenstore, one socket to libvirtd) so libvirt can collect events from different sources. It becomes a bit difficuilt at that point. avahi has that problem too. They have a "give me functions to register and unregister filehandles"-style api for that. And some helper functions to make that easy for apps using the event loops of the usual gui libs (glib/gtk, qt). cheers, Gerd
On Wed, Jul 18, 2007 at 01:23:22PM +0200, Gerd Hoffmann wrote:
That way except for the fd everything is synchronous and purely within the application flow of control. The problem is how to then feed the fd when "something occurs" especially since the something is happening in a different process or even a different domain.
What troubling events do you have in mind? For remote you'll have the socket to select() on. As I far I know the very same protocol is used for qemu, so you probably have a socket too, right? For xen you can just use the xenstore fd. Dunno on the network stuff. Who manages the network? Probably the libvirtd daemon too? Then the notifications can go through the libvirt <-> libvirtd socket connection, so you have a fd too.
If you give directly the fd of the socket used to talk internally to the daemon then you must allow for case where there is data to read but no even will be generated in the function, because since you don't have the library filtering you can't be sure what's available is something the application is interested in. For example if doing a migration a lot of traffic may happen on multiple sockets before the user see an event he's actually interested into. There is so much nastyness once you break the purely synchronous model.
What might be needed is allowing multiple file handles (one socket to xenstore, one socket to libvirtd) so libvirt can collect events from different sources. It becomes a bit difficuilt at that point. avahi has that problem too. They have a "give me functions to register and unregister filehandles"-style api for that. And some helper functions to make that easy for apps using the event loops of the usual gui libs (glib/gtk, qt).
Could be a possibility, the other one is to create one thread internal to the library but limited purely to doing I/O, still I find this disgusting. Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Daniel Veillard wrote:
If you give directly the fd of the socket used to talk internally to the daemon then you must allow for case where there is data to read but no even will be generated in the function, because since you don't have the library filtering you can't be sure what's available is something the application is interested in.
I don't see that as a problem. Just process the data and return "no event for you yet" to the application.
For example if doing a migration a lot of traffic may happen on multiple sockets before the user see an event he's actually interested into. There is so much nastyness once you break the purely synchronous model.
I don't expect to be notified truely async, i.e. while some virDomMigrate() call is in progress. I just want be able to -- for example -- get notified about new domains showing up while sitting around idle without polling once per second. Not sending notifications while other operations are in progress is fine, just queue them up and send them once the migration is done. cheers, Gerd
Hi,
I don't expect to be notified truely async, i.e. while some virDomMigrate() call is in progress.
Uhm, while thinking about it: Even when not designing the library api async, it still is at the wire, i.e. you can have a cross over from a server side notification and a client-side request. Hmm. No easy way out without opening a separate communication channel for the notifications ... cheers, Gerd
Daniel Veillard wrote:
On Wed, Jul 18, 2007 at 01:23:22PM +0200, Gerd Hoffmann wrote:
That way except for the fd everything is synchronous and purely within the application flow of control. The problem is how to then feed the fd when "something occurs" especially since the something is happening in a different process or even a different domain. What troubling events do you have in mind? For remote you'll have the socket to select() on. As I far I know the very same protocol is used for qemu, so you probably have a socket too, right? For xen you can just use the xenstore fd. Dunno on the network stuff. Who manages the network? Probably the libvirtd daemon too? Then the notifications can go through the libvirt <-> libvirtd socket connection, so you have a fd too.
If you give directly the fd of the socket used to talk internally to the daemon then you must allow for case where there is data to read but no even will be generated in the function, because since you don't have the library filtering you can't be sure what's available is something the application is interested in. For example if doing a migration a lot of traffic may happen on multiple sockets before the user see an event he's actually interested into. There is so much nastyness once you break the purely synchronous model.
I think, specifically with migration, the problem case is: client -- sends domainMigrate --> source server migration starts domain disappears from source server source server -- sends domainStop event --> client later, domain appears on destination server destination server -- sends domainStart event --> client source server -- sends reply to domainMigrate --> client The domainStop event from the source server is the problem message. It appears in between the request and the reply for an ordinary RPC. This argues for having two separate connections, one for synchronous RPC, one for events. That in itself is not very pleasant, particularly if the connections are going over ssh, or if connections are otherwise expensive to set up or maintain (hello TLS). Oh and BTW, writing and reading TLS messages does not translate simply into write(2) and read(2) on the underlying fd. TLS tends to chat backwards and forwards. The remote driver goes to a lot of trouble to deal with this.
What might be needed is allowing multiple file handles (one socket to xenstore, one socket to libvirtd) so libvirt can collect events from different sources. It becomes a bit difficuilt at that point. avahi has that problem too. They have a "give me functions to register and unregister filehandles"-style api for that. And some helper functions to make that easy for apps using the event loops of the usual gui libs (glib/gtk, qt).
Could be a possibility, the other one is to create one thread internal to the library but limited purely to doing I/O, still I find this disgusting.
Mmmm, not nice. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
Gerd Hoffmann wrote:
xenstore doesn't calls you back. xenstore gives you a file handle you can select() on and a function you can call when data is available to figure which watch did fire.
Yes, my mistake. I was -- cough -- reading the kernel interface to xenbus, not the xenstore userspace interface. As you rightly say the userspace interface gives you a fd which you have to select() on, see the C example here: http://wiki.xensource.com/xenwiki/XenStoreReference
We would also need to avoid mixing ordinary request/response with the notifications
Why? Just disallowing notifications while a request is in flight should be enougth I think.
I'd like to have something which integrates nicely with the usual select() based event loops, like the xenstore model. Sort of "here is a file handle, and if data comes in call this function." The function in turn could either call callbacks or return some event struct.
I can see this working. In the Xen (local) case we can hand back the Xenstore fd itself for applications to monitor. In the Xen remote case the server side has already got an event loop which can look for fd events. They'll be sent synchronously to the client. The client will have to hand out the remote connection fd for applications to select() on. In the qemu case I'm not sure how to modify qemu_driver.c to handle this. One for Dan Berrange to comment on. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Wed, Jul 18, 2007 at 12:57:37PM +0100, Richard W.M. Jones wrote:
We would also need to avoid mixing ordinary request/response with the notifications
Why? Just disallowing notifications while a request is in flight should be enougth I think.
I'd like to have something which integrates nicely with the usual select() based event loops, like the xenstore model. Sort of "here is a file handle, and if data comes in call this function." The function in turn could either call callbacks or return some event struct.
I can see this working.
In the Xen (local) case we can hand back the Xenstore fd itself for applications to monitor.
In the Xen remote case the server side has already got an event loop which can look for fd events. They'll be sent synchronously to the client. The client will have to hand out the remote connection fd for applications to select() on.
In the qemu case I'm not sure how to modify qemu_driver.c to handle this. One for Dan Berrange to comment on.
We should not explicitly expose 'the xenstore filehandle', or 'the libvirtd' filehandle per-se. Instead we should provide an API which lets application register callbacks for adding/removing file handles to an event loop. We in fact already have this - look at the src/event.h file. So if we make this part of the official public API, then applications merely need provide an impl of the virEventAddHandleFunc and virEventRemoveHandleFunc. The underlying libvirt drivers can thus invoke these callbacks as required, be it with the XenStored filehandle, a XenAPI filehandle, a libvirtd fd or something else. We can provide sample impls of the virEventAddHandleFunc and virEventRemoveHandleFunc code for GLib and/or QT in a libvirtglib.so and libvirtqt.so add-on libraries. This allows for easy integration with most GUI apps, while also allowing integration with custom home brew event loops. This is how DBus provides event loop integration without tieing itself to a specific impl & it works very nicely. I outline this previously in the bottom half of this mail. http://www.redhat.com/archives/libvir-list/2007-May/msg00083.html Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Daniel P. Berrange wrote:
We should not explicitly expose 'the xenstore filehandle', or 'the libvirtd' filehandle per-se. Instead we should provide an API which lets application register callbacks for adding/removing file handles to an event loop.
I outline this previously in the bottom half of this mail.
http://www.redhat.com/archives/libvir-list/2007-May/msg00083.html
YES! That is the way to go I guess ... cheers, Gerd
Daniel P. Berrange wrote:
Instead we should provide an API which lets application register callbacks for adding/removing file handles to an event loop.
Understood ...
We in fact already have this - look at the src/event.h file. So if we make this part of the official public API, then applications merely need provide an impl of the virEventAddHandleFunc and virEventRemoveHandleFunc. The underlying libvirt drivers can thus invoke these callbacks as required, be it with the XenStored filehandle, a XenAPI filehandle, a libvirtd fd or something else.
If I understand what you're saying here, you are proposing that libvirt provides a call like this following (don't worry about specifics): virEventDomainNotify (virConnectPtr, int state, void (*f)(...)); The implementation of this call is that it sets up a xenstore watch, and registers the xenstore fd with the previously defined virEventAddHandleFunc (implemented by the main app / app's event loop code). When the event happens, xenstore fd gets some data available, app's event loop triggers this, and calls into libvirt, which then calls the app's callback function ("f"). Is that right? In the remote case it's more tricky. What does the remote's callback function do when it gets an event? (This could happen pretty asynchronously relative to other communications going on -- eg. in the middle of a round of TLS chatter). Choices seem to be: (1) Send the data OOB, unencrypted[1]. (2) Piggy-back the notification on to the reply to the next call [not asynchronous]. (3) Try to work out where we are in the TLS message and if necessary queue the event up and wait for a "safe" place to insert the message in the protocol. (3) seems fairly do-able. Rich. [1] TLS 1.1 includes support for OOB data, but it's not really clear if GnuTLS supports that. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Wed, Jul 18, 2007 at 03:45:10PM +0100, Richard W.M. Jones wrote:
Daniel P. Berrange wrote:
Instead we should provide an API which lets application register callbacks for adding/removing file handles to an event loop.
Understood ...
We in fact already have this - look at the src/event.h file. So if we make this part of the official public API, then applications merely need provide an impl of the virEventAddHandleFunc and virEventRemoveHandleFunc. The underlying libvirt drivers can thus invoke these callbacks as required, be it with the XenStored filehandle, a XenAPI filehandle, a libvirtd fd or something else.
If I understand what you're saying here, you are proposing that libvirt provides a call like this following (don't worry about specifics):
virEventDomainNotify (virConnectPtr, int state, void (*f)(...));
The implementation of this call is that it sets up a xenstore watch, and registers the xenstore fd with the previously defined virEventAddHandleFunc (implemented by the main app / app's event loop code).
When the event happens, xenstore fd gets some data available, app's event loop triggers this, and calls into libvirt, which then calls the app's callback function ("f").
Is that right?
Yep, that's pretty much spot on.
In the remote case it's more tricky. What does the remote's callback function do when it gets an event? (This could happen pretty asynchronously relative to other communications going on -- eg. in the middle of a round of TLS chatter). Choices seem to be: (1) Send the data OOB, unencrypted[1]. (2) Piggy-back the notification on to the reply to the next call [not asynchronous]. (3) Try to work out where we are in the TLS message and if necessary queue the event up and wait for a "safe" place to insert the message in the protocol. (3) seems fairly do-able.
The daemon has a buffer of outgoing data - and it appends to this buffer discrete 'XDR' messages. So we should be able to simply append another XDR message per notification & not have to overly worry about things being mixed up in the stream. When the client is deserializing replies during normal calls it'll have to keep an eye out for messages which are in fact notifications, rather than its expected reply, but again that should not be too hard since we're dealing with discrete XDR encoded messages. So AFAICT we don't need any OOB or piggybacking stuff - just queue up the data to send as normal.
[1] TLS 1.1 includes support for OOB data, but it's not really clear if GnuTLS supports that.
Yeah I get the impression that is fairly experimental stuff. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Daniel P. Berrange wrote:
The daemon has a buffer of outgoing data - and it appends to this buffer discrete 'XDR' messages. So we should be able to simply append another XDR message per notification & not have to overly worry about things being mixed up in the stream. When the client is deserializing replies during normal calls it'll have to keep an eye out for messages which are in fact notifications, rather than its expected reply, but again that should not be too hard since we're dealing with discrete XDR encoded messages. So AFAICT we don't need any OOB or piggybacking stuff - just queue up the data to send as normal.
Obviously at the moment, remote calls are completely synchronous on the client side. remote_internal.c:call() is a function which serialises the args, writes them to the socket, then sits there waiting for a reply. I guess we're not planning to make this asynchronous? (Or are we??) Unless we change remote_internal.c, events which happen outside of calls won't be delivered upwards asynchronously, but only when the app happens to make a libvirt call. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Wed, Jul 18, 2007 at 04:13:38PM +0100, Richard W.M. Jones wrote:
Daniel P. Berrange wrote:
The daemon has a buffer of outgoing data - and it appends to this buffer discrete 'XDR' messages. So we should be able to simply append another XDR message per notification & not have to overly worry about things being mixed up in the stream. When the client is deserializing replies during normal calls it'll have to keep an eye out for messages which are in fact notifications, rather than its expected reply, but again that should not be too hard since we're dealing with discrete XDR encoded messages. So AFAICT we don't need any OOB or piggybacking stuff - just queue up the data to send as normal.
Obviously at the moment, remote calls are completely synchronous on the client side. remote_internal.c:call() is a function which serialises the args, writes them to the socket, then sits there waiting for a reply. I guess we're not planning to make this asynchronous? (Or are we??)
It'll be a little tricky, but I don't think we'd want to make it asynchronous. If we just adapt it so that if the packet is gets back is a notification we put it aside & read another packet for its real reply. Once its got the real reply then it can dispatch any notifications it queued up. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Wed, Jul 18, 2007 at 03:18:03PM +0100, Daniel P. Berrange wrote:
On Wed, Jul 18, 2007 at 12:57:37PM +0100, Richard W.M. Jones wrote:
In the qemu case I'm not sure how to modify qemu_driver.c to handle this. One for Dan Berrange to comment on.
We should not explicitly expose 'the xenstore filehandle', or 'the libvirtd' filehandle per-se. Instead we should provide an API which lets application register callbacks for adding/removing file handles to an event loop.
We in fact already have this - look at the src/event.h file. So if we make this part of the official public API, then applications merely need provide an impl of the virEventAddHandleFunc and virEventRemoveHandleFunc. The underlying libvirt drivers can thus invoke these callbacks as required, be it with the XenStored filehandle, a XenAPI filehandle, a libvirtd fd or something else.
We can provide sample impls of the virEventAddHandleFunc and virEventRemoveHandleFunc code for GLib and/or QT in a libvirtglib.so and libvirtqt.so add-on libraries. This allows for easy integration with most GUI apps, while also allowing integration with custom home brew event loops. This is how DBus provides event loop integration without tieing itself to a specific impl & it works very nicely.
Hum, on one hand you have a simple "here is an fd, register it and call this when there is something to read" and on the other hand there is an indirect callback architecture, with possibly multiple shared libs. Isn't this a bit over the top ? Plus I really prefer to have an API not based on callbacks but on synchronous calls driven by the application. My own experience is that it is way more comfortable with the later at least for most programmers (e.g. SAX vs. Reader XML APIs). Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Wed, Jul 18, 2007 at 10:46:02AM -0400, Daniel Veillard wrote:
On Wed, Jul 18, 2007 at 03:18:03PM +0100, Daniel P. Berrange wrote:
On Wed, Jul 18, 2007 at 12:57:37PM +0100, Richard W.M. Jones wrote:
In the qemu case I'm not sure how to modify qemu_driver.c to handle this. One for Dan Berrange to comment on.
We should not explicitly expose 'the xenstore filehandle', or 'the libvirtd' filehandle per-se. Instead we should provide an API which lets application register callbacks for adding/removing file handles to an event loop.
We in fact already have this - look at the src/event.h file. So if we make this part of the official public API, then applications merely need provide an impl of the virEventAddHandleFunc and virEventRemoveHandleFunc. The underlying libvirt drivers can thus invoke these callbacks as required, be it with the XenStored filehandle, a XenAPI filehandle, a libvirtd fd or something else.
We can provide sample impls of the virEventAddHandleFunc and virEventRemoveHandleFunc code for GLib and/or QT in a libvirtglib.so and libvirtqt.so add-on libraries. This allows for easy integration with most GUI apps, while also allowing integration with custom home brew event loops. This is how DBus provides event loop integration without tieing itself to a specific impl & it works very nicely.
Hum, on one hand you have a simple "here is an fd, register it and call this when there is something to read" and on the other hand there is an indirect callback architecture, with possibly multiple shared libs. Isn't this a bit over the top ? Plus I really prefer to have an API not based on callbacks but on synchronous calls driven by the application.
Because this assumes that a single FD is all you'll ever need to monitor which isn't neccessarily the case with all drivers. The add/remove handle funcs let the driver provide the application with FD as & when they are needed. From what I understand of XenAPI, the way they provide state notifications requires multiple FDs, and we shouldn't artifically limit ourselves since there's many other virt systems we still may support in the future which may have have a single pre-determined 'fd'. The callbacks really aren't that hard to understand and we already have this stuff in the internal API since we need it for QEMU driver, so we wouldn't be adding any new code to libvirt in that regard - merely reusing existing infrastructure.
My own experience is that it is way more comfortable with the later at least for most programmers (e.g. SAX vs. Reader XML APIs).
Not exactly a great comparison, as SAX / XML Readers aren't trying to monitor an arbitrary external dynamically changing state system. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Wed, Jul 18, 2007 at 10:56:42AM +0100, Richard W.M. Jones wrote:
I had some private discussions with a developer who wants to start using libvirt, but requires asynch notifications of domain start/stop (and other stuff, but this will do for the moment). Right now he's using XenStore watches. The alternative of polling didn't look very appealing for him.
A rough API for this might look like this:
typedef int virEventHandle;
/* Register a callback to catch domain start events. */ virEventHandle virEventDomainStart (virConnectPtr, void (*f) (virConnectPtr, int domid));
/* Register a callback to catch domain stop events. */ virEventHandle virEventDomainStop (virConnectPtr, void (*f) (virConnectPtr, int domid));
/* Unregister any type of callback. */ int virEventCancel (virConnectPtr, virEventHandle handle);
Yuk - this is going to give an ever growing number of callbacks - there are many interesting states besides Start & Start. Pause, resume, save, restore. Probably want something that takes void (*virDomainStateNotify)(virDomainPtr dom, int state); The 'state' value could be any virDomainState constant - though we should not notify for changes betweeen 'nostate', 'running' and 'blocked' because they all effectively just mean 'running' and Xen domains ossicilate between them many times a second. Or maybe define a different set of 'state' transitions to represent the actiohn triggered, rather than the resulting state ? Any you could register with: virDomainRegisterStateNotify(virDomainPtr, virDomainStateNotify); virDomainRegisterStateNotify(virDomainPtr); A pre-requisite of using these notifications would be that the application had registered an event loop implementation, as per http://www.redhat.com/archives/libvir-list/2007-May/msg00083.html See src/event.c and srv/event.h for actual impls we already have. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
participants (4)
-
Daniel P. Berrange -
Daniel Veillard -
Gerd Hoffmann -
Richard W.M. Jones