Re: [libvirt] [RFC][scale] new API for querying domains stats

From: "Daniel P. Berrange" <berrange(a)redhat.com&gt; To: "Francesco Romani" <fromani(a)redhat.com&gt; Cc: libvir-list(a)redhat.com Sent: Tuesday, July 1, 2014 10:35:21 AM Subject: Re: [libvirt] [RFC][scale] new API for querying domains stats

> We [in VDSM] currently use these APIs for our sempling: > virDomainBlockInfo > virDomainGetInfo > virDomainGetCPUStats > virDomainBlockStats > virDomainBlockStatsFlags > virDomainInterfaceStats > virDomainGetVcpusFlags > virDomainGetMetadata Why do you need to call virDomainGetMetadata so often ? That merely contains a opaque data blob that can only have come from VDSM itself, so I'm surprised you need to call that at all frequently.

> please note that we are much more concerned about thread reduction then > about performance numbers. We had report of thread number becoming a > real harm, while performance so far is not yet a concern > (https://bugzilla.redhat.com/show_bug.cgi?id=1102147#c54) > > * bulk APIs for querying domain stats > (https://bugzilla.redhat.com/show_bug.cgi?id=1113116) > would be really welcome as well. It is quite independent from the > previous bullet point > and would help us greatly with scale. If we did the first bullet point, we'd be adding another ~10 APIs for async variants. If we then did the second bullet point we'd be adding another ~10 APIs for bulk querying. So while you're right that they are independent, it would be desirable to address them both at the same time, so we only need to add 10 new APIs in total, not 20.

For the async API design, I could see two potential designs 1. A custom callback to run per API typedef (void)(*virDomainBlockInfoCallback)(virDomainPtr dom, bool isError, virDomainBlockInfoPtr info, void *opaque); int virDomainGetBlockInfoAsync(virDomainPtr dom, const char *disk, virDomainBlockInfoCallback cb, void *opaque, unsigned int flags); 2. A standard callback and a pair of APIs typedef void *virDomainAsyncResult; typedef (void)(*virDomainAsyncCallback)(virDomainPtr dom, virDomainAsyncResult res); void virDomainGetBlockInfoAsync(virDomainPtr dom, const char *disk, virDomainBlockInfoCallback cb, void *opaque, unsigned int flags); int virDomainGetBlockInfoFinish(virDomainPtr dom, virDomainAsyncResult res, virDomainBlockInfoPtr info); This second approach is the way GIO works (see example in this page https://developer.gnome.org/gio/stable/GAsyncResult.html ). The main difference between them really is probably the way you get error reporting from the APIs. In the first example, libvirt would raise an error before it invoked the callback, with isError set to True. In the second example, the Finish() func would raise the error and return -1.