Re: [libvirt] [RFC][scale] new API for querying domains stats

> > This is made only worse by the fact that VDSM is a python 2.7 application, > > and notoriously > > python 2.x behaves very badly with threads. We are already working to > > improve our code, > > but I'd like to bring the discussion here and see if and when the querying > > API can be improved. > > > > We currently use these APIs for our sempling: > > virDomainBlockInfo > > virDomainGetInfo > > virDomainGetCPUStats > > virDomainBlockStats > > virDomainBlockStatsFlags > > virDomainInterfaceStats > > virDomainGetVcpusFlags > > virDomainGetMetadata > > > > What we'd like to have is > > > > * asynchronous APIs for querying domain stats > > (https://bugzilla.redhat.com/show_bug.cgi?id=1113106) > > This would be just awesome. Either a single callback or a different one > > per call is fine > > (let's discuss this!). > > please note that we are much more concerned about thread reduction then > > about performance > > numbers. We had report of thread number becoming a real harm, while > > performance so far > > is not yet a concern > > (https://bugzilla.redhat.com/show_bug.cgi?id=1102147#c54) > > I'm not a big fan of this approach. I mean, IIRC python has this Big > Python Lock, which effectively prevents two threads run concurrently. It has the GIL, yes. Only one thread can run python code at any given time. This however it is not true for extensions modules written in C which if carefully designed (read: coded to properly release the GIL) can run concurrently. This is one of the reasons while threading in python it is tolerated for I/O, evne though never recommended. AFAIK/IIRC the code the libvirt module for python allows this, so we should be good to go.