Re: [libvirt] [RFC][scale] new API for querying domains stats

On Fri, Jul 04, 2014 at 12:11:54PM +0100, Richard W.M. Jones wrote: > On Tue, Jul 01, 2014 at 03:09:13AM -0400, Francesco Romani wrote: > > I'd like to discuss possible APIs and plans for new query APIs in libvirt. > > > > I'm one of the oVirt (http://www.ovirt.org) developers, and I write code for VDSM; > > VDSM is the node management daemon, which is in charge, among many other things, to > > gather the host and statistics per Domain/VM. > > > > Right now we aim for a number of VM per node in the (few) hundreds, but we have big plans > > to scale much more, and to possibly reach thousands in a not so distant future. > > At the moment, we use one thread per VM to gather the VM stats (CPU, network, disk), > > and of course this obviously scales poorly. > > I'll just note here that a bug has been opened for virt-top, which > is similar to this. > > If a domain has a large number of disks (256 virtio-scsi disks in the > customer's case), then virt-top spends so long fetching the data for > each separate disk, it can take 30-40 seconds between updates. > > The same thing would happen if you had lots of domains, each with a > few disks, but with the total adding up to hundreds of disks. > > The same thing would happen if you substitute network interfaces for disks. > > What would help for us: > > - A way to get information for multiple objects in a single domain > > - A way to get information for multiple objects across multiple domains I'd say that we want something similar to the virDomainListAllDomains() API for stats. ie we shouldn't try to pass in the full list of domains or paths we want info for. We should just list all domains, optionally using flags to filter based on some characteristic, eg exclude inactive. Similarly always list stats for all disks.