Re: [libvirt] [RFC][scale] new API for querying domains stats

From: "Richard W.M. Jones" <rjones(a)redhat.com&gt; To: "Francesco Romani" <fromani(a)redhat.com&gt; Cc: libvir-list(a)redhat.com Sent: Friday, July 4, 2014 1:11:54 PM Subject: Re: [libvirt] [RFC][scale] new API for querying domains stats

> Right now we aim for a number of VM per node in the (few) hundreds, but we > have big plans > to scale much more, and to possibly reach thousands in a not so distant > future. > At the moment, we use one thread per VM to gather the VM stats (CPU, > network, disk), > and of course this obviously scales poorly. I'll just note here that a bug has been opened for virt-top, which is similar to this. If a domain has a large number of disks (256 virtio-scsi disks in the customer's case), then virt-top spends so long fetching the data for each separate disk, it can take 30-40 seconds between updates. The same thing would happen if you had lots of domains, each with a few disks, but with the total adding up to hundreds of disks. The same thing would happen if you substitute network interfaces for disks. What would help for us: - A way to get information for multiple objects in a single domain - A way to get information for multiple objects across multiple domains in as few API round trips as possible.