Re: [libvirt] [RFC][scale] new API for querying domains stats
On Tue, Jul 01, 2014 at 03:09:13AM -0400, Francesco Romani wrote:
I'd like to discuss possible APIs and plans for new query APIs in
libvirt.
I'm one of the oVirt (http://www.ovirt.org) developers, and I write code for VDSM;
VDSM is the node management daemon, which is in charge, among many other things, to
gather the host and statistics per Domain/VM.
Right now we aim for a number of VM per node in the (few) hundreds, but we have big
plans
to scale much more, and to possibly reach thousands in a not so distant future.
At the moment, we use one thread per VM to gather the VM stats (CPU, network, disk),
and of course this obviously scales poorly.
I'll just note here that a bug has been opened for virt-top, which
is similar to this.
If a domain has a large number of disks (256 virtio-scsi disks in the
customer's case), then virt-top spends so long fetching the data for
each separate disk, it can take 30-40 seconds between updates.
The same thing would happen if you had lots of domains, each with a
few disks, but with the total adding up to hundreds of disks.
The same thing would happen if you substitute network interfaces for disks.
What would help for us:
- A way to get information for multiple objects in a single domain
- A way to get information for multiple objects across multiple domains
in as few API round trips as possible.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/