Re: [libvirt] [RFC PATCH 2/2] lib: Add note that bulk stats API queries may overrun RPC buffers
On Tue, May 23, 2017 at 10:04:44 +0100, Daniel Berrange wrote:
On Tue, May 23, 2017 at 10:51:47AM +0200, Martin Kletzander wrote:
> On Tue, May 23, 2017 at 09:32:03AM +0100, Daniel P. Berrange wrote:
> > On Mon, May 22, 2017 at 06:00:13PM +0200, Peter Krempa wrote:
> > > Hint that the users should limit the number of VMs queried in the bulk
> > > stats API.
[...]
> Or we can give a hint on what the limit is and let users figure
out the
> sensible number. It is not only based on the number of VMs. I believe
> you can hit the limit with one or two VMs if you have lot of devices
> that we report statistics for. Limiting the number of VMs to a
> particular number would not help as much in this case. But we can
> combine both approaches.
Urgh, that's even worse. Apps can't simply split queries into blocks of
N vms, because any single VM in that list might have huge number of disks.
So to use this at all reliably you have to query the XML config of every
guest to see what devices are present and then split up the queries into
variable number of VMs :-(
This just adds to my feeling that we should consider this API a failed
experiment
Given this metric, any API in libvirt that takes XML is failed in the
same sense. You can easily have a VM that exceeds the 4MiB string RPC
limit (and 16, or 32) MiB in this sense anyways.
Along with metadata, you can reach the limit even easier.
A disk information dump returned with this API has slightly above 600
bytes, so let's say 1k, with this metric you can have a VM with 16k
disks/nics/whatever, which is reasonable. With the bump to 32MiB, you
can obviously have even more.
Since this API makes sense (saves time) even if called for a singe VM I
don't really think this is a big problem. If you say that the number of
VMs you should query is let's say 2 or 4, you can have giant guests.
The only non-scalable part is if you have lots of giant guests. You
can't really fix that.
Attachments:
- signature.asc (application/pgp-signature — 833 bytes)