Re: [libvirt] [PATCH v2] lib: Add note that bulk stats API queries may overrun RPC buffers

On Tue, May 23, 2017 at 05:29:52PM +0200, Peter Krempa wrote: > On Tue, May 23, 2017 at 17:19:53 +0200, Martin Kletzander wrote: >> On Tue, May 23, 2017 at 05:07:40PM +0200, Michal Privoznik wrote: >> > On 05/23/2017 04:35 PM, Martin Kletzander wrote: >> > > On Tue, May 23, 2017 at 04:23:30PM +0200, Peter Krempa wrote: > > [...] > >> > > > + * Note that this API is prone to exceeding maximum RPC if >> querying >> > > > too many VMs >> > > > + * with lots of statistics. It's suggested to query in batches of >> > > > 10VMs, which >> > > > + * should be good enough for VMs with 3000 disks + networks. >> > > > + * >> > > >> > > Coming to think about it... Why don't we just batch this >> ourselves under >> > > the hood and just return the merged result? >> > >> > Because: >> > >> > https://www.redhat.com/archives/libvir-list/2017-May/msg00088.html >> > >> >> Not on the RPC level, the API would just be syntax sugar to >> virDomainListGetStats() if a flag was passed >> (e.g. >> VIR_DOMAIN_GET_ALL_STATS_I_DONT_REALLY_CARE_IF_THIS_IS_DONE_IN_ONE_LIBVIRT_CALL) >> > > Also compared to a full fragmentation of the returned data, this would > result into a worst-case-scenario memory usage of MAX_SIZE * > NVMS_QUERIED_IN_ORIGINAL_CALL, when compared to an unbounded memory use > of the full fragmentation approach. If I get what you are saying, then the same would happen if the mgmt app (or client) implemented it themselves. We would basically just provide the guessing logic.