On Mon, Apr 22, 2013 at 06:18:04PM +0200, Viktor Mihajlovski wrote:
On 04/22/2013 04:34 PM, Daniel P. Berrange wrote:
>On Mon, Apr 22, 2013 at 04:22:26PM +0200, Viktor Mihajlovski wrote:
>I'm not against this in general, but before we enlarge this so much
>there needs to be some code work to make RPC message encoding more
>efficient. Currently virNetMessageEncodeHeader will allocate
>VIR_NET_MESSAGE_MAX immediately, regardless of whether the message
>being encoded acutally needs so much. The problem is that we can't
>tell ahead of time how much space a message needs. We just can't go
>on doing this if we're going to have such large max message size.
>We need to start with small allocs & grow if we hit the size limit.
So something along the lines of allocating VIR_NET_MESSAGE_INITIAL
bytes, maybe well below 1M, and then bumping it in the
EncodePayloadxxx calls if required. Do you foresee any pitfalls
one should consider?
We have an ever increasing variability in RPC message size - some of
them will remain absolutely tiny - 24 bytes in size, while others can
be huge. Deciding on the optimal initial allocation size is a tradeoff
between the overheads of frequently re-alloc'ing to enlarge the buffer,
vs the overhead of allocting too much memory and/or calling memset()
on the buffers.
I'd suggest an initial size of just a perhaps as small as a few 10's
of KB ? Then do a loop doubling allocation each time until it hits
the hard limit. In cases where we even up triggering the re-alloc
loop many times, we'll presumably be on large hardware with many
many VMs, so the overhead of the re-allocs will be dwarved by all
the other activity on the machine.
Daniel
--
|:
http://berrange.com -o-
http://www.flickr.com/photos/dberrange/ :|
|:
http://libvirt.org -o-
http://virt-manager.org :|
|:
http://autobuild.org -o-
http://search.cpan.org/~danberr/ :|
|:
http://entangle-photo.org -o-
http://live.gnome.org/gtk-vnc :|