On Mon, Dec 07, 2015 at 02:46:59PM +0100, Michal Privoznik wrote:
Dear list,
I'd like to hear your opinion on the following bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1282859
Long story short, Imagine the following scenario:
1. Create 4GB file full of zeroes
2. virsh vol-download it
What happens is that all those 4GB are transferred byte after byte
through our RPC system. Not only this puts needles pressure on our event
loop, it's suboptimal for network and other resources too.
I'd like to explore our options here keeping in mind that the original
volume might have been sparse and we ought to keep it sparse on the
destination too.
In the bug the reporter (Matthew Booth) suggests introducing new type of
RPC message that will let us keep our APIs unchanged. The source will
scan the file for windows of zeroes bigger than some value. When found
the new type of message is passed to client without need to copy those
zeroes. Yes, this is very similar to RLE.
If we are going that way, should we enable users to put a compression
program in between read()/write() and our RPC? Well, should we let users
to choose what compression program we will put there? Because there are
better compression algorithms than RLE.
It only looks like compression if you're solely looking at the network
data transfer. A keep feature of sparse support is that we preserve
the sparseness on both sides.
ie, if I have a sparse raw file locally, and vol-upload it, it should
remain a sparse file on the server. Likewise vol-downloading a sparse
file should let me create a sparse file locally. For this reason the
RPC program must explicitly represent data holes, and not merely
consider them a type of compression algorithm, as that would not let
us preserve the holes on both ends of the stream.
Regards,
Daniel
--
|:
http://berrange.com -o-
http://www.flickr.com/photos/dberrange/ :|
|:
http://libvirt.org -o-
http://virt-manager.org :|
|:
http://autobuild.org -o-
http://search.cpan.org/~danberr/ :|
|:
http://entangle-photo.org -o-
http://live.gnome.org/gtk-vnc :|