[libvirt-users] Shouldn't vol-upload / virStorageVolUpload() be doing some format conversion?
Hi, I just tried using vol-update to copy an image file to a storage pool, and I noticed that, at least in my configuration (qemu backend, storage pool of type "dir"), this command does not appear to be doing any kind of format conversion. It simply copied the source file as-is over the file in which the target volume was stored with no checks whether that even makes sense. The help text to this command says it supports offset and length, which - kinda, sorta - implies that I can write a block of bytes at offset n to a volume, attach that volume to a domain, and then the OS in the guest domain will find exactly the bytes I just wrote at exactly offset n in its newly attached harddisk/CDROM/whatever. Instead, trying to this with anything but raw images would probably just break the image file. AFAICS, in its current form, this command and this API-call are only useful in exactly two scenarios: - Both the source and target image are in raw format - Source and target image are in the same non-raw format (like qcow2 or vmdk) AND you are copying the entire volume at once. In all other cases - source and target are in different formats, source and target are in non-raw and you are trying to write only parts of a volume - this can only lead to a broken image file. As far as I see it, at the layer of abstraction libvirt offers, "volume" should not be synonymous with "the file the file the volume is stored in on disk", but rather with "the volume as a guest machine will get to see it". Regards, Guido
On 04/12/2013 12:06 PM, Guido Winkelmann wrote:
Hi,
I just tried using vol-update to copy an image file to a storage pool, and I noticed that, at least in my configuration (qemu backend, storage pool of type "dir"), this command does not appear to be doing any kind of format conversion. It simply copied the source file as-is over the file in which the target volume was stored with no checks whether that even makes sense.
The help text to this command says it supports offset and length, which - kinda, sorta - implies that I can write a block of bytes at offset n to a volume, attach that volume to a domain, and then the OS in the guest domain will find exactly the bytes I just wrote at exactly offset n in its newly attached harddisk/CDROM/whatever. Instead, trying to this with anything but raw images would probably just break the image file.
AFAICS, in its current form, this command and this API-call are only useful in exactly two scenarios:
- Both the source and target image are in raw format
- Source and target image are in the same non-raw format (like qcow2 or vmdk) AND you are copying the entire volume at once.
Yeah, and I'd LOVE to enhance that by use of flags, so that we can allow raw->formatted and formatted->raw conversion by adding a couple of flags (that is, are we uploading the offset in relation to the HOST's view of the file, or in relation to the GUEST's view of the file). Going from format A to format B would be a matter of hooking up a volumeDownload from one format to a volumeUpload in the other format, so it might also make sense to enhance the virStream API to make it easier to pipeline two streams end-to-end. But someone has to write the patches.
In all other cases - source and target are in different formats, source and target are in non-raw and you are trying to write only parts of a volume - this can only lead to a broken image file.
Usually, you want to be copying the offsets in relation to the guest's view of the storage; it is very seldom that you would want to copy the host's view of a non-raw file (except, as you note, when copying the entire file including metadata). The fact that the existing command treats offsets relative to the host's view is annoying; but for backward compatibility, I think we're stuck with it.
As far as I see it, at the layer of abstraction libvirt offers, "volume" should not be synonymous with "the file the file the volume is stored in on disk", but rather with "the volume as a guest machine will get to see it".
There's certainly lots of room for improvement in the virStorageVol* API. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Hi, I am considering starting a project libvirtvol under 2-clause BSD or maybe Boost License in order to facilitate things like this. The goal would be to have a consistent API to work with various hard disk image formats such as qcow2, vmdk or vhd, offering operations such as create, open, close, read, write, grow, shrink and discard (aka TRIM or hole-punching). Do you think this would be a good idea? Or do you maybe know of some other library that already solves this problem? Of course, I can't make any guarantees about how this will work out and how long it will take me. Guido
On 04/15/2013 11:52 AM, Guido Winkelmann wrote:
Hi,
I am considering starting a project libvirtvol under 2-clause BSD or maybe Boost License in order to facilitate things like this. The goal would be to have a consistent API to work with various hard disk image formats such as qcow2, vmdk or vhd, offering operations such as create, open, close, read, write, grow, shrink and discard (aka TRIM or hole-punching).
Do you think this would be a good idea? Or do you maybe know of some other library that already solves this problem?
libqblock was proposed in the qemu lists, although at the moment it has some flaws (it defers to qemu's default behavior of glib memory management, which is to abort() rather than gracefully report OOM errors back to the caller, which is rather disappointing for any library-quality API). Discussion on that library has stalled, though, since the last real post on the topic was in February: https://lists.gnu.org/archive/html/qemu-devel/2013-02/msg05022.html -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
participants (2)
-
Eric Blake -
Guido Winkelmann