[libvirt] Semantic change in v4.3 slows down image creation and snapshots (on ZFS)
Hi, when discussing bug [1] triggered by [2] Daniel and I have found that the change [3] which is in since v4.3 was a semantic change to the behavior of libvirt. virt-manager used and still does issue a volume XML like: <volume> <name>disk.qcow2</name> <capacity>16106127360</capacity> <allocation>16106127360</allocation> <target> <format type="qcow2"/> <features> <lazy_refcounts/> </features> </target> </volume> Up until the commit above that was using preallocation=metadata, but now uses preallocation=falloc. In many cases (read: Fileystems) that won't be very user-visible, but if you are running on ZFS (or anything else that can't fallocate well) this is made worse by [4] slowing down these operations a lot. After the POV for virt-manager seems to have settled on the bug [1] referenced the remaining question is if libvirt should revert/modify the change [3] - and that we agreed required a wider discussion to be had here, hence this mail. [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1759454 [2]: https://bugs.launchpad.net/virt-manager/+bug/1847105 [3]: https://libvirt.org/git/?p=libvirt.git;a=commit;h=c9ec7088c7a3f4cd26bb471f1f... [4]: https://github.com/zfsonlinux/zfs/issues/326 -- Christian Ehrhardt Software Engineer, Ubuntu Server Canonical Ltd
On Wed, Oct 09, 2019 at 13:39:54 +0200, Christian Ehrhardt wrote:
Hi, when discussing bug [1] triggered by [2] Daniel and I have found that the change [3] which is in since v4.3 was a semantic change to the behavior of libvirt.
virt-manager used and still does issue a volume XML like: <volume> <name>disk.qcow2</name> <capacity>16106127360</capacity> <allocation>16106127360</allocation> <target> <format type="qcow2"/> <features> <lazy_refcounts/> </features> </target> </volume>
Up until the commit above that was using preallocation=metadata, but now uses preallocation=falloc. In many cases (read: Fileystems) that won't be very user-visible, but if you are running on ZFS (or anything else that can't fallocate well) this is made worse by [4] slowing down these operations a lot.
Note that for raw files we would do preallocation for the full image but only when fast allocation is supported. See 'createRawFile'.
After the POV for virt-manager seems to have settled on the bug [1] referenced the remaining question is if libvirt should revert/modify the change [3] - and that we agreed required a wider discussion to be had here, hence this mail.
Semantically the change above would make sense if it weren't coupled to the presence of VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA. The flags semantics imply that metadata allocation is requested and thus metadata should be allocated. On the other hand doing full allocation if the flag is not present but the XML calls for it would semantically be okay IMO. I'm not sure if that is what you expect though.
On Wed, Oct 09, 2019 at 02:25:09PM +0200, Peter Krempa wrote:
On Wed, Oct 09, 2019 at 13:39:54 +0200, Christian Ehrhardt wrote:
Hi, when discussing bug [1] triggered by [2] Daniel and I have found that the change [3] which is in since v4.3 was a semantic change to the behavior of libvirt.
virt-manager used and still does issue a volume XML like: <volume> <name>disk.qcow2</name> <capacity>16106127360</capacity> <allocation>16106127360</allocation> <target> <format type="qcow2"/> <features> <lazy_refcounts/> </features> </target> </volume>
Up until the commit above that was using preallocation=metadata, but now uses preallocation=falloc. In many cases (read: Fileystems) that won't be very user-visible, but if you are running on ZFS (or anything else that can't fallocate well) this is made worse by [4] slowing down these operations a lot.
Note that for raw files we would do preallocation for the full image but only when fast allocation is supported. See 'createRawFile'.
Hmm, yes, i see we report a fatal error if fallocate() does work, which is good. Annoyingly QEMU uses posix_fallocate() which automagically falls back to writing zeros, which is very bad. AFAIK it isn't possible to ask QEMU to allocate only if it is fast :-(
After the POV for virt-manager seems to have settled on the bug [1] referenced the remaining question is if libvirt should revert/modify the change [3] - and that we agreed required a wider discussion to be had here, hence this mail.
Semantically the change above would make sense if it weren't coupled to the presence of VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA.
The flags semantics imply that metadata allocation is requested and thus metadata should be allocated.
On the other hand doing full allocation if the flag is not present but the XML calls for it would semantically be okay IMO.
Yeah, I agree with this.
I'm not sure if that is what you expect though.
virt-manager passes VOL_CREATE_PREALLOC_METADATA, so honouring that flag correctly would fix the problem reported with ZFS. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Wed, Oct 9, 2019 at 8:33 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
On Wed, Oct 09, 2019 at 02:25:09PM +0200, Peter Krempa wrote:
On Wed, Oct 09, 2019 at 13:39:54 +0200, Christian Ehrhardt wrote:
Up until the commit above that was using preallocation=metadata, but now uses preallocation=falloc.
Semantically the change above would make sense if it weren't coupled to the presence of VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA.
The flags semantics imply that metadata allocation is requested and thus metadata should be allocated.
On the other hand doing full allocation if the flag is not present but the XML calls for it would semantically be okay IMO.
Yeah, I agree with this.
I'm not sure if that is what you expect though.
virt-manager passes VOL_CREATE_PREALLOC_METADATA, so honouring that flag correctly would fix the problem reported with ZFS.
I know that this is an old message, but I'm just getting around to updating some of my libvirt-python scripts and I'm encountering a similar problem to virt-manager when using libvirt >=4.3, without the ZFS part. Prior to libvirt v4.3, my script created qcow2-format StorageVolumes with a <capacity/> element, and no <allocation/> element. It used the flag VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA to request metadata-only prealloc, which the documentation of virStorageVolCreateXml describes as creating a sparse file:
Since 1.0.1 VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA in flags can be used to get higher performance with qcow2 image files which don't support full preallocation, by creating a sparse image file with metadata.
The patch identified in this thread does not create a sparse file even in the presence of this flag when the requested capacity is equal to the requested allocation. You could just update the documentation to match current behavior, but I think that this is a bug and that qemu-img's prealloc=metadata option should always be used when VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA is specified. It also seems that the documentation is out-of-date in regards to qcow2 not supporting full preallocation. As mentioned previously, using the StorageVolume's <allocation /> element as an input does not work for qcow2. The requested allocation size is never honored by qemu-img. There is no concept of partial preallocation. There is only full prealloc and sparse metadata prealloc. Thanks, Mike
participants (4)
-
Christian Ehrhardt -
Daniel P. Berrangé -
Mike Haboustak -
Peter Krempa