General use of zstd instead of zlib compression
Hi, currently i use on all VM's qcow2-Images with zlib compression. If i do an Backup, the Backup-Image will be compressed with zstd Level 3 to shrink the image for transfering it over not so fast internet connections. So, why not directly using zstd compression on the images. Are there any reason's not to do that ? As i always use virt-manager for administration, i patched qemu (V9.2.2) to create on default zstd compressed images (only 1 change in line 3525). So newly created images do have compression type zstd, which work's (qemi-img info). I see one unusual thing. If i do an qemi-img convert with compression_type=zstd the size of the converted image is much smaller than the original file while "qemu-img info" shows on both as compression type zstd. Do they use different compression levels, maybe ? If i now do an virsh backup-begin <domain>, the backup-Image does also has a bigger size than the original, while showing zstd as compression type (qemu-img info). If i convert it with the similar command as above, both converted images has nearly the same size. Even if i copy the smaller converted image to the original and boot the vm from the smaller image, the backup-image (after backup-begin) is bigger. So, i am confused. Are there any explanations about the different image sizes or what's going on here ? best regards Michael
On Fri, Mar 14, 2025 at 11:12:09 -0000, Michael Niehren wrote:
Hi,
currently i use on all VM's qcow2-Images with zlib compression. If i do an Backup, the Backup-Image will be compressed with zstd Level 3 to shrink the image for transfering it over not so fast internet connections.
Libvirt doesn't override the defaults that qemu selects when creating images.
So, why not directly using zstd compression on the images. Are there any reason's not to do that ?
One reason is compatibility. Libvirt wants the images to stay compatible thus we must not switch to anything new by default.
As i always use virt-manager for administration, i patched qemu (V9.2.2) to create on default zstd compressed images (only 1 change in line 3525). So newly created images do have compression type zstd, which work's (qemi-img info). I see one unusual thing. If i do an qemi-img convert with compression_type=zstd the size of the converted image is much smaller than the original file while "qemu-img info" shows on both as compression type zstd. Do they use different compression levels, maybe ?
Setting the compression algorithm on the qcow2 image doesn't mean that qemu will write compressed data into it. This is controlled either by the use of 'filter-compress' blockdev layer or e.g. the compress parameter of the backup job.
If i now do an virsh backup-begin <domain>, the backup-Image does also has a bigger size than the original, while showing zstd as compression type (qemu-img info). If i convert it with the similar command as above, both converted images has nearly the same size. Even if i copy the smaller converted image to the original and boot the vm from the smaller image, the backup-image (after backup-begin) is bigger.
As noted above the fact that any compression algorithm is enabled doesn't mean that compressed data will be written. I don't see us ever wanting to enable compression for the main disk image for performance reasons. Said that makes likely sense to allow enabling compression for the backup job as an option and should be relatively straightforward to do. The more intricate bit will be wiring up XML for selecting non-default compression format for the image created by libvirt (otherwise '--reuse-external' and manual creation would be needed.
On Fri, Mar 14, 2025 at 13:11:00 +0100, Peter Krempa wrote:
On Fri, Mar 14, 2025 at 11:12:09 -0000, Michael Niehren wrote:
[...]
As noted above the fact that any compression algorithm is enabled doesn't mean that compressed data will be written.
I don't see us ever wanting to enable compression for the main disk image for performance reasons.
Said that makes likely sense to allow enabling compression for the backup job as an option and should be relatively straightforward to do.
Actually; since this would be a new flag we could also make it use zstd instead of the default for images libvirt creates. Although zstd support can be compiled out of qemu so we'd have to keep the documentation alibistic to allow both options and also wire up a capability flag.
The more intricate bit will be wiring up XML for selecting non-default compression format for the image created by libvirt (otherwise '--reuse-external' and manual creation would be needed.
An implementation only in the backup job would be great, that would fit my needs. Maybe some more flags in the backupxml file would be useful. I used for compressing on the zstd call "--sparse --rsyncable -B128 " as the files are later transfered with rsync. The compression level would also be useful to set. By the way, is the backup qcow2 file written from qemu or from libvirt, when calling backup-begin ?
On Fri, Mar 14, 2025 at 13:33:04 -0000, Michael Niehren wrote:
An implementation only in the backup job would be great, that would fit my needs.
Maybe some more flags in the backupxml file would be useful. I used for compressing on the zstd call "--sparse --rsyncable -B128 " as the files are later transfered with rsync. The compression level would also be useful to set.
qcow2 doesn't seem to support any other configuration options for compression: ## # @BlockdevCreateOptionsQcow2: # # Driver specific image creation options for qcow2. # # @file: Node to create the image format on # # @data-file: Node to use as an external data file in which all guest # data is stored so that only metadata remains in the qcow2 file # (since: 4.0) # # @data-file-raw: True if the external data file must stay valid as a # standalone (read-only) raw image without looking at qcow2 # metadata (default: false; since: 4.0) # # @extended-l2: True to make the image have extended L2 entries # (default: false; since 5.2) # # @size: Size of the virtual disk in bytes # # @version: Compatibility level (default: v3) # # @backing-file: File name of the backing file if a backing file # should be used # # @backing-fmt: Name of the block driver to use for the backing file # # @encrypt: Encryption options if the image should be encrypted # # @cluster-size: qcow2 cluster size in bytes (default: 65536) # # @preallocation: Preallocation mode for the new image (default: off; # allowed values: off, falloc, full, metadata) # # @lazy-refcounts: True if refcounts may be updated lazily # (default: off) # # @refcount-bits: Width of reference counts in bits (default: 16) # # @compression-type: The image cluster compression method # (default: zlib, since 5.1) # # Since: 2.12 ## { 'struct': 'BlockdevCreateOptionsQcow2', 'data': { 'file': 'BlockdevRef', '*data-file': 'BlockdevRef', '*data-file-raw': 'bool', '*extended-l2': 'bool', 'size': 'size', '*version': 'BlockdevQcow2Version', '*backing-file': 'str', '*backing-fmt': 'BlockdevDriver', '*encrypt': 'QCryptoBlockCreateOptions', '*cluster-size': 'size', '*preallocation': 'PreallocMode', '*lazy-refcounts': 'bool', '*refcount-bits': 'int', '*compression-type':'Qcow2CompressionType' } }
By the way, is the backup qcow2 file written from qemu or from libvirt, when calling backup-begin ?
Normally libvirt creates an empty file, applies security labelling for qemu to be able to access it. Then qemu is instructed to open the backing file (blockdev-add), format it with a qcow2 header (blockdev-create) and then start the backup job (transaction + blockdev-backup). In case '--reuse-external' flag is used, libvirt expects that the file was pre-created by the user with proper metadata (most importantly proper size + whatever configuration the user wants). Libvirt applies security labels and then uses blockdev-add + (transaction + blockdev-backup) to do the backup. The actual backup is always done by qemu in push mode. In pull mode the same as above happens for the 'scratch' temporary file needed to do the backup. libvirt then instructs qemu to start a NBD server where the user can pull the backup from.
ok, so only setting the compression type to zstd in the backupxml would be possible from libvirt, right ?
one more ... what would happen, if i create the qcow2 image with zstd compression enabled and use the '--reuse-external' flag ? Would qemu write compressed data in it or not ?
On Fri, Mar 14, 2025 at 14:18:03 -0000, Michael Niehren wrote:
one more ...
what would happen, if i create the qcow2 image with zstd compression enabled and use the '--reuse-external' flag ? Would qemu write compressed data in it or not ?
It would not be compressed, as I've explained earlier (On technical lists it's preferred if context from what you're replying to is not trimmed so that it can be reffred back to) actual compression of blocks needs to be requested explicitly. The qcow2 header setting merely states which algorithm should be used if blocks are compressed.
On Fri, Mar 14, 2025 at 03:34:46PM +0100, Peter Krempa wrote:
On Fri, Mar 14, 2025 at 14:18:03 -0000, Michael Niehren wrote:
one more ...
what would happen, if i create the qcow2 image with zstd compression enabled and use the '--reuse-external' flag ? Would qemu write compressed data in it or not ?
It would not be compressed, as I've explained earlier (On technical lists it's preferred if context from what you're replying to is not trimmed so that it can be reffred back to) actual compression of blocks needs to be requested explicitly. The qcow2 header setting merely states which algorithm should be used if blocks are compressed.
Also, AFAIK, qemu will not write compressed data during "normal" usage by a running VM. The compression only gets applied during full image conversion processes. iow, 'qemu-img convert' can be used to create a compressed image, from a non-compressed image. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, Mar 17, 2025 at 09:31:55 +0000, Daniel P. Berrangé wrote:
On Fri, Mar 14, 2025 at 03:34:46PM +0100, Peter Krempa wrote:
On Fri, Mar 14, 2025 at 14:18:03 -0000, Michael Niehren wrote:
one more ...
what would happen, if i create the qcow2 image with zstd compression enabled and use the '--reuse-external' flag ? Would qemu write compressed data in it or not ?
It would not be compressed, as I've explained earlier (On technical lists it's preferred if context from what you're replying to is not trimmed so that it can be reffred back to) actual compression of blocks needs to be requested explicitly. The qcow2 header setting merely states which algorithm should be used if blocks are compressed.
Also, AFAIK, qemu will not write compressed data during "normal" usage by a running VM. The compression only gets applied during full image
So there is the 'compress' filter in qemu although based on the only bit of "documentation" it has (commit message) QCOW2 supports compressed writes only for previously unallocated clusters so in fact it is used only for backups IIUC.
So to archive my goal to have an zstd compressed backup image, i can currently not use the new approach with backup-begin ? Instead i have to use the old approach with the overlay file and use qemu-img convert compression_type=zstd ... for doing the backup and if finished blockcommit the overlay file. am i right ?
participants (3)
-
Daniel P. Berrangé -
Michael Niehren -
Peter Krempa