[RFC PATCH 0/3] qemu: Propagate cluster size to new qcow2 images
Cluster size may have performatnce implications. Propagate the cluster size used by the original image to any copy or overlay. Peter Krempa (3): qemu: monitor: Detect image cluster size from 'query-named-block-nodes' qemu: block: Allow specifying cluster size when using 'blockdev-create' qemuBlockStorageSourceCreateDetectSize: Propagate cluster size for 'qcow2' src/qemu/qemu_block.c | 6 ++++++ src/qemu/qemu_monitor.h | 3 +++ src/qemu/qemu_monitor_json.c | 3 +++ src/util/virstoragefile.h | 1 + 4 files changed, 13 insertions(+) -- 2.26.2
Configuring the cluster size of an image may have performance implications. This patch allows us to detect cluster size for existing images so that we will be able to propagate it to new images which are based on existing images e.g. during snapshots/block-copy/etc. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_monitor.h | 3 +++ src/qemu/qemu_monitor_json.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index d20a15c202..369d63ce12 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -696,6 +696,9 @@ struct _qemuBlockNamedNodeData { qemuBlockNamedNodeDataBitmapPtr *bitmaps; size_t nbitmaps; + + /* the cluster size of the image is valid only when > 0 */ + unsigned long long clusterSize; }; virHashTablePtr diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index 9cdf6c0f7f..e6d2e7d4db 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -2997,6 +2997,9 @@ qemuMonitorJSONBlockGetNamedNodeDataWorker(size_t pos G_GNUC_UNUSED, if (virJSONValueObjectGetNumberUlong(img, "actual-size", &ent->physical) < 0) ent->physical = ent->capacity; + /* try looking up the cluster size */ + ignore_value(virJSONValueObjectGetNumberUlong(img, "cluster-size", &ent->clusterSize)); + if ((bitmaps = virJSONValueObjectGetArray(val, "dirty-bitmaps"))) qemuMonitorJSONBlockGetNamedNodeDataBitmaps(bitmaps, ent); -- 2.26.2
'blockdev-create' allows us to create the image with a custom cluster size if we wish to. Wire it up for 'qcow2'. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_block.c | 1 + src/util/virstoragefile.h | 1 + 2 files changed, 2 insertions(+) diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index 26c1b42428..0113c64f0f 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -2315,6 +2315,7 @@ qemuBlockStorageSourceCreateGetFormatPropsQcow2(virStorageSourcePtr src, "s:file", src->nodestorage, "U:size", src->capacity, "S:version", qcow2version, + "P:cluster-size", src->clusterSize, NULL) < 0) return -1; diff --git a/src/util/virstoragefile.h b/src/util/virstoragefile.h index f73b3ee005..87763cf389 100644 --- a/src/util/virstoragefile.h +++ b/src/util/virstoragefile.h @@ -318,6 +318,7 @@ struct _virStorageSource { unsigned long long capacity; /* in bytes, 0 if unknown */ unsigned long long allocation; /* in bytes, 0 if unknown */ unsigned long long physical; /* in bytes, 0 if unknown */ + unsigned long long clusterSize; /* in bytes, 0 if unknown */ bool has_allocation; /* Set to true when provided in XML */ size_t nseclabels; -- 2.26.2
Propagate the cluster size from the original image as the user might have configured a custom cluster size for performance reasons. Propagate the cluster size of a qcow2 image to the new overlay or copy. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_block.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index 0113c64f0f..6e8fe545b4 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -2789,6 +2789,11 @@ qemuBlockStorageSourceCreateDetectSize(virHashTablePtr blockNamedNodeData, return -1; } + /* propagate cluster size if the images are compatible */ + if (templ->format == VIR_STORAGE_FILE_QCOW2 && + src->format == VIR_STORAGE_FILE_QCOW2) + src->clusterSize = entry->clusterSize; + if (src->format == VIR_STORAGE_FILE_RAW) { src->physical = entry->capacity; } else { -- 2.26.2
On 8/26/20 9:56 AM, Peter Krempa wrote:
Cluster size may have performatnce implications. Propagate the cluster size used by the original image to any copy or overlay.
Peter Krempa (3): qemu: monitor: Detect image cluster size from 'query-named-block-nodes' qemu: block: Allow specifying cluster size when using 'blockdev-create' qemuBlockStorageSourceCreateDetectSize: Propagate cluster size for 'qcow2'
It's not as flexible as allowing the user to customize sizes, but it definitely makes sense and is a sane heuristic, better than adccidentally creating an overlay with 64k clusters atop a base image with 1M clusters. Series: Reviewed-by: Eric Blake <eblake@redhat.com> -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On Wed, Aug 26, 2020 at 10:32:04 -0500, Eric Blake wrote:
On 8/26/20 9:56 AM, Peter Krempa wrote:
Cluster size may have performatnce implications. Propagate the cluster size used by the original image to any copy or overlay.
Peter Krempa (3): qemu: monitor: Detect image cluster size from 'query-named-block-nodes' qemu: block: Allow specifying cluster size when using 'blockdev-create' qemuBlockStorageSourceCreateDetectSize: Propagate cluster size for 'qcow2'
It's not as flexible as allowing the user to customize sizes, but it definitely makes sense and is a sane heuristic, better than adccidentally creating an overlay with 64k clusters atop a base image with 1M clusters.
Actually, I'll modify the condition in patch 3 to also check that the configured cluster size in the new image is 0 before applying one from the template image. That way it will be simpler to wire up configurable cluster size by just dealing with the XML.
Series: Reviewed-by: Eric Blake <eblake@redhat.com>
Thanks I'll push it soon.
participants (2)
-
Eric Blake -
Peter Krempa