[PATCH 0/6] qemu: Add detection of 'blockdev-reopen'
Note that the capability update patch is waiting for a series of fixes as current qemu master breaks with libvirt. Full version can be fetched from: git fetch https://gitlab.com/pipo.sk/libvirt.git blockdev-reopen Peter Krempa (6): tests: qemucapabilities: Update capabilities for qemu-6.1.0-rc0 on x86_64 qemuBlockReopenFormat: Extract @src handling and monitor invocation qemuBlockReopenFormatMon: Adapt to new argument format of 'blockdev-reopen' qemumonitorjsontest: Add test case for 'blockdev-reopen' qemu: capabilities: Enable detection of QEMU_CAPS_BLOCKDEV_REOPEN NEWS: Mention enablement of incremental backups NEWS.rst | 6 + src/qemu/qemu_block.c | 32 +- src/qemu/qemu_block.h | 5 + src/qemu/qemu_capabilities.c | 1 + .../domaincapsdata/qemu_6.1.0-q35.x86_64.xml | 2 +- .../domaincapsdata/qemu_6.1.0-tcg.x86_64.xml | 2 +- tests/domaincapsdata/qemu_6.1.0.x86_64.xml | 2 +- .../caps_6.1.0.x86_64.replies | 4245 +++++++++-------- .../caps_6.1.0.x86_64.xml | 8 +- tests/qemumonitorjsontest.c | 28 + 10 files changed, 2312 insertions(+), 2019 deletions(-) -- 2.31.1
Update to XXXXXXX (I'll update once paolos fixes are merged) Notable changes are: - stabilization of 'blockdev-reopen' - addition of the 'vmx-tsc-scaling' cpu flag - Supported display types are now in the schema only if they are compiled in. - rbd image encryption - 'aio-max-batch' iothread property Based on: https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg05922.html --- Please fetch the full version from the git repository. .../caps_6.1.0.x86_64.replies | 4245 +++++++++-------- .../caps_6.1.0.x86_64.xml | 6 +- 2 files changed, 2240 insertions(+), 2011 deletions(-) diff --git a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.replies b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.replies index 5fa8a07aa5..dac4f2fb2e 100644 --- a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.replies +++ b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.replies @@ -17,11 +17,11 @@ { "return": { "qemu": { - "micro": 50, + "micro": 90, "minor": 0, "major": 6 }, - "package": "v6.0.0-1910-gd0ac9a6147" + "package": "v6.1.0-rc0-47-g8e5c9da9b5" }, "id": "libvirt-2" } [...] diff --git a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml index a833dc9396..9d04a8b4a5 100644 --- a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml @@ -260,10 +260,10 @@ <flag name='virtio-vga-gl'/> <flag name='confidential-guest-support'/> <flag name='query-display-options'/> - <version>6000050</version> + <version>6000090</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100243</microcodeVersion> - <package>v6.0.0-1910-gd0ac9a6147</package> + <package>v6.1.0-rc0-47-g8e5c9da9b5</package> <arch>x86_64</arch> <hostCPU type='kvm' model='base' migratability='yes'> <property name='vmx-entry-load-rtit-ctl' type='boolean' value='false'/> @@ -285,6 +285,7 @@ <property name='bus-lock-detect' type='boolean' value='false'/> <property name='xsavec' type='boolean' value='true' migratable='yes'/> <property name='intel-pt' type='boolean' value='false'/> + <property name='vmx-tsc-scaling' type='boolean' value='false'/> <property name='vmx-cr8-store-exit' type='boolean' value='false'/> <property name='vmx-rdseed-exit' type='boolean' value='false'/> <property name='vmx-eptp-switching' type='boolean' value='false'/> @@ -1652,6 +1653,7 @@ <property name='bus-lock-detect' type='boolean' value='false'/> <property name='xsavec' type='boolean' value='false'/> <property name='intel-pt' type='boolean' value='false'/> + <property name='vmx-tsc-scaling' type='boolean' value='false'/> <property name='vmx-cr8-store-exit' type='boolean' value='false'/> <property name='vmx-rdseed-exit' type='boolean' value='false'/> <property name='vmx-eptp-switching' type='boolean' value='false'/> -- 2.31.1
On Fri, Jul 23, 2021 at 11:43:53 +0200, Peter Krempa wrote:
Update to XXXXXXX (I'll update once paolos fixes are merged)
Notable changes are: - stabilization of 'blockdev-reopen' - addition of the 'vmx-tsc-scaling' cpu flag - Supported display types are now in the schema only if they are compiled in. - rbd image encryption - 'aio-max-batch' iothread property
Based on:
https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg05922.html ---
Paolos series was pulled to qemu master. I've updated the commit message to: Update to v6.1.0-rc0-48-g7b7ca8ebde Notable changes are: - stabilization of 'blockdev-reopen' - addition of the 'vmx-tsc-scaling' cpu flag - Supported display types are now in the schema only if they are compiled in. - rbd image encryption - 'aio-max-batch' iothread property Signed-off-by: Peter Krempa <pkrempa@redhat.com>
This will simplify testing of the blockdev-reopen code once it's enabled. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_block.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index 6627d044cd..c96be8b52d 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -3299,6 +3299,22 @@ qemuBlockBitmapsHandleCommitFinish(virStorageSource *topsrc, } +static int +qemuBlockReopenFormatMon(qemuMonitor *mon, + virStorageSource *src) +{ + g_autoptr(virJSONValue) reopenprops = NULL; + + if (!(reopenprops = qemuBlockStorageSourceGetBlockdevProps(src, src->backingStore))) + return -1; + + if (qemuMonitorBlockdevReopen(mon, &reopenprops) < 0) + return -1; + + return 0; +} + + /** * qemuBlockReopenFormat: * @vm: domain object @@ -3316,7 +3332,6 @@ qemuBlockReopenFormat(virDomainObj *vm, { qemuDomainObjPrivate *priv = vm->privateData; virQEMUDriver *driver = priv->driver; - g_autoptr(virJSONValue) reopenprops = NULL; int rc; /* If we are lacking the object here, qemu might have opened an image with @@ -3327,13 +3342,10 @@ qemuBlockReopenFormat(virDomainObj *vm, return -1; } - if (!(reopenprops = qemuBlockStorageSourceGetBlockdevProps(src, src->backingStore))) - return -1; - if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0) return -1; - rc = qemuMonitorBlockdevReopen(priv->mon, &reopenprops); + rc = qemuBlockReopenFormatMon(priv->mon, src); if (qemuDomainObjExitMonitor(driver, vm) < 0 || rc < 0) return -1; -- 2.31.1
This function was added prior 'blockdev-reopen' being stable and qemu changed the arguments to actually contain an array of block node definitions to reopen. In our case we are just changing between read-only and read-write modes and thus we can keep operating on the nodes one-by-one. Modify the code to add the wrapper array. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_block.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index c96be8b52d..d9b876e5c1 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -3304,8 +3304,18 @@ qemuBlockReopenFormatMon(qemuMonitor *mon, virStorageSource *src) { g_autoptr(virJSONValue) reopenprops = NULL; + g_autoptr(virJSONValue) srcprops = NULL; + g_autoptr(virJSONValue) reopenoptions = virJSONValueNewArray(); - if (!(reopenprops = qemuBlockStorageSourceGetBlockdevProps(src, src->backingStore))) + if (!(srcprops = qemuBlockStorageSourceGetBlockdevProps(src, src->backingStore))) + return -1; + + if (virJSONValueArrayAppend(reopenoptions, &srcprops) < 0) + return -1; + + if (virJSONValueObjectCreate(&reopenprops, + "a:options", &reopenoptions, + NULL) < 0) return -1; if (qemuMonitorBlockdevReopen(mon, &reopenprops) < 0) -- 2.31.1
Export 'qemuBlockReopenFormatMon' and use it in a new test case wich will validate the arguments against the QMP schema. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_block.c | 2 +- src/qemu/qemu_block.h | 5 +++++ tests/qemumonitorjsontest.c | 28 ++++++++++++++++++++++++++++ 3 files changed, 34 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index d9b876e5c1..eedcfd927f 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -3299,7 +3299,7 @@ qemuBlockBitmapsHandleCommitFinish(virStorageSource *topsrc, } -static int +int qemuBlockReopenFormatMon(qemuMonitor *mon, virStorageSource *src) { diff --git a/src/qemu/qemu_block.h b/src/qemu/qemu_block.h index ff7048eb6c..54601a48a9 100644 --- a/src/qemu/qemu_block.h +++ b/src/qemu/qemu_block.h @@ -265,6 +265,11 @@ qemuBlockBitmapsHandleCommitFinish(virStorageSource *topsrc, GHashTable *blockNamedNodeData, virJSONValue **actions); +/* only for use in qemumonitorjsontest */ +int +qemuBlockReopenFormatMon(qemuMonitor *mon, + virStorageSource *src); + int qemuBlockReopenReadWrite(virDomainObj *vm, virStorageSource *src, diff --git a/tests/qemumonitorjsontest.c b/tests/qemumonitorjsontest.c index 0b321e8ed8..2122d9d999 100644 --- a/tests/qemumonitorjsontest.c +++ b/tests/qemumonitorjsontest.c @@ -2800,6 +2800,33 @@ testQemuMonitorJSONBlockExportAdd(const void *opaque) return 0; } + +static int +testQemuMonitorJSONBlockdevReopen(const void *opaque) +{ + const testGenericData *data = opaque; + g_autoptr(qemuMonitorTest) test = NULL; + g_autoptr(virStorageSource) src = virStorageSourceNew(); + + if (!(test = qemuMonitorTestNewSchema(data->xmlopt, data->schema))) + return -1; + + src->format = VIR_STORAGE_FILE_QCOW2; + src->readonly = true; + src->nodeformat = g_strdup("test node"); + src->nodestorage = g_strdup("backing nodename"); + src->backingStore = virStorageSourceNew(); + + if (qemuMonitorTestAddItem(test, "blockdev-reopen", "{\"return\":{}}") < 0) + return -1; + + if (qemuBlockReopenFormatMon(qemuMonitorTestGetMonitor(test), src) < 0) + return -1; + + return 0; +} + + static int testQemuMonitorJSONqemuMonitorJSONGetCPUModelComparison(const void *opaque) { @@ -2997,6 +3024,7 @@ mymain(void) DO_TEST(GetIOThreads); DO_TEST(Transaction); DO_TEST(BlockExportAdd); + DO_TEST(BlockdevReopen); DO_TEST_SIMPLE("qmp_capabilities", qemuMonitorJSONSetCapabilities); DO_TEST_SIMPLE("system_powerdown", qemuMonitorJSONSystemPowerdown); DO_TEST_SIMPLE("system_reset", qemuMonitorJSONSystemReset); -- 2.31.1
Base it on the presence of the "blockdev-reopen" QMP command. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_capabilities.c | 1 + tests/domaincapsdata/qemu_6.1.0-q35.x86_64.xml | 2 +- tests/domaincapsdata/qemu_6.1.0-tcg.x86_64.xml | 2 +- tests/domaincapsdata/qemu_6.1.0.x86_64.xml | 2 +- tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml | 2 ++ 5 files changed, 6 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 6cea8c2eca..c31c80eaf3 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -1182,6 +1182,7 @@ struct virQEMUCapsStringFlags virQEMUCapsCommands[] = { { "query-cpu-model-comparison", QEMU_CAPS_QUERY_CPU_MODEL_COMPARISON }, { "block-export-add", QEMU_CAPS_BLOCK_EXPORT_ADD }, { "query-display-options", QEMU_CAPS_QUERY_DISPLAY_OPTIONS }, + { "blockdev-reopen", QEMU_CAPS_BLOCKDEV_REOPEN }, }; struct virQEMUCapsStringFlags virQEMUCapsMigration[] = { diff --git a/tests/domaincapsdata/qemu_6.1.0-q35.x86_64.xml b/tests/domaincapsdata/qemu_6.1.0-q35.x86_64.xml index 62bc6bdf05..98b9c06cd1 100644 --- a/tests/domaincapsdata/qemu_6.1.0-q35.x86_64.xml +++ b/tests/domaincapsdata/qemu_6.1.0-q35.x86_64.xml @@ -216,7 +216,7 @@ <vmcoreinfo supported='yes'/> <genid supported='yes'/> <backingStoreInput supported='yes'/> - <backup supported='no'/> + <backup supported='yes'/> <sev supported='no'/> </features> </domainCapabilities> diff --git a/tests/domaincapsdata/qemu_6.1.0-tcg.x86_64.xml b/tests/domaincapsdata/qemu_6.1.0-tcg.x86_64.xml index 69d5f7e404..a9b4011557 100644 --- a/tests/domaincapsdata/qemu_6.1.0-tcg.x86_64.xml +++ b/tests/domaincapsdata/qemu_6.1.0-tcg.x86_64.xml @@ -219,7 +219,7 @@ <vmcoreinfo supported='yes'/> <genid supported='yes'/> <backingStoreInput supported='yes'/> - <backup supported='no'/> + <backup supported='yes'/> <sev supported='no'/> </features> </domainCapabilities> diff --git a/tests/domaincapsdata/qemu_6.1.0.x86_64.xml b/tests/domaincapsdata/qemu_6.1.0.x86_64.xml index a21c131d41..91a845204b 100644 --- a/tests/domaincapsdata/qemu_6.1.0.x86_64.xml +++ b/tests/domaincapsdata/qemu_6.1.0.x86_64.xml @@ -216,7 +216,7 @@ <vmcoreinfo supported='yes'/> <genid supported='yes'/> <backingStoreInput supported='yes'/> - <backup supported='no'/> + <backup supported='yes'/> <sev supported='no'/> </features> </domainCapabilities> diff --git a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml index 9d04a8b4a5..ab89df1ed0 100644 --- a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml @@ -206,6 +206,7 @@ <flag name='dbus-vmstate'/> <flag name='vhost-user-gpu'/> <flag name='vhost-user-vga'/> + <flag name='incremental-backup'/> <flag name='ramfb'/> <flag name='blockdev-file-dynamic-auto-read-only'/> <flag name='savevm-monitor-nodes'/> @@ -217,6 +218,7 @@ <flag name='vhost-user-fs'/> <flag name='query-named-block-nodes.flat'/> <flag name='blockdev-snapshot.allow-write-only-overlay'/> + <flag name='blockdev-reopen'/> <flag name='storage.werror'/> <flag name='fsdev.multidevs'/> <flag name='virtio.packed'/> -- 2.31.1
Now that 'blockdev-reopen' will be stable in the upcoming qemu versions we can finally enable incremental backups. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- NEWS.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index d95750e776..a7a7022f8d 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -13,6 +13,12 @@ v7.6.0 (unreleased) * **New features** + * qemu: Incremental backup support via ``virDomainBackupBegin`` + + libvirt-7.6 along with the unreleased qemu-6.1 will fully support the change + block tracking features (block-dirty-bitmaps) to be able to do incremental + backups and management of the checkpoint states via the appropriate APIs. + * **Improvements** * **Bug fixes** -- 2.31.1
On Fri, Jul 23, 2021 at 11:43:52AM +0200, Peter Krempa wrote:
Note that the capability update patch is waiting for a series of fixes as current qemu master breaks with libvirt.
Full version can be fetched from:
git fetch https://gitlab.com/pipo.sk/libvirt.git blockdev-reopen
Peter Krempa (6): tests: qemucapabilities: Update capabilities for qemu-6.1.0-rc0 on x86_64 qemuBlockReopenFormat: Extract @src handling and monitor invocation qemuBlockReopenFormatMon: Adapt to new argument format of 'blockdev-reopen' qemumonitorjsontest: Add test case for 'blockdev-reopen' qemu: capabilities: Enable detection of QEMU_CAPS_BLOCKDEV_REOPEN NEWS: Mention enablement of incremental backups
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
participants (2)
-
Pavel Hrdina -
Peter Krempa