[PATCH v2 1/1] qemu: add support for zero-page-detection migration parameter
QEMU supports a migration parameter to determine whether zero-page checking should be performed and if yes, where it should be performed (i.e., in the main migration thread or multifd threads). Add support for this in libvirt in the form of a boolean VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION migration parameter. Setting this to true with multifd will offload the check to multifd threads. Without multifd enabled, the check will happen in the main migration thread. Setting this to false disables zero-page checking. When the parameter is not set, QEMU's default is kept. Signed-off-by: Tejus GK <tejus.gk@nutanix.com> --- docs/manpages/virsh.rst | 6 ++++++ include/libvirt/libvirt-domain.h | 16 ++++++++++++++++ src/qemu/qemu_migration.c | 1 + src/qemu/qemu_migration_params.c | 30 ++++++++++++++++++++++++++++++ src/qemu/qemu_migration_params.h | 1 + tools/virsh-domain.c | 11 +++++++++++ 6 files changed, 65 insertions(+) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index f533a10776..b5a796a00a 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3802,6 +3802,7 @@ migrate [--bandwidth bandwidth] [--tls-destination hostname] [--disks-uri URI] [--copy-storage-synchronous-writes] [--available-switchover-bandwidth bandwidth] + [--zero-page-detection] Migrate domain to another host. Add *--live* for live migration; <--p2p> for peer-2-peer migration; *--direct* for direct migration; or *--tunnelled* @@ -3953,6 +3954,11 @@ parallel connections. The number of such connections can be set using network link between the source and the target and thus speeding up the migration. +*--zero-page-detection* requests pages containing only zeros to be detected by +the parallel migration threads rather than the main migration thread. Without +*--parallel*, zero pages are detected by the main migration thread. When the +option is not set, the hypervisor default is used. + Running migration can be canceled by interrupting virsh (usually using ``Ctrl-C``) or by ``domjobabort`` command sent from another virsh instance. diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 9ee153c6eb..0e68a572c3 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1468,6 +1468,22 @@ typedef enum { */ # define VIR_MIGRATE_PARAM_TLS_DESTINATION "tls.destination" +/** + * VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION: + * + * virDomainMigrate* params field: whether zero page detection should be + * performed during migration. As VIR_TYPED_PARAM_BOOLEAN. + * + * When enabled with parallel migration, zero pages are detected by the multifd + * threads rather than the main migration thread. When enabled without parallel + * migration, zero pages are detected by the main migration thread. + * When disabled, zero pages are transferred as normal pages. + * When not set, the hypervisor default is used. + * + * Since: 12.9.0 + */ +# define VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION "zero_page_detection" + /* Domain migration. */ virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 76661de09a..13113ceaff 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -110,6 +110,7 @@ const virTypedParamValidationTemplate qemuMigrationParametersValidation[] = { { VIR_MIGRATE_PARAM_TLS_DESTINATION, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_DISKS_URI, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_BANDWIDTH_AVAIL_SWITCHOVER, VIR_TYPED_PARAM_ULLONG }, + { VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, VIR_TYPED_PARAM_BOOLEAN }, { "", 0 } }; diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index dd47516742..e1ead04068 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -131,6 +131,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "multifd-zstd-level", "avail-switchover-bandwidth", "direct-io", + "zero-page-detection", ); typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysOnItem; @@ -332,6 +333,9 @@ static const qemuMigrationParamInfoItem qemuMigrationParamInfo[] = { [QEMU_MIGRATION_PARAM_DIRECT_IO] = { .type = QEMU_MIGRATION_PARAM_TYPE_BOOL, }, + [QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION] = { + .type = QEMU_MIGRATION_PARAM_TYPE_STRING, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) == QEMU_MIGRATION_PARAM_LAST); @@ -692,6 +696,29 @@ qemuMigrationParamsSetCompression(virTypedParameterPtr params, } +static int +qemuMigrationParamsSetZeroPageDetection(virTypedParameterPtr params, + int nparams, + qemuMigrationParams *migParams) +{ + int zeroPageDetection; + int rc; + + if ((rc = virTypedParamsGetBoolean(params, nparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + &zeroPageDetection)) < 0) + return -1; + + if (rc == 1) { + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].value.s = + g_strdup(zeroPageDetection ? "multifd" : "none"); + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].set = true; + } + + return 0; +} + + void qemuMigrationParamsSetBlockDirtyBitmapMapping(qemuMigrationParams *migParams, virJSONValue **params) @@ -792,6 +819,9 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr params, if (qemuMigrationParamsSetCompression(params, nparams, flags, migParams) < 0) return NULL; + if (qemuMigrationParamsSetZeroPageDetection(params, nparams, migParams) < 0) + return NULL; + return g_steal_pointer(&migParams); } diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index b7a829b85a..131fe82fc0 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -67,6 +67,7 @@ typedef enum { QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL, QEMU_MIGRATION_PARAM_AVAIL_SWITCHOVER_BANDWIDTH, QEMU_MIGRATION_PARAM_DIRECT_IO, + QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION, QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 2c64f19a19..18854ad0c3 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -11432,6 +11432,10 @@ static const vshCmdOptDef opts_migrate[] = { .type = VSH_OT_INT, .help = N_("bandwidth (in MiB/s) available for the final phase of migration") }, + {.name = "zero-page-detection", + .type = VSH_OT_BOOL, + .help = N_("enable zero page detection in parallel migration threads") + }, {.name = NULL} }; @@ -11733,6 +11737,13 @@ doMigrate(void *opaque) goto save_error; } + if (vshCommandOptBool(cmd, "zero-page-detection")) { + if (virTypedParamsAddBoolean(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + true) < 0) + goto save_error; + } + if ((rv = vshCommandOptInt(ctl, cmd, "comp-zlib-level", &intOpt)) < 0) { goto out; } else if (rv > 0) { -- 2.43.7
On Wed, Oct 21, 2026 at 10:58:02 +0000, Tejus GK wrote:
QEMU supports a migration parameter to determine whether zero-page checking should be performed and if yes, where it should be performed (i.e., in the main migration thread or multifd threads). Add support for this in libvirt in the form of a boolean VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION migration parameter.
Setting this to true with multifd will offload the check to multifd threads. Without multifd enabled, the check will happen in the main migration thread. Setting this to false disables zero-page checking. When the parameter is not set, QEMU's default is kept.
Signed-off-by: Tejus GK <tejus.gk@nutanix.com> --- docs/manpages/virsh.rst | 6 ++++++ include/libvirt/libvirt-domain.h | 16 ++++++++++++++++ src/qemu/qemu_migration.c | 1 + src/qemu/qemu_migration_params.c | 30 ++++++++++++++++++++++++++++++ src/qemu/qemu_migration_params.h | 1 + tools/virsh-domain.c | 11 +++++++++++ 6 files changed, 65 insertions(+)
I talked about this feature with Peter Xu and realized I missed one important thing when looking at the QEMU code. The QEMU default for this parameter is not "legacy". QEMU defaults to "multifd" and only machine types 8.2 and older override it to "legacy". In other words QEMU is already doing the right thing by default which makes this patch mostly useless. Theoretically users could want to disable zero page detection although I'm not sure what would be the benefit of doing so, or perhaps to force "multifd" for old machine types. That said I think we can just drop this patch, but since I started writing my comments before talking to Peter, I'm sending them anyway in case you wanted to come up with an explanation of why this patch is useful. I forgot to mention this in v1... we usually split such additions into three commits: - public API (include/libvirt/libvirt-domain.h) - virsh (docs/manpages/virsh.rst, tools/virsh-domain.c) - qemu implementation (src/qemu/*) It's not a big deal for such a small change, but it makes individual commits focused on a single layer.
diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index f533a10776..b5a796a00a 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3802,6 +3802,7 @@ migrate [--bandwidth bandwidth] [--tls-destination hostname] [--disks-uri URI] [--copy-storage-synchronous-writes] [--available-switchover-bandwidth bandwidth] + [--zero-page-detection]
Migrate domain to another host. Add *--live* for live migration; <--p2p> for peer-2-peer migration; *--direct* for direct migration; or *--tunnelled* @@ -3953,6 +3954,11 @@ parallel connections. The number of such connections can be set using network link between the source and the target and thus speeding up the migration.
+*--zero-page-detection* requests pages containing only zeros to be detected by +the parallel migration threads rather than the main migration thread. Without +*--parallel*, zero pages are detected by the main migration thread. When the +option is not set, the hypervisor default is used. +
virsh options should provide the same functionality that is available via C API, this would only cover half of the new functionality
Running migration can be canceled by interrupting virsh (usually using ``Ctrl-C``) or by ``domjobabort`` command sent from another virsh instance.
diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 9ee153c6eb..0e68a572c3 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1468,6 +1468,22 @@ typedef enum { */ # define VIR_MIGRATE_PARAM_TLS_DESTINATION "tls.destination"
+/** + * VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION: + * + * virDomainMigrate* params field: whether zero page detection should be + * performed during migration. As VIR_TYPED_PARAM_BOOLEAN. + * + * When enabled with parallel migration, zero pages are detected by the multifd + * threads rather than the main migration thread. When enabled without parallel + * migration, zero pages are detected by the main migration thread.
I would consider this an implementation detail. The parameter is called "zero_page_detection" so we should describe it as such. When specified it would force enable/disable zero page detection and unused would let the hypervisor decide what to do.
+ * When disabled, zero pages are transferred as normal pages. + * When not set, the hypervisor default is used. + * + * Since: 12.9.0 + */ +# define VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION "zero_page_detection" + /* Domain migration. */ virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 76661de09a..13113ceaff 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -110,6 +110,7 @@ const virTypedParamValidationTemplate qemuMigrationParametersValidation[] = { { VIR_MIGRATE_PARAM_TLS_DESTINATION, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_DISKS_URI, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_BANDWIDTH_AVAIL_SWITCHOVER, VIR_TYPED_PARAM_ULLONG }, + { VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, VIR_TYPED_PARAM_BOOLEAN }, { "", 0 } };
diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index dd47516742..e1ead04068 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -131,6 +131,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "multifd-zstd-level", "avail-switchover-bandwidth", "direct-io", + "zero-page-detection", );
typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysOnItem; @@ -332,6 +333,9 @@ static const qemuMigrationParamInfoItem qemuMigrationParamInfo[] = { [QEMU_MIGRATION_PARAM_DIRECT_IO] = { .type = QEMU_MIGRATION_PARAM_TYPE_BOOL, }, + [QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION] = { + .type = QEMU_MIGRATION_PARAM_TYPE_STRING, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) == QEMU_MIGRATION_PARAM_LAST);
@@ -692,6 +696,29 @@ qemuMigrationParamsSetCompression(virTypedParameterPtr params, }
+static int +qemuMigrationParamsSetZeroPageDetection(virTypedParameterPtr params, + int nparams, + qemuMigrationParams *migParams) +{ + int zeroPageDetection; + int rc; + + if ((rc = virTypedParamsGetBoolean(params, nparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + &zeroPageDetection)) < 0) + return -1; + + if (rc == 1) { + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].value.s = + g_strdup(zeroPageDetection ? "multifd" : "none"); + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].set = true; + }
You need to check if the parameter even exists in QEMU as we support versions >= 7.2 and the parameter was introduced in QEMU 9.0. With old versions setting VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION would be allowed (such QEMU uses the equivalent of "legacy"), but disabling it would not be supported and should report an error.
+ + return 0; +} + + void qemuMigrationParamsSetBlockDirtyBitmapMapping(qemuMigrationParams *migParams, virJSONValue **params) @@ -792,6 +819,9 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr params, if (qemuMigrationParamsSetCompression(params, nparams, flags, migParams) < 0) return NULL;
+ if (qemuMigrationParamsSetZeroPageDetection(params, nparams, migParams) < 0) + return NULL; + return g_steal_pointer(&migParams); }
diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index b7a829b85a..131fe82fc0 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -67,6 +67,7 @@ typedef enum { QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL, QEMU_MIGRATION_PARAM_AVAIL_SWITCHOVER_BANDWIDTH, QEMU_MIGRATION_PARAM_DIRECT_IO, + QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION,
QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 2c64f19a19..18854ad0c3 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -11432,6 +11432,10 @@ static const vshCmdOptDef opts_migrate[] = { .type = VSH_OT_INT, .help = N_("bandwidth (in MiB/s) available for the final phase of migration") }, + {.name = "zero-page-detection", + .type = VSH_OT_BOOL, + .help = N_("enable zero page detection in parallel migration threads") + }, {.name = NULL} };
@@ -11733,6 +11737,13 @@ doMigrate(void *opaque) goto save_error; }
+ if (vshCommandOptBool(cmd, "zero-page-detection")) { + if (virTypedParamsAddBoolean(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + true) < 0) + goto save_error; + } +
The implementation matches the description of the new option, but it does not match what the typed parameter can do. The parameter can be unspecified, enabled or disabled, but this option can only be unspecified or enabled.
if ((rv = vshCommandOptInt(ctl, cmd, "comp-zlib-level", &intOpt)) < 0) { goto out; } else if (rv > 0) {
Jirka
On 27 Oct 2026, at 3:29 PM, Jiri Denemark <jdenemar@redhat.com> wrote:
!-------------------------------------------------------------------| CAUTION: External Email
|-------------------------------------------------------------------!
On Wed, Oct 21, 2026 at 10:58:02 +0000, Tejus GK wrote:
QEMU supports a migration parameter to determine whether zero-page checking should be performed and if yes, where it should be performed (i.e., in the main migration thread or multifd threads). Add support for this in libvirt in the form of a boolean VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION migration parameter.
Setting this to true with multifd will offload the check to multifd threads. Without multifd enabled, the check will happen in the main migration thread. Setting this to false disables zero-page checking. When the parameter is not set, QEMU's default is kept.
Signed-off-by: Tejus GK <tejus.gk@nutanix.com> --- docs/manpages/virsh.rst | 6 ++++++ include/libvirt/libvirt-domain.h | 16 ++++++++++++++++ src/qemu/qemu_migration.c | 1 + src/qemu/qemu_migration_params.c | 30 ++++++++++++++++++++++++++++++ src/qemu/qemu_migration_params.h | 1 + tools/virsh-domain.c | 11 +++++++++++ 6 files changed, 65 insertions(+)
I talked about this feature with Peter Xu and realized I missed one important thing when looking at the QEMU code. The QEMU default for this parameter is not "legacy". QEMU defaults to "multifd" and only machine types 8.2 and older override it to "legacy". In other words QEMU is already doing the right thing by default which makes this patch mostly useless. Theoretically users could want to disable zero page detection although I'm not sure what would be the benefit of doing so, or perhaps to force "multifd" for old machine types.
That said I think we can just drop this patch, but since I started writing my comments before talking to Peter, I'm sending them anyway in case you wanted to come up with an explanation of why this patch is useful.
I forgot to mention this in v1... we usually split such additions into three commits: - public API (include/libvirt/libvirt-domain.h) - virsh (docs/manpages/virsh.rst, tools/virsh-domain.c) - qemu implementation (src/qemu/*)
It's not a big deal for such a small change, but it makes individual commits focused on a single layer.
diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index f533a10776..b5a796a00a 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3802,6 +3802,7 @@ migrate [--bandwidth bandwidth] [--tls-destination hostname] [--disks-uri URI] [--copy-storage-synchronous-writes] [--available-switchover-bandwidth bandwidth] + [--zero-page-detection]
Migrate domain to another host. Add *--live* for live migration; <--p2p> for peer-2-peer migration; *--direct* for direct migration; or *--tunnelled* @@ -3953,6 +3954,11 @@ parallel connections. The number of such connections can be set using network link between the source and the target and thus speeding up the migration.
+*--zero-page-detection* requests pages containing only zeros to be detected by +the parallel migration threads rather than the main migration thread. Without +*--parallel*, zero pages are detected by the main migration thread. When the +option is not set, the hypervisor default is used. +
virsh options should provide the same functionality that is available via C API, this would only cover half of the new functionality
Running migration can be canceled by interrupting virsh (usually using ``Ctrl-C``) or by ``domjobabort`` command sent from another virsh instance.
diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 9ee153c6eb..0e68a572c3 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1468,6 +1468,22 @@ typedef enum { */ # define VIR_MIGRATE_PARAM_TLS_DESTINATION "tls.destination"
+/** + * VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION: + * + * virDomainMigrate* params field: whether zero page detection should be + * performed during migration. As VIR_TYPED_PARAM_BOOLEAN. + * + * When enabled with parallel migration, zero pages are detected by the multifd + * threads rather than the main migration thread. When enabled without parallel + * migration, zero pages are detected by the main migration thread.
I would consider this an implementation detail. The parameter is called "zero_page_detection" so we should describe it as such. When specified it would force enable/disable zero page detection and unused would let the hypervisor decide what to do.
+ * When disabled, zero pages are transferred as normal pages. + * When not set, the hypervisor default is used. + * + * Since: 12.9.0 + */ +# define VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION "zero_page_detection" + /* Domain migration. */ virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 76661de09a..13113ceaff 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -110,6 +110,7 @@ const virTypedParamValidationTemplate qemuMigrationParametersValidation[] = { { VIR_MIGRATE_PARAM_TLS_DESTINATION, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_DISKS_URI, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_BANDWIDTH_AVAIL_SWITCHOVER, VIR_TYPED_PARAM_ULLONG }, + { VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, VIR_TYPED_PARAM_BOOLEAN }, { "", 0 } };
diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index dd47516742..e1ead04068 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -131,6 +131,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "multifd-zstd-level", "avail-switchover-bandwidth", "direct-io", + "zero-page-detection", );
typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysOnItem; @@ -332,6 +333,9 @@ static const qemuMigrationParamInfoItem qemuMigrationParamInfo[] = { [QEMU_MIGRATION_PARAM_DIRECT_IO] = { .type = QEMU_MIGRATION_PARAM_TYPE_BOOL, }, + [QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION] = { + .type = QEMU_MIGRATION_PARAM_TYPE_STRING, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) == QEMU_MIGRATION_PARAM_LAST);
@@ -692,6 +696,29 @@ qemuMigrationParamsSetCompression(virTypedParameterPtr params, }
+static int +qemuMigrationParamsSetZeroPageDetection(virTypedParameterPtr params, + int nparams, + qemuMigrationParams *migParams) +{ + int zeroPageDetection; + int rc; + + if ((rc = virTypedParamsGetBoolean(params, nparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + &zeroPageDetection)) < 0) + return -1; + + if (rc == 1) { + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].value.s = + g_strdup(zeroPageDetection ? "multifd" : "none"); + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].set = true; + }
You need to check if the parameter even exists in QEMU as we support versions >= 7.2 and the parameter was introduced in QEMU 9.0. With old versions setting VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION would be allowed (such QEMU uses the equivalent of "legacy"), but disabling it would not be supported and should report an error.
+ + return 0; +} + + void qemuMigrationParamsSetBlockDirtyBitmapMapping(qemuMigrationParams *migParams, virJSONValue **params) @@ -792,6 +819,9 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr params, if (qemuMigrationParamsSetCompression(params, nparams, flags, migParams) < 0) return NULL;
+ if (qemuMigrationParamsSetZeroPageDetection(params, nparams, migParams) < 0) + return NULL; + return g_steal_pointer(&migParams); }
diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index b7a829b85a..131fe82fc0 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -67,6 +67,7 @@ typedef enum { QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL, QEMU_MIGRATION_PARAM_AVAIL_SWITCHOVER_BANDWIDTH, QEMU_MIGRATION_PARAM_DIRECT_IO, + QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION,
QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 2c64f19a19..18854ad0c3 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -11432,6 +11432,10 @@ static const vshCmdOptDef opts_migrate[] = { .type = VSH_OT_INT, .help = N_("bandwidth (in MiB/s) available for the final phase of migration") }, + {.name = "zero-page-detection", + .type = VSH_OT_BOOL, + .help = N_("enable zero page detection in parallel migration threads") + }, {.name = NULL} };
@@ -11733,6 +11737,13 @@ doMigrate(void *opaque) goto save_error; }
+ if (vshCommandOptBool(cmd, "zero-page-detection")) { + if (virTypedParamsAddBoolean(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + true) < 0) + goto save_error; + } +
The implementation matches the description of the new option, but it does not match what the typed parameter can do. The parameter can be unspecified, enabled or disabled, but this option can only be unspecified or enabled.
if ((rv = vshCommandOptInt(ctl, cmd, "comp-zlib-level", &intOpt)) < 0) { goto out; } else if (rv > 0) {
Jirka
Hi Jiri, Peter! Thank you for the review again! My main goal was to add this as an override for the older machine types you mentioned. A VM with an older machine type, but new enough QEMU will not have multifd zero-page checking kicking in for it, even though QEMU supports it. And we seem to be having a lot of such VMs in production, which don’t end up seeing the benefits of this offload. With the patch, clients could programmatically make QEMU use the offload for such VMs, without having to reboot the VM to change the machine type. I can post a v3 based on your suggestions above, if you are okay with keeping the API for this. Please let me know your thoughts on it. Regards, Tejus
On 27 Oct 2026, at 3:29 PM, Jiri Denemark <jdenemar@redhat.com> wrote:
!-------------------------------------------------------------------| CAUTION: External Email
|-------------------------------------------------------------------!
On Wed, Oct 21, 2026 at 10:58:02 +0000, Tejus GK wrote:
QEMU supports a migration parameter to determine whether zero-page checking should be performed and if yes, where it should be performed (i.e., in the main migration thread or multifd threads). Add support for this in libvirt in the form of a boolean VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION migration parameter.
Setting this to true with multifd will offload the check to multifd threads. Without multifd enabled, the check will happen in the main migration thread. Setting this to false disables zero-page checking. When the parameter is not set, QEMU's default is kept.
Signed-off-by: Tejus GK <tejus.gk@nutanix.com> --- docs/manpages/virsh.rst | 6 ++++++ include/libvirt/libvirt-domain.h | 16 ++++++++++++++++ src/qemu/qemu_migration.c | 1 + src/qemu/qemu_migration_params.c | 30 ++++++++++++++++++++++++++++++ src/qemu/qemu_migration_params.h | 1 + tools/virsh-domain.c | 11 +++++++++++ 6 files changed, 65 insertions(+)
I talked about this feature with Peter Xu and realized I missed one important thing when looking at the QEMU code. The QEMU default for this parameter is not "legacy". QEMU defaults to "multifd" and only machine types 8.2 and older override it to "legacy". In other words QEMU is already doing the right thing by default which makes this patch mostly useless. Theoretically users could want to disable zero page detection although I'm not sure what would be the benefit of doing so, or perhaps to force "multifd" for old machine types.
That said I think we can just drop this patch, but since I started writing my comments before talking to Peter, I'm sending them anyway in case you wanted to come up with an explanation of why this patch is useful.
I forgot to mention this in v1... we usually split such additions into three commits: - public API (include/libvirt/libvirt-domain.h) - virsh (docs/manpages/virsh.rst, tools/virsh-domain.c) - qemu implementation (src/qemu/*)
It's not a big deal for such a small change, but it makes individual commits focused on a single layer.
diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index f533a10776..b5a796a00a 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3802,6 +3802,7 @@ migrate [--bandwidth bandwidth] [--tls-destination hostname] [--disks-uri URI] [--copy-storage-synchronous-writes] [--available-switchover-bandwidth bandwidth] + [--zero-page-detection]
Migrate domain to another host. Add *--live* for live migration; <--p2p> for peer-2-peer migration; *--direct* for direct migration; or *--tunnelled* @@ -3953,6 +3954,11 @@ parallel connections. The number of such connections can be set using network link between the source and the target and thus speeding up the migration.
+*--zero-page-detection* requests pages containing only zeros to be detected by +the parallel migration threads rather than the main migration thread. Without +*--parallel*, zero pages are detected by the main migration thread. When the +option is not set, the hypervisor default is used. +
virsh options should provide the same functionality that is available via C API, this would only cover half of the new functionality
Running migration can be canceled by interrupting virsh (usually using ``Ctrl-C``) or by ``domjobabort`` command sent from another virsh instance.
diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 9ee153c6eb..0e68a572c3 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1468,6 +1468,22 @@ typedef enum { */ # define VIR_MIGRATE_PARAM_TLS_DESTINATION "tls.destination"
+/** + * VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION: + * + * virDomainMigrate* params field: whether zero page detection should be + * performed during migration. As VIR_TYPED_PARAM_BOOLEAN. + * + * When enabled with parallel migration, zero pages are detected by the multifd + * threads rather than the main migration thread. When enabled without parallel + * migration, zero pages are detected by the main migration thread.
I would consider this an implementation detail. The parameter is called "zero_page_detection" so we should describe it as such. When specified it would force enable/disable zero page detection and unused would let the hypervisor decide what to do.
+ * When disabled, zero pages are transferred as normal pages. + * When not set, the hypervisor default is used. + * + * Since: 12.9.0 + */ +# define VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION "zero_page_detection" + /* Domain migration. */ virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 76661de09a..13113ceaff 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -110,6 +110,7 @@ const virTypedParamValidationTemplate qemuMigrationParametersValidation[] = { { VIR_MIGRATE_PARAM_TLS_DESTINATION, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_DISKS_URI, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_BANDWIDTH_AVAIL_SWITCHOVER, VIR_TYPED_PARAM_ULLONG }, + { VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, VIR_TYPED_PARAM_BOOLEAN }, { "", 0 } };
diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index dd47516742..e1ead04068 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -131,6 +131,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "multifd-zstd-level", "avail-switchover-bandwidth", "direct-io", + "zero-page-detection", );
typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysOnItem; @@ -332,6 +333,9 @@ static const qemuMigrationParamInfoItem qemuMigrationParamInfo[] = { [QEMU_MIGRATION_PARAM_DIRECT_IO] = { .type = QEMU_MIGRATION_PARAM_TYPE_BOOL, }, + [QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION] = { + .type = QEMU_MIGRATION_PARAM_TYPE_STRING, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) == QEMU_MIGRATION_PARAM_LAST);
@@ -692,6 +696,29 @@ qemuMigrationParamsSetCompression(virTypedParameterPtr params, }
+static int +qemuMigrationParamsSetZeroPageDetection(virTypedParameterPtr params, + int nparams, + qemuMigrationParams *migParams) +{ + int zeroPageDetection; + int rc; + + if ((rc = virTypedParamsGetBoolean(params, nparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + &zeroPageDetection)) < 0) + return -1; + + if (rc == 1) { + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].value.s = + g_strdup(zeroPageDetection ? "multifd" : "none"); + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].set = true; + }
You need to check if the parameter even exists in QEMU as we support versions >= 7.2 and the parameter was introduced in QEMU 9.0. With old versions setting VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION would be allowed (such QEMU uses the equivalent of "legacy"), but disabling it would not be supported and should report an error.
+ + return 0; +} + + void qemuMigrationParamsSetBlockDirtyBitmapMapping(qemuMigrationParams *migParams, virJSONValue **params) @@ -792,6 +819,9 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr params, if (qemuMigrationParamsSetCompression(params, nparams, flags, migParams) < 0) return NULL;
+ if (qemuMigrationParamsSetZeroPageDetection(params, nparams, migParams) < 0) + return NULL; + return g_steal_pointer(&migParams); }
diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index b7a829b85a..131fe82fc0 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -67,6 +67,7 @@ typedef enum { QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL, QEMU_MIGRATION_PARAM_AVAIL_SWITCHOVER_BANDWIDTH, QEMU_MIGRATION_PARAM_DIRECT_IO, + QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION,
QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 2c64f19a19..18854ad0c3 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -11432,6 +11432,10 @@ static const vshCmdOptDef opts_migrate[] = { .type = VSH_OT_INT, .help = N_("bandwidth (in MiB/s) available for the final phase of migration") }, + {.name = "zero-page-detection", + .type = VSH_OT_BOOL, + .help = N_("enable zero page detection in parallel migration threads") + }, {.name = NULL} };
@@ -11733,6 +11737,13 @@ doMigrate(void *opaque) goto save_error; }
+ if (vshCommandOptBool(cmd, "zero-page-detection")) { + if (virTypedParamsAddBoolean(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + true) < 0) + goto save_error; + } +
The implementation matches the description of the new option, but it does not match what the typed parameter can do. The parameter can be unspecified, enabled or disabled, but this option can only be unspecified or enabled.
if ((rv = vshCommandOptInt(ctl, cmd, "comp-zlib-level", &intOpt)) < 0) { goto out; } else if (rv > 0) {
Jirka
Hi Jiri, Peter! Thank you for the review again! My main goal was to add this as an override for the older machine types you mentioned. A VM with an older machine type, but new enough QEMU will not have multifd zero-page checking kicking in for it, even though QEMU supports it. And we seem to be having a lot of such VMs in production, which don’t end up seeing the benefits of this offload. With the patch, clients could programmatically make QEMU use the offload for such VMs, without having to reboot the VM to change the machine type. I can post a v3 based on your suggestions above, if you are okay with keeping the API for this. Please let me know your thoughts on it. Regards, Tejus
On 28 Oct 2026, at 1:47 PM, Tejus GK <tejus.gk@nutanix.com> wrote:
On 27 Oct 2026, at 3:29 PM, Jiri Denemark <jdenemar@redhat.com> wrote:
!-------------------------------------------------------------------| CAUTION: External Email
|-------------------------------------------------------------------!
On Wed, Oct 21, 2026 at 10:58:02 +0000, Tejus GK wrote:
QEMU supports a migration parameter to determine whether zero-page checking should be performed and if yes, where it should be performed (i.e., in the main migration thread or multifd threads). Add support for this in libvirt in the form of a boolean VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION migration parameter.
Setting this to true with multifd will offload the check to multifd threads. Without multifd enabled, the check will happen in the main migration thread. Setting this to false disables zero-page checking. When the parameter is not set, QEMU's default is kept.
Signed-off-by: Tejus GK <tejus.gk@nutanix.com> --- docs/manpages/virsh.rst | 6 ++++++ include/libvirt/libvirt-domain.h | 16 ++++++++++++++++ src/qemu/qemu_migration.c | 1 + src/qemu/qemu_migration_params.c | 30 ++++++++++++++++++++++++++++++ src/qemu/qemu_migration_params.h | 1 + tools/virsh-domain.c | 11 +++++++++++ 6 files changed, 65 insertions(+)
I talked about this feature with Peter Xu and realized I missed one important thing when looking at the QEMU code. The QEMU default for this parameter is not "legacy". QEMU defaults to "multifd" and only machine types 8.2 and older override it to "legacy". In other words QEMU is already doing the right thing by default which makes this patch mostly useless. Theoretically users could want to disable zero page detection although I'm not sure what would be the benefit of doing so, or perhaps to force "multifd" for old machine types.
That said I think we can just drop this patch, but since I started writing my comments before talking to Peter, I'm sending them anyway in case you wanted to come up with an explanation of why this patch is useful.
I forgot to mention this in v1... we usually split such additions into three commits: - public API (include/libvirt/libvirt-domain.h) - virsh (docs/manpages/virsh.rst, tools/virsh-domain.c) - qemu implementation (src/qemu/*)
It's not a big deal for such a small change, but it makes individual commits focused on a single layer.
diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index f533a10776..b5a796a00a 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3802,6 +3802,7 @@ migrate [--bandwidth bandwidth] [--tls-destination hostname] [--disks-uri URI] [--copy-storage-synchronous-writes] [--available-switchover-bandwidth bandwidth] + [--zero-page-detection]
Migrate domain to another host. Add *--live* for live migration; <--p2p> for peer-2-peer migration; *--direct* for direct migration; or *--tunnelled* @@ -3953,6 +3954,11 @@ parallel connections. The number of such connections can be set using network link between the source and the target and thus speeding up the migration.
+*--zero-page-detection* requests pages containing only zeros to be detected by +the parallel migration threads rather than the main migration thread. Without +*--parallel*, zero pages are detected by the main migration thread. When the +option is not set, the hypervisor default is used. +
virsh options should provide the same functionality that is available via C API, this would only cover half of the new functionality
Running migration can be canceled by interrupting virsh (usually using ``Ctrl-C``) or by ``domjobabort`` command sent from another virsh instance.
diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 9ee153c6eb..0e68a572c3 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1468,6 +1468,22 @@ typedef enum { */ # define VIR_MIGRATE_PARAM_TLS_DESTINATION "tls.destination"
+/** + * VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION: + * + * virDomainMigrate* params field: whether zero page detection should be + * performed during migration. As VIR_TYPED_PARAM_BOOLEAN. + * + * When enabled with parallel migration, zero pages are detected by the multifd + * threads rather than the main migration thread. When enabled without parallel + * migration, zero pages are detected by the main migration thread.
I would consider this an implementation detail. The parameter is called "zero_page_detection" so we should describe it as such. When specified it would force enable/disable zero page detection and unused would let the hypervisor decide what to do.
+ * When disabled, zero pages are transferred as normal pages. + * When not set, the hypervisor default is used. + * + * Since: 12.9.0 + */ +# define VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION "zero_page_detection" + /* Domain migration. */ virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 76661de09a..13113ceaff 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -110,6 +110,7 @@ const virTypedParamValidationTemplate qemuMigrationParametersValidation[] = { { VIR_MIGRATE_PARAM_TLS_DESTINATION, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_DISKS_URI, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_BANDWIDTH_AVAIL_SWITCHOVER, VIR_TYPED_PARAM_ULLONG }, + { VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, VIR_TYPED_PARAM_BOOLEAN }, { "", 0 } };
diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index dd47516742..e1ead04068 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -131,6 +131,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "multifd-zstd-level", "avail-switchover-bandwidth", "direct-io", + "zero-page-detection", );
typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysOnItem; @@ -332,6 +333,9 @@ static const qemuMigrationParamInfoItem qemuMigrationParamInfo[] = { [QEMU_MIGRATION_PARAM_DIRECT_IO] = { .type = QEMU_MIGRATION_PARAM_TYPE_BOOL, }, + [QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION] = { + .type = QEMU_MIGRATION_PARAM_TYPE_STRING, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) == QEMU_MIGRATION_PARAM_LAST);
@@ -692,6 +696,29 @@ qemuMigrationParamsSetCompression(virTypedParameterPtr params, }
+static int +qemuMigrationParamsSetZeroPageDetection(virTypedParameterPtr params, + int nparams, + qemuMigrationParams *migParams) +{ + int zeroPageDetection; + int rc; + + if ((rc = virTypedParamsGetBoolean(params, nparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + &zeroPageDetection)) < 0) + return -1; + + if (rc == 1) { + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].value.s = + g_strdup(zeroPageDetection ? "multifd" : "none"); + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].set = true; + }
You need to check if the parameter even exists in QEMU as we support versions >= 7.2 and the parameter was introduced in QEMU 9.0. With old versions setting VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION would be allowed (such QEMU uses the equivalent of "legacy"), but disabling it would not be supported and should report an error.
+ + return 0; +} + + void qemuMigrationParamsSetBlockDirtyBitmapMapping(qemuMigrationParams *migParams, virJSONValue **params) @@ -792,6 +819,9 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr params, if (qemuMigrationParamsSetCompression(params, nparams, flags, migParams) < 0) return NULL;
+ if (qemuMigrationParamsSetZeroPageDetection(params, nparams, migParams) < 0) + return NULL; + return g_steal_pointer(&migParams); }
diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index b7a829b85a..131fe82fc0 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -67,6 +67,7 @@ typedef enum { QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL, QEMU_MIGRATION_PARAM_AVAIL_SWITCHOVER_BANDWIDTH, QEMU_MIGRATION_PARAM_DIRECT_IO, + QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION,
QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 2c64f19a19..18854ad0c3 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -11432,6 +11432,10 @@ static const vshCmdOptDef opts_migrate[] = { .type = VSH_OT_INT, .help = N_("bandwidth (in MiB/s) available for the final phase of migration") }, + {.name = "zero-page-detection", + .type = VSH_OT_BOOL, + .help = N_("enable zero page detection in parallel migration threads") + }, {.name = NULL} };
@@ -11733,6 +11737,13 @@ doMigrate(void *opaque) goto save_error; }
+ if (vshCommandOptBool(cmd, "zero-page-detection")) { + if (virTypedParamsAddBoolean(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + true) < 0) + goto save_error; + } +
The implementation matches the description of the new option, but it does not match what the typed parameter can do. The parameter can be unspecified, enabled or disabled, but this option can only be unspecified or enabled.
if ((rv = vshCommandOptInt(ctl, cmd, "comp-zlib-level", &intOpt)) < 0) { goto out; } else if (rv > 0) {
Jirka
Hi Jiri, Peter!
Thank you for the review again! My main goal was to add this as an override for the older machine types you mentioned. A VM with an older machine type, but new enough QEMU will not have multifd zero-page checking kicking in for it, even though QEMU supports it. And we seem to be having a lot of such VMs in production, which don’t end up seeing the benefits of this offload. With the patch, clients could programmatically make QEMU use the offload for such VMs, without having to reboot the VM to change the machine type.
I can post a v3 based on your suggestions above, if you are okay with keeping the API for this. Please let me know your thoughts on it.
Regards, Tejus
Hi, a gentle ping for the above :) Regards, Tejus
On Fri, Nov 06, 2026 at 11:37:56AM +0000, Tejus GK wrote:
On 28 Oct 2026, at 1:47 PM, Tejus GK <tejus.gk@nutanix.com> wrote:
On 27 Oct 2026, at 3:29 PM, Jiri Denemark <jdenemar@redhat.com> wrote:
!-------------------------------------------------------------------| CAUTION: External Email
|-------------------------------------------------------------------!
On Wed, Oct 21, 2026 at 10:58:02 +0000, Tejus GK wrote:
QEMU supports a migration parameter to determine whether zero-page checking should be performed and if yes, where it should be performed (i.e., in the main migration thread or multifd threads). Add support for this in libvirt in the form of a boolean VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION migration parameter.
Setting this to true with multifd will offload the check to multifd threads. Without multifd enabled, the check will happen in the main migration thread. Setting this to false disables zero-page checking. When the parameter is not set, QEMU's default is kept.
Signed-off-by: Tejus GK <tejus.gk@nutanix.com> --- docs/manpages/virsh.rst | 6 ++++++ include/libvirt/libvirt-domain.h | 16 ++++++++++++++++ src/qemu/qemu_migration.c | 1 + src/qemu/qemu_migration_params.c | 30 ++++++++++++++++++++++++++++++ src/qemu/qemu_migration_params.h | 1 + tools/virsh-domain.c | 11 +++++++++++ 6 files changed, 65 insertions(+)
I talked about this feature with Peter Xu and realized I missed one important thing when looking at the QEMU code. The QEMU default for this parameter is not "legacy". QEMU defaults to "multifd" and only machine types 8.2 and older override it to "legacy". In other words QEMU is already doing the right thing by default which makes this patch mostly useless. Theoretically users could want to disable zero page detection although I'm not sure what would be the benefit of doing so, or perhaps to force "multifd" for old machine types.
That said I think we can just drop this patch, but since I started writing my comments before talking to Peter, I'm sending them anyway in case you wanted to come up with an explanation of why this patch is useful.
I forgot to mention this in v1... we usually split such additions into three commits: - public API (include/libvirt/libvirt-domain.h) - virsh (docs/manpages/virsh.rst, tools/virsh-domain.c) - qemu implementation (src/qemu/*)
It's not a big deal for such a small change, but it makes individual commits focused on a single layer.
diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index f533a10776..b5a796a00a 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3802,6 +3802,7 @@ migrate [--bandwidth bandwidth] [--tls-destination hostname] [--disks-uri URI] [--copy-storage-synchronous-writes] [--available-switchover-bandwidth bandwidth] + [--zero-page-detection]
Migrate domain to another host. Add *--live* for live migration; <--p2p> for peer-2-peer migration; *--direct* for direct migration; or *--tunnelled* @@ -3953,6 +3954,11 @@ parallel connections. The number of such connections can be set using network link between the source and the target and thus speeding up the migration.
+*--zero-page-detection* requests pages containing only zeros to be detected by +the parallel migration threads rather than the main migration thread. Without +*--parallel*, zero pages are detected by the main migration thread. When the +option is not set, the hypervisor default is used. +
virsh options should provide the same functionality that is available via C API, this would only cover half of the new functionality
Running migration can be canceled by interrupting virsh (usually using ``Ctrl-C``) or by ``domjobabort`` command sent from another virsh instance.
diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 9ee153c6eb..0e68a572c3 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1468,6 +1468,22 @@ typedef enum { */ # define VIR_MIGRATE_PARAM_TLS_DESTINATION "tls.destination"
+/** + * VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION: + * + * virDomainMigrate* params field: whether zero page detection should be + * performed during migration. As VIR_TYPED_PARAM_BOOLEAN. + * + * When enabled with parallel migration, zero pages are detected by the multifd + * threads rather than the main migration thread. When enabled without parallel + * migration, zero pages are detected by the main migration thread.
I would consider this an implementation detail. The parameter is called "zero_page_detection" so we should describe it as such. When specified it would force enable/disable zero page detection and unused would let the hypervisor decide what to do.
+ * When disabled, zero pages are transferred as normal pages. + * When not set, the hypervisor default is used. + * + * Since: 12.9.0 + */ +# define VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION "zero_page_detection" + /* Domain migration. */ virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 76661de09a..13113ceaff 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -110,6 +110,7 @@ const virTypedParamValidationTemplate qemuMigrationParametersValidation[] = { { VIR_MIGRATE_PARAM_TLS_DESTINATION, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_DISKS_URI, VIR_TYPED_PARAM_STRING }, { VIR_MIGRATE_PARAM_BANDWIDTH_AVAIL_SWITCHOVER, VIR_TYPED_PARAM_ULLONG }, + { VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, VIR_TYPED_PARAM_BOOLEAN }, { "", 0 } };
diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index dd47516742..e1ead04068 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -131,6 +131,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "multifd-zstd-level", "avail-switchover-bandwidth", "direct-io", + "zero-page-detection", );
typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysOnItem; @@ -332,6 +333,9 @@ static const qemuMigrationParamInfoItem qemuMigrationParamInfo[] = { [QEMU_MIGRATION_PARAM_DIRECT_IO] = { .type = QEMU_MIGRATION_PARAM_TYPE_BOOL, }, + [QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION] = { + .type = QEMU_MIGRATION_PARAM_TYPE_STRING, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) == QEMU_MIGRATION_PARAM_LAST);
@@ -692,6 +696,29 @@ qemuMigrationParamsSetCompression(virTypedParameterPtr params, }
+static int +qemuMigrationParamsSetZeroPageDetection(virTypedParameterPtr params, + int nparams, + qemuMigrationParams *migParams) +{ + int zeroPageDetection; + int rc; + + if ((rc = virTypedParamsGetBoolean(params, nparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + &zeroPageDetection)) < 0) + return -1; + + if (rc == 1) { + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].value.s = + g_strdup(zeroPageDetection ? "multifd" : "none"); + migParams->params[QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION].set = true; + }
You need to check if the parameter even exists in QEMU as we support versions >= 7.2 and the parameter was introduced in QEMU 9.0. With old versions setting VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION would be allowed (such QEMU uses the equivalent of "legacy"), but disabling it would not be supported and should report an error.
+ + return 0; +} + + void qemuMigrationParamsSetBlockDirtyBitmapMapping(qemuMigrationParams *migParams, virJSONValue **params) @@ -792,6 +819,9 @@ qemuMigrationParamsFromFlags(virTypedParameterPtr params, if (qemuMigrationParamsSetCompression(params, nparams, flags, migParams) < 0) return NULL;
+ if (qemuMigrationParamsSetZeroPageDetection(params, nparams, migParams) < 0) + return NULL; + return g_steal_pointer(&migParams); }
diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index b7a829b85a..131fe82fc0 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -67,6 +67,7 @@ typedef enum { QEMU_MIGRATION_PARAM_MULTIFD_ZSTD_LEVEL, QEMU_MIGRATION_PARAM_AVAIL_SWITCHOVER_BANDWIDTH, QEMU_MIGRATION_PARAM_DIRECT_IO, + QEMU_MIGRATION_PARAM_ZERO_PAGE_DETECTION,
QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 2c64f19a19..18854ad0c3 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -11432,6 +11432,10 @@ static const vshCmdOptDef opts_migrate[] = { .type = VSH_OT_INT, .help = N_("bandwidth (in MiB/s) available for the final phase of migration") }, + {.name = "zero-page-detection", + .type = VSH_OT_BOOL, + .help = N_("enable zero page detection in parallel migration threads") + }, {.name = NULL} };
@@ -11733,6 +11737,13 @@ doMigrate(void *opaque) goto save_error; }
+ if (vshCommandOptBool(cmd, "zero-page-detection")) { + if (virTypedParamsAddBoolean(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_ZERO_PAGE_DETECTION, + true) < 0) + goto save_error; + } +
The implementation matches the description of the new option, but it does not match what the typed parameter can do. The parameter can be unspecified, enabled or disabled, but this option can only be unspecified or enabled.
if ((rv = vshCommandOptInt(ctl, cmd, "comp-zlib-level", &intOpt)) < 0) { goto out; } else if (rv > 0) {
Jirka
Hi Jiri, Peter!
Thank you for the review again! My main goal was to add this as an override for the older machine types you mentioned. A VM with an older machine type, but new enough QEMU will not have multifd zero-page checking kicking in for it, even though QEMU supports it. And we seem to be having a lot of such VMs in production, which don’t end up seeing the benefits of this offload. With the patch, clients could programmatically make QEMU use the offload for such VMs, without having to reboot the VM to change the machine type.
I can post a v3 based on your suggestions above, if you are okay with keeping the API for this. Please let me know your thoughts on it.
Regards, Tejus
Hi, a gentle ping for the above :)
If with the comment addressed first from Jiri (making sure it only applies to QEMU that supports zero-page-detection=multifd), it looks working, but then it'll be an API likely nobody will use after a short while: all people will be using multifd detections by default. Another option might be that Libvirt detects such support on both sides and enable it if both support it, silently with no API at all. I don't see a drawback indeed; multifd should always be better. QEMU can't do such for old machine types because QEMU doesn't have a way to know if the other side can understand the new protocol, but Libvirt knows. Then at least we can get rid of this API to maint. Jiri may have a better final word to say, though. Thanks, -- Peter Xu
On Fri, Nov 06, 2026 at 09:06:52 -0500, Peter Xu wrote:
If with the comment addressed first from Jiri (making sure it only applies to QEMU that supports zero-page-detection=multifd), it looks working, but then it'll be an API likely nobody will use after a short while: all people will be using multifd detections by default.
Another option might be that Libvirt detects such support on both sides and enable it if both support it, silently with no API at all. I don't see a drawback indeed; multifd should always be better.
QEMU can't do such for old machine types because QEMU doesn't have a way to know if the other side can understand the new protocol, but Libvirt knows. Then at least we can get rid of this API to maint.
Oh so this is the reason behind the legacy default for older machine types. In this case you are right, the best option for libvirt would be to check that both sides support this parameter and if so, unconditionally switch it to multifd. Jirka
participants (3)
-
Jiri Denemark -
Peter Xu -
Tejus GK