On 11/30/23 01:25, Hyman Huang wrote: This needs to be 10.0.0 because even by the time you sent your patches we were in feature freeze for 9.10.0. Michal

Introudce option to enable dirty-limit convergence algorithim during live migration. Signed-off-by: Hyman Huang <yong.huang(a)smartx.com&gt; --- docs/manpages/virsh.rst | 10 +++++++++- src/libvirt-domain.c | 8 ++++++++ tools/virsh-domain.c | 10 ++++++++++ 3 files changed, 27 insertions(+), 1 deletion(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 3e7a4c6c22..d1d20cb5ba 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3376,7 +3376,7 @@ migrate [--compressed] [--comp-methods method-list] [--comp-mt-level] [--comp-mt-threads] [--comp-mt-dthreads] [--comp-xbzrle-cache] [--comp-zlib-level] [--comp-zstd-level] - [--auto-converge] [auto-converge-initial] + [--dirty-limit] [--auto-converge] [auto-converge-initial] [auto-converge-increment] [--persistent-xml file] [--tls] [--postcopy-bandwidth bandwidth] [--parallel [--parallel-connections connections]] @@ -3432,6 +3432,14 @@ source or destination host and the ``migrate`` command will report an error leaving the domain active on both hosts. To recover from such situation repeat the original ``migrate`` command with an additional *--postcopy-resume* flag. +*--dirty-limit* forces convergence using dirty-limit algorithms during live +migration. For QEMU/KVM, this means migration will throttle vCPUs as needed to +keep their dirty page rate within the migration parameter ``vcpu-dirty-limit`` +(1 megabits/s by default). This can improve the responsiveness of large guests +during live migration and result in more stable read performance. It requires +dirty-ring size configuration and conflicts with the traditional +``auto-converge`` algorithm. + *--auto-converge* forces convergence during live migration. The initial guest CPU throttling rate can be set with *auto-converge-initial*. If the initial throttling rate is not enough to ensure convergence, the rate is diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 77a9682ecb..dde60ab05b 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -4227,6 +4227,10 @@ virDomainMigrate3(virDomainPtr domain, /* Now checkout the destination */ virCheckReadOnlyGoto(dconn->flags, error); + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_MIGRATE_AUTO_CONVERGE, + VIR_MIGRATE_DIRTY_LIMIT, + error); + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_MIGRATE_NON_SHARED_DISK, VIR_MIGRATE_NON_SHARED_INC, error); @@ -4652,6 +4656,10 @@ virDomainMigrateToURI3(virDomainPtr domain, virCheckDomainReturn(domain, -1); virCheckReadOnlyGoto(domain->conn->flags, error); + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_MIGRATE_AUTO_CONVERGE, + VIR_MIGRATE_DIRTY_LIMIT, + error); + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_MIGRATE_TUNNELLED, VIR_MIGRATE_PARALLEL, error); diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 66f933dead..3f34230e6b 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -10981,6 +10981,10 @@ static const vshCmdOptDef opts_migrate[] = { .type = VSH_OT_BOOL, .help = N_("compress repeated pages during live migration") }, + {.name = "dirty-limit", + .type = VSH_OT_BOOL, + .help = N_("force convergence with dirty-limit algorithim during live migration") + }, {.name = "auto-converge", .type = VSH_OT_BOOL, .help = N_("force convergence during live migration") @@ -11167,6 +11171,7 @@ doMigrate(void *opaque) { "change-protection", VIR_MIGRATE_CHANGE_PROTECTION }, { "unsafe", VIR_MIGRATE_UNSAFE }, { "compressed", VIR_MIGRATE_COMPRESSED }, + { "dirty-limit", VIR_MIGRATE_DIRTY_LIMIT }, { "auto-converge", VIR_MIGRATE_AUTO_CONVERGE }, { "rdma-pin-all", VIR_MIGRATE_RDMA_PIN_ALL }, { "offline", VIR_MIGRATE_OFFLINE }, @@ -11193,6 +11198,11 @@ doMigrate(void *opaque) flags |= flagmap[i].migflag; } + if (flags & VIR_MIGRATE_DIRTY_LIMIT && flags & VIR_MIGRATE_AUTO_CONVERGE) { + vshError(ctl, "'--dirty-limit' conflicts with '--auto-converge'"); + goto out; + } + if (flags & VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES && !(flags & (VIR_MIGRATE_NON_SHARED_DISK | VIR_MIGRATE_NON_SHARED_DISK))) { vshError(ctl, "'--copy-storage-synchronous-writes' requires one of '--copy-storage-all', '--copy-storage-inc'"); -- 2.39.1

在 2023/12/20 22:27, Michal Prívozník 写道: Yes, that is what we would do after this series. Actually, Daniel commented on the patchset I supplied for that function, the specifics areas follows: https://patchew.org/Libvirt/169142376567.1452.17600824269401067909-0@git.... And i'm refactoring on that series and the work is kind of slow. IMHO, there can be two separate series: migration using the dirty-limit capability and the upper limit on the dirty page rate. So i posted the migration series firstly instead of dirty page rate upper limit. :) Ok, get it. > >> if (flags & VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES && >> !(flags & (VIR_MIGRATE_NON_SHARED_DISK | VIR_MIGRATE_NON_SHARED_DISK))) { >> vshError(ctl, "'--copy-storage-synchronous-writes' requires one of '--copy-storage-all', '--copy-storage-inc'"); > Also looks spurious, but pre-existing. > > Michal >

Signed-off-by: Hyman Huang <yong.huang(a)smartx.com&gt; --- NEWS.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index f12734c2a1..f44b350a68 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -17,6 +17,14 @@ v9.10.0 (unreleased) * **New features** + * QEMU: Support dirty-limit live migration + + Introduce ``VIR_MIGRATE_DIRTY_LIMIT`` flag for live migration API, + which can enable the dirty-limit convergence algorithm during live + migration and reap more stable read performance for guest vCPUs. + + Also, add the corresponding ``--dirty-limit`` option to ``virsh migrate``. + * Introduce pipewire audio backend The QEMU hypervisor driver now allows setting ``pipewire`` backend for -- 2.39.1