[libvirt] [PATCH] qemu: Use unlimited speed when migrating to file
This reverts commit 61f2b6ba5fdef0613d6351b99913b5ef468144ce and most of commit d8916dc8e2f612ab3ce46f32c4bfeb0bd73f6007, which effectively brings back commit ef1065cf5acad13767c054758cbe7f4e8af5d241 written by Jim Fehlig: The qemu migration speed default is 32MiB/s as defined in migration.c /* Migration speed throttling */ static int64_t max_throttle = (32 << 20); There's no need to throttle migration when targeting a file, so set migration speed to unlimited prior to migration, and restore to libvirt default value after migration. Default units is MB for migrate_set_speed monitor command, so (INT64_MAX / (1024 * 1024)) is used for unlimited migration speed. This was reverted because migration to file could not be canceled and even monitored since qemu were not processing any monitor commands until the migration finished. This is now different as we make sure the file descriptor we pass to qemu is able to properly report EAGAIN. Recent qemu changes might have helped as well. I tested managedsave with this patch in and indeed, it is 10x faster while I can still monitor its progress. --- src/qemu/qemu_domain.h | 7 +++++++ src/qemu/qemu_migration.c | 17 +++++++++++++++++ 2 files changed, 24 insertions(+), 0 deletions(-) diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index f8e943f..be9e34d 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -39,6 +39,13 @@ (1 << VIR_DOMAIN_VIRT_XEN)) # define QEMU_DOMAIN_DEFAULT_MIG_BANDWIDTH_MAX 32 +# if ULONG_MAX == 4294967295 +/* Qemu has a 64-bit limit, but we are limited by our historical choice of + * representing bandwidth in a long instead of a 64-bit int. */ +# define QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX ULONG_MAX +# else +# define QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX (INT64_MAX / (1024 * 1024)) +# endif # define JOB_MASK(job) (1 << (job - 1)) # define DEFAULT_JOB_MASK \ diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 81b2d5b..c0cf636 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2943,6 +2943,16 @@ qemuMigrationToFile(struct qemud_driver *driver, virDomainObjPtr vm, bool restoreLabel = false; virCommandPtr cmd = NULL; int pipeFD[2] = { -1, -1 }; + unsigned long saveMigBandwidth = priv->migMaxBandwidth; + + /* Increase migration bandwidth to unlimited since target is a file. + * Failure to change migration speed is not fatal. */ + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) == 0) { + qemuMonitorSetMigrationSpeed(priv->mon, + QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX); + priv->migMaxBandwidth = QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX; + qemuDomainObjExitMonitorWithDriver(driver, vm); + } if (qemuCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATE_QEMU_FD) && (!compressor || pipe(pipeFD) == 0)) { @@ -3054,6 +3064,13 @@ qemuMigrationToFile(struct qemud_driver *driver, virDomainObjPtr vm, ret = 0; cleanup: + /* Restore max migration bandwidth */ + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) == 0) { + qemuMonitorSetMigrationSpeed(priv->mon, saveMigBandwidth); + priv->migMaxBandwidth = saveMigBandwidth; + qemuDomainObjExitMonitorWithDriver(driver, vm); + } + VIR_FORCE_CLOSE(pipeFD[0]); VIR_FORCE_CLOSE(pipeFD[1]); virCommandFree(cmd); -- 1.7.8.5
On 03/21/2012 08:11 AM, Jiri Denemark wrote:
This reverts commit 61f2b6ba5fdef0613d6351b99913b5ef468144ce and most of commit d8916dc8e2f612ab3ce46f32c4bfeb0bd73f6007, which effectively brings back commit ef1065cf5acad13767c054758cbe7f4e8af5d241 written by Jim Fehlig:
That history makes it look like we can't make up our minds :) But also important in the churn is indeed the intermediate changes to our migration code passing in a pipe, and improvements in qemu.
The qemu migration speed default is 32MiB/s as defined in migration.c
/* Migration speed throttling */ static int64_t max_throttle = (32 << 20);
There's no need to throttle migration when targeting a file, so set migration speed to unlimited prior to migration, and restore to libvirt default value after migration.
Default units is MB for migrate_set_speed monitor command, so (INT64_MAX / (1024 * 1024)) is used for unlimited migration speed.
This was reverted because migration to file could not be canceled and even monitored since qemu were not processing any monitor commands until the migration finished. This is now different as we make sure the file descriptor we pass to qemu is able to properly report EAGAIN. Recent qemu changes might have helped as well.
I tested managedsave with this patch in and indeed, it is 10x faster while I can still monitor its progress. --- src/qemu/qemu_domain.h | 7 +++++++ src/qemu/qemu_migration.c | 17 +++++++++++++++++ 2 files changed, 24 insertions(+), 0 deletions(-)
ACK. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Wed, Mar 21, 2012 at 08:40:23 -0600, Eric Blake wrote:
On 03/21/2012 08:11 AM, Jiri Denemark wrote:
This reverts commit 61f2b6ba5fdef0613d6351b99913b5ef468144ce and most of commit d8916dc8e2f612ab3ce46f32c4bfeb0bd73f6007, which effectively brings back commit ef1065cf5acad13767c054758cbe7f4e8af5d241 written by Jim Fehlig:
That history makes it look like we can't make up our minds :) But also important in the churn is indeed the intermediate changes to our migration code passing in a pipe, and improvements in qemu.
ACK.
Pushed, thanks. Jirka
participants (2)
-
Eric Blake -
Jiri Denemark