[libvirt] [PATCH 0/5] Implement virDomainMigrateGetMaxSpeed in qemu driver
This is a rebase of uncommitted patches in the series originally posted here https://www.redhat.com/archives/libvir-list/2011-August/msg01373.html Jim Fehlig (5): Store max migration bandwidth in qemuDomainObjPrivate struct Impl virDomainMigrateGetMaxSpeed in qemu driver Save migration speed in qemuDomainMigrateSetMaxSpeed Set qemu migration speed unlimited when migrating to file Use max bandwidth from qemuDomainObjPrivate struct when migrating src/qemu/qemu_domain.c | 2 + src/qemu/qemu_domain.h | 5 +++ src/qemu/qemu_driver.c | 65 +++++++++++++++++++++++++++++++------------- src/qemu/qemu_migration.c | 21 +++++++++++++- 4 files changed, 72 insertions(+), 21 deletions(-) -- 1.7.5.4
From: Jim Fehlig <jfehlig@novell.com> The maximum bandwidth that can be consumed when migrating a domain is better classified as an operational vs configuration parameter of the dommain. As such, store this parameter in qemuDomainObjPrivate structure. --- src/qemu/qemu_domain.c | 2 ++ src/qemu/qemu_domain.h | 5 +++++ 2 files changed, 7 insertions(+), 0 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 982bad6..d2cc2f0 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -215,6 +215,8 @@ static void *qemuDomainObjPrivateAlloc(void) if (qemuDomainObjInitJob(priv) < 0) VIR_FREE(priv); + priv->migMaxBandwidth = QEMU_DOMAIN_DEFAULT_MIG_BANDWIDTH_MAX; + return priv; } diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 55875fe..e44ff8a 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -36,6 +36,9 @@ (1 << VIR_DOMAIN_VIRT_KVM) | \ (1 << VIR_DOMAIN_VIRT_XEN)) +# define QEMU_DOMAIN_DEFAULT_MIG_BANDWIDTH_MAX (32 << 20) +# define QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX (INT64_MAX / (1024 * 1024)) + # define JOB_MASK(job) (1 << (job - 1)) # define DEFAULT_JOB_MASK \ (JOB_MASK(QEMU_JOB_QUERY) | \ @@ -115,6 +118,8 @@ struct _qemuDomainObjPrivate { bool fakeReboot; int jobs_queued; + + unsigned long migMaxBandwidth; }; struct qemuDomainWatchdogEvent -- 1.7.5.4
From: Jim Fehlig <jfehlig@novell.com> --- src/qemu/qemu_driver.c | 33 +++++++++++++++++++++++++++++++++ 1 files changed, 33 insertions(+), 0 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 321b07b..458787d 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -8880,6 +8880,38 @@ cleanup: return ret; } +static int +qemuDomainMigrateGetMaxSpeed(virDomainPtr dom, + unsigned long *bandwidth, + unsigned int flags) +{ + struct qemud_driver *driver = dom->conn->privateData; + virDomainObjPtr vm; + int ret = -1; + + virCheckFlags(0, -1); + + qemuDriverLock(driver); + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + qemuDriverUnlock(driver); + + if (!vm) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(dom->uuid, uuidstr); + qemuReportError(VIR_ERR_NO_DOMAIN, + _("no domain with matching uuid '%s'"), uuidstr); + goto cleanup; + } + + *bandwidth = vm->privateData->migMaxBandwidth; + ret = 0; + +cleanup: + if (vm) + virDomainObjUnlock(vm); + return ret; +} + static int qemuDomainSnapshotIsAllowed(virDomainObjPtr vm) { int i; @@ -10606,6 +10638,7 @@ static virDriver qemuDriver = { .domainAbortJob = qemuDomainAbortJob, /* 0.7.7 */ .domainMigrateSetMaxDowntime = qemuDomainMigrateSetMaxDowntime, /* 0.8.0 */ .domainMigrateSetMaxSpeed = qemuDomainMigrateSetMaxSpeed, /* 0.9.0 */ + .domainMigrateGetMaxSpeed = qemuDomainMigrateGetMaxSpeed, /* 0.9.5 */ .domainEventRegisterAny = qemuDomainEventRegisterAny, /* 0.8.0 */ .domainEventDeregisterAny = qemuDomainEventDeregisterAny, /* 0.8.0 */ .domainManagedSave = qemuDomainManagedSave, /* 0.8.0 */ -- 1.7.5.4
From: Jim Fehlig <jfehlig@novell.com> Now that migration speed is stored in qemuDomainObjPrivate structure, save the new value when invoking qemuDomainMigrateSetMaxSpeed(). Allow setting migration speed on inactive domain too. --- src/qemu/qemu_driver.c | 36 +++++++++++++++--------------------- 1 files changed, 15 insertions(+), 21 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 458787d..0ecd257 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -8848,31 +8848,25 @@ qemuDomainMigrateSetMaxSpeed(virDomainPtr dom, return -1; } - if (qemuDomainObjBeginJob(driver, vm, QEMU_JOB_MIGRATION_OP) < 0) - goto cleanup; - - if (!virDomainObjIsActive(vm)) { - qemuReportError(VIR_ERR_OPERATION_INVALID, - "%s", _("domain is not running")); - goto endjob; - } - priv = vm->privateData; + if (virDomainObjIsActive(vm)) { + if (qemuDomainObjBeginJob(driver, vm, QEMU_JOB_MIGRATION_OP) < 0) + goto cleanup; - if (priv->job.asyncJob != QEMU_ASYNC_JOB_MIGRATION_OUT) { - qemuReportError(VIR_ERR_OPERATION_INVALID, - "%s", _("domain is not being migrated")); - goto endjob; - } + VIR_DEBUG("Setting migration bandwidth to %luMbs", bandwidth); + qemuDomainObjEnterMonitor(driver, vm); + ret = qemuMonitorSetMigrationSpeed(priv->mon, bandwidth); + qemuDomainObjExitMonitor(driver, vm); - VIR_DEBUG("Setting migration bandwidth to %luMbs", bandwidth); - qemuDomainObjEnterMonitor(driver, vm); - ret = qemuMonitorSetMigrationSpeed(priv->mon, bandwidth); - qemuDomainObjExitMonitor(driver, vm); + if (ret == 0) + priv->migMaxBandwidth = bandwidth; -endjob: - if (qemuDomainObjEndJob(driver, vm) == 0) - vm = NULL; + if (qemuDomainObjEndJob(driver, vm) == 0) + vm = NULL; + } else { + priv->migMaxBandwidth = bandwidth; + ret = 0; + } cleanup: if (vm) -- 1.7.5.4
From: Jim Fehlig <jfehlig@novell.com> The qemu migration speed default is 32MiB/s as defined in migration.c /* Migration speed throttling */ static int64_t max_throttle = (32 << 20); There's no need to throttle migration when targeting a file, so set migration speed to unlimited prior to migration, and restore to libvirt default value after migration. Default units is MB for migrate_set_speed monitor command, so (INT64_MAX / (1024 * 1024)) is used for unlimited migration speed. Tested with both json and text monitors. --- src/qemu/qemu_migration.c | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 7fd4192..b882d06 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2718,6 +2718,16 @@ qemuMigrationToFile(struct qemud_driver *driver, virDomainObjPtr vm, bool restoreLabel = false; virCommandPtr cmd = NULL; int pipeFD[2] = { -1, -1 }; + unsigned long saveMigBandwidth = priv->migMaxBandwidth; + + /* Increase migration bandwidth to unlimited since target is a file. + * Failure to change migration speed is not fatal. */ + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) == 0) { + qemuMonitorSetMigrationSpeed(priv->mon, + QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX); + priv->migMaxBandwidth = QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX; + qemuDomainObjExitMonitorWithDriver(driver, vm); + } if (qemuCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATE_QEMU_FD) && (!compressor || pipe(pipeFD) == 0)) { @@ -2829,6 +2839,13 @@ qemuMigrationToFile(struct qemud_driver *driver, virDomainObjPtr vm, ret = 0; cleanup: + /* Restore max migration bandwidth */ + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) == 0) { + qemuMonitorSetMigrationSpeed(priv->mon, saveMigBandwidth); + priv->migMaxBandwidth = saveMigBandwidth; + qemuDomainObjExitMonitorWithDriver(driver, vm); + } + VIR_FORCE_CLOSE(pipeFD[0]); VIR_FORCE_CLOSE(pipeFD[1]); virCommandFree(cmd); -- 1.7.5.4
From: Jim Fehlig <jfehlig@novell.com> Adjust qemuMigrationRun() to use migMaxBandwidth in qemuDomainObjPrivate structure when setting qemu migration speed. Caller-specified 'resource' parameter overrides migMaxBandwidth. --- src/qemu/qemu_migration.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index b882d06..d9f8d93 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1437,6 +1437,7 @@ qemuMigrationRun(struct qemud_driver *driver, qemuMigrationCookiePtr mig = NULL; qemuMigrationIOThreadPtr iothread = NULL; int fd = -1; + unsigned long migrate_speed = resource ? resource : priv->migMaxBandwidth; VIR_DEBUG("driver=%p, vm=%p, cookiein=%s, cookieinlen=%d, " "cookieout=%p, cookieoutlen=%p, flags=%lx, resource=%lu, " @@ -1472,8 +1473,7 @@ qemuMigrationRun(struct qemud_driver *driver, QEMU_ASYNC_JOB_MIGRATION_OUT) < 0) goto cleanup; - if (resource > 0 && - qemuMonitorSetMigrationSpeed(priv->mon, resource) < 0) { + if (qemuMonitorSetMigrationSpeed(priv->mon, migrate_speed) < 0) { qemuDomainObjExitMonitorWithDriver(driver, vm); goto cleanup; } -- 1.7.5.4
On 09/14/2011 10:43 AM, Jim Fehlig wrote:
This is a rebase of uncommitted patches in the series originally posted here
https://www.redhat.com/archives/libvir-list/2011-August/msg01373.html
Jim Fehlig (5): Store max migration bandwidth in qemuDomainObjPrivate struct Impl virDomainMigrateGetMaxSpeed in qemu driver Save migration speed in qemuDomainMigrateSetMaxSpeed Set qemu migration speed unlimited when migrating to file Use max bandwidth from qemuDomainObjPrivate struct when migrating
ACK series - it seems silly to have a new virDomainMigrateGetMaxSpeed in the public symbols for 0.9.5 without at least one driver implementing it, and this looks like reasonable low-impact additions. Go ahead and push. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Eric Blake wrote:
On 09/14/2011 10:43 AM, Jim Fehlig wrote:
This is a rebase of uncommitted patches in the series originally posted here
https://www.redhat.com/archives/libvir-list/2011-August/msg01373.html
Jim Fehlig (5): Store max migration bandwidth in qemuDomainObjPrivate struct Impl virDomainMigrateGetMaxSpeed in qemu driver Save migration speed in qemuDomainMigrateSetMaxSpeed Set qemu migration speed unlimited when migrating to file Use max bandwidth from qemuDomainObjPrivate struct when migrating
ACK series - it seems silly to have a new virDomainMigrateGetMaxSpeed in the public symbols for 0.9.5 without at least one driver implementing it, and this looks like reasonable low-impact additions. Go ahead and push.
Thanks, pushed.
participants (2)
-
Eric Blake -
Jim Fehlig