[libvirt] [PATCH v6 0/5] migration: add multithread compression

Add means to turn multithread compression on during migration. Add means to pass compression parameters in migration command. WARNING! This should be pushed only after https://www.redhat.com/archives/libvir-list/2016-March/msg01506.html or virsh domain will break. Changes from v5 =============== Originally i just wanted to send a rebase as i discovered that automatic rebase goes wrong for the patch 1 but later found a couple of bugs in the patch itself. Here they are: 1. fix qemuDomainMigratePrepare2 and qemuDomainMigratePerform3 passing NULL for compression. Changes from v4 =============== 1. Clean up documentation and comments. 2. Stop keeping compression options in flags internally. Move flags data into generic compression structure. 3. Use existing libvirt enum infrastructure to deal with compression methods. This makes parse and dump function less painful. 4. Use booleans for 'set' flags instead of bitsets. 5. Othes minor changes on Jiri comments. Eli Qiao (1): qemumonitorjsontest: add test for getting multithread compress params Nikolay Shirokovskiy (2): migration: qemu: add option to select compression methods qemu: migration: support setting compession parameters ShaoHe Feng (2): qemu: monitor: add migration parameters accessors virsh: add compression options for migration include/libvirt/libvirt-domain.h | 42 +++++++ src/qemu/qemu_driver.c | 60 +++++++-- src/qemu/qemu_migration.c | 262 +++++++++++++++++++++++++++++++++++---- src/qemu/qemu_migration.h | 36 ++++++ src/qemu/qemu_monitor.c | 24 +++- src/qemu/qemu_monitor.h | 18 +++ src/qemu/qemu_monitor_json.c | 110 ++++++++++++++++ src/qemu/qemu_monitor_json.h | 5 + tests/qemumonitorjsontest.c | 61 +++++++++ tools/virsh-domain.c | 76 ++++++++++++ tools/virsh.pod | 18 ++- 11 files changed, 671 insertions(+), 41 deletions(-) -- 1.8.3.1

Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> --- include/libvirt/libvirt-domain.h | 9 ++ src/qemu/qemu_driver.c | 60 +++++++++++--- src/qemu/qemu_migration.c | 173 +++++++++++++++++++++++++++++++++------ src/qemu/qemu_migration.h | 27 ++++++ src/qemu/qemu_monitor.c | 2 +- src/qemu/qemu_monitor.h | 1 + 6 files changed, 234 insertions(+), 38 deletions(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 8220ab0..cc8f7de 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -787,6 +787,15 @@ typedef enum { */ # define VIR_MIGRATE_PARAM_DISKS_PORT "disks_port" +/** + * VIR_MIGRATE_PARAM_COMPRESSION: + * + * virDomainMigrate* params multiple field: name of the method used to + * compress migration traffic. The parameter may be specified multiple times + * if more than one method should be used. + */ +# define VIR_MIGRATE_PARAM_COMPRESSION "compression" + /* Domain migration. */ virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index eaabe58..44e8aa0 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -12087,6 +12087,7 @@ qemuDomainMigratePrepare2(virConnectPtr dconn, virQEMUDriverPtr driver = dconn->privateData; virDomainDefPtr def = NULL; char *origname = NULL; + qemuMigrationCompressionPtr compression = NULL; int ret = -1; virCheckFlags(QEMU_MIGRATION_FLAGS, -1); @@ -12101,6 +12102,9 @@ qemuDomainMigratePrepare2(virConnectPtr dconn, goto cleanup; } + if (!(compression = qemuMigrationCompressionParse(NULL, 0, flags))) + goto cleanup; + if (virLockManagerPluginUsesState(driver->lockManager)) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Cannot use migrate v2 protocol with lock manager %s"), @@ -12121,9 +12125,11 @@ qemuDomainMigratePrepare2(virConnectPtr dconn, ret = qemuMigrationPrepareDirect(driver, dconn, NULL, 0, NULL, NULL, /* No cookies */ uri_in, uri_out, - &def, origname, NULL, 0, NULL, 0, flags); + &def, origname, NULL, 0, NULL, 0, + compression, flags); cleanup: + VIR_FREE(compression); VIR_FREE(origname); virDomainDefFree(def); return ret; @@ -12144,6 +12150,7 @@ qemuDomainMigratePerform(virDomainPtr dom, virDomainObjPtr vm; int ret = -1; const char *dconnuri = NULL; + qemuMigrationCompressionPtr compression = NULL; virCheckFlags(QEMU_MIGRATION_FLAGS, -1); @@ -12154,6 +12161,9 @@ qemuDomainMigratePerform(virDomainPtr dom, goto cleanup; } + if (!(compression = qemuMigrationCompressionParse(NULL, 0, flags))) + goto cleanup; + if (!(vm = qemuDomObjFromDomain(dom))) goto cleanup; @@ -12175,11 +12185,12 @@ qemuDomainMigratePerform(virDomainPtr dom, */ ret = qemuMigrationPerform(driver, dom->conn, vm, NULL, dconnuri, uri, NULL, NULL, 0, NULL, 0, - cookie, cookielen, + compression, cookie, cookielen, NULL, NULL, /* No output cookies in v2 */ flags, dname, resource, false); cleanup: + VIR_FREE(compression); return ret; } @@ -12323,6 +12334,7 @@ qemuDomainMigratePrepare3(virConnectPtr dconn, virQEMUDriverPtr driver = dconn->privateData; virDomainDefPtr def = NULL; char *origname = NULL; + qemuMigrationCompressionPtr compression = NULL; int ret = -1; virCheckFlags(QEMU_MIGRATION_FLAGS, -1); @@ -12337,6 +12349,9 @@ qemuDomainMigratePrepare3(virConnectPtr dconn, goto cleanup; } + if (!(compression = qemuMigrationCompressionParse(NULL, 0, flags))) + goto cleanup; + if (!(def = qemuMigrationPrepareDef(driver, dom_xml, dname, &origname))) goto cleanup; @@ -12347,9 +12362,11 @@ qemuDomainMigratePrepare3(virConnectPtr dconn, cookiein, cookieinlen, cookieout, cookieoutlen, uri_in, uri_out, - &def, origname, NULL, 0, NULL, 0, flags); + &def, origname, NULL, 0, NULL, 0, + compression, flags); cleanup: + VIR_FREE(compression); VIR_FREE(origname); virDomainDefFree(def); return ret; @@ -12377,6 +12394,7 @@ qemuDomainMigratePrepare3Params(virConnectPtr dconn, int nmigrate_disks; const char **migrate_disks = NULL; char *origname = NULL; + qemuMigrationCompressionPtr compression = NULL; int ret = -1; virCheckFlagsGoto(QEMU_MIGRATION_FLAGS, cleanup); @@ -12407,6 +12425,9 @@ qemuDomainMigratePrepare3Params(virConnectPtr dconn, if (nmigrate_disks < 0) goto cleanup; + if (!(compression = qemuMigrationCompressionParse(params, nparams, flags))) + goto cleanup; + if (flags & VIR_MIGRATE_TUNNELLED) { /* this is a logical error; we never should have gotten here with * VIR_MIGRATE_TUNNELLED set @@ -12428,10 +12449,11 @@ qemuDomainMigratePrepare3Params(virConnectPtr dconn, cookieout, cookieoutlen, uri_in, uri_out, &def, origname, listenAddress, - nmigrate_disks, migrate_disks, - nbdPort, flags); + nmigrate_disks, migrate_disks, nbdPort, + compression, flags); cleanup: + VIR_FREE(compression); VIR_FREE(migrate_disks); VIR_FREE(origname); virDomainDefFree(def); @@ -12551,22 +12573,32 @@ qemuDomainMigratePerform3(virDomainPtr dom, { virQEMUDriverPtr driver = dom->conn->privateData; virDomainObjPtr vm; + qemuMigrationCompressionPtr compression = NULL; + int ret = -1; virCheckFlags(QEMU_MIGRATION_FLAGS, -1); if (!(vm = qemuDomObjFromDomain(dom))) return -1; + if (!(compression = qemuMigrationCompressionParse(NULL, 0, flags))) + return -1; + if (virDomainMigratePerform3EnsureACL(dom->conn, vm->def) < 0) { virDomainObjEndAPI(&vm); - return -1; + goto cleanup; } - return qemuMigrationPerform(driver, dom->conn, vm, xmlin, - dconnuri, uri, NULL, NULL, 0, NULL, 0, - cookiein, cookieinlen, - cookieout, cookieoutlen, - flags, dname, resource, true); + ret = qemuMigrationPerform(driver, dom->conn, vm, xmlin, + dconnuri, uri, NULL, NULL, 0, NULL, 0, + compression, + cookiein, cookieinlen, + cookieout, cookieoutlen, + flags, dname, resource, true); + + cleanup: + VIR_FREE(compression); + return ret; } static int @@ -12591,6 +12623,7 @@ qemuDomainMigratePerform3Params(virDomainPtr dom, const char **migrate_disks = NULL; unsigned long long bandwidth = 0; int nbdPort = 0; + qemuMigrationCompressionPtr compression = NULL; int ret = -1; virCheckFlags(QEMU_MIGRATION_FLAGS, -1); @@ -12627,6 +12660,9 @@ qemuDomainMigratePerform3Params(virDomainPtr dom, if (nmigrate_disks < 0) goto cleanup; + if (!(compression = qemuMigrationCompressionParse(params, nparams, flags))) + goto cleanup; + if (!(vm = qemuDomObjFromDomain(dom))) goto cleanup; @@ -12638,9 +12674,11 @@ qemuDomainMigratePerform3Params(virDomainPtr dom, ret = qemuMigrationPerform(driver, dom->conn, vm, dom_xml, dconnuri, uri, graphicsuri, listenAddress, nmigrate_disks, migrate_disks, nbdPort, + compression, cookiein, cookieinlen, cookieout, cookieoutlen, flags, dname, bandwidth, true); cleanup: + VIR_FREE(compression); VIR_FREE(migrate_disks); return ret; } diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index e72c874..8fe0f0d 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -79,6 +79,11 @@ VIR_ENUM_IMPL(qemuMigrationJobPhase, QEMU_MIGRATION_PHASE_LAST, "finish3", ); +VIR_ENUM_IMPL(qemuMigrationCompressMethod, QEMU_MIGRATION_COMPRESS_LAST, + "xbzrle", + "mt", +); + enum qemuMigrationCookieFlags { QEMU_MIGRATION_COOKIE_FLAG_GRAPHICS, QEMU_MIGRATION_COOKIE_FLAG_LOCKSTATE, @@ -3414,6 +3419,29 @@ qemuMigrationPrepareIncoming(virDomainObjPtr vm, } static int +qemuMigrationSetCompression(virQEMUDriverPtr driver, + virDomainObjPtr vm, + qemuDomainAsyncJob job, + qemuMigrationCompressionPtr compression) +{ + if (qemuMigrationSetOption(driver, vm, + QEMU_MONITOR_MIGRATION_CAPS_XBZRLE, + compression->methods & + (1ULL << QEMU_MIGRATION_COMPRESS_XBZRLE), + job) < 0) + return -1; + + if (qemuMigrationSetOption(driver, vm, + QEMU_MONITOR_MIGRATION_CAPS_COMPRESS, + compression->methods & + (1ULL << QEMU_MIGRATION_COMPRESS_MULTITHREAD), + job) < 0) + return -1; + + return 0; +} + +static int qemuMigrationPrepareAny(virQEMUDriverPtr driver, virConnectPtr dconn, const char *cookiein, @@ -3430,6 +3458,7 @@ qemuMigrationPrepareAny(virQEMUDriverPtr driver, size_t nmigrate_disks, const char **migrate_disks, int nbdPort, + qemuMigrationCompressionPtr compression, unsigned long flags) { virDomainObjPtr vm = NULL; @@ -3627,10 +3656,8 @@ qemuMigrationPrepareAny(virQEMUDriverPtr driver, dataFD[1] = -1; /* 'st' owns the FD now & will close it */ } - if (qemuMigrationSetOption(driver, vm, - QEMU_MONITOR_MIGRATION_CAPS_XBZRLE, - flags & VIR_MIGRATE_COMPRESSED, - QEMU_ASYNC_JOB_MIGRATION_IN) < 0) + if (qemuMigrationSetCompression(driver, vm, QEMU_ASYNC_JOB_MIGRATION_IN, + compression) < 0) goto stopjob; if (STREQ_NULLABLE(protocol, "rdma") && @@ -3768,6 +3795,7 @@ qemuMigrationPrepareTunnel(virQEMUDriverPtr driver, const char *origname, unsigned long flags) { + qemuMigrationCompressionPtr compression = NULL; int ret; VIR_DEBUG("driver=%p, dconn=%p, cookiein=%s, cookieinlen=%d, " @@ -3782,9 +3810,14 @@ qemuMigrationPrepareTunnel(virQEMUDriverPtr driver, return -1; } + if (!(compression = qemuMigrationCompressionParse(NULL, 0, flags))) + return -1; + ret = qemuMigrationPrepareAny(driver, dconn, cookiein, cookieinlen, cookieout, cookieoutlen, def, origname, - st, NULL, 0, false, NULL, 0, NULL, 0, flags); + st, NULL, 0, false, NULL, 0, NULL, 0, + compression, flags); + VIR_FREE(compression); return ret; } @@ -3827,6 +3860,7 @@ qemuMigrationPrepareDirect(virQEMUDriverPtr driver, size_t nmigrate_disks, const char **migrate_disks, int nbdPort, + qemuMigrationCompressionPtr compression, unsigned long flags) { unsigned short port = 0; @@ -3949,7 +3983,8 @@ qemuMigrationPrepareDirect(virQEMUDriverPtr driver, cookieout, cookieoutlen, def, origname, NULL, uri ? uri->scheme : "tcp", port, autoPort, listenAddress, - nmigrate_disks, migrate_disks, nbdPort, flags); + nmigrate_disks, migrate_disks, nbdPort, + compression, flags); cleanup: virURIFree(uri); VIR_FREE(hostname); @@ -4457,7 +4492,8 @@ qemuMigrationRun(virQEMUDriverPtr driver, virConnectPtr dconn, const char *graphicsuri, size_t nmigrate_disks, - const char **migrate_disks) + const char **migrate_disks, + qemuMigrationCompressionPtr compression) { int ret = -1; unsigned int migrate_flags = QEMU_MONITOR_MIGRATE_BACKGROUND; @@ -4541,10 +4577,8 @@ qemuMigrationRun(virQEMUDriverPtr driver, goto cleanup; } - if (qemuMigrationSetOption(driver, vm, - QEMU_MONITOR_MIGRATION_CAPS_XBZRLE, - flags & VIR_MIGRATE_COMPRESSED, - QEMU_ASYNC_JOB_MIGRATION_OUT) < 0) + if (qemuMigrationSetCompression(driver, vm, QEMU_ASYNC_JOB_MIGRATION_OUT, + compression) < 0) goto cleanup; if (qemuMigrationSetOption(driver, vm, @@ -4785,7 +4819,8 @@ static int doNativeMigrate(virQEMUDriverPtr driver, virConnectPtr dconn, const char *graphicsuri, size_t nmigrate_disks, - const char **migrate_disks) + const char **migrate_disks, + qemuMigrationCompressionPtr compression) { qemuDomainObjPrivatePtr priv = vm->privateData; virURIPtr uribits = NULL; @@ -4835,7 +4870,8 @@ static int doNativeMigrate(virQEMUDriverPtr driver, ret = qemuMigrationRun(driver, vm, cookiein, cookieinlen, cookieout, cookieoutlen, flags, resource, &spec, dconn, - graphicsuri, nmigrate_disks, migrate_disks); + graphicsuri, nmigrate_disks, migrate_disks, + compression); if (spec.destType == MIGRATION_DEST_FD) VIR_FORCE_CLOSE(spec.dest.fd.qemu); @@ -4859,7 +4895,8 @@ static int doTunnelMigrate(virQEMUDriverPtr driver, virConnectPtr dconn, const char *graphicsuri, size_t nmigrate_disks, - const char **migrate_disks) + const char **migrate_disks, + qemuMigrationCompressionPtr compression) { virNetSocketPtr sock = NULL; int ret = -1; @@ -4896,7 +4933,8 @@ static int doTunnelMigrate(virQEMUDriverPtr driver, ret = qemuMigrationRun(driver, vm, cookiein, cookieinlen, cookieout, cookieoutlen, flags, resource, &spec, dconn, - graphicsuri, nmigrate_disks, migrate_disks); + graphicsuri, nmigrate_disks, migrate_disks, + compression); cleanup: if (spec.destType == MIGRATION_DEST_FD) { @@ -4934,6 +4972,7 @@ static int doPeer2PeerMigrate2(virQEMUDriverPtr driver, bool cancelled; virStreamPtr st = NULL; unsigned long destflags; + qemuMigrationCompressionPtr compression = NULL; VIR_DEBUG("driver=%p, sconn=%p, dconn=%p, vm=%p, dconnuri=%s, " "flags=%lx, dname=%s, resource=%lu", @@ -4955,6 +4994,9 @@ static int doPeer2PeerMigrate2(virQEMUDriverPtr driver, destflags = flags & ~(VIR_MIGRATE_ABORT_ON_ERROR | VIR_MIGRATE_AUTO_CONVERGE); + if (!(compression = qemuMigrationCompressionParse(NULL, 0, flags))) + goto cleanup; + VIR_DEBUG("Prepare2 %p", dconn); if (flags & VIR_MIGRATE_TUNNELLED) { /* @@ -5007,12 +5049,12 @@ static int doPeer2PeerMigrate2(virQEMUDriverPtr driver, ret = doTunnelMigrate(driver, vm, st, NULL, 0, NULL, NULL, flags, resource, dconn, - NULL, 0, NULL); + NULL, 0, NULL, compression); else ret = doNativeMigrate(driver, vm, uri_out, cookie, cookielen, NULL, NULL, /* No out cookie with v2 migration */ - flags, resource, dconn, NULL, 0, NULL); + flags, resource, dconn, NULL, 0, NULL, compression); /* Perform failed. Make sure Finish doesn't overwrite the error */ if (ret < 0) @@ -5054,6 +5096,7 @@ static int doPeer2PeerMigrate2(virQEMUDriverPtr driver, } VIR_FREE(uri_out); VIR_FREE(cookie); + VIR_FREE(compression); return ret; } @@ -5077,6 +5120,7 @@ doPeer2PeerMigrate3(virQEMUDriverPtr driver, size_t nmigrate_disks, const char **migrate_disks, int nbdPort, + qemuMigrationCompressionPtr compression, unsigned long long bandwidth, bool useParams, unsigned long flags) @@ -5159,6 +5203,10 @@ doPeer2PeerMigrate3(virQEMUDriverPtr driver, VIR_MIGRATE_PARAM_DISKS_PORT, nbdPort) < 0) goto cleanup; + + if (qemuMigrationCompressionDump(compression, ¶ms, &nparams, + &maxparams, &flags) < 0) + goto cleanup; } if (virDomainObjGetState(vm, NULL) == VIR_DOMAIN_PAUSED) @@ -5244,13 +5292,13 @@ doPeer2PeerMigrate3(virQEMUDriverPtr driver, cookiein, cookieinlen, &cookieout, &cookieoutlen, flags, bandwidth, dconn, graphicsuri, - nmigrate_disks, migrate_disks); + nmigrate_disks, migrate_disks, compression); } else { ret = doNativeMigrate(driver, vm, uri, cookiein, cookieinlen, &cookieout, &cookieoutlen, flags, bandwidth, dconn, graphicsuri, - nmigrate_disks, migrate_disks); + nmigrate_disks, migrate_disks, compression); } /* Perform failed. Make sure Finish doesn't overwrite the error */ @@ -5426,6 +5474,7 @@ static int doPeer2PeerMigrate(virQEMUDriverPtr driver, size_t nmigrate_disks, const char **migrate_disks, int nbdPort, + qemuMigrationCompressionPtr compression, unsigned long flags, const char *dname, unsigned long resource, @@ -5549,7 +5598,7 @@ static int doPeer2PeerMigrate(virQEMUDriverPtr driver, ret = doPeer2PeerMigrate3(driver, sconn, dconn, dconnuri, vm, xmlin, dname, uri, graphicsuri, listenAddress, nmigrate_disks, migrate_disks, nbdPort, - resource, useParams, flags); + compression, resource, useParams, flags); } else { ret = doPeer2PeerMigrate2(driver, sconn, dconn, vm, dconnuri, flags, dname, resource); @@ -5587,6 +5636,7 @@ qemuMigrationPerformJob(virQEMUDriverPtr driver, size_t nmigrate_disks, const char **migrate_disks, int nbdPort, + qemuMigrationCompressionPtr compression, const char *cookiein, int cookieinlen, char **cookieout, @@ -5623,12 +5673,13 @@ qemuMigrationPerformJob(virQEMUDriverPtr driver, ret = doPeer2PeerMigrate(driver, conn, vm, xmlin, dconnuri, uri, graphicsuri, listenAddress, nmigrate_disks, migrate_disks, nbdPort, - flags, dname, resource, &v3proto); + compression, flags, dname, resource, &v3proto); } else { qemuMigrationJobSetPhase(driver, vm, QEMU_MIGRATION_PHASE_PERFORM2); ret = doNativeMigrate(driver, vm, uri, cookiein, cookieinlen, cookieout, cookieoutlen, - flags, resource, NULL, NULL, 0, NULL); + flags, resource, NULL, NULL, 0, NULL, + compression); } if (ret < 0) goto endjob; @@ -5689,6 +5740,7 @@ qemuMigrationPerformPhase(virQEMUDriverPtr driver, const char *graphicsuri, size_t nmigrate_disks, const char **migrate_disks, + qemuMigrationCompressionPtr compression, const char *cookiein, int cookieinlen, char **cookieout, @@ -5714,7 +5766,7 @@ qemuMigrationPerformPhase(virQEMUDriverPtr driver, ret = doNativeMigrate(driver, vm, uri, cookiein, cookieinlen, cookieout, cookieoutlen, flags, resource, NULL, graphicsuri, - nmigrate_disks, migrate_disks); + nmigrate_disks, migrate_disks, compression); if (ret < 0) { if (qemuMigrationRestoreDomainState(conn, vm)) { @@ -5757,6 +5809,7 @@ qemuMigrationPerform(virQEMUDriverPtr driver, size_t nmigrate_disks, const char **migrate_disks, int nbdPort, + qemuMigrationCompressionPtr compression, const char *cookiein, int cookieinlen, char **cookieout, @@ -5787,7 +5840,7 @@ qemuMigrationPerform(virQEMUDriverPtr driver, return qemuMigrationPerformJob(driver, conn, vm, xmlin, dconnuri, uri, graphicsuri, listenAddress, nmigrate_disks, migrate_disks, nbdPort, - cookiein, cookieinlen, + compression, cookiein, cookieinlen, cookieout, cookieoutlen, flags, dname, resource, v3proto); } else { @@ -5801,14 +5854,14 @@ qemuMigrationPerform(virQEMUDriverPtr driver, return qemuMigrationPerformPhase(driver, conn, vm, uri, graphicsuri, nmigrate_disks, migrate_disks, - cookiein, cookieinlen, + compression, cookiein, cookieinlen, cookieout, cookieoutlen, flags, resource); } else { return qemuMigrationPerformJob(driver, conn, vm, xmlin, NULL, uri, graphicsuri, listenAddress, nmigrate_disks, migrate_disks, nbdPort, - cookiein, cookieinlen, + compression, cookiein, cookieinlen, cookieout, cookieoutlen, flags, dname, resource, v3proto); } @@ -6558,3 +6611,71 @@ qemuMigrationErrorReport(virQEMUDriverPtr driver, virSetError(err); virFreeError(err); } + +/* don't ever pass NULL params with non zero nparams */ +qemuMigrationCompressionPtr +qemuMigrationCompressionParse(virTypedParameterPtr params, int nparams, + unsigned long flags) +{ + size_t i; + qemuMigrationCompressionPtr compression = NULL; + + if (VIR_ALLOC(compression) < 0) + return NULL; + + for (i = 0; i < nparams; i++) { + int method; + + if (STRNEQ(params[i].field, VIR_MIGRATE_PARAM_COMPRESSION)) + continue; + + if ((method = qemuMigrationCompressMethodTypeFromString(params[i].value.s)) < 0) { + virReportError(VIR_ERR_INVALID_ARG, + _("Unsupported compression method '%s'"), + params[i].value.s); + goto error; + } + + if (compression->methods & (1ULL << method)) { + virReportError(VIR_ERR_INVALID_ARG, + _("Compression method '%s' is specified twice"), + params[i].value.s); + goto error; + } + + compression->methods |= 1ULL << method; + } + + if (!compression->methods && (flags & VIR_MIGRATE_COMPRESSED)) + compression->methods = 1ULL << QEMU_MIGRATION_COMPRESS_XBZRLE; + + return compression; + + error: + VIR_FREE(compression); + return NULL; +} + +int +qemuMigrationCompressionDump(qemuMigrationCompressionPtr compression, + virTypedParameterPtr *params, + int *nparams, int *maxparams, + unsigned long *flags) +{ + size_t i; + + if (compression->methods == 1ULL << QEMU_MIGRATION_COMPRESS_XBZRLE) { + *flags |= VIR_MIGRATE_COMPRESSED; + return 0; + } + + for (i = QEMU_MIGRATION_COMPRESS_XBZRLE; i < QEMU_MIGRATION_COMPRESS_LAST; ++i) { + if ((compression->methods & (1ULL << i)) && + virTypedParamsAddString(params, nparams, maxparams, + VIR_MIGRATE_PARAM_COMPRESSION, + qemuMigrationCompressMethodTypeToString(i)) < 0) + return -1; + } + + return 0; +} diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index a927bea..cb51ca5 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -25,6 +25,9 @@ # include "qemu_conf.h" # include "qemu_domain.h" +typedef struct _qemuMigrationCompression qemuMigrationCompression; +typedef qemuMigrationCompression *qemuMigrationCompressionPtr; + /* All supported qemu migration flags. */ # define QEMU_MIGRATION_FLAGS \ (VIR_MIGRATE_LIVE | \ @@ -55,6 +58,8 @@ VIR_MIGRATE_PARAM_MIGRATE_DISKS, VIR_TYPED_PARAM_STRING | \ VIR_TYPED_PARAM_MULTIPLE, \ VIR_MIGRATE_PARAM_DISKS_PORT, VIR_TYPED_PARAM_INT, \ + VIR_MIGRATE_PARAM_COMPRESSION, VIR_TYPED_PARAM_STRING | \ + VIR_TYPED_PARAM_MULTIPLE, \ NULL @@ -74,6 +79,26 @@ typedef enum { } qemuMigrationJobPhase; VIR_ENUM_DECL(qemuMigrationJobPhase) +typedef enum { + QEMU_MIGRATION_COMPRESS_XBZRLE = 0, + QEMU_MIGRATION_COMPRESS_MULTITHREAD, + + QEMU_MIGRATION_COMPRESS_LAST +} qemuMigrationCompressMethod; +VIR_ENUM_DECL(qemuMigrationCompressMethod) + +struct _qemuMigrationCompression { + unsigned long long methods; +}; + +qemuMigrationCompressionPtr +qemuMigrationCompressionParse(virTypedParameterPtr params, int nparams, + unsigned long flags); +int qemuMigrationCompressionDump(qemuMigrationCompressionPtr compression, + virTypedParameterPtr *params, + int *nparams, int *maxparams, + unsigned long *flags); + int qemuMigrationJobStart(virQEMUDriverPtr driver, virDomainObjPtr vm, qemuDomainAsyncJob job) @@ -137,6 +162,7 @@ int qemuMigrationPrepareDirect(virQEMUDriverPtr driver, size_t nmigrate_disks, const char **migrate_disks, int nbdPort, + qemuMigrationCompressionPtr compression, unsigned long flags); int qemuMigrationPerform(virQEMUDriverPtr driver, @@ -150,6 +176,7 @@ int qemuMigrationPerform(virQEMUDriverPtr driver, size_t nmigrate_disks, const char **migrate_disks, int nbdPort, + qemuMigrationCompressionPtr compression, const char *cookiein, int cookieinlen, char **cookieout, diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index b7e4fa9..ac9befa 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -168,7 +168,7 @@ VIR_ENUM_IMPL(qemuMonitorMigrationStatus, VIR_ENUM_IMPL(qemuMonitorMigrationCaps, QEMU_MONITOR_MIGRATION_CAPS_LAST, "xbzrle", "auto-converge", "rdma-pin-all", "events", - "postcopy-ram") + "postcopy-ram", "compress") VIR_ENUM_IMPL(qemuMonitorVMStatus, QEMU_MONITOR_VM_STATUS_LAST, diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index bb74917..78efcd7 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -551,6 +551,7 @@ typedef enum { QEMU_MONITOR_MIGRATION_CAPS_RDMA_PIN_ALL, QEMU_MONITOR_MIGRATION_CAPS_EVENTS, QEMU_MONITOR_MIGRATION_CAPS_POSTCOPY, + QEMU_MONITOR_MIGRATION_CAPS_COMPRESS, QEMU_MONITOR_MIGRATION_CAPS_LAST } qemuMonitorMigrationCaps; -- 1.8.3.1

On Thu, Apr 14, 2016 at 13:33:48 +0300, Nikolay Shirokovskiy wrote:
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> --- include/libvirt/libvirt-domain.h | 9 ++ src/qemu/qemu_driver.c | 60 +++++++++++--- src/qemu/qemu_migration.c | 173 +++++++++++++++++++++++++++++++++------ src/qemu/qemu_migration.h | 27 ++++++ src/qemu/qemu_monitor.c | 2 +- src/qemu/qemu_monitor.h | 1 + 6 files changed, 234 insertions(+), 38 deletions(-) ... diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index eaabe58..44e8aa0 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c ... @@ -12551,22 +12573,32 @@ qemuDomainMigratePerform3(virDomainPtr dom, { virQEMUDriverPtr driver = dom->conn->privateData; virDomainObjPtr vm; + qemuMigrationCompressionPtr compression = NULL; + int ret = -1;
virCheckFlags(QEMU_MIGRATION_FLAGS, -1);
if (!(vm = qemuDomObjFromDomain(dom))) return -1;
s/return -1/goto cleanup/
+ if (!(compression = qemuMigrationCompressionParse(NULL, 0, flags))) + return -1; +
And move this if statement before qemuDomObjFromDomain(), otherwise you'd need to call virDomainObjEndAPI() here in case of error, which would be rather ugly.
if (virDomainMigratePerform3EnsureACL(dom->conn, vm->def) < 0) { virDomainObjEndAPI(&vm); - return -1; + goto cleanup; }
- return qemuMigrationPerform(driver, dom->conn, vm, xmlin, - dconnuri, uri, NULL, NULL, 0, NULL, 0, - cookiein, cookieinlen, - cookieout, cookieoutlen, - flags, dname, resource, true); + ret = qemuMigrationPerform(driver, dom->conn, vm, xmlin, + dconnuri, uri, NULL, NULL, 0, NULL, 0, + compression, + cookiein, cookieinlen, + cookieout, cookieoutlen, + flags, dname, resource, true); + + cleanup: + VIR_FREE(compression); + return ret; }
static int
...
diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index e72c874..8fe0f0d 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c ... @@ -6558,3 +6611,71 @@ qemuMigrationErrorReport(virQEMUDriverPtr driver, virSetError(err); virFreeError(err); } + +/* don't ever pass NULL params with non zero nparams */ +qemuMigrationCompressionPtr +qemuMigrationCompressionParse(virTypedParameterPtr params, int nparams, + unsigned long flags) +{ + size_t i; + qemuMigrationCompressionPtr compression = NULL; + + if (VIR_ALLOC(compression) < 0) + return NULL; + + for (i = 0; i < nparams; i++) { + int method; + + if (STRNEQ(params[i].field, VIR_MIGRATE_PARAM_COMPRESSION)) + continue; + + if ((method = qemuMigrationCompressMethodTypeFromString(params[i].value.s)) < 0) {
This line is too long. ACK with the following squashed in: diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index cc8f7de..d813280 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -791,8 +791,9 @@ typedef enum { * VIR_MIGRATE_PARAM_COMPRESSION: * * virDomainMigrate* params multiple field: name of the method used to - * compress migration traffic. The parameter may be specified multiple times - * if more than one method should be used. + * compress migration traffic. Supported compression methods: xbzrle, mt. + * The parameter may be specified multiple times if more than one method + * should be used. */ # define VIR_MIGRATE_PARAM_COMPRESSION "compression" diff --git i/src/qemu/qemu_driver.c w/src/qemu/qemu_driver.c index 44e8aa0..e795062 100644 --- i/src/qemu/qemu_driver.c +++ w/src/qemu/qemu_driver.c @@ -12578,12 +12578,12 @@ qemuDomainMigratePerform3(virDomainPtr dom, virCheckFlags(QEMU_MIGRATION_FLAGS, -1); - if (!(vm = qemuDomObjFromDomain(dom))) - return -1; - if (!(compression = qemuMigrationCompressionParse(NULL, 0, flags))) return -1; + if (!(vm = qemuDomObjFromDomain(dom))) + goto cleanup; + if (virDomainMigratePerform3EnsureACL(dom->conn, vm->def) < 0) { virDomainObjEndAPI(&vm); goto cleanup; diff --git i/src/qemu/qemu_migration.c w/src/qemu/qemu_migration.c index 35b5748..f30fbfb 100644 --- i/src/qemu/qemu_migration.c +++ w/src/qemu/qemu_migration.c @@ -3434,7 +3434,7 @@ qemuMigrationSetCompression(virQEMUDriverPtr driver, if (qemuMigrationSetOption(driver, vm, QEMU_MONITOR_MIGRATION_CAPS_COMPRESS, compression->methods & - (1ULL << QEMU_MIGRATION_COMPRESS_MULTITHREAD), + (1ULL << QEMU_MIGRATION_COMPRESS_MT), job) < 0) return -1; @@ -6612,9 +6612,11 @@ qemuMigrationErrorReport(virQEMUDriverPtr driver, virFreeError(err); } + /* don't ever pass NULL params with non zero nparams */ qemuMigrationCompressionPtr -qemuMigrationCompressionParse(virTypedParameterPtr params, int nparams, +qemuMigrationCompressionParse(virTypedParameterPtr params, + int nparams, unsigned long flags) { size_t i; @@ -6629,7 +6631,8 @@ qemuMigrationCompressionParse(virTypedParameterPtr params, int nparams, if (STRNEQ(params[i].field, VIR_MIGRATE_PARAM_COMPRESSION)) continue; - if ((method = qemuMigrationCompressMethodTypeFromString(params[i].value.s)) < 0) { + method = qemuMigrationCompressMethodTypeFromString(params[i].value.s); + if (method < 0) { virReportError(VIR_ERR_INVALID_ARG, _("Unsupported compression method '%s'"), params[i].value.s); @@ -6659,7 +6662,8 @@ qemuMigrationCompressionParse(virTypedParameterPtr params, int nparams, int qemuMigrationCompressionDump(qemuMigrationCompressionPtr compression, virTypedParameterPtr *params, - int *nparams, int *maxparams, + int *nparams, + int *maxparams, unsigned long *flags) { size_t i; @@ -6669,7 +6673,7 @@ qemuMigrationCompressionDump(qemuMigrationCompressionPtr compression, return 0; } - for (i = QEMU_MIGRATION_COMPRESS_XBZRLE; i < QEMU_MIGRATION_COMPRESS_LAST; ++i) { + for (i = 0; i < QEMU_MIGRATION_COMPRESS_LAST; ++i) { if ((compression->methods & (1ULL << i)) && virTypedParamsAddString(params, nparams, maxparams, VIR_MIGRATE_PARAM_COMPRESSION, diff --git i/src/qemu/qemu_migration.h w/src/qemu/qemu_migration.h index cb51ca5..ec5145b 100644 --- i/src/qemu/qemu_migration.h +++ w/src/qemu/qemu_migration.h @@ -81,7 +81,7 @@ VIR_ENUM_DECL(qemuMigrationJobPhase) typedef enum { QEMU_MIGRATION_COMPRESS_XBZRLE = 0, - QEMU_MIGRATION_COMPRESS_MULTITHREAD, + QEMU_MIGRATION_COMPRESS_MT, QEMU_MIGRATION_COMPRESS_LAST } qemuMigrationCompressMethod; @@ -92,11 +92,13 @@ struct _qemuMigrationCompression { }; qemuMigrationCompressionPtr -qemuMigrationCompressionParse(virTypedParameterPtr params, int nparams, +qemuMigrationCompressionParse(virTypedParameterPtr params, + int nparams, unsigned long flags); int qemuMigrationCompressionDump(qemuMigrationCompressionPtr compression, virTypedParameterPtr *params, - int *nparams, int *maxparams, + int *nparams, + int *maxparams, unsigned long *flags); int qemuMigrationJobStart(virQEMUDriverPtr driver,

From: ShaoHe Feng <shaohe.feng@intel.com> Signed-off-by: ShaoHe Feng <shaohe.feng@intel.com> Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> --- src/qemu/qemu_monitor.c | 22 +++++++++ src/qemu/qemu_monitor.h | 17 +++++++ src/qemu/qemu_monitor_json.c | 110 +++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 5 ++ 4 files changed, 154 insertions(+) diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index ac9befa..83551a8 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -2177,6 +2177,28 @@ qemuMonitorSetMigrationCacheSize(qemuMonitorPtr mon, int +qemuMonitorGetMigrationCompression(qemuMonitorPtr mon, + qemuMonitorMigrationCompressionPtr compress) +{ + QEMU_CHECK_MONITOR_JSON(mon); + + return qemuMonitorJSONGetMigrationCompression(mon, compress); +} + +int +qemuMonitorSetMigrationCompression(qemuMonitorPtr mon, + qemuMonitorMigrationCompressionPtr compress) +{ + VIR_DEBUG("level=%d threads=%d dthreads=%d", + compress->level, compress->threads, compress->dthreads); + + QEMU_CHECK_MONITOR_JSON(mon); + + return qemuMonitorJSONSetMigrationCompression(mon, compress); +} + + +int qemuMonitorGetMigrationStats(qemuMonitorPtr mon, qemuMonitorMigrationStatsPtr stats) { diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 78efcd7..bd5d006 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -487,6 +487,23 @@ int qemuMonitorGetMigrationCacheSize(qemuMonitorPtr mon, int qemuMonitorSetMigrationCacheSize(qemuMonitorPtr mon, unsigned long long cacheSize); +typedef struct _qemuMonitorMigrationCompression qemuMonitorMigrationCompression; +typedef qemuMonitorMigrationCompression *qemuMonitorMigrationCompressionPtr; +struct _qemuMonitorMigrationCompression { + bool level_set; + bool threads_set; + bool dthreads_set; + + int level; + int threads; + int dthreads; +}; + +int qemuMonitorGetMigrationCompression(qemuMonitorPtr mon, + qemuMonitorMigrationCompressionPtr compress); +int qemuMonitorSetMigrationCompression(qemuMonitorPtr mon, + qemuMonitorMigrationCompressionPtr compress); + typedef enum { QEMU_MONITOR_MIGRATION_STATUS_INACTIVE, QEMU_MONITOR_MIGRATION_STATUS_SETUP, diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index 78af83e..29d6c8c 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -2521,6 +2521,116 @@ qemuMonitorJSONSetMigrationCacheSize(qemuMonitorPtr mon, } +int +qemuMonitorJSONGetMigrationCompression(qemuMonitorPtr mon, + qemuMonitorMigrationCompressionPtr compress) +{ + int ret = -1; + virJSONValuePtr result; + virJSONValuePtr cmd = NULL; + virJSONValuePtr reply = NULL; + + if (!(cmd = qemuMonitorJSONMakeCommand("query-migrate-parameters", NULL))) + return -1; + + if ((ret = qemuMonitorJSONCommand(mon, cmd, &reply)) < 0) + goto cleanup; + + if ((ret = qemuMonitorJSONCheckError(cmd, reply)) < 0) + goto cleanup; + + if (!(result = virJSONValueObjectGet(reply, "return"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("query-migrate-parameters reply was missing " + "'return' data")); + goto cleanup; + } + + if (virJSONValueObjectGetNumberInt(result, "compress-level", + &compress->level) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed/missing compress-level " + "in migrate parameters")); + goto cleanup; + } + compress->level_set = true; + + if (virJSONValueObjectGetNumberInt(result, "compress-threads", + &compress->threads) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed/missing compress-threads " + "in migrate parameters")); + goto cleanup; + } + compress->threads_set = true; + + if (virJSONValueObjectGetNumberInt(result, "decompress-threads", + &compress->dthreads) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed/missing decompress-threads " + "in migrate parameters")); + goto cleanup; + } + compress->dthreads_set = true; + + ret = 0; + cleanup: + virJSONValueFree(cmd); + virJSONValueFree(reply); + return ret; +} + +int +qemuMonitorJSONSetMigrationCompression(qemuMonitorPtr mon, + qemuMonitorMigrationCompressionPtr compress) +{ + int ret = -1; + virJSONValuePtr cmd = NULL; + virJSONValuePtr args = NULL; + virJSONValuePtr reply = NULL; + + if (!(cmd = virJSONValueNewObject())) + goto cleanup; + + if (virJSONValueObjectAppendString(cmd, "execute", + "migrate-set-parameters") < 0) + goto cleanup; + + if (!(args = virJSONValueNewObject())) + goto cleanup; + + if (compress->level_set && + virJSONValueObjectAppendNumberInt(args, "compress-level", + compress->level) < 0) + goto cleanup; + + if (compress->threads_set && + virJSONValueObjectAppendNumberInt(args, "compress-threads", + compress->threads) < 0) + goto cleanup; + + if (compress->dthreads_set && + virJSONValueObjectAppendNumberInt(args, "decompress-threads", + compress->dthreads) < 0) + goto cleanup; + + if (virJSONValueObjectAppend(cmd, "arguments", args) < 0) + goto cleanup; + args = NULL; + + if ((ret = qemuMonitorJSONCommand(mon, cmd, &reply)) < 0) + goto cleanup; + + ret = qemuMonitorJSONCheckError(cmd, reply); + + cleanup: + virJSONValueFree(cmd); + virJSONValueFree(args); + virJSONValueFree(reply); + return ret; +} + + static int qemuMonitorJSONGetMigrationStatsReply(virJSONValuePtr reply, qemuMonitorMigrationStatsPtr stats) diff --git a/src/qemu/qemu_monitor_json.h b/src/qemu/qemu_monitor_json.h index d155f52..5cbee1a 100644 --- a/src/qemu/qemu_monitor_json.h +++ b/src/qemu/qemu_monitor_json.h @@ -126,6 +126,11 @@ int qemuMonitorJSONGetMigrationCacheSize(qemuMonitorPtr mon, int qemuMonitorJSONSetMigrationCacheSize(qemuMonitorPtr mon, unsigned long long cacheSize); +int qemuMonitorJSONGetMigrationCompression(qemuMonitorPtr mon, + qemuMonitorMigrationCompressionPtr compress); +int qemuMonitorJSONSetMigrationCompression(qemuMonitorPtr mon, + qemuMonitorMigrationCompressionPtr compress); + int qemuMonitorJSONGetMigrationStats(qemuMonitorPtr mon, qemuMonitorMigrationStatsPtr stats); -- 1.8.3.1

On Thu, Apr 14, 2016 at 13:33:49 +0300, Nikolay Shirokovskiy wrote:
From: ShaoHe Feng <shaohe.feng@intel.com>
Signed-off-by: ShaoHe Feng <shaohe.feng@intel.com> Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> --- src/qemu/qemu_monitor.c | 22 +++++++++ src/qemu/qemu_monitor.h | 17 +++++++ src/qemu/qemu_monitor_json.c | 110 +++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 5 ++ 4 files changed, 154 insertions(+)
ACK Jirka

From: Eli Qiao <liyong.qiao@intel.com> Signed-off-by: Eli Qiao <liyong.qiao@intel.com> Signed-off-by: ShaoHe Feng <shaohe.feng@intel.com> Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> --- tests/qemumonitorjsontest.c | 61 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/tests/qemumonitorjsontest.c b/tests/qemumonitorjsontest.c index 1be0bee..ca15659 100644 --- a/tests/qemumonitorjsontest.c +++ b/tests/qemumonitorjsontest.c @@ -1593,6 +1593,66 @@ testQemuMonitorJSONqemuMonitorJSONGetBlockStatsInfo(const void *data) } static int +testQemuMonitorJSONqemuMonitorJSONGetMigrationCompression(const void *data) +{ + virDomainXMLOptionPtr xmlopt = (virDomainXMLOptionPtr)data; + qemuMonitorTestPtr test = qemuMonitorTestNewSimple(true, xmlopt); + qemuMonitorMigrationCompression compress; + int ret = -1; + + if (!test) + return -1; + + if (qemuMonitorTestAddItem(test, "query-migrate-parameters", + "{" + " \"return\": {" + " \"decompress-threads\": 2," + " \"compress-threads\": 8," + " \"compress-level\": 1" + " }" + "}") < 0) { + goto cleanup; + } + + if (qemuMonitorJSONGetMigrationCompression(qemuMonitorTestGetMonitor(test), + &compress) < 0) + goto cleanup; + + if (!compress.level_set || + !compress.threads_set || + !compress.dthreads_set) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + "One of level, threads or dthreads set flags is not set"); + return -1; + } + + if (compress.level != 1) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "Invalid decompress-threads: %d, expected 1", + compress.level); + goto cleanup; + } + if (compress.threads != 8) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "Invalid decompress-threads: %d, expected 8", + compress.threads); + goto cleanup; + } + if (compress.dthreads != 2) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "Invalid decompress-threads: %d, expected 2", + compress.dthreads); + goto cleanup; + } + ret = 0; + + cleanup: + qemuMonitorTestFree(test); + return ret; +} + + +static int testQemuMonitorJSONqemuMonitorJSONGetMigrationCacheSize(const void *data) { virDomainXMLOptionPtr xmlopt = (virDomainXMLOptionPtr)data; @@ -2333,6 +2393,7 @@ mymain(void) DO_TEST(qemuMonitorJSONGetBlockInfo); DO_TEST(qemuMonitorJSONGetBlockStatsInfo); DO_TEST(qemuMonitorJSONGetMigrationCacheSize); + DO_TEST(qemuMonitorJSONGetMigrationCompression); DO_TEST(qemuMonitorJSONGetMigrationStats); DO_TEST(qemuMonitorJSONGetChardevInfo); DO_TEST(qemuMonitorJSONSetBlockIoThrottle); -- 1.8.3.1

On Thu, Apr 14, 2016 at 13:33:50 +0300, Nikolay Shirokovskiy wrote:
From: Eli Qiao <liyong.qiao@intel.com>
Signed-off-by: Eli Qiao <liyong.qiao@intel.com> Signed-off-by: ShaoHe Feng <shaohe.feng@intel.com> Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> --- tests/qemumonitorjsontest.c | 61 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+)
ACK with the following trivial diff squashed in... Jirka diff --git i/tests/qemumonitorjsontest.c w/tests/qemumonitorjsontest.c index ca15659..829de90 100644 --- i/tests/qemumonitorjsontest.c +++ w/tests/qemumonitorjsontest.c @@ -1622,7 +1622,7 @@ testQemuMonitorJSONqemuMonitorJSONGetMigrationCompression(const void *data) !compress.threads_set || !compress.dthreads_set) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - "One of level, threads or dthreads set flags is not set"); + "One of level, threads or dthreads flags is not set"); return -1; }

Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> --- include/libvirt/libvirt-domain.h | 33 +++++++++++++++ src/qemu/qemu_migration.c | 91 +++++++++++++++++++++++++++++++++++++++- src/qemu/qemu_migration.h | 9 ++++ 3 files changed, 132 insertions(+), 1 deletion(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index cc8f7de..ccf0c3e 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -796,6 +796,39 @@ typedef enum { */ # define VIR_MIGRATE_PARAM_COMPRESSION "compression" +/** + * VIR_MIGRATE_PARAM_COMPRESSION_MT_LEVEL: + * + * virDomainMigrate* params field: the level of compression for multithread + * compression as VIR_TYPED_PARAM_INT. Accepted values are in range 0-9. + * 0 is no compression, 1 is maximum speed and 9 is maximum compression. + */ +# define VIR_MIGRATE_PARAM_COMPRESSION_MT_LEVEL "compression.mt.level" + +/** + * VIR_MIGRATE_PARAM_COMPRESSION_MT_THREADS: + * + * virDomainMigrate* params field: the number of compression threads for + * multithread compression as VIR_TYPED_PARAM_INT. + */ +# define VIR_MIGRATE_PARAM_COMPRESSION_MT_THREADS "compression.mt.threads" + +/** + * VIR_MIGRATE_PARAM_COMPRESSION_MT_DTHREADS: + * + * virDomainMigrate* params field: the number of decompression threads for + * multithread compression as VIR_TYPED_PARAM_INT. + */ +# define VIR_MIGRATE_PARAM_COMPRESSION_MT_DTHREADS "compression.mt.dthreads" + +/** + * VIR_MIGRATE_PARAM_COMPRESSION_XBZRLE_CACHE: + * + * virDomainMigrate* params field: the size of page cache for xbzrle + * compression as VIR_TYPED_PARAM_ULLONG. + */ +# define VIR_MIGRATE_PARAM_COMPRESSION_XBZRLE_CACHE "compression.xbzrle.cache" + /* Domain migration. */ virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 8fe0f0d..099ff92 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3424,6 +3424,10 @@ qemuMigrationSetCompression(virQEMUDriverPtr driver, qemuDomainAsyncJob job, qemuMigrationCompressionPtr compression) { + int ret = -1; + qemuDomainObjPrivatePtr priv = vm->privateData; + qemuMonitorMigrationCompressionPtr params = &compression->params; + if (qemuMigrationSetOption(driver, vm, QEMU_MONITOR_MIGRATION_CAPS_XBZRLE, compression->methods & @@ -3438,7 +3442,25 @@ qemuMigrationSetCompression(virQEMUDriverPtr driver, job) < 0) return -1; - return 0; + if (qemuDomainObjEnterMonitorAsync(driver, vm, job) < 0) + return -1; + + if ((params->level_set || params->threads_set || params->dthreads_set) && + qemuMonitorSetMigrationCompression(priv->mon, params) < 0) + goto cleanup; + + if (compression->xbzrle_cache_set && + qemuMonitorSetMigrationCacheSize(priv->mon, + compression->xbzrle_cache) < 0) + goto cleanup; + + ret = 0; + + cleanup: + if (qemuDomainObjExitMonitor(driver, vm) < 0) + ret = -1; + + return ret; } static int @@ -6619,6 +6641,7 @@ qemuMigrationCompressionParse(virTypedParameterPtr params, int nparams, { size_t i; qemuMigrationCompressionPtr compression = NULL; + qemuMonitorMigrationCompressionPtr cparams; if (VIR_ALLOC(compression) < 0) return NULL; @@ -6646,6 +6669,47 @@ qemuMigrationCompressionParse(virTypedParameterPtr params, int nparams, compression->methods |= 1ULL << method; } +#define GET_PARAM(PARAM, TYPE, PARENT, VALUE) \ + do { \ + int rc; \ + \ + if ((rc = virTypedParamsGet ## TYPE(params, nparams, \ + PARAM, &PARENT->VALUE)) < 0) \ + goto error; \ + \ + if (rc == 1) \ + PARENT->VALUE ## _set = true; \ + } while (0) + + cparams = &compression->params; + + if (params) { + GET_PARAM(VIR_MIGRATE_PARAM_COMPRESSION_MT_LEVEL, Int, + cparams, level); + GET_PARAM(VIR_MIGRATE_PARAM_COMPRESSION_MT_THREADS, Int, + cparams, threads); + GET_PARAM(VIR_MIGRATE_PARAM_COMPRESSION_MT_DTHREADS, Int, + cparams, dthreads); + GET_PARAM(VIR_MIGRATE_PARAM_COMPRESSION_XBZRLE_CACHE, ULLong, + compression, xbzrle_cache); + } + +#undef GET_PARAM + + if ((cparams->level_set || cparams->threads_set || cparams->dthreads_set) && + !(compression->methods & (1ULL << QEMU_MIGRATION_COMPRESS_MULTITHREAD))) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("Turn multithread compression on to tune it")); + goto error; + } + + if (compression->xbzrle_cache_set && + !(compression->methods & (1ULL << QEMU_MIGRATION_COMPRESS_XBZRLE))) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("Turn xbzrle compression on to tune it")); + goto error; + } + if (!compression->methods && (flags & VIR_MIGRATE_COMPRESSED)) compression->methods = 1ULL << QEMU_MIGRATION_COMPRESS_XBZRLE; @@ -6663,6 +6727,7 @@ qemuMigrationCompressionDump(qemuMigrationCompressionPtr compression, unsigned long *flags) { size_t i; + qemuMonitorMigrationCompressionPtr cparams = &compression->params; if (compression->methods == 1ULL << QEMU_MIGRATION_COMPRESS_XBZRLE) { *flags |= VIR_MIGRATE_COMPRESSED; @@ -6677,5 +6742,29 @@ qemuMigrationCompressionDump(qemuMigrationCompressionPtr compression, return -1; } + if (cparams->level_set && + virTypedParamsAddInt(params, nparams, maxparams, + VIR_MIGRATE_PARAM_COMPRESSION_MT_LEVEL, + cparams->level) < 0) + return -1; + + if (cparams->threads_set && + virTypedParamsAddInt(params, nparams, maxparams, + VIR_MIGRATE_PARAM_COMPRESSION_MT_THREADS, + cparams->threads) < 0) + return -1; + + if (cparams->dthreads_set && + virTypedParamsAddInt(params, nparams, maxparams, + VIR_MIGRATE_PARAM_COMPRESSION_MT_DTHREADS, + cparams->dthreads) < 0) + return -1; + + if (compression->xbzrle_cache_set && + virTypedParamsAddULLong(params, nparams, maxparams, + VIR_MIGRATE_PARAM_COMPRESSION_XBZRLE_CACHE, + compression->xbzrle_cache) < 0) + return -1; + return 0; } diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index cb51ca5..ea49928 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -60,6 +60,10 @@ typedef qemuMigrationCompression *qemuMigrationCompressionPtr; VIR_MIGRATE_PARAM_DISKS_PORT, VIR_TYPED_PARAM_INT, \ VIR_MIGRATE_PARAM_COMPRESSION, VIR_TYPED_PARAM_STRING | \ VIR_TYPED_PARAM_MULTIPLE, \ + VIR_MIGRATE_PARAM_COMPRESSION_MT_LEVEL, VIR_TYPED_PARAM_INT, \ + VIR_MIGRATE_PARAM_COMPRESSION_MT_THREADS, VIR_TYPED_PARAM_INT, \ + VIR_MIGRATE_PARAM_COMPRESSION_MT_DTHREADS, VIR_TYPED_PARAM_INT, \ + VIR_MIGRATE_PARAM_COMPRESSION_XBZRLE_CACHE, VIR_TYPED_PARAM_ULLONG, \ NULL @@ -89,6 +93,11 @@ VIR_ENUM_DECL(qemuMigrationCompressMethod) struct _qemuMigrationCompression { unsigned long long methods; + + qemuMonitorMigrationCompression params; + + bool xbzrle_cache_set; + unsigned long long xbzrle_cache; }; qemuMigrationCompressionPtr -- 1.8.3.1

On Thu, Apr 14, 2016 at 13:33:51 +0300, Nikolay Shirokovskiy wrote:
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> --- include/libvirt/libvirt-domain.h | 33 +++++++++++++++ src/qemu/qemu_migration.c | 91 +++++++++++++++++++++++++++++++++++++++- src/qemu/qemu_migration.h | 9 ++++ 3 files changed, 132 insertions(+), 1 deletion(-) ... diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 8fe0f0d..099ff92 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c ... @@ -6663,6 +6727,7 @@ qemuMigrationCompressionDump(qemuMigrationCompressionPtr compression, unsigned long *flags) { size_t i; + qemuMonitorMigrationCompressionPtr cparams = &compression->params;
if (compression->methods == 1ULL << QEMU_MIGRATION_COMPRESS_XBZRLE) {
You should also check whether xbzrle_cache is set, otherwise you'd lose that settings if only xbzrle method was chosen.
*flags |= VIR_MIGRATE_COMPRESSED;
ACK with the following trivial patch squashed in... Jirka diff --git i/src/qemu/qemu_migration.c w/src/qemu/qemu_migration.c index 3e4243a..b9e5fc2 100644 --- i/src/qemu/qemu_migration.c +++ w/src/qemu/qemu_migration.c @@ -6700,7 +6700,7 @@ qemuMigrationCompressionParse(virTypedParameterPtr params, #undef GET_PARAM if ((cparams->level_set || cparams->threads_set || cparams->dthreads_set) && - !(compression->methods & (1ULL << QEMU_MIGRATION_COMPRESS_MULTITHREAD))) { + !(compression->methods & (1ULL << QEMU_MIGRATION_COMPRESS_MT))) { virReportError(VIR_ERR_INVALID_ARG, "%s", _("Turn multithread compression on to tune it")); goto error; @@ -6733,7 +6733,8 @@ qemuMigrationCompressionDump(qemuMigrationCompressionPtr compression, size_t i; qemuMonitorMigrationCompressionPtr cparams = &compression->params; - if (compression->methods == 1ULL << QEMU_MIGRATION_COMPRESS_XBZRLE) { + if (compression->methods == 1ULL << QEMU_MIGRATION_COMPRESS_XBZRLE && + !compression->xbzrle_cache_set) { *flags |= VIR_MIGRATE_COMPRESSED; return 0; }

From: ShaoHe Feng <shaohe.feng@intel.com> Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> --- tools/virsh-domain.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 18 ++++++++++--- 2 files changed, 91 insertions(+), 3 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 6d4265c..800829e 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -9783,6 +9783,26 @@ static const vshCmdOptDef opts_migrate[] = { .type = VSH_OT_BOOL, .help = N_("compress repeated pages during live migration") }, + {.name = "comp-methods", + .type = VSH_OT_STRING, + .help = N_("comma separated list of compression methods to be used") + }, + {.name = "comp-mt-level", + .type = VSH_OT_INT, + .help = N_("compress level for multithread compression") + }, + {.name = "comp-mt-threads", + .type = VSH_OT_INT, + .help = N_("number of compession threads for multithread compression") + }, + {.name = "comp-mt-dthreads", + .type = VSH_OT_INT, + .help = N_("number of decompession threads for multithread compression") + }, + {.name = "comp-xbzrle-cache", + .type = VSH_OT_INT, + .help = N_("page cache size for xbzrle compression") + }, {.name = "auto-converge", .type = VSH_OT_BOOL, .help = N_("force convergence during live migration") @@ -9863,6 +9883,9 @@ doMigrate(void *opaque) virTypedParameterPtr params = NULL; int nparams = 0; int maxparams = 0; + int intOpt = 0; + unsigned long long ullOpt = 0; + int rv; virConnectPtr dconn = data->dconn; sigemptyset(&sigmask); @@ -9930,6 +9953,59 @@ doMigrate(void *opaque) VIR_FREE(val); } + if (vshCommandOptStringReq(ctl, cmd, "comp-methods", &opt) < 0) + goto out; + if (opt) { + char **val = virStringSplit(opt, ",", 0); + + if (virTypedParamsAddStringList(¶ms, + &nparams, + &maxparams, + VIR_MIGRATE_PARAM_COMPRESSION, + (const char **)val) < 0) { + VIR_FREE(val); + goto save_error; + } + + VIR_FREE(val); + } + + if ((rv = vshCommandOptInt(ctl, cmd, "comp-mt-level", &intOpt)) < 0) { + goto out; + } else if (rv > 0) { + if (virTypedParamsAddInt(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_COMPRESSION_MT_LEVEL, + intOpt) < 0) + goto save_error; + } + + if ((rv = vshCommandOptInt(ctl, cmd, "comp-mt-threads", &intOpt)) < 0) { + goto out; + } else if (rv > 0) { + if (virTypedParamsAddInt(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_COMPRESSION_MT_THREADS, + intOpt) < 0) + goto save_error; + } + + if ((rv = vshCommandOptInt(ctl, cmd, "comp-mt-dthreads", &intOpt)) < 0) { + goto out; + } else if (rv > 0) { + if (virTypedParamsAddInt(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_COMPRESSION_MT_DTHREADS, + intOpt) < 0) + goto save_error; + } + + if ((rv = vshCommandOptULongLong(ctl, cmd, "comp-xbzrle-cache", &ullOpt)) < 0) { + goto out; + } else if (rv > 0) { + if (virTypedParamsAddULLong(¶ms, &nparams, &maxparams, + VIR_MIGRATE_PARAM_COMPRESSION_XBZRLE_CACHE, + ullOpt) < 0) + goto save_error; + } + if (vshCommandOptStringReq(ctl, cmd, "xml", &opt) < 0) goto out; if (opt) { diff --git a/tools/virsh.pod b/tools/virsh.pod index d2cc5b2..7cf5552 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -1537,11 +1537,14 @@ to the I<uri> namespace is displayed instead of being modified. =item B<migrate> [I<--live>] [I<--offline>] [I<--direct>] [I<--p2p> [I<--tunnelled>]] [I<--persistent>] [I<--undefinesource>] [I<--suspend>] [I<--copy-storage-all>] [I<--copy-storage-inc>] [I<--change-protection>] [I<--unsafe>] [I<--verbose>] -[I<--compressed>] [I<--abort-on-error>] [I<--auto-converge>] [I<--postcopy>] +[I<--abort-on-error>] [I<--auto-converge>] [I<--postcopy>] [I<--postcopy-after-precopy>] I<domain> I<desturi> [I<migrateuri>] [I<graphicsuri>] [I<listen-address>] [I<dname>] [I<--timeout> B<seconds> [I<--timeout-suspend> | I<--timeout-postcopy>]] [I<--xml> B<file>] [I<--migrate-disks> B<disk-list>] [I<--disks-port> B<port>] +[I<--compressed>] [I<--comp-methods> B<method-list>] +[I<--comp-mt-level>] [I<--comp-mt-threads>] [I<--comp-mt-dthreads>] +[I<--comp-xbzrle-cache>] Migrate domain to another host. Add I<--live> for live migration; <--p2p> for peer-2-peer migration; I<--direct> for direct migration; or I<--tunnelled> @@ -1564,8 +1567,7 @@ enforces that no incompatible configuration changes will be made to the domain while the migration is underway; this flag is implicitly enabled when supported by the hypervisor, but can be explicitly used to reject the migration if the hypervisor lacks change protection support. I<--verbose> displays the progress -of migration. I<--compressed> activates compression of memory pages that have -to be transferred repeatedly during live migration. I<--abort-on-error> cancels +of migration. I<--abort-on-error> cancels the migration if a soft error (for example I/O error) happens during the migration. I<--auto-converge> forces convergence during live migration. I<--postcopy> enables post-copy logic in migration, but does not @@ -1602,6 +1604,16 @@ I<--timeout-postcopy> is used, virsh will switch migration from pre-copy to post-copy upon timeout; migration has to be started with I<--postcopy> option for this to work. +I<--compressed> activates compression, the compression method is chosen +with I<--comp-methods>. Supported methods are "mt" and "xbzrle" and +can be used in any combination. When no methods are specified, a hypervisor +default methods will be used. QEMU defaults to "xbzrle". Compression methods +can be tuned further. I<--comp-mt-level> sets compression level. +Values are in range from 0 to 9, where 1 is maximum speed and 9 is maximum +compression. I<--comp-mt-threads> and I<--comp-mt-dthreads> set the number +of compress threads on source and the number of decompress threads on target +respectively. I<--comp-xbzrle-cache> sets size of page cache in bytes. + Running migration can be canceled by interrupting virsh (usually using C<Ctrl-C>) or by B<domjobabort> command sent from another virsh instance. -- 1.8.3.1

On Thu, Apr 14, 2016 at 13:33:52 +0300, Nikolay Shirokovskiy wrote:
From: ShaoHe Feng <shaohe.feng@intel.com>
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com> --- tools/virsh-domain.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 18 ++++++++++--- 2 files changed, 91 insertions(+), 3 deletions(-)
ACK Jirka

On Thu, Apr 14, 2016 at 13:33:47 +0300, Nikolay Shirokovskiy wrote:
Add means to turn multithread compression on during migration. Add means to pass compression parameters in migration command.
I found a few nits in some of the patches, but otherwise the series looks good. Please, look at the changes I suggested in individual replies and let me know if you agree with them. If so, I'll squash them in your patches and push the result. Thanks. Jirka

On 14.04.2016 15:39, Jiri Denemark wrote:
On Thu, Apr 14, 2016 at 13:33:47 +0300, Nikolay Shirokovskiy wrote:
Add means to turn multithread compression on during migration. Add means to pass compression parameters in migration command.
I found a few nits in some of the patches, but otherwise the series looks good. Please, look at the changes I suggested in individual replies and let me know if you agree with them. If so, I'll squash them in your patches and push the result.
Thanks.
Jirka
I checked all the diffs you sent and agree with all of them. Thanks for reviewing! Nikolay

On Thu, Apr 14, 2016 at 15:55:31 +0300, Nikolay Shirokovskiy wrote:
On 14.04.2016 15:39, Jiri Denemark wrote:
On Thu, Apr 14, 2016 at 13:33:47 +0300, Nikolay Shirokovskiy wrote:
Add means to turn multithread compression on during migration. Add means to pass compression parameters in migration command.
I found a few nits in some of the patches, but otherwise the series looks good. Please, look at the changes I suggested in individual replies and let me know if you agree with them. If so, I'll squash them in your patches and push the result.
Thanks.
Jirka
I checked all the diffs you sent and agree with all of them. Thanks for reviewing!
OK, I pushed this series. Jirka
participants (2)
-
Jiri Denemark
-
Nikolay Shirokovskiy