[libvirt] [PATCH 0/3] libxl: support Xen migration stream V2
Hi all, This patch adds support for Xen migration stream V2 to the libvirt libxl driver. In the process it fixes save/restore and migration tests in OSSTest, which have been failing since libvirt commit e7440656. Patch1 changes the libxl API requirement from version 4.2 to 4.4, enabling use of an extended libxl_domain_create_restore() function that allows passing restore parameters such as stream version. Patch2 adds support for migration stream V2 in the basic save/restore logic, taking advantage of a save image header that includes a version field. The version is set to '2' if the host produces a V2 migration stream. This patch fixes the failing save/restore tests in OSSTest. Patch3 adds support for migration stream V2 in the migration logic. The migration code does not use the save image header and instead uses libvirt's migration protocol to transfer domain configuration and other such goodies from source to destination. The patch enables sending version information in the Begin and Prepare migration phases so the correct stream version information can be passed to libxl_domain_create_restore(). An upshot of this approach is safely detecting and aborting a migration from a V2 host to a V1-only host. This patch fixes the failing migration tests in OSSTest. Jim Fehlig (3): libxl: switch to using libxl_domain_create_restore from v4.4 API libxl: support Xen migration stream V2 in save/restore libxl: support migration stream V2 in migration configure.ac | 9 +- src/libxl/libxl_conf.h | 6 +- src/libxl/libxl_domain.c | 46 ++++++++-- src/libxl/libxl_domain.h | 14 ++- src/libxl/libxl_driver.c | 27 +++--- src/libxl/libxl_migration.c | 204 +++++++++++++++++++++++++++++++++++++++++++- src/libxl/libxl_migration.h | 6 +- 7 files changed, 282 insertions(+), 30 deletions(-) -- 2.1.4
In LIBXL_API_VERSION 0x040400, the libxl_domain_create_restore API gained a parameter for specifying restore parameters. Switch to using version 0x040400, which will be useful in a subsequent commit to specify the Xen migration stream version when restoring. Signed-off-by: Jim Fehlig <jfehlig@suse.com> --- configure.ac | 9 +++++---- src/libxl/libxl_domain.c | 6 +++++- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/configure.ac b/configure.ac index 88e2e20..0da0f75 100644 --- a/configure.ac +++ b/configure.ac @@ -875,10 +875,11 @@ if test "$with_libxl" != "no" ; then fi fi -# Until there is a need to use enhancements of libxl APIs such as -# libxl_domain_create_restore and libxl_set_vcpuaffinity, stick with -# the APIs as defined in libxl API version 4.2.0. -LIBXL_CFLAGS="$LIBXL_CFLAGS -DLIBXL_API_VERSION=0x040200" +# LIBXL_API_VERSION 4.4.0 introduced a new parameter to +# libxl_domain_create_restore for specifying restore parameters. +# The libxl driver will make use of this new parameter for specifying +# the Xen migration stream version. +LIBXL_CFLAGS="$LIBXL_CFLAGS -DLIBXL_API_VERSION=0x040400" LIBS="$old_LIBS" CFLAGS="$old_CFLAGS" diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 14a900c..32ad946 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -1028,6 +1028,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, virDomainObjPtr vm, libxlDriverConfigPtr cfg; virHostdevManagerPtr hostdev_mgr = driver->hostdevMgr; libxl_asyncprogress_how aop_console_how; + libxl_domain_restore_params params; libxl_domain_config_init(&d_config); @@ -1115,8 +1116,11 @@ libxlDomainStart(libxlDriverPrivatePtr driver, virDomainObjPtr vm, ret = libxl_domain_create_new(cfg->ctx, &d_config, &domid, NULL, &aop_console_how); } else { + libxl_domain_restore_params_init(¶ms); ret = libxl_domain_create_restore(cfg->ctx, &d_config, &domid, - restore_fd, NULL, &aop_console_how); + restore_fd, ¶ms, NULL, + &aop_console_how); + libxl_domain_restore_params_dispose(¶ms); } virObjectLock(vm); -- 2.1.4
On Mon, May 02, 2016 at 07:01:17PM -0600, Jim Fehlig wrote:
In LIBXL_API_VERSION 0x040400, the libxl_domain_create_restore API gained a parameter for specifying restore parameters. Switch to using version 0x040400, which will be useful in a subsequent commit to specify the Xen migration stream version when restoring.
Signed-off-by: Jim Fehlig <jfehlig@suse.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
--- configure.ac | 9 +++++---- src/libxl/libxl_domain.c | 6 +++++- 2 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/configure.ac b/configure.ac index 88e2e20..0da0f75 100644 --- a/configure.ac +++ b/configure.ac @@ -875,10 +875,11 @@ if test "$with_libxl" != "no" ; then fi fi
-# Until there is a need to use enhancements of libxl APIs such as -# libxl_domain_create_restore and libxl_set_vcpuaffinity, stick with -# the APIs as defined in libxl API version 4.2.0. -LIBXL_CFLAGS="$LIBXL_CFLAGS -DLIBXL_API_VERSION=0x040200" +# LIBXL_API_VERSION 4.4.0 introduced a new parameter to +# libxl_domain_create_restore for specifying restore parameters. +# The libxl driver will make use of this new parameter for specifying +# the Xen migration stream version. +LIBXL_CFLAGS="$LIBXL_CFLAGS -DLIBXL_API_VERSION=0x040400" LIBS="$old_LIBS" CFLAGS="$old_CFLAGS"
diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 14a900c..32ad946 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -1028,6 +1028,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, virDomainObjPtr vm, libxlDriverConfigPtr cfg; virHostdevManagerPtr hostdev_mgr = driver->hostdevMgr; libxl_asyncprogress_how aop_console_how; + libxl_domain_restore_params params;
libxl_domain_config_init(&d_config);
@@ -1115,8 +1116,11 @@ libxlDomainStart(libxlDriverPrivatePtr driver, virDomainObjPtr vm, ret = libxl_domain_create_new(cfg->ctx, &d_config, &domid, NULL, &aop_console_how); } else { + libxl_domain_restore_params_init(¶ms); ret = libxl_domain_create_restore(cfg->ctx, &d_config, &domid, - restore_fd, NULL, &aop_console_how); + restore_fd, ¶ms, NULL, + &aop_console_how); + libxl_domain_restore_params_dispose(¶ms); } virObjectLock(vm);
-- 2.1.4
On Mon, May 02, Jim Fehlig wrote:
In LIBXL_API_VERSION 0x040400, the libxl_domain_create_restore API gained a parameter for specifying restore parameters. Switch to using version 0x040400, which will be useful in a subsequent commit to specify the Xen migration stream version when restoring.
This breaks compilation with xen-4.3 etc. Is this intentional? Olaf
On 05/11/2016 12:01 AM, Olaf Hering wrote:
On Mon, May 02, Jim Fehlig wrote:
In LIBXL_API_VERSION 0x040400, the libxl_domain_create_restore API gained a parameter for specifying restore parameters. Switch to using version 0x040400, which will be useful in a subsequent commit to specify the Xen migration stream version when restoring. This breaks compilation with xen-4.3 etc. Is this intentional?
Yes. Did you see my message elsewhere in the thread when I notified I was pushing the series? https://www.redhat.com/archives/libvir-list/2016-May/msg00686.html Regards, Jim
On Wed, May 11, Jim Fehlig wrote:
https://www.redhat.com/archives/libvir-list/2016-May/msg00686.html
Perhaps configure.ac should be changed to reject libxl upfront. It looks like the assignment of LIBXL_CFLAGS should be moved up, so that AC_CHECK_LIB gets the -DLIBXL_API_VERSION as well. I will provide a patch for that. Olaf
Olaf Hering wrote:
On Wed, May 11, Jim Fehlig wrote:
https://www.redhat.com/archives/libvir-list/2016-May/msg00686.html
Perhaps configure.ac should be changed to reject libxl upfront.
Yes, good idea.
It looks like the assignment of LIBXL_CFLAGS should be moved up, so that AC_CHECK_LIB gets the -DLIBXL_API_VERSION as well. I will provide a patch for that.
Ok, thanks for volunteering! Regards, Jim
Xen 4.6 introduced a new migration stream commonly referred to as "migration V2". Xen 4.6 and newer always produce this new stream, whereas Xen 4.5 and older always produce the legacy stream. Support for migration stream V2 can be detected at build time with LIBXL_HAVE_SRM_V2 from libxl.h. The legacy and V2 streams are not compatible, but a V2 host can accept and convert a legacy stream. Commit e7440656 changed the libxl driver to use the lowest libxl API version possible (version 0x040200) to ensure the driver builds against older Xen releases. The old 4.2 restore API does not support specifying a stream version and assumes a legacy stream, even if the incoming stream is migration V2. Thinking it has been given a legacy stream, libxl will fail to convert an incoming stream that is already V2, which causes the entire restore operation to fail. Xen's libvirt-related OSSTest has been failing since commit e7440656 landed in libvirt.git master. One of the more recent failures can be seen here http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg00071.html This patch changes the call to libxl_domain_create_restore() to include the stream version if LIBXL_HAVE_SRM_V2 is defined. The version field of the libxlSavefileHeader struct is also updated to '2' when LIBXL_HAVE_SRM_V2 is defined, ensuring the stream version in the header matches the actual stream version produced by Xen. Along with bumping the libxl API requirement to 0x040400, this patch fixes save/restore on a migration V2 Xen host. Oddly, migration has never used the libxlSavefileHeader. It handles passing configuration in the Begin and Prepare phases, and then calls libxl directly to transfer domain state/memory in the Perform phase. A subsequent patch will add stream version handling in the Begin and Prepare phase handshaking, which will fix the migration related OSSTest failures. Signed-off-by: Jim Fehlig <jfehlig@suse.com> --- src/libxl/libxl_conf.h | 6 +++++- src/libxl/libxl_domain.c | 40 ++++++++++++++++++++++++++++++++++++---- src/libxl/libxl_domain.h | 14 ++++++++++---- src/libxl/libxl_driver.c | 13 ++++++++----- src/libxl/libxl_migration.c | 2 +- 5 files changed, 60 insertions(+), 15 deletions(-) diff --git a/src/libxl/libxl_conf.h b/src/libxl/libxl_conf.h index 24e2911..c5b9429 100644 --- a/src/libxl/libxl_conf.h +++ b/src/libxl/libxl_conf.h @@ -148,7 +148,11 @@ struct _libxlDriverPrivate { }; # define LIBXL_SAVE_MAGIC "libvirt-xml\n \0 \r" -# define LIBXL_SAVE_VERSION 1 +# ifdef LIBXL_HAVE_SRM_V2 +# define LIBXL_SAVE_VERSION 2 +# else +# define LIBXL_SAVE_VERSION 1 +# endif typedef struct _libxlSavefileHeader libxlSavefileHeader; typedef libxlSavefileHeader *libxlSavefileHeaderPtr; diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 32ad946..5fa1bd9 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -514,7 +514,7 @@ libxlDomainShutdownThread(void *opaque) } libxlDomainDestroyInternal(driver, vm); libxlDomainCleanup(driver, vm); - if (libxlDomainStart(driver, vm, false, -1) < 0) { + if (libxlDomainStartNew(driver, vm, false) < 0) { virErrorPtr err = virGetLastError(); VIR_ERROR(_("Failed to restart VM '%s': %s"), vm->def->name, err ? err->message : _("unknown error")); @@ -1006,14 +1006,23 @@ libxlDomainCreateIfaceNames(virDomainDefPtr def, libxl_domain_config *d_config) } +#ifdef LIBXL_HAVE_SRM_V2 +# define LIBXL_DOMSTART_RESTORE_VER_ATTR /* empty */ +#else +# define LIBXL_DOMSTART_RESTORE_VER_ATTR ATTRIBUTE_UNUSED +#endif + /* * Start a domain through libxenlight. * * virDomainObjPtr must be locked and a job acquired on invocation */ -int -libxlDomainStart(libxlDriverPrivatePtr driver, virDomainObjPtr vm, - bool start_paused, int restore_fd) +static int +libxlDomainStart(libxlDriverPrivatePtr driver, + virDomainObjPtr vm, + bool start_paused, + int restore_fd, + uint32_t restore_ver LIBXL_DOMSTART_RESTORE_VER_ATTR) { libxl_domain_config d_config; virDomainDefPtr def = NULL; @@ -1049,6 +1058,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, virDomainObjPtr vm, goto cleanup; restore_fd = managed_save_fd; + restore_ver = hdr.version; if (STRNEQ(vm->def->name, def->name) || memcmp(vm->def->uuid, def->uuid, VIR_UUID_BUFLEN)) { @@ -1117,6 +1127,9 @@ libxlDomainStart(libxlDriverPrivatePtr driver, virDomainObjPtr vm, &domid, NULL, &aop_console_how); } else { libxl_domain_restore_params_init(¶ms); +#ifdef LIBXL_HAVE_SRM_V2 + params.stream_version = restore_ver; +#endif ret = libxl_domain_create_restore(cfg->ctx, &d_config, &domid, restore_fd, ¶ms, NULL, &aop_console_how); @@ -1203,6 +1216,25 @@ libxlDomainStart(libxlDriverPrivatePtr driver, virDomainObjPtr vm, return ret; } +int +libxlDomainStartNew(libxlDriverPrivatePtr driver, + virDomainObjPtr vm, + bool start_paused) +{ + return libxlDomainStart(driver, vm, start_paused, -1, LIBXL_SAVE_VERSION); +} + +int +libxlDomainStartRestore(libxlDriverPrivatePtr driver, + virDomainObjPtr vm, + bool start_paused, + int restore_fd, + uint32_t restore_ver) +{ + return libxlDomainStart(driver, vm, start_paused, + restore_fd, restore_ver); +} + bool libxlDomainDefCheckABIStability(libxlDriverPrivatePtr driver, virDomainDefPtr src, diff --git a/src/libxl/libxl_domain.h b/src/libxl/libxl_domain.h index 1c1eba3..c53adaa 100644 --- a/src/libxl/libxl_domain.h +++ b/src/libxl/libxl_domain.h @@ -142,10 +142,16 @@ libxlDomainSetVcpuAffinities(libxlDriverPrivatePtr driver, virDomainObjPtr vm); int -libxlDomainStart(libxlDriverPrivatePtr driver, - virDomainObjPtr vm, - bool start_paused, - int restore_fd); +libxlDomainStartNew(libxlDriverPrivatePtr driver, + virDomainObjPtr vm, + bool start_paused); + +int +libxlDomainStartRestore(libxlDriverPrivatePtr driver, + virDomainObjPtr vm, + bool start_paused, + int restore_fd, + uint32_t restore_ver); bool libxlDomainDefCheckABIStability(libxlDriverPrivatePtr driver, diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index bf97c9c..8977ae2 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -323,7 +323,7 @@ libxlAutostartDomain(virDomainObjPtr vm, } if (vm->autostart && !virDomainObjIsActive(vm) && - libxlDomainStart(driver, vm, false, -1) < 0) { + libxlDomainStartNew(driver, vm, false) < 0) { err = virGetLastError(); VIR_ERROR(_("Failed to autostart VM '%s': %s"), vm->def->name, @@ -998,8 +998,8 @@ libxlDomainCreateXML(virConnectPtr conn, const char *xml, goto cleanup; } - if (libxlDomainStart(driver, vm, (flags & VIR_DOMAIN_START_PAUSED) != 0, - -1) < 0) { + if (libxlDomainStartNew(driver, vm, + (flags & VIR_DOMAIN_START_PAUSED) != 0) < 0) { if (!vm->persistent) { virDomainObjListRemove(driver->domains, vm); vm = NULL; @@ -1818,7 +1818,9 @@ libxlDomainRestoreFlags(virConnectPtr conn, const char *from, goto cleanup; } - ret = libxlDomainStart(driver, vm, (flags & VIR_DOMAIN_SAVE_PAUSED) != 0, fd); + ret = libxlDomainStartRestore(driver, vm, + (flags & VIR_DOMAIN_SAVE_PAUSED) != 0, + fd, hdr.version); if (ret < 0 && !vm->persistent) virDomainObjListRemove(driver->domains, vm); @@ -2681,7 +2683,8 @@ libxlDomainCreateWithFlags(virDomainPtr dom, goto endjob; } - ret = libxlDomainStart(driver, vm, (flags & VIR_DOMAIN_START_PAUSED) != 0, -1); + ret = libxlDomainStartNew(driver, vm, + (flags & VIR_DOMAIN_START_PAUSED) != 0); if (ret < 0) goto endjob; dom->id = vm->def->id; diff --git a/src/libxl/libxl_migration.c b/src/libxl/libxl_migration.c index ab1f76e..1d4ec5e 100644 --- a/src/libxl/libxl_migration.c +++ b/src/libxl/libxl_migration.c @@ -106,7 +106,7 @@ libxlDoMigrateReceive(void *opaque) * Always start the domain paused. If needed, unpause in the * finish phase, after transfer of the domain is complete. */ - ret = libxlDomainStart(driver, vm, true, recvfd); + ret = libxlDomainStartRestore(driver, vm, true, recvfd, LIBXL_SAVE_VERSION); if (ret < 0 && !vm->persistent) remove_dom = true; -- 2.1.4
Similar to "support Xen migration stream V2 in save/restore", add support for indicating the migration stream version in the migration code. To accomplish this, add a minimal migration cookie in the libxl driver that is passed between source and destination hosts. Initially, the cookie is only used in the Begin and Prepare phases of migration to communicate the version of the migration stream produced by the source. Signed-off-by: Jim Fehlig <jfehlig@suse.com> --- src/libxl/libxl_driver.c | 14 +-- src/libxl/libxl_migration.c | 204 +++++++++++++++++++++++++++++++++++++++++++- src/libxl/libxl_migration.h | 6 +- 3 files changed, 213 insertions(+), 11 deletions(-) diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index 8977ae2..062d6f8 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -5151,8 +5151,8 @@ static char * libxlDomainMigrateBegin3Params(virDomainPtr domain, virTypedParameterPtr params, int nparams, - char **cookieout ATTRIBUTE_UNUSED, - int *cookieoutlen ATTRIBUTE_UNUSED, + char **cookieout, + int *cookieoutlen, unsigned int flags) { const char *xmlin = NULL; @@ -5193,15 +5193,16 @@ libxlDomainMigrateBegin3Params(virDomainPtr domain, return NULL; } - return libxlDomainMigrationBegin(domain->conn, vm, xmlin); + return libxlDomainMigrationBegin(domain->conn, vm, xmlin, + cookieout, cookieoutlen); } static int libxlDomainMigratePrepare3Params(virConnectPtr dconn, virTypedParameterPtr params, int nparams, - const char *cookiein ATTRIBUTE_UNUSED, - int cookieinlen ATTRIBUTE_UNUSED, + const char *cookiein, + int cookieinlen, char **cookieout ATTRIBUTE_UNUSED, int *cookieoutlen ATTRIBUTE_UNUSED, char **uri_out, @@ -5240,7 +5241,8 @@ libxlDomainMigratePrepare3Params(virConnectPtr dconn, if (virDomainMigratePrepare3ParamsEnsureACL(dconn, def) < 0) goto error; - if (libxlDomainMigrationPrepare(dconn, &def, uri_in, uri_out, flags) < 0) + if (libxlDomainMigrationPrepare(dconn, &def, uri_in, uri_out, + cookiein, cookieinlen, flags) < 0) goto error; return 0; diff --git a/src/libxl/libxl_migration.c b/src/libxl/libxl_migration.c index 1d4ec5e..ad54960 100644 --- a/src/libxl/libxl_migration.c +++ b/src/libxl/libxl_migration.c @@ -48,6 +48,18 @@ VIR_LOG_INIT("libxl.libxl_migration"); +typedef struct _libxlMigrationCookie libxlMigrationCookie; +typedef libxlMigrationCookie *libxlMigrationCookiePtr; +struct _libxlMigrationCookie { + /* Host properties */ + char *srcHostname; + uint32_t xenMigStreamVer; + + /* Guest properties */ + unsigned char uuid[VIR_UUID_BUFLEN]; + char *name; +}; + typedef struct _libxlMigrationDstArgs { virObject parent; @@ -55,6 +67,7 @@ typedef struct _libxlMigrationDstArgs { virConnectPtr conn; virDomainObjPtr vm; unsigned int flags; + libxlMigrationCookiePtr migcookie; /* for freeing listen sockets */ virNetSocketPtr *socks; @@ -63,11 +76,166 @@ typedef struct _libxlMigrationDstArgs { static virClassPtr libxlMigrationDstArgsClass; + +static void +libxlMigrationCookieFree(libxlMigrationCookiePtr mig) +{ + if (!mig) + return; + + VIR_FREE(mig->srcHostname); + VIR_FREE(mig->name); + VIR_FREE(mig); +} + +static libxlMigrationCookiePtr +libxlMigrationCookieNew(virDomainObjPtr dom) +{ + libxlMigrationCookiePtr mig = NULL; + + if (VIR_ALLOC(mig) < 0) + goto error; + + if (VIR_STRDUP(mig->name, dom->def->name) < 0) + goto error; + + memcpy(mig->uuid, dom->def->uuid, VIR_UUID_BUFLEN); + + if (!(mig->srcHostname = virGetHostname())) + goto error; + + mig->xenMigStreamVer = LIBXL_SAVE_VERSION; + + return mig; + + error: + libxlMigrationCookieFree(mig); + return NULL; +} + + +static int +libxlMigrationBakeCookie(libxlMigrationCookiePtr mig, + char **cookieout, + int *cookieoutlen) +{ + virBuffer buf = VIR_BUFFER_INITIALIZER; + char uuidstr[VIR_UUID_STRING_BUFLEN]; + + if (!cookieout || !cookieoutlen) + return 0; + + *cookieoutlen = 0; + virUUIDFormat(mig->uuid, uuidstr); + + virBufferAddLit(&buf, "<libxl-migration>\n"); + virBufferAdjustIndent(&buf, 2); + virBufferEscapeString(&buf, "<name>%s</name>\n", mig->name); + virBufferAsprintf(&buf, "<uuid>%s</uuid>\n", uuidstr); + virBufferEscapeString(&buf, "<hostname>%s</hostname>\n", mig->srcHostname); + virBufferAsprintf(&buf, "<migration-stream-version>%u</migration-stream-version>\n", mig->xenMigStreamVer); + virBufferAdjustIndent(&buf, -2); + virBufferAddLit(&buf, "</libxl-migration>\n"); + + if (virBufferCheckError(&buf) < 0) + return -1; + + *cookieout = virBufferContentAndReset(&buf); + *cookieoutlen = strlen(*cookieout) + 1; + + VIR_DEBUG("cookielen=%d cookie=%s", *cookieoutlen, *cookieout); + + return 0; +} + +static int +libxlMigrationEatCookie(const char *cookiein, + int cookieinlen, + libxlMigrationCookiePtr *migout) +{ + libxlMigrationCookiePtr mig = NULL; + xmlDocPtr doc = NULL; + xmlXPathContextPtr ctxt = NULL; + char *uuidstr = NULL; + int ret = -1; + + /* + * Assume a legacy (V1) migration stream if request came from a + * source host without cookie support, and hence no way to + * specify a stream version. + */ + if (!cookiein || !cookieinlen) { + if (VIR_ALLOC(mig) < 0) + return -1; + + mig->xenMigStreamVer = 1; + *migout = mig; + return 0; + } + + if (cookiein[cookieinlen-1] != '\0') { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Migration cookie was not NULL terminated")); + return -1; + } + + VIR_DEBUG("cookielen=%d cookie='%s'", cookieinlen, NULLSTR(cookiein)); + + if (VIR_ALLOC(mig) < 0) + return -1; + + if (!(doc = virXMLParseStringCtxt(cookiein, + _("(libxl_migration_cookie)"), + &ctxt))) + goto error; + + /* Extract domain name */ + if (!(mig->name = virXPathString("string(./name[1])", ctxt))) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("missing name element in migration data")); + goto error; + } + + /* Extract domain uuid */ + uuidstr = virXPathString("string(./uuid[1])", ctxt); + if (!uuidstr) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("missing uuid element in migration data")); + goto error; + } + if (virUUIDParse(uuidstr, mig->uuid) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("malformed uuid element")); + goto error; + } + + if (virXPathUInt("string(./migration-stream-version[1])", + ctxt, &mig->xenMigStreamVer) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("missing Xen migration stream version")); + goto error; + } + + *migout = mig; + ret = 0; + goto cleanup; + + error: + libxlMigrationCookieFree(mig); + + cleanup: + VIR_FREE(uuidstr); + xmlXPathFreeContext(ctxt); + xmlFreeDoc(doc); + return ret; +} + static void libxlMigrationDstArgsDispose(void *obj) { libxlMigrationDstArgs *args = obj; + libxlMigrationCookieFree(args->migcookie); VIR_FREE(args->socks); } @@ -106,7 +274,8 @@ libxlDoMigrateReceive(void *opaque) * Always start the domain paused. If needed, unpause in the * finish phase, after transfer of the domain is complete. */ - ret = libxlDomainStartRestore(driver, vm, true, recvfd, LIBXL_SAVE_VERSION); + ret = libxlDomainStartRestore(driver, vm, true, recvfd, + args->migcookie->xenMigStreamVer); if (ret < 0 && !vm->persistent) remove_dom = true; @@ -227,10 +396,13 @@ libxlDomainMigrationIsAllowed(virDomainDefPtr def) char * libxlDomainMigrationBegin(virConnectPtr conn, virDomainObjPtr vm, - const char *xmlin) + const char *xmlin, + char **cookieout, + int *cookieoutlen) { libxlDriverPrivatePtr driver = conn->privateData; libxlDriverConfigPtr cfg = libxlDriverConfigGet(driver); + libxlMigrationCookiePtr mig; virDomainDefPtr tmpdef = NULL; virDomainDefPtr def; char *xml = NULL; @@ -238,6 +410,12 @@ libxlDomainMigrationBegin(virConnectPtr conn, if (libxlDomainObjBeginJob(driver, vm, LIBXL_JOB_MODIFY) < 0) goto cleanup; + if (!(mig = libxlMigrationCookieNew(vm))) + goto endjob; + + if (libxlMigrationBakeCookie(mig, cookieout, cookieoutlen) < 0) + goto endjob; + if (xmlin) { if (!(tmpdef = virDomainDefParseString(xmlin, cfg->caps, driver->xmlopt, @@ -308,9 +486,12 @@ libxlDomainMigrationPrepare(virConnectPtr dconn, virDomainDefPtr *def, const char *uri_in, char **uri_out, + const char *cookiein, + int cookieinlen, unsigned int flags) { libxlDriverPrivatePtr driver = dconn->privateData; + libxlMigrationCookiePtr mig = NULL; virDomainObjPtr vm = NULL; char *hostname = NULL; unsigned short port; @@ -323,6 +504,16 @@ libxlDomainMigrationPrepare(virConnectPtr dconn, size_t i; int ret = -1; + if (libxlMigrationEatCookie(cookiein, cookieinlen, &mig) < 0) + goto error; + + if (mig->xenMigStreamVer > LIBXL_SAVE_VERSION) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, + _("Xen migration stream version '%d' is not supported on this host"), + mig->xenMigStreamVer); + goto error; + } + if (!(vm = virDomainObjListAdd(driver->domains, *def, driver->xmlopt, VIR_DOMAIN_OBJ_LIST_ADD_LIVE | @@ -409,6 +600,7 @@ libxlDomainMigrationPrepare(virConnectPtr dconn, args->flags = flags; args->socks = socks; args->nsocks = nsocks; + args->migcookie = mig; for (i = 0; i < nsocks; i++) { if (virNetSocketSetBlocking(socks[i], true) < 0) @@ -479,11 +671,14 @@ libxlDoMigrateP2P(libxlDriverPrivatePtr driver, char *uri_out = NULL; char *dom_xml = NULL; unsigned long destflags; + char *cookieout = NULL; + int cookieoutlen; bool cancelled = true; virErrorPtr orig_err = NULL; int ret = -1; - dom_xml = libxlDomainMigrationBegin(sconn, vm, xmlin); + dom_xml = libxlDomainMigrationBegin(sconn, vm, xmlin, + &cookieout, &cookieoutlen); if (!dom_xml) goto cleanup; @@ -509,7 +704,7 @@ libxlDoMigrateP2P(libxlDriverPrivatePtr driver, VIR_DEBUG("Prepare3"); virObjectUnlock(vm); ret = dconn->driver->domainMigratePrepare3Params - (dconn, params, nparams, NULL, 0, NULL, NULL, &uri_out, destflags); + (dconn, params, nparams, cookieout, cookieoutlen, NULL, NULL, &uri_out, destflags); virObjectLock(vm); if (ret == -1) @@ -580,6 +775,7 @@ libxlDoMigrateP2P(libxlDriverPrivatePtr driver, virFreeError(orig_err); } + VIR_FREE(cookieout); VIR_FREE(dom_xml); VIR_FREE(uri_out); virTypedParamsFree(params, nparams); diff --git a/src/libxl/libxl_migration.h b/src/libxl/libxl_migration.h index 0f83bb4..c02026d 100644 --- a/src/libxl/libxl_migration.h +++ b/src/libxl/libxl_migration.h @@ -42,7 +42,9 @@ char * libxlDomainMigrationBegin(virConnectPtr conn, virDomainObjPtr vm, - const char *xmlin); + const char *xmlin, + char **cookieout, + int *cookieoutlen); virDomainDefPtr libxlDomainMigrationPrepareDef(libxlDriverPrivatePtr driver, @@ -54,6 +56,8 @@ libxlDomainMigrationPrepare(virConnectPtr dconn, virDomainDefPtr *def, const char *uri_in, char **uri_out, + const char *cookiein, + int cookieinlen, unsigned int flags); int -- 2.1.4
On 03/05/16 02:01, Jim Fehlig wrote:
Hi all,
This patch adds support for Xen migration stream V2 to the libvirt libxl driver. In the process it fixes save/restore and migration tests in OSSTest, which have been failing since libvirt commit e7440656.
Patch1 changes the libxl API requirement from version 4.2 to 4.4, enabling use of an extended libxl_domain_create_restore() function that allows passing restore parameters such as stream version.
Patch2 adds support for migration stream V2 in the basic save/restore logic, taking advantage of a save image header that includes a version field. The version is set to '2' if the host produces a V2 migration stream. This patch fixes the failing save/restore tests in OSSTest.
Patch3 adds support for migration stream V2 in the migration logic. The migration code does not use the save image header and instead uses libvirt's migration protocol to transfer domain configuration and other such goodies from source to destination. The patch enables sending version information in the Begin and Prepare migration phases so the correct stream version information can be passed to libxl_domain_create_restore(). An upshot of this approach is safely detecting and aborting a migration from a V2 host to a V1-only host. This patch fixes the failing migration tests in OSSTest.
LGTM (although I can't comment on the libvirt internals). ~Andrew
On Mon, May 02, 2016 at 07:01:16PM -0600, Jim Fehlig wrote:
Hi all,
This patch adds support for Xen migration stream V2 to the libvirt libxl driver. In the process it fixes save/restore and migration tests in OSSTest, which have been failing since libvirt commit e7440656.
Patch1 changes the libxl API requirement from version 4.2 to 4.4, enabling use of an extended libxl_domain_create_restore() function that allows passing restore parameters such as stream version.
Patch2 adds support for migration stream V2 in the basic save/restore logic, taking advantage of a save image header that includes a version field. The version is set to '2' if the host produces a V2 migration stream. This patch fixes the failing save/restore tests in OSSTest.
Patch3 adds support for migration stream V2 in the migration logic. The migration code does not use the save image header and instead uses libvirt's migration protocol to transfer domain configuration and other such goodies from source to destination. The patch enables sending version information in the Begin and Prepare migration phases so the correct stream version information can be passed to libxl_domain_create_restore(). An upshot of this approach is safely detecting and aborting a migration from a V2 host to a V1-only host. This patch fixes the failing migration tests in OSSTest.
The general approach looks good to me. Wei.
On 05/03/2016 07:19 AM, Wei Liu wrote:
On Mon, May 02, 2016 at 07:01:16PM -0600, Jim Fehlig wrote:
Hi all,
This patch adds support for Xen migration stream V2 to the libvirt libxl driver. In the process it fixes save/restore and migration tests in OSSTest, which have been failing since libvirt commit e7440656.
Patch1 changes the libxl API requirement from version 4.2 to 4.4, enabling use of an extended libxl_domain_create_restore() function that allows passing restore parameters such as stream version.
Patch2 adds support for migration stream V2 in the basic save/restore logic, taking advantage of a save image header that includes a version field. The version is set to '2' if the host produces a V2 migration stream. This patch fixes the failing save/restore tests in OSSTest.
Patch3 adds support for migration stream V2 in the migration logic. The migration code does not use the save image header and instead uses libvirt's migration protocol to transfer domain configuration and other such goodies from source to destination. The patch enables sending version information in the Begin and Prepare migration phases so the correct stream version information can be passed to libxl_domain_create_restore(). An upshot of this approach is safely detecting and aborting a migration from a V2 host to a V1-only host. This patch fixes the failing migration tests in OSSTest.
The general approach looks good to me.
Any comments on this series from libvirt maintainers? Would be nice to get this series (or a variant) committed, fixing the daily OSSTest failures. For your viewing pleasure, here's a link to today's failure http://logs.test-lab.xenproject.org/osstest/logs/93880/ Regards, Jim
On Mon, May 02, 2016 at 07:01:16PM -0600, Jim Fehlig wrote:
Hi all,
This patch adds support for Xen migration stream V2 to the libvirt libxl driver. In the process it fixes save/restore and migration tests in OSSTest, which have been failing since libvirt commit e7440656.
Patch1 changes the libxl API requirement from version 4.2 to 4.4, enabling use of an extended libxl_domain_create_restore() function that allows passing restore parameters such as stream version.
Patch2 adds support for migration stream V2 in the basic save/restore logic, taking advantage of a save image header that includes a version field. The version is set to '2' if the host produces a V2 migration stream. This patch fixes the failing save/restore tests in OSSTest.
Patch3 adds support for migration stream V2 in the migration logic. The migration code does not use the save image header and instead uses libvirt's migration protocol to transfer domain configuration and other such goodies from source to destination. The patch enables sending version information in the Begin and Prepare migration phases so the correct stream version information can be passed to libxl_domain_create_restore(). An upshot of this approach is safely detecting and aborting a migration from a V2 host to a V1-only host. This patch fixes the failing migration tests in OSSTest.
Jim Fehlig (3): libxl: switch to using libxl_domain_create_restore from v4.4 API libxl: support Xen migration stream V2 in save/restore libxl: support migration stream V2 in migration
configure.ac | 9 +- src/libxl/libxl_conf.h | 6 +- src/libxl/libxl_domain.c | 46 ++++++++-- src/libxl/libxl_domain.h | 14 ++- src/libxl/libxl_driver.c | 27 +++--- src/libxl/libxl_migration.c | 204 +++++++++++++++++++++++++++++++++++++++++++- src/libxl/libxl_migration.h | 6 +- 7 files changed, 282 insertions(+), 30 deletions(-)
ACK series Jan
Ján Tomko wrote:
On Mon, May 02, 2016 at 07:01:16PM -0600, Jim Fehlig wrote:
Hi all,
This patch adds support for Xen migration stream V2 to the libvirt libxl driver. In the process it fixes save/restore and migration tests in OSSTest, which have been failing since libvirt commit e7440656.
Patch1 changes the libxl API requirement from version 4.2 to 4.4, enabling use of an extended libxl_domain_create_restore() function that allows passing restore parameters such as stream version.
Patch2 adds support for migration stream V2 in the basic save/restore logic, taking advantage of a save image header that includes a version field. The version is set to '2' if the host produces a V2 migration stream. This patch fixes the failing save/restore tests in OSSTest.
Patch3 adds support for migration stream V2 in the migration logic. The migration code does not use the save image header and instead uses libvirt's migration protocol to transfer domain configuration and other such goodies from source to destination. The patch enables sending version information in the Begin and Prepare migration phases so the correct stream version information can be passed to libxl_domain_create_restore(). An upshot of this approach is safely detecting and aborting a migration from a V2 host to a V1-only host. This patch fixes the failing migration tests in OSSTest.
Jim Fehlig (3): libxl: switch to using libxl_domain_create_restore from v4.4 API libxl: support Xen migration stream V2 in save/restore libxl: support migration stream V2 in migration
configure.ac | 9 +- src/libxl/libxl_conf.h | 6 +- src/libxl/libxl_domain.c | 46 ++++++++-- src/libxl/libxl_domain.h | 14 ++- src/libxl/libxl_driver.c | 27 +++--- src/libxl/libxl_migration.c | 204 +++++++++++++++++++++++++++++++++++++++++++- src/libxl/libxl_migration.h | 6 +- 7 files changed, 282 insertions(+), 30 deletions(-)
ACK series
Thanks, pushed now. Note to all: With this change, libvirt.git will no longer build with Xen < 4.4. Wearing either upstream maintainer or distro packager hat, I don't really care :-). With upstream hat on, Xen 4.2 is no longer maintained. Xen 4.3 saw the end of general support in Jan 2015 and will end the subsequent 18 months of security support in early July. So in a little over a month 4.3 will be dead. With distro packager hat on, I'm not interested in using a new libvirt with such an old Xen. I hope others feel the same. If not, patches are welcome. Regards, Jim
participants (5)
-
Andrew Cooper -
Jim Fehlig -
Ján Tomko -
Olaf Hering -
Wei Liu