[libvirt RFCv6 00/27] multifd save restore prototype
This a respin of the multifd save prototype, rebased on post 8.3.0 release. There are still quite a few rough edges: KNOWN ISSUES: 1) this applies only to virsh save and virsh restore for now (no managed save etc). 2) error handling is not good yet, especially during resume, errors may leave behind a qemu process and such. May need some help find all of these cases 3) the compression part is demonstrative only, there needs to be more attention to compression options, and detecting the compression used to store the multifd saves. ... changes from v4: * runIO renamed to virFileDiskCopy and rethought arguments * renamed new APIs from ...ParametersFlags to ...Params * introduce the new virDomainSaveParams and virDomainRestoreParams without any additional parameters, so they can be upstreamed first. * solved the issue in the gendispatch.pl script generating code that was missing the conn parameter. --- changes from v3: * reordered series to have all helper-related change at the start * solved all reported issues from ninja test, including documentation * fixed most broken migration capabilities code (still imperfect likely) * added G_GNUC_UNUSED as needed * after multifd restore, added what I think were the missing operations: qemuProcessRefreshState(), qemuProcessStartCPUs() - most importantly, virDomainObjSave() The domain now starts running after restore without further encouragement * removed the sleep(10) from the multifd-helper changes from v2: * added ability to restore the VM from disk using multifd * fixed the multifd-helper to work in both directions, assuming the need to listen for save, and connect for restore. * fixed a large number of bugs, and probably introduced some :-) Thanks for your thoughts, Claudio Claudio Fontana (27): iohelper: introduce new struct to carry copy operation parameters iohelper: refactor copy operation as a separate function iohelper: move runIO function to virfile.c virfile: rename runIO to virFileDiskCopy virfile: change virFileDiskCopy arguments to extend beyond stdin, stdout virfile: add comment about the use of SEEK_END in virFileDiskCopy multifd-helper: new helper for parallel save/restore libvirt: introduce virDomainSaveParams public API libvirt: introduce virDomainRestoreParams public API remote: Add RPC support for the virDomainSaveParams API gendispatch: add DomainRestoreParams as requiring conn argument remote: Add RPC support for the virDomainRestoreParams API qemu: add implementation for virDomainSaveParams API qemu: add implementation for virDomainRestoreParams API libvirt: add new VIR_DOMAIN_SAVE_PARALLEL flag and parameter qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in save qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in restore qemu: saveimage: introduce virQEMUSaveFd qemu: wire up saveimage code with the multifd helper qemu: capabilities: add multifd to the probed migration capabilities qemu: implement qemuMigrationSrcToFilesMultiFd qemu: add parameter to qemuMigrationDstRun to skip waiting qemu: implement qemuSaveImageLoadMultiFd tools: add parallel parameter to virsh save command tools: add parallel parameter to virsh restore command docs: update refs to virDomainSaveParams and virDomainRestoreParams qemu: add migration parameter multifd-compression docs/formatsnapshot.rst | 5 +- docs/manpages/virsh.rst | 34 +- include/libvirt/libvirt-domain.h | 49 ++ po/POTFILES.in | 1 + src/driver-hypervisor.h | 14 + src/libvirt-domain.c | 99 +++- src/libvirt_private.syms | 1 + src/libvirt_public.syms | 6 + src/qemu/qemu_capabilities.c | 6 + src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_driver.c | 239 +++++++-- src/qemu/qemu_migration.c | 155 ++++-- src/qemu/qemu_migration.h | 16 +- src/qemu/qemu_migration_params.c | 71 ++- src/qemu/qemu_migration_params.h | 15 + src/qemu/qemu_process.c | 3 +- src/qemu/qemu_process.h | 5 +- src/qemu/qemu_saveimage.c | 496 ++++++++++++++---- src/qemu/qemu_saveimage.h | 49 +- src/qemu/qemu_snapshot.c | 6 +- src/remote/remote_driver.c | 2 + src/remote/remote_protocol.x | 29 +- src/remote_protocol-structs | 17 + src/rpc/gendispatch.pl | 5 +- src/util/iohelper.c | 162 +----- src/util/meson.build | 19 + src/util/multifd-helper.c | 249 +++++++++ src/util/virfile.c | 218 ++++++++ src/util/virfile.h | 2 + src/util/virthread.c | 5 + src/util/virthread.h | 1 + .../caps_4.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + .../caps_4.0.0.riscv32.xml | 1 + .../caps_4.0.0.riscv64.xml | 1 + .../qemucapabilitiesdata/caps_4.0.0.s390x.xml | 1 + .../caps_4.0.0.x86_64.xml | 1 + .../caps_4.1.0.x86_64.xml | 1 + .../caps_4.2.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + .../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 + .../caps_4.2.0.x86_64.xml | 1 + .../caps_5.0.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 2 + .../caps_5.0.0.riscv64.xml | 2 + .../caps_5.0.0.x86_64.xml | 2 + .../qemucapabilitiesdata/caps_5.1.0.sparc.xml | 2 + .../caps_5.1.0.x86_64.xml | 2 + .../caps_5.2.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 2 + .../caps_5.2.0.riscv64.xml | 2 + .../qemucapabilitiesdata/caps_5.2.0.s390x.xml | 2 + .../caps_5.2.0.x86_64.xml | 2 + .../caps_6.0.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_6.0.0.s390x.xml | 2 + .../caps_6.0.0.x86_64.xml | 2 + .../caps_6.1.0.x86_64.xml | 2 + .../caps_6.2.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 2 + .../caps_6.2.0.x86_64.xml | 2 + .../caps_7.0.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 2 + .../caps_7.0.0.x86_64.xml | 2 + tools/virsh-domain.c | 96 +++- 64 files changed, 1686 insertions(+), 446 deletions(-) create mode 100644 src/util/multifd-helper.c -- 2.35.3
this is in preparation for a minor refactoring of the copy function itself out of runIO(). Signed-off-by: Claudio Fontana <cfontana@suse.de> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> --- src/util/iohelper.c | 95 +++++++++++++++++++++++++-------------------- 1 file changed, 53 insertions(+), 42 deletions(-) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index 2c91bf4f93..c13746a547 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -45,6 +45,16 @@ # define O_DIRECT 0 #endif +struct runIOParams { + bool isBlockDev; + bool isDirect; + bool isWrite; + int fdin; + const char *fdinname; + int fdout; + const char *fdoutname; +}; + static int runIO(const char *path, int fd, int oflags) { @@ -53,13 +63,9 @@ runIO(const char *path, int fd, int oflags) size_t buflen = 1024*1024; intptr_t alignMask = 64*1024 - 1; int ret = -1; - int fdin, fdout; - const char *fdinname, *fdoutname; - unsigned long long total = 0; - bool direct = O_DIRECT && ((oflags & O_DIRECT) != 0); - off_t end = 0; + off_t total = 0; struct stat sb; - bool isBlockDev = false; + struct runIOParams p; #if WITH_POSIX_MEMALIGN if (posix_memalign(&base, alignMask + 1, buflen)) @@ -77,34 +83,23 @@ runIO(const char *path, int fd, int oflags) fd, path); goto cleanup; } - isBlockDev = S_ISBLK(sb.st_mode); + p.isBlockDev = S_ISBLK(sb.st_mode); + p.isDirect = O_DIRECT && (oflags & O_DIRECT); switch (oflags & O_ACCMODE) { case O_RDONLY: - fdin = fd; - fdinname = path; - fdout = STDOUT_FILENO; - fdoutname = "stdout"; - /* To make the implementation simpler, we give up on any - * attempt to use O_DIRECT in a non-trivial manner. */ - if (!isBlockDev && direct && ((end = lseek(fd, 0, SEEK_CUR)) != 0)) { - virReportSystemError(end < 0 ? errno : EINVAL, "%s", - _("O_DIRECT read needs entire seekable file")); - goto cleanup; - } + p.isWrite = false; + p.fdin = fd; + p.fdinname = path; + p.fdout = STDOUT_FILENO; + p.fdoutname = "stdout"; break; case O_WRONLY: - fdin = STDIN_FILENO; - fdinname = "stdin"; - fdout = fd; - fdoutname = path; - /* To make the implementation simpler, we give up on any - * attempt to use O_DIRECT in a non-trivial manner. */ - if (!isBlockDev && direct && (end = lseek(fd, 0, SEEK_END)) != 0) { - virReportSystemError(end < 0 ? errno : EINVAL, "%s", - _("O_DIRECT write needs empty seekable file")); - goto cleanup; - } + p.isWrite = true; + p.fdin = STDIN_FILENO; + p.fdinname = "stdin"; + p.fdout = fd; + p.fdoutname = path; break; case O_RDWR: @@ -114,6 +109,22 @@ runIO(const char *path, int fd, int oflags) (oflags & O_ACCMODE)); goto cleanup; } + /* To make the implementation simpler, we give up on any + * attempt to use O_DIRECT in a non-trivial manner. */ + if (!p.isBlockDev && p.isDirect) { + off_t off; + if (p.isWrite) { + if ((off = lseek(fd, 0, SEEK_END)) != 0) { + virReportSystemError(off < 0 ? errno : EINVAL, "%s", + _("O_DIRECT write needs empty seekable file")); + goto cleanup; + } + } else if ((off = lseek(fd, 0, SEEK_CUR)) != 0) { + virReportSystemError(off < 0 ? errno : EINVAL, "%s", + _("O_DIRECT read needs entire seekable file")); + goto cleanup; + } + } while (1) { ssize_t got; @@ -124,16 +135,16 @@ runIO(const char *path, int fd, int oflags) * writes will be aligned. * In other cases using saferead reduces number of syscalls. */ - if (fdin == fd && direct) { - if ((got = read(fdin, buf, buflen)) < 0 && + if (!p.isWrite && p.isDirect) { + if ((got = read(p.fdin, buf, buflen)) < 0 && errno == EINTR) continue; } else { - got = saferead(fdin, buf, buflen); + got = saferead(p.fdin, buf, buflen); } if (got < 0) { - virReportSystemError(errno, _("Unable to read %s"), fdinname); + virReportSystemError(errno, _("Unable to read %s"), p.fdinname); goto cleanup; } if (got == 0) @@ -142,35 +153,35 @@ runIO(const char *path, int fd, int oflags) total += got; /* handle last write size align in direct case */ - if (got < buflen && direct && fdout == fd) { + if (got < buflen && p.isDirect && p.isWrite) { ssize_t aligned_got = (got + alignMask) & ~alignMask; memset(buf + got, 0, aligned_got - got); - if (safewrite(fdout, buf, aligned_got) < 0) { - virReportSystemError(errno, _("Unable to write %s"), fdoutname); + if (safewrite(p.fdout, buf, aligned_got) < 0) { + virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); goto cleanup; } - if (!isBlockDev && ftruncate(fd, total) < 0) { - virReportSystemError(errno, _("Unable to truncate %s"), fdoutname); + if (!p.isBlockDev && ftruncate(p.fdout, total) < 0) { + virReportSystemError(errno, _("Unable to truncate %s"), p.fdoutname); goto cleanup; } break; } - if (safewrite(fdout, buf, got) < 0) { - virReportSystemError(errno, _("Unable to write %s"), fdoutname); + if (safewrite(p.fdout, buf, got) < 0) { + virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); goto cleanup; } } /* Ensure all data is written */ - if (virFileDataSync(fdout) < 0) { + if (virFileDataSync(p.fdout) < 0) { if (errno != EINVAL && errno != EROFS) { /* fdatasync() may fail on some special FDs, e.g. pipes */ - virReportSystemError(errno, _("unable to fsync %s"), fdoutname); + virReportSystemError(errno, _("unable to fsync %s"), p.fdoutname); goto cleanup; } } -- 2.35.3
Signed-off-by: Claudio Fontana <cfontana@suse.de> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> --- src/util/iohelper.c | 131 +++++++++++++++++++++++++------------------- 1 file changed, 75 insertions(+), 56 deletions(-) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index c13746a547..1584321839 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -55,17 +55,23 @@ struct runIOParams { const char *fdoutname; }; -static int -runIO(const char *path, int fd, int oflags) +/** + * runIOCopy: execute the IO copy based on the passed parameters + * @p: the IO parameters + * + * Execute the copy based on the passed parameters. + * + * Returns: size transfered, or < 0 on error. + */ + +static off_t +runIOCopy(const struct runIOParams p) { g_autofree void *base = NULL; /* Location to be freed */ char *buf = NULL; /* Aligned location within base */ size_t buflen = 1024*1024; intptr_t alignMask = 64*1024 - 1; - int ret = -1; off_t total = 0; - struct stat sb; - struct runIOParams p; #if WITH_POSIX_MEMALIGN if (posix_memalign(&base, alignMask + 1, buflen)) @@ -77,6 +83,67 @@ runIO(const char *path, int fd, int oflags) buf = (char *) (((intptr_t) base + alignMask) & ~alignMask); #endif + while (1) { + ssize_t got; + + /* If we read with O_DIRECT from file we can't use saferead as + * it can lead to unaligned read after reading last bytes. + * If we write with O_DIRECT use should use saferead so that + * writes will be aligned. + * In other cases using saferead reduces number of syscalls. + */ + if (!p.isWrite && p.isDirect) { + if ((got = read(p.fdin, buf, buflen)) < 0 && + errno == EINTR) + continue; + } else { + got = saferead(p.fdin, buf, buflen); + } + + if (got < 0) { + virReportSystemError(errno, _("Unable to read %s"), p.fdinname); + return -2; + } + if (got == 0) + break; + + total += got; + + /* handle last write size align in direct case */ + if (got < buflen && p.isDirect && p.isWrite) { + ssize_t aligned_got = (got + alignMask) & ~alignMask; + + memset(buf + got, 0, aligned_got - got); + + if (safewrite(p.fdout, buf, aligned_got) < 0) { + virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); + return -3; + } + + if (!p.isBlockDev && ftruncate(p.fdout, total) < 0) { + virReportSystemError(errno, _("Unable to truncate %s"), p.fdoutname); + return -4; + } + + break; + } + + if (safewrite(p.fdout, buf, got) < 0) { + virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); + return -3; + } + } + return total; +} + +static int +runIO(const char *path, int fd, int oflags) +{ + int ret = -1; + off_t total = 0; + struct stat sb; + struct runIOParams p; + if (fstat(fd, &sb) < 0) { virReportSystemError(errno, _("Unable to access file descriptor %d path %s"), @@ -125,57 +192,9 @@ runIO(const char *path, int fd, int oflags) goto cleanup; } } - - while (1) { - ssize_t got; - - /* If we read with O_DIRECT from file we can't use saferead as - * it can lead to unaligned read after reading last bytes. - * If we write with O_DIRECT use should use saferead so that - * writes will be aligned. - * In other cases using saferead reduces number of syscalls. - */ - if (!p.isWrite && p.isDirect) { - if ((got = read(p.fdin, buf, buflen)) < 0 && - errno == EINTR) - continue; - } else { - got = saferead(p.fdin, buf, buflen); - } - - if (got < 0) { - virReportSystemError(errno, _("Unable to read %s"), p.fdinname); - goto cleanup; - } - if (got == 0) - break; - - total += got; - - /* handle last write size align in direct case */ - if (got < buflen && p.isDirect && p.isWrite) { - ssize_t aligned_got = (got + alignMask) & ~alignMask; - - memset(buf + got, 0, aligned_got - got); - - if (safewrite(p.fdout, buf, aligned_got) < 0) { - virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); - goto cleanup; - } - - if (!p.isBlockDev && ftruncate(p.fdout, total) < 0) { - virReportSystemError(errno, _("Unable to truncate %s"), p.fdoutname); - goto cleanup; - } - - break; - } - - if (safewrite(p.fdout, buf, got) < 0) { - virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); - goto cleanup; - } - } + total = runIOCopy(p); + if (total < 0) + goto cleanup; /* Ensure all data is written */ if (virFileDataSync(p.fdout) < 0) { -- 2.35.3
where it can be reused by other helpers. No changes other than the move. Note that this makes iohelper now dependent on -lutil, because unused (for iohelper) parts of virfile.c contain calls to openpty(3). Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 175 ------------------------------------------- src/util/meson.build | 4 + src/util/virfile.c | 172 ++++++++++++++++++++++++++++++++++++++++++ src/util/virfile.h | 2 + 4 files changed, 178 insertions(+), 175 deletions(-) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index 1584321839..b3aaf82b9f 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -41,181 +41,6 @@ #define VIR_FROM_THIS VIR_FROM_STORAGE -#ifndef O_DIRECT -# define O_DIRECT 0 -#endif - -struct runIOParams { - bool isBlockDev; - bool isDirect; - bool isWrite; - int fdin; - const char *fdinname; - int fdout; - const char *fdoutname; -}; - -/** - * runIOCopy: execute the IO copy based on the passed parameters - * @p: the IO parameters - * - * Execute the copy based on the passed parameters. - * - * Returns: size transfered, or < 0 on error. - */ - -static off_t -runIOCopy(const struct runIOParams p) -{ - g_autofree void *base = NULL; /* Location to be freed */ - char *buf = NULL; /* Aligned location within base */ - size_t buflen = 1024*1024; - intptr_t alignMask = 64*1024 - 1; - off_t total = 0; - -#if WITH_POSIX_MEMALIGN - if (posix_memalign(&base, alignMask + 1, buflen)) - abort(); - buf = base; -#else - buf = g_new0(char, buflen + alignMask); - base = buf; - buf = (char *) (((intptr_t) base + alignMask) & ~alignMask); -#endif - - while (1) { - ssize_t got; - - /* If we read with O_DIRECT from file we can't use saferead as - * it can lead to unaligned read after reading last bytes. - * If we write with O_DIRECT use should use saferead so that - * writes will be aligned. - * In other cases using saferead reduces number of syscalls. - */ - if (!p.isWrite && p.isDirect) { - if ((got = read(p.fdin, buf, buflen)) < 0 && - errno == EINTR) - continue; - } else { - got = saferead(p.fdin, buf, buflen); - } - - if (got < 0) { - virReportSystemError(errno, _("Unable to read %s"), p.fdinname); - return -2; - } - if (got == 0) - break; - - total += got; - - /* handle last write size align in direct case */ - if (got < buflen && p.isDirect && p.isWrite) { - ssize_t aligned_got = (got + alignMask) & ~alignMask; - - memset(buf + got, 0, aligned_got - got); - - if (safewrite(p.fdout, buf, aligned_got) < 0) { - virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); - return -3; - } - - if (!p.isBlockDev && ftruncate(p.fdout, total) < 0) { - virReportSystemError(errno, _("Unable to truncate %s"), p.fdoutname); - return -4; - } - - break; - } - - if (safewrite(p.fdout, buf, got) < 0) { - virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); - return -3; - } - } - return total; -} - -static int -runIO(const char *path, int fd, int oflags) -{ - int ret = -1; - off_t total = 0; - struct stat sb; - struct runIOParams p; - - if (fstat(fd, &sb) < 0) { - virReportSystemError(errno, - _("Unable to access file descriptor %d path %s"), - fd, path); - goto cleanup; - } - p.isBlockDev = S_ISBLK(sb.st_mode); - p.isDirect = O_DIRECT && (oflags & O_DIRECT); - - switch (oflags & O_ACCMODE) { - case O_RDONLY: - p.isWrite = false; - p.fdin = fd; - p.fdinname = path; - p.fdout = STDOUT_FILENO; - p.fdoutname = "stdout"; - break; - case O_WRONLY: - p.isWrite = true; - p.fdin = STDIN_FILENO; - p.fdinname = "stdin"; - p.fdout = fd; - p.fdoutname = path; - break; - - case O_RDWR: - default: - virReportSystemError(EINVAL, - _("Unable to process file with flags %d"), - (oflags & O_ACCMODE)); - goto cleanup; - } - /* To make the implementation simpler, we give up on any - * attempt to use O_DIRECT in a non-trivial manner. */ - if (!p.isBlockDev && p.isDirect) { - off_t off; - if (p.isWrite) { - if ((off = lseek(fd, 0, SEEK_END)) != 0) { - virReportSystemError(off < 0 ? errno : EINVAL, "%s", - _("O_DIRECT write needs empty seekable file")); - goto cleanup; - } - } else if ((off = lseek(fd, 0, SEEK_CUR)) != 0) { - virReportSystemError(off < 0 ? errno : EINVAL, "%s", - _("O_DIRECT read needs entire seekable file")); - goto cleanup; - } - } - total = runIOCopy(p); - if (total < 0) - goto cleanup; - - /* Ensure all data is written */ - if (virFileDataSync(p.fdout) < 0) { - if (errno != EINVAL && errno != EROFS) { - /* fdatasync() may fail on some special FDs, e.g. pipes */ - virReportSystemError(errno, _("unable to fsync %s"), p.fdoutname); - goto cleanup; - } - } - - ret = 0; - - cleanup: - if (VIR_CLOSE(fd) < 0 && - ret == 0) { - virReportSystemError(errno, _("Unable to close %s"), path); - ret = -1; - } - return ret; -} - static const char *program_name; G_GNUC_NORETURN static void diff --git a/src/util/meson.build b/src/util/meson.build index 24350a3e67..84ef13ba32 100644 --- a/src/util/meson.build +++ b/src/util/meson.build @@ -175,6 +175,7 @@ keycode_dep = declare_dependency( io_helper_sources = [ 'iohelper.c', + 'virfile.c', ] virt_util_lib = static_library( @@ -213,6 +214,9 @@ if conf.has('WITH_LIBVIRTD') files(io_helper_sources), dtrace_gen_headers, ], + 'deps': [ + libutil_dep, + ], } endif diff --git a/src/util/virfile.c b/src/util/virfile.c index 130b0fbace..b033a27264 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4577,3 +4577,175 @@ virFileSetCOW(const char *path, return 0; #endif /* ! __linux__ */ } + +struct runIOParams { + bool isBlockDev; + bool isDirect; + bool isWrite; + int fdin; + const char *fdinname; + int fdout; + const char *fdoutname; +}; + +/** + * runIOCopy: execute the IO copy based on the passed parameters + * @p: the IO parameters + * + * Execute the copy based on the passed parameters. + * + * Returns: size transfered, or < 0 on error. + */ + +static off_t +runIOCopy(const struct runIOParams p) +{ + g_autofree void *base = NULL; /* Location to be freed */ + char *buf = NULL; /* Aligned location within base */ + size_t buflen = 1024*1024; + intptr_t alignMask = 64*1024 - 1; + off_t total = 0; + +#if WITH_POSIX_MEMALIGN + if (posix_memalign(&base, alignMask + 1, buflen)) + abort(); + buf = base; +#else + buf = g_new0(char, buflen + alignMask); + base = buf; + buf = (char *) (((intptr_t) base + alignMask) & ~alignMask); +#endif + + while (1) { + ssize_t got; + + /* If we read with O_DIRECT from file we can't use saferead as + * it can lead to unaligned read after reading last bytes. + * If we write with O_DIRECT use should use saferead so that + * writes will be aligned. + * In other cases using saferead reduces number of syscalls. + */ + if (!p.isWrite && p.isDirect) { + if ((got = read(p.fdin, buf, buflen)) < 0 && + errno == EINTR) + continue; + } else { + got = saferead(p.fdin, buf, buflen); + } + + if (got < 0) { + virReportSystemError(errno, _("Unable to read %s"), p.fdinname); + return -2; + } + if (got == 0) + break; + + total += got; + + /* handle last write size align in direct case */ + if (got < buflen && p.isDirect && p.isWrite) { + ssize_t aligned_got = (got + alignMask) & ~alignMask; + + memset(buf + got, 0, aligned_got - got); + + if (safewrite(p.fdout, buf, aligned_got) < 0) { + virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); + return -3; + } + + if (!p.isBlockDev && ftruncate(p.fdout, total) < 0) { + virReportSystemError(errno, _("Unable to truncate %s"), p.fdoutname); + return -4; + } + + break; + } + + if (safewrite(p.fdout, buf, got) < 0) { + virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); + return -3; + } + } + return total; +} + + +off_t +runIO(const char *path, int fd, int oflags) +{ + int ret = -1; + off_t total = 0; + struct stat sb; + struct runIOParams p; + + if (fstat(fd, &sb) < 0) { + virReportSystemError(errno, + _("Unable to access file descriptor %d path %s"), + fd, path); + goto cleanup; + } + p.isBlockDev = S_ISBLK(sb.st_mode); + p.isDirect = O_DIRECT && (oflags & O_DIRECT); + + switch (oflags & O_ACCMODE) { + case O_RDONLY: + p.isWrite = false; + p.fdin = fd; + p.fdinname = path; + p.fdout = STDOUT_FILENO; + p.fdoutname = "stdout"; + break; + case O_WRONLY: + p.isWrite = true; + p.fdin = STDIN_FILENO; + p.fdinname = "stdin"; + p.fdout = fd; + p.fdoutname = path; + break; + + case O_RDWR: + default: + virReportSystemError(EINVAL, + _("Unable to process file with flags %d"), + (oflags & O_ACCMODE)); + goto cleanup; + } + /* To make the implementation simpler, we give up on any + * attempt to use O_DIRECT in a non-trivial manner. */ + if (!p.isBlockDev && p.isDirect) { + off_t off; + if (p.isWrite) { + if ((off = lseek(fd, 0, SEEK_END)) != 0) { + virReportSystemError(off < 0 ? errno : EINVAL, "%s", + _("O_DIRECT write needs empty seekable file")); + goto cleanup; + } + } else if ((off = lseek(fd, 0, SEEK_CUR)) != 0) { + virReportSystemError(off < 0 ? errno : EINVAL, "%s", + _("O_DIRECT read needs entire seekable file")); + goto cleanup; + } + } + total = runIOCopy(p); + if (total < 0) + goto cleanup; + + /* Ensure all data is written */ + if (virFileDataSync(p.fdout) < 0) { + if (errno != EINVAL && errno != EROFS) { + /* fdatasync() may fail on some special FDs, e.g. pipes */ + virReportSystemError(errno, _("unable to fsync %s"), p.fdoutname); + goto cleanup; + } + } + + ret = 0; + + cleanup: + if (VIR_CLOSE(fd) < 0 && + ret == 0) { + virReportSystemError(errno, _("Unable to close %s"), path); + ret = -1; + } + return ret; +} diff --git a/src/util/virfile.h b/src/util/virfile.h index b04386f6e6..0dc336e339 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -383,3 +383,5 @@ int virFileDataSync(int fd); int virFileSetCOW(const char *path, virTristateBool state); + +off_t runIO(const char *path, int fd, int oflags); -- 2.35.3
On Thu, May 05, 2022 at 02:52:17PM +0200, Claudio Fontana wrote:
where it can be reused by other helpers. No changes other than the move.
Note that this makes iohelper now dependent on -lutil, because unused (for iohelper) parts of virfile.c contain calls to openpty(3).
Needs -lacl too on F35 at least.
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 175 ------------------------------------------- src/util/meson.build | 4 + src/util/virfile.c | 172 ++++++++++++++++++++++++++++++++++++++++++ src/util/virfile.h | 2 + 4 files changed, 178 insertions(+), 175 deletions(-)
diff --git a/src/util/meson.build b/src/util/meson.build index 24350a3e67..84ef13ba32 100644 --- a/src/util/meson.build +++ b/src/util/meson.build @@ -175,6 +175,7 @@ keycode_dep = declare_dependency(
io_helper_sources = [ 'iohelper.c', + 'virfile.c', ]
virt_util_lib = static_library( @@ -213,6 +214,9 @@ if conf.has('WITH_LIBVIRTD') files(io_helper_sources), dtrace_gen_headers, ], + 'deps': [
Adding 'acl_dep' here is needed
+ libutil_dep, + ], } endif
diff --git a/src/util/virfile.c b/src/util/virfile.c index 130b0fbace..b033a27264 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4577,3 +4577,175 @@ virFileSetCOW(const char *path, return 0; #endif /* ! __linux__ */ } + +struct runIOParams { + bool isBlockDev; + bool isDirect; + bool isWrite; + int fdin; + const char *fdinname; + int fdout; + const char *fdoutname; +}; + +/** + * runIOCopy: execute the IO copy based on the passed parameters + * @p: the IO parameters + * + * Execute the copy based on the passed parameters. + * + * Returns: size transfered, or < 0 on error. + */ + +static off_t +runIOCopy(const struct runIOParams p) +{ + g_autofree void *base = NULL; /* Location to be freed */ + char *buf = NULL; /* Aligned location within base */ + size_t buflen = 1024*1024; + intptr_t alignMask = 64*1024 - 1; + off_t total = 0; + +#if WITH_POSIX_MEMALIGN + if (posix_memalign(&base, alignMask + 1, buflen)) + abort(); + buf = base; +#else + buf = g_new0(char, buflen + alignMask); + base = buf; + buf = (char *) (((intptr_t) base + alignMask) & ~alignMask); +#endif
For reasons I don't understand 'WITH_POSIX_MEMALIGN' is defined on mingw, but posix_memalign doesn't exist in header files and thus we fail to compile. THe original iohelper.c file is not compiled at all unless WITH_LIBVIRTD is set, which is not on mingw, so we didn't see this problem before WITH_POSIX_MEMALIGN. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 5/6/22 1:02 PM, Daniel P. Berrangé wrote:
On Thu, May 05, 2022 at 02:52:17PM +0200, Claudio Fontana wrote:
where it can be reused by other helpers. No changes other than the move.
Note that this makes iohelper now dependent on -lutil, because unused (for iohelper) parts of virfile.c contain calls to openpty(3).
Needs -lacl too on F35 at least.
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 175 ------------------------------------------- src/util/meson.build | 4 + src/util/virfile.c | 172 ++++++++++++++++++++++++++++++++++++++++++ src/util/virfile.h | 2 + 4 files changed, 178 insertions(+), 175 deletions(-)
diff --git a/src/util/meson.build b/src/util/meson.build index 24350a3e67..84ef13ba32 100644 --- a/src/util/meson.build +++ b/src/util/meson.build @@ -175,6 +175,7 @@ keycode_dep = declare_dependency(
io_helper_sources = [ 'iohelper.c', + 'virfile.c', ]
virt_util_lib = static_library( @@ -213,6 +214,9 @@ if conf.has('WITH_LIBVIRTD') files(io_helper_sources), dtrace_gen_headers, ], + 'deps': [
Adding 'acl_dep' here is needed
+ libutil_dep, + ], } endif
diff --git a/src/util/virfile.c b/src/util/virfile.c index 130b0fbace..b033a27264 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4577,3 +4577,175 @@ virFileSetCOW(const char *path, return 0; #endif /* ! __linux__ */ } + +struct runIOParams { + bool isBlockDev; + bool isDirect; + bool isWrite; + int fdin; + const char *fdinname; + int fdout; + const char *fdoutname; +}; + +/** + * runIOCopy: execute the IO copy based on the passed parameters + * @p: the IO parameters + * + * Execute the copy based on the passed parameters. + * + * Returns: size transfered, or < 0 on error. + */ + +static off_t +runIOCopy(const struct runIOParams p) +{ + g_autofree void *base = NULL; /* Location to be freed */ + char *buf = NULL; /* Aligned location within base */ + size_t buflen = 1024*1024; + intptr_t alignMask = 64*1024 - 1; + off_t total = 0; + +#if WITH_POSIX_MEMALIGN + if (posix_memalign(&base, alignMask + 1, buflen)) + abort(); + buf = base; +#else + buf = g_new0(char, buflen + alignMask); + base = buf; + buf = (char *) (((intptr_t) base + alignMask) & ~alignMask); +#endif
For reasons I don't understand 'WITH_POSIX_MEMALIGN' is defined on mingw, but posix_memalign doesn't exist in header files and thus we fail to compile.
THe original iohelper.c file is not compiled at all unless WITH_LIBVIRTD is set, which is not on mingw, so we didn't see this problem before WITH_POSIX_MEMALIGN.
With regards, Daniel
Should I go back and put this stuff outside virFile.c then, would simplify things in my view, this is a very specialized use anyway no? Thanks, Claudio
On Fri, May 06, 2022 at 01:29:57PM +0200, Claudio Fontana wrote:
On 5/6/22 1:02 PM, Daniel P. Berrangé wrote:
On Thu, May 05, 2022 at 02:52:17PM +0200, Claudio Fontana wrote:
where it can be reused by other helpers. No changes other than the move.
Note that this makes iohelper now dependent on -lutil, because unused (for iohelper) parts of virfile.c contain calls to openpty(3).
Needs -lacl too on F35 at least.
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 175 ------------------------------------------- src/util/meson.build | 4 + src/util/virfile.c | 172 ++++++++++++++++++++++++++++++++++++++++++ src/util/virfile.h | 2 + 4 files changed, 178 insertions(+), 175 deletions(-)
diff --git a/src/util/meson.build b/src/util/meson.build index 24350a3e67..84ef13ba32 100644 --- a/src/util/meson.build +++ b/src/util/meson.build @@ -175,6 +175,7 @@ keycode_dep = declare_dependency(
io_helper_sources = [ 'iohelper.c', + 'virfile.c', ]
virt_util_lib = static_library( @@ -213,6 +214,9 @@ if conf.has('WITH_LIBVIRTD') files(io_helper_sources), dtrace_gen_headers, ], + 'deps': [
Adding 'acl_dep' here is needed
+ libutil_dep, + ], } endif
diff --git a/src/util/virfile.c b/src/util/virfile.c index 130b0fbace..b033a27264 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4577,3 +4577,175 @@ virFileSetCOW(const char *path, return 0; #endif /* ! __linux__ */ } + +struct runIOParams { + bool isBlockDev; + bool isDirect; + bool isWrite; + int fdin; + const char *fdinname; + int fdout; + const char *fdoutname; +}; + +/** + * runIOCopy: execute the IO copy based on the passed parameters + * @p: the IO parameters + * + * Execute the copy based on the passed parameters. + * + * Returns: size transfered, or < 0 on error. + */ + +static off_t +runIOCopy(const struct runIOParams p) +{ + g_autofree void *base = NULL; /* Location to be freed */ + char *buf = NULL; /* Aligned location within base */ + size_t buflen = 1024*1024; + intptr_t alignMask = 64*1024 - 1; + off_t total = 0; + +#if WITH_POSIX_MEMALIGN + if (posix_memalign(&base, alignMask + 1, buflen)) + abort(); + buf = base; +#else + buf = g_new0(char, buflen + alignMask); + base = buf; + buf = (char *) (((intptr_t) base + alignMask) & ~alignMask); +#endif
For reasons I don't understand 'WITH_POSIX_MEMALIGN' is defined on mingw, but posix_memalign doesn't exist in header files and thus we fail to compile.
THe original iohelper.c file is not compiled at all unless WITH_LIBVIRTD is set, which is not on mingw, so we didn't see this problem before WITH_POSIX_MEMALIGN.
With regards, Daniel
Should I go back and put this stuff outside virFile.c then, would simplify things in my view, this is a very specialized use anyway no?
It is fine in virfile, I'd just ifdef the header + source together using !WIN32. I've found the bug wrt posix_memalign mis-detection https://github.com/mesonbuild/meson/issues/1083 which i'll fix separately With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 5/6/22 1:02 PM, Daniel P. Berrangé wrote:
On Thu, May 05, 2022 at 02:52:17PM +0200, Claudio Fontana wrote:
where it can be reused by other helpers. No changes other than the move.
Note that this makes iohelper now dependent on -lutil, because unused (for iohelper) parts of virfile.c contain calls to openpty(3).
Needs -lacl too on F35 at least.
Can you help me figure out where this need comes from, to ensure we are not missing anything else? On my SUSE systems it's not needed, maybe some other OS needs more -l parameters? Thanks, Claudio
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 175 ------------------------------------------- src/util/meson.build | 4 + src/util/virfile.c | 172 ++++++++++++++++++++++++++++++++++++++++++ src/util/virfile.h | 2 + 4 files changed, 178 insertions(+), 175 deletions(-)
diff --git a/src/util/meson.build b/src/util/meson.build index 24350a3e67..84ef13ba32 100644 --- a/src/util/meson.build +++ b/src/util/meson.build @@ -175,6 +175,7 @@ keycode_dep = declare_dependency(
io_helper_sources = [ 'iohelper.c', + 'virfile.c', ]
virt_util_lib = static_library( @@ -213,6 +214,9 @@ if conf.has('WITH_LIBVIRTD') files(io_helper_sources), dtrace_gen_headers, ], + 'deps': [
Adding 'acl_dep' here is needed
+ libutil_dep, + ], } endif
diff --git a/src/util/virfile.c b/src/util/virfile.c index 130b0fbace..b033a27264 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4577,3 +4577,175 @@ virFileSetCOW(const char *path, return 0; #endif /* ! __linux__ */ } + +struct runIOParams { + bool isBlockDev; + bool isDirect; + bool isWrite; + int fdin; + const char *fdinname; + int fdout; + const char *fdoutname; +}; + +/** + * runIOCopy: execute the IO copy based on the passed parameters + * @p: the IO parameters + * + * Execute the copy based on the passed parameters. + * + * Returns: size transfered, or < 0 on error. + */ + +static off_t +runIOCopy(const struct runIOParams p) +{ + g_autofree void *base = NULL; /* Location to be freed */ + char *buf = NULL; /* Aligned location within base */ + size_t buflen = 1024*1024; + intptr_t alignMask = 64*1024 - 1; + off_t total = 0; + +#if WITH_POSIX_MEMALIGN + if (posix_memalign(&base, alignMask + 1, buflen)) + abort(); + buf = base; +#else + buf = g_new0(char, buflen + alignMask); + base = buf; + buf = (char *) (((intptr_t) base + alignMask) & ~alignMask); +#endif
For reasons I don't understand 'WITH_POSIX_MEMALIGN' is defined on mingw, but posix_memalign doesn't exist in header files and thus we fail to compile.
THe original iohelper.c file is not compiled at all unless WITH_LIBVIRTD is set, which is not on mingw, so we didn't see this problem before WITH_POSIX_MEMALIGN.
With regards, Daniel
On Fri, May 06, 2022 at 01:48:33PM +0200, Claudio Fontana wrote:
On 5/6/22 1:02 PM, Daniel P. Berrangé wrote:
On Thu, May 05, 2022 at 02:52:17PM +0200, Claudio Fontana wrote:
where it can be reused by other helpers. No changes other than the move.
Note that this makes iohelper now dependent on -lutil, because unused (for iohelper) parts of virfile.c contain calls to openpty(3).
Needs -lacl too on F35 at least.
Can you help me figure out where this need comes from, to ensure we are not missing anything else? On my SUSE systems it's not needed, maybe some other OS needs more -l parameters?
Just let the gitlab CI run, and if it passes all jobs, we're fine. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 5/6/22 1:02 PM, Daniel P. Berrangé wrote:
On Thu, May 05, 2022 at 02:52:17PM +0200, Claudio Fontana wrote:
where it can be reused by other helpers. No changes other than the move.
Note that this makes iohelper now dependent on -lutil, because unused (for iohelper) parts of virfile.c contain calls to openpty(3).
Needs -lacl too on F35 at least.
do we end up needing the whole of the virt_util_lib dependencies for this? dependencies: [ acl_dep, audit_dep, capng_dep, devmapper_dep, gnutls_dep, intl_dep, libm_dep, libnl_dep, libutil_dep, numactl_dep, secdriver_dep, src_dep, thread_dep, win32_dep, yajl_dep, ] iohelper did not need -lutil or anything else before, just a file. going back to just a separate helper_runio.c/h would make sense to me, but let me know. Thanks, Claudio
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 175 ------------------------------------------- src/util/meson.build | 4 + src/util/virfile.c | 172 ++++++++++++++++++++++++++++++++++++++++++ src/util/virfile.h | 2 + 4 files changed, 178 insertions(+), 175 deletions(-)
diff --git a/src/util/meson.build b/src/util/meson.build index 24350a3e67..84ef13ba32 100644 --- a/src/util/meson.build +++ b/src/util/meson.build @@ -175,6 +175,7 @@ keycode_dep = declare_dependency(
io_helper_sources = [ 'iohelper.c', + 'virfile.c', ]
virt_util_lib = static_library( @@ -213,6 +214,9 @@ if conf.has('WITH_LIBVIRTD') files(io_helper_sources), dtrace_gen_headers, ], + 'deps': [
Adding 'acl_dep' here is needed
+ libutil_dep, + ], } endif
diff --git a/src/util/virfile.c b/src/util/virfile.c index 130b0fbace..b033a27264 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4577,3 +4577,175 @@ virFileSetCOW(const char *path, return 0; #endif /* ! __linux__ */ } + +struct runIOParams { + bool isBlockDev; + bool isDirect; + bool isWrite; + int fdin; + const char *fdinname; + int fdout; + const char *fdoutname; +}; + +/** + * runIOCopy: execute the IO copy based on the passed parameters + * @p: the IO parameters + * + * Execute the copy based on the passed parameters. + * + * Returns: size transfered, or < 0 on error. + */ + +static off_t +runIOCopy(const struct runIOParams p) +{ + g_autofree void *base = NULL; /* Location to be freed */ + char *buf = NULL; /* Aligned location within base */ + size_t buflen = 1024*1024; + intptr_t alignMask = 64*1024 - 1; + off_t total = 0; + +#if WITH_POSIX_MEMALIGN + if (posix_memalign(&base, alignMask + 1, buflen)) + abort(); + buf = base; +#else + buf = g_new0(char, buflen + alignMask); + base = buf; + buf = (char *) (((intptr_t) base + alignMask) & ~alignMask); +#endif
For reasons I don't understand 'WITH_POSIX_MEMALIGN' is defined on mingw, but posix_memalign doesn't exist in header files and thus we fail to compile.
THe original iohelper.c file is not compiled at all unless WITH_LIBVIRTD is set, which is not on mingw, so we didn't see this problem before WITH_POSIX_MEMALIGN.
With regards, Daniel
On Fri, May 06, 2022 at 01:51:53PM +0200, Claudio Fontana wrote:
On 5/6/22 1:02 PM, Daniel P. Berrangé wrote:
On Thu, May 05, 2022 at 02:52:17PM +0200, Claudio Fontana wrote:
where it can be reused by other helpers. No changes other than the move.
Note that this makes iohelper now dependent on -lutil, because unused (for iohelper) parts of virfile.c contain calls to openpty(3).
Needs -lacl too on F35 at least.
do we end up needing the whole of the virt_util_lib dependencies for this?
dependencies: [ acl_dep, audit_dep, capng_dep, devmapper_dep, gnutls_dep, intl_dep, libm_dep, libnl_dep, libutil_dep, numactl_dep, secdriver_dep, src_dep, thread_dep, win32_dep, yajl_dep, ]
iohelper did not need -lutil or anything else before, just a file. going back to just a separate helper_runio.c/h would make sense to me, but let me know.
The linker will discard all the .o files that don't have any functions referenced, so we'll not need most of what virt_util_lib depends on, just the virfile.o deps. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 5/6/22 1:54 PM, Daniel P. Berrangé wrote:
On Fri, May 06, 2022 at 01:51:53PM +0200, Claudio Fontana wrote:
On 5/6/22 1:02 PM, Daniel P. Berrangé wrote:
On Thu, May 05, 2022 at 02:52:17PM +0200, Claudio Fontana wrote:
where it can be reused by other helpers. No changes other than the move.
Note that this makes iohelper now dependent on -lutil, because unused (for iohelper) parts of virfile.c contain calls to openpty(3).
Needs -lacl too on F35 at least.
do we end up needing the whole of the virt_util_lib dependencies for this?
dependencies: [ acl_dep, audit_dep, capng_dep, devmapper_dep, gnutls_dep, intl_dep, libm_dep, libnl_dep, libutil_dep, numactl_dep, secdriver_dep, src_dep, thread_dep, win32_dep, yajl_dep, ]
iohelper did not need -lutil or anything else before, just a file. going back to just a separate helper_runio.c/h would make sense to me, but let me know.
The linker will discard all the .o files that don't have any functions referenced, so we'll not need most of what virt_util_lib depends on, just the virfile.o deps.
Aye aye, Thanks, Claudio
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 2 +- src/util/virfile.c | 2 +- src/util/virfile.h | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index b3aaf82b9f..ce10ccb905 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -97,7 +97,7 @@ main(int argc, char **argv) usage(EXIT_FAILURE); } - if (fd < 0 || runIO(path, fd, oflags) < 0) + if (fd < 0 || virFileDiskCopy(path, fd, oflags) < 0) goto error; return 0; diff --git a/src/util/virfile.c b/src/util/virfile.c index b033a27264..30b0eee074 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4671,7 +4671,7 @@ runIOCopy(const struct runIOParams p) off_t -runIO(const char *path, int fd, int oflags) +virFileDiskCopy(const char *path, int fd, int oflags) { int ret = -1; off_t total = 0; diff --git a/src/util/virfile.h b/src/util/virfile.h index 0dc336e339..59efb760f3 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -384,4 +384,4 @@ int virFileDataSync(int fd); int virFileSetCOW(const char *path, virTristateBool state); -off_t runIO(const char *path, int fd, int oflags); +off_t virFileDiskCopy(const char *path, int fd, int oflags); -- 2.35.3
On Thu, May 05, 2022 at 02:52:18PM +0200, Claudio Fontana wrote:
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 2 +- src/util/virfile.c | 2 +- src/util/virfile.h | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 17 +--------- src/util/virfile.c | 82 ++++++++++++++++++++++++++++++++++----------- src/util/virfile.h | 2 +- 3 files changed, 64 insertions(+), 37 deletions(-) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index ce10ccb905..055540c8c4 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -58,7 +58,6 @@ int main(int argc, char **argv) { const char *path; - int oflags = -1; int fd = -1; program_name = argv[0]; @@ -79,25 +78,11 @@ main(int argc, char **argv) program_name, argv[3]); exit(EXIT_FAILURE); } -#ifdef F_GETFL - oflags = fcntl(fd, F_GETFL); -#else - /* Stupid mingw. */ - if (fd == STDIN_FILENO) - oflags = O_RDONLY; - else if (fd == STDOUT_FILENO) - oflags = O_WRONLY; -#endif - if (oflags < 0) { - fprintf(stderr, _("%s: unable to determine access mode of fd %d"), - program_name, fd); - exit(EXIT_FAILURE); - } } else { /* unknown argc pattern */ usage(EXIT_FAILURE); } - if (fd < 0 || virFileDiskCopy(path, fd, oflags) < 0) + if (fd < 0 || virFileDiskCopy(fd, path, -1, "stdio") < 0) goto error; return 0; diff --git a/src/util/virfile.c b/src/util/virfile.c index 30b0eee074..87d3e53819 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4669,19 +4669,64 @@ runIOCopy(const struct runIOParams p) return total; } +/** + * virFileDiskCopy: run IO to copy data between storage and a pipe or socket. + * + * @disk_fd: the already open regular file or block device + * @disk_path: the pathname corresponding to disk_fd (for error reporting) + * @remote_fd: the pipe or socket + * Use -1 to auto-choose between STDIN or STDOUT. + * @remote_path: the pathname corresponding to remote_fd (for error reporting) + * + * Note that the direction of the transfer is detected based on the @disk_fd + * file access mode (man 2 open). Therefore @disk_fd must be opened with + * O_RDONLY or O_WRONLY. O_RDWR is not supported. + * + * virFileDiskCopy always closes the file descriptor disk_fd, + * and any error during close(2) is reported and considered a failure. + * + * Returns: bytes transferred or < 0 on failure. + */ off_t -virFileDiskCopy(const char *path, int fd, int oflags) +virFileDiskCopy(int disk_fd, const char *disk_path, int remote_fd, const char *remote_path) { int ret = -1; off_t total = 0; struct stat sb; struct runIOParams p; + int oflags = -1; + +#ifdef F_GETFL + oflags = fcntl(disk_fd, F_GETFL); +#else /* !F_GETFL */ + /* + * mingw still does not support F_GETFL, and does not seem willing to + * support it in the future. So we need a hack to get iohelper working, + * specifically possible only when using stdio. + */ + if (disk_fd == STDIN_FILENO) { + oflags = O_RDONLY; + } else if (disk_fd == STDOUT_FILENO) { + oflags = O_WRONLY; + } else { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, + _("%s: disk_fd must be stdin or stdout, due to lack of F_GETFL (mingw)"), + __FUNCTION__); + goto cleanup; + } +#endif /* !F_GETFL */ - if (fstat(fd, &sb) < 0) { + if (oflags < 0) { virReportSystemError(errno, - _("Unable to access file descriptor %d path %s"), - fd, path); + _("unable to determine access mode of %s"), + disk_path); + goto cleanup; + } + if (fstat(disk_fd, &sb) < 0) { + virReportSystemError(errno, + _("unable to stat file descriptor %d path %s"), + disk_fd, disk_path); goto cleanup; } p.isBlockDev = S_ISBLK(sb.st_mode); @@ -4690,23 +4735,21 @@ virFileDiskCopy(const char *path, int fd, int oflags) switch (oflags & O_ACCMODE) { case O_RDONLY: p.isWrite = false; - p.fdin = fd; - p.fdinname = path; - p.fdout = STDOUT_FILENO; - p.fdoutname = "stdout"; + p.fdin = disk_fd; + p.fdinname = disk_path; + p.fdout = remote_fd >= 0 ? remote_fd : STDOUT_FILENO; + p.fdoutname = remote_path; break; case O_WRONLY: p.isWrite = true; - p.fdin = STDIN_FILENO; - p.fdinname = "stdin"; - p.fdout = fd; - p.fdoutname = path; + p.fdin = remote_fd >= 0 ? remote_fd : STDIN_FILENO; + p.fdinname = remote_path; + p.fdout = disk_fd; + p.fdoutname = disk_path; break; - case O_RDWR: default: - virReportSystemError(EINVAL, - _("Unable to process file with flags %d"), + virReportSystemError(EINVAL, _("Unable to process file with flags %d"), (oflags & O_ACCMODE)); goto cleanup; } @@ -4715,12 +4758,12 @@ virFileDiskCopy(const char *path, int fd, int oflags) if (!p.isBlockDev && p.isDirect) { off_t off; if (p.isWrite) { - if ((off = lseek(fd, 0, SEEK_END)) != 0) { + if ((off = lseek(disk_fd, 0, SEEK_END)) != 0) { virReportSystemError(off < 0 ? errno : EINVAL, "%s", _("O_DIRECT write needs empty seekable file")); goto cleanup; } - } else if ((off = lseek(fd, 0, SEEK_CUR)) != 0) { + } else if ((off = lseek(disk_fd, 0, SEEK_CUR)) != 0) { virReportSystemError(off < 0 ? errno : EINVAL, "%s", _("O_DIRECT read needs entire seekable file")); goto cleanup; @@ -4742,9 +4785,8 @@ virFileDiskCopy(const char *path, int fd, int oflags) ret = 0; cleanup: - if (VIR_CLOSE(fd) < 0 && - ret == 0) { - virReportSystemError(errno, _("Unable to close %s"), path); + if (VIR_CLOSE(disk_fd) < 0 && ret == 0) { + virReportSystemError(errno, _("Unable to close %s"), disk_path); ret = -1; } return ret; diff --git a/src/util/virfile.h b/src/util/virfile.h index 59efb760f3..8e378efe30 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -384,4 +384,4 @@ int virFileDataSync(int fd); int virFileSetCOW(const char *path, virTristateBool state); -off_t virFileDiskCopy(const char *path, int fd, int oflags); +off_t virFileDiskCopy(int disk_fd, const char *disk_path, int remote_fd, const char *remote_path); -- 2.35.3
On Thu, May 05, 2022 at 02:52:19PM +0200, Claudio Fontana wrote:
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 17 +--------- src/util/virfile.c | 82 ++++++++++++++++++++++++++++++++++----------- src/util/virfile.h | 2 +- 3 files changed, 64 insertions(+), 37 deletions(-)
diff --git a/src/util/iohelper.c b/src/util/iohelper.c index ce10ccb905..055540c8c4 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -58,7 +58,6 @@ int main(int argc, char **argv) { const char *path; - int oflags = -1; int fd = -1;
program_name = argv[0]; @@ -79,25 +78,11 @@ main(int argc, char **argv) program_name, argv[3]); exit(EXIT_FAILURE); } -#ifdef F_GETFL - oflags = fcntl(fd, F_GETFL); -#else - /* Stupid mingw. */ - if (fd == STDIN_FILENO) - oflags = O_RDONLY; - else if (fd == STDOUT_FILENO) - oflags = O_WRONLY; -#endif - if (oflags < 0) { - fprintf(stderr, _("%s: unable to determine access mode of fd %d"), - program_name, fd); - exit(EXIT_FAILURE); - } } else { /* unknown argc pattern */ usage(EXIT_FAILURE); }
- if (fd < 0 || virFileDiskCopy(path, fd, oflags) < 0) + if (fd < 0 || virFileDiskCopy(fd, path, -1, "stdio") < 0) goto error;
return 0; diff --git a/src/util/virfile.c b/src/util/virfile.c index 30b0eee074..87d3e53819 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4669,19 +4669,64 @@ runIOCopy(const struct runIOParams p) return total; }
+/** + * virFileDiskCopy: run IO to copy data between storage and a pipe or socket. + * + * @disk_fd: the already open regular file or block device + * @disk_path: the pathname corresponding to disk_fd (for error reporting) + * @remote_fd: the pipe or socket + * Use -1 to auto-choose between STDIN or STDOUT. + * @remote_path: the pathname corresponding to remote_fd (for error reporting) + * + * Note that the direction of the transfer is detected based on the @disk_fd + * file access mode (man 2 open). Therefore @disk_fd must be opened with + * O_RDONLY or O_WRONLY. O_RDWR is not supported. + * + * virFileDiskCopy always closes the file descriptor disk_fd, + * and any error during close(2) is reported and considered a failure. + * + * Returns: bytes transferred or < 0 on failure. + */
off_t -virFileDiskCopy(const char *path, int fd, int oflags) +virFileDiskCopy(int disk_fd, const char *disk_path, int remote_fd, const char *remote_path) { int ret = -1; off_t total = 0; struct stat sb; struct runIOParams p; + int oflags = -1; + +#ifdef F_GETFL + oflags = fcntl(disk_fd, F_GETFL); +#else /* !F_GETFL */ + /* + * mingw still does not support F_GETFL, and does not seem willing to + * support it in the future. So we need a hack to get iohelper working, + * specifically possible only when using stdio. + */ + if (disk_fd == STDIN_FILENO) { + oflags = O_RDONLY; + } else if (disk_fd == STDOUT_FILENO) { + oflags = O_WRONLY; + } else { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, + _("%s: disk_fd must be stdin or stdout, due to lack of F_GETFL (mingw)"), + __FUNCTION__); + goto cleanup; + } +#endif /* !F_GETFL */
With this issue, and the posix_memalign issue, I'm inclined to say we should just take the easy way out and hide virFileDiskCopy behind #ifndef WIN32, since we don't even need it on WIN32. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 5/6/22 1:03 PM, Daniel P. Berrangé wrote:
On Thu, May 05, 2022 at 02:52:19PM +0200, Claudio Fontana wrote:
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/iohelper.c | 17 +--------- src/util/virfile.c | 82 ++++++++++++++++++++++++++++++++++----------- src/util/virfile.h | 2 +- 3 files changed, 64 insertions(+), 37 deletions(-)
diff --git a/src/util/iohelper.c b/src/util/iohelper.c index ce10ccb905..055540c8c4 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -58,7 +58,6 @@ int main(int argc, char **argv) { const char *path; - int oflags = -1; int fd = -1;
program_name = argv[0]; @@ -79,25 +78,11 @@ main(int argc, char **argv) program_name, argv[3]); exit(EXIT_FAILURE); } -#ifdef F_GETFL - oflags = fcntl(fd, F_GETFL); -#else - /* Stupid mingw. */ - if (fd == STDIN_FILENO) - oflags = O_RDONLY; - else if (fd == STDOUT_FILENO) - oflags = O_WRONLY; -#endif - if (oflags < 0) { - fprintf(stderr, _("%s: unable to determine access mode of fd %d"), - program_name, fd); - exit(EXIT_FAILURE); - } } else { /* unknown argc pattern */ usage(EXIT_FAILURE); }
- if (fd < 0 || virFileDiskCopy(path, fd, oflags) < 0) + if (fd < 0 || virFileDiskCopy(fd, path, -1, "stdio") < 0) goto error;
return 0; diff --git a/src/util/virfile.c b/src/util/virfile.c index 30b0eee074..87d3e53819 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4669,19 +4669,64 @@ runIOCopy(const struct runIOParams p) return total; }
+/** + * virFileDiskCopy: run IO to copy data between storage and a pipe or socket. + * + * @disk_fd: the already open regular file or block device + * @disk_path: the pathname corresponding to disk_fd (for error reporting) + * @remote_fd: the pipe or socket + * Use -1 to auto-choose between STDIN or STDOUT. + * @remote_path: the pathname corresponding to remote_fd (for error reporting) + * + * Note that the direction of the transfer is detected based on the @disk_fd + * file access mode (man 2 open). Therefore @disk_fd must be opened with + * O_RDONLY or O_WRONLY. O_RDWR is not supported. + * + * virFileDiskCopy always closes the file descriptor disk_fd, + * and any error during close(2) is reported and considered a failure. + * + * Returns: bytes transferred or < 0 on failure. + */
off_t -virFileDiskCopy(const char *path, int fd, int oflags) +virFileDiskCopy(int disk_fd, const char *disk_path, int remote_fd, const char *remote_path) { int ret = -1; off_t total = 0; struct stat sb; struct runIOParams p; + int oflags = -1; + +#ifdef F_GETFL + oflags = fcntl(disk_fd, F_GETFL); +#else /* !F_GETFL */ + /* + * mingw still does not support F_GETFL, and does not seem willing to + * support it in the future. So we need a hack to get iohelper working, + * specifically possible only when using stdio. + */ + if (disk_fd == STDIN_FILENO) { + oflags = O_RDONLY; + } else if (disk_fd == STDOUT_FILENO) { + oflags = O_WRONLY; + } else { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, + _("%s: disk_fd must be stdin or stdout, due to lack of F_GETFL (mingw)"), + __FUNCTION__); + goto cleanup; + } +#endif /* !F_GETFL */
With this issue, and the posix_memalign issue, I'm inclined to say we should just take the easy way out and hide virFileDiskCopy behind #ifndef WIN32, since we don't even need it on WIN32.
Ah ok this works too. Thanks! Claudio
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/virfile.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/util/virfile.c b/src/util/virfile.c index 87d3e53819..64d7f6948b 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4758,6 +4758,10 @@ virFileDiskCopy(int disk_fd, const char *disk_path, int remote_fd, const char *r if (!p.isBlockDev && p.isDirect) { off_t off; if (p.isWrite) { + /* + * note: for write we do not only check that disk_fd is seekable, + * we also want to know that the file is empty, so we need SEEK_END. + */ if ((off = lseek(disk_fd, 0, SEEK_END)) != 0) { virReportSystemError(off < 0 ? errno : EINVAL, "%s", _("O_DIRECT write needs empty seekable file")); -- 2.35.3
On Thu, May 05, 2022 at 02:52:20PM +0200, Claudio Fontana wrote:
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/util/virfile.c | 4 ++++ 1 file changed, 4 insertions(+)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
For the save direction, this helper listens on a unix socket which QEMU connects to for multifd migration to files. For the restore direction, this helper connects to a unix socket QEMU listens at for multifd migration from files. The file descriptors are passed as command line parameters. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- po/POTFILES.in | 1 + src/libvirt_private.syms | 1 + src/util/meson.build | 15 +++ src/util/multifd-helper.c | 249 ++++++++++++++++++++++++++++++++++++++ src/util/virthread.c | 5 + src/util/virthread.h | 1 + 6 files changed, 272 insertions(+) create mode 100644 src/util/multifd-helper.c diff --git a/po/POTFILES.in b/po/POTFILES.in index 0d9adb0758..4efb330262 100644 --- a/po/POTFILES.in +++ b/po/POTFILES.in @@ -241,6 +241,7 @@ @SRCDIR@src/storage_file/storage_source_backingstore.c @SRCDIR@src/test/test_driver.c @SRCDIR@src/util/iohelper.c +@SRCDIR@src/util/multifd-helper.c @SRCDIR@src/util/viralloc.c @SRCDIR@src/util/virarptable.c @SRCDIR@src/util/viraudit.c diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 97bfca906b..5f2bee985e 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -3427,6 +3427,7 @@ virThreadCreateFull; virThreadID; virThreadIsSelf; virThreadJoin; +virThreadJoinRet; virThreadMaxName; virThreadSelf; virThreadSelfID; diff --git a/src/util/meson.build b/src/util/meson.build index 84ef13ba32..2f8b57eabf 100644 --- a/src/util/meson.build +++ b/src/util/meson.build @@ -178,6 +178,11 @@ io_helper_sources = [ 'virfile.c', ] +multifd_helper_sources = [ + 'multifd-helper.c', + 'virfile.c', +] + virt_util_lib = static_library( 'virt_util', [ @@ -218,6 +223,16 @@ if conf.has('WITH_LIBVIRTD') libutil_dep, ], } + virt_helpers += { + 'name': 'libvirt_multifd_helper', + 'sources': [ + files(multifd_helper_sources), + dtrace_gen_headers, + ], + 'deps': [ + libutil_dep, + ], + } endif util_inc_dir = include_directories('.') diff --git a/src/util/multifd-helper.c b/src/util/multifd-helper.c new file mode 100644 index 0000000000..59657578e2 --- /dev/null +++ b/src/util/multifd-helper.c @@ -0,0 +1,249 @@ +/* + * multifd-helper.c: listens on Unix socket to perform I/O to multiple files + * + * Copyright (C) 2022 SUSE LLC + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * This has been written to support QEMU multifd migration to file, + * allowing better use of cpu resources to speed up the save/restore. + */ + +#include <config.h> + +#include <unistd.h> +#include <fcntl.h> +#include <stdlib.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/socket.h> +#include <sys/un.h> + +#include "virthread.h" +#include "virfile.h" +#include "virerror.h" +#include "virstring.h" +#include "virgettext.h" + +#define VIR_FROM_THIS VIR_FROM_STORAGE + +typedef struct _multiFdConnData multiFdConnData; +struct _multiFdConnData { + int clientfd; + int filefd; + int oflags; + const char *path; + virThread tid; + + off_t total; +}; + +typedef struct _multiFdThreadArgs multiFdThreadArgs; +struct _multiFdThreadArgs { + int nchannels; + multiFdConnData *conn; /* contains main fd + nchannels */ + const char *sun_path; /* unix socket name to use for the server */ + struct sockaddr_un serv_addr; + + off_t total; +}; + +static void clientThreadFunc(void *a) +{ + multiFdConnData *c = a; + c->total = virFileDiskCopy(c->filefd, c->path, c->clientfd, "socket"); +} + +static off_t waitClientThreads(multiFdConnData *conn, int n) +{ + int idx; + off_t total = 0; + for (idx = 0; idx < n; idx++) { + multiFdConnData *c = &conn[idx]; + if (virThreadJoinRet(&c->tid) < 0) { + total = -1; + } else if (total >= 0) { + total += c->total; + } + if (VIR_CLOSE(c->clientfd) < 0) { + total = -1; + } + } + return total; +} + +static void loadThreadFunc(void *a) +{ + multiFdThreadArgs *args = a; + int idx; + args->total = -1; + + for (idx = 0; idx < args->nchannels + 1; idx++) { + /* Perform outgoing connections */ + multiFdConnData *c = &args->conn[idx]; + c->clientfd = socket(AF_UNIX, SOCK_STREAM, 0); + if (c->clientfd < 0) { + virReportSystemError(errno, "%s", _("loadThread: socket() failed")); + goto cleanup; + } + if (connect(c->clientfd, (const struct sockaddr *)&args->serv_addr, + sizeof(struct sockaddr_un)) < 0) { + virReportSystemError(errno, "%s", _("loadThread: connect() failed")); + goto cleanup; + } + if (virThreadCreate(&c->tid, true, &clientThreadFunc, c) < 0) { + virReportSystemError(errno, "%s", _("loadThread: client thread creation failed")); + goto cleanup; + } + } + args->total = waitClientThreads(args->conn, args->nchannels + 1); + + cleanup: + for (idx = 0; idx < args->nchannels + 1; idx++) { + multiFdConnData *c = &args->conn[idx]; + VIR_FORCE_CLOSE(c->clientfd); + } +} + +static void saveThreadFunc(void *a) +{ + multiFdThreadArgs *args = a; + int idx; + const char buf[1] = {'R'}; + int sockfd; + + if ((sockfd = socket(AF_UNIX, SOCK_STREAM, 0)) < 0) { + virReportSystemError(errno, "%s", _("saveThread: socket() failed")); + return; + } + unlink(args->sun_path); + if (bind(sockfd, (struct sockaddr *)&args->serv_addr, sizeof(args->serv_addr)) < 0) { + virReportSystemError(errno, "%s", _("saveThread: bind() failed")); + goto cleanup; + } + if (listen(sockfd, args->nchannels + 1) < 0) { + virReportSystemError(errno, "%s", _("saveThread: listen() failed")); + goto cleanup; + } + + /* signal that the server is ready */ + if (safewrite(STDOUT_FILENO, &buf, 1) != 1) { + virReportSystemError(errno, "%s", _("saveThread: safewrite failed")); + goto cleanup; + } + + for (idx = 0; idx < args->nchannels + 1; idx++) { + /* Wait for incoming connection. */ + multiFdConnData *c = &args->conn[idx]; + if ((c->clientfd = accept(sockfd, NULL, NULL)) < 0) { + virReportSystemError(errno, "%s", _("saveThread: accept() failed")); + goto cleanup; + } + if (virThreadCreate(&c->tid, true, &clientThreadFunc, c) < 0) { + virReportSystemError(errno, "%s", _("saveThread: client thread creation failed")); + goto cleanup; + } + } + + args->total = waitClientThreads(args->conn, args->nchannels + 1); + + cleanup: + for (idx = 0; idx < args->nchannels + 1; idx++) { + multiFdConnData *c = &args->conn[idx]; + VIR_FORCE_CLOSE(c->clientfd); + } + if (VIR_CLOSE(sockfd) < 0) + args->total = -1; +} + +static const char *program_name; + +G_GNUC_NORETURN static void +usage(int status) +{ + if (status) { + fprintf(stderr, _("%s: try --help for more details"), program_name); + } else { + fprintf(stderr, _("Usage: %s UNIX_SOCNAME N MAINFD FD0 FD1 ... FDn"), program_name); + } + exit(status); +} + +int +main(int argc, char **argv) +{ + virThread tid; + virThreadFunc func = saveThreadFunc; + multiFdThreadArgs args = { 0 }; + int idx; + + /* sleep(10); */ + + program_name = argv[0]; + + if (virGettextInitialize() < 0 || + virErrorInitialize() < 0) { + fprintf(stderr, _("%s: initialization failed"), program_name); + exit(EXIT_FAILURE); + } + + if (argc > 1 && STREQ(argv[1], "--help")) + usage(EXIT_SUCCESS); + if (argc < 4) + usage(EXIT_FAILURE); + + args.sun_path = argv[1]; + if (virStrToLong_i(argv[2], NULL, 10, &args.nchannels) < 0) + fprintf(stderr, _("%s: malformed number of channels N %s"), program_name, argv[2]); + + if (argc < 4 + args.nchannels) + usage(EXIT_FAILURE); + + args.conn = g_new0(multiFdConnData, args.nchannels + 1); + + for (idx = 3; idx < 3 + args.nchannels + 1; idx++) { + multiFdConnData *c = &args.conn[idx - 3]; + + if (virStrToLong_i(argv[idx], NULL, 10, &c->filefd) < 0) { + fprintf(stderr, _("%s: malformed FD %s"), program_name, argv[idx]); + usage(EXIT_FAILURE); + } +#ifndef F_GETFL +#error "multifd-helper requires F_GETFL parameter of fcntl" +#endif + c->oflags = fcntl(c->filefd, F_GETFL); + if ((c->oflags & O_ACCMODE) == O_RDONLY) { + func = loadThreadFunc; + } + } + + /* initialize server address structure */ + memset(&args.serv_addr, 0, sizeof(args.serv_addr)); + args.serv_addr.sun_family = AF_UNIX; + virStrcpyStatic(args.serv_addr.sun_path, args.sun_path); + + if (virThreadCreate(&tid, true, func, &args) < 0) { + virReportSystemError(errno, _("%s: failed to create server thread"), program_name); + exit(EXIT_FAILURE); + } + + if (virThreadJoinRet(&tid) < 0) + exit(EXIT_FAILURE); + + if (args.total < 0) + exit(EXIT_FAILURE); + + exit(EXIT_SUCCESS); +} diff --git a/src/util/virthread.c b/src/util/virthread.c index 5422bb74fd..0f6c6a68fa 100644 --- a/src/util/virthread.c +++ b/src/util/virthread.c @@ -348,6 +348,11 @@ void virThreadJoin(virThread *thread) pthread_join(thread->thread, NULL); } +int virThreadJoinRet(virThread *thread) +{ + return pthread_join(thread->thread, NULL); +} + void virThreadCancel(virThread *thread) { pthread_cancel(thread->thread); diff --git a/src/util/virthread.h b/src/util/virthread.h index 23abe0b6c9..5cecb9bd8a 100644 --- a/src/util/virthread.h +++ b/src/util/virthread.h @@ -89,6 +89,7 @@ int virThreadCreateFull(virThread *thread, void virThreadSelf(virThread *thread); bool virThreadIsSelf(virThread *thread); void virThreadJoin(virThread *thread); +int virThreadJoinRet(virThread *thread); size_t virThreadMaxName(void); -- 2.35.3
create an alternative to virDomainSaveFlags that is more extendible via parameters. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- include/libvirt/libvirt-domain.h | 26 +++++++++++++++++ src/driver-hypervisor.h | 7 +++++ src/libvirt-domain.c | 48 ++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 5 ++++ 4 files changed, 86 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 2edef9c4e1..52c86f3813 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1562,6 +1562,10 @@ int virDomainSaveFlags (virDomainPtr domain, const char *to, const char *dxml, unsigned int flags); +int virDomainSaveParams (virDomainPtr domain, + virTypedParameterPtr params, + int nparams, + unsigned int flags); int virDomainRestore (virConnectPtr conn, const char *from); int virDomainRestoreFlags (virConnectPtr conn, @@ -1569,6 +1573,28 @@ int virDomainRestoreFlags (virConnectPtr conn, const char *dxml, unsigned int flags); +/** + * VIR_SAVE_PARAM_FILE: + * + * the parameter used to specify the savestate file to save to or restore from. + * + * Since: 8.4.0 + */ +# define VIR_SAVE_PARAM_FILE "file" + +/** + * VIR_SAVE_PARAM_DXML: + * + * an optional parameter used to adjust guest xml on restore. + * If the hypervisor supports it, it can be used to alter + * host-specific portions of the domain XML that will be used when + * restoring an image. For example, it is possible to alter the + * device while the domain is stopped. + * + * Since: 8.4.0 + */ +# define VIR_SAVE_PARAM_DXML "dxml" + /* See below for virDomainSaveImageXMLFlags */ char * virDomainSaveImageGetXMLDesc (virConnectPtr conn, const char *file, diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index 4423eb0885..434a7d7cc4 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -240,6 +240,12 @@ typedef int const char *dxml, unsigned int flags); +typedef int +(*virDrvDomainSaveParams)(virDomainPtr domain, + virTypedParameterPtr params, + int nparams, + unsigned int flags); + typedef int (*virDrvDomainRestore)(virConnectPtr conn, const char *from); @@ -1489,6 +1495,7 @@ struct _virHypervisorDriver { virDrvDomainGetControlInfo domainGetControlInfo; virDrvDomainSave domainSave; virDrvDomainSaveFlags domainSaveFlags; + virDrvDomainSaveParams domainSaveParams; virDrvDomainRestore domainRestore; virDrvDomainRestoreFlags domainRestoreFlags; virDrvDomainSaveImageGetXMLDesc domainSaveImageGetXMLDesc; diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 27993829a0..ca2fe5e99d 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -999,6 +999,54 @@ virDomainSaveFlags(virDomainPtr domain, const char *to, return -1; } +/** + * virDomainSaveParams: + * @domain: a domain object + * @params: save parameters + * @nparams: number of save parameters + * @flags: bitwise-OR of virDomainSaveRestoreFlags + * + * This method extends virDomainSaveFlags by adding parameters. + * + * Returns 0 in case of success and -1 in case of failure. + * + * Since: 8.4.0 + */ +int +virDomainSaveParams(virDomainPtr domain, + virTypedParameterPtr params, int nparams, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "params=%p, nparams=%d, flags=0x%x", + params, nparams, flags); + VIR_TYPED_PARAMS_DEBUG(params, nparams); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + conn = domain->conn; + + virCheckReadOnlyGoto(conn->flags, error); + + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_SAVE_RUNNING, + VIR_DOMAIN_SAVE_PAUSED, + error); + + if (conn->driver->domainSaveParams) { + if (conn->driver->domainSaveParams(domain, params, nparams, flags) < 0) + goto error; + return 0; + } + + virReportUnsupportedError(); + + error: + virDispatchError(domain->conn); + return -1; +} + /** * virDomainRestore: diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index f93692c427..32ffbbaf84 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -916,4 +916,9 @@ LIBVIRT_8.0.0 { virDomainSetLaunchSecurityState; } LIBVIRT_7.8.0; +LIBVIRT_8.4.0 { + global: + virDomainSaveParams; +} LIBVIRT_8.0.0; + # .... define new API here using predicted next version number .... -- 2.35.3
On Thu, May 05, 2022 at 02:52:22PM +0200, Claudio Fontana wrote:
create an alternative to virDomainSaveFlags that is more extendible via parameters.
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- include/libvirt/libvirt-domain.h | 26 +++++++++++++++++ src/driver-hypervisor.h | 7 +++++ src/libvirt-domain.c | 48 ++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 5 ++++ 4 files changed, 86 insertions(+)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
add new API in order to be able to extend parameters to the domain restore operation. We will use it to fit the existing arguments of VirDomainRestoreFlags for now. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- include/libvirt/libvirt-domain.h | 4 +++ src/driver-hypervisor.h | 7 +++++ src/libvirt-domain.c | 45 ++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 1 + 4 files changed, 57 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 52c86f3813..d8375583fd 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1572,6 +1572,10 @@ int virDomainRestoreFlags (virConnectPtr conn, const char *from, const char *dxml, unsigned int flags); +int virDomainRestoreParams (virConnectPtr conn, + virTypedParameterPtr params, + int nparams, + unsigned int flags); /** * VIR_SAVE_PARAM_FILE: diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index 434a7d7cc4..69516e8fea 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -256,6 +256,12 @@ typedef int const char *dxml, unsigned int flags); +typedef int +(*virDrvDomainRestoreParams)(virConnectPtr conn, + virTypedParameterPtr params, + int nparams, + unsigned int flags); + typedef char * (*virDrvDomainSaveImageGetXMLDesc)(virConnectPtr conn, const char *file, @@ -1498,6 +1504,7 @@ struct _virHypervisorDriver { virDrvDomainSaveParams domainSaveParams; virDrvDomainRestore domainRestore; virDrvDomainRestoreFlags domainRestoreFlags; + virDrvDomainRestoreParams domainRestoreParams; virDrvDomainSaveImageGetXMLDesc domainSaveImageGetXMLDesc; virDrvDomainSaveImageDefineXML domainSaveImageDefineXML; virDrvDomainCoreDump domainCoreDump; diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index ca2fe5e99d..0d6afcc1c6 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -1181,6 +1181,51 @@ virDomainRestoreFlags(virConnectPtr conn, const char *from, const char *dxml, } +/** + * virDomainRestoreParams: + * @conn: pointer to the hypervisor connection + * @params: restore parameters + * @nparams: number of restore parameters + * @flags: bitwise-OR of virDomainSaveRestoreFlags + * + * This method extends virDomainRestoreFlags by adding parameters. + * + * Returns 0 in case of success and -1 in case of failure. + * + * Since: 8.4.0 + */ +int +virDomainRestoreParams(virConnectPtr conn, + virTypedParameterPtr params, int nparams, + unsigned int flags) +{ + VIR_DEBUG("conn=%p, params=%p, nparams=%d, flags=0x%x", + conn, params, nparams, flags); + VIR_TYPED_PARAMS_DEBUG(params, nparams); + + virResetLastError(); + + virCheckConnectReturn(conn, -1); + virCheckReadOnlyGoto(conn->flags, error); + + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_SAVE_RUNNING, + VIR_DOMAIN_SAVE_PAUSED, + error); + + if (conn->driver->domainRestoreParams) { + if (conn->driver->domainRestoreParams(conn, params, nparams, flags) < 0) + goto error; + return 0; + } + + virReportUnsupportedError(); + + error: + virDispatchError(conn); + return -1; +} + + /** * virDomainSaveImageGetXMLDesc: * @conn: pointer to the hypervisor connection diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 32ffbbaf84..9f58b52924 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -919,6 +919,7 @@ LIBVIRT_8.0.0 { LIBVIRT_8.4.0 { global: virDomainSaveParams; + virDomainRestoreParams; } LIBVIRT_8.0.0; # .... define new API here using predicted next version number .... -- 2.35.3
On Thu, May 05, 2022 at 02:52:23PM +0200, Claudio Fontana wrote:
add new API in order to be able to extend parameters to the domain restore operation. We will use it to fit the existing arguments of VirDomainRestoreFlags for now.
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- include/libvirt/libvirt-domain.h | 4 +++ src/driver-hypervisor.h | 7 +++++ src/libvirt-domain.c | 45 ++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 1 + 4 files changed, 57 insertions(+)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 17 ++++++++++++++++- src/remote_protocol-structs | 9 +++++++++ 3 files changed, 26 insertions(+), 1 deletion(-) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 7e7a21fcab..65fc19643a 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -8446,6 +8446,7 @@ static virHypervisorDriver hypervisor_driver = { .domainGetControlInfo = remoteDomainGetControlInfo, /* 0.9.3 */ .domainSave = remoteDomainSave, /* 0.3.0 */ .domainSaveFlags = remoteDomainSaveFlags, /* 0.9.4 */ + .domainSaveParams = remoteDomainSaveParams, /* 8.4.0 */ .domainRestore = remoteDomainRestore, /* 0.3.0 */ .domainRestoreFlags = remoteDomainRestoreFlags, /* 0.9.4 */ .domainSaveImageGetXMLDesc = remoteDomainSaveImageGetXMLDesc, /* 0.9.4 */ diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 4f13cef662..ef52ff346a 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -230,6 +230,9 @@ const REMOTE_NODE_MEMORY_PARAMETERS_MAX = 64; /* Upper limit on migrate parameters */ const REMOTE_DOMAIN_MIGRATE_PARAM_LIST_MAX = 64; +/* Upper limit on save/restore parameters */ +const REMOTE_DOMAIN_SAVE_PARAMS_MAX = 64; + /* Upper limit on number of job stats */ const REMOTE_DOMAIN_JOB_STATS_MAX = 64; @@ -980,6 +983,12 @@ struct remote_domain_save_flags_args { unsigned int flags; }; +struct remote_domain_save_params_args { + remote_nonnull_domain dom; + remote_typed_param params<REMOTE_DOMAIN_SAVE_PARAMS_MAX>; + unsigned int flags; +}; + struct remote_domain_restore_args { remote_nonnull_string from; }; @@ -6920,5 +6929,11 @@ enum remote_procedure { * @generate: both * @acl: domain:write */ - REMOTE_PROC_DOMAIN_SET_LAUNCH_SECURITY_STATE = 439 + REMOTE_PROC_DOMAIN_SET_LAUNCH_SECURITY_STATE = 439, + + /** + * @generate: both + * @acl: domain:hibernate + */ + REMOTE_PROC_DOMAIN_SAVE_PARAMS = 440 }; diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index d88176781d..f5e0a60396 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -563,6 +563,14 @@ struct remote_domain_save_flags_args { remote_string dxml; u_int flags; }; +struct remote_domain_save_params_args { + remote_nonnull_domain dom; + struct { + u_int params_len; + remote_typed_param * params_val; + } params; + u_int flags; +}; struct remote_domain_restore_args { remote_nonnull_string from; }; @@ -3689,4 +3697,5 @@ enum remote_procedure { REMOTE_PROC_NETWORK_CREATE_XML_FLAGS = 437, REMOTE_PROC_DOMAIN_EVENT_MEMORY_DEVICE_SIZE_CHANGE = 438, REMOTE_PROC_DOMAIN_SET_LAUNCH_SECURITY_STATE = 439, + REMOTE_PROC_DOMAIN_SAVE_PARAMS = 440, }; -- 2.35.3
On Thu, May 05, 2022 at 02:52:24PM +0200, Claudio Fontana wrote:
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 17 ++++++++++++++++- src/remote_protocol-structs | 9 +++++++++ 3 files changed, 26 insertions(+), 1 deletion(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
in preparation for the addition of DomainRestoreParams, add it to the list of methods requiring a conn first argument. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/rpc/gendispatch.pl | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/rpc/gendispatch.pl b/src/rpc/gendispatch.pl index 9f5bf0e316..a64ff3e73f 100755 --- a/src/rpc/gendispatch.pl +++ b/src/rpc/gendispatch.pl @@ -637,7 +637,10 @@ elsif ($mode eq "server") { } elsif ($args_member =~ m/^remote_typed_param (\S+)<(\S+)>;/) { push(@vars_list, "virTypedParameterPtr $1 = NULL"); push(@vars_list, "int n$1 = 0"); - if ($call->{ProcName} eq "NodeSetMemoryParameters") { + + # NB: if your new API starts with remote_typed_params, enter it here if you need + # the conn arg to be passed first! + if ($call->{ProcName} eq "NodeSetMemoryParameters" || $call->{ProcName} eq "DomainRestoreParams") { push(@args_list, $conn_var); } push(@args_list, "$1"); -- 2.35.3
On Thu, May 05, 2022 at 02:52:25PM +0200, Claudio Fontana wrote:
in preparation for the addition of DomainRestoreParams, add it to the list of methods requiring a conn first argument.
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/rpc/gendispatch.pl | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 14 +++++++++++++- src/remote_protocol-structs | 8 ++++++++ 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 65fc19643a..423f5f9fb9 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -8449,6 +8449,7 @@ static virHypervisorDriver hypervisor_driver = { .domainSaveParams = remoteDomainSaveParams, /* 8.4.0 */ .domainRestore = remoteDomainRestore, /* 0.3.0 */ .domainRestoreFlags = remoteDomainRestoreFlags, /* 0.9.4 */ + .domainRestoreParams = remoteDomainRestoreParams, /* 8.4.0 */ .domainSaveImageGetXMLDesc = remoteDomainSaveImageGetXMLDesc, /* 0.9.4 */ .domainSaveImageDefineXML = remoteDomainSaveImageDefineXML, /* 0.9.4 */ .domainCoreDump = remoteDomainCoreDump, /* 0.3.0 */ diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index ef52ff346a..085631c11b 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -999,6 +999,11 @@ struct remote_domain_restore_flags_args { unsigned int flags; }; +struct remote_domain_restore_params_args { + remote_typed_param params<REMOTE_DOMAIN_SAVE_PARAMS_MAX>; + unsigned int flags; +}; + struct remote_domain_save_image_get_xml_desc_args { remote_nonnull_string file; unsigned int flags; @@ -6935,5 +6940,12 @@ enum remote_procedure { * @generate: both * @acl: domain:hibernate */ - REMOTE_PROC_DOMAIN_SAVE_PARAMS = 440 + REMOTE_PROC_DOMAIN_SAVE_PARAMS = 440, + + /** + * @generate: both + * @acl: domain:start + * @acl: domain:write + */ + REMOTE_PROC_DOMAIN_RESTORE_PARAMS = 441 }; diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index f5e0a60396..4ffdce5679 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -579,6 +579,13 @@ struct remote_domain_restore_flags_args { remote_string dxml; u_int flags; }; +struct remote_domain_restore_params_args { + struct { + u_int params_len; + remote_typed_param * params_val; + } params; + u_int flags; +}; struct remote_domain_save_image_get_xml_desc_args { remote_nonnull_string file; u_int flags; @@ -3698,4 +3705,5 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_EVENT_MEMORY_DEVICE_SIZE_CHANGE = 438, REMOTE_PROC_DOMAIN_SET_LAUNCH_SECURITY_STATE = 439, REMOTE_PROC_DOMAIN_SAVE_PARAMS = 440, + REMOTE_PROC_DOMAIN_RESTORE_PARAMS = 441, }; -- 2.35.3
On Thu, May 05, 2022 at 02:52:26PM +0200, Claudio Fontana wrote:
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 14 +++++++++++++- src/remote_protocol-structs | 8 ++++++++ 3 files changed, 22 insertions(+), 1 deletion(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_driver.c | 53 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 464c080409..4a74713738 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2804,6 +2804,58 @@ qemuDomainSave(virDomainPtr dom, const char *path) return qemuDomainSaveFlags(dom, path, NULL, 0); } +static int +qemuDomainSaveParams(virDomainPtr dom, + virTypedParameterPtr params, int nparams, + unsigned int flags) +{ + const char *to = NULL; + const char *dxml = NULL; + virQEMUDriver *driver = dom->conn->privateData; + int compressed; + g_autoptr(virCommand) compressor = NULL; + int ret = -1; + virDomainObj *vm = NULL; + g_autoptr(virQEMUDriverConfig) cfg = NULL; + + virCheckFlags(VIR_DOMAIN_SAVE_BYPASS_CACHE | + VIR_DOMAIN_SAVE_RUNNING | + VIR_DOMAIN_SAVE_PAUSED, -1); + + if (virTypedParamsValidate(params, nparams, + VIR_SAVE_PARAM_FILE, VIR_TYPED_PARAM_STRING, + VIR_SAVE_PARAM_DXML, VIR_TYPED_PARAM_STRING, + NULL) < 0) + return -1; + + if (virTypedParamsGetString(params, nparams, VIR_SAVE_PARAM_FILE, &to) < 0) + return -1; + if (virTypedParamsGetString(params, nparams, VIR_SAVE_PARAM_DXML, &dxml) < 0) + return -1; + + cfg = virQEMUDriverGetConfig(driver); + if ((compressed = qemuSaveImageGetCompressionProgram(cfg->saveImageFormat, + &compressor, + "save", false)) < 0) + goto cleanup; + + if (!(vm = qemuDomainObjFromDomain(dom))) + goto cleanup; + + if (virDomainSaveParamsEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (virDomainObjCheckActive(vm) < 0) + goto cleanup; + + ret = qemuDomainSaveInternal(driver, vm, to, compressed, + compressor, dxml, flags); + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + static char * qemuDomainManagedSavePath(virQEMUDriver *driver, virDomainObj *vm) { @@ -20829,6 +20881,7 @@ static virHypervisorDriver qemuHypervisorDriver = { .domainGetControlInfo = qemuDomainGetControlInfo, /* 0.9.3 */ .domainSave = qemuDomainSave, /* 0.2.0 */ .domainSaveFlags = qemuDomainSaveFlags, /* 0.9.4 */ + .domainSaveParams = qemuDomainSaveParams, /* 8.4.0 */ .domainRestore = qemuDomainRestore, /* 0.2.0 */ .domainRestoreFlags = qemuDomainRestoreFlags, /* 0.9.4 */ .domainSaveImageGetXMLDesc = qemuDomainSaveImageGetXMLDesc, /* 0.9.4 */ -- 2.35.3
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_driver.c | 51 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 44 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 4a74713738..e3582f62a7 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5806,12 +5806,12 @@ static int qemuNodeGetSecurityModel(virConnectPtr conn, return 0; } - static int -qemuDomainRestoreFlags(virConnectPtr conn, - const char *path, - const char *dxml, - unsigned int flags) +qemuDomainRestoreInternal(virConnectPtr conn, + const char *path, + const char *dxml, + unsigned int flags, + int (*ensureACL)(virConnectPtr, virDomainDef *)) { virQEMUDriver *driver = conn->privateData; qemuDomainObjPrivate *priv = NULL; @@ -5840,7 +5840,7 @@ qemuDomainRestoreFlags(virConnectPtr conn, if (fd < 0) goto cleanup; - if (virDomainRestoreFlagsEnsureACL(conn, def) < 0) + if (ensureACL(conn, def) < 0) goto cleanup; if (virHookPresent(VIR_HOOK_DRIVER_QEMU)) { @@ -5908,11 +5908,47 @@ qemuDomainRestoreFlags(virConnectPtr conn, return ret; } +static int +qemuDomainRestoreFlags(virConnectPtr conn, + const char *path, + const char *dxml, + unsigned int flags) +{ + return qemuDomainRestoreInternal(conn, path, dxml, flags, + virDomainRestoreFlagsEnsureACL); +} + static int qemuDomainRestore(virConnectPtr conn, const char *path) { - return qemuDomainRestoreFlags(conn, path, NULL, 0); + return qemuDomainRestoreInternal(conn, path, NULL, 0, + virDomainRestoreEnsureACL); +} + +static int +qemuDomainRestoreParams(virConnectPtr conn, + virTypedParameterPtr params, int nparams, + unsigned int flags) +{ + const char *path = NULL; + const char *dxml = NULL; + int ret = -1; + + if (virTypedParamsValidate(params, nparams, + VIR_SAVE_PARAM_FILE, VIR_TYPED_PARAM_STRING, + VIR_SAVE_PARAM_DXML, VIR_TYPED_PARAM_STRING, + NULL) < 0) + return -1; + + if (virTypedParamsGetString(params, nparams, VIR_SAVE_PARAM_FILE, &path) < 0) + return -1; + if (virTypedParamsGetString(params, nparams, VIR_SAVE_PARAM_DXML, &dxml) < 0) + return -1; + + ret = qemuDomainRestoreInternal(conn, path, dxml, flags, + virDomainRestoreParamsEnsureACL); + return ret; } static char * @@ -20884,6 +20920,7 @@ static virHypervisorDriver qemuHypervisorDriver = { .domainSaveParams = qemuDomainSaveParams, /* 8.4.0 */ .domainRestore = qemuDomainRestore, /* 0.2.0 */ .domainRestoreFlags = qemuDomainRestoreFlags, /* 0.9.4 */ + .domainRestoreParams = qemuDomainRestoreParams, /* 8.4.0 */ .domainSaveImageGetXMLDesc = qemuDomainSaveImageGetXMLDesc, /* 0.9.4 */ .domainSaveImageDefineXML = qemuDomainSaveImageDefineXML, /* 0.9.4 */ .domainCoreDump = qemuDomainCoreDump, /* 0.7.0 */ -- 2.35.3
in order to enable parallel save functionality, we will need an opportune new flag and a parameter to specify the number of extra connections to use. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- include/libvirt/libvirt-domain.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index d8375583fd..d130fbf481 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1554,6 +1554,7 @@ typedef enum { VIR_DOMAIN_SAVE_RUNNING = 1 << 1, /* Favor running over paused (Since: 0.9.5) */ VIR_DOMAIN_SAVE_PAUSED = 1 << 2, /* Favor paused over running (Since: 0.9.5) */ VIR_DOMAIN_SAVE_RESET_NVRAM = 1 << 3, /* Re-initialize NVRAM from template (Since: 8.1.0) */ + VIR_DOMAIN_SAVE_PARALLEL = 1 << 4, /* Parallel Save/Restore to multiple files (Since: 8.4.0) */ } virDomainSaveRestoreFlags; int virDomainSave (virDomainPtr domain, @@ -1581,6 +1582,8 @@ int virDomainRestoreParams (virConnectPtr conn, * VIR_SAVE_PARAM_FILE: * * the parameter used to specify the savestate file to save to or restore from. + * For parallel saves, this is the main file, with the extra connections adding suffix + * .1 .2 .3 ... up to VIR_SAVE_PARAM_PARALLEL_CONNECTIONS. * * Since: 8.4.0 */ @@ -1599,6 +1602,21 @@ int virDomainRestoreParams (virConnectPtr conn, */ # define VIR_SAVE_PARAM_DXML "dxml" +/** + * VIR_SAVE_PARAM_PARALLEL_CONNECTIONS: + * + * this optional parameter mirrors the migration parameter + * VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS. + * + * This parameter is used when saving or restoring state files + * in parallel using the flag VIR_DOMAIN_SAVE_PARALLEL . + * It specifies the number of extra files to save/restore + * using parallel connections. + * + * Since: 8.4.0 + */ +# define VIR_SAVE_PARAM_PARALLEL_CONNECTIONS "parallel.connections" + /* See below for virDomainSaveImageXMLFlags */ char * virDomainSaveImageGetXMLDesc (virConnectPtr conn, const char *file, -- 2.35.3
and its companion param VIR_SAVE_PARAM_PARALLEL_CONNECTIONS Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_driver.c | 17 +++++++++++------ src/qemu/qemu_saveimage.c | 1 + src/qemu/qemu_saveimage.h | 1 + src/qemu/qemu_snapshot.c | 2 +- 4 files changed, 14 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index e3582f62a7..e276a66815 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2641,7 +2641,7 @@ static int qemuDomainSaveInternal(virQEMUDriver *driver, virDomainObj *vm, const char *path, int compressed, virCommand *compressor, - const char *xmlin, unsigned int flags) + const char *xmlin, int nconn, unsigned int flags) { g_autofree char *xml = NULL; bool was_running = false; @@ -2722,7 +2722,7 @@ qemuDomainSaveInternal(virQEMUDriver *driver, xml = NULL; ret = qemuSaveImageCreate(driver, vm, path, data, compressor, - flags, VIR_ASYNC_JOB_SAVE); + nconn, flags, VIR_ASYNC_JOB_SAVE); if (ret < 0) goto endjob; @@ -2791,7 +2791,7 @@ qemuDomainSaveFlags(virDomainPtr dom, const char *path, const char *dxml, goto cleanup; ret = qemuDomainSaveInternal(driver, vm, path, compressed, - compressor, dxml, flags); + compressor, dxml, -1, flags); cleanup: virDomainObjEndAPI(&vm); @@ -2815,16 +2815,19 @@ qemuDomainSaveParams(virDomainPtr dom, int compressed; g_autoptr(virCommand) compressor = NULL; int ret = -1; + int nconn = 0; virDomainObj *vm = NULL; g_autoptr(virQEMUDriverConfig) cfg = NULL; virCheckFlags(VIR_DOMAIN_SAVE_BYPASS_CACHE | VIR_DOMAIN_SAVE_RUNNING | - VIR_DOMAIN_SAVE_PAUSED, -1); + VIR_DOMAIN_SAVE_PAUSED | + VIR_DOMAIN_SAVE_PARALLEL, -1); if (virTypedParamsValidate(params, nparams, VIR_SAVE_PARAM_FILE, VIR_TYPED_PARAM_STRING, VIR_SAVE_PARAM_DXML, VIR_TYPED_PARAM_STRING, + VIR_SAVE_PARAM_PARALLEL_CONNECTIONS, VIR_TYPED_PARAM_INT, NULL) < 0) return -1; @@ -2832,6 +2835,8 @@ qemuDomainSaveParams(virDomainPtr dom, return -1; if (virTypedParamsGetString(params, nparams, VIR_SAVE_PARAM_DXML, &dxml) < 0) return -1; + if (virTypedParamsGetInt(params, nparams, VIR_SAVE_PARAM_PARALLEL_CONNECTIONS, &nconn) < 0) + return -1; cfg = virQEMUDriverGetConfig(driver); if ((compressed = qemuSaveImageGetCompressionProgram(cfg->saveImageFormat, @@ -2849,7 +2854,7 @@ qemuDomainSaveParams(virDomainPtr dom, goto cleanup; ret = qemuDomainSaveInternal(driver, vm, to, compressed, - compressor, dxml, flags); + compressor, dxml, nconn, flags); cleanup: virDomainObjEndAPI(&vm); @@ -2906,7 +2911,7 @@ qemuDomainManagedSave(virDomainPtr dom, unsigned int flags) VIR_INFO("Saving state of domain '%s' to '%s'", vm->def->name, name); ret = qemuDomainSaveInternal(driver, vm, name, compressed, - compressor, NULL, flags); + compressor, NULL, -1, flags); if (ret == 0) vm->hasManagedSave = true; diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 4fd4c5cfcd..7c76db359e 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -258,6 +258,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, const char *path, virQEMUSaveData *data, virCommand *compressor, + int nconn G_GNUC_UNUSED, unsigned int flags, virDomainAsyncJob asyncJob) { diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 391cd55ed0..b3d5c02fd6 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -96,6 +96,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, const char *path, virQEMUSaveData *data, virCommand *compressor, + int nconn, unsigned int flags, virDomainAsyncJob asyncJob); diff --git a/src/qemu/qemu_snapshot.c b/src/qemu/qemu_snapshot.c index b62fab7bb3..2e445e8296 100644 --- a/src/qemu/qemu_snapshot.c +++ b/src/qemu/qemu_snapshot.c @@ -1457,7 +1457,7 @@ qemuSnapshotCreateActiveExternal(virQEMUDriver *driver, memory_existing = virFileExists(snapdef->memorysnapshotfile); if ((ret = qemuSaveImageCreate(driver, vm, snapdef->memorysnapshotfile, - data, compressor, 0, + data, compressor, -1, 0, VIR_ASYNC_JOB_SNAPSHOT)) < 0) goto cleanup; -- 2.35.3
and its companion param VIR_SAVE_PARAM_PARALLEL_CONNECTIONS. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_driver.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index e276a66815..ff2be6ffe9 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5815,6 +5815,7 @@ static int qemuDomainRestoreInternal(virConnectPtr conn, const char *path, const char *dxml, + int nchannels G_GNUC_UNUSED, unsigned int flags, int (*ensureACL)(virConnectPtr, virDomainDef *)) { @@ -5834,7 +5835,8 @@ qemuDomainRestoreInternal(virConnectPtr conn, virCheckFlags(VIR_DOMAIN_SAVE_BYPASS_CACHE | VIR_DOMAIN_SAVE_RUNNING | VIR_DOMAIN_SAVE_PAUSED | - VIR_DOMAIN_SAVE_RESET_NVRAM, -1); + VIR_DOMAIN_SAVE_RESET_NVRAM | + VIR_DOMAIN_SAVE_PARALLEL, -1); if (flags & VIR_DOMAIN_SAVE_RESET_NVRAM) reset_nvram = true; @@ -5919,7 +5921,7 @@ qemuDomainRestoreFlags(virConnectPtr conn, const char *dxml, unsigned int flags) { - return qemuDomainRestoreInternal(conn, path, dxml, flags, + return qemuDomainRestoreInternal(conn, path, dxml, -1, flags, virDomainRestoreFlagsEnsureACL); } @@ -5927,7 +5929,7 @@ static int qemuDomainRestore(virConnectPtr conn, const char *path) { - return qemuDomainRestoreInternal(conn, path, NULL, 0, + return qemuDomainRestoreInternal(conn, path, NULL, -1, 0, virDomainRestoreEnsureACL); } @@ -5939,10 +5941,12 @@ qemuDomainRestoreParams(virConnectPtr conn, const char *path = NULL; const char *dxml = NULL; int ret = -1; + int nchannels = 2; if (virTypedParamsValidate(params, nparams, VIR_SAVE_PARAM_FILE, VIR_TYPED_PARAM_STRING, VIR_SAVE_PARAM_DXML, VIR_TYPED_PARAM_STRING, + VIR_SAVE_PARAM_PARALLEL_CONNECTIONS, VIR_TYPED_PARAM_INT, NULL) < 0) return -1; @@ -5950,8 +5954,10 @@ qemuDomainRestoreParams(virConnectPtr conn, return -1; if (virTypedParamsGetString(params, nparams, VIR_SAVE_PARAM_DXML, &dxml) < 0) return -1; + if (virTypedParamsGetInt(params, nparams, VIR_SAVE_PARAM_PARALLEL_CONNECTIONS, &nchannels) < 0) + return -1; - ret = qemuDomainRestoreInternal(conn, path, dxml, flags, + ret = qemuDomainRestoreInternal(conn, path, dxml, nchannels, flags, virDomainRestoreParamsEnsureACL); return ret; } -- 2.35.3
use this data type to encapsulate the pathname, file descriptor, wrapper, and need to unlink. Adapt qemuSaveImageCreate and qemuSaveImageOpen to use a virQEMUSaveFd instead of a plain file descriptor. This makes management of wrapper, need_unlink etc much easier. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_driver.c | 100 +++++++++------ src/qemu/qemu_saveimage.c | 257 +++++++++++++++++++++++--------------- src/qemu/qemu_saveimage.h | 27 +++- 3 files changed, 234 insertions(+), 150 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index ff2be6ffe9..e4cdd93b50 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5825,12 +5825,13 @@ qemuDomainRestoreInternal(virConnectPtr conn, virDomainObj *vm = NULL; g_autofree char *xmlout = NULL; const char *newxml = dxml; - int fd = -1; int ret = -1; virQEMUSaveData *data = NULL; - virFileWrapperFd *wrapperFd = NULL; + virQEMUSaveFd saveFd = QEMU_SAVEFD_INVALID; bool hook_taint = false; bool reset_nvram = false; + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + int oflags = O_RDONLY; virCheckFlags(VIR_DOMAIN_SAVE_BYPASS_CACHE | VIR_DOMAIN_SAVE_RUNNING | @@ -5841,10 +5842,18 @@ qemuDomainRestoreInternal(virConnectPtr conn, if (flags & VIR_DOMAIN_SAVE_RESET_NVRAM) reset_nvram = true; - fd = qemuSaveImageOpen(driver, NULL, path, &def, &data, - (flags & VIR_DOMAIN_SAVE_BYPASS_CACHE) != 0, - &wrapperFd, false, false); - if (fd < 0) + if (flags & VIR_DOMAIN_SAVE_BYPASS_CACHE) { + if (virFileDirectFdFlag() < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("bypass cache unsupported by this system")); + return -1; + } + oflags |= O_DIRECT; + } + if (virQEMUSaveFdInit(&saveFd, path, 0, oflags, cfg) < 0) + return -1; + + if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; if (ensureACL(conn, def) < 0) @@ -5898,16 +5907,13 @@ qemuDomainRestoreInternal(virConnectPtr conn, flags) < 0) goto cleanup; - ret = qemuSaveImageStartVM(conn, driver, vm, &fd, data, path, + ret = qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, false, reset_nvram, VIR_ASYNC_JOB_START); qemuProcessEndJob(vm); cleanup: - VIR_FORCE_CLOSE(fd); - if (virFileWrapperFdClose(wrapperFd) < 0) - ret = -1; - virFileWrapperFdFree(wrapperFd); + ret = virQEMUSaveFdFini(&saveFd, vm, ret); virQEMUSaveDataFree(data); if (vm && ret < 0) qemuDomainRemoveInactive(driver, vm); @@ -5969,15 +5975,16 @@ qemuDomainSaveImageGetXMLDesc(virConnectPtr conn, const char *path, virQEMUDriver *driver = conn->privateData; char *ret = NULL; g_autoptr(virDomainDef) def = NULL; - int fd = -1; virQEMUSaveData *data = NULL; + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + virQEMUSaveFd saveFd = QEMU_SAVEFD_INVALID; virCheckFlags(VIR_DOMAIN_SAVE_IMAGE_XML_SECURE, NULL); - fd = qemuSaveImageOpen(driver, NULL, path, &def, &data, - false, NULL, false, false); + if (virQEMUSaveFdInit(&saveFd, path, 0, O_RDONLY, cfg) < 0) + return NULL; - if (fd < 0) + if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; if (virDomainSaveImageGetXMLDescEnsureACL(conn, def) < 0) @@ -5987,7 +5994,8 @@ qemuDomainSaveImageGetXMLDesc(virConnectPtr conn, const char *path, cleanup: virQEMUSaveDataFree(data); - VIR_FORCE_CLOSE(fd); + if (virQEMUSaveFdFini(&saveFd, NULL, ret ? 0 : -1) < 0) + ret = NULL; return ret; } @@ -5999,22 +6007,23 @@ qemuDomainSaveImageDefineXML(virConnectPtr conn, const char *path, int ret = -1; g_autoptr(virDomainDef) def = NULL; g_autoptr(virDomainDef) newdef = NULL; - int fd = -1; virQEMUSaveData *data = NULL; + virQEMUSaveFd saveFd = QEMU_SAVEFD_INVALID; + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); int state = -1; virCheckFlags(VIR_DOMAIN_SAVE_RUNNING | VIR_DOMAIN_SAVE_PAUSED, -1); + if (virQEMUSaveFdInit(&saveFd, path, 0, O_RDWR, cfg) < 0) + return -1; + if (flags & VIR_DOMAIN_SAVE_RUNNING) state = 1; else if (flags & VIR_DOMAIN_SAVE_PAUSED) state = 0; - fd = qemuSaveImageOpen(driver, NULL, path, &def, &data, - false, NULL, true, false); - - if (fd < 0) + if (qemuSaveImageOpen(driver, NULL, &def, &data, false, &saveFd) < 0) goto cleanup; if (virDomainSaveImageDefineXMLEnsureACL(conn, def) < 0) @@ -6041,15 +6050,15 @@ qemuDomainSaveImageDefineXML(virConnectPtr conn, const char *path, VIR_DOMAIN_XML_MIGRATABLE))) goto cleanup; - if (lseek(fd, 0, SEEK_SET) != 0) { + if (lseek(saveFd.fd, 0, SEEK_SET) != 0) { virReportSystemError(errno, _("cannot seek in '%s'"), path); goto cleanup; } - if (virQEMUSaveDataWrite(data, fd, path) < 0) + if (virQEMUSaveDataWrite(data, saveFd.fd, path) < 0) goto cleanup; - if (VIR_CLOSE(fd) < 0) { + if (virQEMUSaveFdClose(&saveFd, NULL) < 0) { virReportSystemError(errno, _("failed to write header data to '%s'"), path); goto cleanup; } @@ -6057,8 +6066,8 @@ qemuDomainSaveImageDefineXML(virConnectPtr conn, const char *path, ret = 0; cleanup: - VIR_FORCE_CLOSE(fd); virQEMUSaveDataFree(data); + ret = virQEMUSaveFdFini(&saveFd, NULL, ret); return ret; } @@ -6070,8 +6079,9 @@ qemuDomainManagedSaveGetXMLDesc(virDomainPtr dom, unsigned int flags) g_autofree char *path = NULL; char *ret = NULL; g_autoptr(virDomainDef) def = NULL; - int fd = -1; virQEMUSaveData *data = NULL; + virQEMUSaveFd saveFd = QEMU_SAVEFD_INVALID; + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); qemuDomainObjPrivate *priv; virCheckFlags(VIR_DOMAIN_SAVE_IMAGE_XML_SECURE, NULL); @@ -6093,15 +6103,19 @@ qemuDomainManagedSaveGetXMLDesc(virDomainPtr dom, unsigned int flags) goto cleanup; } - if ((fd = qemuSaveImageOpen(driver, priv->qemuCaps, path, &def, &data, - false, NULL, false, false)) < 0) + if (virQEMUSaveFdInit(&saveFd, path, 0, O_RDONLY, cfg) < 0) + goto cleanup; + + if (qemuSaveImageOpen(driver, priv->qemuCaps, &def, &data, false, + &saveFd) < 0) goto cleanup; ret = qemuDomainDefFormatXML(driver, priv->qemuCaps, def, flags); cleanup: virQEMUSaveDataFree(data); - VIR_FORCE_CLOSE(fd); + if (virQEMUSaveFdFini(&saveFd, vm, ret ? 0 : -1) < 0) + ret = NULL; virDomainObjEndAPI(&vm); return ret; } @@ -6152,16 +6166,25 @@ qemuDomainObjRestore(virConnectPtr conn, { g_autoptr(virDomainDef) def = NULL; qemuDomainObjPrivate *priv = vm->privateData; - int fd = -1; int ret = -1; g_autofree char *xmlout = NULL; virQEMUSaveData *data = NULL; - virFileWrapperFd *wrapperFd = NULL; + virQEMUSaveFd saveFd = QEMU_SAVEFD_INVALID; + int oflags = O_RDONLY; + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); - fd = qemuSaveImageOpen(driver, NULL, path, &def, &data, - bypass_cache, &wrapperFd, false, true); - if (fd < 0) { - if (fd == -3) + if (bypass_cache) { + if (virFileDirectFdFlag() < 0) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("bypass cache unsupported by this system")); + return -1; + } + oflags |= O_DIRECT; + } + + ret = qemuSaveImageOpen(driver, NULL, &def, &data, true, &saveFd); + if (ret < 0) { + if (ret == -3) ret = 1; goto cleanup; } @@ -6205,15 +6228,12 @@ qemuDomainObjRestore(virConnectPtr conn, virDomainObjAssignDef(vm, &def, true, NULL); - ret = qemuSaveImageStartVM(conn, driver, vm, &fd, data, path, + ret = qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, start_paused, reset_nvram, asyncJob); cleanup: virQEMUSaveDataFree(data); - VIR_FORCE_CLOSE(fd); - if (virFileWrapperFdClose(wrapperFd) < 0) - ret = -1; - virFileWrapperFdFree(wrapperFd); + ret = virQEMUSaveFdFini(&saveFd, vm, ret); return ret; } diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 7c76db359e..5a569fa52e 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -248,6 +248,123 @@ qemuSaveImageGetCompressionCommand(virQEMUSaveFormat compression) return ret; } +/* + * virQEMUSaveFdInit: initialize a virQEMUSaveFd + * + * @saveFd: the structure to initialize + * @base: the main file name + * @idx: 0 for the main file, >0 for multifd channels. + * @oflags the file descriptor open flags + * @cfg: the driver config + * + * Returns -1 on error, 0 on success, + * and in both cases virQEMUSaveFdFini must be called to free resources. + */ +int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const char *base, int idx, + int oflags, virQEMUDriverConfig *cfg) +{ + unsigned int wrapperFlags = VIR_FILE_WRAPPER_NON_BLOCKING; + bool isCreat = oflags & O_CREAT; + bool isDirect = O_DIRECT && (oflags & O_DIRECT); + + if (isDirect) + wrapperFlags |= VIR_FILE_WRAPPER_BYPASS_CACHE; + if (idx > 0) { + saveFd->path = g_strdup_printf("%s.%d", base, idx); + } else { + saveFd->path = g_strdup(base); + } + saveFd->wrapper = NULL; + if (isCreat) { + saveFd->fd = virQEMUFileOpenAs(cfg->user, cfg->group, false, saveFd->path, + oflags, &saveFd->need_unlink); + } else { + saveFd->fd = qemuDomainOpenFile(cfg, NULL, saveFd->path, oflags, NULL); + } + if (saveFd->fd < 0) + return -1; + /* + * no wrapper required for the multifd channels. + * For O_CREAT, we always add the wrapper for the main file. + * For !O_CREAT, we only add the wrapper if using O_DIRECT. + */ + if (idx == 0 && (isDirect || isCreat)) { + saveFd->wrapper = virFileWrapperFdNew(&saveFd->fd, saveFd->path, wrapperFlags); + if (!saveFd->wrapper) + return -1; + } + return 0; +} + +/* + * virQEMUSaveFdClose: close a virQEMUSaveFd descriptor with normal close. + * + * @saveFd: the saveFd structure with the file descriptors to close. + * @vm: the virDomainObj (necessary to release lock), or NULL. + * + * If saveFd is NULL, the function will return success. + * + * Returns -1 on error, 0 on success. + */ +int virQEMUSaveFdClose(virQEMUSaveFd *saveFd, virDomainObj *vm) +{ + if (!saveFd) + return 0; + + if (VIR_CLOSE(saveFd->fd) < 0) { + virReportSystemError(errno, _("unable to close %s"), saveFd->path); + return -1; + } + if (vm) { + if (qemuDomainFileWrapperFDClose(vm, saveFd->wrapper) < 0) + return -1; + } else { + if (virFileWrapperFdClose(saveFd->wrapper) < 0) + return -1; + } + return 0; +} + +/* + * virQEMUSaveFdFini: finalize a virQEMUSaveFd + * + * @saveFd: the saveFd structure containing the resources to free. + * @vm: the virDomainObj (necessary to release lock for long close ops), or NULL. + * @ret: the current operation result (< 0 is failure) + * + * If saveFd is NULL, the return value will be unchanged. + * + * Returns ret, or -1 if an error is detected. + */ +int virQEMUSaveFdFini(virQEMUSaveFd *saveFd, virDomainObj *vm, int ret) +{ + if (!saveFd) + return ret; + VIR_FORCE_CLOSE(saveFd->fd); + if (vm) { + if (qemuDomainFileWrapperFDClose(vm, saveFd->wrapper) < 0) + ret = -1; + } else { + if (virFileWrapperFdClose(saveFd->wrapper) < 0) + ret = -1; + } + + if (ret < 0 && saveFd->need_unlink && saveFd->path) { + if (unlink(saveFd->path) < 0) { + virReportSystemError(errno, _("cannot remove file: %s"), + saveFd->path); + } + } + if (saveFd->wrapper) { + virFileWrapperFdFree(saveFd->wrapper); + saveFd->wrapper = NULL; + } + + g_free(saveFd->path); + saveFd->path = NULL; + return ret; +} + /* Helper function to execute a migration to file with a correct save header * the caller needs to make sure that the processors are stopped and do all other @@ -263,41 +380,33 @@ qemuSaveImageCreate(virQEMUDriver *driver, virDomainAsyncJob asyncJob) { g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); - bool needUnlink = false; + virQEMUSaveFd saveFd = QEMU_SAVEFD_INVALID; + unsigned int oflags = O_WRONLY | O_TRUNC | O_CREAT; int ret = -1; - int fd = -1; - int directFlag = 0; - virFileWrapperFd *wrapperFd = NULL; - unsigned int wrapperFlags = VIR_FILE_WRAPPER_NON_BLOCKING; - /* Obtain the file handle. */ - if ((flags & VIR_DOMAIN_SAVE_BYPASS_CACHE)) { - wrapperFlags |= VIR_FILE_WRAPPER_BYPASS_CACHE; - directFlag = virFileDirectFdFlag(); - if (directFlag < 0) { + if (flags & VIR_DOMAIN_SAVE_BYPASS_CACHE) { + if (virFileDirectFdFlag() < 0) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("bypass cache unsupported by this system")); - goto cleanup; + return -1; } + oflags |= O_DIRECT; } - fd = virQEMUFileOpenAs(cfg->user, cfg->group, false, path, - O_WRONLY | O_TRUNC | O_CREAT | directFlag, - &needUnlink); - if (fd < 0) + if (virQEMUSaveFdInit(&saveFd, path, 0, oflags, cfg) < 0) { goto cleanup; - - if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, fd) < 0) + } + if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, saveFd.fd) < 0) goto cleanup; - if (!(wrapperFd = virFileWrapperFdNew(&fd, path, wrapperFlags))) + if (virQEMUSaveDataWrite(data, saveFd.fd, saveFd.path) < 0) goto cleanup; - if (virQEMUSaveDataWrite(data, fd, path) < 0) - goto cleanup; /* Perform the migration */ - if (qemuMigrationSrcToFile(driver, vm, fd, compressor, asyncJob) < 0) + if (qemuMigrationSrcToFile(driver, vm, saveFd.fd, compressor, asyncJob) < 0) + goto cleanup; + if (virQEMUSaveFdClose(&saveFd, vm) < 0) goto cleanup; /* Touch up file header to mark image complete. */ @@ -306,29 +415,17 @@ qemuSaveImageCreate(virQEMUDriver *driver, * up to seek backwards on wrapperFd. The reopened fd will * trigger a single page of file system cache pollution, but * that's acceptable. */ - if (VIR_CLOSE(fd) < 0) { - virReportSystemError(errno, _("unable to close %s"), path); - goto cleanup; - } - - if (qemuDomainFileWrapperFDClose(vm, wrapperFd) < 0) - goto cleanup; - if ((fd = qemuDomainOpenFile(cfg, vm->def, path, O_WRONLY, NULL)) < 0 || - virQEMUSaveDataFinish(data, &fd, path) < 0) + if ((saveFd.fd = qemuDomainOpenFile(cfg, vm->def, saveFd.path, O_WRONLY, NULL)) < 0 || + virQEMUSaveDataFinish(data, &saveFd.fd, saveFd.path) < 0) goto cleanup; ret = 0; - cleanup: - VIR_FORCE_CLOSE(fd); - if (qemuDomainFileWrapperFDClose(vm, wrapperFd) < 0) - ret = -1; - virFileWrapperFdFree(wrapperFd); - if (ret < 0 && needUnlink) - unlink(path); + cleanup: + ret = virQEMUSaveFdFini(&saveFd, vm, ret); return ret; } @@ -421,94 +518,49 @@ qemuSaveImageGetCompressionProgram(const char *imageFormat, * @path: path of the save image * @ret_def: returns domain definition created from the XML stored in the image * @ret_data: returns structure filled with data from the image header - * @bypass_cache: bypass cache when opening the file - * @wrapperFd: returns the file wrapper structure - * @open_write: open the file for writing (for updates) - * @unlink_corrupt: remove the image file if it is corrupted + * @unlink_corrupt: mark the image file for removal if it is corrupted + * @saveFd: the save file * - * Returns the opened fd of the save image file and fills the appropriate fields - * on success. On error returns -1 on most failures, -3 if corrupt image was - * unlinked (no error raised). + * Returns 0 on success or -1 on failure. + * On success, the appropriate fields are filled. */ int qemuSaveImageOpen(virQEMUDriver *driver, virQEMUCaps *qemuCaps, - const char *path, virDomainDef **ret_def, virQEMUSaveData **ret_data, - bool bypass_cache, - virFileWrapperFd **wrapperFd, - bool open_write, - bool unlink_corrupt) + bool unlink_corrupt, + virQEMUSaveFd *saveFd) { - g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); - VIR_AUTOCLOSE fd = -1; - int ret = -1; g_autoptr(virQEMUSaveData) data = NULL; virQEMUSaveHeader *header; g_autoptr(virDomainDef) def = NULL; - int oflags = open_write ? O_RDWR : O_RDONLY; size_t xml_len; size_t cookie_len; - if (bypass_cache) { - int directFlag = virFileDirectFdFlag(); - if (directFlag < 0) { - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("bypass cache unsupported by this system")); - return -1; - } - oflags |= directFlag; - } - - if ((fd = qemuDomainOpenFile(cfg, NULL, path, oflags, NULL)) < 0) - return -1; - - if (bypass_cache && - !(*wrapperFd = virFileWrapperFdNew(&fd, path, - VIR_FILE_WRAPPER_BYPASS_CACHE))) - return -1; data = g_new0(virQEMUSaveData, 1); header = &data->header; - if (saferead(fd, header, sizeof(*header)) != sizeof(*header)) { - if (unlink_corrupt) { - if (unlink(path) < 0) { - virReportSystemError(errno, - _("cannot remove corrupt file: %s"), - path); - return -1; - } else { - return -3; - } - } - + if (saferead(saveFd->fd, header, sizeof(*header)) != sizeof(*header)) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("failed to read qemu header")); + if (unlink_corrupt) { + saveFd->need_unlink = true; + } return -1; } if (memcmp(header->magic, QEMU_SAVE_MAGIC, sizeof(header->magic)) != 0) { - if (memcmp(header->magic, QEMU_SAVE_PARTIAL, sizeof(header->magic)) == 0) { - if (unlink_corrupt) { - if (unlink(path) < 0) { - virReportSystemError(errno, - _("cannot remove corrupt file: %s"), - path); - return -1; - } else { - return -3; - } - } - + if (memcmp(header->magic, QEMU_SAVE_PARTIAL, sizeof(header->magic)) == 0) virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("save image is incomplete")); - return -1; + else + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("image magic is incorrect")); + if (unlink_corrupt) { + saveFd->need_unlink = true; } - - virReportError(VIR_ERR_OPERATION_FAILED, "%s", - _("image magic is incorrect")); return -1; } @@ -539,7 +591,7 @@ qemuSaveImageOpen(virQEMUDriver *driver, data->xml = g_new0(char, xml_len); - if (saferead(fd, data->xml, xml_len) != xml_len) { + if (saferead(saveFd->fd, data->xml, xml_len) != xml_len) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("failed to read domain XML")); return -1; @@ -548,7 +600,7 @@ qemuSaveImageOpen(virQEMUDriver *driver, if (cookie_len > 0) { data->cookie = g_new0(char, cookie_len); - if (saferead(fd, data->cookie, cookie_len) != cookie_len) { + if (saferead(saveFd->fd, data->cookie, cookie_len) != cookie_len) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("failed to read cookie")); return -1; @@ -564,10 +616,7 @@ qemuSaveImageOpen(virQEMUDriver *driver, *ret_def = g_steal_pointer(&def); *ret_data = g_steal_pointer(&data); - ret = fd; - fd = -1; - - return ret; + return 0; } int diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index b3d5c02fd6..5dc63f3661 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -54,6 +54,24 @@ struct _virQEMUSaveData { }; +typedef struct _virQEMUSaveFd virQEMUSaveFd; +struct _virQEMUSaveFd { + char *path; + int fd; + bool need_unlink; + virFileWrapperFd *wrapper; +}; + +#define QEMU_SAVEFD_INVALID (virQEMUSaveFd) { .path = NULL, .fd = -1, .need_unlink = false, .wrapper = NULL } + +int virQEMUSaveFdInit(virQEMUSaveFd *saveFd, const char *base, int idx, + int oflags, virQEMUDriverConfig *cfg) + ATTRIBUTE_NONNULL(5); + +int virQEMUSaveFdClose(virQEMUSaveFd *saveFd, virDomainObj *vm); + +int virQEMUSaveFdFini(virQEMUSaveFd *saveFd, virDomainObj *vm, int ret); + virDomainDef * qemuSaveImageUpdateDef(virQEMUDriver *driver, virDomainDef *def, @@ -74,14 +92,11 @@ qemuSaveImageStartVM(virConnectPtr conn, int qemuSaveImageOpen(virQEMUDriver *driver, virQEMUCaps *qemuCaps, - const char *path, virDomainDef **ret_def, virQEMUSaveData **ret_data, - bool bypass_cache, - virFileWrapperFd **wrapperFd, - bool open_write, - bool unlink_corrupt) - ATTRIBUTE_NONNULL(3) ATTRIBUTE_NONNULL(4); + bool unlink_corrupt, + virQEMUSaveFd *saveFd) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(3) ATTRIBUTE_NONNULL(6); int qemuSaveImageGetCompressionProgram(const char *imageFormat, -- 2.35.3
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_saveimage.c | 130 ++++++++++++++++++++++++++++++++++++-- src/qemu/qemu_saveimage.h | 11 ++++ 2 files changed, 137 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 5a569fa52e..65d9a3fef5 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -17,6 +17,7 @@ */ #include <config.h> +#include <configmake.h> #include "qemu_saveimage.h" #include "qemu_domain.h" @@ -365,6 +366,93 @@ int virQEMUSaveFdFini(virQEMUSaveFd *saveFd, virDomainObj *vm, int ret) return ret; } +/* + * qemuSaveImageFreeMultiFd: free all multifd virQEMUSaveFds. + * @multiFd: the array of saveFds + * @vm: the virDomainObj, to release lock + * @nconn: number of multifd channels + * @ret: the current operation result (< 0 is failure) + * + * If multiFd is NULL, the return value will be unchanged. + * + * Returns ret, or -1 if an error is detected. + */ +int qemuSaveImageFreeMultiFd(virQEMUSaveFd *multiFd, virDomainObj *vm, int nconn, int ret) +{ + int idx; + + if (!multiFd) + return ret; + + for (idx = 0; idx < nconn; idx++) { + ret = virQEMUSaveFdFini(&multiFd[idx], vm, ret); + } + /* + * do it again to unlink all in the error case, + * if error happened in the middle of previous loop. + */ + for (idx = 0; idx < nconn; idx++) { + ret = virQEMUSaveFdFini(&multiFd[idx], vm, ret); + } + g_free(multiFd); + return ret; +} + +/* + * qemuSaveImageCloseMultiFd: perform normal close on all multifd virQEMUSaveFds. + * If multiFd is NULL, the function will return success. + * + * Returns -1 on error, 0 on success. + */ + +int qemuSaveImageCloseMultiFd(virQEMUSaveFd *multiFd, int nconn, virDomainObj *vm) +{ + int idx; + + if (!multiFd) + return 0; + + for (idx = 0; idx < nconn; idx++) { + if (virQEMUSaveFdClose(&multiFd[idx], vm) < 0) { + return -1; + } + } + return 0; +} + +/* + * qemuSaveImageCreateMultiFd: allocate and initialize all multifd virQEMUSaveFds. + * + * Returns the new array of virQEMUSaveFds, or NULL on error. + */ + +virQEMUSaveFd * +qemuSaveImageCreateMultiFd(virQEMUDriver *driver, virDomainObj *vm, + virCommand *cmd, const char *path, + int oflags, virQEMUDriverConfig *cfg, + int nconn) +{ + virQEMUSaveFd *multiFd = g_new0(virQEMUSaveFd, nconn); + int idx; + + for (idx = 0; idx < nconn; idx++) { + virQEMUSaveFd *m = &multiFd[idx]; + if (virQEMUSaveFdInit(m, path, idx + 1, oflags, cfg) < 0 || + qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, m->fd) < 0) { + + virQEMUSaveFdFini(m, vm, -1); + goto error; + } + virCommandAddArgFormat(cmd, "%d", m->fd); + virCommandPassFD(cmd, m->fd, 0); + } + return multiFd; + + error: + qemuSaveImageFreeMultiFd(multiFd, vm, nconn, -1); + return NULL; +} + /* Helper function to execute a migration to file with a correct save header * the caller needs to make sure that the processors are stopped and do all other @@ -381,6 +469,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, { g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); virQEMUSaveFd saveFd = QEMU_SAVEFD_INVALID; + virQEMUSaveFd *multiFd = NULL; unsigned int oflags = O_WRONLY | O_TRUNC | O_CREAT; int ret = -1; @@ -402,10 +491,43 @@ qemuSaveImageCreate(virQEMUDriver *driver, if (virQEMUSaveDataWrite(data, saveFd.fd, saveFd.path) < 0) goto cleanup; + if (flags & VIR_DOMAIN_SAVE_PARALLEL) { + g_autoptr(virCommand) cmd = NULL; + g_autofree char *helper_path = NULL; + qemuDomainObjPrivate *priv = vm->privateData; + g_autofree char *sun_path = g_strdup_printf("%s/save-multifd.sock", priv->libDir); + char buf[1]; + int helper_out = -1; + if (!(helper_path = virFileFindResource("libvirt_multifd_helper", + abs_top_builddir "/src", + LIBEXECDIR))) + goto cleanup; + cmd = virCommandNewArgList(helper_path, sun_path, NULL); + virCommandAddArgFormat(cmd, "%d", nconn); + virCommandAddArgFormat(cmd, "%d", saveFd.fd); + virCommandPassFD(cmd, saveFd.fd, 0); + virCommandSetOutputFD(cmd, &helper_out); /* should create pipe automagically */ + + /* Perform parallel multifd migration to files (main fd + channels) */ + if (!(multiFd = qemuSaveImageCreateMultiFd(driver, vm, cmd, saveFd.path, oflags, cfg, nconn))) + goto cleanup; + if (virCommandRunAsync(cmd, NULL) < 0) + goto cleanup; + if (saferead(helper_out, &buf, 1) != 1 || buf[0] != 'R') + goto cleanup; + if (chown(sun_path, cfg->user, cfg->group) < 0) + goto cleanup; + /* still using single fd migration for now */ + if (qemuMigrationSrcToFile(driver, vm, saveFd.fd, compressor, asyncJob) < 0) + goto cleanup; + if (qemuSaveImageCloseMultiFd(multiFd, nconn, vm) < 0) + goto cleanup; + } else { + /* Perform non-parallel migration to file */ + if (qemuMigrationSrcToFile(driver, vm, saveFd.fd, compressor, asyncJob) < 0) + goto cleanup; + } - /* Perform the migration */ - if (qemuMigrationSrcToFile(driver, vm, saveFd.fd, compressor, asyncJob) < 0) - goto cleanup; if (virQEMUSaveFdClose(&saveFd, vm) < 0) goto cleanup; @@ -424,7 +546,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, cleanup: - + ret = qemuSaveImageFreeMultiFd(multiFd, vm, nconn, ret); ret = virQEMUSaveFdFini(&saveFd, vm, ret); return ret; } diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 5dc63f3661..b775c5eb08 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -72,6 +72,17 @@ int virQEMUSaveFdClose(virQEMUSaveFd *saveFd, virDomainObj *vm); int virQEMUSaveFdFini(virQEMUSaveFd *saveFd, virDomainObj *vm, int ret); +virQEMUSaveFd * +qemuSaveImageCreateMultiFd(virQEMUDriver *driver, virDomainObj *vm, + virCommand *cmd, const char *path, + int oflags, virQEMUDriverConfig *cfg, + int nconn) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3) ATTRIBUTE_NONNULL(4) ATTRIBUTE_NONNULL(6); + +int qemuSaveImageCloseMultiFd(virQEMUSaveFd *multiFd, int nconn, virDomainObj *vm); + +int qemuSaveImageFreeMultiFd(virQEMUSaveFd *multiFd, virDomainObj *vm, int nconn, int ret); + virDomainDef * qemuSaveImageUpdateDef(virQEMUDriver *driver, virDomainDef *def, -- 2.35.3
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_capabilities.c | 4 ++++ src/qemu/qemu_capabilities.h | 3 +++ tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.riscv32.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.riscv64.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_4.1.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_4.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_4.2.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml | 1 + tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml | 1 + tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml | 1 + 34 files changed, 39 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 1ed4cda7f0..581b6a40df 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -672,6 +672,9 @@ VIR_ENUM_IMPL(virQEMUCaps, "virtio-iommu-pci", /* QEMU_CAPS_DEVICE_VIRTIO_IOMMU_PCI */ "virtio-iommu.boot-bypass", /* QEMU_CAPS_VIRTIO_IOMMU_BOOT_BYPASS */ "virtio-net.rss", /* QEMU_CAPS_VIRTIO_NET_RSS */ + + /* 430 */ + "migrate-multifd", /* QEMU_CAPS_MIGRATE_MULTIFD */ ); @@ -1230,6 +1233,7 @@ struct virQEMUCapsStringFlags virQEMUCapsCommands[] = { struct virQEMUCapsStringFlags virQEMUCapsMigration[] = { { "rdma-pin-all", QEMU_CAPS_MIGRATE_RDMA }, + { "multifd", QEMU_CAPS_MIGRATE_MULTIFD }, }; /* Use virQEMUCapsQMPSchemaQueries for querying parameters of events */ diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index 9b240e47fb..b089f83da1 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -648,6 +648,9 @@ typedef enum { /* virQEMUCapsFlags grouping marker for syntax-check */ QEMU_CAPS_VIRTIO_IOMMU_BOOT_BYPASS, /* virtio-iommu.boot-bypass */ QEMU_CAPS_VIRTIO_NET_RSS, /* virtio-net rss feature */ + /* 430 */ + QEMU_CAPS_MIGRATE_MULTIFD, /* migrate can set multifd parameter */ + QEMU_CAPS_LAST /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml index 5adf904fc4..4ca2cfa81c 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml @@ -148,6 +148,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700240</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml index a84adc2610..1db978eb4c 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml @@ -153,6 +153,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900240</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.riscv32.xml b/tests/qemucapabilitiesdata/caps_4.0.0.riscv32.xml index c494254c4d..251d4dfd29 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.riscv32.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.riscv32.xml @@ -145,6 +145,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>0</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.riscv64.xml b/tests/qemucapabilitiesdata/caps_4.0.0.riscv64.xml index d2582fa297..a4af47c6a4 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.riscv64.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.riscv64.xml @@ -145,6 +145,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>0</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.s390x.xml b/tests/qemucapabilitiesdata/caps_4.0.0.s390x.xml index 4f36186044..2bab764867 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.s390x.xml @@ -115,6 +115,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>39100240</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_4.0.0.x86_64.xml index 18e5ebd4f4..aa8a9812e5 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.x86_64.xml @@ -188,6 +188,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100240</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_4.1.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_4.1.0.x86_64.xml index 12c5ebe6f3..bd89f0c6b2 100644 --- a/tests/qemucapabilitiesdata/caps_4.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_4.1.0.x86_64.xml @@ -195,6 +195,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4001000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100241</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_4.2.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_4.2.0.aarch64.xml index ee536b7b63..369ef707b9 100644 --- a/tests/qemucapabilitiesdata/caps_4.2.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_4.2.0.aarch64.xml @@ -163,6 +163,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4001050</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml index 10f5a9e2c5..16c867a46b 100644 --- a/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml @@ -160,6 +160,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4001050</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_4.2.0.s390x.xml b/tests/qemucapabilitiesdata/caps_4.2.0.s390x.xml index 069777a49b..b584ba7352 100644 --- a/tests/qemucapabilitiesdata/caps_4.2.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_4.2.0.s390x.xml @@ -128,6 +128,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>39100242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_4.2.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_4.2.0.x86_64.xml index 6b61214a0b..5023028678 100644 --- a/tests/qemucapabilitiesdata/caps_4.2.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_4.2.0.x86_64.xml @@ -206,6 +206,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='virtio-blk.queue-size'/> + <flag name='migrate-multifd'/> <version>4002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml index 4fd02e786d..c45b2e6cf6 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml @@ -175,6 +175,7 @@ <flag name='virtio-blk.queue-size'/> <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> + <flag name='migrate-multifd'/> <version>5000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700241</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml index f2f3558fdc..a3ad743d70 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml @@ -181,6 +181,7 @@ <flag name='virtio-blk.queue-size'/> <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> + <flag name='migrate-multifd'/> <version>5000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900241</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml b/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml index 557949d6d6..e1b5cac26b 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml @@ -167,6 +167,7 @@ <flag name='virtio-blk.queue-size'/> <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> + <flag name='migrate-multifd'/> <version>5000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>0</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml index f301d8a926..796adb9066 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml @@ -215,6 +215,7 @@ <flag name='virtio-blk.queue-size'/> <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> + <flag name='migrate-multifd'/> <version>5000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100241</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml b/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml index 3a330ebdc0..cb203df125 100644 --- a/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml +++ b/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml @@ -87,6 +87,7 @@ <flag name='input-linux'/> <flag name='query-display-options'/> <flag name='memory-backend-file.prealloc-threads'/> + <flag name='migrate-multifd'/> <version>5001000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>0</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml index 53fcbf3417..7479d942a2 100644 --- a/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml @@ -219,6 +219,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>5001000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml index 824224302c..268d1444ad 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml @@ -182,6 +182,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>5002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml index b949f88b5a..eabf4b600c 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml @@ -186,6 +186,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>5002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml b/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml index 873923992d..0dbaf5a5ec 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml @@ -172,6 +172,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>5002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>0</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml b/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml index 5e9560d7b7..b0fbab9cb5 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml @@ -140,6 +140,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>5002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>39100243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml index 3998da9253..1a1717bf2a 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml @@ -223,6 +223,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>5002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml index 51d3628eeb..1c18d122e2 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml @@ -190,6 +190,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>6000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml b/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml index 2e5d0f197a..8fa4cb2307 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml @@ -148,6 +148,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>6000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>39100242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml index 3498d6255b..70c67202b1 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml @@ -232,6 +232,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>6000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml index ddeca62290..a5ec77878f 100644 --- a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml @@ -236,6 +236,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>6001000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml index 5538940372..92d8ceff7e 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml @@ -201,6 +201,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>6001050</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700244</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml index 9c9d9aa08e..f219912927 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml @@ -197,6 +197,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>6002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900244</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml index dba5ecaf87..38fd3878ea 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml @@ -238,6 +238,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>6002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100244</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml index 257b0f625d..522e225c8f 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml @@ -209,6 +209,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-iommu.boot-bypass'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>6002092</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml index 1ddca7d767..1eb43799c0 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml @@ -210,6 +210,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-iommu.boot-bypass'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>6002092</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml index 8074c97ecd..e5023c4219 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml @@ -242,6 +242,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-iommu.boot-bypass'/> <flag name='virtio-net.rss'/> + <flag name='migrate-multifd'/> <version>7000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100243</microcodeVersion> -- 2.35.3
implement a function similar to qemuMigrationSrcToFile that migrates to multiple files using QEMU multifd. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_migration.c | 129 ++++++++++++++++++++----------- src/qemu/qemu_migration.h | 7 ++ src/qemu/qemu_migration_params.c | 22 ++++++ src/qemu/qemu_migration_params.h | 9 +++ src/qemu/qemu_saveimage.c | 5 +- 5 files changed, 124 insertions(+), 48 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index b735bdb391..371db6a41c 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -5896,13 +5896,14 @@ qemuMigrationDstFinish(virQEMUDriver *driver, return dom; } - /* Helper function called while vm is active. */ -int -qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, - int fd, - virCommand *compressor, - virDomainAsyncJob asyncJob) +static int +qemuMigrationSrcToFileAux(virQEMUDriver *driver, virDomainObj *vm, + int fd, + virCommand *compressor, + virDomainAsyncJob asyncJob, + const char *sun_path, + int nchannels) { qemuDomainObjPrivate *priv = vm->privateData; bool bwParam = virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATION_PARAM_BANDWIDTH); @@ -5913,24 +5914,24 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, char *errbuf = NULL; virErrorPtr orig_err = NULL; g_autoptr(qemuMigrationParams) migParams = NULL; + bool needParams = (bwParam || sun_path); if (qemuMigrationSetDBusVMState(driver, vm) < 0) return -1; + if (sun_path && !virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATE_MULTIFD)) + return -1; + /* Increase migration bandwidth to unlimited since target is a file. * Failure to change migration speed is not fatal. */ - if (bwParam) { - if (!(migParams = qemuMigrationParamsNew())) - return -1; + if (needParams && !((migParams = qemuMigrationParamsNew()))) + return -1; + if (bwParam) { if (qemuMigrationParamsSetULL(migParams, QEMU_MIGRATION_PARAM_MAX_BANDWIDTH, QEMU_DOMAIN_MIG_BANDWIDTH_MAX * 1024 * 1024) < 0) return -1; - - if (qemuMigrationParamsApply(driver, vm, asyncJob, migParams) < 0) - return -1; - priv->migMaxBandwidth = QEMU_DOMAIN_MIG_BANDWIDTH_MAX; } else { if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) == 0) { @@ -5941,6 +5942,17 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, } } + if (sun_path) { + qemuMigrationParamsSetCap(migParams, QEMU_MIGRATION_CAP_MULTIFD); + if (qemuMigrationParamsSetInt(migParams, + QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS, + nchannels) < 0) + return -1; + } + + if (needParams && qemuMigrationParamsApply(driver, vm, asyncJob, migParams) < 0) + return -1; + if (!virDomainObjIsActive(vm)) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("guest unexpectedly quit")); @@ -5948,45 +5960,53 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, return -1; } - if (compressor && virPipe(pipeFD) < 0) + if (!sun_path && compressor && virPipe(pipeFD) < 0) return -1; - /* All right! We can use fd migration, which means that qemu - * doesn't have to open() the file, so while we still have to - * grant SELinux access, we can do it on fd and avoid cleanup - * later, as well as skip futzing with cgroup. */ - if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, - compressor ? pipeFD[1] : fd) < 0) - goto cleanup; - if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0) goto cleanup; - if (!compressor) { - rc = qemuMonitorMigrateToFd(priv->mon, - QEMU_MONITOR_MIGRATE_BACKGROUND, - fd); + if (sun_path) { + rc = qemuMonitorMigrateToSocket(priv->mon, + QEMU_MONITOR_MIGRATE_BACKGROUND, + sun_path); } else { - virCommandSetInputFD(compressor, pipeFD[0]); - virCommandSetOutputFD(compressor, &fd); - virCommandSetErrorBuffer(compressor, &errbuf); - virCommandDoAsyncIO(compressor); - if (virSetCloseExec(pipeFD[1]) < 0) { - virReportSystemError(errno, "%s", - _("Unable to set cloexec flag")); - qemuDomainObjExitMonitor(vm); - goto cleanup; - } - if (virCommandRunAsync(compressor, NULL) < 0) { - qemuDomainObjExitMonitor(vm); + /* + * All right! We can use fd migration, which means that qemu + * doesn't have to open() the file, so while we still have to + * grant SELinux access, we can do it on fd and avoid cleanup + * later, as well as skip futzing with cgroup. + */ + if (qemuSecuritySetImageFDLabel(driver->securityManager, vm->def, + compressor ? pipeFD[1] : fd) < 0) goto cleanup; + + if (!compressor) { + rc = qemuMonitorMigrateToFd(priv->mon, + QEMU_MONITOR_MIGRATE_BACKGROUND, + fd); + } else { + virCommandSetInputFD(compressor, pipeFD[0]); + virCommandSetOutputFD(compressor, &fd); + virCommandSetErrorBuffer(compressor, &errbuf); + virCommandDoAsyncIO(compressor); + if (virSetCloseExec(pipeFD[1]) < 0) { + virReportSystemError(errno, "%s", + _("Unable to set cloexec flag")); + qemuDomainObjExitMonitor(vm); + goto cleanup; + } + if (virCommandRunAsync(compressor, NULL) < 0) { + qemuDomainObjExitMonitor(vm); + goto cleanup; + } + rc = qemuMonitorMigrateToFd(priv->mon, + QEMU_MONITOR_MIGRATE_BACKGROUND, + pipeFD[1]); + if (VIR_CLOSE(pipeFD[0]) < 0 || + VIR_CLOSE(pipeFD[1]) < 0) + VIR_WARN("failed to close intermediate pipe"); } - rc = qemuMonitorMigrateToFd(priv->mon, - QEMU_MONITOR_MIGRATE_BACKGROUND, - pipeFD[1]); - if (VIR_CLOSE(pipeFD[0]) < 0 || - VIR_CLOSE(pipeFD[1]) < 0) - VIR_WARN("failed to close intermediate pipe"); } qemuDomainObjExitMonitor(vm); if (rc < 0) @@ -6007,7 +6027,7 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, goto cleanup; } - if (compressor && virCommandWait(compressor, NULL) < 0) + if (!sun_path && compressor && virCommandWait(compressor, NULL) < 0) goto cleanup; qemuDomainEventEmitJobCompleted(driver, vm); @@ -6046,6 +6066,25 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, return ret; } +int +qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainObj *vm, + int fd, + virCommand *compressor, + virDomainAsyncJob asyncJob) +{ + return qemuMigrationSrcToFileAux(driver, vm, fd, compressor, + asyncJob, NULL, -1); +} + +int +qemuMigrationSrcToFilesMultiFd(virQEMUDriver *driver, virDomainObj *vm, + virDomainAsyncJob asyncJob, + const char *sun_path, + int nchannels) +{ + return qemuMigrationSrcToFileAux(driver, vm, -1, NULL, + asyncJob, sun_path, nchannels); +} int qemuMigrationSrcCancel(virQEMUDriver *driver, diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index a8afa66119..ddc8e65489 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -213,6 +213,13 @@ qemuMigrationSrcToFile(virQEMUDriver *driver, virDomainAsyncJob asyncJob) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; +int +qemuMigrationSrcToFilesMultiFd(virQEMUDriver *driver, virDomainObj *vm, + virDomainAsyncJob asyncJob, + const char *sun_path, + int nchannels) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; + int qemuMigrationSrcCancel(virQEMUDriver *driver, virDomainObj *vm); diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index df2384b213..36174a66d8 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -1109,6 +1109,28 @@ qemuMigrationParamsFetch(virQEMUDriver *driver, } +void +qemuMigrationParamsSetCap(qemuMigrationParams *migParams, + virQEMUCapsFlags flag) +{ + ignore_value(virBitmapSetBit(migParams->caps, flag)); +} + + +int +qemuMigrationParamsSetInt(qemuMigrationParams *migParams, + qemuMigrationParam param, + int value) +{ + if (qemuMigrationParamsCheckType(param, QEMU_MIGRATION_PARAM_TYPE_INT) < 0) + return -1; + + migParams->params[param].value.i = value; + migParams->params[param].set = true; + return 0; +} + + int qemuMigrationParamsSetULL(qemuMigrationParams *migParams, qemuMigrationParam param, diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index 4a8815e776..99af73b4a4 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -123,6 +123,15 @@ qemuMigrationParamsFetch(virQEMUDriver *driver, int asyncJob, qemuMigrationParams **migParams); +void +qemuMigrationParamsSetCap(qemuMigrationParams *migParams, + virQEMUCapsFlags flag); + +int +qemuMigrationParamsSetInt(qemuMigrationParams *migParams, + qemuMigrationParam param, + int value); + int qemuMigrationParamsSetULL(qemuMigrationParams *migParams, qemuMigrationParam param, diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 65d9a3fef5..d8b6635511 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -463,7 +463,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, const char *path, virQEMUSaveData *data, virCommand *compressor, - int nconn G_GNUC_UNUSED, + int nconn, unsigned int flags, virDomainAsyncJob asyncJob) { @@ -517,8 +517,7 @@ qemuSaveImageCreate(virQEMUDriver *driver, goto cleanup; if (chown(sun_path, cfg->user, cfg->group) < 0) goto cleanup; - /* still using single fd migration for now */ - if (qemuMigrationSrcToFile(driver, vm, saveFd.fd, compressor, asyncJob) < 0) + if (qemuMigrationSrcToFilesMultiFd(driver, vm, asyncJob, sun_path, nconn) < 0) goto cleanup; if (qemuSaveImageCloseMultiFd(multiFd, nconn, vm) < 0) goto cleanup; -- 2.35.3
The distinction on whether to wait for the migration completion or not was made on the async job type, but with the future addition of multifd migration from files, we need a way to avoid waiting, so we can prepare multifd migration parameters before starting the transfers. Adapt all callers. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_driver.c | 8 ++++---- src/qemu/qemu_migration.c | 18 ++++++++++-------- src/qemu/qemu_migration.h | 3 ++- src/qemu/qemu_process.c | 3 ++- src/qemu/qemu_process.h | 5 +++-- src/qemu/qemu_saveimage.c | 4 +++- src/qemu/qemu_saveimage.h | 1 + src/qemu/qemu_snapshot.c | 4 ++-- 8 files changed, 27 insertions(+), 19 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index e4cdd93b50..480f9b4ed9 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -1630,7 +1630,7 @@ static virDomainPtr qemuDomainCreateXML(virConnectPtr conn, } if (qemuProcessStart(conn, driver, vm, NULL, VIR_ASYNC_JOB_START, - NULL, -1, NULL, NULL, + NULL, -1, NULL, false, NULL, VIR_NETDEV_VPORT_PROFILE_OP_CREATE, start_flags) < 0) { virDomainAuditStart(vm, "booted", false); @@ -5908,7 +5908,7 @@ qemuDomainRestoreInternal(virConnectPtr conn, goto cleanup; ret = qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, - false, reset_nvram, VIR_ASYNC_JOB_START); + false, reset_nvram, true, VIR_ASYNC_JOB_START); qemuProcessEndJob(vm); @@ -6229,7 +6229,7 @@ qemuDomainObjRestore(virConnectPtr conn, virDomainObjAssignDef(vm, &def, true, NULL); ret = qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, - start_paused, reset_nvram, asyncJob); + start_paused, reset_nvram, true, asyncJob); cleanup: virQEMUSaveDataFree(data); @@ -6492,7 +6492,7 @@ qemuDomainObjStart(virConnectPtr conn, } ret = qemuProcessStart(conn, driver, vm, NULL, asyncJob, - NULL, -1, NULL, NULL, + NULL, -1, NULL, false, NULL, VIR_NETDEV_VPORT_PROFILE_OP_CREATE, start_flags); virDomainAuditStart(vm, "booted", ret >= 0); if (ret >= 0) { diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 371db6a41c..542428ab8e 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2139,7 +2139,8 @@ int qemuMigrationDstRun(virQEMUDriver *driver, virDomainObj *vm, const char *uri, - virDomainAsyncJob asyncJob) + virDomainAsyncJob asyncJob, + bool wait) { qemuDomainObjPrivate *priv = vm->privateData; int rv; @@ -2160,14 +2161,15 @@ qemuMigrationDstRun(virQEMUDriver *driver, if (rv < 0) return -1; - if (asyncJob == VIR_ASYNC_JOB_MIGRATION_IN) { - /* qemuMigrationDstWaitForCompletion is called from the Finish phase */ - return 0; + if (wait) { + /* + * the Migration Finish phase, as well as the multifd load from files, + * need to call qemuMigrationDstWaitForCompletion separately, not here. + */ + if (qemuMigrationDstWaitForCompletion(driver, vm, asyncJob, false) < 0) + return -1; } - if (qemuMigrationDstWaitForCompletion(driver, vm, asyncJob, false) < 0) - return -1; - return 0; } @@ -3041,7 +3043,7 @@ qemuMigrationDstPrepareAny(virQEMUDriver *driver, } if (qemuMigrationDstRun(driver, vm, incoming->uri, - VIR_ASYNC_JOB_MIGRATION_IN) < 0) + VIR_ASYNC_JOB_MIGRATION_IN, false) < 0) goto stopjob; if (qemuProcessFinishStartup(driver, vm, VIR_ASYNC_JOB_MIGRATION_IN, diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index ddc8e65489..c3c48c19c0 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -255,7 +255,8 @@ int qemuMigrationDstRun(virQEMUDriver *driver, virDomainObj *vm, const char *uri, - virDomainAsyncJob asyncJob); + virDomainAsyncJob asyncJob, + bool wait); void qemuMigrationAnyPostcopyFailed(virQEMUDriver *driver, diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index b0b00eb0a2..0a1e7985fb 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -7788,6 +7788,7 @@ qemuProcessStart(virConnectPtr conn, const char *migrateFrom, int migrateFd, const char *migratePath, + bool wait_incoming, virDomainMomentObj *snapshot, virNetDevVPortProfileOp vmop, unsigned int flags) @@ -7850,7 +7851,7 @@ qemuProcessStart(virConnectPtr conn, relabel = true; if (incoming) { - if (qemuMigrationDstRun(driver, vm, incoming->uri, asyncJob) < 0) + if (qemuMigrationDstRun(driver, vm, incoming->uri, asyncJob, wait_incoming) < 0) goto stop; } else { /* Refresh state of devices from QEMU. During migration this happens diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index f81bfd930a..5a1d005cb0 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -86,8 +86,9 @@ int qemuProcessStart(virConnectPtr conn, virCPUDef *updatedCPU, virDomainAsyncJob asyncJob, const char *migrateFrom, - int stdin_fd, - const char *stdin_path, + int fd, + const char *migratePath, + bool wait_incoming, virDomainMomentObj *snapshot, virNetDevVPortProfileOp vmop, unsigned int flags); diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index d8b6635511..d58e070bf7 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -749,6 +749,7 @@ qemuSaveImageStartVM(virConnectPtr conn, const char *path, bool start_paused, bool reset_nvram, + bool wait_incoming, virDomainAsyncJob asyncJob) { qemuDomainObjPrivate *priv = vm->privateData; @@ -803,7 +804,8 @@ qemuSaveImageStartVM(virConnectPtr conn, priv->disableSlirp = true; if (qemuProcessStart(conn, driver, vm, cookie ? cookie->cpu : NULL, - asyncJob, "stdio", *fd, path, NULL, + asyncJob, "stdio", *fd, path, wait_incoming, + NULL, VIR_NETDEV_VPORT_PROFILE_OP_RESTORE, start_flags) == 0) started = true; diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index b775c5eb08..7be0892dde 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -97,6 +97,7 @@ qemuSaveImageStartVM(virConnectPtr conn, const char *path, bool start_paused, bool reset_nvram, + bool wait_incoming, virDomainAsyncJob asyncJob) ATTRIBUTE_NONNULL(4) ATTRIBUTE_NONNULL(5) ATTRIBUTE_NONNULL(6); diff --git a/src/qemu/qemu_snapshot.c b/src/qemu/qemu_snapshot.c index 2e445e8296..626a5a14b9 100644 --- a/src/qemu/qemu_snapshot.c +++ b/src/qemu/qemu_snapshot.c @@ -2092,7 +2092,7 @@ qemuSnapshotRevertActive(virDomainObj *vm, rc = qemuProcessStart(snapshot->domain->conn, driver, vm, cookie ? cookie->cpu : NULL, - VIR_ASYNC_JOB_START, NULL, -1, NULL, snap, + VIR_ASYNC_JOB_START, NULL, -1, NULL, false, snap, VIR_NETDEV_VPORT_PROFILE_OP_CREATE, start_flags); virDomainAuditStart(vm, "from-snapshot", rc >= 0); @@ -2215,7 +2215,7 @@ qemuSnapshotRevertInactive(virDomainObj *vm, start_flags |= paused ? VIR_QEMU_PROCESS_START_PAUSED : 0; rc = qemuProcessStart(snapshot->domain->conn, driver, vm, NULL, - VIR_ASYNC_JOB_START, NULL, -1, NULL, NULL, + VIR_ASYNC_JOB_START, NULL, -1, NULL, false, NULL, VIR_NETDEV_VPORT_PROFILE_OP_CREATE, start_flags); virDomainAuditStart(vm, "from-snapshot", rc >= 0); -- 2.35.3
use multifd to restore parallel saves. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_driver.c | 12 +++-- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_migration.h | 6 +++ src/qemu/qemu_saveimage.c | 111 +++++++++++++++++++++++++++++++++++++- src/qemu/qemu_saveimage.h | 9 +++- 5 files changed, 133 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 480f9b4ed9..6b238c7677 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5815,7 +5815,7 @@ static int qemuDomainRestoreInternal(virConnectPtr conn, const char *path, const char *dxml, - int nchannels G_GNUC_UNUSED, + int nchannels, unsigned int flags, int (*ensureACL)(virConnectPtr, virDomainDef *)) { @@ -5907,8 +5907,14 @@ qemuDomainRestoreInternal(virConnectPtr conn, flags) < 0) goto cleanup; - ret = qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, - false, reset_nvram, true, VIR_ASYNC_JOB_START); + if (flags & VIR_DOMAIN_SAVE_PARALLEL) { + ret = qemuSaveImageLoadMultiFd(conn, vm, oflags, data, reset_nvram, + &saveFd, nchannels, VIR_ASYNC_JOB_START); + + } else { + ret = qemuSaveImageStartVM(conn, driver, vm, &saveFd.fd, data, path, + false, reset_nvram, true, VIR_ASYNC_JOB_START); + } qemuProcessEndJob(vm); diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 542428ab8e..7bab913fe5 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1933,7 +1933,7 @@ qemuMigrationSrcWaitForCompletion(virQEMUDriver *driver, } -static int +int qemuMigrationDstWaitForCompletion(virQEMUDriver *driver, virDomainObj *vm, virDomainAsyncJob asyncJob, diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index c3c48c19c0..38f4877cf0 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -191,6 +191,12 @@ qemuMigrationDstFinish(virQEMUDriver *driver, int retcode, bool v3proto); +int +qemuMigrationDstWaitForCompletion(virQEMUDriver *driver, + virDomainObj *vm, + virDomainAsyncJob asyncJob, + bool postcopy); + int qemuMigrationSrcConfirm(virQEMUDriver *driver, virDomainObj *vm, diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index d58e070bf7..2bc81035ae 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -551,6 +551,106 @@ qemuSaveImageCreate(virQEMUDriver *driver, } +int qemuSaveImageLoadMultiFd(virConnectPtr conn, virDomainObj *vm, int oflags, + virQEMUSaveData *data, bool reset_nvram, + virQEMUSaveFd *saveFd, int nchannels, + virDomainAsyncJob asyncJob) +{ + virQEMUDriver *driver = conn->privateData; + qemuDomainObjPrivate *priv = vm->privateData; + virQEMUSaveFd *multiFd = NULL; + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autoptr(virCommand) cmd = NULL; + g_autofree char *helper_path = NULL; + g_autofree char *sun_path = g_strdup_printf("%s/restore-multifd.sock", cfg->saveDir); + int ret = -1; + + if (!(helper_path = virFileFindResource("libvirt_multifd_helper", + abs_top_builddir "/src", + LIBEXECDIR))) + goto cleanup; + cmd = virCommandNewArgList(helper_path, sun_path, NULL); + virCommandAddArgFormat(cmd, "%d", nchannels); + virCommandAddArgFormat(cmd, "%d", saveFd->fd); + virCommandPassFD(cmd, saveFd->fd, 0); + + /* Perform parallel multifd migration from files (main fd + channels) */ + if (!(multiFd = qemuSaveImageCreateMultiFd(driver, vm, cmd, saveFd->path, oflags, cfg, nchannels))) + goto cleanup; + if (qemuSaveImageStartVM(conn, driver, vm, NULL, data, sun_path, + false, reset_nvram, false, asyncJob) < 0) + goto cleanup; + if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATE_MULTIFD)) { + virReportError(VIR_ERR_OPERATION_FAILED, "%s", + _("QEMU multifd not supported")); + goto cleanup; + } else { + g_autoptr(qemuMigrationParams) migParams = qemuMigrationParamsNew(); + bool bwParam = virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATION_PARAM_BANDWIDTH); + + if (bwParam) { + if (qemuMigrationParamsSetULL(migParams, + QEMU_MIGRATION_PARAM_MAX_BANDWIDTH, + QEMU_DOMAIN_MIG_BANDWIDTH_MAX * 1024 * 1024) < 0) + goto cleanup; + priv->migMaxBandwidth = QEMU_DOMAIN_MIG_BANDWIDTH_MAX; + } else { + if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) == 0) { + qemuMonitorSetMigrationSpeed(priv->mon, + QEMU_DOMAIN_MIG_BANDWIDTH_MAX); + priv->migMaxBandwidth = QEMU_DOMAIN_MIG_BANDWIDTH_MAX; + qemuDomainObjExitMonitor(vm); + } + } + qemuMigrationParamsSetCap(migParams, QEMU_MIGRATION_CAP_MULTIFD); + if (qemuMigrationParamsSetInt(migParams, + QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS, + nchannels) < 0) + goto cleanup; + if (qemuMigrationParamsApply(driver, vm, asyncJob, migParams) < 0) + goto cleanup; + + if (!virDomainObjIsActive(vm)) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("guest unexpectedly quit")); + goto cleanup; + } + /* multifd helper can now connect, then wait for migration to complete */ + if (virCommandRunAsync(cmd, NULL) < 0) + goto cleanup; + + if (qemuMigrationDstWaitForCompletion(driver, vm, asyncJob, false) < 0) + goto cleanup; + + if (qemuSaveImageCloseMultiFd(multiFd, nchannels, vm) < 0) + goto cleanup; + + if (qemuProcessRefreshState(driver, vm, asyncJob) < 0) + goto cleanup; + + /* run 'cont' on the destination */ + if (qemuProcessStartCPUs(driver, vm, + VIR_DOMAIN_RUNNING_RESTORED, + asyncJob) < 0) { + if (virGetLastErrorCode() == VIR_ERR_OK) + virReportError(VIR_ERR_OPERATION_FAILED, + "%s", _("failed to resume domain")); + goto cleanup; + } + if (virDomainObjSave(vm, driver->xmlopt, cfg->stateDir) < 0) { + VIR_WARN("Failed to save status on vm %s", vm->def->name); + goto cleanup; + } + } + qemuDomainEventEmitJobCompleted(driver, vm); + ret = 0; + + cleanup: + ret = qemuSaveImageFreeMultiFd(multiFd, vm, nchannels, ret); + return ret; +} + + /* qemuSaveImageGetCompressionProgram: * @imageFormat: String representation from qemu.conf for the compression * image format being used (dump, save, or snapshot). @@ -757,6 +857,7 @@ qemuSaveImageStartVM(virConnectPtr conn, bool started = false; virObjectEvent *event; VIR_AUTOCLOSE intermediatefd = -1; + g_autofree char *migrate_from = NULL; g_autoptr(virCommand) cmd = NULL; g_autofree char *errbuf = NULL; g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); @@ -803,8 +904,14 @@ qemuSaveImageStartVM(virConnectPtr conn, if (cookie && !cookie->slirpHelper) priv->disableSlirp = true; + if (fd) { + migrate_from = g_strdup("stdio"); + } else { + migrate_from = g_strdup_printf("unix://%s", path); + } + if (qemuProcessStart(conn, driver, vm, cookie ? cookie->cpu : NULL, - asyncJob, "stdio", *fd, path, wait_incoming, + asyncJob, migrate_from, fd ? *fd : -1, path, wait_incoming, NULL, VIR_NETDEV_VPORT_PROFILE_OP_RESTORE, start_flags) == 0) @@ -828,7 +935,7 @@ qemuSaveImageStartVM(virConnectPtr conn, VIR_DEBUG("Decompression binary stderr: %s", NULLSTR(errbuf)); virErrorRestore(&orig_err); } - if (VIR_CLOSE(*fd) < 0) { + if (fd && VIR_CLOSE(*fd) < 0) { virReportSystemError(errno, _("cannot close file: %s"), path); rc = -1; } diff --git a/src/qemu/qemu_saveimage.h b/src/qemu/qemu_saveimage.h index 7be0892dde..ae7b3faa17 100644 --- a/src/qemu/qemu_saveimage.h +++ b/src/qemu/qemu_saveimage.h @@ -99,7 +99,7 @@ qemuSaveImageStartVM(virConnectPtr conn, bool reset_nvram, bool wait_incoming, virDomainAsyncJob asyncJob) - ATTRIBUTE_NONNULL(4) ATTRIBUTE_NONNULL(5) ATTRIBUTE_NONNULL(6); + ATTRIBUTE_NONNULL(5) ATTRIBUTE_NONNULL(6); int qemuSaveImageOpen(virQEMUDriver *driver, @@ -117,6 +117,13 @@ qemuSaveImageGetCompressionProgram(const char *imageFormat, bool use_raw_on_fail) ATTRIBUTE_NONNULL(2); +int qemuSaveImageLoadMultiFd(virConnectPtr conn, virDomainObj *vm, int oflags, + virQEMUSaveData *data, bool reset_nvram, + virQEMUSaveFd *saveFd, int nchannels, + virDomainAsyncJob asyncJob) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(4) + ATTRIBUTE_NONNULL(6) G_GNUC_WARN_UNUSED_RESULT; + int qemuSaveImageCreate(virQEMUDriver *driver, virDomainObj *vm, -- 2.35.3
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- docs/manpages/virsh.rst | 23 ++++++++++++++----- tools/virsh-domain.c | 49 +++++++++++++++++++++++++++++++++-------- 2 files changed, 57 insertions(+), 15 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index e73e590754..e9012b85d1 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3803,15 +3803,18 @@ save :: save domain state-file [--bypass-cache] [--xml file] + [--parallel] [--parallel-connections connections] [{--running | --paused}] [--verbose] -Saves a running domain (RAM, but not disk state) to a state file so that -it can be restored -later. Once saved, the domain will no longer be running on the -system, thus the memory allocated for the domain will be free for -other domains to use. ``virsh restore`` restores from this state file. +Saves a paused or running domain (RAM, but not disk state) to one or more +state files, so that it can be restored later. +Once saved, the domain will no longer be running on the system, +thus the memory allocated for the domain will be free for +other domains to use. ``virsh restore`` restores from state file/s. + If *--bypass-cache* is specified, the save will avoid the file system -cache, although this may slow down the operation. +cache; depending on the specific scenario this may slow down or speed up +the operation. The progress may be monitored using ``domjobinfo`` virsh command and canceled with ``domjobabort`` command (sent by another virsh instance). Another option @@ -3833,6 +3836,14 @@ based on the state the domain was in when the save was done; passing either the *--running* or *--paused* flag will allow overriding which state the ``restore`` should use. +*--parallel* option will cause the save data to be sent over multiple +parallel connections to multiple files. The main save file is specified +with ``state-file``, and a number of additional connections can be +set using *--parallel-connections*, which will save to files named +``state-file``.1 , ``state-file``.2 ... up to ``connections``. + +Parallel connections may help in speeding up the save operation. + Domain saved state files assume that disk images will be unchanged between the creation and restore point. For a more complete system restore point, where the disk state is saved alongside the memory diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index ba492e807e..4e39834118 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -4174,6 +4174,14 @@ static const vshCmdOptDef opts_save[] = { .type = VSH_OT_BOOL, .help = N_("avoid file system cache when saving") }, + {.name = "parallel", + .type = VSH_OT_BOOL, + .help = N_("enable parallel save to files") + }, + {.name = "parallel-connections", + .type = VSH_OT_INT, + .help = N_("number of connections/files for parallel save") + }, {.name = "xml", .type = VSH_OT_STRING, .completer = virshCompletePathLocalExisting, @@ -4203,6 +4211,11 @@ doSave(void *opaque) g_autoptr(virshDomain) dom = NULL; const char *name = NULL; const char *to = NULL; + virTypedParameterPtr params = NULL; + int nparams = 0; + int maxparams = 0; + int intOpt = 0; + int rv = -1; unsigned int flags = 0; const char *xmlfile = NULL; g_autofree char *xml = NULL; @@ -4216,29 +4229,46 @@ doSave(void *opaque) goto out_sig; #endif /* !WIN32 */ - if (vshCommandOptStringReq(ctl, cmd, "file", &to) < 0) + if ((rv = vshCommandOptStringReq(ctl, cmd, "file", &to)) < 0) { goto out; - + } else { + if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_SAVE_PARAM_FILE, to) < 0) + goto out; + } if (vshCommandOptBool(cmd, "bypass-cache")) flags |= VIR_DOMAIN_SAVE_BYPASS_CACHE; + if (vshCommandOptBool(cmd, "parallel")) + flags |= VIR_DOMAIN_SAVE_PARALLEL; + if ((rv = vshCommandOptInt(ctl, cmd, "parallel-connections", &intOpt)) < 0) { + goto out; + } else if (rv > 0) { + if (virTypedParamsAddInt(¶ms, &nparams, &maxparams, + VIR_SAVE_PARAM_PARALLEL_CONNECTIONS, intOpt) < 0) + goto out; + } if (vshCommandOptBool(cmd, "running")) flags |= VIR_DOMAIN_SAVE_RUNNING; if (vshCommandOptBool(cmd, "paused")) flags |= VIR_DOMAIN_SAVE_PAUSED; - if (vshCommandOptStringReq(ctl, cmd, "xml", &xmlfile) < 0) + if ((rv = vshCommandOptStringReq(ctl, cmd, "xml", &xmlfile)) < 0) goto out; if (!(dom = virshCommandOptDomain(ctl, cmd, &name))) goto out; - if (xmlfile && - virFileReadAll(xmlfile, VSH_MAX_XML_FILE, &xml) < 0) { - vshReportError(ctl); - goto out; + if (xmlfile) { + if (virFileReadAll(xmlfile, VSH_MAX_XML_FILE, &xml) < 0) { + vshReportError(ctl); + goto out; + } else if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_SAVE_PARAM_DXML, xml) < 0) + goto out; } - - if (flags || xml) { + if (flags & VIR_DOMAIN_SAVE_PARALLEL) { + rc = virDomainSaveParams(dom, params, nparams, flags); + } else if (flags || xml) { rc = virDomainSaveFlags(dom, to, xml, flags); } else { rc = virDomainSave(dom, to); @@ -4252,6 +4282,7 @@ doSave(void *opaque) data->ret = 0; out: + virTypedParamsFree(params, nparams); #ifndef WIN32 pthread_sigmask(SIG_SETMASK, &oldsigmask, NULL); out_sig: -- 2.35.3
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- docs/manpages/virsh.rst | 11 +++++++++- tools/virsh-domain.c | 47 ++++++++++++++++++++++++++++++++++------- 2 files changed, 49 insertions(+), 9 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index e9012b85d1..3bf9b54806 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3754,12 +3754,14 @@ restore :: restore state-file [--bypass-cache] [--xml file] + [--parallel] [--parallel-connections connections] [{--running | --paused}] [--reset-nvram] Restores a domain from a ``virsh save`` state file. See *save* for more info. If *--bypass-cache* is specified, the restore will avoid the file system -cache, although this may slow down the operation. +cache; depending on the specific scenario this may slow down or speed up +the operation. *--xml* ``file`` is usually omitted, but can be used to supply an alternative XML file for use on the restored guest with changes only @@ -3775,6 +3777,13 @@ domain should be started in. If *--reset-nvram* is specified, any existing NVRAM file will be deleted and re-initialized from its pristine template. +*--parallel* option will cause the save data to be loaded from multiple +state files over parallel connections. The main save file is specified +with ``state-file``, and the number of additional channels can be set +using *--parallel-connections* + +Parallel connections may help in speeding up the save operation. + ``Note``: To avoid corrupting file system contents within the domain, you should not reuse the saved state file for a second ``restore`` unless you have also reverted all storage volumes back to the same contents as when diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 4e39834118..9fdb4e43d0 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -5312,6 +5312,14 @@ static const vshCmdOptDef opts_restore[] = { .type = VSH_OT_BOOL, .help = N_("avoid file system cache when restoring") }, + {.name = "parallel", + .type = VSH_OT_BOOL, + .help = N_("enable parallel restore") + }, + {.name = "parallel-connections", + .type = VSH_OT_INT, + .help = N_("number of connections/files for parallel restore") + }, {.name = "xml", .type = VSH_OT_STRING, .completer = virshCompletePathLocalExisting, @@ -5336,17 +5344,36 @@ static bool cmdRestore(vshControl *ctl, const vshCmd *cmd) { const char *from = NULL; + virTypedParameterPtr params = NULL; + int nparams = 0; + int maxparams = 0; + int intOpt = 0; unsigned int flags = 0; const char *xmlfile = NULL; g_autofree char *xml = NULL; virshControl *priv = ctl->privData; - int rc; + int rc = -1; - if (vshCommandOptStringReq(ctl, cmd, "file", &from) < 0) - return false; + if (vshCommandOptStringReq(ctl, cmd, "file", &from) < 0) { + goto out; + } else { + if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_SAVE_PARAM_FILE, from) < 0) + goto out; + } if (vshCommandOptBool(cmd, "bypass-cache")) flags |= VIR_DOMAIN_SAVE_BYPASS_CACHE; + if (vshCommandOptBool(cmd, "parallel")) + flags |= VIR_DOMAIN_SAVE_PARALLEL; + if ((rc = vshCommandOptInt(ctl, cmd, "parallel-connections", &intOpt)) < 0) { + goto out; + } else if (rc > 0) { + rc = -1; + if (virTypedParamsAddInt(¶ms, &nparams, &maxparams, + VIR_SAVE_PARAM_PARALLEL_CONNECTIONS, intOpt) < 0) + goto out; + } if (vshCommandOptBool(cmd, "running")) flags |= VIR_DOMAIN_SAVE_RUNNING; if (vshCommandOptBool(cmd, "paused")) @@ -5355,13 +5382,15 @@ cmdRestore(vshControl *ctl, const vshCmd *cmd) flags |= VIR_DOMAIN_SAVE_RESET_NVRAM; if (vshCommandOptStringReq(ctl, cmd, "xml", &xmlfile) < 0) - return false; + goto out; if (xmlfile && virFileReadAll(xmlfile, VSH_MAX_XML_FILE, &xml) < 0) - return false; + goto out; - if (flags || xml) { + if (flags & VIR_DOMAIN_SAVE_PARALLEL) { + rc = virDomainRestoreParams(priv->conn, params, nparams, flags); + } else if (flags || xml) { rc = virDomainRestoreFlags(priv->conn, from, xml, flags); } else { rc = virDomainRestore(priv->conn, from); @@ -5369,11 +5398,13 @@ cmdRestore(vshControl *ctl, const vshCmd *cmd) if (rc < 0) { vshError(ctl, _("Failed to restore domain from %s"), from); - return false; + goto out; } vshPrintExtra(ctl, _("Domain restored from %s\n"), from); - return true; + out: + virTypedParamsFree(params, nparams); + return rc >= 0; } /* -- 2.35.3
Signed-off-by: Claudio Fontana <cfontana@suse.de> --- docs/formatsnapshot.rst | 5 +++-- include/libvirt/libvirt-domain.h | 1 + src/libvirt-domain.c | 6 +++--- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/formatsnapshot.rst b/docs/formatsnapshot.rst index 9b874e4890..dd742a3063 100644 --- a/docs/formatsnapshot.rst +++ b/docs/formatsnapshot.rst @@ -38,8 +38,9 @@ full system Libvirt can manage all three types of snapshots. For now, VM state (memory) snapshots are created only by the ``virDomainSave()``, ``virDomainSaveFlags``, -and ``virDomainManagedSave()`` functions, and restored via the -``virDomainRestore()``, ``virDomainRestoreFlags()``, ``virDomainCreate()``, and +``virDomainSaveParams`` and ``virDomainManagedSave()`` functions, and restored +via the ``virDomainRestore()``, ``virDomainRestoreFlags()``, +``virDomainRestoreParams``, ``virDomainCreate()``, and ``virDomainCreateWithFlags()`` functions (as well as via domain autostart). With managed snapshots, libvirt tracks all information internally; with save images, the user tracks the snapshot file, but libvirt provides functions such as diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index d130fbf481..5e858f854b 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1544,6 +1544,7 @@ int virDomainPMWakeup (virDomainPtr domain, * virDomainSaveRestoreFlags: * * Flags for use in virDomainSaveFlags(), virDomainManagedSave(), + * virDomainSaveParams(), virDomainRestoreParams(), * virDomainRestoreFlags(), and virDomainSaveImageDefineXML(). Not all * flags apply to all these functions. * diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 0d6afcc1c6..d1d62daa71 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -859,9 +859,9 @@ virDomainPMWakeup(virDomainPtr dom, * listed as running anymore (this ends the life of a transient domain). * Use virDomainRestore() to restore a domain after saving. * - * See virDomainSaveFlags() for more control. Also, a save file can - * be inspected or modified slightly with virDomainSaveImageGetXMLDesc() - * and virDomainSaveImageDefineXML(). + * See virDomainSaveFlags() and virDomainSaveParams() for more control. + * Also, a save file can be inspected or modified slightly with + * virDomainSaveImageGetXMLDesc() and virDomainSaveImageDefineXML(). * * Returns 0 in case of success and -1 in case of failure. * -- 2.35.3
use zstd which is the only really interesting one. Signed-off-by: Claudio Fontana <cfontana@suse.de> --- src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_migration.c | 6 +++ src/qemu/qemu_migration_params.c | 49 +++++++++---------- src/qemu/qemu_migration_params.h | 6 +++ src/qemu/qemu_saveimage.c | 6 +++ .../caps_5.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 1 + .../caps_5.0.0.riscv64.xml | 1 + .../caps_5.0.0.x86_64.xml | 1 + .../qemucapabilitiesdata/caps_5.1.0.sparc.xml | 1 + .../caps_5.1.0.x86_64.xml | 1 + .../caps_5.2.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 1 + .../caps_5.2.0.riscv64.xml | 1 + .../qemucapabilitiesdata/caps_5.2.0.s390x.xml | 1 + .../caps_5.2.0.x86_64.xml | 1 + .../caps_6.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_6.0.0.s390x.xml | 1 + .../caps_6.0.0.x86_64.xml | 1 + .../caps_6.1.0.x86_64.xml | 1 + .../caps_6.2.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 1 + .../caps_6.2.0.x86_64.xml | 1 + .../caps_7.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 1 + .../caps_7.0.0.x86_64.xml | 1 + 27 files changed, 66 insertions(+), 25 deletions(-) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 581b6a40df..9c2d5643eb 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -675,6 +675,7 @@ VIR_ENUM_IMPL(virQEMUCaps, /* 430 */ "migrate-multifd", /* QEMU_CAPS_MIGRATE_MULTIFD */ + "migration-param.multifd-compression", /* QEMU_CAPS_MIGRATION_PARAM_MULTIFD_COMPRESSION */ ); @@ -1612,6 +1613,7 @@ static struct virQEMUCapsStringFlags virQEMUCapsQMPSchemaQueries[] = { { "migrate-set-parameters/arg-type/downtime-limit", QEMU_CAPS_MIGRATION_PARAM_DOWNTIME }, { "migrate-set-parameters/arg-type/xbzrle-cache-size", QEMU_CAPS_MIGRATION_PARAM_XBZRLE_CACHE_SIZE }, { "migrate-set-parameters/arg-type/block-bitmap-mapping/bitmaps/transform", QEMU_CAPS_MIGRATION_PARAM_BLOCK_BITMAP_MAPPING }, + { "migrate-set-parameters/arg-type/multifd-compression", QEMU_CAPS_MIGRATION_PARAM_MULTIFD_COMPRESSION }, { "nbd-server-start/arg-type/tls-creds", QEMU_CAPS_NBD_TLS }, { "nbd-server-add/arg-type/bitmap", QEMU_CAPS_NBD_BITMAP }, { "netdev_add/arg-type/+vhost-vdpa", QEMU_CAPS_NETDEV_VHOST_VDPA }, diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index b089f83da1..e226b1a51a 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -650,6 +650,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for syntax-check */ /* 430 */ QEMU_CAPS_MIGRATE_MULTIFD, /* migrate can set multifd parameter */ + QEMU_CAPS_MIGRATION_PARAM_MULTIFD_COMPRESSION, /* multifd-compression in migrate-set-parameters */ QEMU_CAPS_LAST /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 7bab913fe5..f9c86de67e 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -5950,6 +5950,12 @@ qemuMigrationSrcToFileAux(virQEMUDriver *driver, virDomainObj *vm, QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS, nchannels) < 0) return -1; + if (virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATION_PARAM_MULTIFD_COMPRESSION)) { + if (qemuMigrationParamsSetString(migParams, + QEMU_MIGRATION_PARAM_MULTIFD_COMPRESSION, + "zstd") < 0) + return -1; + } } if (needParams && qemuMigrationParamsApply(driver, vm, asyncJob, migParams) < 0) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index 36174a66d8..f6b9dc337d 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -115,6 +115,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "xbzrle-cache-size", "max-postcopy-bandwidth", "multifd-channels", + "multifd-compression", ); typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysOnItem; @@ -234,6 +235,7 @@ static const qemuMigrationParamType qemuMigrationParamTypes[] = { [QEMU_MIGRATION_PARAM_XBZRLE_CACHE_SIZE] = QEMU_MIGRATION_PARAM_TYPE_ULL, [QEMU_MIGRATION_PARAM_MAX_POSTCOPY_BANDWIDTH] = QEMU_MIGRATION_PARAM_TYPE_ULL, [QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS] = QEMU_MIGRATION_PARAM_TYPE_INT, + [QEMU_MIGRATION_PARAM_MULTIFD_COMPRESSION] = QEMU_MIGRATION_PARAM_TYPE_STRING, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamTypes) == QEMU_MIGRATION_PARAM_LAST); @@ -898,31 +900,6 @@ qemuMigrationParamsApply(virQEMUDriver *driver, } -/** - * qemuMigrationParamsSetString: - * @migrParams: migration parameter object - * @param: parameter to set - * @value: new value - * - * Enables and sets the migration parameter @param in @migrParams. Returns 0 on - * success and -1 on error. Libvirt error is reported. - */ -static int -qemuMigrationParamsSetString(qemuMigrationParams *migParams, - qemuMigrationParam param, - const char *value) -{ - if (qemuMigrationParamsCheckType(param, QEMU_MIGRATION_PARAM_TYPE_STRING) < 0) - return -1; - - migParams->params[param].value.s = g_strdup(value); - - migParams->params[param].set = true; - - return 0; -} - - /* qemuMigrationParamsEnableTLS * @driver: pointer to qemu driver * @vm: domain object @@ -1144,6 +1121,28 @@ qemuMigrationParamsSetULL(qemuMigrationParams *migParams, return 0; } +/** + * qemuMigrationParamsSetString: + * @migrParams: migration parameter object + * @param: parameter to set + * @value: new value + * + * Enables and sets the migration parameter @param in @migrParams. Returns 0 on + * success and -1 on error. Libvirt error is reported. + */ +int +qemuMigrationParamsSetString(qemuMigrationParams *migParams, + qemuMigrationParam param, + const char *value) +{ + if (qemuMigrationParamsCheckType(param, QEMU_MIGRATION_PARAM_TYPE_STRING) < 0) + return -1; + + migParams->params[param].value.s = g_strdup(value); + migParams->params[param].set = true; + return 0; +} + /** * Returns -1 on error, diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index 99af73b4a4..23a4e0c8a2 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -60,6 +60,7 @@ typedef enum { QEMU_MIGRATION_PARAM_XBZRLE_CACHE_SIZE, QEMU_MIGRATION_PARAM_MAX_POSTCOPY_BANDWIDTH, QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS, + QEMU_MIGRATION_PARAM_MULTIFD_COMPRESSION, QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; @@ -137,6 +138,11 @@ qemuMigrationParamsSetULL(qemuMigrationParams *migParams, qemuMigrationParam param, unsigned long long value); +int +qemuMigrationParamsSetString(qemuMigrationParams *migParams, + qemuMigrationParam param, + const char *value); + int qemuMigrationParamsGetULL(qemuMigrationParams *migParams, qemuMigrationParam param, diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 2bc81035ae..a1e9e1c3e8 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -607,6 +607,12 @@ int qemuSaveImageLoadMultiFd(virConnectPtr conn, virDomainObj *vm, int oflags, QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS, nchannels) < 0) goto cleanup; + if (virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_MIGRATION_PARAM_MULTIFD_COMPRESSION)) { + if (qemuMigrationParamsSetString(migParams, + QEMU_MIGRATION_PARAM_MULTIFD_COMPRESSION, + "zstd") < 0) + goto cleanup; + } if (qemuMigrationParamsApply(driver, vm, asyncJob, migParams) < 0) goto cleanup; diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml index c45b2e6cf6..1d032789b7 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.aarch64.xml @@ -176,6 +176,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700241</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml index a3ad743d70..4748bf791e 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml @@ -182,6 +182,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900241</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml b/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml index e1b5cac26b..859638fb1c 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.riscv64.xml @@ -168,6 +168,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>0</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml index 796adb9066..305ae0bf26 100644 --- a/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.0.0.x86_64.xml @@ -216,6 +216,7 @@ <flag name='memory-backend-file.prealloc-threads'/> <flag name='virtio-iommu-pci'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100241</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml b/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml index cb203df125..e0ff08a445 100644 --- a/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml +++ b/tests/qemucapabilitiesdata/caps_5.1.0.sparc.xml @@ -88,6 +88,7 @@ <flag name='query-display-options'/> <flag name='memory-backend-file.prealloc-threads'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5001000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>0</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml index 7479d942a2..9f089f551e 100644 --- a/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml @@ -220,6 +220,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5001000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml index 268d1444ad..3de49adf5b 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.aarch64.xml @@ -183,6 +183,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml index eabf4b600c..ec2ee445a1 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.ppc64.xml @@ -187,6 +187,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml b/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml index 0dbaf5a5ec..cf250f2e42 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.riscv64.xml @@ -173,6 +173,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>0</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml b/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml index b0fbab9cb5..6dc5f25f36 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.s390x.xml @@ -141,6 +141,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>39100243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml index 1a1717bf2a..69bfdcea01 100644 --- a/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml @@ -224,6 +224,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>5002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml index 1c18d122e2..cd25ee8ccb 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.aarch64.xml @@ -191,6 +191,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>6000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml b/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml index 8fa4cb2307..659d952b08 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.s390x.xml @@ -149,6 +149,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>6000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>39100242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml index 70c67202b1..7a2fdde901 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml @@ -233,6 +233,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>6000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100242</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml index a5ec77878f..f8c8266153 100644 --- a/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.1.0.x86_64.xml @@ -237,6 +237,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>6001000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml index 92d8ceff7e..09f7787797 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.aarch64.xml @@ -202,6 +202,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>6001050</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700244</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml index f219912927..5c3342d67a 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.ppc64.xml @@ -198,6 +198,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>6002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900244</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml index 38fd3878ea..2ce9f7bb6c 100644 --- a/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml @@ -239,6 +239,7 @@ <flag name='virtio-iommu-pci'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>6002000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100244</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml b/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml index 522e225c8f..9de4d8bc51 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.aarch64.xml @@ -210,6 +210,7 @@ <flag name='virtio-iommu.boot-bypass'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>6002092</version> <kvmVersion>0</kvmVersion> <microcodeVersion>61700243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml index 1eb43799c0..9a0e30b25a 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.ppc64.xml @@ -211,6 +211,7 @@ <flag name='virtio-iommu.boot-bypass'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>6002092</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900243</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml index e5023c4219..250993cd3f 100644 --- a/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_7.0.0.x86_64.xml @@ -243,6 +243,7 @@ <flag name='virtio-iommu.boot-bypass'/> <flag name='virtio-net.rss'/> <flag name='migrate-multifd'/> + <flag name='migration-param.multifd-compression'/> <version>7000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100243</microcodeVersion> -- 2.35.3
Hi Daniel, will repost a v7 today with many more improvements, in terms of split and in terms of multifd-compression. Thanks! Claudio On 5/5/22 2:52 PM, Claudio Fontana wrote:
This a respin of the multifd save prototype, rebased on post 8.3.0 release.
There are still quite a few rough edges:
KNOWN ISSUES:
1) this applies only to virsh save and virsh restore for now (no managed save etc).
2) error handling is not good yet, especially during resume, errors may leave behind a qemu process and such. May need some help find all of these cases
3) the compression part is demonstrative only, there needs to be more attention to compression options, and detecting the compression used to store the multifd saves.
...
changes from v4:
* runIO renamed to virFileDiskCopy and rethought arguments
* renamed new APIs from ...ParametersFlags to ...Params
* introduce the new virDomainSaveParams and virDomainRestoreParams without any additional parameters, so they can be upstreamed first.
* solved the issue in the gendispatch.pl script generating code that was missing the conn parameter.
---
changes from v3:
* reordered series to have all helper-related change at the start
* solved all reported issues from ninja test, including documentation
* fixed most broken migration capabilities code (still imperfect likely)
* added G_GNUC_UNUSED as needed
* after multifd restore, added what I think were the missing operations:
qemuProcessRefreshState(), qemuProcessStartCPUs() - most importantly, virDomainObjSave()
The domain now starts running after restore without further encouragement
* removed the sleep(10) from the multifd-helper
changes from v2:
* added ability to restore the VM from disk using multifd
* fixed the multifd-helper to work in both directions, assuming the need to listen for save, and connect for restore.
* fixed a large number of bugs, and probably introduced some :-)
Thanks for your thoughts,
Claudio
Claudio Fontana (27): iohelper: introduce new struct to carry copy operation parameters iohelper: refactor copy operation as a separate function iohelper: move runIO function to virfile.c virfile: rename runIO to virFileDiskCopy virfile: change virFileDiskCopy arguments to extend beyond stdin, stdout virfile: add comment about the use of SEEK_END in virFileDiskCopy multifd-helper: new helper for parallel save/restore libvirt: introduce virDomainSaveParams public API libvirt: introduce virDomainRestoreParams public API remote: Add RPC support for the virDomainSaveParams API gendispatch: add DomainRestoreParams as requiring conn argument remote: Add RPC support for the virDomainRestoreParams API qemu: add implementation for virDomainSaveParams API qemu: add implementation for virDomainRestoreParams API libvirt: add new VIR_DOMAIN_SAVE_PARALLEL flag and parameter qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in save qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in restore qemu: saveimage: introduce virQEMUSaveFd qemu: wire up saveimage code with the multifd helper qemu: capabilities: add multifd to the probed migration capabilities qemu: implement qemuMigrationSrcToFilesMultiFd qemu: add parameter to qemuMigrationDstRun to skip waiting qemu: implement qemuSaveImageLoadMultiFd tools: add parallel parameter to virsh save command tools: add parallel parameter to virsh restore command docs: update refs to virDomainSaveParams and virDomainRestoreParams qemu: add migration parameter multifd-compression
docs/formatsnapshot.rst | 5 +- docs/manpages/virsh.rst | 34 +- include/libvirt/libvirt-domain.h | 49 ++ po/POTFILES.in | 1 + src/driver-hypervisor.h | 14 + src/libvirt-domain.c | 99 +++- src/libvirt_private.syms | 1 + src/libvirt_public.syms | 6 + src/qemu/qemu_capabilities.c | 6 + src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_driver.c | 239 +++++++-- src/qemu/qemu_migration.c | 155 ++++-- src/qemu/qemu_migration.h | 16 +- src/qemu/qemu_migration_params.c | 71 ++- src/qemu/qemu_migration_params.h | 15 + src/qemu/qemu_process.c | 3 +- src/qemu/qemu_process.h | 5 +- src/qemu/qemu_saveimage.c | 496 ++++++++++++++---- src/qemu/qemu_saveimage.h | 49 +- src/qemu/qemu_snapshot.c | 6 +- src/remote/remote_driver.c | 2 + src/remote/remote_protocol.x | 29 +- src/remote_protocol-structs | 17 + src/rpc/gendispatch.pl | 5 +- src/util/iohelper.c | 162 +----- src/util/meson.build | 19 + src/util/multifd-helper.c | 249 +++++++++ src/util/virfile.c | 218 ++++++++ src/util/virfile.h | 2 + src/util/virthread.c | 5 + src/util/virthread.h | 1 + .../caps_4.0.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + .../caps_4.0.0.riscv32.xml | 1 + .../caps_4.0.0.riscv64.xml | 1 + .../qemucapabilitiesdata/caps_4.0.0.s390x.xml | 1 + .../caps_4.0.0.x86_64.xml | 1 + .../caps_4.1.0.x86_64.xml | 1 + .../caps_4.2.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + .../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 + .../caps_4.2.0.x86_64.xml | 1 + .../caps_5.0.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 2 + .../caps_5.0.0.riscv64.xml | 2 + .../caps_5.0.0.x86_64.xml | 2 + .../qemucapabilitiesdata/caps_5.1.0.sparc.xml | 2 + .../caps_5.1.0.x86_64.xml | 2 + .../caps_5.2.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 2 + .../caps_5.2.0.riscv64.xml | 2 + .../qemucapabilitiesdata/caps_5.2.0.s390x.xml | 2 + .../caps_5.2.0.x86_64.xml | 2 + .../caps_6.0.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_6.0.0.s390x.xml | 2 + .../caps_6.0.0.x86_64.xml | 2 + .../caps_6.1.0.x86_64.xml | 2 + .../caps_6.2.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 2 + .../caps_6.2.0.x86_64.xml | 2 + .../caps_7.0.0.aarch64.xml | 2 + .../qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 2 + .../caps_7.0.0.x86_64.xml | 2 + tools/virsh-domain.c | 96 +++- 64 files changed, 1686 insertions(+), 446 deletions(-) create mode 100644 src/util/multifd-helper.c
participants (2)
-
Claudio Fontana -
Daniel P. Berrangé