[libvirt RFCv8 00/27] multifd save restore prototype
by Claudio Fontana
This is v8 of the multifd save prototype, which fixes a few bugs,
adds a few more code splits, and records the number of channels
as well as the compression algorithm, so the restore command is
more user-friendly.
It is now possible to just say:
virsh save mydomain /mnt/saves/mysave --parallel
virsh restore /mnt/saves/mysave --parallel
and things work with the default of 2 channels, no compression.
It is also possible to say of course:
virsh save mydomain /mnt/saves/mysave --parallel
--parallel-connections 16 --parallel-compression zstd
virsh restore /mnt/saves/mysave --parallel
and things also work fine, due to channels and compression
being stored in the main save file.
---
changes from v7:
* [ base params API and iohelper refactoring upstreamed ]
* extended the QEMU save image format more, to record the nr
of multifd channels on save. Made the data header struct packed.
* removed --parallel-connections from the restore command, as now
it is useless due to QEMU save image format extension.
* separate out patches to expose migration_params APIs to saveimage,
including qemuMigrationParamsSetString, SetCap, SetInt.
* fixed bugs in the ImageOpen patch (missing saveFd init), removed
some whitespace, and fixed some convoluted code paths for return
value -3.
Claudio Fontana (27):
multifd-helper: new helper for parallel save/restore
tools: prepare doSave to use parameters
tools: prepare cmdRestore to use parameters
libvirt: add new VIR_DOMAIN_SAVE_PARALLEL flag and parameter
qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in save
qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in restore
qemu: saveimage: introduce virQEMUSaveFd
qemu: saveimage: convert qemuSaveImageCreate to use virQEMUSaveFd
qemu: saveimage: convert qemuSaveImageOpen to use virQEMUSaveFd
qemu: saveimage: add virQEMUSaveFd APIs for multifd
qemu: saveimage: wire up saveimage code with the multifd helper
qemu: capabilities: add multifd to the probed migration capabilities
qemu: saveimage: add multifd related fields to save format
qemu: migration_params: add APIs to set Int and Cap
qemu: migration: implement qemuMigrationSrcToFilesMultiFd
qemu: add parameter to qemuMigrationDstRun to skip waiting
qemu: implement qemuSaveImageLoadMultiFd
tools: add parallel parameter to virsh save command
tools: add parallel parameter to virsh restore command
qemu: add migration parameter multifd-compression
libvirt: add new VIR_SAVE_PARAM_PARALLEL_COMPRESSION
qemu: saveimage: add parallel compression argument to ImageCreate
qemu: saveimage: add stub support for multifd compression parameter
qemu: migration: expose qemuMigrationParamsSetString
qemu: saveimage: implement multifd-compression in parallel save
qemu: saveimage: restore compressed parallel images
tools: add parallel-compression parameter to virsh save command
docs/manpages/virsh.rst | 39 +-
include/libvirt/libvirt-domain.h | 29 +
po/POTFILES.in | 1 +
src/libvirt_private.syms | 1 +
src/qemu/qemu_capabilities.c | 6 +
src/qemu/qemu_capabilities.h | 4 +
src/qemu/qemu_driver.c | 140 +++--
src/qemu/qemu_migration.c | 160 +++--
src/qemu/qemu_migration.h | 16 +-
src/qemu/qemu_migration_params.c | 71 ++-
src/qemu/qemu_migration_params.h | 15 +
src/qemu/qemu_process.c | 3 +-
src/qemu/qemu_process.h | 5 +-
src/qemu/qemu_saveimage.c | 557 ++++++++++++++----
src/qemu/qemu_saveimage.h | 57 +-
src/qemu/qemu_snapshot.c | 6 +-
src/util/meson.build | 16 +
src/util/multifd-helper.c | 249 ++++++++
src/util/virthread.c | 5 +
src/util/virthread.h | 1 +
.../caps_4.0.0.aarch64.xml | 1 +
.../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 +
.../caps_4.0.0.riscv32.xml | 1 +
.../caps_4.0.0.riscv64.xml | 1 +
.../qemucapabilitiesdata/caps_4.0.0.s390x.xml | 1 +
.../caps_4.0.0.x86_64.xml | 1 +
.../caps_4.1.0.x86_64.xml | 1 +
.../caps_4.2.0.aarch64.xml | 1 +
.../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 +
.../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 +
.../caps_4.2.0.x86_64.xml | 1 +
.../caps_5.0.0.aarch64.xml | 2 +
.../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 2 +
.../caps_5.0.0.riscv64.xml | 2 +
.../caps_5.0.0.x86_64.xml | 2 +
.../qemucapabilitiesdata/caps_5.1.0.sparc.xml | 2 +
.../caps_5.1.0.x86_64.xml | 2 +
.../caps_5.2.0.aarch64.xml | 2 +
.../qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 2 +
.../caps_5.2.0.riscv64.xml | 2 +
.../qemucapabilitiesdata/caps_5.2.0.s390x.xml | 2 +
.../caps_5.2.0.x86_64.xml | 2 +
.../caps_6.0.0.aarch64.xml | 2 +
.../qemucapabilitiesdata/caps_6.0.0.s390x.xml | 2 +
.../caps_6.0.0.x86_64.xml | 2 +
.../caps_6.1.0.x86_64.xml | 2 +
.../caps_6.2.0.aarch64.xml | 2 +
.../qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 2 +
.../caps_6.2.0.x86_64.xml | 2 +
.../caps_7.0.0.aarch64.xml | 2 +
.../qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 2 +
.../caps_7.0.0.x86_64.xml | 2 +
tools/virsh-domain.c | 101 +++-
53 files changed, 1254 insertions(+), 281 deletions(-)
create mode 100644 src/util/multifd-helper.c
--
2.35.3
2 years, 6 months
[libvirt][PATCH v12 0/6] Support query and use SGX
by Lin Yang
This patch series provides support for enabling Intel's Software
Guard Extensions (SGX) feature in guest VM.
Giving the SGX support in QEMU had been merged. Intel SGX is a
set of instructions that increases the security of application code
and data, giving them more protection from disclosure or modification.
Developers can partition sensitive information into enclaves, which
are areas of execution in memory with more security protection.
The typical flow looks below at very high level:
1. Calls virConnectGetDomainCapabilities API to domain capabilities
that includes the following SGX information.
<feature>
...
<sgx supported='yes'>
<epc_size unit='KiB'>N</epc_size>
</sgx>
...
</feature>
2. User requests to start a guest calling virCreateXML() with SGX
requirement. It does not support NUMA yet, since latest QEMU 6.2
release does not support NUMA.
It should contain
<devices>
...
<memory model='sgx-epc'>
<target>
<size unit='KiB'>N</size>
</target>
</memory>
...
</devices>
Please note that SGX NUMA support will be implemented in future patches.
Haibin Huang (4):
Get SGX capabilities form QMP
Convert QMP capabilities to domain capabilities
Adding of the QEMU CAPS flag for unit test
conf: expose SGX feature in domain capabilities
Lin Yang (2):
conf: Introduce SGX EPC element into device memory xml
qemu: Add command-line to generate SGX EPC memory backend
docs/formatdomain.rst | 9 +-
docs/formatdomaincaps.rst | 26 ++++
src/conf/domain_capabilities.c | 29 ++++
src/conf/domain_capabilities.h | 13 ++
src/conf/domain_conf.c | 6 +
src/conf/domain_conf.h | 1 +
src/conf/domain_validate.c | 16 ++
src/conf/schemas/domaincaps.rng | 22 ++-
src/conf/schemas/domaincommon.rng | 1 +
src/libvirt_private.syms | 1 +
src/qemu/qemu_alias.c | 6 +-
src/qemu/qemu_capabilities.c | 141 ++++++++++++++++++
src/qemu/qemu_capabilities.h | 4 +
src/qemu/qemu_capspriv.h | 4 +
src/qemu/qemu_command.c | 54 ++++++-
src/qemu/qemu_domain.c | 38 +++--
src/qemu/qemu_domain_address.c | 6 +
src/qemu/qemu_driver.c | 1 +
src/qemu/qemu_monitor.c | 10 ++
src/qemu/qemu_monitor.h | 3 +
src/qemu/qemu_monitor_json.c | 104 ++++++++++++-
src/qemu/qemu_monitor_json.h | 9 ++
src/qemu/qemu_process.c | 2 +
src/qemu/qemu_validate.c | 8 +
src/security/security_apparmor.c | 1 +
src/security/security_dac.c | 2 +
src/security/security_selinux.c | 2 +
tests/domaincapsdata/bhyve_basic.x86_64.xml | 1 +
tests/domaincapsdata/bhyve_fbuf.x86_64.xml | 1 +
tests/domaincapsdata/bhyve_uefi.x86_64.xml | 1 +
tests/domaincapsdata/empty.xml | 1 +
tests/domaincapsdata/libxl-xenfv.xml | 1 +
tests/domaincapsdata/libxl-xenpv.xml | 1 +
.../domaincapsdata/qemu_2.11.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_2.11.0-tcg.x86_64.xml | 1 +
tests/domaincapsdata/qemu_2.11.0.s390x.xml | 1 +
tests/domaincapsdata/qemu_2.11.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_2.12.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_2.12.0-tcg.x86_64.xml | 1 +
.../qemu_2.12.0-virt.aarch64.xml | 1 +
tests/domaincapsdata/qemu_2.12.0.aarch64.xml | 1 +
tests/domaincapsdata/qemu_2.12.0.ppc64.xml | 1 +
tests/domaincapsdata/qemu_2.12.0.s390x.xml | 1 +
tests/domaincapsdata/qemu_2.12.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_3.0.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_3.0.0-tcg.x86_64.xml | 1 +
tests/domaincapsdata/qemu_3.0.0.ppc64.xml | 1 +
tests/domaincapsdata/qemu_3.0.0.s390x.xml | 1 +
tests/domaincapsdata/qemu_3.0.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_3.1.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_3.1.0-tcg.x86_64.xml | 1 +
tests/domaincapsdata/qemu_3.1.0.ppc64.xml | 1 +
tests/domaincapsdata/qemu_3.1.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_4.0.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_4.0.0-tcg.x86_64.xml | 1 +
.../qemu_4.0.0-virt.aarch64.xml | 1 +
tests/domaincapsdata/qemu_4.0.0.aarch64.xml | 1 +
tests/domaincapsdata/qemu_4.0.0.ppc64.xml | 1 +
tests/domaincapsdata/qemu_4.0.0.s390x.xml | 1 +
tests/domaincapsdata/qemu_4.0.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_4.1.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_4.1.0-tcg.x86_64.xml | 1 +
tests/domaincapsdata/qemu_4.1.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_4.2.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_4.2.0-tcg.x86_64.xml | 1 +
.../qemu_4.2.0-virt.aarch64.xml | 1 +
tests/domaincapsdata/qemu_4.2.0.aarch64.xml | 1 +
tests/domaincapsdata/qemu_4.2.0.ppc64.xml | 1 +
tests/domaincapsdata/qemu_4.2.0.s390x.xml | 1 +
tests/domaincapsdata/qemu_4.2.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_5.0.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_5.0.0-tcg.x86_64.xml | 1 +
.../qemu_5.0.0-virt.aarch64.xml | 1 +
tests/domaincapsdata/qemu_5.0.0.aarch64.xml | 1 +
tests/domaincapsdata/qemu_5.0.0.ppc64.xml | 1 +
tests/domaincapsdata/qemu_5.0.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_5.1.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_5.1.0-tcg.x86_64.xml | 1 +
tests/domaincapsdata/qemu_5.1.0.sparc.xml | 1 +
tests/domaincapsdata/qemu_5.1.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_5.2.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_5.2.0-tcg.x86_64.xml | 1 +
.../qemu_5.2.0-virt.aarch64.xml | 1 +
tests/domaincapsdata/qemu_5.2.0.aarch64.xml | 1 +
tests/domaincapsdata/qemu_5.2.0.ppc64.xml | 1 +
tests/domaincapsdata/qemu_5.2.0.s390x.xml | 1 +
tests/domaincapsdata/qemu_5.2.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_6.0.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_6.0.0-tcg.x86_64.xml | 1 +
.../qemu_6.0.0-virt.aarch64.xml | 1 +
tests/domaincapsdata/qemu_6.0.0.aarch64.xml | 1 +
tests/domaincapsdata/qemu_6.0.0.s390x.xml | 1 +
tests/domaincapsdata/qemu_6.0.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_6.1.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_6.1.0-tcg.x86_64.xml | 1 +
tests/domaincapsdata/qemu_6.1.0.x86_64.xml | 1 +
.../domaincapsdata/qemu_6.2.0-q35.x86_64.xml | 4 +
.../domaincapsdata/qemu_6.2.0-tcg.x86_64.xml | 4 +
.../qemu_6.2.0-virt.aarch64.xml | 1 +
tests/domaincapsdata/qemu_6.2.0.aarch64.xml | 1 +
tests/domaincapsdata/qemu_6.2.0.ppc64.xml | 1 +
tests/domaincapsdata/qemu_6.2.0.x86_64.xml | 4 +
.../domaincapsdata/qemu_7.0.0-q35.x86_64.xml | 4 +
.../domaincapsdata/qemu_7.0.0-tcg.x86_64.xml | 4 +
.../qemu_7.0.0-virt.aarch64.xml | 1 +
tests/domaincapsdata/qemu_7.0.0.aarch64.xml | 1 +
tests/domaincapsdata/qemu_7.0.0.ppc64.xml | 1 +
tests/domaincapsdata/qemu_7.0.0.x86_64.xml | 4 +
.../domaincapsdata/qemu_7.1.0-q35.x86_64.xml | 1 +
.../domaincapsdata/qemu_7.1.0-tcg.x86_64.xml | 1 +
tests/domaincapsdata/qemu_7.1.0.x86_64.xml | 1 +
.../caps_6.2.0.x86_64.replies | 22 ++-
.../caps_6.2.0.x86_64.xml | 5 +
.../caps_7.0.0.x86_64.replies | 22 ++-
.../caps_7.0.0.x86_64.xml | 5 +
.../caps_7.1.0.x86_64.replies | 21 ++-
.../sgx-epc.x86_64-6.2.0.args | 37 +++++
tests/qemuxml2argvdata/sgx-epc.xml | 36 +++++
tests/qemuxml2argvtest.c | 2 +
.../sgx-epc.x86_64-6.2.0.xml | 52 +++++++
tests/qemuxml2xmltest.c | 2 +
121 files changed, 785 insertions(+), 40 deletions(-)
create mode 100644 tests/qemuxml2argvdata/sgx-epc.x86_64-6.2.0.args
create mode 100644 tests/qemuxml2argvdata/sgx-epc.xml
create mode 100644 tests/qemuxml2xmloutdata/sgx-epc.x86_64-6.2.0.xml
--
2.25.1
2 years, 6 months
[PATCH 00/10] virclosecallbacks: Cleanups (part 1)
by Peter Krempa
This series simplifies some aspects of close callbacks in preparation of
further refactors.
Peter Krempa (10):
virclosecallbacks.h: Reformat header to contemporary style
virObjectLockGuard: Require that returned value is used
qemuMigrationSrcBegin: Automatically free 'xml' variable on error
bhyve: Store 'driver' in VM private data
lxc: Store 'driver' in VM private data
qemuMigrationSrcCleanup: Use 'driver' from VM private data
qemuProcessAutoDestroy: Use 'driver' from VM private data
lxcProcessAutoDestroy: Use 'driver' from VM private data
bhyveProcessAutoDestroy: Use 'driver' from VM private data
virclosecallbacks: Don't pass opqaue pointer to callback invocation
src/bhyve/bhyve_domain.c | 8 ++++++--
src/bhyve/bhyve_domain.h | 2 ++
src/bhyve/bhyve_driver.c | 2 +-
src/bhyve/bhyve_process.c | 6 +++---
src/hypervisor/virclosecallbacks.c | 5 ++---
src/hypervisor/virclosecallbacks.h | 29 +++++++++++++++++------------
src/lxc/lxc_domain.c | 4 +++-
src/lxc/lxc_domain.h | 1 +
src/lxc/lxc_driver.c | 2 +-
src/lxc/lxc_process.c | 8 +++-----
src/qemu/qemu_driver.c | 2 +-
src/qemu/qemu_migration.c | 13 +++++--------
src/qemu/qemu_process.c | 5 ++---
src/util/virobject.h | 2 +-
14 files changed, 48 insertions(+), 41 deletions(-)
--
2.35.3
2 years, 6 months
[PATCH 0/9] qemu: fd: Address differences between FD passing types
by Peter Krempa
Originally the intention was to have one set of helpers but it turned
out to be too much hassle and too obscure semantics.
This series adds a new helper qemuFDPassDirect and simplifies the
internals greatly by using a distinct set of APIs.
Peter Krempa (9):
qemu: fd: Add a distinct set of APIs for 'direct' fd passing mode
qemu: Convert passing of 'vhostfd' to 'qemuFDPassDirect'
qemu: Convert passing of 'tapfds' to 'qemuFDPassDirect'
qemu: Convert passing of 'slirpfd' to 'qemuFDPassDirect'
qemu: Use 'qemuFDPassDirect' for 'unix' chardevs
qemuFDPassValidate: Don't validate file descriptors
qemu: fd: Remove support for 'direct' fd passing from 'qemuFDPass'
qemuFDPassTransferCommand: Remove return value
qemu: fd: Fix documentation for FD set related functions
src/qemu/qemu_command.c | 55 +++-----
src/qemu/qemu_domain.c | 7 +-
src/qemu/qemu_domain.h | 7 +-
src/qemu/qemu_fd.c | 265 ++++++++++++++++++++---------------
src/qemu/qemu_fd.h | 24 +++-
src/qemu/qemu_hotplug.c | 15 +-
src/qemu/qemu_interface.c | 8 +-
src/qemu/qemu_monitor_json.c | 4 +-
src/qemu/qemu_process.c | 5 +-
src/qemu/qemu_slirp.c | 4 +-
tests/qemuxml2argvmock.c | 10 +-
tests/testutilsqemu.c | 26 ++--
12 files changed, 232 insertions(+), 198 deletions(-)
--
2.35.3
2 years, 6 months
[PATCH 0/3] ui: Remove deprecated sdl parameters and switch to QAPI parser
by Thomas Huth
The "-display sdl" option still uses a hand-crafted parser for its
parameters since some of them used underscores which is forbidden
in QAPI. Now that they've been deprecated and the deprecation period
is over, we can remove the problematic parameters and switch to use
the QAPI parser instead.
While we're at it, also remove the deprecated "-sdl" and "-curses" options.
Thomas Huth (3):
ui: Remove deprecated parameters of the "-display sdl" option
ui: Switch "-display sdl" to use the QAPI parser
ui: Remove deprecated options "-sdl" and "-curses"
docs/about/deprecated.rst | 26 -------
docs/about/removed-features.rst | 27 +++++++
qapi/ui.json | 17 ++++-
include/sysemu/sysemu.h | 2 -
softmmu/globals.c | 2 -
softmmu/vl.c | 128 +-------------------------------
ui/sdl2.c | 13 ++++
qemu-options.hx | 56 +-------------
8 files changed, 61 insertions(+), 210 deletions(-)
--
2.27.0
2 years, 6 months
[libvirt PATCH 0/7] po: Various fixes and cleanups
by Andrea Bolognani
msgid ""
"*** BLURB HERE ***"
msgstr ""
Andrea Bolognani (7):
po: Drop unwanted comments from potfile
syntax-check: Introduce sc_pot_comments
po: Drop prefixes from POTFILES.in
po: Don't generate POTFILES
syntax-check: Don't exclude src/false.c from sc_po_check
po: Sort LINGUAS
syntax-check: Introduce sc_linguas_sorting
build-aux/syntax-check.mk | 32 +++-
po/LINGUAS | 8 +-
po/POTFILES | 384 ++++++++++++++++++++++++++++++++++++++
po/POTFILES.in | 384 --------------------------------------
po/libvirt.pot | 216 ---------------------
po/meson.build | 13 +-
6 files changed, 414 insertions(+), 623 deletions(-)
create mode 100644 po/POTFILES
delete mode 100644 po/POTFILES.in
--
2.35.3
2 years, 6 months
[PATCH 00/11] Clean up arguments for qemuBuildCommandLine
by Peter Krempa
This series cleans up the arguments.
Note that patches 10,11 can be dropped if we decide that we want to keep
the '-S' argument off the command line bulit for 'virsh
domxml-to-native' for qemu definitions.
Peter Krempa (11):
virConnectDomainXMLToNative: Add note about dynamically configured
features
docs: drvqemu: Decrease expectations about command line from 'virsh
domxml-to-native'
qemu: command: Don't hide 'vhost' fds from 'standalone' command line
qemuBuildCommandLine: Convert 'standalone' flag to use 'flags'
qemu: Store state of FIPS in virQEMUDriver
qemuBuildCommandLine: Sanitize debug logging
qemuConnectDomainXMLToNative: Refactor cleanup
qemuBuildCommandLine: Remove 'driver' argument
qemuBuildCommandLine: Inline qemuCheckFips
qemuBuildCommandLine: Don't avoid '-S' flag for 'domxml-to-native'
conversion
qemuBuildCommandLine: Remove 'flags' argument
docs/drvqemu.rst | 14 +++++--
src/libvirt-domain.c | 7 ++++
src/qemu/qemu_command.c | 85 ++++++++++++----------------------------
src/qemu/qemu_command.h | 23 ++++-------
src/qemu/qemu_conf.h | 1 +
src/qemu/qemu_driver.c | 37 +++++++++--------
src/qemu/qemu_hotplug.c | 2 +-
src/qemu/qemu_process.c | 24 +++---------
src/qemu/qemu_process.h | 7 +---
tests/qemuxml2argvtest.c | 10 ++---
10 files changed, 82 insertions(+), 128 deletions(-)
--
2.35.3
2 years, 6 months
[libvirt PATCH 0/2] Minor fd passing fixes
by Jonathon Jongsma
Following up Peter's series, here's a couple additional minor cleanups that
simply remove some unused functions related to fd passing.
Jonathon Jongsma (2):
util: remove virCommandPassFDGetFDIndex()
util: Remove virCommandPassFDIndex()
src/libvirt_private.syms | 2 --
src/util/vircommand.c | 55 ++--------------------------------------
src/util/vircommand.h | 3 ---
3 files changed, 2 insertions(+), 58 deletions(-)
--
2.35.3
2 years, 6 months
[libvirt RFCv10 00/32] multifd save restore prototype
by Claudio Fontana
This is v10 of the multifd save prototype, which improves the
DIRECT I/O APIs and makes changes to the on disk format
backward and forward compatible, avoiding the need to alter
the QEMU_SAVE_VERSION.
This series demonstrates the DIRECT I/O friendly read/write code
working by using the block-friently multifd streams supplied by
the multifd helper, but could also work with block-aware QEMU streams
if ever available in the future.
KNOWN ISSUES:
a) still applies only to save/restore (no managed save etc)
b) multifd: saves to multiple files instead of just one file
---
changes from v9:
* exposed virFileDirectAlign
* separated the >= 2 QEMU_SAVE_VERSION change in own patch
* reworked the write code to add the alignment padding to the
data_len, making the on disk format compatible when loaded
from an older libvirt.
* reworked the read code to use direct I/O APIs only for actual
direct I/O file descriptors, so as to make old images work
with newer libvirt.
---
changes from v8:
* rebased on master
* reordered patches to add more upstreamable content at the start
* split introduction of virQEMUSaveFd, so the first part is multifd-free
* new virQEMUSaveDataRead as a mirror of virQEMUSaveDataWrite
* introduced virFileDirect API, using it in virFileDisk operations and
for virQEMUSaveRead and virQEMUSaveWrite
---
changes from v7:
* [ base params API and iohelper refactoring upstreamed ]
* extended the QEMU save image format more, to record the nr
of multifd channels on save. Made the data header struct packed.
* removed --parallel-connections from the restore command, as now
it is useless due to QEMU save image format extension.
* separate out patches to expose migration_params APIs to saveimage,
including qemuMigrationParamsSetString, SetCap, SetInt.
* fixed bugs in the ImageOpen patch (missing saveFd init), removed
some whitespace, and fixed some convoluted code paths for return
value -3.
---
changes from v6:
* improved error path handling, with error messages and especially
cancellation of qemu process on error during restore.
* split patches more and reordered them to keep general refactoring
at the beginning before the --parallel stuff is introduced.
* improved multifd compression support, including adding an enum
and extending the QEMU save image format to record the compression
used on save, and pick it up automatically on restore.
---
changes from v4:
* runIO renamed to virFileDiskCopy and rethought arguments
* renamed new APIs from ...ParametersFlags to ...Params
* introduce the new virDomainSaveParams and virDomainRestoreParams
without any additional parameters, so they can be upstreamed first.
* solved the issue in the gendispatch.pl script generating code that
was missing the conn parameter.
---
changes from v3:
* reordered series to have all helper-related change at the start
* solved all reported issues from ninja test, including documentation
* fixed most broken migration capabilities code (still imperfect likely)
* added G_GNUC_UNUSED as needed
* after multifd restore, added what I think were the missing operations:
qemuProcessRefreshState(),
qemuProcessStartCPUs() - most importantly,
virDomainObjSave()
The domain now starts running after restore without further encouragement
* removed the sleep(10) from the multifd-helper
---
changes from v2:
* added ability to restore the VM from disk using multifd
* fixed the multifd-helper to work in both directions,
assuming the need to listen for save, and connect for restore.
* fixed a large number of bugs, and probably introduced some :-)
---
Claudio Fontana (32):
virfile: introduce virFileDirect APIs
virfile: use virFileDirect API in runIOCopy
qemu: saveimage: rework image read/write to be O_DIRECT friendly
qemu: saveimage: assume future formats will also support compression
virfile: virFileDiskCopy: prepare for O_DIRECT files without wrapper
qemu: saveimage: introduce virQEMUSaveFd
qemu: saveimage: convert qemuSaveImageCreate to use virQEMUSaveFd
qemu: saveimage: convert qemuSaveImageOpen to use virQEMUSaveFd
tools: prepare doSave to use parameters
tools: prepare cmdRestore to use parameters
libvirt: add new VIR_DOMAIN_SAVE_PARALLEL flag and parameter
qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in save
qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in restore
multifd-helper: new helper for parallel save/restore
qemu: saveimage: add virQEMUSaveFd APIs for multifd
qemu: saveimage: wire up saveimage code with the multifd helper
qemu: capabilities: add multifd to the probed migration capabilities
qemu: saveimage: add multifd related fields to save format
qemu: migration_params: add APIs to set Int and Cap
qemu: migration: implement qemuMigrationSrcToFilesMultiFd for save
qemu: add parameter to qemuMigrationDstRun to skip waiting
qemu: implement qemuSaveImageLoadMultiFd for restore
tools: add parallel parameter to virsh save command
tools: add parallel parameter to virsh restore command
qemu: add migration parameter multifd-compression
libvirt: add new VIR_DOMAIN_SAVE_PARAM_PARALLEL_COMPRESSION
qemu: saveimage: add parallel compression argument to ImageCreate
qemu: saveimage: add stub support for multifd compression parameter
qemu: migration: expose qemuMigrationParamsSetString
qemu: saveimage: implement multifd-compression in parallel save
qemu: saveimage: restore compressed parallel images
tools: add parallel-compression parameter to virsh save command
docs/manpages/virsh.rst | 39 +-
include/libvirt/libvirt-domain.h | 29 +
po/POTFILES.in | 1 +
src/libvirt_private.syms | 9 +
src/qemu/qemu_capabilities.c | 4 +
src/qemu/qemu_capabilities.h | 2 +
src/qemu/qemu_driver.c | 146 ++--
src/qemu/qemu_migration.c | 160 ++--
src/qemu/qemu_migration.h | 16 +-
src/qemu/qemu_migration_params.c | 71 +-
src/qemu/qemu_migration_params.h | 15 +
src/qemu/qemu_process.c | 3 +-
src/qemu/qemu_process.h | 5 +-
src/qemu/qemu_saveimage.c | 793 ++++++++++++++----
src/qemu/qemu_saveimage.h | 71 +-
src/qemu/qemu_snapshot.c | 6 +-
src/util/meson.build | 16 +
src/util/multifd-helper.c | 247 ++++++
src/util/virfile.c | 316 ++++++-
src/util/virfile.h | 11 +
src/util/virthread.c | 5 +
src/util/virthread.h | 1 +
.../caps_4.0.0.aarch64.xml | 1 +
.../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 +
.../caps_4.0.0.riscv32.xml | 1 +
.../caps_4.0.0.riscv64.xml | 1 +
.../qemucapabilitiesdata/caps_4.0.0.s390x.xml | 1 +
.../caps_4.0.0.x86_64.xml | 1 +
.../caps_4.1.0.x86_64.xml | 1 +
.../caps_4.2.0.aarch64.xml | 1 +
.../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 +
.../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 +
.../caps_4.2.0.x86_64.xml | 1 +
.../caps_5.0.0.aarch64.xml | 2 +
.../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 2 +
.../caps_5.0.0.riscv64.xml | 2 +
.../caps_5.0.0.x86_64.xml | 2 +
.../qemucapabilitiesdata/caps_5.1.0.sparc.xml | 2 +
.../caps_5.1.0.x86_64.xml | 2 +
.../caps_5.2.0.aarch64.xml | 2 +
.../qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 2 +
.../caps_5.2.0.riscv64.xml | 2 +
.../qemucapabilitiesdata/caps_5.2.0.s390x.xml | 2 +
.../caps_5.2.0.x86_64.xml | 2 +
.../caps_6.0.0.aarch64.xml | 2 +
.../qemucapabilitiesdata/caps_6.0.0.s390x.xml | 2 +
.../caps_6.0.0.x86_64.xml | 2 +
.../caps_6.1.0.x86_64.xml | 2 +
.../caps_6.2.0.aarch64.xml | 2 +
.../qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 2 +
.../caps_6.2.0.x86_64.xml | 2 +
.../caps_7.0.0.aarch64.xml | 2 +
.../qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 2 +
.../caps_7.0.0.x86_64.xml | 2 +
.../caps_7.1.0.x86_64.xml | 2 +
tools/virsh-domain.c | 101 ++-
56 files changed, 1718 insertions(+), 404 deletions(-)
create mode 100644 src/util/multifd-helper.c
--
2.35.3
2 years, 6 months
