- Devel - libvirt List Archives

[libvirt] [PATCH v7 00/23] Incremental backups
by Eric Blake 27 Mar '19

27 Mar '19

I've finished rebasing my earlier v4/v5 patches on top of cleanups earlier in the series (v6 didn't include the second half of the series). I've pushed a tag backup-v7 to both my libvirt.git and libvirt-python.git repos to match: https://repo.or.cz/libvirt/ericb.git/shortlog/refs/tags/backup-v7 https://repo.or.cz/libvirt-python/ericb.git/shortlog/refs/tags/backup-v7 Here's hoping we're happy enough with the API, including the fact that rudimentary operation of pull mode backups work with qemu 4.0-rc1, for this to make it into the 5.2 release. Diffs from v6 - couple more early easy cleanups - add Dan's R-b where appropriate - drop any attempt at virDomainGetJobIds(), instead: - document that job id 0 always works for the current backup job (if there is just one), which is just fine for now because qemu 4.0 doesn't permit parallel jobs (and we can do full job id support for a later release) - split some more patches; finish moving backup code to a new backup_conf.c and not piggybacked onto checkpoint_conf.c Eric Blake (23): snapshot: Drop pointless function virDomainMomentIsCurrentName snapshot: Allow NULL to virDomainSnapshotObjGetDef snapshot: Refactor qemu to utilize virDomainMoment more backup: Document new XML for checkpoints backup: Document new XML for backups backup: Introduce virDomainCheckpoint APIs backup: Introduce virDomainBackup APIs backup: Document nuances between different state capture APIs backup: Parse and output checkpoint XML backup: Allow for lists of checkpoint objects squash to checkpoint list backup: Add new domain:checkpoint access control backup: Implement backup APIs for remote driver backup: Parse and output backup XML backup: Implement virsh support for checkpoints backup: Implement virsh support for backup backup: Add new qemu monitor interactions backup: qemu: Implement metadata tracking for checkpoint APIs backup: Wire up qemu checkpoint commands over QMP backup: qemu: Implement framework for backup job APIs backup: Wire up qemu full pull backup commands over QMP backup: qemu: Wire up qemu full push backup commands over QMP backup: implement qemu incremental pull backup include/libvirt/libvirt-domain-checkpoint.h | 161 ++ include/libvirt/libvirt-domain-snapshot.h | 8 +- include/libvirt/libvirt-domain.h | 47 +- include/libvirt/libvirt.h | 5 +- src/access/viraccessperm.h | 8 +- src/conf/backup_conf.h | 96 + src/conf/checkpoint_conf.h | 103 ++ src/conf/domain_conf.h | 2 + src/conf/virconftypes.h | 12 + src/conf/virdomaincheckpointobjlist.h | 74 + src/conf/virdomainmomentobjlist.h | 9 +- src/conf/virdomainobjlist.h | 9 +- src/conf/virdomainsnapshotobjlist.h | 4 +- src/driver-hypervisor.h | 81 +- src/qemu/qemu_block.h | 3 + src/qemu/qemu_blockjob.h | 1 + src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.h | 39 +- src/qemu/qemu_monitor.h | 21 +- src/qemu/qemu_monitor_json.h | 21 +- tools/virsh-checkpoint.h | 29 + tools/virsh-completer.h | 4 + tools/virsh-util.h | 3 + tools/virsh.h | 1 + docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 10 +- docs/domainstatecapture.html.in | 315 ++++ docs/format.html.in | 2 + docs/formatbackup.html.in | 184 ++ docs/formatcheckpoint.html.in | 204 +++ docs/formatsnapshot.html.in | 2 + docs/index.html.in | 4 +- docs/schemas/domainbackup.rng | 219 +++ docs/schemas/domaincheckpoint.rng | 87 + examples/object-events/event-test.c | 3 + libvirt.spec.in | 3 + mingw-libvirt.spec.in | 6 + po/POTFILES | 3 + src/Makefile.am | 2 + src/access/viraccessperm.c | 5 +- src/conf/Makefile.inc.am | 6 + src/conf/backup_conf.c | 544 ++++++ src/conf/checkpoint_conf.c | 639 +++++++ src/conf/domain_conf.c | 8 +- src/conf/snapshot_conf.c | 2 +- src/conf/virdomaincheckpointobjlist.c | 222 +++ src/conf/virdomainmomentobjlist.c | 49 +- src/conf/virdomainobjlist.c | 13 +- src/conf/virdomainsnapshotobjlist.c | 11 +- src/libvirt-domain-checkpoint.c | 750 ++++++++ src/libvirt-domain-snapshot.c | 89 + src/libvirt-domain.c | 239 ++- src/libvirt_private.syms | 33 +- src/libvirt_public.syms | 20 + src/qemu/qemu_block.c | 12 + src/qemu/qemu_capabilities.c | 6 + src/qemu/qemu_conf.c | 5 + src/qemu/qemu_domain.c | 249 ++- src/qemu/qemu_driver.c | 1609 ++++++++++++++++- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_monitor.c | 65 +- src/qemu/qemu_monitor_json.c | 201 +- src/qemu/qemu_process.c | 7 + src/remote/remote_daemon_dispatch.c | 22 +- src/remote/remote_driver.c | 34 +- src/remote/remote_protocol.x | 259 ++- src/remote_protocol-structs | 139 ++ src/rpc/gendispatch.pl | 34 +- src/test/test_driver.c | 7 +- tests/Makefile.am | 13 +- tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/redefine.xml | 63 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlin/size.xml | 4 + tests/domaincheckpointxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlout/sample.xml | 12 + tests/domaincheckpointxml2xmlout/size.xml | 11 + tests/domaincheckpointxml2xmltest.c | 212 +++ .../caps_4.0.0.riscv32.xml | 2 + .../caps_4.0.0.riscv64.xml | 2 + .../caps_4.0.0.x86_64.xml | 2 + tests/qemumonitorjsontest.c | 2 +- tests/virschematest.c | 4 + tools/Makefile.am | 3 +- tools/virsh-checkpoint.c | 1370 ++++++++++++++ tools/virsh-completer.c | 53 +- tools/virsh-domain-monitor.c | 25 +- tools/virsh-domain.c | 270 ++- tools/virsh-snapshot.c | 37 +- tools/virsh-util.c | 11 + tools/virsh.c | 2 + tools/virsh.pod | 302 +++- 100 files changed, 9373 insertions(+), 163 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 src/conf/backup_conf.h create mode 100644 src/conf/checkpoint_conf.h create mode 100644 src/conf/virdomaincheckpointobjlist.h create mode 100644 tools/virsh-checkpoint.h create mode 100644 docs/domainstatecapture.html.in create mode 100644 docs/formatbackup.html.in create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/checkpoint_conf.c create mode 100644 src/conf/virdomaincheckpointobjlist.c create mode 100644 src/libvirt-domain-checkpoint.c create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/redefine.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlin/size.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/size.xml create mode 100644 tests/domaincheckpointxml2xmltest.c create mode 100644 tools/virsh-checkpoint.c -- 2.20.1

4 63

[libvirt] [PATCH] tests: don't abort in fopen(/proc/mounts)
by Daniel P. Berrangé 27 Mar '19

27 Mar '19

The mock fopen() function will abort if "/proc/mounts" is requested with "r" permissions and VIR_CGROUP_MOCK_FILENAME env var is not set. Unfortunately this is triggering by the libselinux library constructor when it tries to read /proc/mounts to find out if selinuxfs is mounted in an unusual place. This, however, only affects libselinux in Debian as that opens with "r", while in Fedora / RHEL it opens "re" and thus luckily never triggered the abort(), instead getting an EACCESS. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- tests/vircgroupmock.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/vircgroupmock.c b/tests/vircgroupmock.c index 06bd0a5f29..9c67a44b0d 100644 --- a/tests/vircgroupmock.c +++ b/tests/vircgroupmock.c @@ -460,8 +460,10 @@ FILE *fopen(const char *path, const char *mode) } if (type) { - if (!filename) - abort(); + if (!filename) { + errno = EACCES; + return NULL; + } if (virAsprintfQuiet(&filepath, "%s/vircgroupdata/%s.%s", abs_srcdir, filename, type) < 0) { abort(); -- 2.20.1

2 3

[libvirt] [PATCH v6 0/9] Incremental backups, focus on API
by Eric Blake 27 Mar '19

27 Mar '19

I'm fairly confident that these API are ready to go (that is, I've got qemu code in the wings to implement these API for the qemu driver, as demonstrated at last KVM forum, and it shouldn't be too hard to add support in the test driver to get some 'make check' coverage similar to what I recently added for snapshots). I'm hoping the APIs make it in for 5.2, even if I'm still dealing with review churn on the later parts of my v5 series (there has been a lot of rebasing from earlier review comments, so v5 is currently still the most recent version that I was able to run demos with, although I hope to post the rest of v6 soon). The biggest diffs compared to v4/v5 are addressing many of John Ferlan's grammar suggestions, removing some TODO items from the public API (although implementing those changes, such as the new VIR_DOMAIN_SNAPSHOT_CREATE_QUIESCE, is still on my todo list), splitting some large patches (checkpoints and backups are now separated), moving the backup APIs into libvirt-domain.c instead of piggybacking on libvirt-domain-checkpoint.c, and adding 2 more API (virDomainSnapshotCreateXML2 to create a checkpoint at the same time as a snapshot; virDomainListJobIds to learn the job ids of all currently running backup jobs [the initial implementation will only ever have job id 1 due to qemu 4.0 limitations, but the API was designed with future expansion in mind]). I've also got some preliminary patches to snapshot in this posting based on things noticed while working checkpoints. 001/9:[down] 'Revert "snapshot: Add virDomainSnapshotObjListParse"' 002/9:[down] 'Revert "snapshot: Add virDomainSnapshotObjListFormat"' 003/9:[down] 'snapshot: Various doc tweaks' 004/9:[0005] [FC] 'backup: Introduce virDomainCheckpointPtr' 005/9:[down] 'backup: Document new XML for checkpoints' 006/9:[0562] [FC] 'backup: Document new XML for backups' 007/9:[0173] [FC] 'backup: Introduce virDomainCheckpoint APIs' 008/9:[0296] [FC] 'backup: Introduce virDomainBackup APIs' 009/9:[0039] [FC] 'backup: Document nuances between different state capture APIs' Eric Blake (9): Revert "snapshot: Add virDomainSnapshotObjListParse" Revert "snapshot: Add virDomainSnapshotObjListFormat" snapshot: Various doc tweaks backup: Introduce virDomainCheckpointPtr backup: Document new XML for checkpoints backup: Document new XML for backups backup: Introduce virDomainCheckpoint APIs backup: Introduce virDomainBackup APIs backup: Document nuances between different state capture APIs include/libvirt/virterror.h | 7 +- src/util/virerror.c | 12 +- include/libvirt/libvirt-domain-checkpoint.h | 161 ++++ include/libvirt/libvirt-domain-snapshot.h | 7 +- include/libvirt/libvirt-domain.h | 59 +- include/libvirt/libvirt.h | 3 +- src/conf/snapshot_conf.h | 7 - src/conf/virdomainmomentobjlist.h | 5 +- src/conf/virdomainsnapshotobjlist.h | 13 - src/datatypes.h | 33 +- src/driver-hypervisor.h | 87 ++- src/qemu/qemu_blockjob.h | 1 + docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 10 +- docs/domainstatecapture.html.in | 313 ++++++++ docs/format.html.in | 2 + docs/formatbackup.html.in | 181 +++++ docs/formatcheckpoint.html.in | 204 +++++ docs/formatsnapshot.html.in | 82 +- docs/index.html.in | 4 +- docs/schemas/domainbackup.rng | 219 ++++++ docs/schemas/domaincheckpoint.rng | 94 +++ examples/object-events/event-test.c | 3 + libvirt.spec.in | 3 + mingw-libvirt.spec.in | 6 + po/POTFILES | 1 + src/Makefile.am | 2 + src/conf/domain_conf.c | 2 +- src/conf/snapshot_conf.c | 2 +- src/conf/virdomainsnapshotobjlist.c | 166 ---- src/datatypes.c | 63 +- src/libvirt-domain-checkpoint.c | 749 +++++++++++++++++++ src/libvirt-domain-snapshot.c | 311 ++++---- src/libvirt-domain.c | 289 ++++++- src/libvirt_private.syms | 4 +- src/libvirt_public.syms | 21 + tests/Makefile.am | 4 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlout/empty.xml | 10 + tests/domaincheckpointxml2xmlout/sample.xml | 16 + tests/virschematest.c | 4 + tools/virsh-domain.c | 8 +- 50 files changed, 2794 insertions(+), 431 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 docs/domainstatecapture.html.in create mode 100644 docs/formatbackup.html.in create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 src/libvirt-domain-checkpoint.c create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml -- 2.20.1

5 28

[libvirt] [PATCH 1/2] qemu: Fix "boolen" typo in API doc
by Christophe Fergeau 27 Mar '19

27 Mar '19

This also adjusts the argument name which should be 'isListen' in both cases rather than 'listen'. Signed-off-by: Christophe Fergeau <cfergeau(a)redhat.com> --- src/qemu/qemu_command.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index f81d20e5f7..bfa7211894 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -876,7 +876,7 @@ qemuBuildRBDSecinfoURI(virBufferPtr buf, /* qemuBuildTLSx509BackendProps: * @tlspath: path to the TLS credentials - * @listen: boolen listen for client or server setting + * @isListen: boolen listen for client or server setting * @verifypeer: boolean to enable peer verification (form of authorization) * @alias: alias for the TLS credentials object * @secalias: if one exists, the alias of the security object for passwordid @@ -917,7 +917,7 @@ qemuBuildTLSx509BackendProps(const char *tlspath, /* qemuBuildTLSx509CommandLine: * @cmd: Pointer to command * @tlspath: path to the TLS credentials - * @listen: boolen listen for client or server setting + * @isListen: boolean listen for client or server setting * @verifypeer: boolean to enable peer verification (form of authorization) * @certEncSecretAlias: alias of a 'secret' object for decrypting TLS private key * (optional) -- 2.21.0

2 3

[libvirt] [PATCH] news: Document parallel migration
by Jiri Denemark 27 Mar '19

27 Mar '19

Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- docs/news.xml | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index 83e965e0f3..2067830848 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -104,6 +104,18 @@ E.g. <code><controller type='xenbus' maxGrantFrames='64'/></code> </description> </change> + <change> + <summary> + qemu: Add support for parallel migration + </summary> + <description> + With QEMU 4.0.0 libvirt can enable parallel migration which causes + the memory pages to be processed in parallel by several threads and + sent to the destination host using several connections at the same + time. This may increase migration speed in case a single thread is + unable to saturate the network link. + </description> + </change> </section> <section title="Removed features"> <change> -- 2.21.0

2 1

[libvirt] [dockerfiles PATCH] Add Dockerfile for i686 cross-compilation
by Andrea Bolognani 27 Mar '19

27 Mar '19

We didn't add this until now because i686 cross-compilation requires special care and lcitool was known to produce broken Dockerfiles, but that's been fixed as of libvirt-jenkins-ci commit f72d09153905 so we can finally go ahead and complete our coverage of non-x86_64 architectures. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- buildenv-debian-sid-cross-i686.Dockerfile | 93 +++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 buildenv-debian-sid-cross-i686.Dockerfile diff --git a/buildenv-debian-sid-cross-i686.Dockerfile b/buildenv-debian-sid-cross-i686.Dockerfile new file mode 100644 index 0000000..0ac238b --- /dev/null +++ b/buildenv-debian-sid-cross-i686.Dockerfile @@ -0,0 +1,93 @@ +FROM debian:sid + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + augeas-tools \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ccache \ + chrony \ + dnsmasq-base \ + dwarves \ + ebtables \ + gcc \ + gettext \ + git \ + iproute2 \ + kmod \ + libc-dev-bin \ + libtool \ + libtool-bin \ + libxml2-utils \ + lvm2 \ + make \ + nfs-common \ + numad \ + open-iscsi \ + parted \ + patch \ + perl \ + pkgconf \ + policykit-1 \ + qemu-utils \ + radvd \ + screen \ + scrub \ + sudo \ + vim \ + xsltproc \ + zfs-fuse && \ + apt-get autoremove -y && \ + apt-get autoclean -y + +RUN export DEBIAN_FRONTEND=noninteractive && \ + dpkg --add-architecture i386 && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + gcc-i686-linux-gnu \ + libacl1-dev:i386 \ + libapparmor-dev:i386 \ + libattr1-dev:i386 \ + libaudit-dev:i386 \ + libavahi-client-dev:i386 \ + libblkid-dev:i386 \ + libc6-dev:i386 \ + libcap-ng-dev:i386 \ + libcurl4-gnutls-dev:i386 \ + libdbus-1-dev:i386 \ + libdevmapper-dev:i386 \ + libfuse-dev:i386 \ + libglusterfs-dev:i386 \ + libgnutls28-dev:i386 \ + libiscsi-dev:i386 \ + libnl-3-dev:i386 \ + libnl-route-3-dev:i386 \ + libnuma-dev:i386 \ + libparted-dev:i386 \ + libpcap0.8-dev:i386 \ + libpciaccess-dev:i386 \ + librbd-dev:i386 \ + libreadline-dev:i386 \ + libsanlock-dev:i386 \ + libsasl2-dev:i386 \ + libselinux1-dev:i386 \ + libssh-gcrypt-dev:i386 \ + libssh2-1-dev:i386 \ + libtirpc-dev:i386 \ + libudev-dev:i386 \ + libxml2-dev:i386 \ + libyajl-dev:i386 \ + xfslibs-dev:i386 && \ + apt-get autoremove -y && \ + apt-get autoclean -y + +ENV ABI "i686-linux-gnu" +ENV CONFIGURE_OPTS "--host=i686-linux-gnu \ + --target=i686-linux-gnu" +ENV PKG_CONFIG_LIBDIR "/usr/lib/i386-linux-gnu/pkgconfig" -- 2.20.1

2 3

[libvirt] [PATCH] spec: Move ldconfig calls from -client to -libs
by Andrea Bolognani 27 Mar '19

27 Mar '19

ldconfig needs to be called after installing or uninstalling shared libraries. For a very long time, libvirt didn't have a separate package containing just the shared libraries, and so it shipped them in the same one as the clients. Since commit 70b4f0e719cd, however, shared libraries have been moved from -client to their own -libs package; unfortunately, the corresponding ldconfig calls were not moved at the same time, which is what this commit takes care of. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- libvirt.spec.in | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/libvirt.spec.in b/libvirt.spec.in index b7a35a0fb1..8ea27c91a3 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1328,6 +1328,16 @@ then exit 1 fi +%post libs +%if 0%{?rhel} == 7 +/sbin/ldconfig +%endif + +%postun libs +%if 0%{?rhel} == 7 +/sbin/ldconfig +%endif + %pre daemon # 'libvirt' group is just to allow password-less polkit access to # libvirtd. The uid number is irrelevant, so we use dynamic allocation @@ -1469,17 +1479,9 @@ exit 0 %systemd_preun libvirt-guests.service %post client - -%if 0%{?rhel} == 7 -/sbin/ldconfig -%endif %systemd_post libvirt-guests.service %postun client - -%if 0%{?rhel} == 7 -/sbin/ldconfig -%endif %systemd_postun libvirt-guests.service %if %{with_sanlock} -- 2.20.1

2 1

[libvirt] [PATCH v2 0/3] qemu: Add support for parallel migration
by Jiri Denemark 27 Mar '19

27 Mar '19

This series adds a new VIR_MIGRATE_PARALLEL flag for migration APIs which will ask the hypervisor to use multiple parallel connections for migrating a domain. The number of parallel connections can be set using VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS typed parameter. Version 2: - rebased Jiri Denemark (3): Public API for parallel migration qemu: Add support for parallel migration virsh: Add options for parallel migration include/libvirt/libvirt-domain.h | 14 ++++++++++++++ src/qemu/qemu_migration.c | 9 ++++++--- src/qemu/qemu_migration.h | 5 ++++- src/qemu/qemu_migration_params.c | 18 ++++++++++++++++++ src/qemu/qemu_migration_params.h | 2 ++ tools/virsh-domain.c | 19 +++++++++++++++++++ tools/virsh.pod | 7 +++++++ 7 files changed, 70 insertions(+), 4 deletions(-) -- 2.20.1

2 8

[libvirt] [PATCH v5 00/20] Incremental Backup API additions
by Eric Blake 27 Mar '19

27 Mar '19

The following is the latest version of my API proposal for incremental backups, and follows along from the demo I presented as part of my KVM Forum 2018 talk: https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-e… The patches are also available via https://repo.or.cz/libvirt/ericb.git at the tag backup-v5. Depends on my earlier bulk redefine for snapshots, which is pending review at v3: https://www.redhat.com/archives/libvir-list/2019-March/msg00144.html I haven't finished addressing all of John Ferlan's review comments from v4 (including the fact that several of these larger patches need to be split for easier reviews, and the fact that I want virDomainSnapshotObj and virDomainCheckpointObj to share a common base class); but I'm posting this now because I _have_ completed: - push mode backups mostly work (event notification is still not polished, but the fact that I can now do both push and pull modes is nice) - bulk mode list/redefine, accessible via 'virsh dumpxml --checkpoints' and 'virsh snapshot-create --redefine-list' - move virDomainBackup() into libvirt-domain.h (libvirt-domain-checkpoint.h is now JUST about standalone checkpoint objects; then both virDomainBackup() and virDomainSnapshotCreateXML() will be able to create a checkpoint in parallel) - rebase to master (5.2 now instead of 5.1, start using VIR_AUTOFREE in more places, ...) I'm still debating about making <domaincheckpoint> a sub-element of <domainbackup> (in which case I can reuse virDomainSnapshotCreateXML() to create a backup) or keeping it as a separate XML argument (in which case I need to add an API for virDomainSnapshotCreateXMLWithSnapshot()). I also need to work out an API for listing all currently running jobs (even if the current backup code can only run one job at a time, we know we want support in the future for parallel backup jobs). So there WILL be a v6. Hopefully, though, we can start trimming off the front of the series, and at least now the tail of the series currently compiles (John gave up review on v4 when he ran into rebase issues - although with our current rate of VIR_AUTOFREE conversions, there may again be rebase issues soon...) since v4: 001/20:[----] [-C] 'domain_conf: Expose virDomainStorageNetworkParseHost' 002/20:[down] 'qemu: clean up qemuDomainRemoveInactiveCommon' 003/20:[----] [--] 'backup: Document nuances between different state capture APIs' 004/20:[0020] [FC] 'backup: Introduce virDomainCheckpointPtr' 005/20:[down] 'wip: backup: Document new XML for backups' 006/20:[0054] [FC] 'backup: Introduce virDomainCheckpoint APIs' 007/20:[0053] [FC] 'backup: Introduce virDomainBackup APIs' 008/20:[0002] [FC] 'backup: Add new domain:checkpoint access control' 009/20:[0093] [FC] 'backup: Implement backup APIs for remote driver' 010/20:[0381] [FC] 'wip: backup: Parse and output checkpoint XML' 011/20:[0001] [FC] 'wip: backup: Parse and output backup XML' 012/20:[0393] [FC] 'backup: Implement virsh support for checkpoints' 013/20:[0058] [FC] 'wip: backup: virsh support for backup' 014/20:[0019] [FC] 'backup: Add new qemu monitor interactions' 015/20:[0161] [FC] 'backup: qemu: Implement metadata tracking for checkpoint APIs' 016/20:[----] [-C] 'wip: backup: Wire up qemu checkpoint commands over QMP' 017/20:[0006] [FC] 'wip: backup: qemu: Implement framework for backup job APIs' 018/20:[0046] [FC] 'backup: Wire up qemu full pull backup commands over QMP' 019/20:[down] 'wip: backup: qemu: Wire up qemu full push backup commands over QMP' 020/20:[0022] [FC] 'backup: implement qemu incremental pull backup' Eric Blake (20): domain_conf: Expose virDomainStorageNetworkParseHost qemu: clean up qemuDomainRemoveInactiveCommon backup: Document nuances between different state capture APIs backup: Introduce virDomainCheckpointPtr wip: backup: Document new XML for backups backup: Introduce virDomainCheckpoint APIs backup: Introduce virDomainBackup APIs backup: Add new domain:checkpoint access control backup: Implement backup APIs for remote driver wip: backup: Parse and output checkpoint XML wip: backup: Parse and output backup XML backup: Implement virsh support for checkpoints wip: backup: virsh support for backup backup: Add new qemu monitor interactions backup: qemu: Implement metadata tracking for checkpoint APIs wip: backup: Wire up qemu checkpoint commands over QMP wip: backup: qemu: Implement framework for backup job APIs backup: Wire up qemu full pull backup commands over QMP wip: backup: qemu: Wire up qemu full push backup commands over QMP backup: implement qemu incremental pull backup include/libvirt/virterror.h | 6 +- src/util/virerror.c | 12 +- include/libvirt/libvirt-domain-checkpoint.h | 161 ++ include/libvirt/libvirt-domain.h | 48 +- include/libvirt/libvirt.h | 3 +- src/access/viraccessperm.h | 8 +- src/conf/checkpoint_conf.h | 237 +++ src/conf/domain_conf.h | 17 + src/conf/virdomainobjlist.h | 9 +- src/datatypes.h | 31 +- src/driver-hypervisor.h | 74 +- src/qemu/qemu_block.h | 3 + src/qemu/qemu_blockjob.h | 1 + src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.h | 32 +- src/qemu/qemu_monitor.h | 21 +- src/qemu/qemu_monitor_json.h | 19 +- tools/virsh-checkpoint.h | 29 + tools/virsh-completer.h | 4 + tools/virsh-util.h | 3 + tools/virsh.h | 1 + docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 10 +- docs/domainstatecapture.html.in | 314 +++ docs/format.html.in | 2 + docs/formatbackup.html.in | 123 ++ docs/formatcheckpoint.html.in | 188 ++ docs/formatsnapshot.html.in | 2 + docs/index.html.in | 4 +- docs/schemas/domainbackup.rng | 185 ++ docs/schemas/domaincheckpoint.rng | 94 + examples/object-events/event-test.c | 3 + libvirt.spec.in | 3 + mingw-libvirt.spec.in | 6 + po/POTFILES | 3 + src/Makefile.am | 2 + src/access/viraccessperm.c | 5 +- src/conf/Makefile.inc.am | 2 + src/conf/checkpoint_conf.c | 1743 +++++++++++++++++ src/conf/domain_conf.c | 73 +- src/conf/virdomainobjlist.c | 12 +- src/datatypes.c | 63 +- src/libvirt-domain-checkpoint.c | 741 +++++++ src/libvirt-domain.c | 217 +- src/libvirt_private.syms | 33 + src/libvirt_public.syms | 19 + src/qemu/qemu_block.c | 12 + src/qemu/qemu_capabilities.c | 6 + src/qemu/qemu_conf.c | 5 + src/qemu/qemu_domain.c | 276 ++- src/qemu/qemu_driver.c | 1661 +++++++++++++++- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_monitor.c | 65 +- src/qemu/qemu_monitor_json.c | 199 +- src/qemu/qemu_process.c | 7 + src/remote/remote_daemon_dispatch.c | 22 +- src/remote/remote_driver.c | 33 +- src/remote/remote_protocol.x | 239 ++- src/remote_protocol-structs | 129 ++ src/rpc/gendispatch.pl | 32 +- tests/Makefile.am | 15 +- tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlout/empty.xml | 10 + tests/domaincheckpointxml2xmlout/sample.xml | 16 + tests/domaincheckpointxml2xmltest.c | 233 +++ .../caps_4.0.0.riscv32.xml | 2 + .../caps_4.0.0.riscv64.xml | 2 + tests/qemumonitorjsontest.c | 2 +- tests/virschematest.c | 4 + tools/Makefile.am | 3 +- tools/virsh-checkpoint.c | 1377 +++++++++++++ tools/virsh-completer.c | 52 +- tools/virsh-domain-monitor.c | 25 +- tools/virsh-domain.c | 277 ++- tools/virsh-util.c | 11 + tools/virsh.c | 2 + tools/virsh.pod | 272 ++- 86 files changed, 9510 insertions(+), 110 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 src/conf/checkpoint_conf.h create mode 100644 tools/virsh-checkpoint.h create mode 100644 docs/domainstatecapture.html.in create mode 100644 docs/formatbackup.html.in create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 src/conf/checkpoint_conf.c create mode 100644 src/libvirt-domain-checkpoint.c create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml create mode 100644 tests/domaincheckpointxml2xmltest.c create mode 100644 tools/virsh-checkpoint.c -- 2.20.1

2 29

[libvirt] [PATCH v2 00/14] qemu_hotplug: refactor device detach functions to fix one serious and other minor bugs
by Laine Stump 26 Mar '19

26 Mar '19

V1: https://www.redhat.com/archives/libvir-list/2019-March/msg01455.html I pushed patches 01-12, since they were acked, and pushing them leaves the code in no worse state than it was before (all tests still pass). There are 4 new patches in V2, created as a result of Peter asking for patches to be split, or for further refactoring. Still to be done: Peter pointed out that qemuDomainUpdateDeviceLlist() really shouldn't be called until after qemu has sent the DEVICE_DELETED event. That is on my list of followups, but the patches here are still a valid improvement even without that (After his comment, I noticed that we are also calling virDomainSaveStatus too soon - it also shouldn't be called until after DEVICE_DELETED). Laine Stump (14): qemu_hotplug: move qemuDomainDetachDeviceLive() to qemu_hotplug.c qemu_hotplug: remove extra function in middle of DetachController call chain qemu_hotplug: pull qemuDomainUpdateDeviceList out of qemuDomainDetachDeviceLive test: replace calls to individual detach functions with one call to main detach qemu_hotplug: make Detach functions called only from qemu_hotplug.c static qemu_hotplug: rename dev to match in qemuDomainDetachDeviceLive qemu_hotplug: separate Chr|Lease from other devices in DetachDevice switch qemu_hotplug: standardize the names/args/calling of qemuDomainDetach*() qemu_hotplug: rename Chr and Lease Detach functions qemu_hotplug: new function qemuDomainRemoveAuditDevice() qemu_hotplug: audit *all* auditable device types in qemuDomainRemoveAuditDevice qemu_hotplug: consolidate all common detach code in qemuDomainDetachDeviceLive qemu_hotplug: delay sending DEVICE_REMOVED event until after *all* teardown qemu_hotplug: don't shutdown net device until the guest has released it src/qemu/qemu_driver.c | 109 +---- src/qemu/qemu_hotplug.c | 919 +++++++++++++++++++--------------------- src/qemu/qemu_hotplug.h | 56 +-- tests/qemuhotplugtest.c | 8 +- 4 files changed, 465 insertions(+), 627 deletions(-) -- 2.20.1

3 34