[libvirt] [PATCH v7 00/23] Incremental backups

I've finished rebasing my earlier v4/v5 patches on top of cleanups earlier in the series (v6 didn't include the second half of the series). I've pushed a tag backup-v7 to both my libvirt.git and libvirt-python.git repos to match: https://repo.or.cz/libvirt/ericb.git/shortlog/refs/tags/backup-v7 https://repo.or.cz/libvirt-python/ericb.git/shortlog/refs/tags/backup-v7 Here's hoping we're happy enough with the API, including the fact that rudimentary operation of pull mode backups work with qemu 4.0-rc1, for this to make it into the 5.2 release. Diffs from v6 - couple more early easy cleanups - add Dan's R-b where appropriate - drop any attempt at virDomainGetJobIds(), instead: - document that job id 0 always works for the current backup job (if there is just one), which is just fine for now because qemu 4.0 doesn't permit parallel jobs (and we can do full job id support for a later release) - split some more patches; finish moving backup code to a new backup_conf.c and not piggybacked onto checkpoint_conf.c Eric Blake (23): snapshot: Drop pointless function virDomainMomentIsCurrentName snapshot: Allow NULL to virDomainSnapshotObjGetDef snapshot: Refactor qemu to utilize virDomainMoment more backup: Document new XML for checkpoints backup: Document new XML for backups backup: Introduce virDomainCheckpoint APIs backup: Introduce virDomainBackup APIs backup: Document nuances between different state capture APIs backup: Parse and output checkpoint XML backup: Allow for lists of checkpoint objects squash to checkpoint list backup: Add new domain:checkpoint access control backup: Implement backup APIs for remote driver backup: Parse and output backup XML backup: Implement virsh support for checkpoints backup: Implement virsh support for backup backup: Add new qemu monitor interactions backup: qemu: Implement metadata tracking for checkpoint APIs backup: Wire up qemu checkpoint commands over QMP backup: qemu: Implement framework for backup job APIs backup: Wire up qemu full pull backup commands over QMP backup: qemu: Wire up qemu full push backup commands over QMP backup: implement qemu incremental pull backup include/libvirt/libvirt-domain-checkpoint.h | 161 ++ include/libvirt/libvirt-domain-snapshot.h | 8 +- include/libvirt/libvirt-domain.h | 47 +- include/libvirt/libvirt.h | 5 +- src/access/viraccessperm.h | 8 +- src/conf/backup_conf.h | 96 + src/conf/checkpoint_conf.h | 103 ++ src/conf/domain_conf.h | 2 + src/conf/virconftypes.h | 12 + src/conf/virdomaincheckpointobjlist.h | 74 + src/conf/virdomainmomentobjlist.h | 9 +- src/conf/virdomainobjlist.h | 9 +- src/conf/virdomainsnapshotobjlist.h | 4 +- src/driver-hypervisor.h | 81 +- src/qemu/qemu_block.h | 3 + src/qemu/qemu_blockjob.h | 1 + src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.h | 39 +- src/qemu/qemu_monitor.h | 21 +- src/qemu/qemu_monitor_json.h | 21 +- tools/virsh-checkpoint.h | 29 + tools/virsh-completer.h | 4 + tools/virsh-util.h | 3 + tools/virsh.h | 1 + docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 10 +- docs/domainstatecapture.html.in | 315 ++++ docs/format.html.in | 2 + docs/formatbackup.html.in | 184 ++ docs/formatcheckpoint.html.in | 204 +++ docs/formatsnapshot.html.in | 2 + docs/index.html.in | 4 +- docs/schemas/domainbackup.rng | 219 +++ docs/schemas/domaincheckpoint.rng | 87 + examples/object-events/event-test.c | 3 + libvirt.spec.in | 3 + mingw-libvirt.spec.in | 6 + po/POTFILES | 3 + src/Makefile.am | 2 + src/access/viraccessperm.c | 5 +- src/conf/Makefile.inc.am | 6 + src/conf/backup_conf.c | 544 ++++++ src/conf/checkpoint_conf.c | 639 +++++++ src/conf/domain_conf.c | 8 +- src/conf/snapshot_conf.c | 2 +- src/conf/virdomaincheckpointobjlist.c | 222 +++ src/conf/virdomainmomentobjlist.c | 49 +- src/conf/virdomainobjlist.c | 13 +- src/conf/virdomainsnapshotobjlist.c | 11 +- src/libvirt-domain-checkpoint.c | 750 ++++++++ src/libvirt-domain-snapshot.c | 89 + src/libvirt-domain.c | 239 ++- src/libvirt_private.syms | 33 +- src/libvirt_public.syms | 20 + src/qemu/qemu_block.c | 12 + src/qemu/qemu_capabilities.c | 6 + src/qemu/qemu_conf.c | 5 + src/qemu/qemu_domain.c | 249 ++- src/qemu/qemu_driver.c | 1609 ++++++++++++++++- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_monitor.c | 65 +- src/qemu/qemu_monitor_json.c | 201 +- src/qemu/qemu_process.c | 7 + src/remote/remote_daemon_dispatch.c | 22 +- src/remote/remote_driver.c | 34 +- src/remote/remote_protocol.x | 259 ++- src/remote_protocol-structs | 139 ++ src/rpc/gendispatch.pl | 34 +- src/test/test_driver.c | 7 +- tests/Makefile.am | 13 +- tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/redefine.xml | 63 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlin/size.xml | 4 + tests/domaincheckpointxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlout/sample.xml | 12 + tests/domaincheckpointxml2xmlout/size.xml | 11 + tests/domaincheckpointxml2xmltest.c | 212 +++ .../caps_4.0.0.riscv32.xml | 2 + .../caps_4.0.0.riscv64.xml | 2 + .../caps_4.0.0.x86_64.xml | 2 + tests/qemumonitorjsontest.c | 2 +- tests/virschematest.c | 4 + tools/Makefile.am | 3 +- tools/virsh-checkpoint.c | 1370 ++++++++++++++ tools/virsh-completer.c | 53 +- tools/virsh-domain-monitor.c | 25 +- tools/virsh-domain.c | 270 ++- tools/virsh-snapshot.c | 37 +- tools/virsh-util.c | 11 + tools/virsh.c | 2 + tools/virsh.pod | 302 +++- 100 files changed, 9373 insertions(+), 163 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 src/conf/backup_conf.h create mode 100644 src/conf/checkpoint_conf.h create mode 100644 src/conf/virdomaincheckpointobjlist.h create mode 100644 tools/virsh-checkpoint.h create mode 100644 docs/domainstatecapture.html.in create mode 100644 docs/formatbackup.html.in create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/checkpoint_conf.c create mode 100644 src/conf/virdomaincheckpointobjlist.c create mode 100644 src/libvirt-domain-checkpoint.c create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/redefine.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlin/size.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/size.xml create mode 100644 tests/domaincheckpointxml2xmltest.c create mode 100644 tools/virsh-checkpoint.c -- 2.20.1