11:47 p.m.
The following is the latest version of my API proposal for
incremental backups, and follows along from the demo I presented
as part of my KVM Forum 2018 talk:
https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup...
The patches are also available via https://repo.or.cz/libvirt/ericb.git
at the tag backup-v5.
Depends on my earlier bulk redefine for snapshots, which is pending
review at v3:
https://www.redhat.com/archives/libvir-list/2019-March/msg00144.html
I haven't finished addressing all of John Ferlan's review comments
from v4 (including the fact that several of these larger patches
need to be split for easier reviews, and the fact that I want
virDomainSnapshotObj and virDomainCheckpointObj to share a common
base class); but I'm posting this now because I _have_ completed:
- push mode backups mostly work (event notification is still not
polished, but the fact that I can now do both push and pull modes
is nice)
- bulk mode list/redefine, accessible via 'virsh dumpxml --checkpoints'
and 'virsh snapshot-create --redefine-list'
- move virDomainBackup() into libvirt-domain.h (libvirt-domain-checkpoint.h
is now JUST about standalone checkpoint objects; then both virDomainBackup()
and virDomainSnapshotCreateXML() will be able to create a checkpoint
in parallel)
- rebase to master (5.2 now instead of 5.1, start using VIR_AUTOFREE
in more places, ...)
I'm still debating about making <domaincheckpoint> a sub-element of
<domainbackup> (in which case I can reuse virDomainSnapshotCreateXML()
to create a backup) or keeping it as a separate XML argument (in which
case I need to add an API for virDomainSnapshotCreateXMLWithSnapshot()).
I also need to work out an API for listing all currently running jobs
(even if the current backup code can only run one job at a time, we know
we want support in the future for parallel backup jobs). So there WILL
be a v6. Hopefully, though, we can start trimming off the front of the
series, and at least now the tail of the series currently compiles (John
gave up review on v4 when he ran into rebase issues - although with our
current rate of VIR_AUTOFREE conversions, there may again be rebase
issues soon...)
since v4:
001/20:[----] [-C] 'domain_conf: Expose virDomainStorageNetworkParseHost'
002/20:[down] 'qemu: clean up qemuDomainRemoveInactiveCommon'
003/20:[----] [--] 'backup: Document nuances between different state capture
APIs'
004/20:[0020] [FC] 'backup: Introduce virDomainCheckpointPtr'
005/20:[down] 'wip: backup: Document new XML for backups'
006/20:[0054] [FC] 'backup: Introduce virDomainCheckpoint APIs'
007/20:[0053] [FC] 'backup: Introduce virDomainBackup APIs'
008/20:[0002] [FC] 'backup: Add new domain:checkpoint access control'
009/20:[0093] [FC] 'backup: Implement backup APIs for remote driver'
010/20:[0381] [FC] 'wip: backup: Parse and output checkpoint XML'
011/20:[0001] [FC] 'wip: backup: Parse and output backup XML'
012/20:[0393] [FC] 'backup: Implement virsh support for checkpoints'
013/20:[0058] [FC] 'wip: backup: virsh support for backup'
014/20:[0019] [FC] 'backup: Add new qemu monitor interactions'
015/20:[0161] [FC] 'backup: qemu: Implement metadata tracking for checkpoint
APIs'
016/20:[----] [-C] 'wip: backup: Wire up qemu checkpoint commands over QMP'
017/20:[0006] [FC] 'wip: backup: qemu: Implement framework for backup job APIs'
018/20:[0046] [FC] 'backup: Wire up qemu full pull backup commands over QMP'
019/20:[down] 'wip: backup: qemu: Wire up qemu full push backup commands over
QMP'
020/20:[0022] [FC] 'backup: implement qemu incremental pull backup'
Eric Blake (20):
domain_conf: Expose virDomainStorageNetworkParseHost
qemu: clean up qemuDomainRemoveInactiveCommon
backup: Document nuances between different state capture APIs
backup: Introduce virDomainCheckpointPtr
wip: backup: Document new XML for backups
backup: Introduce virDomainCheckpoint APIs
backup: Introduce virDomainBackup APIs
backup: Add new domain:checkpoint access control
backup: Implement backup APIs for remote driver
wip: backup: Parse and output checkpoint XML
wip: backup: Parse and output backup XML
backup: Implement virsh support for checkpoints
wip: backup: virsh support for backup
backup: Add new qemu monitor interactions
backup: qemu: Implement metadata tracking for checkpoint APIs
wip: backup: Wire up qemu checkpoint commands over QMP
wip: backup: qemu: Implement framework for backup job APIs
backup: Wire up qemu full pull backup commands over QMP
wip: backup: qemu: Wire up qemu full push backup commands over QMP
backup: implement qemu incremental pull backup
include/libvirt/virterror.h | 6 +-
src/util/virerror.c | 12 +-
include/libvirt/libvirt-domain-checkpoint.h | 161 ++
include/libvirt/libvirt-domain.h | 48 +-
include/libvirt/libvirt.h | 3 +-
src/access/viraccessperm.h | 8 +-
src/conf/checkpoint_conf.h | 237 +++
src/conf/domain_conf.h | 17 +
src/conf/virdomainobjlist.h | 9 +-
src/datatypes.h | 31 +-
src/driver-hypervisor.h | 74 +-
src/qemu/qemu_block.h | 3 +
src/qemu/qemu_blockjob.h | 1 +
src/qemu/qemu_capabilities.h | 4 +
src/qemu/qemu_conf.h | 2 +
src/qemu/qemu_domain.h | 32 +-
src/qemu/qemu_monitor.h | 21 +-
src/qemu/qemu_monitor_json.h | 19 +-
tools/virsh-checkpoint.h | 29 +
tools/virsh-completer.h | 4 +
tools/virsh-util.h | 3 +
tools/virsh.h | 1 +
docs/Makefile.am | 3 +
docs/apibuild.py | 2 +
docs/docs.html.in | 10 +-
docs/domainstatecapture.html.in | 314 +++
docs/format.html.in | 2 +
docs/formatbackup.html.in | 123 ++
docs/formatcheckpoint.html.in | 188 ++
docs/formatsnapshot.html.in | 2 +
docs/index.html.in | 4 +-
docs/schemas/domainbackup.rng | 185 ++
docs/schemas/domaincheckpoint.rng | 94 +
examples/object-events/event-test.c | 3 +
libvirt.spec.in | 3 +
mingw-libvirt.spec.in | 6 +
po/POTFILES | 3 +
src/Makefile.am | 2 +
src/access/viraccessperm.c | 5 +-
src/conf/Makefile.inc.am | 2 +
src/conf/checkpoint_conf.c | 1743 +++++++++++++++++
src/conf/domain_conf.c | 73 +-
src/conf/virdomainobjlist.c | 12 +-
src/datatypes.c | 63 +-
src/libvirt-domain-checkpoint.c | 741 +++++++
src/libvirt-domain.c | 217 +-
src/libvirt_private.syms | 33 +
src/libvirt_public.syms | 19 +
src/qemu/qemu_block.c | 12 +
src/qemu/qemu_capabilities.c | 6 +
src/qemu/qemu_conf.c | 5 +
src/qemu/qemu_domain.c | 276 ++-
src/qemu/qemu_driver.c | 1661 +++++++++++++++-
src/qemu/qemu_migration.c | 2 +-
src/qemu/qemu_monitor.c | 65 +-
src/qemu/qemu_monitor_json.c | 199 +-
src/qemu/qemu_process.c | 7 +
src/remote/remote_daemon_dispatch.c | 22 +-
src/remote/remote_driver.c | 33 +-
src/remote/remote_protocol.x | 239 ++-
src/remote_protocol-structs | 129 ++
src/rpc/gendispatch.pl | 32 +-
tests/Makefile.am | 15 +-
tests/domainbackupxml2xmlin/backup-pull.xml | 9 +
tests/domainbackupxml2xmlin/backup-push.xml | 9 +
tests/domainbackupxml2xmlin/empty.xml | 1 +
tests/domainbackupxml2xmlout/backup-pull.xml | 9 +
tests/domainbackupxml2xmlout/backup-push.xml | 9 +
tests/domainbackupxml2xmlout/empty.xml | 7 +
tests/domaincheckpointxml2xmlin/empty.xml | 1 +
tests/domaincheckpointxml2xmlin/sample.xml | 7 +
tests/domaincheckpointxml2xmlout/empty.xml | 10 +
tests/domaincheckpointxml2xmlout/sample.xml | 16 +
tests/domaincheckpointxml2xmltest.c | 233 +++
.../caps_4.0.0.riscv32.xml | 2 +
.../caps_4.0.0.riscv64.xml | 2 +
tests/qemumonitorjsontest.c | 2 +-
tests/virschematest.c | 4 +
tools/Makefile.am | 3 +-
tools/virsh-checkpoint.c | 1377 +++++++++++++
tools/virsh-completer.c | 52 +-
tools/virsh-domain-monitor.c | 25 +-
tools/virsh-domain.c | 277 ++-
tools/virsh-util.c | 11 +
tools/virsh.c | 2 +
tools/virsh.pod | 272 ++-
86 files changed, 9510 insertions(+), 110 deletions(-)
create mode 100644 include/libvirt/libvirt-domain-checkpoint.h
create mode 100644 src/conf/checkpoint_conf.h
create mode 100644 tools/virsh-checkpoint.h
create mode 100644 docs/domainstatecapture.html.in
create mode 100644 docs/formatbackup.html.in
create mode 100644 docs/formatcheckpoint.html.in
create mode 100644 docs/schemas/domainbackup.rng
create mode 100644 docs/schemas/domaincheckpoint.rng
create mode 100644 src/conf/checkpoint_conf.c
create mode 100644 src/libvirt-domain-checkpoint.c
create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlin/empty.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml
create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml
create mode 100644 tests/domaincheckpointxml2xmltest.c
create mode 100644 tools/virsh-checkpoint.c
--
2.20.1