1:17 p.m.
The following is the latest version of my API proposal for
incremental backups, and follows along from the demo I presented
as part of my KVM Forum 2018 talk:
https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup...
The patches are also available via https://repo.or.cz/libvirt/ericb.git
at the tag backup-v4.
Integration with external snapshots is still not included in these
patches, although I have been playing more with that locally, and
I still haven't gotten a working demonstration of a push-mode
incremental backup.
At this point, I'm posting mainly because there have been enough
changes in qemu (the backup APIs are now stable! and in qemu.git
in time for 4.0) and libvirt (several releases and code cleanups
in between, such that the v3 no longer applies easily), while I
continue to work locally on more features, addressing the review
comments I got on v3, and remove the 'wip' tag on several of the
later patches.
Among other things, I still haven't determined how we want to
integrate checkpoints with external snapshots; we could either have:
virDomainSnapshotCreateXML("<domainsnapshot>...") # existing
virDomainBackupBegin("<domainbackup>...",
"<domaincheckpoint>...") # this series
virDomainSnapshotCheckpointCreateXML("<domainsnapshot>...",
"<domaincheckpoint>...") # new
or to make checkpoint creation part of the snapshot and backup XML, as in:
virDomainSnapshotCreateXML("<domainsnapshot><domaincheckpoint>...</domainsnapshot>")
# extension of existing
virDomainBackupBegin("<domainbackup><domaincheckpoint>...</domainbackup>")
# tweak to this series
I'm also planning to add an API to query which job ids are currently
running (right now, the code only supports one job at a time, and
always calls it job 1, but that is not future-friendly) and an update
to the redefine XML command to make it easier to redefine multiple
checkpoints in one API call.
Since v3:
001/20:[----] [--] 'snapshots: Avoid term 'checkpoint' for full system
snapshot'
002/20:[----] [--] 'domain_conf: Expose virDomainStorageNetworkParseHost'
003/20:[----] [--] 'qemu: Allow optional export name during NBD export'
004/20:[----] [--] 'backup: Document nuances between different state capture
APIs'
005/20:[0032] [FC] 'backup: Introduce virDomainCheckpointPtr'
006/20:[----] [--] 'backup: Document new XML for backups'
007/20:[0013] [FC] 'backup: Introduce virDomainCheckpoint APIs'
008/20:[0017] [FC] 'backup: Introduce virDomainBackup APIs'
009/20:[----] [--] 'backup: Add new domain:checkpoint access control'
010/20:[0104] [FC] 'backup: Implement backup APIs for remote driver'
011/20:[0034] [FC] 'wip: backup: Parse and output checkpoint XML'
012/20:[0013] [FC] 'wip: backup: Parse and output backup XML'
013/20:[0006] [FC] 'backup: Implement virsh support for checkpoints'
014/20:[----] [--] 'wip: backup: virsh support for backup'
015/20:[0126] [FC] 'backup: Add new qemu monitor interactions'
016/20:[0072] [FC] 'backup: qemu: Implement metadata tracking for checkpoint
APIs'
017/20:[0069] [FC] 'wip: backup: Wire up qemu checkpoint commands over QMP'
018/20:[0018] [FC] 'wip: backup: qemu: Implement framework for backup job APIs'
019/20:[0045] [FC] 'backup: Wire up qemu full pull backup commands over QMP'
020/20:[0051] [FC] 'backup: implement qemu incremental pull backup'
Eric Blake (20):
snapshots: Avoid term 'checkpoint' for full system snapshot
domain_conf: Expose virDomainStorageNetworkParseHost
qemu: Allow optional export name during NBD export
backup: Document nuances between different state capture APIs
backup: Introduce virDomainCheckpointPtr
backup: Document new XML for backups
backup: Introduce virDomainCheckpoint APIs
backup: Introduce virDomainBackup APIs
backup: Add new domain:checkpoint access control
backup: Implement backup APIs for remote driver
wip: backup: Parse and output checkpoint XML
wip: backup: Parse and output backup XML
backup: Implement virsh support for checkpoints
wip: backup: virsh support for backup
backup: Add new qemu monitor interactions
backup: qemu: Implement metadata tracking for checkpoint APIs
wip: backup: Wire up qemu checkpoint commands over QMP
wip: backup: qemu: Implement framework for backup job APIs
backup: Wire up qemu full pull backup commands over QMP
backup: implement qemu incremental pull backup
include/libvirt/virterror.h | 7 +-
src/util/virerror.c | 12 +-
include/libvirt/libvirt-domain-checkpoint.h | 176 ++
include/libvirt/libvirt-domain-snapshot.h | 2 +-
include/libvirt/libvirt-domain.h | 17 +-
include/libvirt/libvirt.h | 3 +-
src/access/viraccessperm.h | 8 +-
src/conf/checkpoint_conf.h | 215 +++
src/conf/domain_conf.h | 16 +-
src/datatypes.h | 31 +-
src/driver-hypervisor.h | 74 +-
src/qemu/qemu_block.h | 3 +
src/qemu/qemu_blockjob.h | 1 +
src/qemu/qemu_capabilities.h | 2 +
src/qemu/qemu_conf.h | 2 +
src/qemu/qemu_domain.h | 29 +-
src/qemu/qemu_monitor.h | 21 +-
src/qemu/qemu_monitor_json.h | 19 +-
tools/virsh-checkpoint.h | 29 +
tools/virsh-completer.h | 4 +
tools/virsh-util.h | 3 +
tools/virsh.h | 1 +
docs/Makefile.am | 3 +
docs/apibuild.py | 2 +
docs/docs.html.in | 9 +-
docs/domainstatecapture.html.in | 314 ++++
docs/format.html.in | 1 +
docs/formatcheckpoint.html.in | 291 +++
docs/formatsnapshot.html.in | 33 +-
docs/index.html.in | 3 +-
docs/schemas/domainbackup.rng | 185 ++
docs/schemas/domaincheckpoint.rng | 94 +
examples/object-events/event-test.c | 3 +
libvirt.spec.in | 3 +
mingw-libvirt.spec.in | 6 +
po/POTFILES | 3 +
src/Makefile.am | 2 +
src/access/viraccessperm.c | 5 +-
src/conf/Makefile.inc.am | 2 +
src/conf/checkpoint_conf.c | 1533 ++++++++++++++++
src/conf/domain_conf.c | 52 +-
src/conf/snapshot_conf.c | 4 +-
src/datatypes.c | 62 +-
src/libvirt-domain-checkpoint.c | 925 ++++++++++
src/libvirt-domain-snapshot.c | 7 +-
src/libvirt-domain.c | 8 +-
src/libvirt_private.syms | 30 +
src/libvirt_public.syms | 23 +
src/qemu/qemu_block.c | 12 +
src/qemu/qemu_capabilities.c | 4 +
src/qemu/qemu_conf.c | 5 +
src/qemu/qemu_domain.c | 217 ++-
src/qemu/qemu_driver.c | 1552 +++++++++++++++-
src/qemu/qemu_migration.c | 2 +-
src/qemu/qemu_monitor.c | 65 +-
src/qemu/qemu_monitor_json.c | 197 +-
src/qemu/qemu_process.c | 7 +
src/remote/remote_daemon_dispatch.c | 22 +-
src/remote/remote_driver.c | 33 +-
src/remote/remote_protocol.x | 238 ++-
src/remote_protocol-structs | 129 ++
src/rpc/gendispatch.pl | 32 +-
tests/Makefile.am | 15 +-
tests/domainbackupxml2xmlin/backup-pull.xml | 9 +
tests/domainbackupxml2xmlin/backup-push.xml | 9 +
tests/domainbackupxml2xmlin/empty.xml | 1 +
tests/domainbackupxml2xmlout/backup-pull.xml | 9 +
tests/domainbackupxml2xmlout/backup-push.xml | 9 +
tests/domainbackupxml2xmlout/empty.xml | 7 +
tests/domaincheckpointxml2xmlin/empty.xml | 1 +
tests/domaincheckpointxml2xmlin/sample.xml | 7 +
tests/domaincheckpointxml2xmlout/empty.xml | 10 +
tests/domaincheckpointxml2xmlout/sample.xml | 16 +
tests/domaincheckpointxml2xmltest.c | 231 +++
.../caps_4.0.0.riscv32.xml | 2 +
.../caps_4.0.0.riscv64.xml | 2 +
tests/qemumonitorjsontest.c | 2 +-
tests/virschematest.c | 4 +
tools/Makefile.am | 3 +-
tools/virsh-checkpoint.c | 1578 +++++++++++++++++
tools/virsh-completer.c | 52 +-
tools/virsh-domain.c | 8 +-
tools/virsh-snapshot.c | 2 +-
tools/virsh-util.c | 11 +
tools/virsh.c | 2 +
tools/virsh.pod | 14 +-
86 files changed, 8674 insertions(+), 123 deletions(-)
create mode 100644 include/libvirt/libvirt-domain-checkpoint.h
create mode 100644 src/conf/checkpoint_conf.h
create mode 100644 tools/virsh-checkpoint.h
create mode 100644 docs/domainstatecapture.html.in
create mode 100644 docs/formatcheckpoint.html.in
create mode 100644 docs/schemas/domainbackup.rng
create mode 100644 docs/schemas/domaincheckpoint.rng
create mode 100644 src/conf/checkpoint_conf.c
create mode 100644 src/libvirt-domain-checkpoint.c
create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlin/empty.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
create mode 100644 tests/domainbackupxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml
create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml
create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml
create mode 100644 tests/domaincheckpointxml2xmltest.c
create mode 100644 tools/virsh-checkpoint.c
--
2.20.1