Re: [libvirt] [PATCH 4/8] backup: Document new XML for backups

On 06/26/2018 02:51 PM, Nir Soffer wrote: Matches what the existing formatsnapshot.html.in named its tag. (If you haven't guessed, I'm heavily relying on snapshots as my template for adding this). Okay, I will try to improve these in v2. But (other than answering these good review emails), my current priority is a working demo (to prove the API works) prior to further doc polish. My intent is that if the user does not assign a checkpoint name, then libvirt will default it to the current time in seconds-since-the-Epoch. Then, whatever name is given to the checkpoint (whether chosen by libvirt or assigned by the user) will also be the default name of the bitmap created in each qcow2 volume, but the XML also allows you to name the qcow2 bitmaps something different than the checkpoint name (maybe not a wise idea in the common case, but could come in handy later if you use the _REDEFINE flag to teach libvirt about existing bitmaps that are already present in a qcow2 image rather than placed there by libvirt). Because snapshots had an arbitrary tree, and it was easier to copy from snapshots. Even if we only use a linear tree for now, it is still feasible that in the future, we can facilitate a domain rolling back to the disk state as captured at checkpoint C1, at which point you could then have multiple children C2 (the bitmap created prior to rolling back) and C3 (the bitmap created for tracking changes made after rolling back). Again, for a first cut, I probably will punt and state that snapshots and incremental backups do not play well together yet; but as we get experience and add more code, the API is flexible enough that down the road we really can offer reverting to an arbitrary snapshot and ALSO updating checkpoints to match. Hmm - right now, you can't (that was one Snapshot API that I intentionally did not copy over to Checkpoint), so I should probably reword that. Because we didn't require the user to provide names for snapshots, and generating a name via the current timestamp is still fairly likely to be usable. Because if your guest uses all qcow2 images, and you don't want to exclude any images from the checkpoint, then not specifying <disks> does the right thing with less typing. Just because libvirt tries to have sane defaults doesn't mean you have to rely on them, though. For the same reason that you can support an internal snapshot referencing a qcow2 snapshot with a different name. Yeah, it's probably not a common usage, but there are cases (such as when using _REDEFINE) where it can prove invaluable. You're right that most users won't name qcow2 bitmaps differently from the libvirt checkpoint name. My worry is about future extensibility of the XML. If the XML is too simple, then we may lock ourselves into a corner at not being able to support some other backend implementation of checkpoints (just because qemu implements checkpoints via qcow2 bitmaps does not mean that some other hyperviser won't come along that implements checkpoints via a UUID, so I tried to leave room for <disk checkpoint='uuid' uuid='..-..-...'/> as potential XML for such a hypervisor mapping - and while bitmap names different from checkpoint names is unusual, it is much more likely that UUIDs for multiple disks would have to be different per disk) That would be an output-only XML element, and only if qemu were even modified to expose that information. But yes, I can see how exposing that could be useful. Copy-and-paste from snapshots, but in general, what it would take to start a new VM using a restoration of the backup images corresponding to that checkpoint (that is, the XML is the smaller persistent form, rather than the larger running form; my classic example used to be that the 'inactive domain configuration' omits <alias> tags while the 'running configuration' does not - but since libvirt recently added support for user-settable <alias> tags, that no longer holds...). Your incremental backup would get all changes since the point in time c2 was created (that is, the changes recorded by the merge of bitmaps c2 and c3). Your incremental backup would get all changes since the point in time c1 was created (that is, the changes recorded by the merge of bitmaps c1, c2, and c3). Qemu can't do that right now, so this API can't do it either. Maybe there's a way to add it into the API (and the fact that we used XML leaves that door wide open), but not right now. Or the current proposal of <incremental> serves as the 'from', and a new sibling element <limit> becomes the 'to', if it becomes possible to limit a backup to an earlier point in time than the present call to the API. Possible via the x-nbd-server-add-bitmap command added in qemu commit 767f0c7, coupled with a client that knows how to request NBD_OPT_SET_META_CONTEXT "qemu:dirty-bitmap:foo" then read the bitmap with NBD_CMD_BLOCK_STATUS (I have a hack patch sitting on the qemu list that lets qemu-img behave as such a client: https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg05993.html) Yes, except that the kernel NBD client plugin does not have support for NBD_CMD_BLOCK_STATUS, so reading /dev/nbd0 won't be able to find the dirty blocks. But you could always do it in two steps: first, connect a client that only reads the bitmap (such as qemu-img with my hack), then connect the kernel client so that you can stream just the portions of /dev/nbd0 referenced in the map of the first step. (Or, since both clients would be read-only, you can have them both connected to the qemu server at once) Ultimately, yes, I'd like to support gluster/NBD/sheepdog/... destinations. My initial implementation is less ambitious, and supports just local files (because those are easier to test and therefore produce a demo with). -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org