Re: [libvirt] [RFC v2] external (pull) backup API

Hi, all. This is another RFC on pull backup API. This API provides means to read domain disks in a snapshotted state so that client can back them up as well as means to write domain disks to revert them to backed up state. The previous version of RFC is [1]. I'll also describe the API implementation details to shed light on misc qemu dirty bitmap commands usage.

3. Checkpoints (the most interesting part) First a few facts about qemu dirty bitmaps. Bitmap can be either in active or disable state. In disabled state it does not get changed on guest writes. And oppositely in active state it tracks guest writes. This implementation uses approach with only one active bitmap at a time. This should reduce guest write penalties in the presence of checkpoints.

So on first snapshot we create bitmap B_1. Now it tracks changes from the snapshot 1. On second snapshot we create bitmap B_2 and disable bitmap B1 and so on. Now bitmap B1 keep changes from snaphost 1 to snapshot 2, B2 - changes from snaphot 2 to snapshot 3 and so on. Last bitmap is active and gets most disk change after latest snapshot. Getting changed blocks bitmap from some checkpoint in past till current snapshot is quite simple in this scheme. For example if the last snapshot is 7 then to get changes from snapshot 3 to latest snapshot we need to merge bitmaps B3, B4, B4 and B6. Merge is just logical OR on bitmap bits. Deleting a checkpoint somewhere in the middle of checkpoint sequence requires merging correspondent bitmap to the previous bitmap in this scheme. We use persitent bitmaps in the implementation. This means upon qemu process termination bitmaps are saved in disks images metadata and restored back on qemu process start.

This makes checkpoint a persistent property that is we keep them across domain start/stops. Qemu does not try hard to keep bitmaps. If upon save something goes wrong bitmap is dropped. The same is applied to the migration process too. For backup process it is not critical. If we don't discover a checkpoint we always can make a full backup. Also qemu provides no special means to track order of bitmaps. These facts are critical for implementation with one active bitmap at a time. We need right order of bitmaps upon merge - for snapshot N and block changes from snanpshot K, K < N to N we need to merge bitmaps B_{K}, ..., B_{N-1}. Also if one of the bitmaps to be merged is missing we can't calculate desired block changes too.

*Restore operation nuances* As it was written above to restore a domain one needs to start it in paused state, export domain's disks and write them from backup. However qemu currently does not let export disks for write even for a domain that never starts guests CPU. We have an experimental qemu command option -x-vz-nbd-restore (passed together with -incoming option) to fix it.

*Links* [1] Previous version of RFC https://www.redhat.com/archives/libvir-list/2017-November/msg00514.html