1:22 a.m.
Hello everybody,
I am trying to understand how I can create a setup with two libvirt + kvm +
zfs systems, in which I can:
1. Create a domain on sys-a, with storage space on a zfs filesystem. This
filesystem stores the qcow2 image, any qcow2 disk snapshots, memory dumps
and domain XML definition.
2. Take consistent ZFS snapshots every 15 minutes, and sync them to sys-b.
The snapshots will be replicated using syncoid. These will be using scripts
to quiesce the VM.
3. Given that most of the disk is synced (at most 15 minutes ago), what are
the things I need to ensure (libvirt snapshots, memory snapshot, etc), so
that I can do a live migration while copying just the incremental changes
since the last snapshot I already transferred?
I was trying to use virsh migrate —copy-storage-inc, however I could not
find any working example that would copy just the incremental changes. I
could find the theory of the snapshot backing chain, however
—copy-storage-inc complains that it cannot replicate while there are
snapshots present. (error: Requested operation is not valid: cannot migrate
domain with 1 snapshots).
I this scenario possible?
Dan Vatca
2:50 p.m.
New subject: How to live migrate with copy-storage-inc and zfs snapshot
replication?
On Tue, May 21, 2024 at 09:22:44 +0300, Dan Vatca via Users wrote:
Hello everybody,
I am trying to understand how I can create a setup with two libvirt + kvm +
zfs systems, in which I can:
1. Create a domain on sys-a, with storage space on a zfs filesystem. This
filesystem stores the qcow2 image, any qcow2 disk snapshots, memory dumps
and domain XML definition.
2. Take consistent ZFS snapshots every 15 minutes, and sync them to sys-b.
The snapshots will be replicated using syncoid. These will be using scripts
to quiesce the VM.
3. Given that most of the disk is synced (at most 15 minutes ago), what are
the things I need to ensure (libvirt snapshots, memory snapshot, etc), so
that I can do a live migration while copying just the incremental changes
since the last snapshot I already transferred?
I was trying to use virsh migrate —copy-storage-inc, however I could not
find any working example that would copy just the incremental changes. I
could find the theory of the snapshot backing chain, however
Was this theory involving creation of an external snapshot (new qcow2
overlay), then ensuring that the original image is replicated to the
other host and then using migration with incremental copy?
If yes, the theory is correct, but requires creating the snapshot,
without actual snapshot metadata (just creating the qcow2 overlay).
This is done via 'virsh snapshot-create --no-metadata'. This does
everything to do the snapshot, but doesn't actually record it in the
list of snapshots ...
Alternatively you can delete the sanpshot metadata if you have them
already around (virsh snapshot-delete --metadata).
—copy-storage-inc complains that it cannot replicate while there are
snapshots present. (error: Requested operation is not valid: cannot migrate
domain with 1 snapshots).
... bypassing this check. Now this check has some history behind it, the
gist being that libvirt doesn't actually copy the metadata over, so any
snapshots are unusable on the destination.
Additionally when using --copy-storage-all the disk image chain is
flattened, thus snapshots are not preserved.
Reallistically libvirt should reconsider this check, which is more than
10 years old, as it in certain cases it'd make sense to migrate the VM
even without preserving snapshot state. Or in cases of shared storage
replicating the snapshot metadata would certainly improve usability.
215
days inactive
215
days old
1 comments
2 participants
participants (2)
-
Dan Vatca -
Peter Krempa