[libvirt-users] Live migration with non-shared ZFS volume

Hi all, I'm using ZFS on Linux block volumes as my VM storage and want to do live migrations between hypervisors. If I create ZFS snapshot of used volume on source host, send it do destination host (zfs send/recv) and then run live migration with VIR_MIGRATE_NON_SHARED_DISK flag, the migration works OK. But this procedure copies the whole disk twice which is a huge downside. The best solution would be, if libvirt could send the incremental data since last snapshot itself but this feature is not there (AFAIK). So I am thinking about a workaround: 1. Create snapshot using: "virsh snapshot-create --xmlfile snap.xml --disk-only --no-metadata test-domain" which will start writing snapshot data into temporary qcow2 file <domainsnapshot> <disks> <disk name='/dev/zstore/test-volume'> <source file='/tmp/test-volume.qcow2'/> </disk> </disks> </domainsnapshot> 2. Create snapshot of backing ZFS volume and send it to destination host. 3. Migrate the domain Currently, in step 3 I need to create empty qcow snapshot file on the destination host, otherwise the migration fails with: "Operation not supported: pre-creation of storage targets for incremental storage migration is not supported" My question is: Is it possible to do live migration with blockcommit operation? If not, would it be hard to implement? I imagine it like I would start migration with some special parameter (e.g. VIR_MIGRATE_NON_SHARED_INC_COMMIT) which would only migrate data from qcow snapshot to destination storage. This would ensure the disk consistency and avoid useless whole disk copy. Or do you have any other idea how to solve this? BR. Daniel Kucera.