Re: [libvirt] QEMU migration with non-shared storage
On Thu, 11 Sep 2014, Blair Bethwaite wrote:
A related problem arises mixing RBD and local disks when
drive-mirroring / block-migrating - the RBDs get migrated too, volume
round trips out of ceph -> into source hypervisor -> into dest
hypervisor -> back into ceph. Not great!
This patch fixes that issue (but thus far I've only tested against
1.1.1 on Precise - will do with 1.2 on Trusty next week)
=====
qemuMigrationDriveMirror(virQEMUDriverPt
virDomainBlockJobInfo info;
/* skip shared, RO and source-less disks */
- if (disk->shared || disk->readonly || !disk->src)
+ if (disk->shared || disk->readonly || !disk->src ||
+ (disk->type == VIR_DOMAIN_DISK_TYPE_NETWORK &&
+ disk->protocol == VIR_DOMAIN_DISK_PROTOCOL_RBD))
continue;
VIR_FREE(diskAlias);
=====
So interested in feedback on this and whether it should be pushed up...
I think you would need a corresponding change in
qemuMigrationStartNBDServer as well.
This change would largely solve the problems I'm encountering, but it's
still annoying having to use VIR_MIGRATE_UNSAFE just so that libvirt
doesn't complain about the non-shared disks, even though they're going to
be explicitly copied from the source to destination VM.
I'm going to see if I can find out why RBD is being treated differently
from other disk backends. As I said before, I can't see anything in QEMU
that indicates it behaves differently from the others, and indeed it
appears that *all* backends should be safe since (as far as I can tell)
they're explicitly flushed when the VM is paused. Maybe at the time that
RBD was added to QEMU and libvirt, QEMU only did this explicit flushing
for RBD.
It seems to me that libvirt is also probably going to need some more
detailed logic on deciding when a disk device should be copied or not
(with the understanding that when NBD is *not* being used, that decision
is ultimately up to QEMU only). If we have exceptional cases like RBD,
then the decision can't be made purely according to the shared flag.
- Michael