[libvirt] [PATCH/RFC] Introduce VIR_MIGRATE_FORCE flag to allow for risky migration
Hi, Migration will be disallowed when the vm uses host devices or has snapshots (qemuMigrationIsAllowed)[1]. Would it make sense to introduce a VIR_MIGRATE_FORCE similar to VIR_REVERT_FORCE here? We could then introduce error codes similar to the snapshot case (VIR_ERR_MIGRATE_RISKY). This path is just to illustrate the idea not to be applied as is. Cheers, -- Guido [1] Hopefully we can make migration with snapshots safe in the future by transfering the metadata. A current hack around is to put this onto shared storage and reread it on the destination side after migration.
On Fri, Oct 21, 2011 at 08:16:24PM +0200, Guido Günther wrote:
Hi, Migration will be disallowed when the vm uses host devices or has snapshots (qemuMigrationIsAllowed)[1]. Would it make sense to introduce a VIR_MIGRATE_FORCE similar to VIR_REVERT_FORCE here? We could then introduce error codes similar to the snapshot case (VIR_ERR_MIGRATE_RISKY).
I'm not sure this will actually work out in practice because QEMU itself also checks some of these scenarios and blocks them. So even if libvirt didn't check, the user still wouldnt' be able to force it to migrate. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 10/24/2011 02:00 AM, Daniel P. Berrange wrote:
On Fri, Oct 21, 2011 at 08:16:24PM +0200, Guido Günther wrote:
Hi, Migration will be disallowed when the vm uses host devices or has snapshots (qemuMigrationIsAllowed)[1]. Would it make sense to introduce a VIR_MIGRATE_FORCE similar to VIR_REVERT_FORCE here? We could then introduce error codes similar to the snapshot case (VIR_ERR_MIGRATE_RISKY).
I'm not sure this will actually work out in practice because QEMU itself also checks some of these scenarios and blocks them. So even if libvirt didn't check, the user still wouldnt' be able to force it to migrate.
That's true for hostdev passthrough (qemu refuses that, because you don't have the same hostdevs on the destination), but not so for snapshots (where right now, the only reason we don't permit it is due to a lack of implementation in libvirt - it has no bearing on qemu). Regarding the scenario of snapshot metadata, the biggest problem is that v3 cookies are not large enough to send the description of each snapshot in one rpc call. I've been thinking about that some more; it may be possible to use migrate v3 to send the migration after all, by adding the following to the cookies: in Begin, an fdstream is opened, then cookie includes details about the fdstream identifier. Then that fdstream is used to send a count of the total number of snapshots, followed by a length of each snapshot then the xml for that snapshot. Thus, the cookie is used to set up a second channel between source and destination, where that channel has a defined format for passing an arbitrary amount of data needed to reconstruct the snapshot hierarchy on the destination. I don't know if the fdstream can be run in parallel with the rest of the migration, or if it should be completed prior to the rest of the Prepare steps; at any rate, coordinating overall success or any failures on receiving the fdstream will have to be communicated from the destination back to the source in another cookie. But if we can teach migration v3 to send snapshots, then it might mean that we don't need VIR_MIGRATE_FORCE after all; the only place where snapshots would prevent migration is when either side of the equation doesn't know the new cookie, but those are the same situations where a new flag would not be recognized to have any effect. Besides, you can still manage snapshot migration manually, (albiet painfully), via a series of snapshot-dumpxml on the source, then snapshot-create --redefine on the destination. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On Mon, Oct 24, 2011 at 04:12:08PM -0600, Eric Blake wrote:
On 10/24/2011 02:00 AM, Daniel P. Berrange wrote:
On Fri, Oct 21, 2011 at 08:16:24PM +0200, Guido Günther wrote:
Hi, Migration will be disallowed when the vm uses host devices or has snapshots (qemuMigrationIsAllowed)[1]. Would it make sense to introduce a VIR_MIGRATE_FORCE similar to VIR_REVERT_FORCE here? We could then introduce error codes similar to the snapshot case (VIR_ERR_MIGRATE_RISKY).
I'm not sure this will actually work out in practice because QEMU itself also checks some of these scenarios and blocks them. So even if libvirt didn't check, the user still wouldnt' be able to force it to migrate.
That's true for hostdev passthrough (qemu refuses that, because you don't have the same hostdevs on the destination), but not so for snapshots (where right now, the only reason we don't permit it is due to a lack of implementation in libvirt - it has no bearing on qemu).
Regarding the scenario of snapshot metadata, the biggest problem is that v3 cookies are not large enough to send the description of each snapshot in one rpc call. I've been thinking about that some more; it may be possible to use migrate v3 to send the migration after all, by adding the following to the cookies:
in Begin, an fdstream is opened, then cookie includes details about the fdstream identifier. Then that fdstream is used to send a count of the total number of snapshots, followed by a length of each snapshot then the xml for that snapshot. Thus, the cookie is used to set up a second channel between source and destination, where that channel has a defined format for passing an arbitrary amount of data needed to reconstruct the snapshot hierarchy on the destination. I don't know if the fdstream can be run in parallel with the rest of the migration, or if it should be completed prior to the rest of the Prepare steps; at any rate, coordinating overall success or any failures on receiving the fdstream will have to be communicated from the destination back to the source in another cookie.
But if we can teach migration v3 to send snapshots, then it might mean that we don't need VIR_MIGRATE_FORCE after all; the only place where snapshots would prevent migration is when either side of the equation doesn't know the new cookie, but those are the same situations where a new flag would not be recognized to have any effect. Besides, you can still manage snapshot migration manually, (albiet painfully), via a series of snapshot-dumpxml on the source, then snapshot-create --redefine on the destination.
I can't help feeling that this is getting ridiculously complicated, which in turn makes me question whether it is really worth doing. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Mon, Oct 24, 2011 at 04:12:08PM -0600, Eric Blake wrote:
On 10/24/2011 02:00 AM, Daniel P. Berrange wrote:
On Fri, Oct 21, 2011 at 08:16:24PM +0200, Guido Günther wrote:
Hi, Migration will be disallowed when the vm uses host devices or has snapshots (qemuMigrationIsAllowed)[1]. Would it make sense to introduce a VIR_MIGRATE_FORCE similar to VIR_REVERT_FORCE here? We could then introduce error codes similar to the snapshot case (VIR_ERR_MIGRATE_RISKY).
I'm not sure this will actually work out in practice because QEMU itself also checks some of these scenarios and blocks them. So even if libvirt didn't check, the user still wouldnt' be able to force it to migrate.
That's true for hostdev passthrough (qemu refuses that, because you don't have the same hostdevs on the destination), but not so for snapshots (where right now, the only reason we don't permit it is due to a lack of implementation in libvirt - it has no bearing on qemu).
Regarding the scenario of snapshot metadata, the biggest problem is that v3 cookies are not large enough to send the description of each snapshot in one rpc call. I've been thinking about that some more; it may be possible to use migrate v3 to send the migration after all, by adding the following to the cookies:
in Begin, an fdstream is opened, then cookie includes details about the fdstream identifier. Then that fdstream is used to send a count of the total number of snapshots, followed by a length of each snapshot then the xml for that snapshot. Thus, the cookie is used to set up a second channel between source and destination, where that channel has a defined format for passing an arbitrary amount of data needed to reconstruct the snapshot hierarchy on the destination. I don't know if the fdstream can be run in parallel with the rest of the migration, or if it should be completed prior to the rest of the Prepare steps; at any rate, coordinating overall success or any failures on receiving the fdstream will have to be communicated from the destination back to the source in another cookie.
But if we can teach migration v3 to send snapshots, then it might mean that we don't need VIR_MIGRATE_FORCE after all; the only place where snapshots would prevent migration is when either side of the equation doesn't know the new cookie, but those are the same situations where a new flag would not be recognized to have any effect. Besides, you can still manage snapshot migration manually, (albiet painfully), via a series of snapshot-dumpxml on the source, then snapshot-create --redefine on the destination.
Another solution (for filesystem backed storage pools) would be to put the snapshot information next to the qcow2 images. We'd then have the snapshot information available on the remote side (when not using --copy-storage) and could just reread that information on the other host. I've done this by putting /var/lib/libvirt/qemu/snapshot there and it works out pretty well with some minor modifications. Would that be a scenario worth supporting? Cheers, -- Guido
participants (3)
-
Daniel P. Berrange -
Eric Blake -
Guido Günther