Daniel P. Berrange napsal(a):

On Mon, Oct 12, 2009 at 12:37:22PM +0200, Radek Hlad?k wrote:

...

Short question: What is the best way to run some script on VM migration?

Detailed description: We use own system for volume management. Its based on md and we found it very flexible. Basic idea is that we have md mirror everywhere we can. This suits great for libvirt: We have a disk on iSCSI server, its "partioned" using device mapper, the resulting "partition" is member of md raid1, that raid is exported via iSCSI to the libvirt node, it is on the node included in another md raid1 and that "final" md raid is used as storage by the VM. This provides high level of abstraction that allows us to migrate the storage to another disk, another iSCSI server, etc... The only glitch is on live migration of the VM. It is possible to assemble and run the array on two nodes. Also all superblock modifications are done on first write to the array. However it would be more secure to stop the array after the source vm stops and before destination vm starts. What is the best way to run some script in this particular moment? I know we could create a new storage driver but I am not sure that it is the best option.

There is no explicit support for running a script upon migration in a transactional manner.

It ought to be possible to create a small daemon that listens to libvirt domain lifecycle events, and when an event occurs, it would then invoke a script corresponding to the event. These events are asynchronous to any of the operations though - you'll receive them some fairly short time after the operation is complete. There are quite a few you'll see during migration. On the source side you'll see a STOPPED event, possibly with a detail message of MIGRATED if the hypervisor supports that info (only QEMU does, not Xen). On the dest side you'll see a STARTED event, again possibly with a detail of MIGRATED. If doing non-live migration you'll also see RESUME/SUSPEND events on dest/source hosts.

If you just want to cleanup the volumes on the source after migration, then you could use the STOPPED event to trigger cleanup. There's no way to insert any script between the stop on source & start on dest sequence.

I'll check this out. This idea came to my mind too, but there is the problem with that asynchronism. The reason is that I need the cleanup to occur before first write on the destination machine. The situation is that I have one physical blockdevice with md array member exported to two nodes. I can assemble the array on both nodes in the same time as long as there is only one node using the array at time. However the metadata of the array gets updated on array stop and first write. So the best option would be to stop the array, sync and assemble it (in this order). The second option I can think of is forcing md to update the metadata somehow. Then I should assemble the array on both nodes and stop the source array sometimes after the migration is done. Then I would force the destination machine to fix the metadata and everyting would be fine...

NB, storage drivers won't help you here - they are not involved in migration at all

I thought that the storage driver has some sort of events on "storage start", "storage stop", etc... For example the drbd storage should switch slave to master or something like that? Or am I totally wrong?

Daniel

Radek