5:37 a.m.
Short question: What is the best way to run some script on VM migration?
Detailed description:
We use own system for volume management. Its based on md and we found it very
flexible. Basic idea is that we have md mirror everywhere we can. This suits
great for libvirt: We have a disk on iSCSI server, its "partioned" using
device mapper, the resulting "partition" is member of md raid1, that raid is
exported via iSCSI to the libvirt node, it is on the node included in another
md raid1 and that "final" md raid is used as storage by the VM. This provides
high level of abstraction that allows us to migrate the storage to another
disk, another iSCSI server, etc...
The only glitch is on live migration of the VM. It is possible to assemble and
run the array on two nodes. Also all superblock modifications are done on
first write to the array. However it would be more secure to stop the array
after the source vm stops and before destination vm starts.
What is the best way to run some script in this particular moment? I know we
could create a new storage driver but I am not sure that it is the best option.
--
Open WebMail Project (http://openwebmail.org)
8:15 a.m.
On Mon, Oct 12, 2009 at 12:37:22PM +0200, Radek Hlad?k wrote:
Short question: What is the best way to run some script on VM
migration?
Detailed description:
We use own system for volume management. Its based on md and we found it very
flexible. Basic idea is that we have md mirror everywhere we can. This suits
great for libvirt: We have a disk on iSCSI server, its "partioned" using
device mapper, the resulting "partition" is member of md raid1, that raid is
exported via iSCSI to the libvirt node, it is on the node included in another
md raid1 and that "final" md raid is used as storage by the VM. This provides
high level of abstraction that allows us to migrate the storage to another
disk, another iSCSI server, etc...
The only glitch is on live migration of the VM. It is possible to assemble and
run the array on two nodes. Also all superblock modifications are done on
first write to the array. However it would be more secure to stop the array
after the source vm stops and before destination vm starts.
What is the best way to run some script in this particular moment? I know we
could create a new storage driver but I am not sure that it is the best option.
There is no explicit support for running a script upon migration in a
transactional manner.
It ought to be possible to create a small daemon that listens to libvirt
domain lifecycle events, and when an event occurs, it would then invoke
a script corresponding to the event. These events are asynchronous to
any of the operations though - you'll receive them some fairly short time
after the operation is complete. There are quite a few you'll see during
migration. On the source side you'll see a STOPPED event, possibly with a
detail message of MIGRATED if the hypervisor supports that info (only
QEMU does, not Xen). On the dest side you'll see a STARTED event, again
possibly with a detail of MIGRATED. If doing non-live migration you'll
also see RESUME/SUSPEND events on dest/source hosts.
If you just want to cleanup the volumes on the source after migration,
then you could use the STOPPED event to trigger cleanup. There's no
way to insert any script between the stop on source & start on dest
sequence.
NB, storage drivers won't help you here - they are not involved in
migration at all
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
12:51 p.m.
Daniel P. Berrange napsal(a):
On Mon, Oct 12, 2009 at 12:37:22PM +0200, Radek Hlad?k wrote:
> Short question: What is the best way to run some script on VM migration?
>
> Detailed description:
> We use own system for volume management. Its based on md and we found it very
> flexible. Basic idea is that we have md mirror everywhere we can. This suits
> great for libvirt: We have a disk on iSCSI server, its "partioned" using
> device mapper, the resulting "partition" is member of md raid1, that raid
is
> exported via iSCSI to the libvirt node, it is on the node included in another
> md raid1 and that "final" md raid is used as storage by the VM. This
provides
> high level of abstraction that allows us to migrate the storage to another
> disk, another iSCSI server, etc...
> The only glitch is on live migration of the VM. It is possible to assemble and
> run the array on two nodes. Also all superblock modifications are done on
> first write to the array. However it would be more secure to stop the array
> after the source vm stops and before destination vm starts.
> What is the best way to run some script in this particular moment? I know we
> could create a new storage driver but I am not sure that it is the best option.
There is no explicit support for running a script upon migration in a
transactional manner.
It ought to be possible to create a small daemon that listens to libvirt
domain lifecycle events, and when an event occurs, it would then invoke
a script corresponding to the event. These events are asynchronous to
any of the operations though - you'll receive them some fairly short time
after the operation is complete. There are quite a few you'll see during
migration. On the source side you'll see a STOPPED event, possibly with a
detail message of MIGRATED if the hypervisor supports that info (only
QEMU does, not Xen). On the dest side you'll see a STARTED event, again
possibly with a detail of MIGRATED. If doing non-live migration you'll
also see RESUME/SUSPEND events on dest/source hosts.
If you just want to cleanup the volumes on the source after migration,
then you could use the STOPPED event to trigger cleanup. There's no
way to insert any script between the stop on source & start on dest
sequence.
I'll check this out. This idea came to my mind too, but there is the
problem with that asynchronism. The reason is that I need the cleanup to
occur before first write on the destination machine. The situation is
that I have one physical blockdevice with md array member exported to
two nodes. I can assemble the array on both nodes in the same time as
long as there is only one node using the array at time. However the
metadata of the array gets updated on array stop and first write. So the
best option would be to stop the array, sync and assemble it (in this
order).
The second option I can think of is forcing md to update the metadata
somehow. Then I should assemble the array on both nodes and stop the
source array sometimes after the migration is done. Then I would force
the destination machine to fix the metadata and everyting would be fine...
NB, storage drivers won't help you here - they are not involved in
migration at all
I thought that the storage driver has some sort of events on "storage
start", "storage stop", etc... For example the drbd storage should
switch slave to master or something like that? Or am I totally wrong?
Daniel
Radek
5521
days inactive
5522
days old
2 comments
3 participants
participants (3)
-
Daniel P. Berrange -
Radek Hladik -
Radek Hladík