[libvirt] How to avoid failure of migration/restoring/starting if cdrom is ejected inside guest?
Hello list, There is problem of migration if the changedable medium is ejected inside guest, this is caused by qemu closes the block driver backend once the medium is ejected, but it doesn't gives a way to let libvirt known the fact. So, libvirt will try to migrate the guest with the media still exists. This will cause the failure, as qemu already closes the block driver backend. Actually this could also break domain restoring and starting (if the domain has a managed saving image, supposing the media is ejected before saving or managed saving). It's ideal if qemu could provide some event so that libvirt could known the media is changed immediately, but it's bad news qemu upstream won't make a patch for this in a short time. As a alternative solution, they proposed patch to expose the status of changeable medium via monitor command "info block": http://lists.gnu.org/archive/html/qemu-devel/2011-08/msg00408.html The output of the improved "info block" looks like below: (qemu) info block disk0: removable=0 file=/home/armbru/work/images/test.qcow2 backing_file=test.img ro=0 drv=qcow2 encrypted=0 cd: removable=1 locked=0 ejected file=x.iso ro=1 drv=raw encrypted=0 What libvirt can do with the qemu improvement is checking the meduim status at the time of migration, but it doesn't kill all the bugs, such as for a live migration, one can eject the media inside guest just during the migration process. This probally makes a race and cause the failure just the same. And moreover, this won't solve the problem on restoring and starting with managed saving image, can't get the medium status as the guest is not active. So, I 'm hesitating to use "info block" to resolve the problems, it can't resolve the root problem thoroughly. Or I missed some good ideas? Any thought is welcomed, thanks. By the way, it might be deserving to report the cdrom tray status using the improved "info block" though, though qemu might keep improving the command to output more infomation, such as the media is inserted, but the tray is still open (which means different if we report the tray status as "close" if "info block" outputs "inserted", need to change the codes then)? Patches of qemu side to improve the tray handling: http://lists.nongnu.org/archive/html/qemu-devel/2011-06/msg00381.html Regards Osier
On Mon, Aug 08, 2011 at 05:47:24PM +0800, Osier Yang wrote:
Hello list,
There is problem of migration if the changedable medium is ejected inside guest, this is caused by qemu closes the block driver backend once the medium is ejected, but it doesn't gives a way to let libvirt known the fact. So, libvirt will try to migrate the guest with the media still exists. This will cause the failure, as qemu already closes the block driver backend.
Actually this could also break domain restoring and starting (if the domain has a managed saving image, supposing the media is ejected before saving or managed saving).
It's ideal if qemu could provide some event so that libvirt could known the media is changed immediately, but it's bad news qemu upstream won't make a patch for this in a short time.
As a alternative solution, they proposed patch to expose the status of changeable medium via monitor command "info block":
http://lists.gnu.org/archive/html/qemu-devel/2011-08/msg00408.html
The output of the improved "info block" looks like below:
(qemu) info block disk0: removable=0 file=/home/armbru/work/images/test.qcow2 backing_file=test.img ro=0 drv=qcow2 encrypted=0 cd: removable=1 locked=0 ejected file=x.iso ro=1 drv=raw encrypted=0
What libvirt can do with the qemu improvement is checking the meduim status at the time of migration, but it doesn't kill all the bugs, such as for a live migration, one can eject the media inside guest just during the migration process. This probally makes a race and cause the failure just the same.
After thinking about this a bit, I believe it's not a problem. If the guest has the media in the drive at the start of migration, libvirt will require the media to be present on the destination. If the guest ejects the media subsequent to that check, the operation of the guest on the dst host will be unaffected. The guest will have the media available to it, but the guest is not required to use it. The only question I have is whether the guest might start on the dst with the media back in the drive, which would be incorrect.
And moreover, this won't solve the problem on restoring and starting with managed saving image, can't get the medium status as the guest is not active.
True, but I'm not sure I see an easy solution to that, and I don't think the lack of solution there should hold up a solution to the migration problem.
So, I 'm hesitating to use "info block" to resolve the problems, it can't resolve the root problem thoroughly.
Or I missed some good ideas? Any thought is welcomed, thanks.
By the way, it might be deserving to report the cdrom tray status using the improved "info block" though, though qemu might keep improving the command to output more infomation, such as the media is inserted, but the tray is still open (which means different if we report the tray status as "close" if "info block" outputs "inserted", need to change the codes then)?
IMO, qemu should expose, and we should report to the user 4 states: 1) tray closed, media inserted 2) tray open, media inserted 3) tray closed, media removed 4) tray open, media removed The API for inserting and removing media should be different from opening and closing the drive. Dave
Patches of qemu side to improve the tray handling:
http://lists.nongnu.org/archive/html/qemu-devel/2011-06/msg00381.html
Regards Osier
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
于 2011年08月08日 22:04, Dave Allan 写道:
On Mon, Aug 08, 2011 at 05:47:24PM +0800, Osier Yang wrote:
Hello list,
There is problem of migration if the changedable medium is ejected inside guest, this is caused by qemu closes the block driver backend once the medium is ejected, but it doesn't gives a way to let libvirt known the fact. So, libvirt will try to migrate the guest with the media still exists. This will cause the failure, as qemu already closes the block driver backend.
Actually this could also break domain restoring and starting (if the domain has a managed saving image, supposing the media is ejected before saving or managed saving).
It's ideal if qemu could provide some event so that libvirt could known the media is changed immediately, but it's bad news qemu upstream won't make a patch for this in a short time.
As a alternative solution, they proposed patch to expose the status of changeable medium via monitor command "info block":
http://lists.gnu.org/archive/html/qemu-devel/2011-08/msg00408.html
The output of the improved "info block" looks like below:
(qemu) info block disk0: removable=0 file=/home/armbru/work/images/test.qcow2 backing_file=test.img ro=0 drv=qcow2 encrypted=0 cd: removable=1 locked=0 ejected file=x.iso ro=1 drv=raw encrypted=0
What libvirt can do with the qemu improvement is checking the meduim status at the time of migration, but it doesn't kill all the bugs, such as for a live migration, one can eject the media inside guest just during the migration process. This probally makes a race and cause the failure just the same. After thinking about this a bit, I believe it's not a problem. If the guest has the media in the drive at the start of migration, libvirt will require the media to be present on the destination.
I guess you mean the migration qemu command line will contains the media path present. If so, yes, it's true.
If the guest ejects the media subsequent to that check, the operation of the guest on the dst host will be unaffected. The guest will have the media available to it, but the guest is not required to use it. The only question I have is whether the guest might start on the dst with the media back in the drive, which would be incorrect.
Sorry, I don't explain the problem clearly enough. If the medium is ejected inside guest, and the source of the medium is removed or renamed. libvirt will still tries to do cgroup setting before starting the guest on dest side, that's the real problem. As one will think it's reasonable to remove the source if it's ejected. And we can't prevent one remove the source during the migration (supposing the removing is before libvirt tries to do cgroup setting on dest host.)
And moreover, this won't solve the problem on restoring and starting with managed saving image, can't get the medium status as the guest is not active. True, but I'm not sure I see an easy solution to that, and I don't think the lack of solution there should hold up a solution to the migration problem.
So, I 'm hesitating to use "info block" to resolve the problems, it can't resolve the root problem thoroughly.
Or I missed some good ideas? Any thought is welcomed, thanks.
By the way, it might be deserving to report the cdrom tray status using the improved "info block" though, though qemu might keep improving the command to output more infomation, such as the media is inserted, but the tray is still open (which means different if we report the tray status as "close" if "info block" outputs "inserted", need to change the codes then)? IMO, qemu should expose, and we should report to the user 4 states:
1) tray closed, media inserted 2) tray open, media inserted 3) tray closed, media removed 4) tray open, media removed
The API for inserting and removing media should be different from opening and closing the drive.
Agee on this, we need to wait the qemu side tray about patches in first.
Dave
Patches of qemu side to improve the tray handling:
http://lists.nongnu.org/archive/html/qemu-devel/2011-06/msg00381.html
Regards Osier
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Mon, Aug 08, 2011 at 10:45:19PM +0800, Osier Yang wrote:
于 2011年08月08日 22:04, Dave Allan 写道:
On Mon, Aug 08, 2011 at 05:47:24PM +0800, Osier Yang wrote:
Hello list,
There is problem of migration if the changedable medium is ejected inside guest, this is caused by qemu closes the block driver backend once the medium is ejected, but it doesn't gives a way to let libvirt known the fact. So, libvirt will try to migrate the guest with the media still exists. This will cause the failure, as qemu already closes the block driver backend.
Actually this could also break domain restoring and starting (if the domain has a managed saving image, supposing the media is ejected before saving or managed saving).
It's ideal if qemu could provide some event so that libvirt could known the media is changed immediately, but it's bad news qemu upstream won't make a patch for this in a short time.
As a alternative solution, they proposed patch to expose the status of changeable medium via monitor command "info block":
http://lists.gnu.org/archive/html/qemu-devel/2011-08/msg00408.html
The output of the improved "info block" looks like below:
(qemu) info block disk0: removable=0 file=/home/armbru/work/images/test.qcow2 backing_file=test.img ro=0 drv=qcow2 encrypted=0 cd: removable=1 locked=0 ejected file=x.iso ro=1 drv=raw encrypted=0
What libvirt can do with the qemu improvement is checking the meduim status at the time of migration, but it doesn't kill all the bugs, such as for a live migration, one can eject the media inside guest just during the migration process. This probally makes a race and cause the failure just the same. After thinking about this a bit, I believe it's not a problem. If the guest has the media in the drive at the start of migration, libvirt will require the media to be present on the destination.
I guess you mean the migration qemu command line will contains the media path present. If so, yes, it's true.
If the guest ejects the media subsequent to that check, the operation of the guest on the dst host will be unaffected. The guest will have the media available to it, but the guest is not required to use it. The only question I have is whether the guest might start on the dst with the media back in the drive, which would be incorrect.
Sorry, I don't explain the problem clearly enough.
If the medium is ejected inside guest, and the source of the medium is removed or renamed. libvirt will still tries to do cgroup setting before starting the guest on dest side, that's the real problem. As one will think it's reasonable to remove the source if it's ejected. And we can't prevent one remove the source during the migration (supposing the removing is before libvirt tries to do cgroup setting on dest host.)
Agreed, but that's a problem that I don't think we can solve except by documenting the behavior. The libvirt documentation has always clearly stated that the storage configuration must be the same on the src and dst hosts. We are discussing altering that constraint here in the case of medium that has been ejected, but I think we can be clear that the constraint is not being removed and that a guest's storage configuration must not be changed while a migration is in progress. Dave
And moreover, this won't solve the problem on restoring and starting with managed saving image, can't get the medium status as the guest is not active. True, but I'm not sure I see an easy solution to that, and I don't think the lack of solution there should hold up a solution to the migration problem.
So, I 'm hesitating to use "info block" to resolve the problems, it can't resolve the root problem thoroughly.
Or I missed some good ideas? Any thought is welcomed, thanks.
By the way, it might be deserving to report the cdrom tray status using the improved "info block" though, though qemu might keep improving the command to output more infomation, such as the media is inserted, but the tray is still open (which means different if we report the tray status as "close" if "info block" outputs "inserted", need to change the codes then)? IMO, qemu should expose, and we should report to the user 4 states:
1) tray closed, media inserted 2) tray open, media inserted 3) tray closed, media removed 4) tray open, media removed
The API for inserting and removing media should be different from opening and closing the drive.
Agee on this, we need to wait the qemu side tray about patches in first.
Dave
Patches of qemu side to improve the tray handling:
http://lists.nongnu.org/archive/html/qemu-devel/2011-06/msg00381.html
Regards Osier
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On 08/08/2011 09:22 AM, Dave Allan wrote:
If the medium is ejected inside guest, and the source of the medium is removed or renamed. libvirt will still tries to do cgroup setting before starting the guest on dest side, that's the real problem. As one will think it's reasonable to remove the source if it's ejected. And we can't prevent one remove the source during the migration (supposing the removing is before libvirt tries to do cgroup setting on dest host.)
Agreed, but that's a problem that I don't think we can solve except by documenting the behavior. The libvirt documentation has always clearly stated that the storage configuration must be the same on the src and dst hosts. We are discussing altering that constraint here in the case of medium that has been ejected, but I think we can be clear that the constraint is not being removed and that a guest's storage configuration must not be changed while a migration is in progress.
I thought the qemu folks were working on a series of patches to make cd tray state part of the migration information. That is, if you start a migration, then a guest ejects the cd, then the destination will correctly see the ejected cd, even though the destination was started with the tray closed. That is, libvirt should not have to poll on cd tray state in order to correctly migrate, but should be able to poll on cd tray state in order to report tray state to the curious user. The migration aspect of cd tray state generally has to be solved by qemu - there's nothing libvirt can do if the guest changes cd tray state after migration has started but before it has completed, unless libvirt can poll the source after it has paused but before the destination has been unpaused and issue appropriate monitor commands on the destination to resync state; but that should only be necessary if qemu doesn't migrate cd tray state in the first place, and requires that qemu support tray state monitor commands for libvirt to use. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On Mon, Aug 08, 2011 at 11:53:26AM -0600, Eric Blake wrote:
On 08/08/2011 09:22 AM, Dave Allan wrote:
If the medium is ejected inside guest, and the source of the medium is removed or renamed. libvirt will still tries to do cgroup setting before starting the guest on dest side, that's the real problem. As one will think it's reasonable to remove the source if it's ejected. And we can't prevent one remove the source during the migration (supposing the removing is before libvirt tries to do cgroup setting on dest host.)
Agreed, but that's a problem that I don't think we can solve except by documenting the behavior. The libvirt documentation has always clearly stated that the storage configuration must be the same on the src and dst hosts. We are discussing altering that constraint here in the case of medium that has been ejected, but I think we can be clear that the constraint is not being removed and that a guest's storage configuration must not be changed while a migration is in progress.
I thought the qemu folks were working on a series of patches to make cd tray state part of the migration information. That is, if you start a migration, then a guest ejects the cd, then the destination will correctly see the ejected cd, even though the destination was started with the tray closed. That is, libvirt should not have to poll on cd tray state in order to correctly migrate, but should be able to poll on cd tray state in order to report tray state to the curious user.
The migration aspect of cd tray state generally has to be solved by qemu - there's nothing libvirt can do if the guest changes cd tray state after migration has started but before it has completed, unless libvirt can poll the source after it has paused but before the destination has been unpaused and issue appropriate monitor commands on the destination to resync state; but that should only be necessary if qemu doesn't migrate cd tray state in the first place, and requires that qemu support tray state monitor commands for libvirt to use.
Yeah, that was my concern about having the media appear back in the drive following migration that I mentioned in my first response. I think we're in agreement--I only think we need to poll for the tray state for on migration so that we can correctly decide whether to allow the migration if the media backing storage isn't present. I.e., if the guest has ejected the media from the drive we can permit migration if the media isn't present on the dst. Dave
于 2011年08月09日 02:09, Dave Allan 写道:
If the medium is ejected inside guest, and the source of the medium is removed or renamed. libvirt will still tries to do cgroup setting before starting the guest on dest side, that's the real problem. As one will think it's reasonable to remove the source if it's ejected. And we can't prevent one remove the source during the migration (supposing the removing is before libvirt tries to do cgroup setting on dest host.) Agreed, but that's a problem that I don't think we can solve except by documenting the behavior. The libvirt documentation has always clearly stated that the storage configuration must be the same on the src and dst hosts. We are discussing altering that constraint here in the case of medium that has been ejected, but I think we can be clear that the constraint is not being removed and that a guest's storage configuration must not be changed while a migration is in progress. I thought the qemu folks were working on a series of patches to make cd tray state part of the migration information. That is, if you start a migration, then a guest ejects the cd, then the destination will correctly see the ejected cd, even though the destination was started with the tray closed. That is, libvirt should not have to
On 08/08/2011 09:22 AM, Dave Allan wrote: poll on cd tray state in order to correctly migrate, but should be able to poll on cd tray state in order to report tray state to the curious user.
The migration aspect of cd tray state generally has to be solved by qemu - there's nothing libvirt can do if the guest changes cd tray state after migration has started but before it has completed, unless libvirt can poll the source after it has paused but before the destination has been unpaused and issue appropriate monitor commands on the destination to resync state; but that should only be necessary if qemu doesn't migrate cd tray state in the first place, and requires that qemu support tray state monitor commands for libvirt to use. Yeah, that was my concern about having the media appear back in the drive following migration that I mentioned in my first response. I
On Mon, Aug 08, 2011 at 11:53:26AM -0600, Eric Blake wrote: think we're in agreement--I only think we need to poll for the tray state for on migration so that we can correctly decide whether to allow the migration if the media backing storage isn't present. I.e., if the guest has ejected the media from the drive we can permit migration if the media isn't present on the dst.
Dave
I'm not sure if we're in agreement, following is my thought 1) trying to solve the problem that ejects the media inside guest while migration in process (with or without the media source is removed, if don't remove the media source, we starts guest on dest with the media inserted incorrectly, if removing it, we fail early before trying executing the qemu command line while do cgroup setting) doesn't work. 2) But we can do simple checking just before (not during) migration takes place, (if the media is ejected, we starts the guest on dest without the media present). 3) for the report tray status, we need to wait for the qemu patches are done. The coming patches of "info block" can't provide enough info for us. Osier
On Tue, Aug 09, 2011 at 10:07:57AM +0800, Osier Yang wrote:
于 2011年08月09日 02:09, Dave Allan 写道:
If the medium is ejected inside guest, and the source of the medium is removed or renamed. libvirt will still tries to do cgroup setting before starting the guest on dest side, that's the real problem. As one will think it's reasonable to remove the source if it's ejected. And we can't prevent one remove the source during the migration (supposing the removing is before libvirt tries to do cgroup setting on dest host.) Agreed, but that's a problem that I don't think we can solve except by documenting the behavior. The libvirt documentation has always clearly stated that the storage configuration must be the same on the src and dst hosts. We are discussing altering that constraint here in the case of medium that has been ejected, but I think we can be clear that the constraint is not being removed and that a guest's storage configuration must not be changed while a migration is in progress. I thought the qemu folks were working on a series of patches to make cd tray state part of the migration information. That is, if you start a migration, then a guest ejects the cd, then the destination will correctly see the ejected cd, even though the destination was started with the tray closed. That is, libvirt should not have to
On 08/08/2011 09:22 AM, Dave Allan wrote: poll on cd tray state in order to correctly migrate, but should be able to poll on cd tray state in order to report tray state to the curious user.
The migration aspect of cd tray state generally has to be solved by qemu - there's nothing libvirt can do if the guest changes cd tray state after migration has started but before it has completed, unless libvirt can poll the source after it has paused but before the destination has been unpaused and issue appropriate monitor commands on the destination to resync state; but that should only be necessary if qemu doesn't migrate cd tray state in the first place, and requires that qemu support tray state monitor commands for libvirt to use. Yeah, that was my concern about having the media appear back in the drive following migration that I mentioned in my first response. I
On Mon, Aug 08, 2011 at 11:53:26AM -0600, Eric Blake wrote: think we're in agreement--I only think we need to poll for the tray state for on migration so that we can correctly decide whether to allow the migration if the media backing storage isn't present. I.e., if the guest has ejected the media from the drive we can permit migration if the media isn't present on the dst.
Dave
I'm not sure if we're in agreement, following is my thought
1) trying to solve the problem that ejects the media inside guest while migration in process (with or without the media source is removed, if don't remove the media source, we starts guest on dest with the media inserted incorrectly,
This is the problem addressed by Marcus' tray state migration patches to qemu that I mentioned in my earlier email, so as long as that work is accepted by qemu, I don't think we have to worry about this situation.
if removing it, we fail early before trying executing the qemu command line while do cgroup setting) doesn't work.
If libvirt queries for whether the guest has ejected the media before we begin the migration, then we can remove the check for the backing storage on the dst host. If the guest has ejected the media, we don't care whether it's present on the dst, which I think is your point 2). I'm not 100% convinced that it's a problem even if the user removes the storage during migration, and if it is, this is the problem that can be addressed with documenting the restriction that a user cannot remove the backing storage for media while a migration is in progress. Since the current restriction is that storage must be identical on src and dst, this represents something of a relaxation.
2) But we can do simple checking just before (not during) migration takes place, (if the media is ejected, we starts the guest on dest without the media present).
Yes.
3) for the report tray status, we need to wait for the qemu patches are done. The coming patches of "info block" can't provide enough info for us.
Right, libvirt can report the media status, but not the tray status. When qemu provides the tray status libvirt can report that, too. Dave
于 2011年08月09日 11:02, Dave Allan 写道:
On Tue, Aug 09, 2011 at 10:07:57AM +0800, Osier Yang wrote:
于 2011年08月09日 02:09, Dave Allan 写道:
If the medium is ejected inside guest, and the source of the medium is removed or renamed. libvirt will still tries to do cgroup setting before starting the guest on dest side, that's the real problem. As one will think it's reasonable to remove the source if it's ejected. And we can't prevent one remove the source during the migration (supposing the removing is before libvirt tries to do cgroup setting on dest host.) Agreed, but that's a problem that I don't think we can solve except by documenting the behavior. The libvirt documentation has always clearly stated that the storage configuration must be the same on the src and dst hosts. We are discussing altering that constraint here in the case of medium that has been ejected, but I think we can be clear that the constraint is not being removed and that a guest's storage configuration must not be changed while a migration is in progress. I thought the qemu folks were working on a series of patches to make cd tray state part of the migration information. That is, if you start a migration, then a guest ejects the cd, then the destination will correctly see the ejected cd, even though the destination was started with the tray closed. That is, libvirt should not have to
On 08/08/2011 09:22 AM, Dave Allan wrote: poll on cd tray state in order to correctly migrate, but should be able to poll on cd tray state in order to report tray state to the curious user.
The migration aspect of cd tray state generally has to be solved by qemu - there's nothing libvirt can do if the guest changes cd tray state after migration has started but before it has completed, unless libvirt can poll the source after it has paused but before the destination has been unpaused and issue appropriate monitor commands on the destination to resync state; but that should only be necessary if qemu doesn't migrate cd tray state in the first place, and requires that qemu support tray state monitor commands for libvirt to use. Yeah, that was my concern about having the media appear back in the drive following migration that I mentioned in my first response. I
On Mon, Aug 08, 2011 at 11:53:26AM -0600, Eric Blake wrote: think we're in agreement--I only think we need to poll for the tray state for on migration so that we can correctly decide whether to allow the migration if the media backing storage isn't present. I.e., if the guest has ejected the media from the drive we can permit migration if the media isn't present on the dst.
Dave I'm not sure if we're in agreement, following is my thought
1) trying to solve the problem that ejects the media inside guest while migration in process (with or without the media source is removed, if don't remove the media source, we starts guest on dest with the media inserted incorrectly, This is the problem addressed by Marcus' tray state migration patches to qemu that I mentioned in my earlier email, so as long as that work is accepted by qemu, I don't think we have to worry about this situation.
if removing it, we fail early before trying executing the qemu command line while do cgroup setting) doesn't work. If libvirt queries for whether the guest has ejected the media before we begin the migration, then we can remove the check for the backing storage on the dst host. If the guest has ejected the media, we don't care whether it's present on the dst, which I think is your point 2).
Yes
I'm not 100% convinced that it's a problem even if the user removes the storage during migration, and if it is, this is the problem that can be addressed with documenting the restriction that a user cannot remove the backing storage for media while a migration is in progress. Since the current restriction is that storage must be identical on src and dst, this represents something of a relaxation.
Yes, agreed on this in previous mail.
2) But we can do simple checking just before (not during) migration takes place, (if the media is ejected, we starts the guest on dest without the media present). Yes.
3) for the report tray status, we need to wait for the qemu patches are done. The coming patches of "info block" can't provide enough info for us. Right, libvirt can report the media status, but not the tray status. When qemu provides the tray status libvirt can report that, too.
Dave
于 2011年08月09日 01:53, Eric Blake 写道:
On 08/08/2011 09:22 AM, Dave Allan wrote:
If the medium is ejected inside guest, and the source of the medium is removed or renamed. libvirt will still tries to do cgroup setting before starting the guest on dest side, that's the real problem. As one will think it's reasonable to remove the source if it's ejected. And we can't prevent one remove the source during the migration (supposing the removing is before libvirt tries to do cgroup setting on dest host.)
Agreed, but that's a problem that I don't think we can solve except by documenting the behavior. The libvirt documentation has always clearly stated that the storage configuration must be the same on the src and dst hosts. We are discussing altering that constraint here in the case of medium that has been ejected, but I think we can be clear that the constraint is not being removed and that a guest's storage configuration must not be changed while a migration is in progress.
I thought the qemu folks were working on a series of patches to make cd tray state part of the migration information. That is, if you start a migration, then a guest ejects the cd, then the destination will correctly see the ejected cd, even though the destination was started with the tray closed. That is, libvirt should not have to poll on cd tray state in order to correctly migrate, but should be able to poll on cd tray state in order to report tray state to the curious user.
Agree. I don't known they are working out such patches yet. Only known they are trying to work out some event for the tray state.
The migration aspect of cd tray state generally has to be solved by qemu - there's nothing libvirt can do if the guest changes cd tray state after migration has started but before it has completed, unless libvirt can poll the source after it has paused but before the destination has been unpaused and issue appropriate monitor commands on the destination to resync state; but that should only be necessary if qemu doesn't migrate cd tray state in the first place, and requires that qemu support tray state monitor commands for libvirt to use.
As far as I known, qemu still can only report the medium is ejected or inserted soon via "info block", and without minitor commands to change the tray state, or I missed something? Osier
On Tue, Aug 09, 2011 at 09:58:10AM +0800, Osier Yang wrote:
于 2011年08月09日 01:53, Eric Blake 写道:
On 08/08/2011 09:22 AM, Dave Allan wrote:
If the medium is ejected inside guest, and the source of the medium is removed or renamed. libvirt will still tries to do cgroup setting before starting the guest on dest side, that's the real problem. As one will think it's reasonable to remove the source if it's ejected. And we can't prevent one remove the source during the migration (supposing the removing is before libvirt tries to do cgroup setting on dest host.)
Agreed, but that's a problem that I don't think we can solve except by documenting the behavior. The libvirt documentation has always clearly stated that the storage configuration must be the same on the src and dst hosts. We are discussing altering that constraint here in the case of medium that has been ejected, but I think we can be clear that the constraint is not being removed and that a guest's storage configuration must not be changed while a migration is in progress.
I thought the qemu folks were working on a series of patches to make cd tray state part of the migration information. That is, if you start a migration, then a guest ejects the cd, then the destination will correctly see the ejected cd, even though the destination was started with the tray closed. That is, libvirt should not have to poll on cd tray state in order to correctly migrate, but should be able to poll on cd tray state in order to report tray state to the curious user.
Agree. I don't known they are working out such patches yet. Only known they are trying to work out some event for the tray state.
Markus has submitted patches to qemu to migrate the tray state. See, e.g,: http://lists.gnu.org/archive/html/qemu-devel/2011-07/msg01947.html so, assuming that work is accepted, eject post-migration-start will not be a problem.
The migration aspect of cd tray state generally has to be solved by qemu - there's nothing libvirt can do if the guest changes cd tray state after migration has started but before it has completed, unless libvirt can poll the source after it has paused but before the destination has been unpaused and issue appropriate monitor commands on the destination to resync state; but that should only be necessary if qemu doesn't migrate cd tray state in the first place, and requires that qemu support tray state monitor commands for libvirt to use.
As far as I known, qemu still can only report the medium is ejected or inserted soon via "info block", and without minitor commands to change the tray state, or I missed something?
Osier
于 2011年08月08日 23:22, Dave Allan 写道:
On Mon, Aug 08, 2011 at 05:47:24PM +0800, Osier Yang wrote:
Hello list,
There is problem of migration if the changedable medium is ejected inside guest, this is caused by qemu closes the block driver backend once the medium is ejected, but it doesn't gives a way to let libvirt known the fact. So, libvirt will try to migrate the guest with the media still exists. This will cause the failure, as qemu already closes the block driver backend.
Actually this could also break domain restoring and starting (if the domain has a managed saving image, supposing the media is ejected before saving or managed saving).
It's ideal if qemu could provide some event so that libvirt could known the media is changed immediately, but it's bad news qemu upstream won't make a patch for this in a short time.
As a alternative solution, they proposed patch to expose the status of changeable medium via monitor command "info block":
http://lists.gnu.org/archive/html/qemu-devel/2011-08/msg00408.html
The output of the improved "info block" looks like below:
(qemu) info block disk0: removable=0 file=/home/armbru/work/images/test.qcow2 backing_file=test.img ro=0 drv=qcow2 encrypted=0 cd: removable=1 locked=0 ejected file=x.iso ro=1 drv=raw encrypted=0
What libvirt can do with the qemu improvement is checking the meduim status at the time of migration, but it doesn't kill all the bugs, such as for a live migration, one can eject the media inside guest just during the migration process. This probally makes a race and cause the failure just the same. After thinking about this a bit, I believe it's not a problem. If the guest has the media in the drive at the start of migration, libvirt will require the media to be present on the destination. I guess you mean the migration qemu command line will contains
于 2011年08月08日 22:04, Dave Allan 写道: the media path present. If so, yes, it's true.
If the guest ejects the media subsequent to that check, the operation of the guest on the dst host will be unaffected. The guest will have the media available to it, but the guest is not required to use it. The only question I have is whether the guest might start on the dst with the media back in the drive, which would be incorrect.
Sorry, I don't explain the problem clearly enough.
If the medium is ejected inside guest, and the source of the medium is removed or renamed. libvirt will still tries to do cgroup setting before starting the guest on dest side, that's the real problem. As one will think it's reasonable to remove the source if it's ejected. And we can't prevent one remove the source during the migration (supposing the removing is before libvirt tries to do cgroup setting on dest host.) Agreed, but that's a problem that I don't think we can solve except by documenting the behavior. The libvirt documentation has always clearly stated that the storage configuration must be the same on the src and dst hosts. We are discussing altering that constraint here in
On Mon, Aug 08, 2011 at 10:45:19PM +0800, Osier Yang wrote: the case of medium that has been ejected, but I think we can be clear that the constraint is not being removed and that a guest's storage configuration must not be changed while a migration is in progress.
Agree, though changing the doc is not ideal fix, but that's what we can only do currently.
Dave
And moreover, this won't solve the problem on restoring and starting with managed saving image, can't get the medium status as the guest is not active. True, but I'm not sure I see an easy solution to that, and I don't think the lack of solution there should hold up a solution to the migration problem.
So, I 'm hesitating to use "info block" to resolve the problems, it can't resolve the root problem thoroughly.
Or I missed some good ideas? Any thought is welcomed, thanks.
By the way, it might be deserving to report the cdrom tray status using the improved "info block" though, though qemu might keep improving the command to output more infomation, such as the media is inserted, but the tray is still open (which means different if we report the tray status as "close" if "info block" outputs "inserted", need to change the codes then)? IMO, qemu should expose, and we should report to the user 4 states:
1) tray closed, media inserted 2) tray open, media inserted 3) tray closed, media removed 4) tray open, media removed
The API for inserting and removing media should be different from opening and closing the drive. Agee on this, we need to wait the qemu side tray about patches in first.
Dave
Patches of qemu side to improve the tray handling:
http://lists.nongnu.org/archive/html/qemu-devel/2011-06/msg00381.html
Regards Osier
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
participants (3)
-
Dave Allan -
Eric Blake -
Osier Yang