On Mon, Aug 08, 2011 at 05:47:24PM +0800, Osier Yang wrote: > Hello list, > > There is problem of migration if the changedable medium is ejected > inside guest, this is caused by qemu closes the block driver backend > once the medium is ejected, but it doesn't gives a way to let > libvirt known the fact. So, libvirt will try to migrate the guest > with the media still exists. This will cause the failure, as qemu > already closes the block driver backend. > > Actually this could also break domain restoring and starting (if the domain > has a managed saving image, supposing the media is ejected before saving > or managed saving). > > It's ideal if qemu could provide some event so that libvirt could known the > media is changed immediately, but it's bad news qemu upstream won't make > a patch for this in a short time. > > As a alternative solution, they proposed patch to expose the status of > changeable medium via monitor command "info block": > > http://lists.gnu.org/archive/html/qemu-devel/2011-08/msg00408.html > > The output of the improved "info block" looks like below: > > (qemu) info block > disk0: removable=0 file=/home/armbru/work/images/test.qcow2 > backing_file=test.img ro=0 drv=qcow2 encrypted=0 > cd: removable=1 locked=0 ejected file=x.iso ro=1 drv=raw encrypted=0 > > What libvirt can do with the qemu improvement is checking the meduim status > at the time of migration, but it doesn't kill all the bugs, such as for a > live migration, one can eject the media inside guest just during the migration > process. This probally makes a race and cause the failure just the same. After thinking about this a bit, I believe it's not a problem. If the guest has the media in the drive at the start of migration, libvirt will require the media to be present on the destination.

If the guest ejects the media subsequent to that check, the operation of the guest on the dst host will be unaffected. The guest will have the media available to it, but the guest is not required to use it. The only question I have is whether the guest might start on the dst with the media back in the drive, which would be incorrect.

> And moreover, this won't solve the problem on restoring and starting > with managed saving image, can't get the medium status as the guest > is not active. True, but I'm not sure I see an easy solution to that, and I don't think the lack of solution there should hold up a solution to the migration problem. > So, I 'm hesitating to use "info block" to resolve the problems, it > can't resolve the root problem thoroughly. > > Or I missed some good ideas? Any thought is welcomed, thanks. > > By the way, it might be deserving to report the cdrom tray status > using the improved "info block" though, though qemu might keep > improving the command to output more infomation, such as the media > is inserted, but the tray is still open (which means different if we > report the tray status as "close" if "info block" outputs > "inserted", need to change the codes then)? IMO, qemu should expose, and we should report to the user 4 states: 1) tray closed, media inserted 2) tray open, media inserted 3) tray closed, media removed 4) tray open, media removed The API for inserting and removing media should be different from opening and closing the drive.

Dave > Patches of qemu side to improve the tray handling: > > http://lists.nongnu.org/archive/html/qemu-devel/2011-06/msg00381.html > > Regards > Osier > > -- > libvir-list mailing list > libvir-list(a)redhat.com > https://www.redhat.com/mailman/listinfo/libvir-list

> If the medium is ejected inside guest, and the source of the medium > is removed or renamed. libvirt will still tries to do cgroup setting > before starting the guest on dest side, that's the real problem. As > one will think it's reasonable to remove the source if it's > ejected. And we can't prevent one remove the source during the > migration (supposing the removing is before libvirt tries to do > cgroup setting on dest host.) Agreed, but that's a problem that I don't think we can solve except by documenting the behavior. The libvirt documentation has always clearly stated that the storage configuration must be the same on the src and dst hosts. We are discussing altering that constraint here in the case of medium that has been ejected, but I think we can be clear that the constraint is not being removed and that a guest's storage configuration must not be changed while a migration is in progress.

On Mon, Aug 08, 2011 at 11:53:26AM -0600, Eric Blake wrote: > On 08/08/2011 09:22 AM, Dave Allan wrote: >>> If the medium is ejected inside guest, and the source of the medium >>> is removed or renamed. libvirt will still tries to do cgroup setting >>> before starting the guest on dest side, that's the real problem. As >>> one will think it's reasonable to remove the source if it's >>> ejected. And we can't prevent one remove the source during the >>> migration (supposing the removing is before libvirt tries to do >>> cgroup setting on dest host.) >> Agreed, but that's a problem that I don't think we can solve except by >> documenting the behavior. The libvirt documentation has always >> clearly stated that the storage configuration must be the same on the >> src and dst hosts. We are discussing altering that constraint here in >> the case of medium that has been ejected, but I think we can be clear >> that the constraint is not being removed and that a guest's storage >> configuration must not be changed while a migration is in progress. > I thought the qemu folks were working on a series of patches to make > cd tray state part of the migration information. That is, if you > start a migration, then a guest ejects the cd, then the destination > will correctly see the ejected cd, even though the destination was > started with the tray closed. That is, libvirt should not have to > poll on cd tray state in order to correctly migrate, but should be > able to poll on cd tray state in order to report tray state to the > curious user. > > The migration aspect of cd tray state generally has to be solved by > qemu - there's nothing libvirt can do if the guest changes cd tray > state after migration has started but before it has completed, > unless libvirt can poll the source after it has paused but before > the destination has been unpaused and issue appropriate monitor > commands on the destination to resync state; but that should only be > necessary if qemu doesn't migrate cd tray state in the first place, > and requires that qemu support tray state monitor commands for > libvirt to use. Yeah, that was my concern about having the media appear back in the drive following migration that I mentioned in my first response. I think we're in agreement--I only think we need to poll for the tray state for on migration so that we can correctly decide whether to allow the migration if the media backing storage isn't present. I.e., if the guest has ejected the media from the drive we can permit migration if the media isn't present on the dst. Dave

On Tue, Aug 09, 2011 at 10:07:57AM +0800, Osier Yang wrote: > 于 2011年08月09日 02:09, Dave Allan 写道: >> On Mon, Aug 08, 2011 at 11:53:26AM -0600, Eric Blake wrote: >>> On 08/08/2011 09:22 AM, Dave Allan wrote: >>>>> If the medium is ejected inside guest, and the source of the medium >>>>> is removed or renamed. libvirt will still tries to do cgroup setting >>>>> before starting the guest on dest side, that's the real problem. As >>>>> one will think it's reasonable to remove the source if it's >>>>> ejected. And we can't prevent one remove the source during the >>>>> migration (supposing the removing is before libvirt tries to do >>>>> cgroup setting on dest host.) >>>> Agreed, but that's a problem that I don't think we can solve except by >>>> documenting the behavior. The libvirt documentation has always >>>> clearly stated that the storage configuration must be the same on the >>>> src and dst hosts. We are discussing altering that constraint here in >>>> the case of medium that has been ejected, but I think we can be clear >>>> that the constraint is not being removed and that a guest's storage >>>> configuration must not be changed while a migration is in progress. >>> I thought the qemu folks were working on a series of patches to make >>> cd tray state part of the migration information. That is, if you >>> start a migration, then a guest ejects the cd, then the destination >>> will correctly see the ejected cd, even though the destination was >>> started with the tray closed. That is, libvirt should not have to >>> poll on cd tray state in order to correctly migrate, but should be >>> able to poll on cd tray state in order to report tray state to the >>> curious user. >>> >>> The migration aspect of cd tray state generally has to be solved by >>> qemu - there's nothing libvirt can do if the guest changes cd tray >>> state after migration has started but before it has completed, >>> unless libvirt can poll the source after it has paused but before >>> the destination has been unpaused and issue appropriate monitor >>> commands on the destination to resync state; but that should only be >>> necessary if qemu doesn't migrate cd tray state in the first place, >>> and requires that qemu support tray state monitor commands for >>> libvirt to use. >> Yeah, that was my concern about having the media appear back in the >> drive following migration that I mentioned in my first response. I >> think we're in agreement--I only think we need to poll for the tray >> state for on migration so that we can correctly decide whether to >> allow the migration if the media backing storage isn't present. I.e., >> if the guest has ejected the media from the drive we can permit >> migration if the media isn't present on the dst. >> >> Dave > I'm not sure if we're in agreement, following is my thought > > 1) trying to solve the problem that ejects the media inside guest > while migration in process (with or without the media source > is removed, if don't remove the media source, we starts guest > on dest with the media inserted incorrectly, This is the problem addressed by Marcus' tray state migration patches to qemu that I mentioned in my earlier email, so as long as that work is accepted by qemu, I don't think we have to worry about this situation. > if removing it, we fail early before trying executing the qemu > command line while do cgroup setting) doesn't work. If libvirt queries for whether the guest has ejected the media before we begin the migration, then we can remove the check for the backing storage on the dst host. If the guest has ejected the media, we don't care whether it's present on the dst, which I think is your point 2).

I'm not 100% convinced that it's a problem even if the user removes the storage during migration, and if it is, this is the problem that can be addressed with documenting the restriction that a user cannot remove the backing storage for media while a migration is in progress. Since the current restriction is that storage must be identical on src and dst, this represents something of a relaxation.

> 2) But we can do simple checking just before (not during) migration > takes place, (if the media is ejected, we starts the guest on dest > without the media present). Yes. > 3) for the report tray status, we need to wait for the qemu patches > are done. The coming patches of "info block" can't provide enough > info for us. Right, libvirt can report the media status, but not the tray status. When qemu provides the tray status libvirt can report that, too. Dave

于 2011年08月09日 01:53, Eric Blake 写道: >On 08/08/2011 09:22 AM, Dave Allan wrote: >>>If the medium is ejected inside guest, and the source of the medium >>>is removed or renamed. libvirt will still tries to do cgroup setting >>>before starting the guest on dest side, that's the real problem. As >>>one will think it's reasonable to remove the source if it's >>>ejected. And we can't prevent one remove the source during the >>>migration (supposing the removing is before libvirt tries to do >>>cgroup setting on dest host.) >> >>Agreed, but that's a problem that I don't think we can solve except by >>documenting the behavior. The libvirt documentation has always >>clearly stated that the storage configuration must be the same on the >>src and dst hosts. We are discussing altering that constraint here in >>the case of medium that has been ejected, but I think we can be clear >>that the constraint is not being removed and that a guest's storage >>configuration must not be changed while a migration is in progress. > >I thought the qemu folks were working on a series of patches to >make cd tray state part of the migration information. That is, if >you start a migration, then a guest ejects the cd, then the >destination will correctly see the ejected cd, even though the >destination was started with the tray closed. That is, libvirt >should not have to poll on cd tray state in order to correctly >migrate, but should be able to poll on cd tray state in order to >report tray state to the curious user. > Agree. I don't known they are working out such patches yet. Only known they are trying to work out some event for the tray state.

>The migration aspect of cd tray state generally has to be solved >by qemu - there's nothing libvirt can do if the guest changes cd >tray state after migration has started but before it has >completed, unless libvirt can poll the source after it has paused >but before the destination has been unpaused and issue appropriate >monitor commands on the destination to resync state; but that >should only be necessary if qemu doesn't migrate cd tray state in >the first place, and requires that qemu support tray state monitor >commands for libvirt to use. > As far as I known, qemu still can only report the medium is ejected or inserted soon via "info block", and without minitor commands to change the tray state, or I missed something? Osier