On Thu, Sep 13, 2018 at 10:35:09 +0400, Dmitry Melekhov wrote: > After some mistakes yesterday we ( me and my colleague ) think that it > will be wise for libvirt to check config file existence on remote side Which config file?

> and through error if not, > > before migrating, otherwise migration will fail and VM fs can be > damaged, because it is sort of remove of power plug... > > We missed twice yesterday :-( > > Could  you tell me is there already such option or any plans to > implement this? You didn't really say anything about what you did and what happened. Thus it's impossible to help you.

Jirka

On Thu, Sep 13, 2018 at 18:38:57 +0400, Dmitry Melekhov wrote: > > 13.09.2018 17:47, Jiri Denemark пишет: >> On Thu, Sep 13, 2018 at 10:35:09 +0400, Dmitry Melekhov wrote: >>> After some mistakes yesterday we ( me and my colleague ) think that it >>> will be wise for libvirt to check config file existence on remote side >> Which config file? >> > VM config file, namely qemu. Do you mean the domain XML definition found in /etc/libvirt/qemu/*.xml ?

> We forgot to mount shared storage (namely gluster volume), on which we > have config files, twice... If you're talking about the domain XML file, then you should never store them on a shared storage. All XML files in /etc/libvirt are supposed to be local to each host.

Sharing them may cause changes done on one host will not be seen by libvirtd on the other host and they can even get lost when libvirtd on the other host decides to write its version of the domain definition to the file. And migration is designed so that the domain definition does not need to be present on the destination host. In fact, even if it's present, libvirtd will ignore it until you stop the migrated domain and start it again.

> And tried to live migrate VM to another host, where there is no config > file, so VM can't be started, so it died. Twice :-( Error messages or logs would help. Otherwise we have no clue what failed and why.

On Thu, Sep 13, 2018 at 19:37:00 +0400, Dmitry Melekhov wrote: > > 13.09.2018 18:57, Jiri Denemark пишет: >> On Thu, Sep 13, 2018 at 18:38:57 +0400, Dmitry Melekhov wrote: >>> 13.09.2018 17:47, Jiri Denemark пишет: >>>> On Thu, Sep 13, 2018 at 10:35:09 +0400, Dmitry Melekhov wrote: >>>>> After some mistakes yesterday we ( me and my colleague ) think that it >>>>> will be wise for libvirt to check config file existence on remote side >>>> Which config file? >>>> >>> VM config file, namely qemu. >> Do you mean the domain XML definition found in /etc/libvirt/qemu/*.xml ? > Yes and no, xml, but at another location. > >>> We forgot to mount shared storage (namely gluster volume), on which we >>> have config files, twice... >> If you're talking about the domain XML file, then you should never store >> them on a shared storage. All XML files in /etc/libvirt are supposed to >> be local to each host. > They are not stored in /etc/libvirt , but in other location, which is > suitable for sharing. I'm even more confused now. How could libvirt even know about such files? Unless you use transient domains, i.e., point libvirt to the right XML when starting the domain.

> Unfortunately, my colleague left today  for 2 weeks vacations and I have > no access to these hosts, > so I will be able to look in logs or do some tests only in October. > So, I'll write there again after 2 weeks or so. Yeah, it would be nice if you could share the actual domain XML and the command you used for migrating it to the other host, too.

On Fri, Sep 14, 2018 at 16:00:43 +0400, Dmitry Melekhov wrote: > 14.09.2018 15:43, Jiri Denemark пишет: >> On Thu, Sep 13, 2018 at 19:37:00 +0400, Dmitry Melekhov wrote: >>> 13.09.2018 18:57, Jiri Denemark пишет: >>>> On Thu, Sep 13, 2018 at 18:38:57 +0400, Dmitry Melekhov wrote: >>>>> 13.09.2018 17:47, Jiri Denemark пишет: >>>>>> On Thu, Sep 13, 2018 at 10:35:09 +0400, Dmitry Melekhov wrote: >>>>>>> After some mistakes yesterday we ( me and my colleague ) think that it >>>>>>> will be wise for libvirt to check config file existence on remote side >>>>>> Which config file? >>>>>> >>>>> VM config file, namely qemu. >>>> Do you mean the domain XML definition found in /etc/libvirt/qemu/*.xml ? >>> Yes and no, xml, but at another location. >>> >>>>> We forgot to mount shared storage (namely gluster volume), on which we >>>>> have config files, twice... >>>> If you're talking about the domain XML file, then you should never store >>>> them on a shared storage. All XML files in /etc/libvirt are supposed to >>>> be local to each host. >>> They are not stored in /etc/libvirt , but in other location, which is >>> suitable for sharing. >> I'm even more confused now. How could libvirt even know about such >> files? Unless you use transient domains, i.e., point libvirt to the >> right XML when starting the domain. > Yes, it transient, really, we use pacemaker's VirtualDomain > http://www.linux-ha.org/wiki/VirtualDomain_(resource_agent) > > pacemaker creates domain and destroys it. > > As I can see from sources migration looks like: > virsh ${VIRSH_OPTIONS} migrate --live $migrate_opts $DOMAIN_NAME > $remoteuri $migrateuri & > > Don't know which exact options it uses though. > > Do we need xml on another host if domain is transient for live migration? No, the XML is only needed when starting the domain. At the beginning of migration libvirt transfers the XML to libvirtd on the other host so that QEMU with the right parameters can be started there.

Just to make sure, the disk images used by the domain are stored on a shared storage accessible from both hosts, right?