Re: [libvirt] libvirt mdev migration, mdevctl integration

On Mon, Nov 25, 2019 at 06:14:33PM +0100, Cornelia Huck wrote: > On Mon, 18 Nov 2019 19:00:25 +0000 > Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: > > > On Mon, Nov 18, 2019 at 10:06:34AM -0700, Alex Williamson wrote:

> > > An issue here though is that the json will also include the parent > > > device, which we obviously cannot assume is the same (particularly the > > > bus address) on the migration target. We can allow commandline > > > overrides for the parent just as we do above for the UUID when defining > > > the mdev device from json, but it's an open issue who is going to be > > > smart enough (perhaps dumb enough) to claim this responsibility. It > > > would be interesting to understand how libvirt handles other host > > > specific information during migration, for instance if node or processor > > > affinities are part of the VM XML, how is that translated to the > > > target? I could imagine that we could introduce a simple "first > > > available" placement in mdevctl, but maybe there should minimally be a > > > node allocation preference with optional enforcement (similar to > > > numactl), or maybe something above libvirt needs to take this > > > responsibility to prepare the target before we get ourselves into > > > trouble. > > > > I don't think we need to solve placement in libvirt. > > > > The guest XML will just reference the mdev via a UUID that > > was used with virNodeDeviceDefineXML. > > > > The virNodeDeviceDefineXML call where the mdev is first defined > > will set the details of the mdev creation for this specific host. > > The XML used with virNodeDeviceDefineXML can be different on the > > source + target hosts. As long as the UUID is the same in both > > hosts, the VM will associate with it correctly. > > I wonder how to sync up with different placements, but maybe I'm just > missing something. > > Looking at this from the vfio-ccw angle, we can easily have the same > device (as identified by the device number) on different subchannels > (parents). To find out the device number, you need to look at the child > ccw device of the subchannel while it is *not* bound to vfio-ccw, but > to the normal I/O subchannel driver instead. Or ask your admin for the > system definition... This just means that whoever/whatever is invoking "virDomainDeviceDefinXML" or "mdevctl create" will pass different parameters on each host. When migrating a guest the mgmt app can indicate which device should be used for the guest on each host. This is similar issue to migrating a guest which uses a ethNNN device that's got different name on each host ,or a /dev/sdNNN that's different on each host, etc