Re: [libvirt] [RFC PATCH 00/16] Introduce vGPU mdev framework to libvirt

On Mon, Feb 06, 2017 at 01:19:42PM +0100, Erik Skultety wrote: > Finally. It's here. This is the initial suggestion on how libvirt might > interract with the mdev framework, currently only focussing on the non-managed > devices, i.e. those pre-created by the user, since that will be revisited once > we all settled on how the XML should look like, given we might not want to use > the sysfs path directly as an attribute in the domain XML. My proposal on the > XML is the following: > > <hostdev mode='subsystem' type='mdev'> > <source> > <!-- this is the host's physical device address --> > <address domain='0x0000' bus='0x00' slot='0x00' function='0x00'> > <uuid>vGPU_UUID<uuid> > <source> > <!-- target PCI address can be omitted to assign it automatically --> > </hostdev> > > So the mediated device is identified by the physical parent device visible on > the host and a UUID which allows us to construct the sysfs path by ourselves, > which we then put on the QEMU's command line. > > A few remarks if you actually happen to have a machine to test this on: > - right now the mediated devices are one-time use only, i.e. they have to be > recreated before every machine boot > - I wouldn't recommend assigning multiple vGPUs to a single domain > > Once this series is sorted out, we can then continue with 'managed=yes' where > as Laine pointed out [1], we need to figure out how exactly should the > management layer hint libvirt which vGPU type should be used for device > instantiation. You seem to be suggesting that managed=yes with mdev devices would cause create / delete of a mdev device from a specified parent. This is rather different semantics from what managed=yes does with PCI device assignment today. There the managed=yes flag is just about controlling host device driver attachment. ie whether libvirt will manually bind to vfio.ko, or expect the admin to have bound it to vfio.ko before hand. I think it is important to keep that concept as is for mdev too. While we're thinking of mdev purely in terms of KVM + vfio usage, it wouldn't suprise me if there ended up being non-KVM based use cases for mdev. It isn't clear to me that auto-creation of mdev devices as a concept even belongs in the domain XML neccessarily. Looking at two similar areas. For SRIOV NICs, in the domain XML you either specify an explicit VF to use, or you reference a libvirt virtual network. The latter takes care of dynamically providing VFs to VMs. For NPIV, IIRC, the domain XML works similarly either taking an explicit vHBA, or referencing a storage pool to get one more dynamically.

Before we even consider auto-creation though, I think we need to have manual creation designed & integrated in the node device APIs. So in terms of the domain XML, I think the only think we need to provide is the address of the pre-existing mdev device to be used. In this case "address" means the UUID. We should not need anything about the parent device AFAICT.