Re: [libvirt] [Qemu-devel] [RFC v2] libvirt vGPU QEMU integration

From: Daniel P. Berrange [mailto:berrange@redhat.com] Sent: Thursday, September 29, 2016 10:39 PM On Thu, Sep 29, 2016 at 02:35:48PM +0000, Tian, Kevin wrote: > > From: Daniel P. Berrange [mailto:berrange@redhat.com] > > Sent: Thursday, September 29, 2016 4:06 PM > > > > On Wed, Sep 28, 2016 at 12:48:33PM -0700, Neo Jia wrote: > > > On Tue, Sep 20, 2016 at 10:47:53AM +0100, Daniel P. Berrange wrote: > > > > On Tue, Sep 20, 2016 at 02:05:52AM +0530, Kirti Wankhede wrote: > > > > > > > > > > Hi libvirt experts, > > > > > > > > > > Thanks for valuable input on v1 version of RFC. > > > > > > > > > > Quick brief, VFIO based mediated device framework provides a way to > > > > > virtualize their devices without SR-IOV, like NVIDIA vGPU, Intel KVMGT > > > > > and IBM's channel IO. This framework reuses VFIO APIs for all the > > > > > functionalities for mediated devices which are currently being used for > > > > > pass through devices. This framework introduces a set of new sysfs files > > > > > for device creation and its life cycle management. > > > > > > > > > > Here is the summary of discussion on v1: > > > > > 1. Discover mediated device: > > > > > As part of physical device initialization process, vendor driver will > > > > > register their physical devices, which will be used to create virtual > > > > > device (mediated device, aka mdev) to the mediated framework. > > > > > > > > > > Vendor driver should specify mdev_supported_types in directory format. > > > > > This format is class based, for example, display class directory format > > > > > should be as below. We need to define such set for each class of devices > > > > > which would be supported by mediated device framework. > > > > > > > > > > --- mdev_destroy > > > > > --- mdev_supported_types > > > > > |-- 11 > > > > > | |-- create > > > > > | |-- name > > > > > | |-- fb_length > > > > > | |-- resolution > > > > > | |-- heads > > > > > | |-- max_instances > > > > > | |-- params > > > > > | |-- requires_group > > > > > |-- 12 > > > > > | |-- create > > > > > | |-- name > > > > > | |-- fb_length > > > > > | |-- resolution > > > > > | |-- heads > > > > > | |-- max_instances > > > > > | |-- params > > > > > | |-- requires_group > > > > > |-- 13 > > > > > |-- create > > > > > |-- name > > > > > |-- fb_length > > > > > |-- resolution > > > > > |-- heads > > > > > |-- max_instances > > > > > |-- params > > > > > |-- requires_group > > > > > > > > > > > > > > > In the above example directory '11' represents a type id of mdev device. > > > > > 'name', 'fb_length', 'resolution', 'heads', 'max_instance' and > > > > > 'requires_group' would be Read-Only files that vendor would provide to > > > > > describe about that type. > > > > > > > > > > 'create': > > > > > Write-only file. Mandatory. > > > > > Accepts string to create mediated device. > > > > > > > > > > 'name': > > > > > Read-Only file. Mandatory. > > > > > Returns string, the name of that type id. > > > > > > > > Presumably this is a human-targetted title/description of > > > > the device. > > > > > > > > > > > > > > 'fb_length': > > > > > Read-only file. Mandatory. > > > > > Returns <number>{K,M,G}, size of framebuffer. > > > > > > > > > > 'resolution': > > > > > Read-Only file. Mandatory. > > > > > Returns 'hres x vres' format. Maximum supported resolution. > > > > > > > > > > 'heads': > > > > > Read-Only file. Mandatory. > > > > > Returns integer. Number of maximum heads supported. > > > > > > > > None of these should be mandatory as that makes the mdev > > > > useless for non-GPU devices. > > > > > > > > I'd expect to see a 'class' or 'type' attribute in the > > > > directory whcih tells you what kind of mdev it is. A > > > > valid 'class' value would be 'gpu'. The fb_length, > > > > resolution, and heads parameters would only be mandatory > > > > when class==gpu. > > > > > > > > > > Hi Daniel, > > > > > > Here you are proposing to add a class named "gpu", which will make all those gpu > > > related attributes mandatory, which libvirt can allow user to better > > > parse/present a particular mdev configuration? > > > > > > I am just wondering if there is another option that we just make all those > > > attributes that a mdev device can have as optional but still meaningful to > > > libvirt, so libvirt can still parse / recognize them as an class "mdev". > > > > 'mdev' isn't a class - mdev is the name of the kernel module. The class > > refers to the broad capability of the device. class would be things > > like "gpu", "nic", "fpga" or other such things. The point of the class > > is to identify which other attributes will be considered mandatory. > > > > > > Thanks Daniel. This class definition makes sense to me. > > However I'm not sure whether we should define such common mandatory attributes > of a 'gpu' class now. Intel will go with a 2's power sharing of type definition... actual > type name to be finalized, but an example looks like below: > > [GVTG-SKL-x2]: available instances (2) > [GVTG-SKL-x4]: available instances (4) > [GVTG-SKL-x8]: available instances (8) > ... > > User can create different types of vGPUs simultaneously. A GVTG-SKL-x2 type > vGPU will get half of the physical GPU resource, while a GVTG-SKL-x4 type will > get a quarter. However it's unclear to me how we want to enumerate those > resources into resolution or heads. I feel it'd be more reasonable for us to push > initial libvirt mdev support w/o vgpu specific class definition, until we see > a clear value of doing so (at that time we then follow Daniel's guideline to define > mandatory attributes common to all GPU vendors). Libvirt won't report arbitrary vendor define attributes. So if we are not going to define a gpu class & associated attributes, then there will be no reporting of the 'heads', 'resolution', 'fb_length' data described above.