[RFC] Seeking advice on support for generic vDPA device
vDPA devices allow high-performance devices in a virtual machine by providing a wire-speed data path. These devices require a vendor-specific host driver but the data path follows the virtio specification. The support for generic-vdpa device was recently added to qemu in v8.0.0-rc version. https://gitlab.com/qemu-project/qemu/-/commit/b430a2bd2303b9940cc80ec746887f... This allows libvirt to support these deviecs regardless of device type. Currently, libvirt supports vDPA network devices, We need a new solution to support generic-vdpa devices. At present, we have considered several options. Solution A: Add type vdpa under the hostdev device，the device xml looks like <devices> <hostdev mode='subsystem' type='vdpa'> <source dev='/dev/vhost-vdpa-0'/> </hostdev> </devices> This solution is recommended. This configuration requires only that the device supports the vDPA framework, easy for future expansion and same as the qemu solution design idea. Solution B: The current libvirt mode is used to add vDPA devices under different devices. device xml looks like: network: <devices> <interface type='generic-vdpa'> <source dev='/dev/vhost-vdpa-0'/> </interface> </devices> storage: <devices> <disk type='generic-vdpa’> <source dev='/dev/vhost-vdpa-0'/> </disk> </devices> The network interface has the vdpa type. Therefore, a new type is required to distinguish the "generic-vdpa" type. However, this method differentiates device types, which is not convenient for the expansion of other types of vDPA devices. Not recommended. Solution C: Referring to PCI passthrough, Add the driver type (appropriate vendor-specific driver) to the driver of the PCI device. The unbind/bind of the vDPA device driver and the creation of VDPA device are implemented by libvirt. device xml looks like: <devices> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vp-vdpa'/> <source> <address domain='0x0000' bus='0x04' slot='0x01' function='0x01'/> </source> </hostdev> </devices> This solution is related to the vendor-specific driver and supports only the vdpa created by the PCI device. vdpa devices created in vduse mode are not supported. This solution is not universal and is not recommended. What's your proposal?
On Sat, Mar 25, 2023 at 9:14 PM jiangdongxu <jiangdongxu1@huawei.com> wrote:
vDPA devices allow high-performance devices in a virtual machine by providing a wire-speed data path. These devices require a vendor-specific host driver but the data path follows the virtio specification.
The support for generic-vdpa device was recently added to qemu in v8.0.0-rc version. https://gitlab.com/qemu-project/qemu/-/commit/b430a2bd2303b9940cc80ec746887f...
This allows libvirt to support these deviecs regardless of device type. Currently, libvirt supports vDPA network devices, We need a new solution to support generic-vdpa devices.
At present, we have considered several options.
Solution A: Add type vdpa under the hostdev device，the device xml looks like <devices> <hostdev mode='subsystem' type='vdpa'> <source dev='/dev/vhost-vdpa-0'/> </hostdev> </devices> This solution is recommended. This configuration requires only that the device supports the vDPA framework, easy for future expansion and same as the qemu solution design idea.
I prefer this one, it's easier to implement and works for any type of virtio devices and vDPA parents
Solution B: The current libvirt mode is used to add vDPA devices under different devices. device xml looks like: network: <devices> <interface type='generic-vdpa'> <source dev='/dev/vhost-vdpa-0'/> </interface> </devices>
storage: <devices> <disk type='generic-vdpa’> <source dev='/dev/vhost-vdpa-0'/> </disk> </devices>
The network interface has the vdpa type. Therefore, a new type is required to distinguish the "generic-vdpa" type. However, this method differentiates device types, which is not convenient for the expansion of other types of vDPA devices. Not recommended.
This should be the way when we have dedicated block layer support for vhost-vDPA. Which seems not necessary for general vDPA devices.
Solution C: Referring to PCI passthrough, Add the driver type (appropriate vendor-specific driver) to the driver of the PCI device. The unbind/bind of the vDPA device driver and the creation of VDPA device are implemented by libvirt. device xml looks like: <devices> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vp-vdpa'/> <source> <address domain='0x0000' bus='0x04' slot='0x01' function='0x01'/> </source> </hostdev> </devices> This solution is related to the vendor-specific driver and supports only the vdpa created by the PCI device. vdpa devices created in vduse mode are not supported. This solution is not universal and is not recommended.
I agree, this requires PCI specific knowledge which defeats the goal of vDPA to be device agnostic. Thanks
What's your proposal?
participants (2)
-
Jason Wang -
jiangdongxu