4:54 a.m.
When I first put in support for VFIO device assignment, I didn't
realize that groups of devices were quite as common as they actually
are. In particular, I didn't know that often multiple
seemingly-unrelated devices can end up in the same VFIO iommu group
due to unlucky circumstances of hardware - they may share a dma
controller which means that the devices can't truly be isolated from
each other, and thus should not be simultaneously assigned to
different guests (or even used by the host) - all of the devices in a
group should be either assigned to the same guest or, if not assigned
to the guest, should be isolated off in a driver to prevent them
from being used by the host.
The following set of patches makes setting that up easier to deal
with. The end result of all the patches is the following:
1) The virNodeDevice API will be able to detach or re-attach all the
devices in a particular group with a single API call.
2) <hostdev managed='yes'>, <interface type='hostdev'
managed='yes'>,
and <interface type='network' managed='yes'> devices (where the
network is itself a pool of SRIOV Virtual Functions) can specify:
<driver name='vfio' group='auto'/>
and libvirt will automatically detach (and bind to the 'vfio-pci'
driver for assignment/isolation) all devices in the same group as
the device being assigned. Likewise, when the device it detached
from the guest, a check will be made and, if none of the devices in
the same group as the device being detach is still in use by a guest
As usual there are a lot of patches here, but many of them are
extremely simple, so don't be put off by the count.
Also, please note that patches 15/22 - 18/22 provide a new public API
(virNodeDeviceReAttachFlags()). I kept the new API later in the series
just in case someone wanted to backport as much as possible of this
patchset to an old release.
Laine Stump (22):
syntax: virPCIDeviceFree is also a NOP for NULL args
pci: change stubDriver from const char* to char*
pci: new utility functions
pci: eliminate memory leak in virPCIDeviceReattach
pci: make virPCIDeviceDetach consistent in behavior
pci: eliminate repetitive path constructions in
virPCIDeviceBindToStub
pci: eliminate unused driver arg from virPCIDeviceDetach
pci: update stubDriver name in virPCIDeviceBindToStub
pci: rename virPCIDeviceGetVFIOGroupDev to
virPCIDeviceGetIOMMUGroupDev
pci: make virPCIParseDeviceAddress public
pci: new iommu_group functions
pci: optionally detach/reattach all devices in a VFIO group
API & qemu: add ability to detach an entire VFIO group of devices
virsh: add option to detach entire group of devices
API: new virNodeDeviceReAttachFlags
API: implement RPC calls for virNodeDeviceReAttachFlags
qemu: implement virNodeDeviceReAttachFlags
xen: implement virNodeDeviceReAttachFlags
virsh: add option to attach entire group of devices
nodedev: add iommuGroup to node device object
conf: add <driver group='auto'> to hostdev, interface, and networks
qemu: implement backend of <driver group='auto'/>
cfg.mk | 1 +
docs/formatdomain.html.in | 94 ++-
docs/formatnetwork.html.in | 11 +
docs/formatnode.html.in | 63 +-
docs/schemas/domaincommon.rng | 16 +
docs/schemas/network.rng | 8 +
docs/schemas/nodedev.rng | 11 +
include/libvirt/libvirt.h.in | 18 +
src/conf/domain_conf.c | 36 +-
src/conf/domain_conf.h | 13 +
src/conf/network_conf.c | 39 +-
src/conf/network_conf.h | 14 +
src/conf/node_device_conf.c | 86 ++-
src/conf/node_device_conf.h | 5 +-
src/driver.h | 5 +
src/libvirt.c | 60 ++
src/libvirt_private.syms | 11 +-
src/libvirt_public.syms | 4 +
src/network/bridge_driver.c | 22 +
src/node_device/node_device_udev.c | 21 +-
src/qemu/qemu_cgroup.c | 4 +-
src/qemu/qemu_driver.c | 40 +-
src/qemu/qemu_hostdev.c | 39 +-
src/remote/remote_driver.c | 29 +
src/remote/remote_protocol.x | 12 +-
src/remote_protocol-structs | 5 +
src/security/security_apparmor.c | 2 +-
src/security/security_dac.c | 4 +-
src/security/security_selinux.c | 4 +-
src/util/virpci.c | 707 ++++++++++++++++++---
src/util/virpci.h | 37 +-
src/xen/xen_driver.c | 22 +-
tests/networkxml2xmlin/hostdev-pf.xml | 2 +-
tests/networkxml2xmlout/hostdev-pf.xml | 2 +-
tests/nodedevschemadata/pci_8086_10c9_sriov_pf.xml | 16 +
tests/nodedevxml2xmltest.c | 1 +
.../qemuxml2argvdata/qemuxml2argv-hostdev-vfio.xml | 2 +-
.../qemuxml2argv-net-hostdev-vfio.xml | 2 +-
tools/virsh-nodedev.c | 30 +-
39 files changed, 1340 insertions(+), 158 deletions(-)
create mode 100644 tests/nodedevschemadata/pci_8086_10c9_sriov_pf.xml
--
1.7.11.7