[libvirt] VFIO device groups and libvirt
VFIO device assignment has a concept of device "groups"; each group is a
set of devices that must all be assigned to the same guest (alternately,
unassigned devices can be attached to the vfio stub driver). This is to
prevent cross-guest (or guest-host) contamination of devices that share
DMA mapping or some such thing and thus can't be completely isolated
from each other.
The vfio support currently in libvirt works with no problems if no other
devices are in the same "group" as a device being assigned to a guest.
If there are other devices in the group, it fails with a message noting
that the other devices in the group must also be attached to the vfio
stub driver (so that the host can't use them).
I am now adding code to libvirt that, when receiving a request to attach
a device in a group to the vfio stub, will notice the other devices in
the group and attach them to the stub as well; it will then allow
attaching all of these devices to the guest, but not to any other guest.
This will be nice for those who *want* this behavior, but I think that
very often a person trying to assign a device that's in a group doesn't
even realize that it's in a group, and if they knew it was in a group,
they *wouldn't* want to lose all those devices just to assign one (once
a single device in a group is assigned to the guest, none of the other
devices in the group are usable by the host).
An example of this was encountered by Cole the other day. He had a
non-SRIOV network card plugged into his system that he wanted to assign
with VFIO, but it turned out that due to the hardware layout of his
motherboard, this plugin network card was placed in a group with 2
devices that were part of the motherboard chipset. When this occurs, a
user might want to take one of three different courses:
1) attach all those onboard devices to the vfio stub driver too (making
them inaccessible to the host, but allowing the assignment of the target
device via VFIO)
2) use legacy KVM device assignment instead (which is less secure, but
will allow assigning one of the devices while the rest remain available
to the host)
3) try plugging the card they want to assign into a different slot
(which may remove the device from this group and place it in another,
possibly empty, group)
It's difficult to say which the user will want, so I don't want the new
"attach all devices in the group to the stub" code to be automatic.
Instead, I think it should only take effect if some option is specified
in the device's <driver> element. For example:
<hostdev managed='yes'>
<driver name='vfio' group='allow'/>
...
</hostdev>
Without that xml option, the same error message logged now will continue
to be logged. With that xml option, all the devices in the group will be
attached to the vfio stub driver.
Likewise, in virNodeDeviceDetachFlags(), a VIR_NODE_DEVICE_DETACH_GROUP
flag will have the following effect:
Single device in group - no effect
Multiple devices, no flag - detach will fail
Multiple devices, flag set - all devices in the same group as
specified device will be detached by one call.
Does this sound reasonable?