Re: [libvirt] VFIO device groups and libvirt

VFIO device assignment has a concept of device "groups"; each group is a set of devices that must all be assigned to the same guest (alternately, unassigned devices can be attached to the vfio stub driver). This is to prevent cross-guest (or guest-host) contamination of devices that share DMA mapping or some such thing and thus can't be completely isolated from each other. The vfio support currently in libvirt works with no problems if no other devices are in the same "group" as a device being assigned to a guest. If there are other devices in the group, it fails with a message noting that the other devices in the group must also be attached to the vfio stub driver (so that the host can't use them). I am now adding code to libvirt that, when receiving a request to attach a device in a group to the vfio stub, will notice the other devices in the group and attach them to the stub as well; it will then allow attaching all of these devices to the guest, but not to any other guest. This will be nice for those who *want* this behavior, but I think that very often a person trying to assign a device that's in a group doesn't even realize that it's in a group, and if they knew it was in a group, they *wouldn't* want to lose all those devices just to assign one (once a single device in a group is assigned to the guest, none of the other devices in the group are usable by the host). An example of this was encountered by Cole the other day. He had a non-SRIOV network card plugged into his system that he wanted to assign with VFIO, but it turned out that due to the hardware layout of his motherboard, this plugin network card was placed in a group with 2 devices that were part of the motherboard chipset. When this occurs, a user might want to take one of three different courses: 1) attach all those onboard devices to the vfio stub driver too (making them inaccessible to the host, but allowing the assignment of the target device via VFIO) 2) use legacy KVM device assignment instead (which is less secure, but will allow assigning one of the devices while the rest remain available to the host) 3) try plugging the card they want to assign into a different slot (which may remove the device from this group and place it in another, possibly empty, group) It's difficult to say which the user will want, so I don't want the new "attach all devices in the group to the stub" code to be automatic. Instead, I think it should only take effect if some option is specified in the device's <driver> element. For example: <hostdev managed='yes'> <driver name='vfio' group='allow'/> ... </hostdev> Without that xml option, the same error message logged now will continue to be logged. With that xml option, all the devices in the group will be attached to the vfio stub driver. Likewise, in virNodeDeviceDetachFlags(), a VIR_NODE_DEVICE_DETACH_GROUP flag will have the following effect: Single device in group - no effect Multiple devices, no flag - detach will fail Multiple devices, flag set - all devices in the same group as specified device will be detached by one call. Does this sound reasonable?