On 8/26/24 1:07 PM, daggs via Users wrote:
Greetings Michal,
> Sent: Monday, August 26, 2024 at 12:42 PM
> From: "Michal Prívozník" <mprivozn(a)redhat.com>
> To: "daggs" <daggs(a)gmx.com>, users(a)lists.libvirt.org
> Subject: Re: splitting a bt wifi combo between two vms
>
> On 8/26/24 10:37, daggs via Users wrote:
>> Greetings,
>>
>> I have the following combo device:
>> dagg@NCC-5001D ~ $ lsusb -d8087:0029
>> Bus 001 Device 002: ID 8087:0029 Intel Corp. AX200 Bluetooth
>> dagg@NCC-5001D ~ $ lspci -vv -s 03:00.0
>> 03:00.0 Network controller: Intel Corporation Wi-Fi 6 AX200 (rev 1a)
>> Subsystem: Intel Corporation Wi-Fi 6 AX200NGW
>> Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
>> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
>> Interrupt: pin A routed to IRQ 34
>> IOMMU group: 13
>> Region 0: Memory at fc600000 (64-bit, non-prefetchable) [size=16K]
>> Capabilities: <access denied>
>> Kernel modules: iwlwifi
>>
>> I want to pass each of them to a different vm.
>> should I expect problems?
> If you assign a PCI device to a VM then the host has to release it.
> You're not showing the full PCI device config (I assume bluetooth is
> another PCI function), but I'd assume it's all or nothing. I don't think
> I've seen a PCI device that allows detaching a single PCI function
> (apart from SRIO-V, which are specifically designed to do this). IOW, if
> you assign the 03:00.0 PCI device to a VM then the other function
> (bluetooth) has to be detached too at which point it can not be assigned
> to another VM.
>
> This can be verified via virsh:
>
> virsh nodedev-dumpxml pci_0000_03_00_0
>
> in the printed XML you'll find <iommuGroup/> element, e.g.:
>
> <iommuGroup number='14'>
> <address domain='0x0000' bus='0x03' slot='0x00'
function='0x0'/>
> </iommuGroup>
>
> If there's just one <address/> child element, then the device is in its
> own iommu group and can be detached alone. If there are multiple child
> elements, e.g.:
>
> <iommuGroup number='16'>
> <address domain='0x0000' bus='0x07' slot='0x01'
function='0x0'/>
> <address domain='0x0000' bus='0x07' slot='0x04'
function='0x0'/>
> <address domain='0x0000' bus='0x08' slot='0x00'
function='0x0'/>
> <address domain='0x0000' bus='0x07' slot='0x00'
function='0x0'/>
> <address domain='0x0000' bus='0x06' slot='0x00'
function='0x0'/>
> <address domain='0x0000' bus='0x04' slot='0x01'
function='0x0'/>
> </iommuGroup>
>
> then all devices within the iommu group have to be detached from the
> host before giving any device to a VM.
>
> Michal
>
>
that is the thing, from what I can see, this combo device has a pci device which is the
wifi, see:
$ tail /sys/bus/pci/devices/0000\:03\:00.0/{device,vendor}
==> /sys/bus/pci/devices/0000:03:00.0/device <==
0x2723
==> /sys/bus/pci/devices/0000:03:00.0/vendor <==
0x8086
dagg@NCC-5001D ~ $ ll /sys/bus/pci/devices/0000\:03\:00.0/iommu_group
lrwxrwxrwx 1 root root 0 Aug 26 09:44 /sys/bus/pci/devices/0000:03:00.0/iommu_group ->
../../../../../../kernel/iommu_groups/13
while the bt is a usb device, see:
$ tail /sys/bus/usb/devices/1-5/id*
==> /sys/bus/usb/devices/1-5/idProduct <==
0029
==> /sys/bus/usb/devices/1-5/idVendor <==
8087
both of them are AX200
Is this combo device on a PCIe card? If so, I would figure that the USB
device would be on top of a PCI device. Are there no other PCI devices
listed in lspci that might be the "PCI parent" of the USB controller?
(anyway if there are, my guess is that this device would also show up
with iommu group 13 since, as Michel suggested, most multifunction PCI
cards (with Michal's state exception of cards that support SRIOV) have
all the devices in a single iommu group, so you can't have them attached
to different guests (or have one attached to the host and one to a
guest); this is an intentional limitation because both devices use the
same memory/io space and so you would end up with two guests (or a guest
and a host) having access to portions of each other's memory/io.