11:10 p.m.
Hi all,
I'm testing on debian/unstable SR-IOV feature with Broadcom BCM57810
cards and KVM hypervisor:
Compiled against library: libvirt 1.2.1
Using library: libvirt 1.2.1
Using API: QEMU 1.2.1
Running hypervisor: QEMU 1.7.0
bnx2x
-> firmware 7.8.17
-> driver from kernel 3.12.7
8 VFs are created on the first PF. For each VF, a specific mac address
is set manually using "ip link set eth0 vf x mac xx:xx:xx:xx:xx"
command. I run several KVM guests with PCI passthrough (same kernel,
bnx2x driver and firmware as the host), performance is close to bare metal.
Well, that sounds good, until I start capturing the traffic inside each
VM: host traffic is visible as well as traffic destined to other VM.
It's like if internal card switching was inoperable. I made several
tests with different kernels, different PCIe Passthrough method
assignments for libvirt. All failed.
Has anyone successfully experiment SR-IOV with Broadcom cards on linux ?
-----
Some details:
01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810
10 Gigabit Ethernet (rev 10)
01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810
10 Gigabit Ethernet (rev 10)
01:09.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810
10 Gigabit Ethernet Virtual Function
01:09.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810
10 Gigabit Ethernet Virtual Function
01:09.2 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810
10 Gigabit Ethernet Virtual Function
01:09.3 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810
10 Gigabit Ethernet Virtual Function
01:09.4 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810
10 Gigabit Ethernet Virtual Function
01:09.5 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810
10 Gigabit Ethernet Virtual Function
01:09.6 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810
10 Gigabit Ethernet Virtual Function
01:09.7 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810
10 Gigabit Ethernet Virtual Function
# virsh nodedev-dumpxml pci_0000_01_09_0
<device>
<name>pci_0000_01_09_0</name>
<path>/sys/devices/pci0000:00/0000:00:01.0/0000:01:09.0</path>
<parent>pci_0000_00_01_0</parent>
<driver>
<name>vfio-pci</name>
</driver>
<capability type='pci'>
<domain>0</domain>
<bus>1</bus>
<slot>9</slot>
<function>0</function>
<product id='0x16af'>NetXtreme II BCM57810 10 Gigabit Ethernet
Virtual Function</product>
<vendor id='0x14e4'>Broadcom Corporation</vendor>
<capability type='phys_function'>
<address domain='0x0000' bus='0x01' slot='0x00'
function='0x1'/>
</capability>
<iommuGroup number='35'>
<address domain='0x0000' bus='0x01' slot='0x09'
function='0x0'/>
</iommuGroup>
</capability>
</device>
# virsh nodedev-dumpxml pci_0000_01_09_1
<device>
<name>pci_0000_01_09_1</name>
<path>/sys/devices/pci0000:00/0000:00:01.0/0000:01:09.1</path>
<parent>pci_0000_00_01_0</parent>
<driver>
<name>vfio-pci</name>
</driver>
<capability type='pci'>
<domain>0</domain>
<bus>1</bus>
<slot>9</slot>
<function>1</function>
<product id='0x16af'>NetXtreme II BCM57810 10 Gigabit Ethernet
Virtual Function</product>
<vendor id='0x14e4'>Broadcom Corporation</vendor>
<capability type='phys_function'>
<address domain='0x0000' bus='0x01' slot='0x00'
function='0x1'/>
</capability>
<iommuGroup number='36'>
<address domain='0x0000' bus='0x01' slot='0x09'
function='0x1'/>
</iommuGroup>
</capability>
</device>
Guest A XML:
...
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x01' slot='0x09'
function='0x0'/>
</source>
</hostdev>
...
Guest B XML:
...
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x01' slot='0x09'
function='0x1'/>
</source>
</hostdev>
...
--
Université de Nantes - Direction des Systèmes d'Information