[libvirt] libvirt VF indexing issue
libvirt points to wrong VF -channel, when trying to change MAC -address.
libvirt v1 1.0.2
linux driver : ixgbe 3.17.3
Symptom description
----------------------------------
1.) How the VF channels are seen in the Host ( correct in increasing
address order )
command : ls -la /sys/bus/pci/devices/0000\:04\:00.0/
..................
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn0 -> ../0000:04:10.0/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn1 -> ../0000:04:10.2/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn10 -> ../0000:04:12.4/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn11 -> ../0000:04:12.6/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn12 -> ../0000:04:13.0/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn13 -> ../0000:04:13.2/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn2 -> ../0000:04:10.4/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn3 -> ../0000:04:10.6/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn4 -> ../0000:04:11.0/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn5 -> ../0000:04:11.2/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn6 -> ../0000:04:11.4/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn7 -> ../0000:04:11.6/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn8 -> ../0000:04:12.0/
lrwxrwxrwx 1 root root 0 Oct 17 12:25 virtfn9 -> ../0000:04:12.2/
-----------------------
2.) How the libvirt sees them
virsh nodedev-dumpxml pci_0000_04_00_0
<device>
<name>pci_0000_04_00_0</name>
<parent>pci_0000_00_03_0</parent>
<driver>
<name>ixgbe</name>
</driver>
<capability type='pci'>
<domain>0</domain>
<bus>4</bus>
<slot>0</slot>
<function>0</function>
<product id='0x10fb'>82599EB 10-Gigabit SFI/SFP+ Network
Connection</product>
<vendor id='0x8086'>Intel Corporation</vendor>
<capability type='virt_functions'>
<address domain='0x0000' bus='0x04' slot='0x12'
function='0x4'/>
<address domain='0x0000' bus='0x04' slot='0x12'
function='0x6'/>
<address domain='0x0000' bus='0x04' slot='0x13'
function='0x0'/>
<address domain='0x0000' bus='0x04' slot='0x13'
function='0x2'/>
<address domain='0x0000' bus='0x04' slot='0x10'
function='0x0'/>
<address domain='0x0000' bus='0x04' slot='0x10'
function='0x2'/>
<address domain='0x0000' bus='0x04' slot='0x10'
function='0x4'/>
<address domain='0x0000' bus='0x04' slot='0x10'
function='0x6'/>
<address domain='0x0000' bus='0x04' slot='0x11'
function='0x0'/>
<address domain='0x0000' bus='0x04' slot='0x11'
function='0x2'/>
<address domain='0x0000' bus='0x04' slot='0x11'
function='0x4'/>
<address domain='0x0000' bus='0x04' slot='0x11'
function='0x6'/>
<address domain='0x0000' bus='0x04' slot='0x12'
function='0x0'/>
<address domain='0x0000' bus='0x04' slot='0x12'
function='0x2'/>
</capability>
</capability>
</device>
3.) Libvirt sets MAC -address in slot/position 5 instead of 1
ip link show eth10
87: eth10: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 2100 qdisc mq
state UP mode DEFAULT qlen 1000
link/ether 00:1b:21:b9:a5:60 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 92:e2:6b:53:a7:c2, spoof checking off <------ this is the
slot, MAC tried to set
vf 1 MAC 0a:04:38:2c:79:35, spoof checking off
vf 2 MAC b2:46:b6:91:44:2e, spoof checking off
vf 3 MAC de:b6:28:da:20:44, spoof checking off
vf 4 MAC 02:00:00:80:00:01, spoof checking off <--------- MAC sets
in wrong slot
vf 5 MAC 02:00:00:80:00:02, spoof checking off
vf 6 MAC 02:00:00:80:00:20, spoof checking off
vf 7 MAC 02:00:00:80:00:30, spoof checking off
vf 8 MAC 52:54:00:de:f8:08, spoof checking off
vf 9 MAC 5e:b0:d0:34:d6:cc, spoof checking off
vf 10 MAC d2:88:5c:2d:db:b5, spoof checking off
vf 11 MAC 6a:06:8a:9c:31:8b, spoof checking off
vf 12 MAC 4a:38:be:8f:b3:bf, spoof checking off
vf 13 MAC b6:c1:d4:12:d3:3d, spoof checking off
5.) Question : Why this happens in ixgbe -driver and when we try to
use more than 10 VF's/physical port. ?