On 02/20/2015 05:33 PM, Florin Stingaciu wrote:
Hello,
For the last little while I've been trying to configure a VM with a
passthrough interface using SR-IOV. I've managed to configure SR-IOV
on the hypervisor correctly. I'm working with a Mellanox ConnectX3
card with two infiniband ports and hypervisors running Centos 7. I can
see the PCI devices (ie. virtual functions) using the `virsh
nodedev-list` command.
The majority of tutorials online (including the tutorials on the
libvirt site), suggest one configures an SR-IOV passthrough interface
in the following manner in the VM definition file:
<interface type='hostdev' managed='yes'>
<mac address='52:54:00:d7:cd:aa'/>
<source>
<address type='pci' domain='0x0000' bus='0x01'
slot='0x00'
function='0x2'/>
</source>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03'
function='0x0'/>
</interface>
For example the above configuration should configure the second VF as
a passthrough interface for this VM. When I try to launch the VM, I
get the following error:
error: Failed to start domain test
error: internal error: missing IFLA_VF_INFO in netlink response
I've done research on this error quite extensively. This used to be a
very popular error a while back for scenarios where the total number
of VFs is > 64. This is not the case here as I only have 8.
I've was never able to overcome this error but I did manage to get
this working another way. The VM will boot just fine and detect a new
interface if you define one of the VFs as a PCI in the following
manner within the VM definition file:
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x01' slot='0x00'
function='0x2'/>
</source>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x06'
function='0x0'/>
</hostdev>
As you can see the from the source bus and function, it's the same VF
I was trying with the definition above. I'm wondering if there is any
particular reason for this or how could I get around this error.
Normally, I would just use the second configuration, however I'm about
to begin work within an openstack environment where the sriovnicswitch
ml2 plugin will be used for networking. This plugin, when configuring
passthrough interfaces uses the first definition that does not work
and spits out that error.
Considering even the official libvirt tutorials suggest the first way,
I feel like I must be missing something, or something broke and noone
noticed in one of the latest updates. Any help would be greatly
appreciated.
No, the problem is with the drivers for the particular cards you are
using. To my understanding, the Mellanox cards have a bit of a different
architecture, and were not initially designed to be SRIOV-compliant (or
maybe they 'comply differently', I'm not sure of the exact details).
I would recommend looking to Mellanox for updated drivers for the card
for CentOS7.
Alternately, you could test it with the RHEL7.1 beta, which will have
newer Mellanox bits than whatever CentOS7 you're currently working with.