Re: [libvirt] support for configuring all ports of a multiport SRIOV VF when assigning to guest
On 08/05/2017 01:14 PM, Laine Stump wrote:
On 08/03/2017 02:33 AM, Moshe Levi wrote:
> Hi Laine,
>
> I have a few question before I can give my opinion.
> I the Mellanox Card Dual Port that support one PCI with 2 PF is ConnectX-3 and
ConnectX-3 Pro. (maybe others cards I will check this)
> The ConnectX-4 Dual Port and above is implemented with 2 PCI devices per 2 PF.
So is the "multiple netdevs for a single PCI device" hardware model
completely deprecated, and will never show up again in new products?
If that's the case, maybe I shouldn't burden libvirt's config with all
this new config that will only be used for legacy hardware. Perhaps a
better approach would be to stick with the current config, and make it
work properly for VFIO device assignment when a ConnectX-3 card is
configured in single port mode - even that doesn't work correctly now[*]
but it's a more easily solved problem, and can be done with no config
changes.
Opinions about this? If it's a deadend and the existing legacy hardware
can be used in a reasonable manner by setting it to single port mode, I
don't want to add externally-visible knobs to libvirt.
[*]If a VF is configured to be "port 2 only", libvirt would still try to
get/set the MAC and vlan tag with a netlink message to the *port 1*
netdev of the PF, so it would be saving/setting the wrong (nonexistent?)
VF netdev.
I just tried this and find I was partially mistaken. For example, lets
say that I have a PF with netdevs p2p1 (port 1) and p2p2 (port 2), and I
set it up to have the first 4 VFs on port 1 and the 2nd 4 on port 2. It
turns out that I can set the MAC/vlan tag using either p2p1 *or* p2p2
and it will be set in the correct place. However, if I try to *get* the
current MAC/vlan tag for a VF that uses port 2 via the PF netdev for
port 1, then I will just get back 0.
This might seem confusing/misleading, but it actually makes the job of
fixing device assignment for single port mode a bit simpler!
I did send patches yesterday (any reviews/testing appreciated!) that
make everything work properly when saving/setting/restoring the VF
netdev MAC/vlan tags on dual port cards used for macvtap passthrough:
https://www.redhat.com/archives/libvir-list/2017-August/msg00170.html
Those don't fix the situation when doing VFIO device assignment, but
they do at least make macvtap passthrough work correctly (for all VF
netdevs, even when the netdevs are dual port!), and are the first step
in getting it right for VFIO.