Re: [libvirt] Error starting domain: internal error: missing IFLA_VF_INFO in netlink response

-----Original Message----- From: sendmail [mailto:justsendmailnothingelse@gmail.com] On Behalf Of Laine Stump Sent: Thursday, January 08, 2015 11:13 PM To: Yin Olivia-R63875 Cc: libvir-list(a)redhat.com Subject: Re: Error starting domain: internal error: missing IFLA_VF_INFO in netlink response On 01/08/2015 04:40 AM, Hong-Hua.Yin(a)freescale.com wrote: > Hi Laine, > > Sorry to disturb you. > It seemed this issue had been fixed in libvirt-1.2.2/libnl-3.2.22/linux-3.12. But we still got the error on PowerPC platform. > I'll appreciate if you could give any suggestion. We are not sure if any netlink implementation in kernel space is missed. > > The scenario is a little complicated. We installed internal PF and VF > kernel modules and want to use <interface type="hostdev" managed="yes"> syntax to start a guest domain with MAC address. > > # insmod fslinic.ko max_vfs=2 > Freescale 10 Gigabit PCI Express Network Driver fslinic 0000:01:00.0: > Multiqueue Enabled: Rx Queue count = 1, Tx Queue count = 1 fslinic > 0000:01:00.0: Freescale (R) 10 Gigabit Network Connection fslinic > 0000:01:00.1: Multiqueue Enabled: Rx Queue count = 1, Tx Queue count = > 1 fslinic 0000:01:00.1: Freescale (R) 10 Gigabit Network Connection > > # insmod fslinicvf.ko > Freescale 10 Gigabit PCI Express Network Driver > > # lspci -mk > 00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" "" > 01:00.0 "Power PC" "Freescale Semiconductor Inc" "Device 0440" -r20 "" "" > 01:00.1 "Power PC" "Freescale Semiconductor Inc" "Device 0440" -r20 "" "" > 01:00.4 "Power PC" "Freescale Semiconductor Inc" "Device 0000" -r20 "" "" > 01:00.5 "Power PC" "Freescale Semiconductor Inc" "Device 1957" -r20 "" "" > 01:01.0 "Power PC" "Freescale Semiconductor Inc" "Device 0000" -r20 "" "" > 01:01.1 "Power PC" "Freescale Semiconductor Inc" "Device 1957" -r20 "" "" > 0001:00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" "" > 0002:00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" "" What is the output of "virsh nodedev-dumpxml pci_0000_01_00_4"? It should contain an element like this: <capability type='phys_function'> ... This indicates that it is an sriov VF (its PF will be listed in the above <capability> element). If this element is missing, there is something incomplete about the SRIOV implementation for your card's driver.

> > # echo 1957 0000 > /sys/bus/pci/drivers/vfio-pci/new_id > # echo 1957 1957 > /sys/bus/pci/drivers/vfio-pci/new_id You shouldn't need to do the above when managed='yes', since libvirt does it for you (doing it manually like this may or may not have adverse consequences, I haven't tried it)

> > <interface type="hostdev" managed="yes"> > <mac address="00:e0:0c:00:20:01"/> > <source> > <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x04"/> > </source> > </interface> > <memballoon model='virtio'/> > </devices> > </domain> > > root@t4240rdb:/var/volatile# virsh start interface > error: Failed to start domain interface > error: internal error: missing IFLA_VF_INFO in netlink response At some point (it's been quite awhile, so I've forgotten exactly when) the kernel stopped automatically including the IFLA_VF_INFO object in the response to a netlink RTM_GETLINK request. In order for this info to be included, you need to add an IFLA_EXT_MASK with the RTEXT_FILTER_VF bit set into the RTM_GETLINK request. This is done in src/util/virnetdev.c:virNetDevLinkDump() - here are two bugzilla records describing the problem and pointing at the patches added to fix it: https://bugzilla.redhat.com/show_bug.cgi?id=889249 https://bugzilla.redhat.com/show_bug.cgi?id=889319 That particular code isn't compiled into libvirt unless RTEXT_FILTER_VF is #defined, which should be done in /usr/include/linux/rtnetlink.h. If the kernel headers used when building libvirt were missing that #define (or maybe instead of it being #defined, it is defined as a value in an enum?), then RTEXT_FILTER_VF wouldn't be set in the request. Of course if RTEXT_FILTER_VF really doesn't exist, then the kernel should be returning IFLA_VF_INFO without needing the extra flag in the request. Hopefully something in here will point you at the source of your problem.