Re: [libvirt] Error starting domain: internal error: missing IFLA_VF_INFO in netlink response

-----Original Message----- From: Yin Olivia-R63875 Sent: Monday, January 12, 2015 5:48 PM To: 'Laine Stump' Cc: libvir-list(a)redhat.com Subject: RE: Error starting domain: internal error: missing IFLA_VF_INFO in netlink response Hi Laine, Thank you for the reply. I tried to dump the information. > -----Original Message----- > From: sendmail [mailto:justsendmailnothingelse@gmail.com] On Behalf Of > Laine Stump > Sent: Thursday, January 08, 2015 11:13 PM > To: Yin Olivia-R63875 > Cc: libvir-list(a)redhat.com > Subject: Re: Error starting domain: internal error: missing > IFLA_VF_INFO in netlink response > > On 01/08/2015 04:40 AM, Hong-Hua.Yin(a)freescale.com wrote: > > Hi Laine, > > > > Sorry to disturb you. > > It seemed this issue had been fixed in > > libvirt-1.2.2/libnl-3.2.22/linux-3.12. But > we still got the error on PowerPC platform. > > I'll appreciate if you could give any suggestion. We are not sure if > > any netlink > implementation in kernel space is missed. > > > > The scenario is a little complicated. We installed internal PF and > > VF kernel modules and want to use <interface type="hostdev" > > managed="yes"> > syntax to start a guest domain with MAC address. > > > > # insmod fslinic.ko max_vfs=2 > > Freescale 10 Gigabit PCI Express Network Driver fslinic 0000:01:00.0: > > Multiqueue Enabled: Rx Queue count = 1, Tx Queue count = 1 fslinic > > 0000:01:00.0: Freescale (R) 10 Gigabit Network Connection fslinic > > 0000:01:00.1: Multiqueue Enabled: Rx Queue count = 1, Tx Queue count > > = > > 1 fslinic 0000:01:00.1: Freescale (R) 10 Gigabit Network Connection > > > > # insmod fslinicvf.ko > > Freescale 10 Gigabit PCI Express Network Driver > > > > # lspci -mk > > 00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" "" > > 01:00.0 "Power PC" "Freescale Semiconductor Inc" "Device 0440" -r20 "" "" > > 01:00.1 "Power PC" "Freescale Semiconductor Inc" "Device 0440" -r20 "" "" > > 01:00.4 "Power PC" "Freescale Semiconductor Inc" "Device 0000" -r20 "" "" > > 01:00.5 "Power PC" "Freescale Semiconductor Inc" "Device 1957" -r20 "" "" > > 01:01.0 "Power PC" "Freescale Semiconductor Inc" "Device 0000" -r20 "" "" > > 01:01.1 "Power PC" "Freescale Semiconductor Inc" "Device 1957" -r20 "" "" > > 0001:00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" "" > > 0002:00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" "" > > What is the output of "virsh nodedev-dumpxml pci_0000_01_00_4"? It > should contain an element like this: > > <capability type='phys_function'> > ... > > This indicates that it is an sriov VF (its PF will be listed in the > above <capability> element). If this element is missing, there is > something incomplete about the SRIOV implementation for your card's driver. > > It seemed that card driver is OK. ~# virsh nodedev-dumpxml pci_0000_01_00_4 <device> <name>pci_0000_01_00_4</name> <path>/sys/devices/ffe240000.pcie/pci0000:00/0000:00:00.0/0000:01:00.4</path> <parent>pci_0000_00_00_0</parent> <driver> <name>fslinicvf</name> </driver> <capability type='pci'> <domain>0</domain> <bus>1</bus> <slot>0</slot> <function>4</function> <product id='0x0000' /> <vendor id='0x1957' /> <capability type='phys_function'> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </capability> <iommuGroup number='78'> <address domain='0x0000' bus='0x01' slot='0x00' function='0x4'/> </iommuGroup> </capability> </device> > > > > # echo 1957 0000 > /sys/bus/pci/drivers/vfio-pci/new_id > > # echo 1957 1957 > /sys/bus/pci/drivers/vfio-pci/new_id > > > You shouldn't need to do the above when managed='yes', since libvirt > does it for you (doing it manually like this may or may not have > adverse consequences, I haven't tried it) OK, I got it. > > > > > > <interface type="hostdev" managed="yes"> > > <mac address="00:e0:0c:00:20:01"/> > > <source> > > <address type="pci" domain="0x0000" bus="0x01" slot="0x00" > function="0x04"/> > > </source> > > </interface> > > <memballoon model='virtio'/> > > </devices> > > </domain> > > > > root@t4240rdb:/var/volatile# virsh start interface > > error: Failed to start domain interface > > error: internal error: missing IFLA_VF_INFO in netlink response > > At some point (it's been quite awhile, so I've forgotten exactly when) > the kernel stopped automatically including the IFLA_VF_INFO object in > the response to a netlink RTM_GETLINK request. In order for this info > to be included, you need to add an IFLA_EXT_MASK with the > RTEXT_FILTER_VF bit set into the RTM_GETLINK request. This is done in > src/util/virnetdev.c:virNetDevLinkDump() - here are two bugzilla > records describing the problem and pointing at the patches added to fix it: > > https://bugzilla.redhat.com/show_bug.cgi?id=889249 > https://bugzilla.redhat.com/show_bug.cgi?id=889319 > > That particular code isn't compiled into libvirt unless > RTEXT_FILTER_VF is #defined, which should be done in > /usr/include/linux/rtnetlink.h. If the kernel headers used when > building libvirt were missing that #define (or maybe instead of it > being #defined, it is defined as a value in an enum?), then > RTEXT_FILTER_VF wouldn't be set in the request. Of course if > RTEXT_FILTER_VF really doesn't exist, then the kernel should be returning IFLA_VF_INFO without needing the extra flag in the request. > > Hopefully something in here will point you at the source of your problem. We use linux-3.12 with RTEXT_FILTER_VF defined. $ grep -r RTEXT_FILTER_VF ./ ./include/uapi/linux/rtnetlink.h:#define RTEXT_FILTER_VF (1 << 0) So I guess your patches had been compiled into libvirt. Best Regards, Olivia