Re: [libvirt] <interface type='hostdev'>vf configuration cleanup when VM is delete

On 12/15/2015 01:34 PM, Laine Stump wrote: > On 12/13/2015 10:51 AM, Moshe Levi wrote: >> >> Hi, >> >> I have a setup with libvirt 1.3.0 and OpenStack trunk. >> >> Before launched the VM ip link command show the following VF mac/vlan configuration [1] >> >> When I launch a VM with <interface type='hostdev'> via openstack api (OpenStack direct >> port) >> >> I can see that the VF get the mac/vlan according to libvrit xml [2] and ip link >> command [3], but when I delete the VM the mac/vlan config are still shown as in [3] >> and not restored to [1] >> >> Shouldn’t libvirt restore the mac/vlan to [1]. >> >> The same problem exists when using <interface type='direct'> (OpenStack macvtap port) >> but just for the MAC configuration of the VF. >> > > What libvirt does is to restore the MAC address to whatever it was before we set it up > for use with a guest. Although there are some sriov net drivers that (for some > unfathomable reason) think it's cool to assign a random MAC address to each VF at boot > time, the "normal" thing is for the VFs to have a MAC address of all 0's to start with. > So libvirt should be saving 00:00:00:00:00:00 (it will be in the file > /var/run/libvirt/hostdevmgr/$ifname_vf$vfnum) then setting the MAC to use; when done, > libvirt will read the 00:00:00:00:00:00 and use netlink to set the MAC address, but this > is apparently failing. > > I checked on my Fedora 22 system with the igb driver, and found that if the MAC address > was originally set to something other than 0's, it was restored properly by libvirt, but > if it was set to all 0's originally, the attempt to set it back to 0 would fail. > > I then tried doing the same thing with the "ip" utility: > > # ip link set dev p4p2 vf 0 mac 00:00:00:00:00:00 > > and I get the following response: > > RTNETLINK answers: Invalid argument > > So it appears that either the kernel or the NIC driver is refusing to set the MAC > address to all 0's. I'm reasonably certain this is a regression in the kernel, Sigh. It appears that this has "always" been the case - I just checked on a 2.6.32-573 RHEL kernel, and a 3.10.x RHEL7.2 kernel, and 4.1 (Fedora 22) and both of them also refuse to set the MAC address to 00:00:00:00:00:00. I'm not sure if this limitation is in the NIC driver or some basic code in the kernel.

> although I can't say how long it's been there, as I don't normally pay attention to this > (and as I said, many SRIOV NIC drivers don't default their VFs to 0 MAC addresses) > > What distro and kernel are you using for your tests? > > >> [1] - 24: enp3s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master >> ovs-system state UP mode DEFAULT group default qlen 1000 >> >> link/ether e4:1d:2d:a5:f1:22 brd ff:ff:ff:ff:ff:ff >> >> vf 0 MAC 00:00:00:00:00:00, spoof checking off, link-state auto >> >> vf 1 MAC 00:00:00:00:00:00, spoof checking off, link-state auto >> >> vf 2 MAC 00:00:00:00:00:00, spoof checking off, link-state auto >> >> vf 3 MAC 00:00:00:00:00:00, spoof checking off, link-state auto >> >> [2] - <interface type='hostdev' managed='yes'> >> >> <mac address=' fa:16:3e:11:af:fe '/> >> >> <driver name='kvm'/> >> >> <source> >> >> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x7'/> >> >> </source> >> >> <vlan> >> >> <tag id='190'/> >> >> </vlan> >> >> <alias name='hostdev0'/> >> >> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> >> >> </interface> >> >> [3] 24: enp3s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system >> state UP mode DEFAULT group default qlen 1000 >> >> link/ether e4:1d:2d:a5:f1:22 brd ff:ff:ff:ff:ff:ff >> >> vf 0 MAC 00:00:00:00:00:00, spoof checking off, link-state auto >> >> vf 1 MAC 00:00:00:00:00:00, spoof checking off, link-state auto >> >> vf 2 MAC 00:00:00:00:00:00, spoof checking off, link-state auto >> >> vf 3 MAC fa:16:3e:11:af:fe, vlan 190, spoof checking off, link-state enable >> >> >> >> -- >> libvir-list mailing list >> libvir-list(a)redhat.com >> https://www.redhat.com/mailman/listinfo/libvir-list > F15 > > > -- > libvir-list mailing list > libvir-list(a)redhat.com > https://www.redhat.com/mailman/listinfo/libvir-list