[libvirt-users] automatically detaching PCI host devices from guest for SRIOV usage.
Dear List , We are using intel 82599EB based SRIOV capable 10GbE cards on the host machine.The VFs are being assigned to the guest in pass through mode and are providing the network connectivity as expected. However when we shutdown the guest the connectivity does not comes back via the VFs on the guest in next boot unless the VFs are detached from the *guest* *before* initiating a shutdown of the guests. ( the detach is explicitly done via virsh command /usr/local/libvirt/bin/virsh detach-device guest01 ~/nic.xml nic.xml contained <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x04' slot='0x10' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </hostdev> ) The question is anyone else facing similar problem and if so what solution was used? Secondly , is there anyway to automate the detaching of the VFs from the guest when the guest is put in the shutdown sequence ? (or for that matter destroyed). Regards, Rajesh Kumar Mallah Tel. 91-11-46710500 (Ext:303) Cell +919811255597 Website: http://www.tradeindia.com/ \|/ \|/ \|/ stop printing \|/ \|/ \|/ | | | start planting | | |
On 08/20/2012 08:34 PM, Rajesh Kumar Mallah wrote:
Dear List ,
We are using intel 82599EB based SRIOV capable 10GbE cards on the host machine.The VFs are being assigned to the guest in pass through mode and are providing the network connectivity as expected.
However when we shutdown the guest the connectivity does not comes back via the VFs on the guest in next boot unless the VFs are detached from the *guest* *before* initiating a shutdown of the guests. Dear Rajesh,
Please see the following comments.
( the detach is explicitly done via virsh command /usr/local/libvirt/bin/virsh detach-device guest01 ~/nic.xml nic.xml contained <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x04' slot='0x10' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </hostdev> )
The question is anyone else facing similar problem and if so what solution was used?
Secondly , is there anyway to automate the detaching of the VFs from the guest when the guest is put in the shutdown sequence ? (or for that matter destroyed).
When *|managed|* is *"yes"* for a PCI device, it is automatically detached from the host before being passed on to the guest, and automatically *reattached* to the host after the *guest exits* such as shutdown a guest. Regards, Alex
Regards, Rajesh Kumar Mallah Tel. 91-11-46710500 (Ext:303) Cell +919811255597 Website: http://www.tradeindia.com/ \|/ \|/ \|/ stop printing \|/ \|/ \|/ | | | start planting | | |
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
Dear Alex / List , Thanks for the response. The problem is regarding detaching from the guest not host. In our case since ixgbevf driver is NOT present in the host hence the virtual interfaces never really attach themselves to the host. Regards, Rajesh Kumar Mallah Tel. 91-11-46710500 (Ext:303) Cell +919811255597 Website: http://www.tradeindia.com/ \|/ \|/ \|/ stop printing \|/ \|/ \|/ | | | start planting | | | ----- "Alex Jia" <ajia@redhat.com> wrote: | From: "Alex Jia" <ajia@redhat.com> | To: "Rajesh Kumar Mallah" <mallah@tradeindia.com> | Cc: libvirt-users@redhat.com | Sent: Tuesday, August 21, 2012 8:31:21 AM | Subject: Re: [libvirt-users] automatically detaching PCI host devices from guest for SRIOV usage. | | On 08/20/2012 08:34 PM, Rajesh Kumar Mallah wrote: Dear List , We are using intel 82599EB based SRIOV capable 10GbE cards on the host machine.The VFs are being assigned to the guest in pass through mode and are providing the network connectivity as expected. However when we shutdown the guest the connectivity does not comes back via the VFs on the guest in next boot unless the VFs are detached from the *guest* *before* initiating a shutdown of the guests. Dear Rajesh, | | Please see the following comments. | | ( the detach is explicitly done via virsh command /usr/local/libvirt/bin/virsh detach-device guest01 ~/nic.xml nic.xml contained <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x04' slot='0x10' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </hostdev> ) The question is anyone else facing similar problem and if so what solution was used? Secondly , is there anyway to automate the detaching of the VFs from the guest when the guest is put in the shutdown sequence ? (or for that matter destroyed). | When managed is "yes" for a PCI device, it is automatically detached from the host before being passed on to the guest, and automatically reattached to the host after the guest exits such as shutdown a guest. | | Regards, | Alex | | Regards, Rajesh Kumar Mallah Tel. 91-11-46710500 (Ext:303) Cell +919811255597 Website: http://www.tradeindia.com/ \|/ \|/ \|/ stop printing \|/ \|/ \|/ | | | start planting | | | _______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users |
On 08/21/2012 11:22 AM, Rajesh Kumar Mallah wrote:
Dear Alex / List ,
Thanks for the response.
The problem is regarding detaching from the guest not host.
I see, if you attach a PCI device into guest with 'managed' mode(managed='yes') then the device is automatically detached from guest and returned to host after guest exiting, for example. # lspci | grep Eth 00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02) # readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f /sys/bus/pci/drivers/e1000e # virsh start foo Domain foo started # ping -c 3 xx.xx.xx.xx PING xx.xx.xx.xx (xx.xx.xx.xx) 56(84) bytes of data. 64 bytes from nx-in-f160.1e100.net (xx.xx.xx.xx): icmp_seq=1 ttl=49 time=87.8 ms 64 bytes from nx-in-f160.1e100.net (xx.xx.xx.xx): icmp_seq=2 ttl=49 time=85.2 ms 64 bytes from nx-in-f160.1e100.net (xx.xx.xx.xx): icmp_seq=3 ttl=49 time=91.8 ms --- xx.xx.xx.xx ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2093ms rtt min/avg/max/mdev = 85.240/88.327/91.881/2.731 ms # cat hostdev.xml <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x00' slot='0x19' function='0x0'/> </source> </hostdev> # virsh attach-device foo hostdev.xml Device attached successfully # readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f /sys/bus/pci/drivers/pci-stub # ping -c 3 xx.xx.xx.xx connect: Network is unreachable # virsh destroy foo Domain foo destroyed # readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f /sys/bus/pci/drivers/e1000e # ping -c 3 xx.xx.xx.xx PING xx.xx.xx.xx (xx.xx.xx.xx) 56(84) bytes of data. 64 bytes from xx.xx.xx.xx: icmp_seq=1 ttl=49 time=79.7 ms 64 bytes from xx.xx.xx.xx: icmp_seq=2 ttl=49 time=83.2 ms 64 bytes from xx.xx.xx.xx: icmp_seq=3 ttl=49 time=84.5 ms --- xx.xx.xx.xx ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2087ms rtt min/avg/max/mdev = 79.714/82.480/84.513/2.040 ms Notes, need to wait for local network recovery again then check network connectivity. If your network still is disconnect, it may be a bug IMHO, please show your libvirt and qemu-kvm version, or my example is not what you want, please let me know. Thanks, Alex
In our case since ixgbevf driver is NOT present in the host hence the virtual interfaces never really attach themselves to the host.
Regards, Rajesh Kumar Mallah Tel. 91-11-46710500 (Ext:303) Cell +919811255597 Website: http://www.tradeindia.com/ \|/ \|/ \|/ stop printing \|/ \|/ \|/ | | | start planting | | |
----- "Alex Jia" <ajia@redhat.com> wrote: | From: "Alex Jia" <ajia@redhat.com> | To: "Rajesh Kumar Mallah" <mallah@tradeindia.com> | Cc: libvirt-users@redhat.com | Sent: Tuesday, August 21, 2012 8:31:21 AM | Subject: Re: [libvirt-users] automatically detaching PCI host devices from guest for SRIOV usage. | | On 08/20/2012 08:34 PM, Rajesh Kumar Mallah wrote:
Dear List ,
We are using intel 82599EB based SRIOV capable 10GbE cards on the host machine.The VFs are being assigned to the guest in pass through mode and are providing the network connectivity as expected.
However when we shutdown the guest the connectivity does not comes back via the VFs on the guest in next boot unless the VFs are detached from the *guest* *before* initiating a shutdown of the guests.
Dear Rajesh, | | Please see the following comments. | |
( the detach is explicitly done via virsh command /usr/local/libvirt/bin/virsh detach-device guest01 ~/nic.xml nic.xml contained <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x04' slot='0x10' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </hostdev> )
The question is anyone else facing similar problem and if so what solution was used?
Secondly , is there anyway to automate the detaching of the VFs from the guest when the guest is put in the shutdown sequence ? (or for that matter destroyed).
| When *|managed|* is *"yes"* for a PCI device, it is automatically detached from the host before being passed on to the guest, and automatically *reattached* to the host after the *guest exits* such as shutdown a guest. | | Regards, | Alex | |
Regards, Rajesh Kumar Mallah Tel. 91-11-46710500 (Ext:303) Cell +919811255597 Website:http://www.tradeindia.com/ \|/ \|/ \|/ stop printing \|/ \|/ \|/ | | | start planting | | |
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
|
participants (2)
-
Alex Jia -
Rajesh Kumar Mallah