6:58 p.m.
On 01/25/2014 06:54 PM, Ivan Kabaivanov wrote:
Hi all
I have a very weird case of pci passthrough.
I have a machine with 7 network interfaces, all of them intel. Four of them are on one
quad giga ethernet device. If I manually unbind the devices and allow qemu to use them,
with intel IOMMU working, everything works like a charm. Here's how I do it
manually:
root@kybrat (x86_64) ~]$ lspci -nn | grep net
00:19.0 Ethernet controller [0200]: Intel Corporation 82579V Gigabit Network Connection
[8086:1503] (rev 06)
02:00.0 Ethernet controller [0200]: Intel Corporation 82572EI Gigabit Ethernet Controller
(Copper) [8086:107d] (rev 06)
05:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller
[8086:10a4] (rev 06)
05:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller
[8086:10a4] (rev 06)
06:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller
[8086:10a4] (rev 06)
06:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller
[8086:10a4] (rev 06)
0c:00.0 Ethernet controller [0200]: Intel Corporation 82572EI Gigabit Ethernet Controller
(Copper) [8086:107d] (rev 06)
echo "8086 10a4" > /sys/bus/pci/drivers/pci-stub/new_id
echo "8086 107d" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:05:00.0" > /sys/bus/pci/devices/0000\:05\:00.0/driver/unbind
echo "0000:05:00.0" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:05:00.1" > /sys/bus/pci/devices/0000\:05\:00.1/driver/unbind
echo "0000:05:00.1" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:06:00.0" > /sys/bus/pci/devices/0000\:06\:00.0/driver/unbind
echo "0000:06:00.0" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:06:00.1" > /sys/bus/pci/devices/0000\:06\:00.1/driver/unbind
echo "0000:06:00.1" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:0c:00.0" > /sys/bus/pci/devices/0000\:0c\:00.0/driver/unbind
echo "0000:0c:00.0" > /sys/bus/pci/drivers/pci-stub/bind
qemu-system-i386 -enable-kvm \
-m 2048 \
-cpu host \
-machine pc-q35-1.6 \
-drive file=/media/virtual/krym.qcow2,if=virtio \
-net none \
-device pci-assign,host=06:00.1,rombar=0 \
-device pci-assign,host=05:00.0,rombar=0 \
-device pci-assign,host=06:00.0,rombar=0 \
-device pci-assign,host=05:00.1,rombar=0 \
-device pci-assign,host=0c:00.0,rombar=0 \
-cdrom /media/virtual/asg-9.107-33.1.iso \
-boot c \
-vnc :0 \
-runas virtual \
-monitor unix:/media/virtual/control/krym/monitor,server,nowait \
-rtc base=localtime,clock=host,driftfix=none
This starts qemu and pci passthrough works nicely.
Now, if I try to do the same thing through libvirt, virt-install doesn't like the
quad ethernet device pci passthrough and I get the error below:
virt-install --name=krym --ram=2048 --vcpus=1 --cpu=host --import --os-type=linux --disk
path=/media/virtual/krym.qcow2,format=qcow2,bus=virtio --nonetworks --graphics
vnc,port=5900 --virt-type=kvm --machine=q35 --accelerate --host-device=pci_0000_06_00_1
--host-device=pci_0000_05_00_0 --host-device=pci_0000_06_00_0
--host-device=pci_0000_05_00_1 --host-device=pci_0000_0c_00_0
Starting install...
ERROR Requested operation is not valid: PCI device 0000:06:00.1 is not assignable
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
virsh --connect qemu:///system start krym
otherwise, please restart your installation.
More information is needed to make a guess at the source of the problem:
1) What distro/version of Linux are you running.
2) What version of libvirt?
3) After the failed virt-install command, if the new domain is still
defined, can you provide the output of:
virsh dumpxml $domain
(if virt-install is automatically undefining the failed domain, then you
can get the xml for the defined domain by appending "--print-xml" to the
virt-install commandline)
4) Also, send the last several lines of
/var/log/libvirt/qemu/$domain.log, in particular the qemu commandline
that was executed to start the domain, and any subsequent error messages.
(Just one possibility - since you are successfully using the q35
machinetype, it's highly likely that your OS is new enough that it
supports vfio for device passthrough; vfio is more restrictive about
what devices can be assigned to a guest, sometimes requiring seemingly
unrelated devices to be assigned to the same guest, and the most recent
libvirts will prefer vfio over legacy kvm device assignment when
available. So it's possible that your manual running of qemu-kvm works
because you are forcing use of legacy kvm assignment, and when libvirt
attempts to use vfio it is failing. This will be made apparent by the
information I requested above.
If I try to use just one NIC (any NIC, as long as it's not the quad NIC), things do
work:
virt-install --name=krym --ram=2048 --vcpus=1 --cpu=host --import --os-type=linux --disk
path=/media/virtual/krym.qcow2,format=qcow2,bus=virtio --nonetworks --graphics
vnc,port=5900 --virt-type=kvm --machine=q35 --accelerate --host-device=pci_0000_0c_00_0
Starting install...
Creating domain...
| 0 B 00:00:01
Cannot open display:
Run 'virt-viewer --help' to see a full list of available command line options
Domain creation completed. You can restart your domain by running:
virsh --connect qemu:///system start krym
So I guess libvirt has some problem with the intel quad NIC.
I will appreciate any help.
Thanks,
IvanK.
_______________________________________________
libvirt-users mailing list
libvirt-users(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvirt-users