Hi all
I have a very weird case of pci passthrough.
I have a machine with 7 network interfaces, all of them intel. Four of them are on one quad giga ethernet device. If I manually unbind the devices and allow qemu to use them, with intel IOMMU working, everything works like a charm. Here's how I do it manually:
root@kybrat (x86_64) ~]$ lspci -nn | grep net
00:19.0 Ethernet controller [0200]: Intel Corporation 82579V Gigabit Network Connection [8086:1503] (rev 06)
02:00.0 Ethernet controller [0200]: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) [8086:107d] (rev 06)
05:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
05:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
06:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
06:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
0c:00.0 Ethernet controller [0200]: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) [8086:107d] (rev 06)
echo "8086 10a4" > /sys/bus/pci/drivers/pci-stub/new_id
echo "8086 107d" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:05:00.0" > /sys/bus/pci/devices/0000\:05\:00.0/driver/unbind
echo "0000:05:00.0" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:05:00.1" > /sys/bus/pci/devices/0000\:05\:00.1/driver/unbind
echo "0000:05:00.1" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:06:00.0" > /sys/bus/pci/devices/0000\:06\:00.0/driver/unbind
echo "0000:06:00.0" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:06:00.1" > /sys/bus/pci/devices/0000\:06\:00.1/driver/unbind
echo "0000:06:00.1" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:0c:00.0" > /sys/bus/pci/devices/0000\:0c\:00.0/driver/unbind
echo "0000:0c:00.0" > /sys/bus/pci/drivers/pci-stub/bind
qemu-system-i386 -enable-kvm \
-m 2048 \
-cpu host \
-machine pc-q35-1.6 \
-drive file=/media/virtual/krym.qcow2,if=virtio \
-net none \
-device pci-assign,host=06:00.1,rombar=0 \
-device pci-assign,host=05:00.0,rombar=0 \
-device pci-assign,host=06:00.0,rombar=0 \
-device pci-assign,host=05:00.1,rombar=0 \
-device pci-assign,host=0c:00.0,rombar=0 \
-cdrom /media/virtual/asg-9.107-33.1.iso \
-boot c \
-vnc :0 \
-runas virtual \
-monitor unix:/media/virtual/control/krym/monitor,server,nowait \
-rtc base=localtime,clock=host,driftfix=none
This starts qemu and pci passthrough works nicely.
Now, if I try to do the same thing through libvirt, virt-install doesn't like the quad ethernet device pci passthrough and I get the error below:
virt-install --name=krym --ram=2048 --vcpus=1 --cpu=host --import --os-type=linux --disk path=/media/virtual/krym.qcow2,format=qcow2,bus=virtio --nonetworks --graphics vnc,port=5900 --virt-type=kvm --machine=q35 --accelerate --host-device=pci_0000_06_00_1 --host-device=pci_0000_05_00_0 --host-device=pci_0000_06_00_0 --host-device=pci_0000_05_00_1 --host-device=pci_0000_0c_00_0
Starting install...
ERROR Requested operation is not valid: PCI device 0000:06:00.1 is not assignable
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
virsh --connect qemu:///system start krym
otherwise, please restart your installation.
If I try to use just one NIC (any NIC, as long as it's not the quad NIC), things do work:
virt-install --name=krym --ram=2048 --vcpus=1 --cpu=host --import --os-type=linux --disk path=/media/virtual/krym.qcow2,format=qcow2,bus=virtio --nonetworks --graphics vnc,port=5900 --virt-type=kvm --machine=q35 --accelerate --host-device=pci_0000_0c_00_0
Starting install...
Creating domain... | 0 B 00:00:01
Cannot open display:
Run 'virt-viewer --help' to see a full list of available command line options
Domain creation completed. You can restart your domain by running:
virsh --connect qemu:///system start krym
So I guess libvirt has some problem with the intel quad NIC.
I will appreciate any help.
Thanks,
IvanK.
