[libvirt-users] intel quad gigabit nic and pci passthrough
by Ivan Kabaivanov
Hi all
I have a very weird case of pci passthrough.
I have a machine with 7 network interfaces, all of them intel. Four of them are on one quad giga ethernet device. If I manually unbind the devices and allow qemu to use them, with intel IOMMU working, everything works like a charm. Here's how I do it manually:
root@kybrat (x86_64) ~]$ lspci -nn | grep net
00:19.0 Ethernet controller [0200]: Intel Corporation 82579V Gigabit Network Connection [8086:1503] (rev 06)
02:00.0 Ethernet controller [0200]: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) [8086:107d] (rev 06)
05:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
05:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
06:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
06:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
0c:00.0 Ethernet controller [0200]: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) [8086:107d] (rev 06)
echo "8086 10a4" > /sys/bus/pci/drivers/pci-stub/new_id
echo "8086 107d" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:05:00.0" > /sys/bus/pci/devices/0000\:05\:00.0/driver/unbind
echo "0000:05:00.0" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:05:00.1" > /sys/bus/pci/devices/0000\:05\:00.1/driver/unbind
echo "0000:05:00.1" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:06:00.0" > /sys/bus/pci/devices/0000\:06\:00.0/driver/unbind
echo "0000:06:00.0" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:06:00.1" > /sys/bus/pci/devices/0000\:06\:00.1/driver/unbind
echo "0000:06:00.1" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:0c:00.0" > /sys/bus/pci/devices/0000\:0c\:00.0/driver/unbind
echo "0000:0c:00.0" > /sys/bus/pci/drivers/pci-stub/bind
qemu-system-i386 -enable-kvm \
-m 2048 \
-cpu host \
-machine pc-q35-1.6 \
-drive file=/media/virtual/krym.qcow2,if=virtio \
-net none \
-device pci-assign,host=06:00.1,rombar=0 \
-device pci-assign,host=05:00.0,rombar=0 \
-device pci-assign,host=06:00.0,rombar=0 \
-device pci-assign,host=05:00.1,rombar=0 \
-device pci-assign,host=0c:00.0,rombar=0 \
-cdrom /media/virtual/asg-9.107-33.1.iso \
-boot c \
-vnc :0 \
-runas virtual \
-monitor unix:/media/virtual/control/krym/monitor,server,nowait \
-rtc base=localtime,clock=host,driftfix=none
This starts qemu and pci passthrough works nicely.
Now, if I try to do the same thing through libvirt, virt-install doesn't like the quad ethernet device pci passthrough and I get the error below:
virt-install --name=krym --ram=2048 --vcpus=1 --cpu=host --import --os-type=linux --disk path=/media/virtual/krym.qcow2,format=qcow2,bus=virtio --nonetworks --graphics vnc,port=5900 --virt-type=kvm --machine=q35 --accelerate --host-device=pci_0000_06_00_1 --host-device=pci_0000_05_00_0 --host-device=pci_0000_06_00_0 --host-device=pci_0000_05_00_1 --host-device=pci_0000_0c_00_0
Starting install...
ERROR Requested operation is not valid: PCI device 0000:06:00.1 is not assignable
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
virsh --connect qemu:///system start krym
otherwise, please restart your installation.
If I try to use just one NIC (any NIC, as long as it's not the quad NIC), things do work:
virt-install --name=krym --ram=2048 --vcpus=1 --cpu=host --import --os-type=linux --disk path=/media/virtual/krym.qcow2,format=qcow2,bus=virtio --nonetworks --graphics vnc,port=5900 --virt-type=kvm --machine=q35 --accelerate --host-device=pci_0000_0c_00_0
Starting install...
Creating domain... | 0 B 00:00:01
Cannot open display:
Run 'virt-viewer --help' to see a full list of available command line options
Domain creation completed. You can restart your domain by running:
virsh --connect qemu:///system start krym
So I guess libvirt has some problem with the intel quad NIC.
I will appreciate any help.
Thanks,
IvanK.
10 years, 9 months
- 2
- 2
[libvirt-users] Creating a static Virtio-Serial 'Port'
by Matthew Harrold
Hi all,
I'm having some problems getting Virtio-Serial running as I'd like.
I have 10x USB <-> Serial adapters plugged into my VM Host / Hypervisor,
acting as a console server. These are currently operated by the VM Host
itself, but I would like to move them to a guest. I have used ISA-Serial
but it's limited to 4 'ports' per guest.
I've setup Virtio-Serial for the guest and it works, I can pass info via
serial from the Host to Guest. I'd like to use socat to link the physical
USB serial TTY's and the guest serial 'ports' in /dev/pts.
This is works fine and the guest can communicate with the Serial <-> USB
adapter, however the Guest serial 'ports' are in /dev/pts and they are
assigned a number dynamically when the VM is started, so I can't automate
socat to run at boot.
Is it possible to assign static PTY's in the host, to the Guest? Or, if
this needs to be scripted, is there a command I can run to find out which
PTY it's on without parsing the XML from a dump?
Thanks.
10 years, 9 months
- 1
- 0
[libvirt-users] Libvirt Error Creating VMs for Nimbus Cloud
by Joshua McKee
Hello,
I am having an issue with libvirt in my Nimbus cloud setup. When I use the
Nimbus VM creation test scripts on a VMM node (using libvirt), the VM is
created without a problem. However, when I attempt to create a new VM using
Nimbus (which then tries to create the VM on the VMM node), I encounter the
following error:
---------------------------------------------------------------------------------------------------------------------------------------------
$ ./nimbus-cloud-client-022/bin/cloud-client.sh --run --name testimage
--hours 1
Launching workspace.
Workspace Factory Service:
https://yellow:8443/wsrf/services/WorkspaceFactoryService
Creating workspace "vm-035"... done.
IP address: 172.29.99.20
Hostname: pub02
Start time: Fri Jan 31 13:48:57 MST 2014
Shutdown time: Fri Jan 31 14:48:57 MST 2014
Termination time: Fri Jan 31 14:50:57 MST 2014
Waiting for updates.
Problem with vm-035: Unexpected issue
STDERR: 2014-01-31 13:48:44,412 - WARNING - no qemu_img configuration,
copy-on-write support is disabled
libvir: QEMU error : Domain not found: no domain with matching name
'wrksp-33'
libvir: error : An error occurred, but the cause is unknown
2014-01-31 13:48:45,587 - ERROR - Problem creating the VM: An error
occurred, but the cause is unknown
2014-01-31 13:48:45,587 - ERROR - An error occurred, but the cause is
unknown
Traceback (most recent call last):
File
"/opt/nimbus/src/python/workspacecontrol/defaults/lvrt/lvrt_common.py",
line 123, in create
newvm = self._vmm().createXML(xml, 0)
File "/usr/lib/python2.7/dist-packages/libvirt.py", line 2189, in
createXML
if ret is None:raise libvirtError('virDomainCreateXML() failed',
conn=self)
libvirtError: An error occurred, but the cause is unknown
2014-01-31 13:48:45,589 - ERROR - Problem creating the VM: An error
occurred, but the cause is unknown
Traceback (most recent call last):
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 155, in _common_withnetsecurity
_common_withtmplease(nic_set, kernel, local_file_set, c, netsecurity,
platform, justprint)
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 174, in _common_withtmplease
platform.create(local_file_set, nic_set, kernel)
File
"/opt/nimbus/src/python/workspacecontrol/defaults/lvrt/lvrt_common.py",
line 128, in create
raise UnexpectedError(shorterr)
UnexpectedError: Problem creating the VM: An error occurred, but the cause
is unknown
2014-01-31 13:48:45,589 - ERROR - Creation problem: going to back out tmp
space lease
2014-01-31 13:48:45,589 - ERROR - Backed out tmp space lease
2014-01-31 13:48:45,589 - ERROR - Problem creating the VM: An error
occurred, but the cause is unknown
Traceback (most recent call last):
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 137, in _common_withnetbootstrap
_common_withnetsecurity(vm_name, nic_set, kernel, local_file_set, p, c,
netsecurity, platform, justprint)
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 164, in _common_withnetsecurity
raise e
UnexpectedError: Problem creating the VM: An error occurred, but the cause
is unknown
2014-01-31 13:48:45,589 - ERROR - Creation problem: going to back out net
security
2014-01-31 13:48:45,606 - ERROR - Backed out net security
2014-01-31 13:48:45,606 - ERROR - Problem creating the VM: An error
occurred, but the cause is unknown
Traceback (most recent call last):
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 119, in _common_withnics
_common_withnetbootstrap(vm_name, nic_set, kernel, local_file_set, p,
c, netsecurity, platform, justprint)
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 146, in _common_withnetbootstrap
raise e
UnexpectedError: Problem creating the VM: An error occurred, but the cause
is unknown
2014-01-31 13:48:45,607 - ERROR - Creation problem: going to back out net
bootstrap
2014-01-31 13:48:45,607 - ERROR - Backed out net bootstrap
2014-01-31 13:48:45,607 - ERROR - Problem creating the VM: An error
occurred, but the cause is unknown
Traceback (most recent call last):
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 96, in _common
_common_withnics(vm_name, nic_set, kernel, local_file_set, p, c,
localnet, netbootstrap, netsecurity, platform, justprint)
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 128, in _common_withnics
raise e
UnexpectedError: Problem creating the VM: An error occurred, but the cause
is unknown
2014-01-31 13:48:45,607 - ERROR - Creation problem: going to back out net
leases
2014-01-31 13:48:45,607 - ERROR - Backed out net leases
2014-01-31 13:48:45,608 - ERROR - Problem creating the VM: An error
occurred, but the cause is unknown
Traceback (most recent call last):
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core.py", line 79,
in core
_core(vm_name, action, p, c)
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core.py", line 168,
in _core
wc_core_creation.create(vm_name, p, c, async, editing, images, kernels,
localnet, netbootstrap, netlease, netsecurity, persistence, platform)
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 16, in create
_create_local(vm_name, p, c, async, editing, images, kernels, localnet,
netbootstrap, netlease, netsecurity, persistence, platform)
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 31, in _create_local
_common(local_file_set, vm_name, p, c, editing, kernels, localnet,
netbootstrap, netlease, netsecurity, persistence, platform,
justprint=justprint)
File "/opt/nimbus/src/python/workspacecontrol/main/wc_core_creation.py",
line 107, in _common
raise e
UnexpectedError: Problem creating the VM: An error occurred, but the cause
is unknown
Problem executing: Problem creating the VM: An error occurred, but the
cause is unknown
Exiting with error code: 4
Workspace "vm-035" did NOT reach target state "Running"
Problem running 'vm-035'.
---------------------------------------------------------------------------------------------------------------------------------------------
I checked /var/log/libvirtd.log, and found the following:
---------------------------------------------------------------------------------------------------------------------------------------------
2014-01-31 20:48:39.311+0000: 2888: info : libvirt version: 0.9.8
2014-01-31 20:48:39.311+0000: 2888: error : virNetSocketReadWire:1006 : End
of file while reading data: Input/output error
---------------------------------------------------------------------------------------------------------------------------------------------
I don't know if this is relevant, but I also noticed this warning when
creating the VM through the test scripts mentioned above:
---------------------------------------------------------------------------------------------------------------------------------------------
2014-01-31 05:41:44.018+0000: 1351: warning : qemuDomainObjTaint:1134 :
Domain id=1 name='control-test' uuid=95f2d398-6b98-5359-7597-b0f596aae996
is tainted: high-privileges
---------------------------------------------------------------------------------------------------------------------------------------------
I assumed this was a permissions issue, but I double checked and everything
seems in order. I did not have this issue on my previous Nimbus cloud setup.
Thanks,
Josh
10 years, 9 months
- 1
- 1