7:29 a.m.
hi,
i have a small trouble with pci-passthrough.
i have a working configuration passing a tv card into the guest, all is fine and dandy.
as soon as i add my second identical card into the host i cant start the guest anymore,
whether i add the second card to the guest or not. error message is identical in both
cases.
message of virt-manager|virsh|libvirtd.log:
libvirtError: internal error: Unable to reset PCI device
0000:03:00.0: internal error: Active 0000:03:01.0 devices on bus with 0000:03:00.0, not
doing bus reset
the host is a plain kvm server, no services or apps whatsoever accessing the two devices.
the other guests (firewall and fileserver) do not use the cards either.
if i (soft) remove the 2nd card via
echo -n 1 > /sys/bus/pci/devices/0000\:03\:01.0/remove
i can
start the guest with the 1st card assigned.
thx for anyone looking into this.
wbr,tja..
PS:
host:
root@father:~# lspci
00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core
Processor Integrated Graphics Controller (rev 06)
00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI
(rev 05)
00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family
MEI Controller #1 (rev 04)
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-V (rev 05)
00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2
(rev 05)
00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express
Root Port #1 (rev d5)
00:1c.3 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d5)
00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1
(rev 05)
00:1f.0 ISA bridge: Intel Corporation H87 Express LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port
SATA Controller 1 [AHCI mode] (rev 05)
00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller
(rev 05)
02:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 03)
03:00.0 Multimedia controller: Philips Semiconductors SAA7146 (rev 01)
03:01.0 Multimedia controller: Philips Semiconductors SAA7146 (rev 01)
root@father:~# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=13.10
DISTRIB_CODENAME=saucy
DISTRIB_DESCRIPTION="Ubuntu 13.10"
root@father:~# uname -a
Linux father 3.11.0-18-generic #32-Ubuntu SMP Tue Feb 18 21:11:14 UTC 2014 x86_64 x86_64
x86_64 GNU/Linux
root@father:~# libvirtd --version
libvirtd (libvirt) 1.1.1
guest:
root@father:~# cat /etc/libvirt/qemu/tvBackend.xml
<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
virsh edit tvBackend
or other application using the libvirt API.
-->
<domain type='kvm'>
<name>tvBackend</name>
<uuid>f647bb10-7f9a-f94c-33b9-3d99e8e753e0</uuid>
<memory unit='KiB'>1048576</memory>
<currentMemory unit='KiB'>1048576</currentMemory>
<vcpu placement='static'>2</vcpu>
<os>
<type arch='x86_64' machine='pc-i440fx-1.5'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/kvm-spice</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/kvm/tvBackend.img'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x04' function='0x0'/>
</disk>
<disk type='block' device='cdrom'>
<driver name='qemu' type='raw'/>
<target dev='hdc' bus='ide'/>
<readonly/>
<address type='drive' controller='0' bus='1'
target='0' unit='0'/>
</disk>
<controller type='usb' index='0'>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<controller type='ide' index='0'>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x1'/>
</controller>
<interface type='bridge'>
<mac address='52:54:00:10:26:ec'/>
<source bridge='brlan'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
</interface>
<serial type='pty'>
<target port='0'/>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<input type='mouse' bus='ps2'/>
<graphics type='vnc' port='-1' autoport='yes'/>
<video>
<model type='vga' vram='9216' heads='1'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x02' function='0x0'/>
</video>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x03' slot='0x00'
function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x06' function='0x0'/>
</hostdev>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x05' function='0x0'/>
</memballoon>
</devices>
</domain>
--
thomas jagoditsch - tjaSoft
softWareEntwicklung - netzWerkManagement
10:13 a.m.
On 03/12/2014 06:29 AM, Thomas Jagoditsch wrote:
hi,
i have a small trouble with pci-passthrough.
i have a working configuration passing a tv card into the guest, all is fine and dandy.
as soon as i add my second identical card into the host i cant start the guest anymore,
whether i add the second card to the guest or not. error message is identical in both
cases.
message of virt-manager|virsh|libvirtd.log:
> libvirtError: internal error: Unable to reset PCI device 0000:03:00.0: internal
error: Active 0000:03:01.0 devices on bus with 0000:03:00.0, not doing bus reset
the host is a plain kvm server, no services or apps whatsoever accessing the two
devices.
the other guests (firewall and fileserver) do not use the cards either.
The problem is that for kvm device assignment to work properly, the
device needs to be reset by libvirt after detaching it from the host
driver and before passing it to kvm, and these devices you are trying to
reset support neither "function level reset" nor "power management
reset", so libvirt must fallback to resetting the entire bus where the
device is plugged in. But of course it can't do that if the bus contains
other devices that are in use by the host or by other guests.
If your host OS is new enough to support vfio device assignment, I would
suggest using that instead (if you update to the latest upstrem libvirt
it will automatically use VFIO if it is available, otherwise you can add
"<driver name='vfio'/>" to the device definition to assure that it
will
either use vfio or fail. You will also probably need to run "modprobe
vfio" before starting the guest). The reason I suggest this is that VFIO
will automatically handle resetting the assigned devices whenever
necessary, compared to the old KVM device assignment, where libvirt must
always reset the device because it has no better information about
whether or not it is really necessary.
Another thing you can try is plugging one of the cards into a different
slot - if you can find a slot that is on a different bus, then libvirt
will be able to reset the bus containing the card that will be assigned
(since there won't be any other active devices on that same bus).
if i (soft) remove the 2nd card via
> echo -n 1 > /sys/bus/pci/devices/0000\:03\:01.0/remove
i can start the guest with the 1st card assigned.
thx for anyone looking into this.
libvirt attempts to reset the
wbr,tja..
PS:
host:
> root@father:~# lspci
> 00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev
06)
> 00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core
Processor Integrated Graphics Controller (rev 06)
> 00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB
xHCI (rev 05)
> 00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset
Family MEI Controller #1 (rev 04)
> 00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-V (rev 05)
> 00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB
EHCI #2 (rev 05)
> 00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express
Root Port #1 (rev d5)
> 00:1c.3 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d5)
> 00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB
EHCI #1 (rev 05)
> 00:1f.0 ISA bridge: Intel Corporation H87 Express LPC Controller (rev 05)
> 00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port
SATA Controller 1 [AHCI mode] (rev 05)
> 00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller
(rev 05)
> 02:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 03)
> 03:00.0 Multimedia controller: Philips Semiconductors SAA7146 (rev 01)
> 03:01.0 Multimedia controller: Philips Semiconductors SAA7146 (rev 01)
> root@father:~# cat /etc/lsb-release
> DISTRIB_ID=Ubuntu
> DISTRIB_RELEASE=13.10
> DISTRIB_CODENAME=saucy
> DISTRIB_DESCRIPTION="Ubuntu 13.10"
> root@father:~# uname -a
> Linux father 3.11.0-18-generic #32-Ubuntu SMP Tue Feb 18 21:11:14 UTC 2014 x86_64
x86_64 x86_64 GNU/Linux
> root@father:~# libvirtd --version
> libvirtd (libvirt) 1.1.1
guest:
> root@father:~# cat /etc/libvirt/qemu/tvBackend.xml
> <!--
> WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
> OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
> virsh edit tvBackend
> or other application using the libvirt API.
> -->
>
> <domain type='kvm'>
> <name>tvBackend</name>
> <uuid>f647bb10-7f9a-f94c-33b9-3d99e8e753e0</uuid>
> <memory unit='KiB'>1048576</memory>
> <currentMemory unit='KiB'>1048576</currentMemory>
> <vcpu placement='static'>2</vcpu>
> <os>
> <type arch='x86_64'
machine='pc-i440fx-1.5'>hvm</type>
> <boot dev='hd'/>
> </os>
> <features>
> <acpi/>
> <apic/>
> <pae/>
> </features>
> <clock offset='utc'/>
> <on_poweroff>destroy</on_poweroff>
> <on_reboot>restart</on_reboot>
> <on_crash>restart</on_crash>
> <devices>
> <emulator>/usr/bin/kvm-spice</emulator>
> <disk type='file' device='disk'>
> <driver name='qemu' type='raw'/>
> <source file='/kvm/tvBackend.img'/>
> <target dev='vda' bus='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00'
slot='0x04' function='0x0'/>
> </disk>
> <disk type='block' device='cdrom'>
> <driver name='qemu' type='raw'/>
> <target dev='hdc' bus='ide'/>
> <readonly/>
> <address type='drive' controller='0' bus='1'
target='0' unit='0'/>
> </disk>
> <controller type='usb' index='0'>
> <address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x2'/>
> </controller>
> <controller type='pci' index='0'
model='pci-root'/>
> <controller type='ide' index='0'>
> <address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x1'/>
> </controller>
> <interface type='bridge'>
> <mac address='52:54:00:10:26:ec'/>
> <source bridge='brlan'/>
> <model type='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
> </interface>
> <serial type='pty'>
> <target port='0'/>
> </serial>
> <console type='pty'>
> <target type='serial' port='0'/>
> </console>
> <input type='mouse' bus='ps2'/>
> <graphics type='vnc' port='-1' autoport='yes'/>
> <video>
> <model type='vga' vram='9216' heads='1'/>
> <address type='pci' domain='0x0000' bus='0x00'
slot='0x02' function='0x0'/>
> </video>
> <hostdev mode='subsystem' type='pci'
managed='yes'>
> <source>
> <address domain='0x0000' bus='0x03' slot='0x00'
function='0x0'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x00'
slot='0x06' function='0x0'/>
> </hostdev>
> <memballoon model='virtio'>
> <address type='pci' domain='0x0000' bus='0x00'
slot='0x05' function='0x0'/>
> </memballoon>
> </devices>
> </domain>
3908
days inactive
3908
days old
1 comments
2 participants
participants (2)
-
Laine Stump -
Thomas Jagoditsch