4:59 p.m.
On Wed, Sep 25, 2013 at 11:48:28AM +0300, Marcel Apfelbaum wrote:
On Wed, 2013-09-25 at 10:01 +0300, Michael S. Tsirkin wrote:
> On Tue, Sep 24, 2013 at 06:01:02AM -0400, Laine Stump wrote:
> > When I added support for the Q35-based machinetypes to libvirt, I
> > specifically prohibited attaching any PCI devices (with the exception of
> > graphics controllers) to the PCIe root complex,
>
> That's wrong I think. Anything attached to RC is an integrated
> endpoint, and these can be PCI devices.
I couldn't find on PCIe spec any mention that "Root Complex Integrated
EndPoint"
must be PCIe. But, from spec 1.3.2.3:
- A Root Complex Integrated Endpoint must not require I/O resources claimed through
BAR(s).
- A Root Complex Integrated Endpoint must not generate I/O Requests.
- A Root Complex Integrated Endpoint is required to support MSI or MSI-X or both if an
interrupt resource is requested.
Heh PCI-SIG keeps fighting against legacy interrupts and IO.
But lots of hardware happily ignores these rules.
And the reason is simple: software does not enforce them.
Here's integrated stuff on my laptop:
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core
Processor Family Integrated Graphics Controller (rev 09) (prog-if 00
[VGA controller])
Subsystem: Lenovo Device 21cf
Flags: bus master, fast devsel, latency 0, IRQ 43
Memory at f0000000 (64-bit, non-prefetchable) [size=4M]
Memory at e0000000 (64-bit, prefetchable) [size=256M]
I/O ports at 5000 [size=64]
Expansion ROM at <unassigned> [disabled]
Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
Capabilities: [d0] Power Management version 2
Capabilities: [a4] PCI Advanced Features
Kernel driver in use: i915
So it has an IO BAR.
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset
Family USB Enhanced Host Controller #2 (rev 04) (prog-if 20 [EHCI])
Subsystem: Lenovo Device 21cf
Flags: bus master, medium devsel, latency 0, IRQ 16
Memory at f252a000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [50] Power Management version 2
Capabilities: [58] Debug port: BAR=1 offset=00a0
Capabilities: [98] PCI Advanced Features
Kernel driver in use: ehci-pci
So IRQ but no MSI.
I suppose that this restriction can be removed for PCI devices that
1. Actually work when plugged in into RC Integrated EndPoint
2. Respond to the above limitations
These limitations are just guidance for future devices.
They can't change the past, such devices were made.
>
> > and had planned to
> > prevent attaching them to PCIe root ports (ioh3420 device) and PCIe
> > downstream switch ports (xio-3130 device) as well. I did this because,
> > even though qemu currently allows attaching a normal PCI device in any
> > of these three places, the restriction exists for real hardware and I
> > didn't see any guarantee that qemu wouldn't add the restriction in the
> > future in order to more closely emulate real hardware.
> >
> > However, since I did that, I've learned that many of the qemu
"pci"
> > devices really should be considered as "pci or pcie". Gerd Hoffman
lists
> > some of these cases in a bug he filed against libvirt:
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=1003983
> >
> > I would like to loosen up the restrictions in libvirt, but want to make
> > sure that I don't allow something that could later be forbidden by qemu
> > (thus creating a compatibility problem during upgrades). Beyond Gerd's
> > specific requests to allow ehci, uhci, and hda controllers to attach to
> > PCIe ports, are there any other devices that I specifically should or
> > shouldn't allow? (I would rather be conservative in what I allow -
it's
> > easy to allow more things later, but nearly impossible to revoke
> > permission once it's been allowed).
For the moment I would not remove any restrictions, but only the ones
requested and verified by somebody.
>
> IMO, we really need to grow an interface to query this kind of thing.
Basically libvirt needs to know:
1. for (libvirt) controllers: what kind of devices can be plugged in
2. for devices (controller is also a device)
- to which controllers can it be plugged in
- does it support hot-plug?
3. implicit controllers of the machine types (q35 - "pcie-root", i440fx -
"pci-root")
All the above must be exported to libvirt
Implementation options:
1. Add a compliance field on PCI/PCIe devices and controllers stating if it supports
PCI/PCIe or both (and maybe hot-plug)
- consider plug type + compliance to figure out whether a plug can go into a socket
2. Use Markus Armbruster idea of introducing a concept of "plug and sockets":
- dividing the devices into adapters and plugs
- adding sockets to bridges(buses?).
In this way it would be clear which devices can connect to bridges
Any thoughts?
Thanks,
Marcel
It's all not too hard to implement, we just need to know
what kind of interface makes sense for management.
So Cc libvir-list(a)redhat.com .
>
4:53 p.m.
New subject: [libvirt] Attaching PCI devices to the PCIe root complex
Il 25/09/2013 10:59, Michael S. Tsirkin ha scritto:
> > I couldn't find on PCIe spec any mention that "Root
Complex Integrated EndPoint"
> > must be PCIe. But, from spec 1.3.2.3:
> > - A Root Complex Integrated Endpoint must not require I/O resources claimed
through BAR(s).
> > - A Root Complex Integrated Endpoint must not generate I/O Requests.
> > - A Root Complex Integrated Endpoint is required to support MSI or MSI-X or both
if an
> > interrupt resource is requested.
Heh PCI-SIG keeps fighting against legacy interrupts and IO.
But lots of hardware happily ignores these rules.
And the reason is simple: software does not enforce them.
I think it's "must not require", not "must not have". So it's
the usual
rule that applies to PCIe device, i.e. that they should work even if the
OS doesn't enable the I/O BARs.
Then I have no idea what the I/O BAR in i915 is for, and whether the
device can be used without that BAR.
Paolo
5:28 p.m.
New subject: [libvirt] Attaching PCI devices to the PCIe root complex
On Wed, Oct 02, 2013 at 10:53:07AM +0200, Paolo Bonzini wrote:
Il 25/09/2013 10:59, Michael S. Tsirkin ha scritto:
>> > I couldn't find on PCIe spec any mention that "Root Complex
Integrated EndPoint"
>> > must be PCIe. But, from spec 1.3.2.3:
>> > - A Root Complex Integrated Endpoint must not require I/O resources claimed
through BAR(s).
>> > - A Root Complex Integrated Endpoint must not generate I/O Requests.
>> > - A Root Complex Integrated Endpoint is required to support MSI or MSI-X or
both if an
>> > interrupt resource is requested.
> Heh PCI-SIG keeps fighting against legacy interrupts and IO.
> But lots of hardware happily ignores these rules.
> And the reason is simple: software does not enforce them.
I think it's "must not require", not "must not have". So
it's the usual
rule that applies to PCIe device, i.e. that they should work even if the
OS doesn't enable the I/O BARs.
I agree, thanks for pointing this out.
Seems to still apply to the MSI rule.
Then I have no idea what the I/O BAR in i915 is for, and whether the
device can be used without that BAR.
Paolo
4047
days inactive
4054
days old
2 comments
2 participants
participants (2)
-
Michael S. Tsirkin -
Paolo Bonzini