On Thu, Apr 11, 2013 at 07:03:56AM -0400, Laine Stump wrote:
On 04/10/2013 05:26 AM, Daniel P. Berrange wrote:
On Tue, Apr 09, 2013 at 04:06:06PM -0400, Laine Stump wrote:
On Mon, Apr 08, 2013 at 03:32:07PM -0400, Laine Stump wrote: Actually I do wonder if we should reprent a PCI root as two <controller> elements, one representing the actual PCI root device, and the other representing the host bridge that is built-in.
Also we should use the actual model names, not 'pci-root' or 'pcie-root' but rather i440FX for "pc" machine type, and whatever the q35 model name is.
- One PCI root with built-in PCI bus (ie todays' setup)
<controller type="pci-root" index="0"> <model name="i440FX"/> </controller> <controller type="pci" index="0"> <!-- Host bridge --> <address type='pci' domain='0' bus='0' slot='0''/> Isn't this saying that the bridge connects to itself? (since bus 0 is
On 04/09/2013 04:58 AM, Daniel P. Berrange wrote: this bus)
I understand (again, possibly wrongly) that the builtin PCI bus connects to the chipset using its own slot 0 (that's why it's reserved), but that's its address on itself. How is this bridge associated with the pci-root?
Ah, I *think* I see it - the domain attribute of the pci controller is matched to the index of the pci-root controller, correct? But there's still something strange about the <address> of the pci controller being self-referential. Yes, the index of the pci-root matches the 'domain' of <address>
Okay, then the way that libvirt differentiates between a pci bridge that is connected to the root, and one that is connected to a slot of another bridge is 1) the "bus" attribute of the bridge's <address> matches the "index" attribute of the bridge itself, and 2) "slot" is always 0. Correct?
(The corollary of this is that if slot == 0 and bus != index, or bus == index and slot != 0, it is a configuration error).
I'm still unclear on the usefulness of the pci-root controller though - all the necessary information is contained in the pci controller, except for the type of root. But in the case of pcie root, I think you're not allowed to connect a standard bridge to it, only a "dmi-to-pci-bridge" (i82801b11-bridge)
Yes you can connect a pci bridge to pcie-root. It's represented as a root complex integrated device.
</controller> <interface type='direct'> ... <address type='pci' domain='0' bus='0' slot='3'/> </controller>
- One PCI root with built-in PCI bus and extra PCI bridge
<controller type="pci-root" index="0"> <model name="i440FX"/> </controller> <controller type="pci" index="0"> <!-- Host bridge --> <address type='pci' domain='0' bus='0' slot='0'/> </controller> <controller type="pci" index="1"> <!-- Additional bridge --> <address type='pci' domain='0' bus='0' slot='1'/> </controller> <interface type='direct'> ... <address type='pci' domain='0' bus='1' slot='3'/> </controller>
- One PCI root with built-in PCI bus, PCI-E bus and and extra PCI bridge (ie possible q35 setup) Why would a q35 machine have an i440FX pci-root? It shouldn't, that's a typo
<controller type="pci-root" index="0"> <model name="i440FX"/> </controller> <controller type="pci" index="0"> <!-- Host bridge --> <address type='pci' domain='0' bus='0' slot='0'/> </controller> <controller type="pci" index="1"> <!-- Additional bridge --> <address type='pci' domain='0' bus='0' slot='1'/> </controller> <controller type="pci" index="1"> <!-- Additional bridge --> <address type='pci' domain='0' bus='0' slot='1'/> </controller> I think you did a cut-paste here and intended to change something, but didn't - those two bridges are identical. Yep, the slot should be 2 in the second one
<interface type='direct'> ... <address type='pci' domain='0' bus='1' slot='3'/> </controller>
So if we later allowed for mutiple PCI roots, then we'd have something like
<controller type="pci-root" index="0"> <model name="i440FX"/> </controller> <controller type="pci-root" index="1"> <model name="i440FX"/> </controller> <controller type="pci" index="0"> <!-- Host bridge 1 --> <address type='pci' domain='0' bus='0' slot='0''/> </controller> <controller type="pci" index="0"> <!-- Host bridge 2 --> <address type='pci' domain='1' bus='0' slot='0''/> </controller> <interface type='direct'> <!-- NIC on host bridge 2 --> ... <address type='pci' domain='1' bus='0' slot='3'/> </controller>
NB this means that 'index' values can be reused against the <controller>, provided they are setup on different pci-roots.
(also note that it might happen that the bus number in libvirt's config will correspond to the bus numbering that shows up in the guest OS, but that will just be a happy coincidence)
Does this make sense? Yep, I think we're fairly close. What about the other types of pci controllers that are used by PCIe? We should make sure they fit in this model before we settle on it. What do they do ?
Although I've asked that question, and had it answered, several times now, without a tabular list, I'm still unable to get the answer to that question clear in my mind :-/
Here's the controllers that aw has talked about (in a recent response on a different sub-thread):
On 04/05/2013 03:26 PM, Alex Williamson wrote:
For PCIe, we create new buses for root ports (ioh3420), upstream switch ports (xio3130-upstream), downstream switch ports (xio3130-downstream), and the dmi-to-pci bridge (i82801b11-bridge). For PCI, PCI-to-PCI bridges create new buses (pci-bridge and dec-21154).
Alex: what do each of these connect to, and what can be connected to them?
root-port (ioh3420)
this is connected to the root bus. any express device connects here (at most one device but it can be multifunction)
upstream-switch-port (xio3130-upstream)
this connects to a root port or downstream port, as any express device. only downstream ports connect here.
downstream-switch-port (xio3130-downstream)
this connects to the upstream port. any express device connects here (at most one device but it can be multifunction)
dmi-to-pci-bridge (i82801b11-bridge)
this connects to the root bus. any pci (non express) device connects here
We already know/understand this one: pci-bridge; is this identical in behavior/function? dec-21154
Yes, only dec pretends it's an intel bridge device, pci-bridge tells guest it's a generic device. Does not seem to matter with any guest I tried.
One of my goals is to move us away from emulation of specific chips and create more devices like pci-bridge that adhere to the standard, but don't try to emulate a specific device. Then we might have "root-port", "pcie-upstream-switch-port", "pcie-downstream-switch-port", and "dmi-to-pci-bridge" (none of these names have been discussed).