Re: [libvirt] new libvirt "pci" controller type and pcie/q35 (was Re: [PATCH 4/7] add pci-bridge controller type)

On 04/10/2013 05:26 AM, Daniel P. Berrange wrote: > On Tue, Apr 09, 2013 at 04:06:06PM -0400, Laine Stump wrote: >> On 04/09/2013 04:58 AM, Daniel P. Berrange wrote: >>> On Mon, Apr 08, 2013 at 03:32:07PM -0400, Laine Stump wrote: >>> Actually I do wonder if we should reprent a PCI root as two >>> <controller> elements, one representing the actual PCI root >>> device, and the other representing the host bridge that is >>> built-in. >>> >>> Also we should use the actual model names, not 'pci-root' or >>> 'pcie-root' but rather i440FX for "pc" machine type, and whatever >>> the q35 model name is. >>> >>> - One PCI root with built-in PCI bus (ie todays' setup) >>> >>> <controller type="pci-root" index="0"> >>> <model name="i440FX"/> >>> </controller> >>> <controller type="pci" index="0"> <!-- Host bridge --> >>> <address type='pci' domain='0' bus='0' slot='0''/> >> Isn't this saying that the bridge connects to itself? (since bus 0 is >> this bus) >> >> I understand (again, possibly wrongly) that the builtin PCI bus connects >> to the chipset using its own slot 0 (that's why it's reserved), but >> that's its address on itself. How is this bridge associated with the >> pci-root? >> >> Ah, I *think* I see it - the domain attribute of the pci controller is >> matched to the index of the pci-root controller, correct? But there's >> still something strange about the <address> of the pci controller being >> self-referential. > Yes, the index of the pci-root matches the 'domain' of <address> Okay, then the way that libvirt differentiates between a pci bridge that is connected to the root, and one that is connected to a slot of another bridge is 1) the "bus" attribute of the bridge's <address> matches the "index" attribute of the bridge itself, and 2) "slot" is always 0. Correct? (The corollary of this is that if slot == 0 and bus != index, or bus == index and slot != 0, it is a configuration error). I'm still unclear on the usefulness of the pci-root controller though - all the necessary information is contained in the pci controller, except for the type of root. But in the case of pcie root, I think you're not allowed to connect a standard bridge to it, only a "dmi-to-pci-bridge" (i82801b11-bridge)

>>> </controller> >>> <interface type='direct'> >>> ... >>> <address type='pci' domain='0' bus='0' slot='3'/> >>> </controller> >>> >>> - One PCI root with built-in PCI bus and extra PCI bridge >>> >>> <controller type="pci-root" index="0"> >>> <model name="i440FX"/> >>> </controller> >>> <controller type="pci" index="0"> <!-- Host bridge --> >>> <address type='pci' domain='0' bus='0' slot='0'/> >>> </controller> >>> <controller type="pci" index="1"> <!-- Additional bridge --> >>> <address type='pci' domain='0' bus='0' slot='1'/> >>> </controller> >>> <interface type='direct'> >>> ... >>> <address type='pci' domain='0' bus='1' slot='3'/> >>> </controller> >>> >>> - One PCI root with built-in PCI bus, PCI-E bus and and extra PCI bridge >>> (ie possible q35 setup) >> Why would a q35 machine have an i440FX pci-root? > It shouldn't, that's a typo > >>> <controller type="pci-root" index="0"> >>> <model name="i440FX"/> >>> </controller> >>> <controller type="pci" index="0"> <!-- Host bridge --> >>> <address type='pci' domain='0' bus='0' slot='0'/> >>> </controller> >>> <controller type="pci" index="1"> <!-- Additional bridge --> >>> <address type='pci' domain='0' bus='0' slot='1'/> >>> </controller> >>> <controller type="pci" index="1"> <!-- Additional bridge --> >>> <address type='pci' domain='0' bus='0' slot='1'/> >>> </controller> >> I think you did a cut-paste here and intended to change something, but >> didn't - those two bridges are identical. > Yep, the slot should be 2 in the second one > >>> <interface type='direct'> >>> ... >>> <address type='pci' domain='0' bus='1' slot='3'/> >>> </controller> >>> >>> So if we later allowed for mutiple PCI roots, then we'd have something >>> like >>> >>> <controller type="pci-root" index="0"> >>> <model name="i440FX"/> >>> </controller> >>> <controller type="pci-root" index="1"> >>> <model name="i440FX"/> >>> </controller> >>> <controller type="pci" index="0"> <!-- Host bridge 1 --> >>> <address type='pci' domain='0' bus='0' slot='0''/> >>> </controller> >>> <controller type="pci" index="0"> <!-- Host bridge 2 --> >>> <address type='pci' domain='1' bus='0' slot='0''/> >>> </controller> >>> <interface type='direct'> <!-- NIC on host bridge 2 --> >>> ... >>> <address type='pci' domain='1' bus='0' slot='3'/> >>> </controller> >>> >>> >>> NB this means that 'index' values can be reused against the >>> <controller>, provided they are setup on different pci-roots. >>> >>>> (also note that it might happen that the bus number in libvirt's config >>>> will correspond to the bus numbering that shows up in the guest OS, but >>>> that will just be a happy coincidence) >>>> >>>> Does this make sense? >>> Yep, I think we're fairly close. >> What about the other types of pci controllers that are used by PCIe? We >> should make sure they fit in this model before we settle on it. > What do they do ? Although I've asked that question, and had it answered, several times now, without a tabular list, I'm still unable to get the answer to that question clear in my mind :-/ Here's the controllers that aw has talked about (in a recent response on a different sub-thread): On 04/05/2013 03:26 PM, Alex Williamson wrote: > For PCIe, we create new buses for root ports (ioh3420), upstream switch > ports (xio3130-upstream), downstream switch ports (xio3130-downstream), > and the dmi-to-pci bridge (i82801b11-bridge). For PCI, PCI-to-PCI > bridges create new buses (pci-bridge and dec-21154). Alex: what do each of these connect to, and what can be connected to them? root-port (ioh3420)

upstream-switch-port (xio3130-upstream)

downstream-switch-port (xio3130-downstream)

dmi-to-pci-bridge (i82801b11-bridge)

We already know/understand this one: pci-bridge; is this identical in behavior/function? dec-21154

> One of my goals is to move us away from emulation of specific chips and > create more devices like pci-bridge that adhere to the standard, but > don't try to emulate a specific device. Then we might have "root-port", > "pcie-upstream-switch-port", "pcie-downstream-switch-port", and > "dmi-to-pci-bridge" (none of these names have been discussed).