5:19 a.m.
On Tue, Apr 09, 2013 at 09:59:50AM +0100, Daniel P. Berrange wrote:
On Mon, Apr 08, 2013 at 10:37:45PM +0300, Michael S. Tsirkin wrote:
> On Mon, Apr 08, 2013 at 03:32:07PM -0400, Laine Stump wrote:
> > On 04/08/2013 12:48 PM, Daniel P. Berrange wrote:
> > > On Mon, Apr 08, 2013 at 12:37:49PM -0400, Laine Stump wrote:
> > >> I think we're starting to get closer to the concrete problem
that's
> > >> bothering me. As I understand it (and again - "what I
understand" has
> > >> repeatedly been shown to be incorrect in this thread :-):
> > >>
> > >> * Ihere are multiple different types of devices that provide a bus
with
> > >> 1 or more "slots" that PCI devices (e.g., the virtio-net-pci
device, the
> > >> e1000 network device, etc) can be plugged into.
> > >>
> > >> * In the config for those devices, there is a required
(auto-generated
> > >> if not explicitly provided) <address> element that indicates
what
> > >> controller that device is plugged into e.g.:
> > >>
> > >> <interface type='direct'>
> > >> ...
> > >> <address type='pci' domain='0'
bus='0' slot='3' function='0'/>
> > >> ...
> > >> </interface>
> > >>
> > >> * domain is always hardcoded to 0, and in the past bus was also
always
> > >> hardcoded to 0 because until now there has only been a single place
> > >> where PCI devices could be connected - the builtin pci.0 bus, which is
a
> > >> part of the basic "pc" (and some others) virtual machine and
provides 32
> > >> slots.
> > >>
> > >> * Now we are adding the ability to define new PCI buses, for now just
a
> > >> single kind - a pci-bridge controller, which itself must connect to
an
> > >> existing PCI slot, and provides 32 new PCI slots. But in the future
> > >> there will be more different types of controllers that provide one or
> > >> more PCI slots where PCI devices/controllers can be plugged in.
> > >>
> > >> * In these patches adding support for pci-bridge, we are making the
> > >> assumption that there is a 1:1 correspondence between the
"index='n'"
> > >> attribute of the pci-bridge controller and the
"bus='n'" attribute of
> > >> the <address> element in devices that will be plugged into that
> > >> controller. So for example if we have:
> > >>
> > >>
> > >> <controller type='pci-bridge' index='1'>
> > >> <address type='pci' domain='0'
bus='0' slot='10' function='0'/>
> > >> </controller>
> > >>
> > >> and then change the <interface> definition above to say
"bus='1'", that
> > >> interface device will plug into this new bus at slot 3.
> > >>
> > >> * So let's assume that we add a new controller called
"dmi-to-pci-bridge":
> > >>
> > >> <controller type='dmi-to-pci-bridge'
index='0'/>
> > >>
> > >> Ignoring for now the question of what address we give in the
definition
> > >> of *this* device (which is itself problematic - do we need a new
"pcie"
> > >> address type?), if some device is then defined with
> > >>
> > >>
> > >> <address type='pci bus='0' .../>
> > >>
> > >> How do we differentiate between that meaning "the pci-ptp
controller
> > >> that is index='0'" and "the pci-bridge controller
that is index='0'"? Do
> > >> we need to expand our <address> element further? If, as I think
you
> > >> suggest, we have multiple different kinds of controllers that provide
> > >> PCI slots, each with its own namespace, the current pci address
element
> > >> is inadequate to unambiguously describe where a pci device should be
> > >> plugged in.
> > > Hmm yes, you're right - as long as we only have <adress
type='pci'>
> > > then all <controller> elements should use type='pci' too,
and we should
> > > just distinguish based on the model name of the controller. So ignore
> > > my previous suggestion to have 'pci-bridge' and 'pci-root'
types, we
> > > can only use type='pci' on <controller> elements.
> >
> > Okay, so that means we preserve the correlation between
> >
> > <controller type='pci' index='1'>
> >
> > and
> >
> > <address type='pci' bus='1' ..../>
> >
> >
> > Should the <controller> device use, e.g. <model
type='pci-bridge'/> for
> > the model, as is done for <interface> devices? One notable difference is
> > that in the case of <interface> (with the exception of "<model
> > type='virtio'/>"), the model isn't used for anything except
passing
> > directly through to qemu (and very recently validating against a list of
> > known interface models), while in the case of controllers with
> > type='pci', different models will have different rules about what they
> > can connect to and what can connect to them, and they will affect what
> > is valid in other devices.
> >
> > An example on a "pc" machinetype that has the builtin PCI bus, one
extra
> > pci-pci bridge, and an interface device plugged into slot 3 of the
> > pci-bridge:
> >
> > <controller type='pci' index='0'>
> > <model type='pci-root'/> <!-- builtin pci bus -->
> > </controller>
> > <controller type='pci' index='1'>
> > <model type='pci-bridge'/>
> > </controller>
> > <interface type='direct'>
> > ...
> > <address type='pci' bus='1' slot='3'/>
> > </controller>
> >
> > And for a q35 machinetype that has the root pcie, an i82801b11-bridge
> > connected to slot 1e of that, a pci bridge connected to slot 1 of the
> > i82801b11-bridge, and an interface plugged into slot 3 of the pci-bridge:
> >
> > <controller type='pci' index='0'>
> > <model type='pcie-root'/>
> > </controller>
> > <controller type='pci' index='1'>
> > <model type='i82801b11-bridge'/> <!-- [*] -->
> > <address type='pci' bus='0' slot='0x1e'/>
> > </controller>
> > <controller type='pci' index='2'>
> > <model type='pci-bridge'/>
> > <address type='pci' bus='1' slot='1'/>
> > </controller>
> > <interface type='direct'>
> > ...
> > <address type='pci' bus='2' slot='3'/>
> > </controller>
> >
> > (note that controllers with model='(pci|pcie)-root' will not have any
> > <address> element, because they exist in the basic machine so we
don't
> > need to connect them to anywhere.)
> >
> > (also note that it might happen that the bus number in libvirt's config
> > will correspond to the bus numbering that shows up in the guest OS, but
> > that will just be a happy coincidence)
> >
> > Does this make sense?
>
> Confused. So why are you using bus numbers at all?
> It's just wrong.
They are not wrong. We use to link the <address> element to the <controller>
element.
Daniel
I see so it's just a number, not bus number in pci sense?
Maybe name it bus_id or something.