Re: [libvirt] new libvirt "pci" controller type and pcie/q35 (was Re: [PATCH 4/7] add pci-bridge controller type)

On 04/05/2013 01:38 PM, Daniel P. Berrange wrote: > On Fri, Apr 05, 2013 at 12:32:04PM -0400, Laine Stump wrote: >> On 04/03/2013 11:50 AM, Ján Tomko wrote: >>> From: liguang <lig.fnst(a)cn.fujitsu.com&gt; >>> >>> add a new controller type, then one can >>> define a pci-bridge controller like this: >>> <controller type='pci-bridge' index='0'/> >> In the next patch we're prohibiting exactly this config (index='0') >> because the pre-existing pci bus on the "pc-*" machinetypes is already >> named pci.0. If we don't allow it, we shouldn't include it as an example >> in the commit log :-) > NB, it isn't always named 'pci.0' - on many arches it is merely 'pci'. Yeah, I'm just using that as a convenient shorthand. The final decision on whether to use pci.0 or pci happens down in the qemuBuildCommandline(). > >> More on this - one of the things this points out is that there is no >> representation in the config of the pci.0 bus, it's just assumed to >> always be there. That is the case for pc-* machinetypes (and probably >> several others with PCI buses), but for q35, there is no pci.0 bus in >> the basic machine, only a pcie.0; if you want a pci.0 on q35 (which >> *will* be necessary in order to attach any pci devices, so I imagine we >> will always want one), you have to attach a pcie->pci bridge, which is >> the device "i82801b11-bridge", to pcie.0. >> The reason I bring this up here, is I'm wondering: >> >> 1) should we have some representation of the default pci.0 bus in the >> config, even though it is just "always there" for the pc machinetypes >> and there is no way to disable it, and nothing on the commandline that >> specifies its existence? > Yep, we should be aiming for the XML to fully describe the machine > hardware. So since we're adding the concept of PCI controllers/bridges > etc to the XML, we should be auto-adding the default bus to the XML. > >> 2) For the q35 machinetype, should we just always add an >> i82801b11-bridge device and name it pci.0? Or should that need to be >> present in the xml? > We've been burnt before auto-adding stuff that ought to have > been optional. So I'd tend towards only having the minimal > config that is required. If the users want this, let them > explicitly ask for the bridge > > Also from the apps POV the QEMU device name is irrelevant. The > XML config works off the PCI addresses. So there's no need > to force/specialcase a i82801b11-bridge to use the name > 'pci.0'. Sure. I just mean "pci bus 0" (hmm, but actually this does point out a problem with my logic - the same namespace (well, "numbering space") is used for both pcie and pci buses, so on a q35 system, bus=0 is already taken by pcie.0; that means that the first pci bus would need to use a different bus number anyway, so it wouldn't be so easy to switch an existing domain from pc to q35 - every PCI device would need to have its bus number modified. I suppose that's reasonable to expect, though.

>> 3) Most important - depending on the answers to (1) and (2), should we >> maybe name this device "pci", and use a different backend depending on >> index and machinetype? (or alternately explicitly specifiable with a >> <driver> subelement). To be specific, we would have: >> >> <controller type='pci' index='0'/> >> >> which on pc machinetypes would just be a placeholder in the config (and >> always inserted if it wasn't there, for machinetypes that have a pci >> bus). On the q35 machinetype, that same line would equate to adding an >> i82801b11-bridge device (with source defaulting to >> bus=pcie.0,addr=1e.0). This would serve several purposes: >> >> a) on pc machinetypes, it would be a visual aid indicating that pci.0 >> exists, and that index='0' isn't available for a new pci controller. >> >> b) it would make switching a domain config from pc to q35 simpler, since >> pci.0 would always already be in place for attaching pci devices >> (including pci.1, pci.2, etc) >> >> c) it would make the config a true complete description of the machine >> being created. >> >> (I've suggested naming the controller "pci" rather than "pci-bridge" >> because in the case of a "root" bus like pci.0 it seems to not be a >> "bridge", but maybe the name "pci-bridge" is always appropriate, even >> when it's a root bus. Maybe someone with better pci/pcie knowledge can >> provide an opinion on this) > I think "pci" is a little too generic - how about we call it 'pci-root' Okay, so a separate "pci-root" device along with "pci-bridge"? What I was really hoping was to have all PCI buses represented in a common way in the config. How about a controller called "pci" with different types, "root" and "bridge"? And since they use the same numbering space as pcie buses, maybe the pcie controllers (including the root and the hubs and ???) would be different types of PCI controllers. That would make it easier (i.e. *possible*) to avoid collisions in use of bus numbers. Alex or mst, any advice/opinions on how to represent all the different q35 devices that consume bus numbers in a succinct fashion?