Re: [libvirt] Analysis of the effect of adding PCIe root ports
On 10/06/2016 06:58 AM, Andrea Bolognani wrote:
On Wed, 2016-10-05 at 18:36 +0100, Richard W.M. Jones wrote:
>>> (b) It would be nice to turn the whole thing off for people who don't
>>> care about / need hotplugging.
>>
>> I had contemplated having an "availablePCIeSlots" (or something like
>> that) that was either an attribute of the config, or an option in
>> qemu.conf or libvirtd.conf. If we had such a setting, it could be
>> set to "0".
I remember some pushback when this was proposed. Maybe we
should just give up on the idea of providing spare
hotpluggable PCIe slots by default and ask the user to add
them explicitly after all.
We need to have *something* new in the config related to this, otherwise
it will be impossible to add new "available for hotplug" ports at the
same time as new unaddressed endpoint devices are added (unless an extra
port is added for each endpoint device as well, and of course that is
assuming that the user knows about things like auto-added memory balloon
devices, and which devices are going to be PCIe and which will be legacy
PCI, etc). If the "available" ports aren't somehow marked, the endpoint
devices will end up being placed on them.
> Note that changes to libvirt conf files are not usable by libguestfs.
>
> The setting would need to go into the XML, and please also make it
> possible to determine if $random version of libvirt supports the
> setting, either by a version check or something in capabilities.
Note that you can avoid using any PCIe root port at all by
assigning PCI addresses manually. It looks like the overhead
for the small (I'm assuming) number of devices a libguestfs
appliance will use is low enough that you will probably not
want to open that can of worm, though.
I looked in an example domain config Rich pastebin-ed in IRC yesterday.
These are the PCI devices he has:
* <controller type='scsi' model='virtio-scsi'> - currently put on a
pci-bridge, but will end up on a root-port after my patches
* <controller type='usb' model='blah'> x 4 - a set of USB2
controllers.
This will turn into a single USB3 controller on a root-port after my
patches. Alternately, since it seems you don't use it, you could
eliminate it with:
<controller type='usb' model='none'/>
* <controller type='sata'> - this is fixed on 00:1f.2, and there's
nothing we can do to get rid of it or even change its address
* <controller type='pci' model='dmi-to-pci-bridge'> - the need for
this
will be eliminated by my patches
* <controller type='pci' model='pci-bridge'> - also eliminated
* <controller type='virtio-serial'> - currently on the pci-bridge, but
will move to a root-port after my patches
* <memballoon model='virtio'> - Rich, do you actually use this device?
Seems doubtful. If not, then add
<memballoon model='none'/>
to your config to prevent it.
After getting rid of the controllers that will be eliminated by my
patches, and the "default" devices that you can get rid of with manual
intervention (model='none'), you'll be left with the following devices
on pcie-root-ports which *could* be manually moved to pcie-root (thus
eliminating one pcie-root-port from the domain):
1) virtio-scsi controller
2) virtio-serial controller
and nothing else. Manually address those two to be on bus 0 (pcie-root),
and (with my patches) you've reduced your PCI device+controller count
from the current 10 down to 3 (including the sata controller).