Re: [libvirt-users] assigning PCI addresses with bus > 0x09
On 12/20/18 9:20 AM, Daniel P. Berrangé wrote:
On Thu, Dec 20, 2018 at 03:15:34PM +0100, Riccardo Ravaioli wrote:
> Hi,
>
> My goal is to assign PCI addresses to a number of devices (network
> interfaces, disks and PCI devices in PCI-passthrough) without delegating to
> libvirt the generation of those values. This should give me more control
> and for sure more predictability over the hardware configuration of a
> virtual machine and consequently the name of the interfaces in it. I'm
> using libvirt 4.3.0 to create qemu/KVM virtual machines running Linux
> (Debian Stretch).
>
> So, for every device of the type mentioned above, I add this line:
> <address type='pci' domain='0x0000' bus='0x__'
slot='0x__' function='0x0'/>,
> ... with values from 00 to ff in the bus field, and from 00 to 1f in the
> slot field, as described in the documentation.
>
> Long story short, I noticed that as soon as I assign values > 0x09 to the
> bus field, the serial console hangs indefinitely, in both Debian and
> Ubuntu. The VM seems to be started correctly and its state is "running";
in
> the XML file created by libvirt, I see all controllers from 0 the largest
> bus value I assigned, so everything from that side seems ok.
I guess the hang is that you hit some limit in PCI buses.
I think you're right. Each bus requires some amount of IO space, and I
thought I recalled someone saying that all of the available IO space is
exhausted after 7 or 8 buses. This was in relation to PCIe, where each
root port is a bus, and can potentially take up IO space, so possibly in
that context they were talking about the buses *after* the root bus and
pcie-pci-bridge, which would bring us back to the same number you're
getting.
For PCIe our solution was to turn off IO space usage on the
pcie-root-ports, but you can't do that for conventional PCI buses, since
the devices might actually need IO space to function properly and the
standard requires that you provide it.
The real question though is why you need to create sooooo many PCI buses.
Each bus can do 31 devices. Do you really need to have more than 279
devices in your VM ?
And if you do need more than 279 devices, do they all need to be
hot-pluggable? If not, then you can put up to 8 devices on each slot
(functions 0 - 7).
Regards,
Daniel