Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines

On Fri, Nov 04, 2022 at 08:23:52PM +0600, Oleg Vasilev wrote: > On 04.11.2022 20:19, Andrea Bolognani wrote: >> Additionally, after this change it would be impossible to create a >> q35 VM *without* the additional bridge. Most users of the q35 machine >> type are likely using PCIe devices exclusively, and such a change >> would negatively impact them. >> >> In the (hopefully rare) cases in which hotplugging traditional PCI >> devices is actually required, enabling that at the time when the >> domain is defined is already straightforward: just include the >> relevant controller in the XML. >> >> Unfortunately q35, unlike i440fx, doesn't support hotplugging of any >> kind out of the box, so it's up to the user / management application >> to ensure that the necessary controllers are added. This is not >> ideal, but there's no way around it, and it's still preferable to >> forcing unavoidable, potentially useless extra controllers on >> everybody. > > Apparently, there is a way to hotplug pcie-to-pci-bridge described in qemu > doc [1]. Do you think this approach with bus-reserve=1 would be acceptable > to support in libvirt by default? > > [1]: https://github.com/qemu/qemu/blob/master/docs/pcie_pci_bridge.txt#L25 I don't think so. First of all, the document says that the feature is only supported by SeaBIOS, whereas most q35 guests are likely using UEFI firmware. Moreover, we wouldn't be able to just set bus_reserve=1 for all pcie-root-ports, as that would result in a massive waste of PCI bus numbers. So we'd have to select a single pcie-root-port and make sure we don't hotplug anything but a pcie-to-pci-bridge in there. I can't imagine a way to implement this that's not going to be both incredibly complicated and extremely fragile.

At the end of the day, q35 just requires to plan the hotplug capabilities of the VM ahead of time, and this is true both for PCIe and traditional PCI hotplug. At the libvirt level, I don't really see a way around that. Higher-level tools are of course free to offer a more opinionated out of the box experience. I believe both virt-manager and OpenStack nova allocate a few pcie-root-ports by default to ensure PCIe hotplug can work. Perhaps they'd be okay adding a pcie-to-pci-bridge by default too, making the experience for end users a less frustrating one.