On Fri, Nov 04, 2022 at 03:01:38PM +0100, Denis V. Lunev wrote:
On 11/4/22 14:41, Daniel P. Berrangé wrote:
On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
Hotplugging PCI devices on pc-i440fx machines is supported without additional configuration. On q35, pcie-to-pci-bridge needs to be added prior to the machine startup in order to support hotplug [1].
The idea is to support the same workflow for creating VMs in q35, as was in pc. Namely, do not require additional configuration when hotplugging is needed. Otherwise, all libvirt clients need to be updated which there are a lot and they are maintained by different parties.
Instead, a pcie-to-pci-bridge better be created by default, so that PCI slots are readily available. Might be a good idea to make it configurable in the future. The goal of q35 is to provide a PCI-e topology though, and we want devices to be exposed as PCI-e endpoints, with no legacy PCI usage by default.
When a guest XML is provided for cold boot, libvirt will ensure that all devices are placed onto the PCI-e bus via pcie-root-ports
With this proposal hotplugging a device will cause it to be placed on a PCI bus instead, if there are no spare pcie-root-port available.
This behavioural difference between cold plug and hot plug is very undesirable IMHO, as it is counter to our goal of avoiding legacy PCI on PCI-e machines.
that is an interesting point which I have initially missed. I have checked our investigation log and found that we have potentially made a mistake with making compatibility check with e1000 device instead of virtio, which is our default. This could be the reason.
If a mgmt app wants to enable this mis-matched behaviour they can add a pcie-to-pci-bridge when initially cold booting the VM, but we shouldn't add one by default IMHO. We really recommend that apps just pre-populate 10-30 pcie-root-port devices when creating a VM, so there is plenty of space for hotplugging devices later.
this potentially an option, but I think that these port should be created automatically by libvirt to avoid major client rewrites. Will the option to keep the amount of spare ports in /etc/libvirt/qemu.conf work for you?
Our general view is that qemu.conf should not cause changes that affect the guest ABI. For an given input XML, we want applications to have a predictable guest ABI created. Influencing the guest ABI from qemu.conf settings would mean applications have uncertain results for a given XML config, so what works on one host may break on another host. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|