1:14 p.m.
On 08/09/2016 03:54 AM, Daniel P. Berrange wrote:
On Mon, Aug 08, 2016 at 04:56:55AM -0400, Laine Stump wrote:
> When faced with a guest device that requires a PCI address but doesn't
> have one manually assigned in the config, libvirt has always insisted
> (well... *tried* to insist) on auto-assigning an address that is on a
> PCI controller that supports hotplug. One big problem with this is
> that it prevents automatic use of a Q35 (or aarch64/virt) machine's
> pcie-root (since the PCIe root complex doesn't support hotplug).
I'm not really seeing why lack of use of pcie-root is a real problem. It
makes total sense to always use the bridge instead of root if the latter
does not support hotplug. At the time you first create a guest, you may
not even realize that you'll need to be able to hot-unplug a particular
device later. So putting any device on pcie-root is tieing your hands for
the entire lifetime of the guest.
1) simpler bus topology, less devices - the less there is, the less
resources are used, the less can go wrong (among other advantages)
2) In particular, there is a lot of call for making Q35 and aarch64/virt
machines "legacy-PCI-free". But some of the emulated devices are
PCI-only, and until PCIe alternatives are available, this provides a
method of getting the use of these devices (yes, I realize that in
itself screws over the "legacy-PCI-free" demand) at least without
requiring the addition of a pci-bridge or the dreaded dmi-to-pci-bridge.
3) Even for PCIe devices, there currently is no way to have a
hotpluggable device without using a vendor-specific PCI controller that
doesn't necessarily exist on the real hardware that is being emulated
(pcie-root-port emulates an Intel IOH3420, and
pcie-switch-downstream-port is a TI X3130; I have serious doubts that
either of those are used (or will ever be used) in any real aarch64
hardware). Requiring those controllers to be added into the virtual
machine may lead to incompatibilities with OSes that have been designed
only for real hardware and may be lacking the proper drivers for these
controllers.
4) libvirt should not be making policy decisions about things like
whether or not a device should be hotpluggable. On the 440fx, *all*
devices are in hotplug-capable slots, so this is a non-issue. On Q35 and
other machinetypes with a pcie-root, there is a choice, and that choice
should be made by the user/management app, not forced on them by libvirt.
As far as not realizing that you'll need to hot-unplug some device:
that's true and that's why the default is to assume hotplug will be
needed. But certainly for most devices you can say with a very high
degree of certainty that you'll never hot-unplug them. Why force extra
infrastructure for something that will definitely never be used?
Although it shouldn't be the default, we really do need a way to allow
avoiding the extra trappings of hotplug without requiring the user to
specify the full address.
> In order to promote simpler domain configs (more devices on
pcie-root
> rather than on a pci-bridge), this patch adds a new sub-element to all
> guest devices that have a PCI address and support hotplug:
>
> <hotplug require='no'/>
>
> For devices that have hotplug require='no', we turn off the
> VIR_PCI_CONNECT_HOGPLUGGABLE bit in the devFlags when searching for an
> available PCI address. Since pcie-root now allows standard PCI
> devices, this results in those devices being placed on pcie-root
> rather than pci-bridge.
I still really hate the idea of adding elements to the XML config
that have nothing todo with the actual guest ABI config, but instead
are setting logic policy in libvirt. But more generally I don't see
why anyone would ever want to set require=no for this - it just ties
your hands for later.
So that you can explicitly acknowledge that you don't want the extra
trappings that hotplug requires.
> NB: According to the code in qemuDomainAssignDevicePCISlots()
> and qemuDomainAttachDeviceLive(), the following device types are the
> only ones that use a PCI address *and* support hotplug:
>
> <interface>
> <hostdev>
> <rng>
> <serial>
> <disk> (only virtio)
> <controller> (hotplug only for SCSI controllers)
>
> These devices use a PCI address, but don't support hotplug, so the RNG
> doesn't allow them to have a <hotplug> element:
>
> <filesystem>
> <sound>
> <balloon>
> <watchdog>
> <video>
> <shmem>
> <input>
IMHO it is really bad practice to make any assumption about what
devices support hotplug & what don't. In fact if any device type
does not support hotplug I think it is fair to argue that is
simply a bug / missing feature in libvirt right now. I see no
reason why any of those devices should not be able to support
hotplug in general.
Yes, good point; I suppose that if a future version of libvirt adds the
ability to hotplug those devices, someone may be upset that the virtual
machine they created in the past can't hot-unplug those devices. But
does that mean we shouldn't be placing the default video device directly
at 00:1.0, or the default USB controller at 00:1D.x?