Re: [libvirt PATCH 00/10] Revert <acpi-hotplug-bridge>

Starting with commit 58ba0f6a3d7342fba29edbbf2bb9cb5497c870e5, support for overriding the default hotplug method for a guest using the <acpi-bridge-hotplug> subelement of <features>/<pci> was added to libvirt. This uses the QEMU global commandline switch: ICH9-LPC.acpi-pci-hotplug-with-bridge-support=<off|on> that was added to QEMU in qemu-6.1 (along with QEMU switching the default hotplug type for new Q35-based machinetypes from "native PCIe" to "ACPI"). Unfortunately, soon after we pushed the <acpi-bridge-hotplug> patches to libvirt (2 days after, to be exact), during a regular weekly meeting I attend with some of the QEMU developers, along with bugs that had been found in ACPI hotplug since it was made the default, they were also discussing issues they'd found with the implementation of the QEMU commandline switch to change back to native PCIe hotplug.

Apparently the current method used by acpi-pci-hotplug-with-bridge-support is causing "confusion" in guests, so they were talking about the possibility of changing what the switch does (or replacing it), and suggested that libvirt should "hold off" on supporting it for now. (oops) (they didn't know that we had just pushed Ani's patches that used it) Since the current QEMU option is doomed to either changed behavior (which would result in a guest-visible ABI change) or full deprecation and replacement, it would be better for libvirt to not use it at all, and re-implement based on whatever replacement QEMU comes up with. Once a new XML element has appeared in a libvirt upstream release, we are committed to keeping it there "forever". However, since there has not yet been an upstream release since <acpi-bridge-hotplug> was added, there is still time to revert and avoid the endless support/maintenance burden. Not much time though! And that is the purpose of this patch series. They revert, in reverse order, Ani's 4 original patches adding the feature, along with Peter's 6 followup patches that correct a logic error and streamline/beef up the unit tests. (Note that it is still possible that QEMU would figure out a way out of this without modifying their current flag in any significant way, and in that case we could always "re-apply" the original patches. However, if we don't revert before the next release (Andrea has pointed out the freeze for RC1 is next Tuesday, and it would be best to have it done before then), we will no longer have the option to remove it.) There was some conflict resolution necessary after the "git revert" commands, due to other unrelated patches that changed test case data, and due to a couple of Peter's patches also touching up the i440fx pci-root-hotplug disabling test cases (which are *not* being reverted - they work as advertised!). Note that this series involves *re-adding* some things, just to remove them again in a later revert - this is because Peter's patches effectively reverted the addition of a QEMU capability flag that had been added in Ani's original patches. If anyone would prefer, I can squash those patches together so that the extra dance is eliminated from the diffs; it just seemed more mechanical to do it this way though. A more detailed explanation of the issue: Current Behavior of existing QEMU option ======================================== As far as I understand (and keep in mind that I have misunderstood and misinterpreted this *at least* 4 separate times since it was first explained to me!), the effect of this QEMU setting: -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on is to: 1) enable ACPI hotplug (i.e. expose it to the guest) 2) disable native PCIe hotplug (don't expose it to the guest) By having only one of the two types of hotplug enabled, the intent is to force the guest to use the still-enabled type of hotplug. Unfortunately, after QEMU 6.1 was released with acpi-pci-hotplug=on as the default for new machinetypes, problems were encountered with ACPI hotplug, which caused more attention to be called to the QEMU switch, and the people looking into that found that enabling ACPI and disabling native PCIe hotplug doesn't necessarily have the desired effect of causing the guest to use ACPI for hotplug (or maybe it was the opposite direction). Instead, it "gets confused" (a very technical term, I know. You can ask Julia or Michael for a definition :-)). One possible fix that they talked about was changing the behavior of ICH9-LPC.acpi-pci-hotplug-with-bridge-support: Potential Change to Behavior of QEMU option =========================================== I know it's more complex than this (again, Julia or Michael can explain), but my basic understanding of the way that they're currently thinking of modifying the acpi-pci-hotplug-with-bridge-support option is to have everything the same, *except* that when acpi-hotplug=off, ACPI hotplug will *still be enabled* along with native PCIe hotplug; but because guest OSes prefer native hotplug over ACPI, native PCIe hotplug will be chosen in that case. (Presumably this change prevents the "confusion" that is seen with the existing behavior of the option). So essentially, the choice of whether to use ACPI is controlled by enabling/disabling native PCIe hotplug (which *kind of* goes against the libvirt philosophy of "the XML describes the virtual machine that is presented to the guest"; instead it becomes "the XML describes how the guest will behave"). Another possibility would be to completely scrap (well, deprecate and later remove) the current QEMU commandline switch in favor of one or more switches that behave differently but result in the desired behavior. In either case, I think the best course of action for libvirt is to revert the current <acpi-bridge-hotplug>, wait until QEMU settles down with a new workable set of switches, and then re-do libvirt support based on that. Laine Stump (10): Revert "qemu: capabilities: Remove QEMU_CAPS_PIIX4_ACPI_HOTPLUG_BRIDGE" Revert "qemuxml2xmltest: Convert all acpi-hotplug control related tests to DO_TEST_CAPS_LATEST" Revert "qemuxml2argvtest: Add '-enable' variants for ACPI-hotplug related cases" Revert "qemuxml2argvtest: Use real-caps testing for 'acpi-hotplug-bridge-disable'" Revert "qemuValidateDomainDefPCIFeature: Fix validation logic" Revert "qemuValidateDomainDefPCIFeature: un-break error messages" Revert "NEWS: document new acpi pci hotplug config option" Revert "qemu: command: add support for acpi-bridge-hotplug feature" Revert "conf: introduce support for acpi-bridge-hotplug feature" Revert "qemu: capablities: detect acpi-pci-hotplug-with-bridge-support" NEWS.rst | 8 -- docs/formatdomain.rst | 29 ------ docs/schemas/domaincommon.rng | 15 ---- src/conf/domain_conf.c | 89 +------------------ src/conf/domain_conf.h | 9 -- src/qemu/qemu_capabilities.c | 4 +- src/qemu/qemu_capabilities.h | 3 +- src/qemu/qemu_command.c | 19 ---- src/qemu/qemu_validate.c | 41 --------- .../caps_6.1.0.x86_64.xml | 1 - .../caps_6.2.0.x86_64.xml | 1 - ...-hotplug-bridge-disable.aarch64-latest.err | 1 - .../aarch64-acpi-hotplug-bridge-disable.xml | 13 --- ...-hotplug-bridge-disable.x86_64-latest.args | 34 ------- .../pc-i440fx-acpi-hotplug-bridge-disable.xml | 33 ------- ...i-hotplug-bridge-enable.x86_64-latest.args | 34 ------- .../pc-i440fx-acpi-hotplug-bridge-enable.xml | 33 ------- ...pi-hotplug-bridge-disable.x86_64-6.0.0.err | 1 - ...-hotplug-bridge-disable.x86_64-latest.args | 37 -------- .../q35-acpi-hotplug-bridge-disable.xml | 47 ---------- ...cpi-hotplug-bridge-enable.x86_64-6.0.0.err | 1 - ...i-hotplug-bridge-enable.x86_64-latest.args | 37 -------- .../q35-acpi-hotplug-bridge-enable.xml | 47 ---------- tests/qemuxml2argvtest.c | 10 --- ...i-hotplug-bridge-disable.x86_64-latest.xml | 36 -------- ...pi-hotplug-bridge-enable.x86_64-latest.xml | 36 -------- ...cpi-root-hotplug-disable.x86_64-latest.xml | 33 ------- .../pc-i440fx-acpi-root-hotplug-disable.xml | 1 + ...acpi-root-hotplug-enable.x86_64-latest.xml | 33 ------- .../pc-i440fx-acpi-root-hotplug-enable.xml | 1 + ...i-hotplug-bridge-disable.x86_64-latest.xml | 53 ----------- ...pi-hotplug-bridge-enable.x86_64-latest.xml | 53 ----------- tests/qemuxml2xmltest.c | 10 +-- 33 files changed, 9 insertions(+), 794 deletions(-) delete mode 100644 tests/qemuxml2argvdata/aarch64-acpi-hotplug-bridge-disable.aarch64-latest.err delete mode 100644 tests/qemuxml2argvdata/aarch64-acpi-hotplug-bridge-disable.xml delete mode 100644 tests/qemuxml2argvdata/pc-i440fx-acpi-hotplug-bridge-disable.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/pc-i440fx-acpi-hotplug-bridge-disable.xml delete mode 100644 tests/qemuxml2argvdata/pc-i440fx-acpi-hotplug-bridge-enable.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/pc-i440fx-acpi-hotplug-bridge-enable.xml delete mode 100644 tests/qemuxml2argvdata/q35-acpi-hotplug-bridge-disable.x86_64-6.0.0.err delete mode 100644 tests/qemuxml2argvdata/q35-acpi-hotplug-bridge-disable.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/q35-acpi-hotplug-bridge-disable.xml delete mode 100644 tests/qemuxml2argvdata/q35-acpi-hotplug-bridge-enable.x86_64-6.0.0.err delete mode 100644 tests/qemuxml2argvdata/q35-acpi-hotplug-bridge-enable.x86_64-latest.args delete mode 100644 tests/qemuxml2argvdata/q35-acpi-hotplug-bridge-enable.xml delete mode 100644 tests/qemuxml2xmloutdata/pc-i440fx-acpi-hotplug-bridge-disable.x86_64-latest.xml delete mode 100644 tests/qemuxml2xmloutdata/pc-i440fx-acpi-hotplug-bridge-enable.x86_64-latest.xml delete mode 100644 tests/qemuxml2xmloutdata/pc-i440fx-acpi-root-hotplug-disable.x86_64-latest.xml create mode 120000 tests/qemuxml2xmloutdata/pc-i440fx-acpi-root-hotplug-disable.xml delete mode 100644 tests/qemuxml2xmloutdata/pc-i440fx-acpi-root-hotplug-enable.x86_64-latest.xml create mode 120000 tests/qemuxml2xmloutdata/pc-i440fx-acpi-root-hotplug-enable.xml delete mode 100644 tests/qemuxml2xmloutdata/q35-acpi-hotplug-bridge-disable.x86_64-latest.xml delete mode 100644 tests/qemuxml2xmloutdata/q35-acpi-hotplug-bridge-enable.x86_64-latest.xml -- 2.31.1