9:48 a.m.
This patch series adds support for configuring the PCI high memory MMIO
window size for aarch64 virt machine types using the highmem-mmio-size
feature introduced in QEMU v10.0.0 [1]. It allows users to configure the
size of the high memory MMIO window above 4GB, which can be required to
support PCI passthrough with devices that have a large BARs.
The feature is exposed through the existing pcihole64 element associated
with the PCIe root controller:
<controller type='pci' index='0' model='pcie-root'>
<pcihole64 unit='GiB'>512</pcihole64>
</controller>
This existing schema supports the same semantics for the QEMU PC
machine pci-hole64-size parameter and is a natural fit for supporting
the highmem-mmio-size feature on the aarch64 virt machine.
This series is applied over master and depends on the recently merged
patch [2] that added support for QEMU v10.0.0 aarch64 capabilities.
For your convenience, this series is also available on Github [3].
[1] https://github.com/qemu/qemu/commit/f10104aeae3a17f181d5bb37b7fd7dad7fe86cba
[2] https://github.com/nvmochs/libvirt/commit/cea2ee1d28780808172911e5c586478...
[3] git fetch https://github.com/nvmochs/libvirt.git pci_highmem_mmio_size_pcihole64
Signed-off-by: Matthew R. Ochs <mochs(a)nvidia.com>
Changelog:
v3
- Patch 2 updates:
+ Use positive check in qemuBuildGlobalControllerCommandLine
+ Only allow pcihole64 for PCIe root controllers with arm virt
- Added review tags from Daniel P. Berrangé for patch 1 and 3
v2
- Use existing XML scmhema (pcihole64 element) instead of a new one
Matthew R. Ochs (3):
qemu: Add capability for PCI high memory MMIO size
qemu: Add command line support for PCI high memory MMIO size
tests: Add pcihole64 test for virt machine
src/qemu/qemu_capabilities.c | 2 ++
src/qemu/qemu_capabilities.h | 1 +
src/qemu/qemu_command.c | 14 ++++++++-
src/qemu/qemu_validate.c | 4 ++-
.../caps_10.0.0_aarch64.xml | 1 +
.../pcihole64-virt.aarch64-latest.args | 31 +++++++++++++++++++
.../pcihole64-virt.aarch64-latest.xml | 29 +++++++++++++++++
tests/qemuxmlconfdata/pcihole64-virt.xml | 17 ++++++++++
tests/qemuxmlconftest.c | 1 +
9 files changed, 98 insertions(+), 2 deletions(-)
create mode 100644 tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.args
create mode 100644 tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.xml
create mode 100644 tests/qemuxmlconfdata/pcihole64-virt.xml
--
2.46.0
9:48 a.m.
New subject: [PATCH v3 1/3] qemu: Add capability for PCI high memory MMIO size
Add QEMU capability for PCI high memory MMIO size configuration:
- Add QEMU_CAPS_MACHINE_VIRT_HIGHMEM_MMIO_SIZE capability
- Add capability to virt machine properties
- Add highmem-mmio-size virt machine property to aarch64 qemu
10.0.0 capabilities
This allows detecting support for the highmem-mmio-size virt machine
property in QEMU.
Signed-off-by: Matthew R. Ochs <mochs(a)nvidia.com>
Reviewed-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
src/qemu/qemu_capabilities.c | 2 ++
src/qemu/qemu_capabilities.h | 1 +
tests/qemucapabilitiesdata/caps_10.0.0_aarch64.xml | 1 +
3 files changed, 4 insertions(+)
diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c
index c24584bf75fd..b600dd38b8fc 100644
--- a/src/qemu/qemu_capabilities.c
+++ b/src/qemu/qemu_capabilities.c
@@ -732,6 +732,7 @@ VIR_ENUM_IMPL(virQEMUCaps,
/* 475 */
"virtio-scsi.iothread-mapping", /*
QEMU_CAPS_VIRTIO_SCSI_IOTHREAD_MAPPING */
+ "machine.virt.highmem-mmio-size", /*
QEMU_CAPS_MACHINE_VIRT_HIGHMEM_MMIO_SIZE */
);
@@ -1771,6 +1772,7 @@ static struct virQEMUCapsStringFlags virQEMUCapsMachinePropsVirt[] =
{
{ "iommu", QEMU_CAPS_MACHINE_VIRT_IOMMU },
{ "ras", QEMU_CAPS_MACHINE_VIRT_RAS },
{ "aia", QEMU_CAPS_MACHINE_VIRT_AIA },
+ { "highmem-mmio-size", QEMU_CAPS_MACHINE_VIRT_HIGHMEM_MMIO_SIZE },
};
static struct virQEMUCapsStringFlags virQEMUCapsMachinePropsGeneric[] = {
diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h
index 40aa7acef42e..df33212a348a 100644
--- a/src/qemu/qemu_capabilities.h
+++ b/src/qemu/qemu_capabilities.h
@@ -713,6 +713,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for syntax-check
*/
/* 475 */
QEMU_CAPS_VIRTIO_SCSI_IOTHREAD_MAPPING, /* virtio-scsi supports per-virtqueue
iothread mapping */
+ QEMU_CAPS_MACHINE_VIRT_HIGHMEM_MMIO_SIZE, /* -machine
virt,highmem-mmio-size=<size> */
QEMU_CAPS_LAST /* this must always be the last item */
} virQEMUCapsFlags;
diff --git a/tests/qemucapabilitiesdata/caps_10.0.0_aarch64.xml
b/tests/qemucapabilitiesdata/caps_10.0.0_aarch64.xml
index e14e926c290d..7fd50185575a 100644
--- a/tests/qemucapabilitiesdata/caps_10.0.0_aarch64.xml
+++ b/tests/qemucapabilitiesdata/caps_10.0.0_aarch64.xml
@@ -160,6 +160,7 @@
<flag name='blockdev-set-active'/>
<flag name='shim'/>
<flag name='virtio-scsi.iothread-mapping'/>
+ <flag name='machine.virt.highmem-mmio-size'/>
<version>10000000</version>
<microcodeVersion>61700285</microcodeVersion>
<package>v10.0.0</package>
--
2.46.0
9:48 a.m.
New subject: [PATCH v3 2/3] qemu: Add command line support for PCI high memory MMIO size
Add support for generating QEMU command line with PCI high memory MMIO size:
- Add highmem-mmio-size to machine command line generation using
size conveyed through pcihole64
- Add validation for aarch64/virt machine type requirement
- Add capability check for QEMU support
This enables configuring the PCI high memory MMIO window size
for aarch64 virt machine types using the existing pcihole64
element.
Signed-off-by: Matthew R. Ochs <mochs(a)nvidia.com>
---
src/qemu/qemu_command.c | 14 +++++++++++++-
src/qemu/qemu_validate.c | 4 +++-
2 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index e6d308534f87..d1ebb4cdf9e7 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -6195,7 +6195,8 @@ qemuBuildGlobalControllerCommandLine(virCommand *cmd,
for (i = 0; i < def->ncontrollers; i++) {
virDomainControllerDef *cont = def->controllers[i];
if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_PCI &&
- cont->opts.pciopts.pcihole64) {
+ cont->opts.pciopts.pcihole64 &&
+ (qemuDomainIsQ35(def) || qemuDomainIsI440FX(def))) {
const char *hoststr = NULL;
switch (cont->model) {
@@ -7112,6 +7113,17 @@ qemuBuildMachineCommandLine(virCommand *cmd,
qemuBuildMachineACPI(&buf, def, qemuCaps);
+ if (qemuDomainIsARMVirt(def)) {
+ for (i = 0; i < def->ncontrollers; i++) {
+ virDomainControllerDef *cont = def->controllers[i];
+ if (cont->type == VIR_DOMAIN_CONTROLLER_TYPE_PCI &&
+ cont->opts.pciopts.pcihole64) {
+ virBufferAsprintf(&buf, ",highmem-mmio-size=%lluK",
cont->opts.pciopts.pcihole64size);
+ break;
+ }
+ }
+ }
+
virCommandAddArgBuffer(cmd, &buf);
return 0;
diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c
index b2c3c9e2f631..87588024ced4 100644
--- a/src/qemu/qemu_validate.c
+++ b/src/qemu/qemu_validate.c
@@ -4085,7 +4085,9 @@ qemuValidateDomainDeviceDefControllerPCI(const
virDomainControllerDef *cont,
case VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT:
if (pciopts->pcihole64 || pciopts->pcihole64size != 0) {
- if (!qemuDomainIsQ35(def)) {
+ if (!qemuDomainIsQ35(def) &&
+ !(qemuDomainIsARMVirt(def) && virQEMUCapsGet(qemuCaps,
+
QEMU_CAPS_MACHINE_VIRT_HIGHMEM_MMIO_SIZE))) {
virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
_("Setting the 64-bit PCI hole size is not supported
for machine '%1$s'"),
def->os.machine);
--
2.46.0
9:48 a.m.
New subject: [PATCH v3 3/3] tests: Add pcihole64 test for virt machine
Add a pcihole64 test for the aarch64 virt machine that verifies the
value is propagated to the highmem-mmio-size virt machine parameter.
Signed-off-by: Matthew R. Ochs <mochs(a)nvidia.com>
Reviewed-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
.../pcihole64-virt.aarch64-latest.args | 31 +++++++++++++++++++
.../pcihole64-virt.aarch64-latest.xml | 29 +++++++++++++++++
tests/qemuxmlconfdata/pcihole64-virt.xml | 17 ++++++++++
tests/qemuxmlconftest.c | 1 +
4 files changed, 78 insertions(+)
create mode 100644 tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.args
create mode 100644 tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.xml
create mode 100644 tests/qemuxmlconfdata/pcihole64-virt.xml
diff --git a/tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.args
b/tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.args
new file mode 100644
index 000000000000..d2df31ed868e
--- /dev/null
+++ b/tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.args
@@ -0,0 +1,31 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/var/lib/libvirt/qemu/domain--1-foo \
+USER=test \
+LOGNAME=test \
+XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-foo/.local/share \
+XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-foo/.cache \
+XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-foo/.config \
+/usr/bin/qemu-system-aarch64 \
+-name guest=foo,debug-threads=on \
+-S \
+-object
'{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-foo/master-key.aes"}'
\
+-machine
virt,usb=off,gic-version=2,dump-guest-core=off,memory-backend=mach-virt.ram,acpi=off,highmem-mmio-size=536870912K
\
+-accel tcg \
+-cpu cortex-a15 \
+-m size=1048576k \
+-object
'{"qom-type":"memory-backend-ram","id":"mach-virt.ram","size":1073741824}'
\
+-overcommit mem-lock=off \
+-smp 1,sockets=1,cores=1,threads=1 \
+-uuid 6ba7b810-9dad-11d1-80b4-00c04fd430c8 \
+-display none \
+-no-user-config \
+-nodefaults \
+-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \
+-mon chardev=charmonitor,id=monitor,mode=control \
+-rtc base=utc \
+-no-shutdown \
+-boot strict=on \
+-audiodev
'{"id":"audio1","driver":"none"}' \
+-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
+-msg timestamp=on
diff --git a/tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.xml
b/tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.xml
new file mode 100644
index 000000000000..1fb38b429e9f
--- /dev/null
+++ b/tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.xml
@@ -0,0 +1,29 @@
+<domain type='qemu'>
+ <name>foo</name>
+ <uuid>6ba7b810-9dad-11d1-80b4-00c04fd430c8</uuid>
+ <memory unit='KiB'>1048576</memory>
+ <currentMemory unit='KiB'>1048576</currentMemory>
+ <vcpu placement='static'>1</vcpu>
+ <os>
+ <type arch='aarch64' machine='virt'>hvm</type>
+ <boot dev='hd'/>
+ </os>
+ <features>
+ <gic version='2'/>
+ </features>
+ <cpu mode='custom' match='exact' check='none'>
+ <model fallback='forbid'>cortex-a15</model>
+ </cpu>
+ <clock offset='utc'/>
+ <on_poweroff>destroy</on_poweroff>
+ <on_reboot>restart</on_reboot>
+ <on_crash>destroy</on_crash>
+ <devices>
+ <emulator>/usr/bin/qemu-system-aarch64</emulator>
+ <controller type='pci' index='0' model='pcie-root'>
+ <pcihole64 unit='KiB'>536870912</pcihole64>
+ </controller>
+ <audio id='1' type='none'/>
+ <memballoon model='none'/>
+ </devices>
+</domain>
diff --git a/tests/qemuxmlconfdata/pcihole64-virt.xml
b/tests/qemuxmlconfdata/pcihole64-virt.xml
new file mode 100644
index 000000000000..00bb51ab0e9a
--- /dev/null
+++ b/tests/qemuxmlconfdata/pcihole64-virt.xml
@@ -0,0 +1,17 @@
+<domain type='qemu'>
+ <name>foo</name>
+ <uuid>6ba7b810-9dad-11d1-80b4-00c04fd430c8</uuid>
+ <memory unit='KiB'>1048576</memory>
+ <currentMemory unit='KiB'>1048576</currentMemory>
+ <vcpu placement='static'>1</vcpu>
+ <os>
+ <type arch='aarch64' machine='virt'>hvm</type>
+ </os>
+ <devices>
+ <emulator>/usr/bin/qemu-system-aarch64</emulator>
+ <controller type='pci' index='0' model='pcie-root'>
+ <pcihole64 unit='GiB'>512</pcihole64>
+ </controller>
+ <memballoon model='none'/>
+ </devices>
+</domain>
diff --git a/tests/qemuxmlconftest.c b/tests/qemuxmlconftest.c
index 1f31ec810c7a..7fe51ba4f81c 100644
--- a/tests/qemuxmlconftest.c
+++ b/tests/qemuxmlconftest.c
@@ -2587,6 +2587,7 @@ mymain(void)
DO_TEST_CAPS_LATEST("pcihole64");
DO_TEST_CAPS_LATEST("pcihole64-q35");
+ DO_TEST_CAPS_ARCH_LATEST("pcihole64-virt", "aarch64");
DO_TEST_CAPS_ARCH_LATEST("arm-vexpressa9-nodevs", "aarch64");
DO_TEST_CAPS_ARCH_LATEST("arm-vexpressa9-basic", "aarch64");
--
2.46.0
6:47 a.m.
New subject: [PATCH v3 0/3] Add virt machine support for configuring PCI high
memory MMIO size
I have pushed this series
On Thu, May 15, 2025 at 07:48:07AM -0700, Matthew R. Ochs wrote:
This patch series adds support for configuring the PCI high memory
MMIO
window size for aarch64 virt machine types using the highmem-mmio-size
feature introduced in QEMU v10.0.0 [1]. It allows users to configure the
size of the high memory MMIO window above 4GB, which can be required to
support PCI passthrough with devices that have a large BARs.
The feature is exposed through the existing pcihole64 element associated
with the PCIe root controller:
<controller type='pci' index='0' model='pcie-root'>
<pcihole64 unit='GiB'>512</pcihole64>
</controller>
This existing schema supports the same semantics for the QEMU PC
machine pci-hole64-size parameter and is a natural fit for supporting
the highmem-mmio-size feature on the aarch64 virt machine.
This series is applied over master and depends on the recently merged
patch [2] that added support for QEMU v10.0.0 aarch64 capabilities.
For your convenience, this series is also available on Github [3].
[1] https://github.com/qemu/qemu/commit/f10104aeae3a17f181d5bb37b7fd7dad7fe86cba
[2] https://github.com/nvmochs/libvirt/commit/cea2ee1d28780808172911e5c586478...
[3] git fetch https://github.com/nvmochs/libvirt.git pci_highmem_mmio_size_pcihole64
Signed-off-by: Matthew R. Ochs <mochs(a)nvidia.com>
Changelog:
v3
- Patch 2 updates:
+ Use positive check in qemuBuildGlobalControllerCommandLine
+ Only allow pcihole64 for PCIe root controllers with arm virt
- Added review tags from Daniel P. Berrangé for patch 1 and 3
v2
- Use existing XML scmhema (pcihole64 element) instead of a new one
Matthew R. Ochs (3):
qemu: Add capability for PCI high memory MMIO size
qemu: Add command line support for PCI high memory MMIO size
tests: Add pcihole64 test for virt machine
src/qemu/qemu_capabilities.c | 2 ++
src/qemu/qemu_capabilities.h | 1 +
src/qemu/qemu_command.c | 14 ++++++++-
src/qemu/qemu_validate.c | 4 ++-
.../caps_10.0.0_aarch64.xml | 1 +
.../pcihole64-virt.aarch64-latest.args | 31 +++++++++++++++++++
.../pcihole64-virt.aarch64-latest.xml | 29 +++++++++++++++++
tests/qemuxmlconfdata/pcihole64-virt.xml | 17 ++++++++++
tests/qemuxmlconftest.c | 1 +
9 files changed, 98 insertions(+), 2 deletions(-)
create mode 100644 tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.args
create mode 100644 tests/qemuxmlconfdata/pcihole64-virt.aarch64-latest.xml
create mode 100644 tests/qemuxmlconfdata/pcihole64-virt.xml
--
2.46.0
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
17
days inactive
18
days old
5 comments
3 participants
participants (3)
-
Daniel P. Berrangé -
Matt Ochs -
Matthew R. Ochs