[libvirt] [PATCH v7 00/14] PCI passthrough support on s390
Abstract ======== The PCI representation in QEMU has been extended for S390 allowing configuration of zPCI attributes like uid (user-defined identifier) and fid (PCI function identifier). The details can be found here: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg07262.html To support the new zPCI feature of the S390 platform, a new element of PCI address is introduced. It has two optional attributes, @uid and @fid. For example: <hostdev mode='subsystem' type='pci'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x00' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> <zpci uid='0x0003' fid='0x00000027'/> </address> </hostdev> If they are defined by the user, unique values within the guest domain must be used. If they are not specified and the architecture requires them, they are automatically generated with non-conflicting values. zPCI address as an extension of the PCI address are stored in a new structure 'virZPCIDeviceAddress' which is a member of common PCI Address structure. Additionally, two hashtables are used for assignment and reservation of zPCI uid/fid. In support of extending the PCI address, a new PCI address extension flag is introduced. This extension flag allows is not only dedicated for the S390 platform but also other architectures needing certain extensions to PCI address space. Code Base ========= commit in master: 3a1cdb06fd conf: Fix bug in finding alloc through matching vcpus Change Log ========== v6->v7: 1. Optimize some functions' names and code logic. 2. Fixup build error. 3. Add negative test case for patch 9. 4. Use virXMLFormatElement() in virDomainDeviceInfoFormat(). v5->v6: 1. Modify zPCI XML definition. 2. Optimize the logic of zPCI address assignment and reservation. 3. Add extension flag into PCI address structure. 4. Update commit messages. v4->v5: 1. Update the version number. 2. Fixup code style error. 3. Separate qemu code into single patch. 4. Rebase the patches to the new code of master branch. v3->v4: 1. Update docs. 2. Format code style. 3. Optimize zPCI support check. 4. Move the check of zPCI defined in xml but unsupported by Qemu to qemuDomainDeviceDefValidate(). 5. Change zpci address member of PCI address struct from pointer to instance. 6. Modify zpci address definition principle. Currently the user must either define both of uid and fid or not. v2->v3: 1. Revise code style. 2. Update test cases. 3. Introduce qemuDomainCollectPCIAddressExtension() to collect PCI extension addresses. 4. Introduce virDeviceInfoPCIAddressExtensionPresent() to check if zPCI address exists. 5. Optimize zPCI address check logic. 6. Optimize passed parameters of zPCI addr alloc/release/reserve functions. 7. Report enum range error in qemuDomainDeviceSupportZPCI(). 8. Update commit messages. v1->v2: 1. Separate test commit and merge testcases into corresponding commits that introduce the functionalities firstly. 2. Spare some checks for zpci device. 3. Add vsock and controller support. 4. Add uin32 type schema. 5. Rename zpciuid and zpcifid to zpci_uid and zpci_fid. 6. Always return multibus support on S390. Yi Min Zhao (14): conf: Add definitions for 'uid' and 'fid' PCI address attributes qemu: Introduce zPCI capability conf: Introduce a new PCI address extension flag qemu: Enable PCI multi bus for S390 guests qemu: Auto add pci-root for s390/s390x guests conf: Introduce address caching for PCI extensions conf: use virXMLFormatElement() in virDomainDeviceInfoFormat() conf: Introduce parser, formatter for uid and fid qemu: Add zPCI address definition check conf: Allocate/release 'uid' and 'fid' in PCI address qemu: Generate and use zPCI device in QEMU command line qemu: Add hotpluging support for PCI devices on S390 guests docs: Add 'uid' and 'fid' information news: Update news for PCI address extension attributes cfg.mk | 1 + docs/formatdomain.html.in | 10 +- docs/news.xml | 11 + docs/schemas/basictypes.rng | 27 ++ docs/schemas/domaincommon.rng | 1 + src/bhyve/bhyve_device.c | 3 +- src/conf/device_conf.c | 69 ++++ src/conf/device_conf.h | 7 + src/conf/domain_addr.c | 340 +++++++++++++++++- src/conf/domain_addr.h | 27 +- src/conf/domain_conf.c | 50 ++- src/libvirt_private.syms | 7 + src/qemu/qemu_capabilities.c | 6 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 104 ++++++ src/qemu/qemu_command.h | 2 + src/qemu/qemu_domain.c | 37 ++ src/qemu/qemu_domain_address.c | 205 ++++++++++- src/qemu/qemu_hotplug.c | 160 ++++++++- src/util/virpci.c | 26 ++ src/util/virpci.h | 15 + .../caps_2.10.0.s390x.xml | 1 + .../caps_2.11.0.s390x.xml | 1 + .../caps_2.12.0.s390x.xml | 1 + .../qemucapabilitiesdata/caps_2.7.0.s390x.xml | 1 + .../qemucapabilitiesdata/caps_2.8.0.s390x.xml | 1 + .../qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + .../qemucapabilitiesdata/caps_3.0.0.s390x.xml | 1 + .../disk-virtio-s390-zpci.args | 26 ++ .../disk-virtio-s390-zpci.xml | 19 + .../hostdev-vfio-zpci-autogenerate.args | 25 ++ .../hostdev-vfio-zpci-autogenerate.xml | 18 + .../hostdev-vfio-zpci-boundaries.args | 29 ++ .../hostdev-vfio-zpci-boundaries.xml | 30 ++ .../hostdev-vfio-zpci-multidomain-many.args | 39 ++ .../hostdev-vfio-zpci-multidomain-many.xml | 79 ++++ .../hostdev-vfio-zpci-wrong-arch.xml | 34 ++ tests/qemuxml2argvdata/hostdev-vfio-zpci.args | 25 ++ tests/qemuxml2argvdata/hostdev-vfio-zpci.xml | 21 ++ tests/qemuxml2argvtest.c | 22 ++ .../disk-virtio-s390-zpci.xml | 31 ++ .../hostdev-vfio-zpci-autogenerate.xml | 34 ++ .../hostdev-vfio-zpci-boundaries.xml | 48 +++ .../hostdev-vfio-zpci-multidomain-many.xml | 97 +++++ .../qemuxml2xmloutdata/hostdev-vfio-zpci.xml | 32 ++ tests/qemuxml2xmltest.c | 17 + 46 files changed, 1707 insertions(+), 35 deletions(-) create mode 100644 tests/qemuxml2argvdata/disk-virtio-s390-zpci.args create mode 100644 tests/qemuxml2argvdata/disk-virtio-s390-zpci.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-wrong-arch.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci.xml create mode 100644 tests/qemuxml2xmloutdata/disk-virtio-s390-zpci.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci-autogenerate.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci-boundaries.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci-multidomain-many.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci.xml -- Yi Min
Add zPCI definitions in preparation of extending the PCI address with parameters uid (user-defined identifier) and fid (PCI function identifier). Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- cfg.mk | 1 + src/util/virpci.h | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/cfg.mk b/cfg.mk index 4790d0b7e7..1fc6a2dabb 100644 --- a/cfg.mk +++ b/cfg.mk @@ -472,6 +472,7 @@ sc_prohibit_canonicalize_file_name: # Insist on correct types for [pug]id. sc_correct_id_types: @prohibit='\<(int|long) *[pug]id\>' \ + exclude='exempt from syntax-check' \ halt='use pid_t for pid, uid_t for uid, gid_t for gid' \ $(_sc_search_regexp) diff --git a/src/util/virpci.h b/src/util/virpci.h index 16c2eded5e..4cc9298d85 100644 --- a/src/util/virpci.h +++ b/src/util/virpci.h @@ -37,12 +37,20 @@ typedef virPCIDeviceAddress *virPCIDeviceAddressPtr; typedef struct _virPCIDeviceList virPCIDeviceList; typedef virPCIDeviceList *virPCIDeviceListPtr; +typedef struct _virZPCIDeviceAddress virZPCIDeviceAddress; +typedef virZPCIDeviceAddress *virZPCIDeviceAddressPtr; +struct _virZPCIDeviceAddress { + unsigned int uid; /* exempt from syntax-check */ + unsigned int fid; +}; + struct _virPCIDeviceAddress { unsigned int domain; unsigned int bus; unsigned int slot; unsigned int function; int multi; /* virTristateSwitch */ + virZPCIDeviceAddress zpci; }; typedef enum { -- Yi Min
On Fri, 2018-10-19 at 11:40 +0800, Yi Min Zhao wrote:
struct _virPCIDeviceAddress { unsigned int domain; unsigned int bus; unsigned int slot; unsigned int function; int multi; /* virTristateSwitch */ + virZPCIDeviceAddress zpci;
Laine pointed out that it's weird that 'zpci' is not added in the same patch as 'extFlags', and more specifically that the former is introduced before the latter. I believe I pointed out the same thing at some point, or at least I though about doing so :) Now that it's two people noticing the same, I guess it would be nice to address it. Shouldn't be hard at all... Can you please take care of it? -- Andrea Bolognani / Red Hat / Virtualization
Let's introduce zPCI capability. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_3.0.0.s390x.xml | 1 + 9 files changed, 10 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index e228f52ec0..ee6a7462f6 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -508,6 +508,7 @@ VIR_ENUM_IMPL(virQEMUCaps, QEMU_CAPS_LAST, /* 315 */ "vfio-pci.display", "blockdev", + "zpci", ); @@ -1092,6 +1093,7 @@ struct virQEMUCapsStringFlags virQEMUCapsObjectTypes[] = { { "vhost-vsock-device", QEMU_CAPS_DEVICE_VHOST_VSOCK }, { "mch", QEMU_CAPS_DEVICE_MCH }, { "sev-guest", QEMU_CAPS_SEV_GUEST }, + { "zpci", QEMU_CAPS_DEVICE_ZPCI }, }; static struct virQEMUCapsStringFlags virQEMUCapsDevicePropsVirtioBalloon[] = { diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index 934620ed31..cd417c9062 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -492,6 +492,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for syntax-check */ /* 315 */ QEMU_CAPS_VFIO_PCI_DISPLAY, /* -device vfio-pci.display */ QEMU_CAPS_BLOCKDEV, /* -blockdev and blockdev-add are supported */ + QEMU_CAPS_DEVICE_ZPCI, /* -device zpci */ QEMU_CAPS_LAST /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml b/tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml index e000aac384..3c311042f3 100644 --- a/tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml @@ -113,6 +113,7 @@ <flag name='blockdev-del'/> <flag name='vhost-vsock'/> <flag name='egl-headless'/> + <flag name='zpci'/> <version>2010000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>306247</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml b/tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml index 4eb8a39d94..48db1dbf2d 100644 --- a/tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml @@ -120,6 +120,7 @@ <flag name='vhost-vsock'/> <flag name='tpm-emulator'/> <flag name='egl-headless'/> + <flag name='zpci'/> <version>2011000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>345099</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml b/tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml index 79320d5229..4c561f6214 100644 --- a/tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml @@ -128,6 +128,7 @@ <flag name='tpm-emulator'/> <flag name='egl-headless'/> <flag name='vfio-pci.display'/> + <flag name='zpci'/> <version>2012000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>374287</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml b/tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml index b30c31cafc..de87692857 100644 --- a/tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml @@ -100,6 +100,7 @@ <flag name='nbd-tls'/> <flag name='virtual-css-bridge'/> <flag name='sdl-gl'/> + <flag name='zpci'/> <version>2007000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>219140</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml b/tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml index b010f731a5..f3a32ad376 100644 --- a/tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml @@ -103,6 +103,7 @@ <flag name='virtual-css-bridge'/> <flag name='sdl-gl'/> <flag name='vhost-vsock'/> + <flag name='zpci'/> <version>2007093</version> <kvmVersion>0</kvmVersion> <microcodeVersion>244554</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml b/tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml index 5a4371ab83..f1e05ab1c4 100644 --- a/tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml @@ -107,6 +107,7 @@ <flag name='sdl-gl'/> <flag name='blockdev-del'/> <flag name='vhost-vsock'/> + <flag name='zpci'/> <version>2009000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>267973</microcodeVersion> diff --git a/tests/qemucapabilitiesdata/caps_3.0.0.s390x.xml b/tests/qemucapabilitiesdata/caps_3.0.0.s390x.xml index 3b5f9818a5..c841030b2b 100644 --- a/tests/qemucapabilitiesdata/caps_3.0.0.s390x.xml +++ b/tests/qemucapabilitiesdata/caps_3.0.0.s390x.xml @@ -130,6 +130,7 @@ <flag name='tpm-emulator'/> <flag name='egl-headless'/> <flag name='vfio-pci.display'/> + <flag name='zpci'/> <version>3000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>387601</microcodeVersion> -- Yi Min
This patch introduces PCI address extension flag for virDomainDeviceInfo and virPCIDeviceAddress. The extension flag in virDomainDeviceInfo is used internally during calculating PCI extension flag. The one in virPCIDeviceAddress is the duplicate to indicate extension address is being used. Currently only zPCI extension address is introduced to deal with 'uid' and 'fid' on the S390 platform. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- src/conf/device_conf.h | 4 + src/conf/domain_addr.h | 5 ++ src/qemu/qemu_domain_address.c | 140 ++++++++++++++++++++++++++++++++- src/util/virpci.h | 1 + 4 files changed, 148 insertions(+), 2 deletions(-) diff --git a/src/conf/device_conf.h b/src/conf/device_conf.h index 407956bd02..2366e03607 100644 --- a/src/conf/device_conf.h +++ b/src/conf/device_conf.h @@ -166,6 +166,10 @@ struct _virDomainDeviceInfo { * assignment, never saved and never reported. */ int pciConnectFlags; /* enum virDomainPCIConnectFlags */ + /* pciAddrExtFlags is only used internally to calculate PCI + * address extension flags during address assignment. + */ + int pciAddrExtFlags; /* enum virDomainPCIAddressExtensionFlags */ char *loadparm; /* PCI devices will only be automatically placed on a PCI bus diff --git a/src/conf/domain_addr.h b/src/conf/domain_addr.h index 2a9af9c00b..435b3c5d7f 100644 --- a/src/conf/domain_addr.h +++ b/src/conf/domain_addr.h @@ -29,6 +29,11 @@ # define VIR_PCI_ADDRESS_SLOT_LAST 31 # define VIR_PCI_ADDRESS_FUNCTION_LAST 7 +typedef enum { + VIR_PCI_ADDRESS_EXTENSION_NONE = 0, /* no extension */ + VIR_PCI_ADDRESS_EXTENSION_ZPCI = 1 << 0, /* zPCI support */ +} virPCIDeviceAddressExtensionFlags; + typedef enum { VIR_PCI_CONNECT_HOTPLUGGABLE = 1 << 0, /* is hotplug needed/supported */ diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 8a8764cff5..c0374ccaf3 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -509,6 +509,64 @@ qemuDomainAssignVirtioMMIOAddresses(virDomainDefPtr def, } +static bool +qemuDomainDeviceSupportZPCI(virDomainDeviceDefPtr device) +{ + switch ((virDomainDeviceType)device->type) { + case VIR_DOMAIN_DEVICE_CHR: + return false; + + case VIR_DOMAIN_DEVICE_CONTROLLER: + case VIR_DOMAIN_DEVICE_DISK: + case VIR_DOMAIN_DEVICE_LEASE: + case VIR_DOMAIN_DEVICE_FS: + case VIR_DOMAIN_DEVICE_NET: + case VIR_DOMAIN_DEVICE_INPUT: + case VIR_DOMAIN_DEVICE_SOUND: + case VIR_DOMAIN_DEVICE_VIDEO: + case VIR_DOMAIN_DEVICE_HOSTDEV: + case VIR_DOMAIN_DEVICE_WATCHDOG: + case VIR_DOMAIN_DEVICE_GRAPHICS: + case VIR_DOMAIN_DEVICE_HUB: + case VIR_DOMAIN_DEVICE_REDIRDEV: + case VIR_DOMAIN_DEVICE_SMARTCARD: + case VIR_DOMAIN_DEVICE_MEMBALLOON: + case VIR_DOMAIN_DEVICE_NVRAM: + case VIR_DOMAIN_DEVICE_RNG: + case VIR_DOMAIN_DEVICE_SHMEM: + case VIR_DOMAIN_DEVICE_TPM: + case VIR_DOMAIN_DEVICE_PANIC: + case VIR_DOMAIN_DEVICE_MEMORY: + case VIR_DOMAIN_DEVICE_IOMMU: + case VIR_DOMAIN_DEVICE_VSOCK: + break; + + case VIR_DOMAIN_DEVICE_NONE: + case VIR_DOMAIN_DEVICE_LAST: + default: + virReportEnumRangeError(virDomainDeviceType, device->type); + return false; + } + + return true; +} + + +static virPCIDeviceAddressExtensionFlags +qemuDomainDeviceCalculatePCIAddressExtensionFlags(virQEMUCapsPtr qemuCaps, + virDomainDeviceDefPtr dev) +{ + virPCIDeviceAddressExtensionFlags extFlags = 0; + + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_ZPCI) && + qemuDomainDeviceSupportZPCI(dev)) { + extFlags |= VIR_PCI_ADDRESS_EXTENSION_ZPCI; + } + + return extFlags; +} + + /** * qemuDomainDeviceCalculatePCIConnectFlags: * @@ -990,6 +1048,56 @@ qemuDomainFillAllPCIConnectFlags(virDomainDefPtr def, } +/** + * qemuDomainFillDevicePCIExtensionFlagsIter: + * + * @def: the entire DomainDef + * @dev: The device to be checked + * @info: virDomainDeviceInfo within the device + * @opaque: qemu capabilities + * + * Sets the pciAddressExtFlags for a single device's info. Has properly + * formatted arguments to be called by virDomainDeviceInfoIterate(). + * + * Always returns 0 - there is no failure. + */ +static int +qemuDomainFillDevicePCIExtensionFlagsIter(virDomainDefPtr def ATTRIBUTE_UNUSED, + virDomainDeviceDefPtr dev, + virDomainDeviceInfoPtr info, + void *opaque) +{ + virQEMUCapsPtr qemuCaps = opaque; + + info->pciAddrExtFlags = + qemuDomainDeviceCalculatePCIAddressExtensionFlags(qemuCaps, dev); + + return 0; +} + + +/** + * qemuDomainFillAllPCIExtensionFlags: + * + * @def: the entire DomainDef + * @qemuCaps: as you'd expect + * + * Set the info->pciAddressExtFlags for all devices in the domain. + * + * Returns 0 on success or -1 on failure (the only possibility of + * failure would be some internal problem with + * virDomainDeviceInfoIterate()) + */ +static int +qemuDomainFillAllPCIExtensionFlags(virDomainDefPtr def, + virQEMUCapsPtr qemuCaps) +{ + return virDomainDeviceInfoIterate(def, + qemuDomainFillDevicePCIExtensionFlagsIter, + qemuCaps); +} + + /** * qemuDomainFindUnusedIsolationGroupIter: * @def: domain definition @@ -1264,6 +1372,27 @@ qemuDomainFillDevicePCIConnectFlags(virDomainDefPtr def, } +/** + * qemuDomainFillDevicePCIExtensionFlags: + * + * @dev: The device to be checked + * @info: virDomainDeviceInfo within the device + * @qemuCaps: as you'd expect + * + * Set the info->pciAddressExtFlags for a single device. + * + * No return value. + */ +static void +qemuDomainFillDevicePCIExtensionFlags(virDomainDeviceDefPtr dev, + virDomainDeviceInfoPtr info, + virQEMUCapsPtr qemuCaps) +{ + info->pciAddrExtFlags = + qemuDomainDeviceCalculatePCIAddressExtensionFlags(qemuCaps, dev); +} + + static int qemuDomainPCIAddressReserveNextAddr(virDomainPCIAddressSetPtr addrs, virDomainDeviceInfoPtr dev) @@ -2400,6 +2529,9 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, if (qemuDomainFillAllPCIConnectFlags(def, qemuCaps, driver) < 0) goto cleanup; + if (qemuDomainFillAllPCIExtensionFlags(def, qemuCaps) < 0) + goto cleanup; + if (qemuDomainSetupIsolationGroups(def) < 0) goto cleanup; @@ -2435,7 +2567,8 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, */ virDomainDeviceInfo info = { .pciConnectFlags = (VIR_PCI_CONNECT_HOTPLUGGABLE | - VIR_PCI_CONNECT_TYPE_PCI_DEVICE) + VIR_PCI_CONNECT_TYPE_PCI_DEVICE), + .pciAddrExtFlags = VIR_PCI_ADDRESS_EXTENSION_NONE }; bool buses_reserved = true; @@ -2472,7 +2605,8 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, qemuDomainHasPCIeRoot(def)) { virDomainDeviceInfo info = { .pciConnectFlags = (VIR_PCI_CONNECT_HOTPLUGGABLE | - VIR_PCI_CONNECT_TYPE_PCIE_DEVICE) + VIR_PCI_CONNECT_TYPE_PCIE_DEVICE), + .pciAddrExtFlags = VIR_PCI_ADDRESS_EXTENSION_NONE }; /* if there isn't an empty pcie-root-port, this will @@ -2989,6 +3123,8 @@ qemuDomainEnsurePCIAddress(virDomainObjPtr obj, qemuDomainFillDevicePCIConnectFlags(obj->def, dev, priv->qemuCaps, driver); + qemuDomainFillDevicePCIExtensionFlags(dev, info, priv->qemuCaps); + return virDomainPCIAddressEnsureAddr(priv->pciaddrs, info, info->pciConnectFlags); } diff --git a/src/util/virpci.h b/src/util/virpci.h index 4cc9298d85..0e40d86b97 100644 --- a/src/util/virpci.h +++ b/src/util/virpci.h @@ -50,6 +50,7 @@ struct _virPCIDeviceAddress { unsigned int slot; unsigned int function; int multi; /* virTristateSwitch */ + int extFlags; /* enum virPCIDeviceAddressExtensionFlags */ virZPCIDeviceAddress zpci; }; -- Yi Min
QEMU on s390 supports PCI multibus since forever. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- src/qemu/qemu_capabilities.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index ee6a7462f6..bf1b90875b 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -1700,6 +1700,10 @@ bool virQEMUCapsHasPCIMultiBus(virQEMUCapsPtr qemuCaps, return false; } + /* S390 supports PCI-multibus. */ + if (ARCH_IS_S390(def->os.arch)) + return true; + /* If ARM 'virt' supports PCI, it supports multibus. * No extra conditions here for simplicity. */ -- Yi Min
The pci-root depends on zpci capability. So autogenerate pci-root if zpci exists. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- src/qemu/qemu_domain.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index f00f1b3fdb..961755ef84 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -3287,6 +3287,7 @@ qemuDomainDefAddDefaultDevices(virDomainDefPtr def, case VIR_ARCH_S390X: addDefaultUSB = false; addPanicDevice = true; + addPCIRoot = virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_ZPCI); break; case VIR_ARCH_SPARC: -- Yi Min
This patch provides a caching mechanism for the device address extensions uid and fid on S390. For efficient sparse address allocation, we introduce two hash tables for uid/fid which hold the address set information per domain. Also in order to improve performance of searching available value, we introduce our own callbacks for the two hashtables. In this way, uid/fid is saved in hash key and hash value could be any non-NULL pointer due to no operation on hash value. That is also the reason why we don't introduce hash value free callback. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- src/bhyve/bhyve_device.c | 3 +- src/conf/domain_addr.c | 93 +++++++++++++++++++++++++++++++++- src/conf/domain_addr.h | 10 +++- src/qemu/qemu_domain_address.c | 6 ++- 4 files changed, 108 insertions(+), 4 deletions(-) diff --git a/src/bhyve/bhyve_device.c b/src/bhyve/bhyve_device.c index 207ac6a2dd..a59dfe4519 100644 --- a/src/bhyve/bhyve_device.c +++ b/src/bhyve/bhyve_device.c @@ -71,7 +71,8 @@ bhyveDomainPCIAddressSetCreate(virDomainDefPtr def, unsigned int nbuses) { virDomainPCIAddressSetPtr addrs; - if ((addrs = virDomainPCIAddressSetAlloc(nbuses)) == NULL) + if ((addrs = virDomainPCIAddressSetAlloc(nbuses, + VIR_PCI_ADDRESS_EXTENSION_NONE)) == NULL) return NULL; if (virDomainPCIAddressBusSetModel(&addrs->buses[0], diff --git a/src/conf/domain_addr.c b/src/conf/domain_addr.c index 442e6aab94..902837d4ee 100644 --- a/src/conf/domain_addr.c +++ b/src/conf/domain_addr.c @@ -27,6 +27,7 @@ #include "virlog.h" #include "virstring.h" #include "domain_addr.h" +#include "virhashcode.h" #define VIR_FROM_THIS VIR_FROM_DOMAIN @@ -727,8 +728,93 @@ virDomainPCIAddressReleaseAddr(virDomainPCIAddressSetPtr addrs, addrs->buses[addr->bus].slot[addr->slot].functions &= ~(1 << addr->function); } + +static uint32_t +virZPCIAddrKeyCode(const void *name, + uint32_t seed) +{ + unsigned int value = *((unsigned int *)name); + return virHashCodeGen(&value, sizeof(value), seed); +} + + +static bool +virZPCIAddrKeyEqual(const void *namea, + const void *nameb) +{ + return *((unsigned int *)namea) == *((unsigned int *)nameb); +} + + +static void * +virZPCIAddrKeyCopy(const void *name) +{ + unsigned int *copy; + + if (VIR_ALLOC(copy) < 0) + return NULL; + + *copy = *((unsigned int *)name); + return (void *)copy; +} + + +static void +virZPCIAddrKeyFree(void *name) +{ + VIR_FREE(name); +} + + +static void +virDomainPCIAddressSetExtensionFree(virDomainPCIAddressSetPtr addrs) +{ + if (!addrs || !addrs->zpciIds) + return; + + virHashFree(addrs->zpciIds->uids); + virHashFree(addrs->zpciIds->fids); + VIR_FREE(addrs->zpciIds); +} + + +static int +virDomainPCIAddressSetExtensionAlloc(virDomainPCIAddressSetPtr addrs, + virPCIDeviceAddressExtensionFlags extFlags) +{ + if (extFlags & VIR_PCI_ADDRESS_EXTENSION_ZPCI) { + if (addrs->zpciIds) + return 0; + + if (VIR_ALLOC(addrs->zpciIds) < 0) + return -1; + + if (!(addrs->zpciIds->uids = virHashCreateFull(10, NULL, + virZPCIAddrKeyCode, + virZPCIAddrKeyEqual, + virZPCIAddrKeyCopy, + virZPCIAddrKeyFree))) + goto error; + + if (!(addrs->zpciIds->fids = virHashCreateFull(10, NULL, + virZPCIAddrKeyCode, + virZPCIAddrKeyEqual, + virZPCIAddrKeyCopy, + virZPCIAddrKeyFree))) + goto error; + } + + return 0; + + error: + virDomainPCIAddressSetExtensionFree(addrs); + return -1; +} + + virDomainPCIAddressSetPtr -virDomainPCIAddressSetAlloc(unsigned int nbuses) +virDomainPCIAddressSetAlloc(unsigned int nbuses, + virPCIDeviceAddressExtensionFlags extFlags) { virDomainPCIAddressSetPtr addrs; @@ -739,6 +825,10 @@ virDomainPCIAddressSetAlloc(unsigned int nbuses) goto error; addrs->nbuses = nbuses; + + if (virDomainPCIAddressSetExtensionAlloc(addrs, extFlags) < 0) + goto error; + return addrs; error: @@ -753,6 +843,7 @@ virDomainPCIAddressSetFree(virDomainPCIAddressSetPtr addrs) if (!addrs) return; + virDomainPCIAddressSetExtensionFree(addrs); VIR_FREE(addrs->buses); VIR_FREE(addrs); } diff --git a/src/conf/domain_addr.h b/src/conf/domain_addr.h index 435b3c5d7f..8d64d6b795 100644 --- a/src/conf/domain_addr.h +++ b/src/conf/domain_addr.h @@ -116,6 +116,12 @@ typedef struct { } virDomainPCIAddressBus; typedef virDomainPCIAddressBus *virDomainPCIAddressBusPtr; +typedef struct { + virHashTablePtr uids; + virHashTablePtr fids; +} virDomainZPCIAddressIds; +typedef virDomainZPCIAddressIds *virDomainZPCIAddressIdsPtr; + struct _virDomainPCIAddressSet { virDomainPCIAddressBus *buses; size_t nbuses; @@ -125,11 +131,13 @@ struct _virDomainPCIAddressSet { bool areMultipleRootsSupported; /* If true, the guest can use the pcie-to-pci-bridge controller */ bool isPCIeToPCIBridgeSupported; + virDomainZPCIAddressIdsPtr zpciIds; }; typedef struct _virDomainPCIAddressSet virDomainPCIAddressSet; typedef virDomainPCIAddressSet *virDomainPCIAddressSetPtr; -virDomainPCIAddressSetPtr virDomainPCIAddressSetAlloc(unsigned int nbuses); +virDomainPCIAddressSetPtr virDomainPCIAddressSetAlloc(unsigned int nbuses, + virPCIDeviceAddressExtensionFlags extFlags); void virDomainPCIAddressSetFree(virDomainPCIAddressSetPtr addrs); diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index c0374ccaf3..665f6c1445 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -1505,8 +1505,12 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, size_t i; bool hasPCIeRoot = false; virDomainControllerModelPCI defaultModel; + virPCIDeviceAddressExtensionFlags extFlags = VIR_PCI_ADDRESS_EXTENSION_NONE; - if ((addrs = virDomainPCIAddressSetAlloc(nbuses)) == NULL) + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_ZPCI)) + extFlags |= VIR_PCI_ADDRESS_EXTENSION_ZPCI; + + if ((addrs = virDomainPCIAddressSetAlloc(nbuses, extFlags)) == NULL) return NULL; addrs->dryRun = dryRun; -- Yi Min
In order to add zPCI child element for PCI address, we update virDomainDeviceInfoFormat() to format device info by helper function virXMLFormatElement(). Then we could simply format zPCI address into child buffer later. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> --- src/conf/domain_conf.c | 40 ++++++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index e8e0adc819..88964f10c6 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -6400,6 +6400,8 @@ virDomainDeviceInfoFormat(virBufferPtr buf, virDomainDeviceInfoPtr info, unsigned int flags) { + virBuffer attrBuf = VIR_BUFFER_INITIALIZER; + if ((flags & VIR_DOMAIN_DEF_FORMAT_ALLOW_BOOT) && info->bootIndex) { virBufferAsprintf(buf, "<boot order='%u'", info->bootIndex); @@ -6444,13 +6446,13 @@ virDomainDeviceInfoFormat(virBufferPtr buf, info->type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_VIRTIO_S390) return; - virBufferAsprintf(buf, "<address type='%s'", + virBufferAsprintf(&attrBuf, " type='%s'", virDomainDeviceAddressTypeToString(info->type)); switch ((virDomainDeviceAddressType) info->type) { case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI: if (!virPCIDeviceAddressIsEmpty(&info->addr.pci)) { - virBufferAsprintf(buf, " domain='0x%.4x' bus='0x%.2x' " + virBufferAsprintf(&attrBuf, " domain='0x%.4x' bus='0x%.2x' " "slot='0x%.2x' function='0x%.1x'", info->addr.pci.domain, info->addr.pci.bus, @@ -6458,13 +6460,13 @@ virDomainDeviceInfoFormat(virBufferPtr buf, info->addr.pci.function); } if (info->addr.pci.multi) { - virBufferAsprintf(buf, " multifunction='%s'", - virTristateSwitchTypeToString(info->addr.pci.multi)); + virBufferAsprintf(&attrBuf, " multifunction='%s'", + virTristateSwitchTypeToString(info->addr.pci.multi)); } break; case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DRIVE: - virBufferAsprintf(buf, " controller='%d' bus='%d' target='%d' unit='%d'", + virBufferAsprintf(&attrBuf, " controller='%d' bus='%d' target='%d' unit='%d'", info->addr.drive.controller, info->addr.drive.bus, info->addr.drive.target, @@ -6472,34 +6474,34 @@ virDomainDeviceInfoFormat(virBufferPtr buf, break; case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_VIRTIO_SERIAL: - virBufferAsprintf(buf, " controller='%d' bus='%d' port='%d'", + virBufferAsprintf(&attrBuf, " controller='%d' bus='%d' port='%d'", info->addr.vioserial.controller, info->addr.vioserial.bus, info->addr.vioserial.port); break; case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCID: - virBufferAsprintf(buf, " controller='%d' slot='%d'", + virBufferAsprintf(&attrBuf, " controller='%d' slot='%d'", info->addr.ccid.controller, info->addr.ccid.slot); break; case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_USB: - virBufferAsprintf(buf, " bus='%d'", info->addr.usb.bus); + virBufferAsprintf(&attrBuf, " bus='%d'", info->addr.usb.bus); if (virDomainUSBAddressPortIsValid(info->addr.usb.port)) { - virBufferAddLit(buf, " port='"); - virDomainUSBAddressPortFormatBuf(buf, info->addr.usb.port); - virBufferAddLit(buf, "'"); + virBufferAddLit(&attrBuf, " port='"); + virDomainUSBAddressPortFormatBuf(&attrBuf, info->addr.usb.port); + virBufferAddLit(&attrBuf, "'"); } break; case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_SPAPRVIO: if (info->addr.spaprvio.has_reg) - virBufferAsprintf(buf, " reg='0x%llx'", info->addr.spaprvio.reg); + virBufferAsprintf(&attrBuf, " reg='0x%llx'", info->addr.spaprvio.reg); break; case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCW: - virBufferAsprintf(buf, " cssid='0x%x' ssid='0x%x' devno='0x%04x'", + virBufferAsprintf(&attrBuf, " cssid='0x%x' ssid='0x%x' devno='0x%04x'", info->addr.ccw.cssid, info->addr.ccw.ssid, info->addr.ccw.devno); @@ -6510,15 +6512,15 @@ virDomainDeviceInfoFormat(virBufferPtr buf, case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_ISA: if (info->addr.isa.iobase > 0) - virBufferAsprintf(buf, " iobase='0x%x'", info->addr.isa.iobase); + virBufferAsprintf(&attrBuf, " iobase='0x%x'", info->addr.isa.iobase); if (info->addr.isa.irq > 0) - virBufferAsprintf(buf, " irq='0x%x'", info->addr.isa.irq); + virBufferAsprintf(&attrBuf, " irq='0x%x'", info->addr.isa.irq); break; case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM: - virBufferAsprintf(buf, " slot='%u'", info->addr.dimm.slot); + virBufferAsprintf(&attrBuf, " slot='%u'", info->addr.dimm.slot); if (info->addr.dimm.base) - virBufferAsprintf(buf, " base='0x%llx'", info->addr.dimm.base); + virBufferAsprintf(&attrBuf, " base='0x%llx'", info->addr.dimm.base); break; @@ -6528,7 +6530,9 @@ virDomainDeviceInfoFormat(virBufferPtr buf, break; } - virBufferAddLit(buf, "/>\n"); + virXMLFormatElement(buf, "address", &attrBuf, NULL); + + virBufferFreeAndReset(&attrBuf); } static int -- Yi Min
On Fri, 2018-10-19 at 11:40 +0800, Yi Min Zhao wrote:
In order to add zPCI child element for PCI address, we update virDomainDeviceInfoFormat() to format device info by helper function virXMLFormatElement(). Then we could simply format zPCI address into child buffer later.
Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> --- src/conf/domain_conf.c | 40 ++++++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 18 deletions(-)
Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
This patch introduces new XML parser/formatter functions. Uid is 16-bit and non-zero. Fid is 32-bit. They are the two attributes of zpci which is introduced as PCI address element. Zpci element is parsed and formatted along with PCI address. And add the related test cases. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> --- docs/schemas/basictypes.rng | 27 ++++++++++ docs/schemas/domaincommon.rng | 1 + src/conf/device_conf.c | 53 +++++++++++++++++++ src/conf/domain_addr.c | 3 ++ src/conf/domain_conf.c | 12 ++++- src/libvirt_private.syms | 2 + src/util/virpci.c | 26 +++++++++ src/util/virpci.h | 6 +++ .../disk-virtio-s390-zpci.args | 25 +++++++++ .../disk-virtio-s390-zpci.xml | 19 +++++++ tests/qemuxml2argvdata/hostdev-vfio-zpci.args | 23 ++++++++ tests/qemuxml2argvdata/hostdev-vfio-zpci.xml | 21 ++++++++ tests/qemuxml2argvtest.c | 7 +++ .../disk-virtio-s390-zpci.xml | 31 +++++++++++ .../qemuxml2xmloutdata/hostdev-vfio-zpci.xml | 32 +++++++++++ tests/qemuxml2xmltest.c | 6 +++ 16 files changed, 293 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/disk-virtio-s390-zpci.args create mode 100644 tests/qemuxml2argvdata/disk-virtio-s390-zpci.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci.xml create mode 100644 tests/qemuxml2xmloutdata/disk-virtio-s390-zpci.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci.xml diff --git a/docs/schemas/basictypes.rng b/docs/schemas/basictypes.rng index 14a3670e5c..71a6db3bb4 100644 --- a/docs/schemas/basictypes.rng +++ b/docs/schemas/basictypes.rng @@ -65,6 +65,17 @@ </data> </choice> </define> + <define name="uint32"> + <choice> + <data type="string"> + <param name="pattern">(0x)?[0-9a-fA-F]{1,8}</param> + </data> + <data type="unsignedInt"> + <param name="minInclusive">0</param> + <param name="maxInclusive">4294967295</param> + </data> + </choice> + </define> <define name="UUID"> <choice> @@ -111,6 +122,22 @@ </attribute> </optional> </define> + <define name="zpciaddress"> + <optional> + <element name="zpci"> + <optional> + <attribute name="uid"> + <ref name="uint16"/> + </attribute> + </optional> + <optional> + <attribute name="fid"> + <ref name="uint32"/> + </attribute> + </optional> + </element> + </optional> + </define> <!-- a 6 byte MAC address in ASCII-hex format, eg "12:34:56:78:9A:BC" --> <!-- The lowest bit of the 1st byte is the "multicast" bit. a --> diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 099a949cf8..9eeae80452 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -5220,6 +5220,7 @@ <value>pci</value> </attribute> <ref name="pciaddress"/> + <ref name="zpciaddress"/> </group> <group> <attribute name="type"> diff --git a/src/conf/device_conf.c b/src/conf/device_conf.c index 98a419f40f..f9f6b6e38f 100644 --- a/src/conf/device_conf.c +++ b/src/conf/device_conf.c @@ -47,6 +47,45 @@ VIR_ENUM_IMPL(virDomainDeviceAddress, VIR_DOMAIN_DEVICE_ADDRESS_TYPE_LAST, "dimm", ); +static int +virZPCIDeviceAddressParseXML(xmlNodePtr node, + virPCIDeviceAddressPtr addr) +{ + virZPCIDeviceAddress def = { 0 }; + char *uid; + char *fid; + int ret = -1; + + uid = virXMLPropString(node, "uid"); + fid = virXMLPropString(node, "fid"); + + if (uid && + virStrToLong_uip(uid, NULL, 0, &def.uid) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot parse <address> 'uid' attribute")); + goto cleanup; + } + + if (fid && + virStrToLong_uip(fid, NULL, 0, &def.fid) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot parse <address> 'fid' attribute")); + goto cleanup; + } + + if (!virZPCIDeviceAddressIsEmpty(&def) && + !virZPCIDeviceAddressIsValid(&def)) + goto cleanup; + + addr->zpci = def; + ret = 0; + + cleanup: + VIR_FREE(uid); + VIR_FREE(fid); + return ret; +} + int virDomainDeviceInfoCopy(virDomainDeviceInfoPtr dst, virDomainDeviceInfoPtr src) @@ -181,6 +220,8 @@ virPCIDeviceAddressParseXML(xmlNodePtr node, virPCIDeviceAddressPtr addr) { char *domain, *slot, *bus, *function, *multi; + xmlNodePtr cur; + xmlNodePtr zpci = NULL; int ret = -1; memset(addr, 0, sizeof(*addr)); @@ -230,6 +271,18 @@ virPCIDeviceAddressParseXML(xmlNodePtr node, if (!virPCIDeviceAddressIsEmpty(addr) && !virPCIDeviceAddressIsValid(addr, true)) goto cleanup; + cur = node->children; + while (cur) { + if (cur->type == XML_ELEMENT_NODE && + virXMLNodeNameEqual(cur, "zpci")) { + zpci = cur; + } + cur = cur->next; + } + + if (zpci && virZPCIDeviceAddressParseXML(zpci, addr) < 0) + goto cleanup; + ret = 0; cleanup: diff --git a/src/conf/domain_addr.c b/src/conf/domain_addr.c index 902837d4ee..c307d85ee1 100644 --- a/src/conf/domain_addr.c +++ b/src/conf/domain_addr.c @@ -1040,6 +1040,9 @@ virDomainPCIAddressReserveNextAddr(virDomainPCIAddressSetPtr addrs, dev->isolationGroup, false) < 0) return -1; + addr.extFlags = dev->addr.pci.extFlags; + addr.zpci = dev->addr.pci.zpci; + if (!addrs->dryRun) { dev->type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI; dev->addr.pci = addr; diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 88964f10c6..7193cbd84a 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -6401,6 +6401,7 @@ virDomainDeviceInfoFormat(virBufferPtr buf, unsigned int flags) { virBuffer attrBuf = VIR_BUFFER_INITIALIZER; + virBuffer childBuf = VIR_BUFFER_INITIALIZER; if ((flags & VIR_DOMAIN_DEF_FORMAT_ALLOW_BOOT) && info->bootIndex) { virBufferAsprintf(buf, "<boot order='%u'", info->bootIndex); @@ -6463,6 +6464,14 @@ virDomainDeviceInfoFormat(virBufferPtr buf, virBufferAsprintf(&attrBuf, " multifunction='%s'", virTristateSwitchTypeToString(info->addr.pci.multi)); } + + if (!virZPCIDeviceAddressIsEmpty(&info->addr.pci.zpci)) { + virBufferSetChildIndent(&childBuf, buf); + virBufferAsprintf(&childBuf, + "<zpci uid='0x%.4x' fid='0x%.8x'/>\n", + info->addr.pci.zpci.uid, + info->addr.pci.zpci.fid); + } break; case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DRIVE: @@ -6530,9 +6539,10 @@ virDomainDeviceInfoFormat(virBufferPtr buf, break; } - virXMLFormatElement(buf, "address", &attrBuf, NULL); + virXMLFormatElement(buf, "address", &attrBuf, &childBuf); virBufferFreeAndReset(&attrBuf); + virBufferFreeAndReset(&childBuf); } static int diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 335210c31d..7daf52bc0a 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2568,6 +2568,8 @@ virPCIHeaderTypeToString; virPCIIsVirtualFunction; virPCIStubDriverTypeFromString; virPCIStubDriverTypeToString; +virZPCIDeviceAddressIsEmpty; +virZPCIDeviceAddressIsValid; # util/virperf.h diff --git a/src/util/virpci.c b/src/util/virpci.c index 0f680efbe6..efa2d1662a 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -2563,6 +2563,32 @@ virPCIDeviceAddressParse(char *address, #ifdef __linux__ +bool +virZPCIDeviceAddressIsValid(virZPCIDeviceAddressPtr zpci) +{ + /* We don't need to check fid because fid covers + * all range of uint32 type. + */ + if (zpci->uid > VIR_DOMAIN_DEVICE_ZPCI_MAX_UID || + zpci->uid == 0) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid PCI address uid='0x%.4x', " + "must be > 0x0000 and <= 0x%.4x"), + zpci->uid, + VIR_DOMAIN_DEVICE_ZPCI_MAX_UID); + return false; + } + + return true; +} + +bool +virZPCIDeviceAddressIsEmpty(const virZPCIDeviceAddress *addr) +{ + return !(addr->uid || addr->fid); +} + + /* * returns true if equal */ diff --git a/src/util/virpci.h b/src/util/virpci.h index 0e40d86b97..a2e795eff9 100644 --- a/src/util/virpci.h +++ b/src/util/virpci.h @@ -37,6 +37,9 @@ typedef virPCIDeviceAddress *virPCIDeviceAddressPtr; typedef struct _virPCIDeviceList virPCIDeviceList; typedef virPCIDeviceList *virPCIDeviceListPtr; +# define VIR_DOMAIN_DEVICE_ZPCI_MAX_UID UINT16_MAX +# define VIR_DOMAIN_DEVICE_ZPCI_MAX_FID UINT32_MAX + typedef struct _virZPCIDeviceAddress virZPCIDeviceAddress; typedef virZPCIDeviceAddress *virZPCIDeviceAddressPtr; struct _virZPCIDeviceAddress { @@ -239,6 +242,9 @@ char *virPCIDeviceAddressAsString(virPCIDeviceAddressPtr addr) int virPCIDeviceAddressParse(char *address, virPCIDeviceAddressPtr bdf); +bool virZPCIDeviceAddressIsValid(virZPCIDeviceAddressPtr zpci); +bool virZPCIDeviceAddressIsEmpty(const virZPCIDeviceAddress *addr); + int virPCIGetVirtualFunctionInfo(const char *vf_sysfs_device_path, int pfNetDevIdx, char **pfname, diff --git a/tests/qemuxml2argvdata/disk-virtio-s390-zpci.args b/tests/qemuxml2argvdata/disk-virtio-s390-zpci.args new file mode 100644 index 0000000000..8ac435cb3e --- /dev/null +++ b/tests/qemuxml2argvdata/disk-virtio-s390-zpci.args @@ -0,0 +1,25 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/bin/qemu-system-s390x \ +-name QEMUGuest1 \ +-S \ +-machine s390-ccw-virtio,accel=tcg,usb=off,dump-guest-core=off \ +-m 214 \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-QEMUGuest1/monitor.sock,\ +server,nowait \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-virtio-disk0 \ +-device virtio-blk-pci,bus=pci.0,addr=0x8,drive=drive-virtio-disk0,\ +id=virtio-disk0,bootindex=1 \ +-device virtio-balloon-ccw,id=balloon0,devno=fe.0.0000 diff --git a/tests/qemuxml2argvdata/disk-virtio-s390-zpci.xml b/tests/qemuxml2argvdata/disk-virtio-s390-zpci.xml new file mode 100644 index 0000000000..8bf4a23670 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-virtio-s390-zpci.xml @@ -0,0 +1,19 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory>219136</memory> + <vcpu>1</vcpu> + <os> + <type arch='s390x' machine='s390-ccw-virtio'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-s390x</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'> + <zpci uid='0x0019' fid='0x0000001f'/> + </address> + </disk> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci.args b/tests/qemuxml2argvdata/hostdev-vfio-zpci.args new file mode 100644 index 0000000000..d6b1520c47 --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci.args @@ -0,0 +1,23 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/bin/qemu-system-s390x \ +-name QEMUGuest1 \ +-S \ +-machine s390-ccw-virtio,accel=tcg,usb=off,dump-guest-core=off \ +-m 214 \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-QEMUGuest1/monitor.sock,\ +server,nowait \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-device vfio-pci,host=00:00.0,id=hostdev0,bus=pci.0,addr=0x8 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x1 diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci.xml b/tests/qemuxml2argvdata/hostdev-vfio-zpci.xml new file mode 100644 index 0000000000..002b99c52d --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci.xml @@ -0,0 +1,21 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory>219100</memory> + <os> + <type arch='s390x' machine='s390-ccw-virtio'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-s390x</emulator> + <controller type='pci' index='0' model='pci-root'/> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'> + <zpci uid='0x0019' fid='0x0000001f'/> + </address> + </hostdev> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 39a7f1f53c..7b9e6bf2be 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1076,6 +1076,10 @@ mymain(void) QEMU_CAPS_CCW, QEMU_CAPS_VIRTIO_S390); DO_TEST("disk-virtio-scsi-ccw", QEMU_CAPS_VIRTIO_SCSI, QEMU_CAPS_CCW, QEMU_CAPS_VIRTIO_S390); + DO_TEST("disk-virtio-s390-zpci", + QEMU_CAPS_DEVICE_ZPCI, + QEMU_CAPS_CCW, + QEMU_CAPS_VIRTIO_S390); DO_TEST("disk-order", QEMU_CAPS_VIRTIO_BLK_SCSI); DO_TEST("disk-virtio-queues", QEMU_CAPS_VIRTIO_BLK_NUM_QUEUES); @@ -1679,6 +1683,9 @@ mymain(void) DO_TEST_PARSE_ERROR("hostdev-mdev-display-missing-graphics", QEMU_CAPS_DEVICE_VFIO_PCI, QEMU_CAPS_VFIO_PCI_DISPLAY); + DO_TEST("hostdev-vfio-zpci", + QEMU_CAPS_DEVICE_VFIO_PCI, + QEMU_CAPS_DEVICE_ZPCI); DO_TEST("pci-rom", NONE); DO_TEST("pci-rom-disabled", NONE); DO_TEST("pci-rom-disabled-invalid", NONE); diff --git a/tests/qemuxml2xmloutdata/disk-virtio-s390-zpci.xml b/tests/qemuxml2xmloutdata/disk-virtio-s390-zpci.xml new file mode 100644 index 0000000000..37684c82b1 --- /dev/null +++ b/tests/qemuxml2xmloutdata/disk-virtio-s390-zpci.xml @@ -0,0 +1,31 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='s390x' machine='s390-ccw-virtio'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-s390x</emulator> + <disk type='block' device='disk'> + <driver name='qemu' type='raw'/> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'> + <zpci uid='0x0019' fid='0x0000001f'/> + </address> + </disk> + <controller type='pci' index='0' model='pci-root'/> + <memballoon model='virtio'> + <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/> + </memballoon> + <panic model='s390'/> + </devices> +</domain> diff --git a/tests/qemuxml2xmloutdata/hostdev-vfio-zpci.xml b/tests/qemuxml2xmloutdata/hostdev-vfio-zpci.xml new file mode 100644 index 0000000000..fc8c38ab66 --- /dev/null +++ b/tests/qemuxml2xmloutdata/hostdev-vfio-zpci.xml @@ -0,0 +1,32 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219100</memory> + <currentMemory unit='KiB'>219100</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='s390x' machine='s390-ccw-virtio'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-s390x</emulator> + <controller type='pci' index='0' model='pci-root'/> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'> + <zpci uid='0x0019' fid='0x0000001f'/> + </address> + </hostdev> + <memballoon model='virtio'> + <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/> + </memballoon> + <panic model='s390'/> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 89640f641a..53de9726f4 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -376,6 +376,9 @@ mymain(void) QEMU_CAPS_VIRTIO_SCSI); DO_TEST("disk-virtio-scsi-ioeventfd", QEMU_CAPS_VIRTIO_SCSI); + DO_TEST("disk-virtio-s390-zpci", + QEMU_CAPS_DEVICE_ZPCI, + QEMU_CAPS_CCW); DO_TEST("disk-scsi-megasas", QEMU_CAPS_SCSI_MEGASAS); DO_TEST("disk-scsi-mptsas1068", @@ -458,6 +461,9 @@ mymain(void) DO_TEST("hostdev-usb-address", NONE); DO_TEST("hostdev-pci-address", NONE); DO_TEST("hostdev-vfio", NONE); + DO_TEST("hostdev-vfio-zpci", + QEMU_CAPS_DEVICE_ZPCI, + QEMU_CAPS_CCW); DO_TEST("hostdev-mdev-precreated", NONE); DO_TEST("hostdev-mdev-display", QEMU_CAPS_VFIO_PCI_DISPLAY); DO_TEST("pci-rom", NONE); -- Yi Min
On Fri, 2018-10-19 at 11:40 +0800, Yi Min Zhao wrote:
This patch introduces new XML parser/formatter functions. Uid is 16-bit and non-zero. Fid is 32-bit. They are the two attributes of zpci which is introduced as PCI address element. Zpci element is parsed and formatted along with PCI address. And add the related test cases.
Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> --- docs/schemas/basictypes.rng | 27 ++++++++++ docs/schemas/domaincommon.rng | 1 + src/conf/device_conf.c | 53 +++++++++++++++++++ src/conf/domain_addr.c | 3 ++ src/conf/domain_conf.c | 12 ++++- src/libvirt_private.syms | 2 + src/util/virpci.c | 26 +++++++++ src/util/virpci.h | 6 +++ .../disk-virtio-s390-zpci.args | 25 +++++++++ .../disk-virtio-s390-zpci.xml | 19 +++++++ tests/qemuxml2argvdata/hostdev-vfio-zpci.args | 23 ++++++++ tests/qemuxml2argvdata/hostdev-vfio-zpci.xml | 21 ++++++++ tests/qemuxml2argvtest.c | 7 +++ .../disk-virtio-s390-zpci.xml | 31 +++++++++++ .../qemuxml2xmloutdata/hostdev-vfio-zpci.xml | 32 +++++++++++ tests/qemuxml2xmltest.c | 6 +++ 16 files changed, 293 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/disk-virtio-s390-zpci.args create mode 100644 tests/qemuxml2argvdata/disk-virtio-s390-zpci.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci.xml create mode 100644 tests/qemuxml2xmloutdata/disk-virtio-s390-zpci.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci.xml
Reviewed-by: Andrea Bolognani <abologna@redhat.com> -- Andrea Bolognani / Red Hat / Virtualization
We should ensure that QEMU supports zPCI when a zPCI address is defined in XML and otherwise report an error. This patch introduces a generic validation function qemuDomainDeviceDefValidateAddress() which calls qemuDomainDeviceDefValidateZPCIAddress() if address type is PCI address. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- src/qemu/qemu_domain.c | 36 +++++++++++++++++++ .../hostdev-vfio-zpci-wrong-arch.xml | 34 ++++++++++++++++++ tests/qemuxml2argvtest.c | 2 ++ 3 files changed, 72 insertions(+) create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-wrong-arch.xml diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 961755ef84..8bdc3f58e8 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -5797,6 +5797,38 @@ qemuDomainDeviceDefValidateInput(const virDomainInputDef *input, } +static int +qemuDomainDeviceDefValidateZPCIAddress(virDomainDeviceInfoPtr info, + virQEMUCapsPtr qemuCaps) +{ + if (!virZPCIDeviceAddressIsEmpty(&info->addr.pci.zpci) && + !virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_ZPCI)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + "%s", + _("This QEMU binary doesn't support zPCI")); + return -1; + } + + return 0; +} + + +static int +qemuDomainDeviceDefValidateAddress(const virDomainDeviceDef *dev, + virQEMUCapsPtr qemuCaps) +{ + virDomainDeviceInfoPtr info; + + if (!(info = virDomainDeviceGetInfo((virDomainDeviceDef *)dev))) + return 0; + + if (info->type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) + return qemuDomainDeviceDefValidateZPCIAddress(info, qemuCaps); + + return 0; +} + + static int qemuDomainDeviceDefValidate(const virDomainDeviceDef *dev, const virDomainDef *def, @@ -5810,6 +5842,9 @@ qemuDomainDeviceDefValidate(const virDomainDeviceDef *dev, def->emulator))) return -1; + if ((ret = qemuDomainDeviceDefValidateAddress(dev, qemuCaps)) < 0) + goto cleanup; + switch ((virDomainDeviceType)dev->type) { case VIR_DOMAIN_DEVICE_NET: ret = qemuDomainDeviceDefValidateNetwork(dev->data.net); @@ -5885,6 +5920,7 @@ qemuDomainDeviceDefValidate(const virDomainDeviceDef *dev, break; } + cleanup: virObjectUnref(qemuCaps); return ret; } diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci-wrong-arch.xml b/tests/qemuxml2argvdata/hostdev-vfio-zpci-wrong-arch.xml new file mode 100644 index 0000000000..bfb2f83a3b --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci-wrong-arch.xml @@ -0,0 +1,34 @@ +<domain type='qemu'> + <name>QEMUGuest2</name> + <uuid>c7a5fdbd-edaf-9466-926a-d65c16db1809</uuid> + <memory unit='KiB'>219100</memory> + <currentMemory unit='KiB'>219100</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i686</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest2'/> + <target dev='hda' bus='ide'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='pci' index='0' model='pci-root'/> + <hostdev mode='subsystem' type='pci' managed='yes'> + <driver name='vfio'/> + <source> + <address domain='0x0001' bus='00' slot='00' function='0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'> + <zpci uid='0x0019' fid='0x0000001f'/> + </address> + </hostdev> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 7b9e6bf2be..09c52f5f38 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1683,6 +1683,8 @@ mymain(void) DO_TEST_PARSE_ERROR("hostdev-mdev-display-missing-graphics", QEMU_CAPS_DEVICE_VFIO_PCI, QEMU_CAPS_VFIO_PCI_DISPLAY); + DO_TEST_PARSE_ERROR("hostdev-vfio-zpci-wrong-arch", + QEMU_CAPS_DEVICE_VFIO_PCI); DO_TEST("hostdev-vfio-zpci", QEMU_CAPS_DEVICE_VFIO_PCI, QEMU_CAPS_DEVICE_ZPCI); -- Yi Min
This patch adds new functions for reservation, assignment and release to handle the uid/fid. If the uid/fid is defined in the domain XML, they will be reserved directly in the collecting phase. If any of them is not defined, we will find out an available value for them from the zPCI address hashtable, and reserve them. For the hotplug case there might not be a zPCI definition. So allocate and reserve uid/fid the case. Assign if needed and reserve uid/fid for the defined case. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> --- src/conf/device_conf.c | 16 +++ src/conf/device_conf.h | 3 + src/conf/domain_addr.c | 244 +++++++++++++++++++++++++++++++++ src/conf/domain_addr.h | 12 ++ src/libvirt_private.syms | 5 + src/qemu/qemu_domain_address.c | 59 +++++++- 6 files changed, 338 insertions(+), 1 deletion(-) diff --git a/src/conf/device_conf.c b/src/conf/device_conf.c index f9f6b6e38f..44b210d5ec 100644 --- a/src/conf/device_conf.c +++ b/src/conf/device_conf.c @@ -28,6 +28,7 @@ #include "viruuid.h" #include "virbuffer.h" #include "device_conf.h" +#include "domain_addr.h" #include "virstring.h" #define VIR_FROM_THIS VIR_FROM_DEVICE @@ -215,6 +216,21 @@ virDeviceInfoPCIAddressIsPresent(const virDomainDeviceInfo *info) } +bool +virDeviceInfoPCIAddressExtensionIsWanted(const virDomainDeviceInfo *info) +{ + return (info->addr.pci.extFlags & VIR_PCI_ADDRESS_EXTENSION_ZPCI) && + virZPCIDeviceAddressIsEmpty(&info->addr.pci.zpci); +} + +bool +virDeviceInfoPCIAddressExtensionIsPresent(const virDomainDeviceInfo *info) +{ + return (info->addr.pci.extFlags & VIR_PCI_ADDRESS_EXTENSION_ZPCI) && + !virZPCIDeviceAddressIsEmpty(&info->addr.pci.zpci); +} + + int virPCIDeviceAddressParseXML(xmlNodePtr node, virPCIDeviceAddressPtr addr) diff --git a/src/conf/device_conf.h b/src/conf/device_conf.h index 2366e03607..867b633903 100644 --- a/src/conf/device_conf.h +++ b/src/conf/device_conf.h @@ -198,6 +198,9 @@ bool virDomainDeviceAddressIsValid(virDomainDeviceInfoPtr info, bool virDeviceInfoPCIAddressIsWanted(const virDomainDeviceInfo *info); bool virDeviceInfoPCIAddressIsPresent(const virDomainDeviceInfo *info); +bool virDeviceInfoPCIAddressExtensionIsWanted(const virDomainDeviceInfo *info); +bool virDeviceInfoPCIAddressExtensionIsPresent(const virDomainDeviceInfo *info); + int virPCIDeviceAddressParseXML(xmlNodePtr node, virPCIDeviceAddressPtr addr); diff --git a/src/conf/domain_addr.c b/src/conf/domain_addr.c index c307d85ee1..d174558282 100644 --- a/src/conf/domain_addr.c +++ b/src/conf/domain_addr.c @@ -33,6 +33,238 @@ VIR_LOG_INIT("conf.domain_addr"); +static int +virDomainZPCIAddressReserveId(virHashTablePtr set, + unsigned int id, + const char *name) +{ + if (virHashLookup(set, &id)) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("zPCI %s %o is already reserved"), + name, id); + return -1; + } + + if (virHashAddEntry(set, &id, (void*)1) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Failed to reserve %s %o"), + name, id); + return -1; + } + + return 0; +} + + +static int +virDomainZPCIAddressReserveUid(virHashTablePtr set, + virZPCIDeviceAddressPtr addr) +{ + return virDomainZPCIAddressReserveId(set, addr->uid, "uid"); +} + + +static int +virDomainZPCIAddressReserveFid(virHashTablePtr set, + virZPCIDeviceAddressPtr addr) +{ + return virDomainZPCIAddressReserveId(set, addr->fid, "fid"); +} + + +static int +virDomainZPCIAddressAssignId(virHashTablePtr set, + unsigned int *id, + unsigned int min, + unsigned int max, + const char *name) +{ + while (virHashLookup(set, &min)) { + if (min == max) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("There is no more free %s."), + name); + return -1; + } + ++min; + } + *id = min; + + return 0; +} + + +static int +virDomainZPCIAddressAssignUid(virHashTablePtr set, + virZPCIDeviceAddressPtr addr) +{ + return virDomainZPCIAddressAssignId(set, &addr->uid, 1, + VIR_DOMAIN_DEVICE_ZPCI_MAX_UID, "uid"); +} + + +static int +virDomainZPCIAddressAssignFid(virHashTablePtr set, + virZPCIDeviceAddressPtr addr) +{ + return virDomainZPCIAddressAssignId(set, &addr->fid, 0, + VIR_DOMAIN_DEVICE_ZPCI_MAX_FID, "fid"); +} + + +static void +virDomainZPCIAddressReleaseId(virHashTablePtr set, + unsigned int *id, + const char *name) +{ + if (virHashRemoveEntry(set, id) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Release %s %o failed"), + name, *id); + } + + *id = 0; +} + + +static void +virDomainZPCIAddressReleaseUid(virHashTablePtr set, + virZPCIDeviceAddressPtr addr) +{ + virDomainZPCIAddressReleaseId(set, &addr->uid, "uid"); +} + + +static void +virDomainZPCIAddressReleaseFid(virHashTablePtr set, + virZPCIDeviceAddressPtr addr) +{ + virDomainZPCIAddressReleaseId(set, &addr->fid, "fid"); +} + + +static void +virDomainZPCIAddressReleaseIds(virDomainZPCIAddressIdsPtr zpciIds, + virZPCIDeviceAddressPtr addr) +{ + if (!zpciIds || virZPCIDeviceAddressIsEmpty(addr)) + return; + + virDomainZPCIAddressReleaseUid(zpciIds->uids, addr); + + virDomainZPCIAddressReleaseFid(zpciIds->fids, addr); +} + + +static int +virDomainZPCIAddressReserveNextUid(virHashTablePtr uids, + virZPCIDeviceAddressPtr zpci) +{ + if (virDomainZPCIAddressAssignUid(uids, zpci) < 0) + return -1; + + if (virDomainZPCIAddressReserveUid(uids, zpci) < 0) + return -1; + + return 0; +} + + +static int +virDomainZPCIAddressReserveNextFid(virHashTablePtr fids, + virZPCIDeviceAddressPtr zpci) +{ + if (virDomainZPCIAddressAssignFid(fids, zpci) < 0) + return -1; + + if (virDomainZPCIAddressReserveFid(fids, zpci) < 0) + return -1; + + return 0; +} + + +static int +virDomainZPCIAddressReserveAddr(virDomainZPCIAddressIdsPtr zpciIds, + virZPCIDeviceAddressPtr addr) +{ + if (virDomainZPCIAddressReserveUid(zpciIds->uids, addr) < 0) + return -1; + + if (virDomainZPCIAddressReserveFid(zpciIds->fids, addr) < 0) { + virDomainZPCIAddressReleaseUid(zpciIds->uids, addr); + return -1; + } + + return 0; +} + + +static int +virDomainZPCIAddressReserveNextAddr(virDomainZPCIAddressIdsPtr zpciIds, + virZPCIDeviceAddressPtr addr) +{ + if (virDomainZPCIAddressReserveNextUid(zpciIds->uids, addr) < 0) + return -1; + + if (virDomainZPCIAddressReserveNextFid(zpciIds->fids, addr) < 0) { + virDomainZPCIAddressReleaseUid(zpciIds->uids, addr); + return -1; + } + + return 0; +} + + +int +virDomainPCIAddressExtensionReserveAddr(virDomainPCIAddressSetPtr addrs, + virPCIDeviceAddressPtr addr) +{ + if (addr->extFlags & VIR_PCI_ADDRESS_EXTENSION_ZPCI) { + /* Reserve uid/fid to ZPCI device which has defined uid/fid + * in the domain. + */ + return virDomainZPCIAddressReserveAddr(addrs->zpciIds, &addr->zpci); + } + + return 0; +} + + +int +virDomainPCIAddressExtensionReserveNextAddr(virDomainPCIAddressSetPtr addrs, + virPCIDeviceAddressPtr addr) +{ + if (addr->extFlags & VIR_PCI_ADDRESS_EXTENSION_ZPCI) { + virZPCIDeviceAddress zpci = { 0 }; + + if (virDomainZPCIAddressReserveNextAddr(addrs->zpciIds, &zpci) < 0) + return -1; + + if (!addrs->dryRun) + addr->zpci = zpci; + } + + return 0; +} + +static int +virDomainPCIAddressExtensionEnsureAddr(virDomainPCIAddressSetPtr addrs, + virPCIDeviceAddressPtr addr) +{ + if (addr->extFlags & VIR_PCI_ADDRESS_EXTENSION_ZPCI) { + virZPCIDeviceAddressPtr zpci = &addr->zpci; + + if (virZPCIDeviceAddressIsEmpty(zpci)) + return virDomainZPCIAddressReserveNextAddr(addrs->zpciIds, zpci); + else + return virDomainZPCIAddressReserveAddr(addrs->zpciIds, zpci); + } + + return 0; +} + + virDomainPCIConnectFlags virDomainPCIControllerModelToConnectType(virDomainControllerModelPCI model) { @@ -715,12 +947,24 @@ virDomainPCIAddressEnsureAddr(virDomainPCIAddressSetPtr addrs, ret = virDomainPCIAddressReserveNextAddr(addrs, dev, flags, -1); } + dev->addr.pci.extFlags = dev->pciAddrExtFlags; + ret = virDomainPCIAddressExtensionEnsureAddr(addrs, &dev->addr.pci); + cleanup: VIR_FREE(addrStr); return ret; } +void +virDomainPCIAddressExtensionReleaseAddr(virDomainPCIAddressSetPtr addrs, + virPCIDeviceAddressPtr addr) +{ + if (addr->extFlags & VIR_PCI_ADDRESS_EXTENSION_ZPCI) + virDomainZPCIAddressReleaseIds(addrs->zpciIds, &addr->zpci); +} + + void virDomainPCIAddressReleaseAddr(virDomainPCIAddressSetPtr addrs, virPCIDeviceAddressPtr addr) diff --git a/src/conf/domain_addr.h b/src/conf/domain_addr.h index 8d64d6b795..183235309b 100644 --- a/src/conf/domain_addr.h +++ b/src/conf/domain_addr.h @@ -160,6 +160,14 @@ bool virDomainPCIAddressSlotInUse(virDomainPCIAddressSetPtr addrs, virPCIDeviceAddressPtr addr) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); +int virDomainPCIAddressExtensionReserveAddr(virDomainPCIAddressSetPtr addrs, + virPCIDeviceAddressPtr addr) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); + +int virDomainPCIAddressExtensionReserveNextAddr(virDomainPCIAddressSetPtr addrs, + virPCIDeviceAddressPtr addr) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); + int virDomainPCIAddressReserveAddr(virDomainPCIAddressSetPtr addrs, virPCIDeviceAddressPtr addr, virDomainPCIConnectFlags flags, @@ -181,6 +189,10 @@ void virDomainPCIAddressReleaseAddr(virDomainPCIAddressSetPtr addrs, virPCIDeviceAddressPtr addr) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); +void virDomainPCIAddressExtensionReleaseAddr(virDomainPCIAddressSetPtr addrs, + virPCIDeviceAddressPtr addr) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); + void virDomainPCIAddressSetAllMulti(virDomainDefPtr def) ATTRIBUTE_NONNULL(1); diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 7daf52bc0a..1f365f451e 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -93,6 +93,8 @@ virCPUModeTypeToString; # conf/device_conf.h +virDeviceInfoPCIAddressExtensionIsPresent; +virDeviceInfoPCIAddressExtensionIsWanted; virDeviceInfoPCIAddressIsPresent; virDeviceInfoPCIAddressIsWanted; virDomainDeviceAddressIsValid; @@ -119,6 +121,9 @@ virDomainCCWAddressSetFree; virDomainPCIAddressBusIsFullyReserved; virDomainPCIAddressBusSetModel; virDomainPCIAddressEnsureAddr; +virDomainPCIAddressExtensionReleaseAddr; +virDomainPCIAddressExtensionReserveAddr; +virDomainPCIAddressExtensionReserveNextAddr; virDomainPCIAddressReleaseAddr; virDomainPCIAddressReserveAddr; virDomainPCIAddressReserveNextAddr; diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 665f6c1445..a10194794a 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -1402,6 +1402,24 @@ qemuDomainPCIAddressReserveNextAddr(virDomainPCIAddressSetPtr addrs, } +static int +qemuDomainAssignPCIAddressExtension(virDomainDefPtr def ATTRIBUTE_UNUSED, + virDomainDeviceDefPtr device ATTRIBUTE_UNUSED, + virDomainDeviceInfoPtr info, + void *opaque) +{ + virDomainPCIAddressSetPtr addrs = opaque; + virPCIDeviceAddressPtr addr = &info->addr.pci; + + if (info->type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) + addr->extFlags = info->pciAddrExtFlags; + + if (virDeviceInfoPCIAddressExtensionIsWanted(info)) + return virDomainPCIAddressExtensionReserveNextAddr(addrs, addr); + + return 0; +} + static int qemuDomainCollectPCIAddress(virDomainDefPtr def ATTRIBUTE_UNUSED, virDomainDeviceDefPtr device, @@ -1495,6 +1513,31 @@ qemuDomainCollectPCIAddress(virDomainDefPtr def ATTRIBUTE_UNUSED, return ret; } +static int +qemuDomainCollectPCIAddressExtension(virDomainDefPtr def ATTRIBUTE_UNUSED, + virDomainDeviceDefPtr device, + virDomainDeviceInfoPtr info, + void *opaque) +{ + virDomainPCIAddressSetPtr addrs = opaque; + virPCIDeviceAddressPtr addr = &info->addr.pci; + + if (info->type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) + addr->extFlags = info->pciAddrExtFlags; + + if (!virDeviceInfoPCIAddressExtensionIsPresent(info) || + ((device->type == VIR_DOMAIN_DEVICE_HOSTDEV) && + (device->data.hostdev->parent.type != VIR_DOMAIN_DEVICE_NONE))) { + /* If a hostdev has a parent, its info will be a part of the + * parent, and will have its address collected during the scan + * of the parent's device type. + */ + return 0; + } + + return virDomainPCIAddressExtensionReserveAddr(addrs, addr); +} + static virDomainPCIAddressSetPtr qemuDomainPCIAddressSetCreate(virDomainDefPtr def, virQEMUCapsPtr qemuCaps, @@ -1586,6 +1629,12 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, if (virDomainDeviceInfoIterate(def, qemuDomainCollectPCIAddress, addrs) < 0) goto error; + if (virDomainDeviceInfoIterate(def, + qemuDomainCollectPCIAddressExtension, + addrs) < 0) { + goto error; + } + return addrs; error: @@ -2590,6 +2639,9 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, if (qemuDomainAssignDevicePCISlots(def, qemuCaps, addrs) < 0) goto cleanup; + if (virDomainDeviceInfoIterate(def, qemuDomainAssignPCIAddressExtension, addrs) < 0) + goto cleanup; + /* Only for *new* domains with pcie-root (and no other * manually specified PCI controllers in the definition): If, * after assigning addresses/reserving slots for all devices, @@ -2684,6 +2736,9 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, if (qemuDomainAssignDevicePCISlots(def, qemuCaps, addrs) < 0) goto cleanup; + if (virDomainDeviceInfoIterate(def, qemuDomainAssignPCIAddressExtension, addrs) < 0) + goto cleanup; + /* set multi attribute for devices at function 0 of * any slot that has multiple functions in use */ @@ -3143,8 +3198,10 @@ qemuDomainReleaseDeviceAddress(virDomainObjPtr vm, if (!devstr) devstr = info->alias; - if (virDeviceInfoPCIAddressIsPresent(info)) + if (virDeviceInfoPCIAddressIsPresent(info)) { virDomainPCIAddressReleaseAddr(priv->pciaddrs, &info->addr.pci); + virDomainPCIAddressExtensionReleaseAddr(priv->pciaddrs, &info->addr.pci); + } if (virDomainUSBAddressRelease(priv->usbaddrs, info) < 0) VIR_WARN("Unable to release USB address on %s", NULLSTR(devstr)); -- Yi Min
Add new functions to generate zPCI command string and append it to QEMU command line. And the related tests are added. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- src/qemu/qemu_command.c | 104 ++++++++++++++++++ src/qemu/qemu_command.h | 2 + .../disk-virtio-s390-zpci.args | 1 + .../hostdev-vfio-zpci-autogenerate.args | 25 +++++ .../hostdev-vfio-zpci-autogenerate.xml | 18 +++ .../hostdev-vfio-zpci-boundaries.args | 29 +++++ .../hostdev-vfio-zpci-boundaries.xml | 30 +++++ .../hostdev-vfio-zpci-multidomain-many.args | 39 +++++++ .../hostdev-vfio-zpci-multidomain-many.xml | 79 +++++++++++++ tests/qemuxml2argvdata/hostdev-vfio-zpci.args | 2 + tests/qemuxml2argvtest.c | 13 +++ .../hostdev-vfio-zpci-autogenerate.xml | 34 ++++++ .../hostdev-vfio-zpci-boundaries.xml | 48 ++++++++ .../hostdev-vfio-zpci-multidomain-many.xml | 97 ++++++++++++++++ tests/qemuxml2xmltest.c | 11 ++ 15 files changed, 532 insertions(+) create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci-autogenerate.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci-boundaries.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci-multidomain-many.xml diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 269276f2f9..817cb6fddb 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -2195,6 +2195,57 @@ qemuBuildDiskDeviceStr(const virDomainDef *def, return NULL; } +char * +qemuBuildZPCIDevStr(virDomainDeviceInfoPtr dev) +{ + virBuffer buf = VIR_BUFFER_INITIALIZER; + + virBufferAsprintf(&buf, + "zpci,uid=%u,fid=%u,target=%s,id=zpci%u", + dev->addr.pci.zpci.uid, + dev->addr.pci.zpci.fid, + dev->alias, + dev->addr.pci.zpci.uid); + + if (virBufferCheckError(&buf) < 0) { + virBufferFreeAndReset(&buf); + return NULL; + } + + return virBufferContentAndReset(&buf); +} + +static int +qemuCommandAddZPCIDevice(virCommandPtr cmd, + virDomainDeviceInfoPtr dev) +{ + char *devstr = NULL; + + virCommandAddArg(cmd, "-device"); + + if (!(devstr = qemuBuildZPCIDevStr(dev))) + return -1; + + virCommandAddArg(cmd, devstr); + + VIR_FREE(devstr); + return 0; +} + +static int +qemuCommandAddExtDevice(virCommandPtr cmd, + virDomainDeviceInfoPtr dev) +{ + if (dev->type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI || + dev->addr.pci.extFlags == VIR_PCI_ADDRESS_EXTENSION_NONE) { + return 0; + } + + if (dev->addr.pci.extFlags & VIR_PCI_ADDRESS_EXTENSION_ZPCI) + return qemuCommandAddZPCIDevice(cmd, dev); + + return 0; +} static int qemuBuildFloppyCommandLineControllerOptions(virCommandPtr cmd, @@ -2431,6 +2482,9 @@ qemuBuildDiskCommandLine(virCommandPtr cmd, if (!qemuDiskBusNeedsDriveArg(disk->bus)) { if (disk->bus != VIR_DOMAIN_DISK_BUS_FDC || virQEMUCapsGet(qemuCaps, QEMU_CAPS_BLOCKDEV)) { + if (qemuCommandAddExtDevice(cmd, &disk->info) < 0) + return -1; + virCommandAddArg(cmd, "-device"); if (!(optstr = qemuBuildDiskDeviceStr(def, disk, bootindex, @@ -2629,6 +2683,9 @@ qemuBuildFSDevCommandLine(virCommandPtr cmd, virCommandAddArg(cmd, optstr); VIR_FREE(optstr); + if (qemuCommandAddExtDevice(cmd, &fs->info) < 0) + return -1; + virCommandAddArg(cmd, "-device"); if (!(optstr = qemuBuildFSDevStr(def, fs, qemuCaps))) return -1; @@ -3089,6 +3146,11 @@ qemuBuildControllerDevCommandLine(virCommandPtr cmd, goto cleanup; if (devstr) { + if (qemuCommandAddExtDevice(cmd, &cont->info) < 0) { + VIR_FREE(devstr); + goto cleanup; + } + virCommandAddArg(cmd, "-device"); virCommandAddArg(cmd, devstr); VIR_FREE(devstr); @@ -3887,6 +3949,9 @@ qemuBuildWatchdogCommandLine(virCommandPtr cmd, if (!def->watchdog) return 0; + if (qemuCommandAddExtDevice(cmd, &def->watchdog->info) < 0) + return -1; + virCommandAddArg(cmd, "-device"); optstr = qemuBuildWatchdogDevStr(def, watchdog, qemuCaps); @@ -3959,6 +4024,9 @@ qemuBuildMemballoonCommandLine(virCommandPtr cmd, if (qemuBuildVirtioOptionsStr(&buf, def->memballoon->virtio, qemuCaps) < 0) goto error; + if (qemuCommandAddExtDevice(cmd, &def->memballoon->info) < 0) + goto error; + virCommandAddArg(cmd, "-device"); virCommandAddArgBuffer(cmd, &buf); return 0; @@ -4153,6 +4221,9 @@ qemuBuildInputCommandLine(virCommandPtr cmd, virDomainInputDefPtr input = def->inputs[i]; char *devstr = NULL; + if (qemuCommandAddExtDevice(cmd, &input->info) < 0) + return -1; + if (qemuBuildInputDevStr(&devstr, def, input, qemuCaps) < 0) return -1; @@ -4294,6 +4365,9 @@ qemuBuildSoundCommandLine(virCommandPtr cmd, if (sound->model == VIR_DOMAIN_SOUND_MODEL_PCSPK) { virCommandAddArgList(cmd, "-soundhw", "pcspk", NULL); } else { + if (qemuCommandAddExtDevice(cmd, &sound->info) < 0) + return -1; + virCommandAddArg(cmd, "-device"); if (!(str = qemuBuildSoundDevStr(def, sound, qemuCaps))) return -1; @@ -4531,6 +4605,10 @@ qemuBuildVideoCommandLine(virCommandPtr cmd, if (video->primary) { if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_VIDEO_PRIMARY)) { + if (qemuCommandAddExtDevice(cmd, + &def->videos[i]->info) < 0) + return -1; + virCommandAddArg(cmd, "-device"); if (!(str = qemuBuildDeviceVideoStr(def, video, qemuCaps))) @@ -4543,6 +4621,9 @@ qemuBuildVideoCommandLine(virCommandPtr cmd, return -1; } } else { + if (qemuCommandAddExtDevice(cmd, &def->videos[i]->info) < 0) + return -1; + virCommandAddArg(cmd, "-device"); if (!(str = qemuBuildDeviceVideoStr(def, video, qemuCaps))) @@ -5378,6 +5459,10 @@ qemuBuildHostdevCommandLine(virCommandPtr cmd, VIR_COMMAND_PASS_FD_CLOSE_PARENT); } } + + if (qemuCommandAddExtDevice(cmd, hostdev->info) < 0) + return -1; + virCommandAddArg(cmd, "-device"); devstr = qemuBuildPCIHostdevDevStr(def, hostdev, bootIndex, configfd_name, qemuCaps); @@ -5824,6 +5909,9 @@ qemuBuildRNGCommandLine(virLogManagerPtr logManager, virCommandAddArgBuffer(cmd, &buf); /* add the device */ + if (qemuCommandAddExtDevice(cmd, &rng->info) < 0) + return -1; + if (!(tmp = qemuBuildRNGDevStr(def, rng, qemuCaps))) return -1; virCommandAddArgList(cmd, "-device", tmp, NULL); @@ -8288,6 +8376,9 @@ qemuBuildVhostuserCommandLine(virQEMUDriverPtr driver, virCommandAddArg(cmd, "-netdev"); virCommandAddArg(cmd, netdev); + if (qemuCommandAddExtDevice(cmd, &net->info) < 0) + goto cleanup; + if (!(nic = qemuBuildNicDevStr(def, net, bootindex, queues, qemuCaps))) { goto cleanup; @@ -8569,6 +8660,9 @@ qemuBuildInterfaceCommandLine(virQEMUDriverPtr driver, * New way: -netdev type=tap,id=netdev1 -device e1000,id=netdev1 */ if (qemuDomainSupportsNicdev(def, net)) { + if (qemuCommandAddExtDevice(cmd, &net->info) < 0) + goto cleanup; + if (!(nic = qemuBuildNicDevStr(def, net, bootindex, vhostfdSize, qemuCaps))) goto cleanup; @@ -9016,6 +9110,12 @@ qemuBuildShmemCommandLine(virLogManagerPtr logManager, if (!devstr) return -1; + + if (qemuCommandAddExtDevice(cmd, &shmem->info) < 0) { + VIR_FREE(devstr); + return -1; + } + virCommandAddArgList(cmd, "-device", devstr, NULL); VIR_FREE(devstr); @@ -10183,6 +10283,10 @@ qemuBuildVsockCommandLine(virCommandPtr cmd, virCommandPassFD(cmd, priv->vhostfd, VIR_COMMAND_PASS_FD_CLOSE_PARENT); priv->vhostfd = -1; + + if (qemuCommandAddExtDevice(cmd, &vsock->info) < 0) + goto cleanup; + virCommandAddArgList(cmd, "-device", devstr, NULL); ret = 0; diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index 98d4ac90b5..d382cd592a 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -174,6 +174,8 @@ char *qemuBuildRedirdevDevStr(const virDomainDef *def, virDomainRedirdevDefPtr dev, virQEMUCapsPtr qemuCaps); +char *qemuBuildZPCIDevStr(virDomainDeviceInfoPtr dev); + int qemuNetworkPrepareDevices(virDomainDefPtr def); int qemuGetDriveSourceString(virStorageSourcePtr src, diff --git a/tests/qemuxml2argvdata/disk-virtio-s390-zpci.args b/tests/qemuxml2argvdata/disk-virtio-s390-zpci.args index 8ac435cb3e..3daa8316b6 100644 --- a/tests/qemuxml2argvdata/disk-virtio-s390-zpci.args +++ b/tests/qemuxml2argvdata/disk-virtio-s390-zpci.args @@ -20,6 +20,7 @@ server,nowait \ -rtc base=utc \ -no-shutdown \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-virtio-disk0 \ +-device zpci,uid=25,fid=31,target=virtio-disk0,id=zpci25 \ -device virtio-blk-pci,bus=pci.0,addr=0x8,drive=drive-virtio-disk0,\ id=virtio-disk0,bootindex=1 \ -device virtio-balloon-ccw,id=balloon0,devno=fe.0.0000 diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args b/tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args new file mode 100644 index 0000000000..4309cdf2be --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args @@ -0,0 +1,25 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/bin/qemu-system-s390x \ +-name QEMUGuest1 \ +-S \ +-machine s390-ccw-virtio,accel=tcg,usb=off,dump-guest-core=off \ +-m 214 \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-QEMUGuest1/monitor.sock,\ +server,nowait \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-device zpci,uid=1,fid=0,target=hostdev0,id=zpci1 \ +-device vfio-pci,host=00:00.0,id=hostdev0,bus=pci.0,addr=0x1 \ +-device zpci,uid=2,fid=1,target=balloon0,id=zpci2 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2 diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.xml b/tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.xml new file mode 100644 index 0000000000..36161006ab --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.xml @@ -0,0 +1,18 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory>219100</memory> + <os> + <type arch='s390x' machine='s390-ccw-virtio'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-s390x</emulator> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci'/> + </hostdev> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args b/tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args new file mode 100644 index 0000000000..01b06837c8 --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args @@ -0,0 +1,29 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/bin/qemu-system-s390x \ +-name QEMUGuest1 \ +-S \ +-machine s390-ccw-virtio,accel=tcg,usb=off,dump-guest-core=off \ +-m 214 \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-QEMUGuest1/monitor.sock,\ +server,nowait \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-device zpci,uid=3,fid=2,target=pci.1,id=zpci3 \ +-device pci-bridge,chassis_nr=1,id=pci.1,bus=pci.0,addr=0x1 \ +-device zpci,uid=65535,fid=4294967295,target=hostdev0,id=zpci65535 \ +-device vfio-pci,host=ffff:00:00.0,id=hostdev0,bus=pci.1,addr=0x1f \ +-device zpci,uid=1,fid=0,target=hostdev1,id=zpci1 \ +-device vfio-pci,host=00:00.0,id=hostdev1,bus=pci.0,addr=0x2 \ +-device zpci,uid=2,fid=1,target=balloon0,id=zpci2 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.xml b/tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.xml new file mode 100644 index 0000000000..1e6060345b --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.xml @@ -0,0 +1,30 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory>219100</memory> + <os> + <type arch='s390x' machine='s390-ccw-virtio'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-s390x</emulator> + <controller type='pci' index='0' model='pci-root'/> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0xffff' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x01' slot='0x1f' function='0x0'> + <zpci uid='0xffff' fid='0xffffffff'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci'> + <zpci uid='0x0001' fid='0x00000000'/> + </address> + </hostdev> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.args b/tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.args new file mode 100644 index 0000000000..60b6a2e0d2 --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.args @@ -0,0 +1,39 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/bin/qemu-system-s390x \ +-name QEMUGuest1 \ +-S \ +-machine s390-ccw-virtio,accel=tcg,usb=off,dump-guest-core=off \ +-m 214 \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-QEMUGuest1/monitor.sock,\ +server,nowait \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-device zpci,uid=35,fid=63,target=hostdev0,id=zpci35 \ +-device vfio-pci,host=0001:00:00.0,id=hostdev0,bus=pci.0,addr=0x3 \ +-device zpci,uid=53,fid=104,target=hostdev1,id=zpci53 \ +-device vfio-pci,host=0002:00:00.0,id=hostdev1,bus=pci.0,addr=0x1 \ +-device zpci,uid=1,fid=1,target=hostdev2,id=zpci1 \ +-device vfio-pci,host=0003:00:00.0,id=hostdev2,bus=pci.0,addr=0x2 \ +-device zpci,uid=2,fid=2,target=hostdev3,id=zpci2 \ +-device vfio-pci,host=0004:00:00.0,id=hostdev3,bus=pci.0,addr=0x5 \ +-device zpci,uid=83,fid=0,target=hostdev4,id=zpci83 \ +-device vfio-pci,host=0005:00:00.0,id=hostdev4,bus=pci.0,addr=0x7 \ +-device zpci,uid=3,fid=114,target=hostdev5,id=zpci3 \ +-device vfio-pci,host=0006:00:00.0,id=hostdev5,bus=pci.0,addr=0x9 \ +-device zpci,uid=23,fid=3,target=hostdev6,id=zpci23 \ +-device vfio-pci,host=0007:00:00.0,id=hostdev6,bus=pci.0,addr=0x4 \ +-device zpci,uid=4,fid=40,target=hostdev7,id=zpci4 \ +-device vfio-pci,host=0008:00:00.0,id=hostdev7,bus=pci.0,addr=0x6 \ +-device zpci,uid=5,fid=4,target=balloon0,id=zpci5 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.xml b/tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.xml new file mode 100644 index 0000000000..da8305dd6d --- /dev/null +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.xml @@ -0,0 +1,79 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory>219100</memory> + <os> + <type arch='s390x' machine='s390-ccw-virtio'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-s390x</emulator> + <controller type='pci' index='0' model='pci-root'/> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0x0001' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'> + <zpci uid='0x0023' fid='0x0000003f'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0x0002' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci'> + <zpci uid='0x0035' fid='0x00000068'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0x0003' bus='0x00' slot='0x00' function='0x0'/> + </source> + </hostdev> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0x0004' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </hostdev> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0x0005' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'> + <zpci uid='0x0053'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0x0006' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'> + <zpci uid='0x0003' fid='0x00000072'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0x0007' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci'> + <zpci uid='0x0017' fid='0x00000003'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci'> + <driver name='vfio'/> + <source> + <address domain='0x0008' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci'> + <zpci uid='0x0004' fid='0x00000028'/> + </address> + </hostdev> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci.args b/tests/qemuxml2argvdata/hostdev-vfio-zpci.args index d6b1520c47..7ca4257a34 100644 --- a/tests/qemuxml2argvdata/hostdev-vfio-zpci.args +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci.args @@ -19,5 +19,7 @@ server,nowait \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=utc \ -no-shutdown \ +-device zpci,uid=25,fid=31,target=hostdev0,id=zpci25 \ -device vfio-pci,host=00:00.0,id=hostdev0,bus=pci.0,addr=0x8 \ +-device zpci,uid=1,fid=0,target=balloon0,id=zpci1 \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x1 diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 09c52f5f38..bf164f5f8d 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1688,6 +1688,19 @@ mymain(void) DO_TEST("hostdev-vfio-zpci", QEMU_CAPS_DEVICE_VFIO_PCI, QEMU_CAPS_DEVICE_ZPCI); + DO_TEST("hostdev-vfio-zpci-multidomain-many", + QEMU_CAPS_DEVICE_VFIO_PCI, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_ZPCI); + DO_TEST("hostdev-vfio-zpci-autogenerate", + QEMU_CAPS_DEVICE_VFIO_PCI, + QEMU_CAPS_DEVICE_ZPCI); + DO_TEST("hostdev-vfio-zpci-boundaries", + QEMU_CAPS_DEVICE_VFIO_PCI, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_ZPCI); + DO_TEST_PARSE_ERROR("hostdev-vfio-zpci", + QEMU_CAPS_DEVICE_VFIO_PCI); DO_TEST("pci-rom", NONE); DO_TEST("pci-rom-disabled", NONE); DO_TEST("pci-rom-disabled-invalid", NONE); diff --git a/tests/qemuxml2xmloutdata/hostdev-vfio-zpci-autogenerate.xml b/tests/qemuxml2xmloutdata/hostdev-vfio-zpci-autogenerate.xml new file mode 100644 index 0000000000..e94e63bd0a --- /dev/null +++ b/tests/qemuxml2xmloutdata/hostdev-vfio-zpci-autogenerate.xml @@ -0,0 +1,34 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219100</memory> + <currentMemory unit='KiB'>219100</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='s390x' machine='s390-ccw-virtio'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-s390x</emulator> + <controller type='pci' index='0' model='pci-root'/> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> + <zpci uid='0x0001' fid='0x00000000'/> + </address> + </hostdev> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'> + <zpci uid='0x0002' fid='0x00000001'/> + </address> + </memballoon> + <panic model='s390'/> + </devices> +</domain> diff --git a/tests/qemuxml2xmloutdata/hostdev-vfio-zpci-boundaries.xml b/tests/qemuxml2xmloutdata/hostdev-vfio-zpci-boundaries.xml new file mode 100644 index 0000000000..81d2146188 --- /dev/null +++ b/tests/qemuxml2xmloutdata/hostdev-vfio-zpci-boundaries.xml @@ -0,0 +1,48 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219100</memory> + <currentMemory unit='KiB'>219100</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='s390x' machine='s390-ccw-virtio'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-s390x</emulator> + <controller type='pci' index='0' model='pci-root'/> + <controller type='pci' index='1' model='pci-bridge'> + <model name='pci-bridge'/> + <target chassisNr='1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0xffff' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x01' slot='0x1f' function='0x0'> + <zpci uid='0xffff' fid='0xffffffff'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'> + <zpci uid='0x0001' fid='0x00000000'/> + </address> + </hostdev> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'> + <zpci uid='0x0002' fid='0x00000001'/> + </address> + </memballoon> + <panic model='s390'/> + </devices> +</domain> diff --git a/tests/qemuxml2xmloutdata/hostdev-vfio-zpci-multidomain-many.xml b/tests/qemuxml2xmloutdata/hostdev-vfio-zpci-multidomain-many.xml new file mode 100644 index 0000000000..e56106d103 --- /dev/null +++ b/tests/qemuxml2xmloutdata/hostdev-vfio-zpci-multidomain-many.xml @@ -0,0 +1,97 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219100</memory> + <currentMemory unit='KiB'>219100</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='s390x' machine='s390-ccw-virtio'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-s390x</emulator> + <controller type='pci' index='0' model='pci-root'/> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0001' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'> + <zpci uid='0x0023' fid='0x0000003f'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0002' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> + <zpci uid='0x0035' fid='0x00000068'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0003' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'> + <zpci uid='0x0001' fid='0x00000001'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0004' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'> + <zpci uid='0x0002' fid='0x00000002'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0005' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'> + <zpci uid='0x0053' fid='0x00000000'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0006' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'> + <zpci uid='0x0003' fid='0x00000072'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0007' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'> + <zpci uid='0x0017' fid='0x00000003'/> + </address> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <driver name='vfio'/> + <source> + <address domain='0x0008' bus='0x00' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'> + <zpci uid='0x0004' fid='0x00000028'/> + </address> + </hostdev> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'> + <zpci uid='0x0005' fid='0x00000004'/> + </address> + </memballoon> + <panic model='s390'/> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 53de9726f4..4d1a3610ef 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -464,6 +464,17 @@ mymain(void) DO_TEST("hostdev-vfio-zpci", QEMU_CAPS_DEVICE_ZPCI, QEMU_CAPS_CCW); + DO_TEST("hostdev-vfio-zpci-multidomain-many", + QEMU_CAPS_DEVICE_VFIO_PCI, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_ZPCI); + DO_TEST("hostdev-vfio-zpci-autogenerate", + QEMU_CAPS_DEVICE_VFIO_PCI, + QEMU_CAPS_DEVICE_ZPCI); + DO_TEST("hostdev-vfio-zpci-boundaries", + QEMU_CAPS_DEVICE_VFIO_PCI, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_ZPCI); DO_TEST("hostdev-mdev-precreated", NONE); DO_TEST("hostdev-mdev-display", QEMU_CAPS_VFIO_PCI_DISPLAY); DO_TEST("pci-rom", NONE); -- Yi Min
This commit adds hotplug support for PCI devices on S390 guests. There's no need to implement hot unplug for zPCI as QEMU implements an unplug callback which will unplug both PCI and zPCI device in a cascaded way. Currently, the following PCI devices are supported: virtio-blk-pci virtio-net-pci virtio-rng-pci virtio-input-host-pci virtio-keyboard-pci virtio-mouse-pci virtio-tablet-pci vfio-pci SCSIVhost device Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- src/qemu/qemu_hotplug.c | 160 +++++++++++++++++++++++++++++++++++++--- 1 file changed, 151 insertions(+), 9 deletions(-) diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 0a63741b9e..5f756b7267 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -154,6 +154,80 @@ qemuHotplugPrepareDiskAccess(virQEMUDriverPtr driver, } +static int +qemuDomainAttachZPCIDevice(qemuMonitorPtr mon, + virDomainDeviceInfoPtr info) +{ + char *devstr_zpci = NULL; + int ret = -1; + + if (!(devstr_zpci = qemuBuildZPCIDevStr(info))) + goto cleanup; + + if (qemuMonitorAddDevice(mon, devstr_zpci) < 0) + goto cleanup; + + ret = 0; + + cleanup: + VIR_FREE(devstr_zpci); + return ret; +} + + +static int +qemuDomainDetachZPCIDevice(qemuMonitorPtr mon, + virDomainDeviceInfoPtr info) +{ + char *zpciAlias = NULL; + int ret = -1; + + if (virAsprintf(&zpciAlias, "zpci%d", info->addr.pci.zpci.uid) < 0) + goto cleanup; + + if (qemuMonitorDelDevice(mon, zpciAlias) < 0) + goto cleanup; + + ret = 0; + + cleanup: + VIR_FREE(zpciAlias); + return ret; +} + + +static int +qemuDomainAttachExtensionDevice(qemuMonitorPtr mon, + virDomainDeviceInfoPtr info) +{ + if (info->type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI || + info->addr.pci.extFlags == VIR_PCI_ADDRESS_EXTENSION_NONE) { + return 0; + } + + if (info->addr.pci.extFlags & VIR_PCI_ADDRESS_EXTENSION_ZPCI) + return qemuDomainAttachZPCIDevice(mon, info); + + return 0; +} + + +static int +qemuDomainDetachExtensionDevice(qemuMonitorPtr mon, + virDomainDeviceInfoPtr info) +{ + if (info->type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI || + info->addr.pci.extFlags == VIR_PCI_ADDRESS_EXTENSION_NONE) { + return 0; + } + + if (info->addr.pci.extFlags & VIR_PCI_ADDRESS_EXTENSION_ZPCI) + return qemuDomainDetachZPCIDevice(mon, info); + + return 0; +} + + static int qemuHotplugWaitForTrayEject(virDomainObjPtr vm, virDomainDiskDefPtr disk) @@ -845,8 +919,13 @@ qemuDomainAttachDiskGeneric(virQEMUDriverPtr driver, if (qemuHotplugDiskSourceAttach(priv->mon, diskdata) < 0) goto exit_monitor; - if (qemuMonitorAddDevice(priv->mon, devstr) < 0) + if (qemuDomainAttachExtensionDevice(priv->mon, &disk->info) < 0) + goto exit_monitor; + + if (qemuMonitorAddDevice(priv->mon, devstr) < 0) { + ignore_value(qemuDomainDetachExtensionDevice(priv->mon, &disk->info)); goto exit_monitor; + } if (qemuDomainObjExitMonitor(driver, vm) < 0) { ret = -2; @@ -954,7 +1033,16 @@ int qemuDomainAttachControllerDevice(virQEMUDriverPtr driver, goto cleanup; qemuDomainObjEnterMonitor(driver, vm); - ret = qemuMonitorAddDevice(priv->mon, devstr); + + if ((ret = qemuDomainAttachExtensionDevice(priv->mon, + &controller->info)) < 0) { + goto exit_monitor; + } + + if ((ret = qemuMonitorAddDevice(priv->mon, devstr)) < 0) + ignore_value(qemuDomainDetachExtensionDevice(priv->mon, &controller->info)); + + exit_monitor: if (qemuDomainObjExitMonitor(driver, vm) < 0) { releaseaddr = false; ret = -1; @@ -1417,6 +1505,7 @@ qemuDomainAttachNetDevice(virQEMUDriverPtr driver, } if (qemuDomainIsS390CCW(vm->def) && + net->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI && virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_CCW)) { net->info.type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCW; if (!(ccwaddrs = virDomainCCWAddressSetCreateFromDomain(vm->def))) @@ -1486,7 +1575,15 @@ qemuDomainAttachNetDevice(virQEMUDriverPtr driver, goto try_remove; qemuDomainObjEnterMonitor(driver, vm); + + if (qemuDomainAttachExtensionDevice(priv->mon, &net->info) < 0) { + ignore_value(qemuDomainObjExitMonitor(driver, vm)); + virDomainAuditNet(vm, NULL, net, "attach", false); + goto try_remove; + } + if (qemuMonitorAddDevice(priv->mon, nicstr) < 0) { + ignore_value(qemuDomainDetachExtensionDevice(priv->mon, &net->info)); ignore_value(qemuDomainObjExitMonitor(driver, vm)); virDomainAuditNet(vm, NULL, net, "attach", false); goto try_remove; @@ -1703,8 +1800,16 @@ qemuDomainAttachHostPCIDevice(virQEMUDriverPtr driver, goto error; qemuDomainObjEnterMonitor(driver, vm); - ret = qemuMonitorAddDeviceWithFd(priv->mon, devstr, - configfd, configfd_name); + + if ((ret = qemuDomainAttachExtensionDevice(priv->mon, hostdev->info)) < 0) + goto exit_monitor; + + if ((ret = qemuMonitorAddDeviceWithFd(priv->mon, devstr, + configfd, configfd_name)) < 0) { + ignore_value(qemuDomainDetachExtensionDevice(priv->mon, hostdev->info)); + } + + exit_monitor: if (qemuDomainObjExitMonitor(driver, vm) < 0) goto error; @@ -2360,9 +2465,14 @@ qemuDomainAttachRNGDevice(virQEMUDriverPtr driver, if (qemuMonitorAddObject(priv->mon, &props, &objAlias) < 0) goto exit_monitor; - if (qemuMonitorAddDevice(priv->mon, devstr) < 0) + if (qemuDomainAttachExtensionDevice(priv->mon, &rng->info) < 0) goto exit_monitor; + if (qemuMonitorAddDevice(priv->mon, devstr) < 0) { + ignore_value(qemuDomainDetachExtensionDevice(priv->mon, &rng->info)); + goto exit_monitor; + } + if (qemuDomainObjExitMonitor(driver, vm) < 0) { releaseaddr = false; goto cleanup; @@ -2842,8 +2952,16 @@ qemuDomainAttachSCSIVHostDevice(virQEMUDriverPtr driver, qemuDomainObjEnterMonitor(driver, vm); - ret = qemuMonitorAddDeviceWithFd(priv->mon, devstr, vhostfd, vhostfdName); + if ((ret = qemuDomainAttachExtensionDevice(priv->mon, hostdev->info)) < 0) + goto exit_monitor; + + if ((ret = qemuMonitorAddDeviceWithFd(priv->mon, devstr, vhostfd, + vhostfdName)) < 0) { + ignore_value(qemuDomainDetachExtensionDevice(priv->mon, hostdev->info)); + goto exit_monitor; + } + exit_monitor: if (qemuDomainObjExitMonitor(driver, vm) < 0 || ret < 0) goto audit; @@ -3088,8 +3206,13 @@ qemuDomainAttachShmemDevice(virQEMUDriverPtr driver, release_backing = true; - if (qemuMonitorAddDevice(priv->mon, shmstr) < 0) + if (qemuDomainAttachExtensionDevice(priv->mon, &shmem->info) < 0) + goto exit_monitor; + + if (qemuMonitorAddDevice(priv->mon, shmstr) < 0) { + ignore_value(qemuDomainDetachExtensionDevice(priv->mon, &shmem->info)); goto exit_monitor; + } if (qemuDomainObjExitMonitor(driver, vm) < 0) { release_address = false; @@ -3262,9 +3385,15 @@ qemuDomainAttachInputDevice(virQEMUDriverPtr driver, goto cleanup; qemuDomainObjEnterMonitor(driver, vm); - if (qemuMonitorAddDevice(priv->mon, devstr) < 0) + + if (qemuDomainAttachExtensionDevice(priv->mon, &input->info) < 0) goto exit_monitor; + if (qemuMonitorAddDevice(priv->mon, devstr) < 0) { + ignore_value(qemuDomainDetachExtensionDevice(priv->mon, &input->info)); + goto exit_monitor; + } + if (qemuDomainObjExitMonitor(driver, vm) < 0) { releaseaddr = false; goto cleanup; @@ -3341,8 +3470,14 @@ qemuDomainAttachVsockDevice(virQEMUDriverPtr driver, goto cleanup; qemuDomainObjEnterMonitor(driver, vm); - if (qemuMonitorAddDeviceWithFd(priv->mon, devstr, vsockPriv->vhostfd, fdname) < 0) + + if (qemuDomainAttachExtensionDevice(priv->mon, &vsock->info) < 0) + goto exit_monitor; + + if (qemuMonitorAddDeviceWithFd(priv->mon, devstr, vsockPriv->vhostfd, fdname) < 0) { + ignore_value(qemuDomainDetachExtensionDevice(priv->mon, &vsock->info)); goto exit_monitor; + } if (qemuDomainObjExitMonitor(driver, vm) < 0) { releaseaddr = false; @@ -5328,10 +5463,17 @@ int qemuDomainDetachControllerDevice(virQEMUDriverPtr driver, qemuDomainMarkDeviceForRemoval(vm, &detach->info); qemuDomainObjEnterMonitor(driver, vm); + if (detach->type == VIR_DOMAIN_CONTROLLER_TYPE_PCI && + qemuDomainDetachExtensionDevice(priv->mon, &detach->info)) { + goto exit_monitor; + } + if (qemuMonitorDelDevice(priv->mon, detach->info.alias)) { ignore_value(qemuDomainObjExitMonitor(driver, vm)); goto cleanup; } + + exit_monitor: if (qemuDomainObjExitMonitor(driver, vm) < 0) goto cleanup; -- Yi Min
Update 'Device address' section to describe 'zpci' element and its two attributes 'uid' and 'fid'. Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- docs/formatdomain.html.in | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 8189959773..9ca4975d10 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -3925,7 +3925,15 @@ (<span class="since">since 0.9.7, requires QEMU 0.13</span>). <code>multifunction</code> defaults to 'off', but should be set to 'on' for function 0 of a slot that will - have multiple functions used.<br/> + have multiple functions used. + (<span class="since">Since 4.9.0</span>), PCI address extensions + depending on the architecture are supported. For example, PCI + addresses for S390 guests will have a <code>zpci</code> child + element, with two attributes: <code>uid</code> (a hex value + between 0x0001 and 0xffff, inclusive), and <code>fid</code> (a + hex value between 0x00000000 and 0xffffffff, inclusive) used by + PCI devices on S390 for User-defined Identifiers and Function + Identifiers.<br/> <span class="since">Since 1.3.5</span>, some hypervisor drivers may accept an <code><address type='pci'/></code> element with no other attributes as an explicit request to -- Yi Min
On Fri, 2018-10-19 at 11:40 +0800, Yi Min Zhao wrote: [...]
@@ -3925,7 +3925,15 @@ (<span class="since">since 0.9.7, requires QEMU 0.13</span>). <code>multifunction</code> defaults to 'off', but should be set to 'on' for function 0 of a slot that will - have multiple functions used.<br/> + have multiple functions used. + (<span class="since">Since 4.9.0</span>), PCI address extensions
Since 4.10.0 now ;) -- Andrea Bolognani / Red Hat / Virtualization
在 2018/11/5 下午9:22, Andrea Bolognani 写道:
On Fri, 2018-10-19 at 11:40 +0800, Yi Min Zhao wrote: [...]
@@ -3925,7 +3925,15 @@ (<span class="since">since 0.9.7, requires QEMU 0.13</span>). <code>multifunction</code> defaults to 'off', but should be set to 'on' for function 0 of a slot that will - have multiple functions used.<br/> + have multiple functions used. + (<span class="since">Since 4.9.0</span>), PCI address extensions Since 4.10.0 now ;)
Yes. -- Yi Min
Signed-off-by: Yi Min Zhao <zyimin@linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com> Reviewed-by: Stefan Zimmermann <stzi@linux.ibm.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> --- docs/news.xml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index dc08c96352..6c2e0f27f1 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -35,6 +35,17 @@ <libvirt> <release version="v4.9.0" date="unreleased"> <section title="New features"> + <change> + <summary> + qemu: Added support for PCI device on S390 + </summary> + <description> + PCI addresses can now include the new zpci element which contains + uid (user-defined identifier) and fid (PCI function identifier) + attributes and makes the corresponding devices usable by S390 + guests. + </description> + </change> </section> <section title="Improvements"> </section> -- Yi Min
On Fri, 2018-10-19 at 11:40 +0800, Yi Min Zhao wrote: [...]
<section title="New features"> + <change> + <summary> + qemu: Added support for PCI device on S390 + </summary>
s/device/devices/ -- Andrea Bolognani / Red Hat / Virtualization
Kindly ping. @Jan, could you please take a look? @Andrea, I have updated the code as your comments on previous version. 在 2018/10/19 上午11:40, Yi Min Zhao 写道:
Abstract ======== The PCI representation in QEMU has been extended for S390 allowing configuration of zPCI attributes like uid (user-defined identifier) and fid (PCI function identifier). The details can be found here: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg07262.html
To support the new zPCI feature of the S390 platform, a new element of PCI address is introduced. It has two optional attributes, @uid and @fid. For example: <hostdev mode='subsystem' type='pci'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x00' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> <zpci uid='0x0003' fid='0x00000027'/> </address> </hostdev>
If they are defined by the user, unique values within the guest domain must be used. If they are not specified and the architecture requires them, they are automatically generated with non-conflicting values.
zPCI address as an extension of the PCI address are stored in a new structure 'virZPCIDeviceAddress' which is a member of common PCI Address structure. Additionally, two hashtables are used for assignment and reservation of zPCI uid/fid.
In support of extending the PCI address, a new PCI address extension flag is introduced. This extension flag allows is not only dedicated for the S390 platform but also other architectures needing certain extensions to PCI address space.
Code Base ========= commit in master: 3a1cdb06fd conf: Fix bug in finding alloc through matching vcpus
Change Log ========== v6->v7: 1. Optimize some functions' names and code logic. 2. Fixup build error. 3. Add negative test case for patch 9. 4. Use virXMLFormatElement() in virDomainDeviceInfoFormat().
v5->v6: 1. Modify zPCI XML definition. 2. Optimize the logic of zPCI address assignment and reservation. 3. Add extension flag into PCI address structure. 4. Update commit messages.
v4->v5: 1. Update the version number. 2. Fixup code style error. 3. Separate qemu code into single patch. 4. Rebase the patches to the new code of master branch.
v3->v4: 1. Update docs. 2. Format code style. 3. Optimize zPCI support check. 4. Move the check of zPCI defined in xml but unsupported by Qemu to qemuDomainDeviceDefValidate(). 5. Change zpci address member of PCI address struct from pointer to instance. 6. Modify zpci address definition principle. Currently the user must either define both of uid and fid or not.
v2->v3: 1. Revise code style. 2. Update test cases. 3. Introduce qemuDomainCollectPCIAddressExtension() to collect PCI extension addresses. 4. Introduce virDeviceInfoPCIAddressExtensionPresent() to check if zPCI address exists. 5. Optimize zPCI address check logic. 6. Optimize passed parameters of zPCI addr alloc/release/reserve functions. 7. Report enum range error in qemuDomainDeviceSupportZPCI(). 8. Update commit messages.
v1->v2: 1. Separate test commit and merge testcases into corresponding commits that introduce the functionalities firstly. 2. Spare some checks for zpci device. 3. Add vsock and controller support. 4. Add uin32 type schema. 5. Rename zpciuid and zpcifid to zpci_uid and zpci_fid. 6. Always return multibus support on S390.
Yi Min Zhao (14): conf: Add definitions for 'uid' and 'fid' PCI address attributes qemu: Introduce zPCI capability conf: Introduce a new PCI address extension flag qemu: Enable PCI multi bus for S390 guests qemu: Auto add pci-root for s390/s390x guests conf: Introduce address caching for PCI extensions conf: use virXMLFormatElement() in virDomainDeviceInfoFormat() conf: Introduce parser, formatter for uid and fid qemu: Add zPCI address definition check conf: Allocate/release 'uid' and 'fid' in PCI address qemu: Generate and use zPCI device in QEMU command line qemu: Add hotpluging support for PCI devices on S390 guests docs: Add 'uid' and 'fid' information news: Update news for PCI address extension attributes
cfg.mk | 1 + docs/formatdomain.html.in | 10 +- docs/news.xml | 11 + docs/schemas/basictypes.rng | 27 ++ docs/schemas/domaincommon.rng | 1 + src/bhyve/bhyve_device.c | 3 +- src/conf/device_conf.c | 69 ++++ src/conf/device_conf.h | 7 + src/conf/domain_addr.c | 340 +++++++++++++++++- src/conf/domain_addr.h | 27 +- src/conf/domain_conf.c | 50 ++- src/libvirt_private.syms | 7 + src/qemu/qemu_capabilities.c | 6 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 104 ++++++ src/qemu/qemu_command.h | 2 + src/qemu/qemu_domain.c | 37 ++ src/qemu/qemu_domain_address.c | 205 ++++++++++- src/qemu/qemu_hotplug.c | 160 ++++++++- src/util/virpci.c | 26 ++ src/util/virpci.h | 15 + .../caps_2.10.0.s390x.xml | 1 + .../caps_2.11.0.s390x.xml | 1 + .../caps_2.12.0.s390x.xml | 1 + .../qemucapabilitiesdata/caps_2.7.0.s390x.xml | 1 + .../qemucapabilitiesdata/caps_2.8.0.s390x.xml | 1 + .../qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + .../qemucapabilitiesdata/caps_3.0.0.s390x.xml | 1 + .../disk-virtio-s390-zpci.args | 26 ++ .../disk-virtio-s390-zpci.xml | 19 + .../hostdev-vfio-zpci-autogenerate.args | 25 ++ .../hostdev-vfio-zpci-autogenerate.xml | 18 + .../hostdev-vfio-zpci-boundaries.args | 29 ++ .../hostdev-vfio-zpci-boundaries.xml | 30 ++ .../hostdev-vfio-zpci-multidomain-many.args | 39 ++ .../hostdev-vfio-zpci-multidomain-many.xml | 79 ++++ .../hostdev-vfio-zpci-wrong-arch.xml | 34 ++ tests/qemuxml2argvdata/hostdev-vfio-zpci.args | 25 ++ tests/qemuxml2argvdata/hostdev-vfio-zpci.xml | 21 ++ tests/qemuxml2argvtest.c | 22 ++ .../disk-virtio-s390-zpci.xml | 31 ++ .../hostdev-vfio-zpci-autogenerate.xml | 34 ++ .../hostdev-vfio-zpci-boundaries.xml | 48 +++ .../hostdev-vfio-zpci-multidomain-many.xml | 97 +++++ .../qemuxml2xmloutdata/hostdev-vfio-zpci.xml | 32 ++ tests/qemuxml2xmltest.c | 17 + 46 files changed, 1707 insertions(+), 35 deletions(-) create mode 100644 tests/qemuxml2argvdata/disk-virtio-s390-zpci.args create mode 100644 tests/qemuxml2argvdata/disk-virtio-s390-zpci.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-multidomain-many.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-wrong-arch.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci.xml create mode 100644 tests/qemuxml2xmloutdata/disk-virtio-s390-zpci.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci-autogenerate.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci-boundaries.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci-multidomain-many.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-vfio-zpci.xml
-- Yi Min
On Mon, 2018-10-22 at 13:31 +0800, Yi Min Zhao wrote:
Kindly ping.
@Jan, could you please take a look?
@Andrea, I have updated the code as your comments on previous version.
Most libvirt developers are attending KVM Forum right now, so I wouldn't expect any further feedback until November. Sorry about that! -- Andrea Bolognani / Red Hat / Virtualization
On Fri, 2018-10-19 at 11:40 +0800, Yi Min Zhao wrote:
Abstract ======== The PCI representation in QEMU has been extended for S390 allowing configuration of zPCI attributes like uid (user-defined identifier) and fid (PCI function identifier). The details can be found here: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg07262.html
To support the new zPCI feature of the S390 platform, a new element of PCI address is introduced. It has two optional attributes, @uid and @fid. For example: <hostdev mode='subsystem' type='pci'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x00' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> <zpci uid='0x0003' fid='0x00000027'/> </address> </hostdev>
If they are defined by the user, unique values within the guest domain must be used. If they are not specified and the architecture requires them, they are automatically generated with non-conflicting values.
zPCI address as an extension of the PCI address are stored in a new structure 'virZPCIDeviceAddress' which is a member of common PCI Address structure. Additionally, two hashtables are used for assignment and reservation of zPCI uid/fid.
In support of extending the PCI address, a new PCI address extension flag is introduced. This extension flag allows is not only dedicated for the S390 platform but also other architectures needing certain extensions to PCI address space.
So, I was able to sit down with Laine during KVM Forum and we looked over the entire series together. He didn't spot any issues with it except for the couple minor ones that I pointed out separately in response to the corresponding patches; as for myself, I've already provided R-bs for all patches and I have no further objections to this being merged. I also asked Dan to weigh in explicitly on the list regarding his concerns about the XML design, and he promised he'd do that... I think you can post a v8 which addresses the small issues mentioned above (and rebased on top of current master) in the meantime. -- Andrea Bolognani / Red Hat / Virtualization
participants (2)
-
Andrea Bolognani -
Yi Min Zhao