[libvirt] [PATCH v6 00/17] Use more PCIe less legacy PCI
This is a rebase of v5 (, with the addition of patches 16 & 17, which detect the type of VFIO assigned devices and assign them to either an Express or a legacy PCI slot as appropriate, Laine Stump (17): qemu: new functions qemuDomainMachineHasPCI[e]Root() qemu: new functions to calculate/set device pciConnectFlags qemu: set/use info->pciConnectFlags when validating/assigning PCI addresses qemu: set/use proper pciConnectFlags during hotplug qemu: set pciConnectFlags to 0 instead of PCI|HOTPLUGGABLE if device isn't PCI qemu: assign virtio devices to PCIe slot when appropriate qemu: assign e1000e network devices to PCIe slots when appropriate qemu: assign nec-xhci (USB3) controller to a PCIe address when appropriate qemu: only force an available legacy-PCI slot on domains with pci-root qemu: auto-add pcie-root-port/dmi-to-pci-bridge controllers as needed [RFC] qemu: if pci-bridge is in PCIe config w/o dmi-to-pci-bridge, add it qemu: don't force-add a dmi-to-pci-bridge just on principle qemu: add a USB3 controller to Q35 domains by default qemu: try to put ich9 sound device at 00:1B.0 qemu: initially reserve one open pcie-root-port for hotplug qemu: propagate virQEMUDriver object to qemuDomainDeviceCalculatePCIConnectFlags qemu: assign vfio devices to PCIe addresses when appropriate src/conf/device_conf.h | 5 + src/conf/domain_addr.c | 112 ++- src/conf/domain_addr.h | 3 +- src/qemu/qemu_domain.c | 54 +- src/qemu/qemu_domain.h | 2 + src/qemu/qemu_domain_address.c | 880 +++++++++++++++++---- src/qemu/qemu_domain_address.h | 9 +- src/qemu/qemu_hotplug.c | 28 +- src/qemu/qemu_process.c | 13 +- tests/qemuhotplugtest.c | 4 +- tests/qemuxml2argvdata/qemuxml2argv-autoindex.args | 10 +- .../qemuxml2argv-bios-nvram-secure.args | 1 + .../qemuxml2argv-machine-smm-opt.args | 1 + tests/qemuxml2argvdata/qemuxml2argv-pcie-root.args | 3 +- .../qemuxml2argv-q35-default-devices-only.args | 23 + .../qemuxml2argv-q35-default-devices-only.xml | 18 + .../qemuxml2argv-q35-pcie-autoadd.args | 57 ++ .../qemuxml2argv-q35-pcie-autoadd.xml | 51 ++ tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args | 58 ++ tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml | 67 ++ .../qemuxml2argv-q35-pm-disable-fallback.args | 1 + .../qemuxml2argv-q35-pm-disable.args | 1 + .../qemuxml2argv-q35-virt-manager-basic.args | 56 ++ .../qemuxml2argv-q35-virt-manager-basic.xml | 76 ++ .../qemuxml2argv-q35-virtio-pci.args | 58 ++ .../qemuxml2argv-q35-virtio-pci.xml | 1 + tests/qemuxml2argvtest.c | 164 +++- .../qemuxml2xmlout-autoindex.xml | 10 +- .../qemuxml2xmlout-pcie-root.xml | 4 - .../qemuxml2xmlout-q35-default-devices-only.xml | 45 ++ .../qemuxml2xmlout-q35-pcie-autoadd.xml | 148 ++++ .../qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml | 152 ++++ .../qemuxml2xmlout-q35-virt-manager-basic.xml | 121 +++ .../qemuxml2xmlout-q35-virtio-pci.xml | 152 ++++ tests/qemuxml2xmltest.c | 136 +++- 35 files changed, 2285 insertions(+), 239 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args create mode 120000 tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-default-devices-only.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virt-manager-basic.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml -- 2.7.4
These functions provide a simple one line method of learning if the current domain has a pci-root or pcie-root bus. --- src/qemu/qemu_domain.c | 30 ++++++++++++++++++++++++++++++ src/qemu/qemu_domain.h | 2 ++ 2 files changed, 32 insertions(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 8cba755..bd2d167 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -5620,6 +5620,36 @@ qemuDomainMachineIsI440FX(const virDomainDef *def) bool +qemuDomainMachineHasPCIRoot(const virDomainDef *def) +{ + int root = virDomainControllerFind(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 0); + + if (root < 0) + return false; + + if (def->controllers[root]->model != VIR_DOMAIN_CONTROLLER_MODEL_PCI_ROOT) + return false; + + return true; +} + + +bool +qemuDomainMachineHasPCIeRoot(const virDomainDef *def) +{ + int root = virDomainControllerFind(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 0); + + if (root < 0) + return false; + + if (def->controllers[root]->model != VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) + return false; + + return true; +} + + +bool qemuDomainMachineNeedsFDC(const virDomainDef *def) { char *p = STRSKIP(def->os.machine, "pc-q35-"); diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 2ee1829..8a2e0fd 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -655,6 +655,8 @@ virDomainChrDefPtr qemuFindAgentConfig(virDomainDefPtr def); bool qemuDomainMachineIsQ35(const virDomainDef *def); bool qemuDomainMachineIsI440FX(const virDomainDef *def); +bool qemuDomainMachineHasPCIRoot(const virDomainDef *def); +bool qemuDomainMachineHasPCIeRoot(const virDomainDef *def); bool qemuDomainMachineNeedsFDC(const virDomainDef *def); bool qemuDomainMachineIsS390CCW(const virDomainDef *def); bool qemuDomainMachineIsVirt(const virDomainDef *def); -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
These functions provide a simple one line method of learning if the current domain has a pci-root or pcie-root bus. --- src/qemu/qemu_domain.c | 30 ++++++++++++++++++++++++++++++ src/qemu/qemu_domain.h | 2 ++ 2 files changed, 32 insertions(+)
I feel like it's very redundant at this point, but still: ACK -- Andrea Bolognani / Red Hat / Virtualization
The lowest level function of this trio (qemuDomainDeviceCalculatePCIConnectFlags()) aims to be the single authority for the virDomainPCIConnectFlags to use for any given device using a particular arch/machinetype/qemu-binary. qemuDomainFillDevicePCIConnectFlags() sets info->pciConnectFlags in a single device (unless it has no virDomainDeviceInfo, in which case it's a NOP). qemuDomainFillAllPCIConnectFlags() sets info->pciConnectFlags in all devices that have a virDomainDeviceInfo The latter two functions aren't called anywhere yet. This commit is just making them available. Later patches will replace all the current hodge-podge of flag settings with calls to this single authority. --- src/conf/device_conf.h | 5 + src/qemu/qemu_domain_address.c | 367 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 372 insertions(+) diff --git a/src/conf/device_conf.h b/src/conf/device_conf.h index 8443de6..f435fb5 100644 --- a/src/conf/device_conf.h +++ b/src/conf/device_conf.h @@ -142,6 +142,11 @@ typedef struct _virDomainDeviceInfo { /* bootIndex is only used for disk, network interface, hostdev * and redirdev devices */ unsigned int bootIndex; + + /* pciConnectFlags is only used internally during address + * assignment, never saved and never reported. + */ + int pciConnectFlags; /* enum virDomainPCIConnectFlags */ } virDomainDeviceInfo, *virDomainDeviceInfoPtr; diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index b35a95f..22cb402 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -409,6 +409,373 @@ qemuDomainAssignARMVirtioMMIOAddresses(virDomainDefPtr def, } +/** + * qemuDomainDeviceCalculatePCIConnectFlags: + * + * @dev: The device to be checked + * @pcieFlags: flags to use for a known PCI Express device + * @virtioFlags: flags to use for a virtio device (properly vetted + * for the current qemu binary and arch/machinetype) + * + * Lowest level function to determine PCI connectFlags for a + * device. This function relies on the next higher-level function + * determining the value for pcieFlags and virtioFlags in advance - + * this is to make it more efficient to call multiple times. + * + * Returns appropriate virDomainPCIConnectFlags for this device in + * this domain, or 0 if the device doesn't connect using PCI. There + * is no failure. + */ +static virDomainPCIConnectFlags +qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, + virDomainPCIConnectFlags pcieFlags + ATTRIBUTE_UNUSED, + virDomainPCIConnectFlags virtioFlags + ATTRIBUTE_UNUSED) +{ + virDomainPCIConnectFlags pciFlags = (VIR_PCI_CONNECT_TYPE_PCI_DEVICE | + VIR_PCI_CONNECT_HOTPLUGGABLE); + + switch ((virDomainDeviceType) dev->type) { + case VIR_DOMAIN_DEVICE_CONTROLLER: { + virDomainControllerDefPtr cont = dev->data.controller; + + switch ((virDomainControllerType) cont->type) { + case VIR_DOMAIN_CONTROLLER_TYPE_PCI: + return virDomainPCIControllerModelToConnectType(cont->model); + + case VIR_DOMAIN_CONTROLLER_TYPE_SATA: + return pciFlags; + + case VIR_DOMAIN_CONTROLLER_TYPE_USB: + switch ((virDomainControllerModelUSB) cont->model) { + case VIR_DOMAIN_CONTROLLER_MODEL_USB_NEC_XHCI: + return pciFlags; + + case VIR_DOMAIN_CONTROLLER_MODEL_USB_EHCI: + case VIR_DOMAIN_CONTROLLER_MODEL_USB_ICH9_EHCI1: + case VIR_DOMAIN_CONTROLLER_MODEL_USB_ICH9_UHCI1: + case VIR_DOMAIN_CONTROLLER_MODEL_USB_ICH9_UHCI2: + case VIR_DOMAIN_CONTROLLER_MODEL_USB_ICH9_UHCI3: + case VIR_DOMAIN_CONTROLLER_MODEL_USB_VT82C686B_UHCI: + case VIR_DOMAIN_CONTROLLER_MODEL_USB_PIIX3_UHCI: + case VIR_DOMAIN_CONTROLLER_MODEL_USB_PIIX4_UHCI: + case VIR_DOMAIN_CONTROLLER_MODEL_USB_PCI_OHCI: + return pciFlags; + + case VIR_DOMAIN_CONTROLLER_MODEL_USB_QUSB1: /* xen only */ + case VIR_DOMAIN_CONTROLLER_MODEL_USB_QUSB2: /* xen only */ + case VIR_DOMAIN_CONTROLLER_MODEL_USB_NONE: + case VIR_DOMAIN_CONTROLLER_MODEL_USB_LAST: + /* should be 0 */ + return pciFlags; + } + + case VIR_DOMAIN_CONTROLLER_TYPE_IDE: + case VIR_DOMAIN_CONTROLLER_TYPE_SCSI: + case VIR_DOMAIN_CONTROLLER_TYPE_VIRTIO_SERIAL: + return pciFlags; + + case VIR_DOMAIN_CONTROLLER_TYPE_FDC: + case VIR_DOMAIN_CONTROLLER_TYPE_CCID: + case VIR_DOMAIN_CONTROLLER_TYPE_LAST: + /* should be 0 */ + return pciFlags; + } + } + + case VIR_DOMAIN_DEVICE_FS: + /* the only type of filesystem so far is virtio-9p-pci */ + return pciFlags; + + case VIR_DOMAIN_DEVICE_NET: { + virDomainNetDefPtr net = dev->data.net; + + /* NB: a type='hostdev' will use PCI, but its + * address is assigned when we're assigning the + * addresses for other hostdev devices. + */ + if (net->type == VIR_DOMAIN_NET_TYPE_HOSTDEV || + STREQ(net->model, "usb-net")) { + /* should be 0 */ + return pciFlags; + } + return pciFlags; + } + + case VIR_DOMAIN_DEVICE_SOUND: + switch ((virDomainSoundModel) dev->data.sound->model) { + case VIR_DOMAIN_SOUND_MODEL_ES1370: + case VIR_DOMAIN_SOUND_MODEL_AC97: + case VIR_DOMAIN_SOUND_MODEL_ICH6: + case VIR_DOMAIN_SOUND_MODEL_ICH9: + return pciFlags; + + case VIR_DOMAIN_SOUND_MODEL_SB16: + case VIR_DOMAIN_SOUND_MODEL_PCSPK: + case VIR_DOMAIN_SOUND_MODEL_USB: + case VIR_DOMAIN_SOUND_MODEL_LAST: + /* should be 0 */ + return pciFlags; + } + + case VIR_DOMAIN_DEVICE_DISK: + switch ((virDomainDiskBus) dev->data.disk->bus) { + case VIR_DOMAIN_DISK_BUS_VIRTIO: + return pciFlags; /* only virtio disks use PCI */ + + case VIR_DOMAIN_DISK_BUS_IDE: + case VIR_DOMAIN_DISK_BUS_FDC: + case VIR_DOMAIN_DISK_BUS_SCSI: + case VIR_DOMAIN_DISK_BUS_XEN: + case VIR_DOMAIN_DISK_BUS_USB: + case VIR_DOMAIN_DISK_BUS_UML: + case VIR_DOMAIN_DISK_BUS_SATA: + case VIR_DOMAIN_DISK_BUS_SD: + case VIR_DOMAIN_DISK_BUS_LAST: + /* should be 0 */ + return pciFlags; + } + + case VIR_DOMAIN_DEVICE_HOSTDEV: + return pciFlags; + + case VIR_DOMAIN_DEVICE_MEMBALLOON: + switch ((virDomainMemballoonModel) dev->data.memballoon->model) { + case VIR_DOMAIN_MEMBALLOON_MODEL_VIRTIO: + return pciFlags; + + case VIR_DOMAIN_MEMBALLOON_MODEL_XEN: + case VIR_DOMAIN_MEMBALLOON_MODEL_NONE: + case VIR_DOMAIN_MEMBALLOON_MODEL_LAST: + /* should be 0 (not PCI) */ + return pciFlags; + } + + case VIR_DOMAIN_DEVICE_RNG: + switch ((virDomainRNGModel) dev->data.rng->model) { + case VIR_DOMAIN_RNG_MODEL_VIRTIO: + return pciFlags; + + case VIR_DOMAIN_RNG_MODEL_LAST: + /* should be 0 */ + return pciFlags; + } + + case VIR_DOMAIN_DEVICE_WATCHDOG: + /* only one model connects using PCI */ + switch ((virDomainWatchdogModel) dev->data.watchdog->model) { + case VIR_DOMAIN_WATCHDOG_MODEL_I6300ESB: + return pciFlags; + + case VIR_DOMAIN_WATCHDOG_MODEL_IB700: + case VIR_DOMAIN_WATCHDOG_MODEL_DIAG288: + case VIR_DOMAIN_WATCHDOG_MODEL_LAST: + /* should be 0 */ + return pciFlags; + } + + case VIR_DOMAIN_DEVICE_VIDEO: + switch ((virDomainVideoType) dev->data.video->type) { + case VIR_DOMAIN_VIDEO_TYPE_VIRTIO: + case VIR_DOMAIN_VIDEO_TYPE_VGA: + case VIR_DOMAIN_VIDEO_TYPE_CIRRUS: + case VIR_DOMAIN_VIDEO_TYPE_VMVGA: + case VIR_DOMAIN_VIDEO_TYPE_XEN: + case VIR_DOMAIN_VIDEO_TYPE_VBOX: + case VIR_DOMAIN_VIDEO_TYPE_QXL: + case VIR_DOMAIN_VIDEO_TYPE_PARALLELS: + return pciFlags; + + case VIR_DOMAIN_VIDEO_TYPE_LAST: + /* should be 0 */ + return pciFlags; + } + + + case VIR_DOMAIN_DEVICE_SHMEM: + return pciFlags; + + case VIR_DOMAIN_DEVICE_INPUT: + switch ((virDomainInputBus) dev->data.input->bus) { + case VIR_DOMAIN_INPUT_BUS_VIRTIO: + return pciFlags; + + case VIR_DOMAIN_INPUT_BUS_PS2: + case VIR_DOMAIN_INPUT_BUS_USB: + case VIR_DOMAIN_INPUT_BUS_XEN: + case VIR_DOMAIN_INPUT_BUS_PARALLELS: + case VIR_DOMAIN_INPUT_BUS_LAST: + /* should be 0 */ + return pciFlags; + } + + case VIR_DOMAIN_DEVICE_CHR: + switch ((virDomainChrSerialTargetType) dev->data.chr->targetType) { + case VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_PCI: + return pciFlags; + + case VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_ISA: + case VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_USB: + case VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_LAST: + /* should be 0 */ + return pciFlags; + } + + /* These devices don't ever connect with PCI */ + case VIR_DOMAIN_DEVICE_NVRAM: + case VIR_DOMAIN_DEVICE_TPM: + case VIR_DOMAIN_DEVICE_PANIC: + case VIR_DOMAIN_DEVICE_MEMORY: + case VIR_DOMAIN_DEVICE_HUB: + case VIR_DOMAIN_DEVICE_REDIRDEV: + case VIR_DOMAIN_DEVICE_SMARTCARD: + /* These devices don't even have a DeviceInfo */ + case VIR_DOMAIN_DEVICE_LEASE: + case VIR_DOMAIN_DEVICE_GRAPHICS: + case VIR_DOMAIN_DEVICE_IOMMU: + case VIR_DOMAIN_DEVICE_LAST: + case VIR_DOMAIN_DEVICE_NONE: + /* should be 0 */ + return pciFlags; + } + + /* We can never get here, because all cases are covered in the + * switch, and they all return, but the compiler will still + * complain "control reaches end of non-void function" unless + * we add the following return. + */ + return 0; +} + + +typedef struct { + virDomainPCIConnectFlags virtioFlags; + virDomainPCIConnectFlags pcieFlags; +} qemuDomainFillDevicePCIConnectFlagsIterData; + + +/** + * qemuDomainFillDevicePCIConnectFlagsIterInit: + * + * Initialize the iterator data that is used when calling + * qemuDomainCalculateDevicePCIConnectFlags(). + */ +static void +qemuDomainFillDevicePCIConnectFlagsIterInit(virDomainDefPtr def, + virQEMUCapsPtr qemuCaps, + qemuDomainFillDevicePCIConnectFlagsIterData *data) +{ + if (qemuDomainMachineHasPCIeRoot(def)) { + data->pcieFlags = (VIR_PCI_CONNECT_TYPE_PCIE_DEVICE | + VIR_PCI_CONNECT_HOTPLUGGABLE); + } else { + data->pcieFlags = (VIR_PCI_CONNECT_TYPE_PCI_DEVICE | + VIR_PCI_CONNECT_HOTPLUGGABLE); + } + + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY)) { + data->virtioFlags = data->pcieFlags; + } else { + data->virtioFlags = (VIR_PCI_CONNECT_TYPE_PCI_DEVICE | + VIR_PCI_CONNECT_HOTPLUGGABLE); + } +} + + +/** + * qemuDomainFillDevicePCIConnectFlagsIter: + * + * @def: the entire DomainDef + * @dev: The device to be checked + * @info: virDomainDeviceInfo within the device + * @opaque: points to iterator data setup beforehand. + * + * Sets the pciConnectFlags for a single device's info. Has properly + * formatted arguments to be called by virDomainDeviceInfoIterate(). + * + * Always returns 0 - there is no failure. + */ +static int +qemuDomainFillDevicePCIConnectFlagsIter(virDomainDefPtr def ATTRIBUTE_UNUSED, + virDomainDeviceDefPtr dev, + virDomainDeviceInfoPtr info, + void *opaque) +{ + qemuDomainFillDevicePCIConnectFlagsIterData *data = opaque; + + info->pciConnectFlags + = qemuDomainDeviceCalculatePCIConnectFlags(dev, data->pcieFlags, + data->virtioFlags); + return 0; +} + + +/** + * qemuDomainFillAllPCIConnectFlags: + * + * @def: the entire DomainDef + * @qemuCaps: as you'd expect + * + * Set the info->pciConnectFlags for all devices in the domain. + * + * Returns 0 on success or -1 on failure (the only possibility of + * failure would be some internal problem with + * virDomainDeviceInfoIterate()) + */ +static int ATTRIBUTE_UNUSED +qemuDomainFillAllPCIConnectFlags(virDomainDefPtr def, + virQEMUCapsPtr qemuCaps) +{ + qemuDomainFillDevicePCIConnectFlagsIterData data; + + qemuDomainFillDevicePCIConnectFlagsIterInit(def, qemuCaps, &data); + + return virDomainDeviceInfoIterate(def, + qemuDomainFillDevicePCIConnectFlagsIter, + &data); +} + + +/** + * qemuDomainFillDevicePCIConnectFlags: + * + * @def: the entire DomainDef + * @dev: The device to be checked + * @qemuCaps: as you'd expect + * + * Set the info->pciConnectFlags for a single device. + * + * No return value. + */ +static void ATTRIBUTE_UNUSED +qemuDomainFillDevicePCIConnectFlags(virDomainDefPtr def, + virDomainDeviceDefPtr dev, + virQEMUCapsPtr qemuCaps) +{ + virDomainDeviceInfoPtr info = virDomainDeviceGetInfo(dev); + + if (info) { + /* qemuDomainDeviceCalculatePCIConnectFlags() is called with + * the data setup in the ...IterData by ...IterInit() rather + * than setting the values directly here. It may seem like + * pointless posturing, but it's done this way to eliminate + * duplicated setup code while allowing more efficient + * operation when it's being done repeatedly with the device + * iterator (since qemuDomainFillAllPCIConnectFlags() only + * calls ...IterInit() once for all devices). + */ + qemuDomainFillDevicePCIConnectFlagsIterData data; + + qemuDomainFillDevicePCIConnectFlagsIterInit(def, qemuCaps, &data); + + info->pciConnectFlags + = qemuDomainDeviceCalculatePCIConnectFlags(dev, data.pcieFlags, + data.virtioFlags); + } +} + + static int qemuDomainPCIAddressReserveNextAddr(virDomainPCIAddressSetPtr addrs, virDomainDeviceInfoPtr dev, -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
The lowest level function of this trio (qemuDomainDeviceCalculatePCIConnectFlags()) aims to be the single authority for the virDomainPCIConnectFlags to use for any given device using a particular arch/machinetype/qemu-binary.
qemuDomainFillDevicePCIConnectFlags() sets info->pciConnectFlags in a single device (unless it has no virDomainDeviceInfo, in which case it's a NOP).
qemuDomainFillAllPCIConnectFlags() sets info->pciConnectFlags in all devices that have a virDomainDeviceInfo
The latter two functions aren't called anywhere yet. This commit is just making them available. Later patches will replace all the current hodge-podge of flag settings with calls to this single authority. --- src/conf/device_conf.h | 5 + src/qemu/qemu_domain_address.c | 367 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 372 insertions(+)
ACK -- Andrea Bolognani / Red Hat / Virtualization
Set pciConnectFlags in each device's DeviceInfo and then use those flags later when validating existing addresses in qemuDomainCollectPCIAddress() and when assigning new addresses with qemuDomainPCIAddressReserveNextAddr() (rather than scattering the logic about which devices need which type of slot all over the place). Note that the exact flags set by qemuDomainDeviceCalculatePCIConnectFlags() are different from the flags previously set manually in qemuDomainCollectPCIAddress(), but this doesn't matter because all validation of addresses in that case ignores the setting of the HOTPLUGGABLE flag, and treats PCIE_DEVICE and PCI_DEVICE the same (this lax checking was done on purpose, because there are some things that we want to allow the user to specify manually, e.g. assigning a PCIe device to a PCI slot, that we *don't* ever want libvirt to do automatically. The flag settings that we *really* want to match are 1) the old flag settings in qemuDomainAssignDevicePCISlots() (which is HOTPLUGGABLE | PCI_DEVICE for everything except PCI controllers) and 2) the new flag settings done by qemuDomainDeviceCalculatePCIConnectFlags() (which are currently exactly that - HOTPLUGGABLE | PCI_DEVICE for everything except PCI controllers). --- src/qemu/qemu_domain_address.c | 207 +++++++++++++++-------------------------- 1 file changed, 75 insertions(+), 132 deletions(-) diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 22cb402..6d0ace5 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -723,7 +723,7 @@ qemuDomainFillDevicePCIConnectFlagsIter(virDomainDefPtr def ATTRIBUTE_UNUSED, * failure would be some internal problem with * virDomainDeviceInfoIterate()) */ -static int ATTRIBUTE_UNUSED +static int qemuDomainFillAllPCIConnectFlags(virDomainDefPtr def, virQEMUCapsPtr qemuCaps) { @@ -748,7 +748,7 @@ qemuDomainFillAllPCIConnectFlags(virDomainDefPtr def, * * No return value. */ -static void ATTRIBUTE_UNUSED +static void qemuDomainFillDevicePCIConnectFlags(virDomainDefPtr def, virDomainDeviceDefPtr dev, virQEMUCapsPtr qemuCaps) @@ -779,21 +779,20 @@ qemuDomainFillDevicePCIConnectFlags(virDomainDefPtr def, static int qemuDomainPCIAddressReserveNextAddr(virDomainPCIAddressSetPtr addrs, virDomainDeviceInfoPtr dev, - virDomainPCIConnectFlags flags, unsigned int function, bool reserveEntireSlot) { - return virDomainPCIAddressReserveNextAddr(addrs, dev, flags, + return virDomainPCIAddressReserveNextAddr(addrs, dev, + dev->pciConnectFlags, function, reserveEntireSlot); } static int qemuDomainPCIAddressReserveNextSlot(virDomainPCIAddressSetPtr addrs, - virDomainDeviceInfoPtr dev, - virDomainPCIConnectFlags flags) + virDomainDeviceInfoPtr dev) { - return qemuDomainPCIAddressReserveNextAddr(addrs, dev, flags, 0, true); + return qemuDomainPCIAddressReserveNextAddr(addrs, dev, 0, true); } @@ -807,9 +806,6 @@ qemuDomainCollectPCIAddress(virDomainDefPtr def ATTRIBUTE_UNUSED, int ret = -1; virPCIDeviceAddressPtr addr = &info->addr.pci; bool entireSlot; - /* flags may be changed from default below */ - virDomainPCIConnectFlags flags = (VIR_PCI_CONNECT_HOTPLUGGABLE | - VIR_PCI_CONNECT_TYPE_PCI_DEVICE); if (!virDeviceInfoPCIAddressPresent(info) || ((device->type == VIR_DOMAIN_DEVICE_HOSTDEV) && @@ -821,71 +817,6 @@ qemuDomainCollectPCIAddress(virDomainDefPtr def ATTRIBUTE_UNUSED, return 0; } - /* Change flags according to differing requirements of different - * devices. - */ - switch (device->type) { - case VIR_DOMAIN_DEVICE_CONTROLLER: - switch (device->data.controller->type) { - case VIR_DOMAIN_CONTROLLER_TYPE_PCI: - flags = virDomainPCIControllerModelToConnectType(device->data.controller->model); - break; - - case VIR_DOMAIN_CONTROLLER_TYPE_SATA: - /* SATA controllers aren't hot-plugged, and can be put in - * either a PCI or PCIe slot - */ - flags = (VIR_PCI_CONNECT_TYPE_PCI_DEVICE - | VIR_PCI_CONNECT_TYPE_PCIE_DEVICE); - break; - - case VIR_DOMAIN_CONTROLLER_TYPE_USB: - /* allow UHCI and EHCI controllers to be manually placed on - * the PCIe bus (but don't put them there automatically) - */ - switch (device->data.controller->model) { - case VIR_DOMAIN_CONTROLLER_MODEL_USB_EHCI: - case VIR_DOMAIN_CONTROLLER_MODEL_USB_ICH9_EHCI1: - case VIR_DOMAIN_CONTROLLER_MODEL_USB_ICH9_UHCI1: - case VIR_DOMAIN_CONTROLLER_MODEL_USB_ICH9_UHCI2: - case VIR_DOMAIN_CONTROLLER_MODEL_USB_ICH9_UHCI3: - case VIR_DOMAIN_CONTROLLER_MODEL_USB_VT82C686B_UHCI: - flags = VIR_PCI_CONNECT_TYPE_PCI_DEVICE; - break; - case VIR_DOMAIN_CONTROLLER_MODEL_USB_NEC_XHCI: - /* should this be PCIE-only? Or do we need to allow PCI - * for backward compatibility? - */ - flags = (VIR_PCI_CONNECT_TYPE_PCI_DEVICE - | VIR_PCI_CONNECT_TYPE_PCIE_DEVICE); - break; - case VIR_DOMAIN_CONTROLLER_MODEL_USB_PCI_OHCI: - case VIR_DOMAIN_CONTROLLER_MODEL_USB_PIIX3_UHCI: - case VIR_DOMAIN_CONTROLLER_MODEL_USB_PIIX4_UHCI: - /* Allow these for PCI only */ - break; - } - } - break; - - case VIR_DOMAIN_DEVICE_SOUND: - switch (device->data.sound->model) { - case VIR_DOMAIN_SOUND_MODEL_ICH6: - case VIR_DOMAIN_SOUND_MODEL_ICH9: - flags = VIR_PCI_CONNECT_TYPE_PCI_DEVICE; - break; - } - break; - - case VIR_DOMAIN_DEVICE_VIDEO: - /* video cards aren't hot-plugged, and can be put in either a - * PCI or PCIe slot - */ - flags = (VIR_PCI_CONNECT_TYPE_PCI_DEVICE - | VIR_PCI_CONNECT_TYPE_PCIE_DEVICE); - break; - } - /* Ignore implicit controllers on slot 0:0:1.0: * implicit IDE controller on 0:0:1.1 (no qemu command line) * implicit USB controller on 0:0:1.2 (-usb) @@ -928,7 +859,7 @@ qemuDomainCollectPCIAddress(virDomainDefPtr def ATTRIBUTE_UNUSED, entireSlot = (addr->function == 0 && addr->multi != VIR_TRISTATE_SWITCH_ON); - if (virDomainPCIAddressReserveAddr(addrs, addr, flags, + if (virDomainPCIAddressReserveAddr(addrs, addr, info->pciConnectFlags, entireSlot, true) < 0) goto cleanup; @@ -1084,8 +1015,7 @@ qemuDomainValidateDevicePCISlotsPIIX3(virDomainDefPtr def, if (virDomainPCIAddressSlotInUse(addrs, &tmp_addr)) { if (qemuDeviceVideoUsable) { if (qemuDomainPCIAddressReserveNextSlot(addrs, - &primaryVideo->info, - flags) < 0) { + &primaryVideo->info) < 0) { goto cleanup; } } else { @@ -1274,8 +1204,7 @@ qemuDomainValidateDevicePCISlotsQ35(virDomainDefPtr def, if (virDomainPCIAddressSlotInUse(addrs, &tmp_addr)) { if (qemuDeviceVideoUsable) { if (qemuDomainPCIAddressReserveNextSlot(addrs, - &primaryVideo->info, - flags) < 0) + &primaryVideo->info) < 0) goto cleanup; } else { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", @@ -1396,7 +1325,6 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, virDomainPCIAddressSetPtr addrs) { size_t i, j; - virDomainPCIConnectFlags flags = 0; /* initialize to quiet gcc warning */ /* PCI controllers */ for (i = 0; i < def->ncontrollers; i++) { @@ -1410,33 +1338,18 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, !virDeviceInfoPCIAddressWanted(&cont->info)) continue; - /* convert the type of controller into a "CONNECT_TYPE" - * flag to use when searching for the proper - * controller/bus to connect it to on the upstream side. - */ - flags = virDomainPCIControllerModelToConnectType(model); - if (qemuDomainPCIAddressReserveNextSlot(addrs, - &cont->info, flags) < 0) { + if (qemuDomainPCIAddressReserveNextSlot(addrs, &cont->info) < 0) goto error; - } } } - /* all other devices that plug into a PCI slot are treated as a - * PCI endpoint devices that require a hotplug-capable slot - * (except for some special cases which have specific handling - * below) - */ - flags = VIR_PCI_CONNECT_HOTPLUGGABLE | VIR_PCI_CONNECT_TYPE_PCI_DEVICE; - for (i = 0; i < def->nfss; i++) { if (!virDeviceInfoPCIAddressWanted(&def->fss[i]->info)) continue; /* Only support VirtIO-9p-pci so far. If that changes, * we might need to skip devices here */ - if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->fss[i]->info, - flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->fss[i]->info) < 0) goto error; } @@ -1452,7 +1365,8 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, !virDeviceInfoPCIAddressWanted(&net->info)) { continue; } - if (qemuDomainPCIAddressReserveNextSlot(addrs, &net->info, flags) < 0) + + if (qemuDomainPCIAddressReserveNextSlot(addrs, &net->info) < 0) goto error; } @@ -1470,7 +1384,7 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, continue; } - if (qemuDomainPCIAddressReserveNextSlot(addrs, &sound->info, flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, &sound->info) < 0) goto error; } @@ -1537,7 +1451,8 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, if (foundAddr) { /* Reserve this function on the slot we found */ - if (virDomainPCIAddressReserveAddr(addrs, &addr, flags, + if (virDomainPCIAddressReserveAddr(addrs, &addr, + cont->info.pciConnectFlags, false, true) < 0) goto error; @@ -1546,8 +1461,7 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, } else { /* This is the first part of the controller, so need * to find a free slot & then reserve this function */ - if (qemuDomainPCIAddressReserveNextAddr(addrs, - &cont->info, flags, + if (qemuDomainPCIAddressReserveNextAddr(addrs, &cont->info, addr.function, false) < 0) { goto error; @@ -1556,10 +1470,8 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, cont->info.addr.pci.multi = addr.multi; } } else { - if (qemuDomainPCIAddressReserveNextSlot(addrs, - &cont->info, flags) < 0) { - goto error; - } + if (qemuDomainPCIAddressReserveNextSlot(addrs, &cont->info) < 0) + goto error; } } @@ -1590,8 +1502,7 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, goto error; } - if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->disks[i]->info, - flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->disks[i]->info) < 0) goto error; } @@ -1603,8 +1514,8 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, def->hostdevs[i]->source.subsys.type != VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI) continue; - if (qemuDomainPCIAddressReserveNextSlot(addrs, def->hostdevs[i]->info, - flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, + def->hostdevs[i]->info) < 0) goto error; } @@ -1613,8 +1524,8 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, def->memballoon->model == VIR_DOMAIN_MEMBALLOON_MODEL_VIRTIO && virDeviceInfoPCIAddressWanted(&def->memballoon->info)) { - if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->memballoon->info, - flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, + &def->memballoon->info) < 0) goto error; } @@ -1624,8 +1535,7 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, !virDeviceInfoPCIAddressWanted(&def->rngs[i]->info)) continue; - if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->rngs[i]->info, - flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->rngs[i]->info) < 0) goto error; } @@ -1633,8 +1543,7 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, if (def->watchdog && def->watchdog->model == VIR_DOMAIN_WATCHDOG_MODEL_I6300ESB && virDeviceInfoPCIAddressWanted(&def->watchdog->info)) { - if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->watchdog->info, - flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->watchdog->info) < 0) goto error; } @@ -1642,16 +1551,15 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, * assigned address. */ if (def->nvideos > 0 && virDeviceInfoPCIAddressWanted(&def->videos[0]->info)) { - if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->videos[0]->info, - flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->videos[0]->info) < 0) goto error; } for (i = 1; i < def->nvideos; i++) { if (!virDeviceInfoPCIAddressWanted(&def->videos[i]->info)) continue; - if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->videos[i]->info, - flags) < 0) + + if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->videos[i]->info) < 0) goto error; } @@ -1660,8 +1568,7 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, if (!virDeviceInfoPCIAddressWanted(&def->shmems[i]->info)) continue; - if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->shmems[i]->info, - flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->shmems[i]->info) < 0) goto error; } for (i = 0; i < def->ninputs; i++) { @@ -1669,8 +1576,7 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, !virDeviceInfoPCIAddressWanted(&def->inputs[i]->info)) continue; - if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->inputs[i]->info, - flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, &def->inputs[i]->info) < 0) goto error; } for (i = 0; i < def->nparallels; i++) { @@ -1683,7 +1589,7 @@ qemuDomainAssignDevicePCISlots(virDomainDefPtr def, !virDeviceInfoPCIAddressWanted(&chr->info)) continue; - if (qemuDomainPCIAddressReserveNextSlot(addrs, &chr->info, flags) < 0) + if (qemuDomainPCIAddressReserveNextSlot(addrs, &chr->info) < 0) goto error; } for (i = 0; i < def->nchannels; i++) { @@ -1840,8 +1746,6 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, int rv; bool buses_reserved = true; - virDomainPCIConnectFlags flags = VIR_PCI_CONNECT_TYPE_PCI_DEVICE; - for (i = 0; i < def->ncontrollers; i++) { virDomainControllerDefPtr cont = def->controllers[i]; @@ -1853,10 +1757,23 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, nbuses = max_idx + 1; + /* set the connect type flags (pci vs. pcie) in the DeviceInfo + * of all devices. This will be used to pick an appropriate + * bus when assigning addresses. + */ + if (qemuDomainFillAllPCIConnectFlags(def, qemuCaps) < 0) + goto cleanup; + if (nbuses > 0 && virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_PCI_BRIDGE)) { + /* This is a dummy info used to reserve a slot for a legacy + * PCI device that doesn't exist, but may in the future, e.g. + * if another device is hotplugged into the domain. + */ virDomainDeviceInfo info; + info.pciConnectFlags = VIR_PCI_CONNECT_TYPE_PCI_DEVICE; + /* 1st pass to figure out how many PCI bridges we need */ if (!(addrs = qemuDomainPCIAddressSetCreate(def, nbuses, true))) goto cleanup; @@ -1880,24 +1797,50 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, */ if (!buses_reserved && !qemuDomainMachineIsVirt(def) && - qemuDomainPCIAddressReserveNextSlot(addrs, &info, flags) < 0) + qemuDomainPCIAddressReserveNextSlot(addrs, &info) < 0) goto cleanup; if (qemuDomainAssignDevicePCISlots(def, qemuCaps, addrs) < 0) goto cleanup; for (i = 1; i < addrs->nbuses; i++) { + virDomainDeviceDef dev; + int contIndex; virDomainPCIAddressBusPtr bus = &addrs->buses[i]; if ((rv = virDomainDefMaybeAddController( def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, i, bus->model)) < 0) goto cleanup; - /* If we added a new bridge, we will need one more address */ - if (rv > 0 && - qemuDomainPCIAddressReserveNextSlot(addrs, &info, flags) < 0) + + if (rv == 0) + continue; /* no new controller added */ + + /* We did add a new controller, so we will need one more + * address (and we need to set the new controller's + * pciConnectFlags) + */ + contIndex = virDomainControllerFind(def, + VIR_DOMAIN_CONTROLLER_TYPE_PCI, + i); + if (contIndex < 0) { + /* this should never happen - we just added it */ + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Could not find auto-added %s controller " + "with index %zu"), + virDomainControllerModelPCITypeToString(bus->model), + i); + goto cleanup; + } + dev.type = VIR_DOMAIN_DEVICE_CONTROLLER; + dev.data.controller = def->controllers[contIndex]; + /* set connect flags so it will be properly addressed */ + qemuDomainFillDevicePCIConnectFlags(def, &dev, qemuCaps); + if (qemuDomainPCIAddressReserveNextSlot(addrs, + &dev.data.controller->info) < 0) goto cleanup; } + nbuses = addrs->nbuses; virDomainPCIAddressSetFree(addrs); addrs = NULL; -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
Set pciConnectFlags in each device's DeviceInfo and then use those flags later when validating existing addresses in qemuDomainCollectPCIAddress() and when assigning new addresses with qemuDomainPCIAddressReserveNextAddr() (rather than scattering the logic about which devices need which type of slot all over the place).
Note that the exact flags set by qemuDomainDeviceCalculatePCIConnectFlags() are different from the flags previously set manually in qemuDomainCollectPCIAddress(), but this doesn't matter because all validation of addresses in that case ignores the setting of the HOTPLUGGABLE flag, and treats PCIE_DEVICE and PCI_DEVICE the same (this lax checking was done on purpose, because there are some things that we want to allow the user to specify manually, e.g. assigning a PCIe device to a PCI slot, that we *don't* ever want libvirt to do automatically. The flag settings that we *really* want to match are 1) the old flag settings in qemuDomainAssignDevicePCISlots() (which is HOTPLUGGABLE | PCI_DEVICE for everything except PCI controllers) and 2) the new flag settings done by qemuDomainDeviceCalculatePCIConnectFlags() (which are currently exactly that - HOTPLUGGABLE | PCI_DEVICE for everything except PCI controllers). --- src/qemu/qemu_domain_address.c | 207 +++++++++++++++-------------------------- 1 file changed, 75 insertions(+), 132 deletions(-)
ACK -- Andrea Bolognani / Red Hat / Virtualization
Before now, all the qemu hotplug functions assumed that all devices to be hotplugged were legacy PCI endpoint devices (VIR_PCI_CONNECT_TYPE_PCI_DEVICE). This worked out "okay", because all devices *are* legacy PCI endpoint devices on x86/440fx machinetypes, and hotplug didn't work properly on machinetypes using PCIe anyway (hotplugging onto a legacy PCI slot doesn't work, and until commit b87703cf any attempt to manually specify a PCIe address for a hotplugged device would be erroneously rejected). This patch makes all qemu hotplug operations honor the pciConnectFlags set by the single all-knowing function qemuDomainDeviceCalculatePCIConnectFlags(). This is done in 3 steps, but in a single commit since we would have to touch the other points at each step anyway: 1) add a flags argument to the hypervisor-agnostic virDomainPCIAddressEnsureAddr() (previously it hardcoded ..._PCI_DEVICE) 2) add a new qemu-specific function qemuDomainEnsurePCIAddress() which gets the correct pciConnectFlags for the device from qemuDomainDeviceConnectFlags(), then calls virDomainPCIAddressEnsureAddr(). 3) in qemu_hotplug.c replace all calls to virDomainPCIAddressEnsureAddr() with calls to qemuDomainEnsurePCIAddress() So in effect, we're putting a "shim" on top of all calls to virDomainPCIAddressEnsureAddr() that sets the right pciConnectFlags. --- src/conf/domain_addr.c | 10 ++-------- src/conf/domain_addr.h | 3 ++- src/qemu/qemu_domain_address.c | 27 +++++++++++++++++++++++++++ src/qemu/qemu_domain_address.h | 4 ++++ src/qemu/qemu_hotplug.c | 23 ++++++++++++++++------- 5 files changed, 51 insertions(+), 16 deletions(-) diff --git a/src/conf/domain_addr.c b/src/conf/domain_addr.c index 2af9c64..0346471e 100644 --- a/src/conf/domain_addr.c +++ b/src/conf/domain_addr.c @@ -487,17 +487,11 @@ virDomainPCIAddressReserveSlot(virDomainPCIAddressSetPtr addrs, int virDomainPCIAddressEnsureAddr(virDomainPCIAddressSetPtr addrs, - virDomainDeviceInfoPtr dev) + virDomainDeviceInfoPtr dev, + virDomainPCIConnectFlags flags) { int ret = -1; char *addrStr = NULL; - /* Flags should be set according to the particular device, - * but only the caller knows the type of device. Currently this - * function is only used for hot-plug, though, and hot-plug is - * only supported for standard PCI devices, so we can safely use - * the setting below */ - virDomainPCIConnectFlags flags = (VIR_PCI_CONNECT_HOTPLUGGABLE | - VIR_PCI_CONNECT_TYPE_PCI_DEVICE); if (!(addrStr = virDomainPCIAddressAsString(&dev->addr.pci))) goto cleanup; diff --git a/src/conf/domain_addr.h b/src/conf/domain_addr.h index ff80fce..bcec2c7 100644 --- a/src/conf/domain_addr.h +++ b/src/conf/domain_addr.h @@ -144,7 +144,8 @@ int virDomainPCIAddressReserveSlot(virDomainPCIAddressSetPtr addrs, ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); int virDomainPCIAddressEnsureAddr(virDomainPCIAddressSetPtr addrs, - virDomainDeviceInfoPtr dev) + virDomainDeviceInfoPtr dev, + virDomainPCIConnectFlags flags) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); int virDomainPCIAddressReleaseAddr(virDomainPCIAddressSetPtr addrs, diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 6d0ace5..85e9521 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -2140,6 +2140,33 @@ qemuDomainAssignAddresses(virDomainDefPtr def, return 0; } +/** + * qemuDomainEnsurePCIAddress: + * + * if @dev should have a PCI address but doesn't, assign an address on + * a compatible PCI bus, and set it in @dev->...info. If there is an + * address already, validate that it is on a compatible bus, based on + * @dev->...info.pciConnectFlags. + * + * @obj: the virDomainObjPtr for the domain. This will include + * qemuCaps and address cache (if there is one) + * + * @dev: the device that we need to ensure has a PCI address + * + * returns 0 on success -1 on failure. + */ +int +qemuDomainEnsurePCIAddress(virDomainObjPtr obj, + virDomainDeviceDefPtr dev) +{ + qemuDomainObjPrivatePtr priv = obj->privateData; + virDomainDeviceInfoPtr info = virDomainDeviceGetInfo(dev); + + qemuDomainFillDevicePCIConnectFlags(obj->def, dev, priv->qemuCaps); + + return virDomainPCIAddressEnsureAddr(priv->pciaddrs, info, + info->pciConnectFlags); +} void qemuDomainReleaseDeviceAddress(virDomainObjPtr vm, diff --git a/src/qemu/qemu_domain_address.h b/src/qemu/qemu_domain_address.h index 11d6e92..800859c 100644 --- a/src/qemu/qemu_domain_address.h +++ b/src/qemu/qemu_domain_address.h @@ -37,6 +37,10 @@ int qemuDomainAssignAddresses(virDomainDefPtr def, bool newDomain) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); +int qemuDomainEnsurePCIAddress(virDomainObjPtr obj, + virDomainDeviceDefPtr dev) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); + void qemuDomainReleaseDeviceAddress(virDomainObjPtr vm, virDomainDeviceInfoPtr info, const char *devstr); diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index e06862c..41731ab 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -304,6 +304,7 @@ qemuDomainAttachVirtioDiskDevice(virConnectPtr conn, int ret = -1; int rv; qemuDomainObjPrivatePtr priv = vm->privateData; + virDomainDeviceDef dev = { VIR_DOMAIN_DEVICE_DISK, { .disk = disk } }; virErrorPtr orig_err; char *devstr = NULL; char *drivestr = NULL; @@ -344,7 +345,7 @@ qemuDomainAttachVirtioDiskDevice(virConnectPtr conn, goto error; } else if (!disk->info.type || disk->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) { - if (virDomainPCIAddressEnsureAddr(priv->pciaddrs, &disk->info) < 0) + if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) goto error; } releaseaddr = true; @@ -462,6 +463,8 @@ int qemuDomainAttachControllerDevice(virQEMUDriverPtr driver, const char* type = virDomainControllerTypeToString(controller->type); char *devstr = NULL; qemuDomainObjPrivatePtr priv = vm->privateData; + virDomainDeviceDef dev = { VIR_DOMAIN_DEVICE_CONTROLLER, + { .controller = controller } }; virDomainCCWAddressSetPtr ccwaddrs = NULL; bool releaseaddr = false; @@ -501,7 +504,7 @@ int qemuDomainAttachControllerDevice(virQEMUDriverPtr driver, if (controller->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE || controller->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) { - if (virDomainPCIAddressEnsureAddr(priv->pciaddrs, &controller->info) < 0) + if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) goto cleanup; } else if (controller->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCW) { if (!(ccwaddrs = qemuDomainCCWAddrSetCreateFromDomain(vm->def))) @@ -935,6 +938,7 @@ qemuDomainAttachNetDevice(virQEMUDriverPtr driver, virDomainNetDefPtr net) { qemuDomainObjPrivatePtr priv = vm->privateData; + virDomainDeviceDef dev = { VIR_DOMAIN_DEVICE_NET, { .net = net } }; char **tapfdName = NULL; int *tapfd = NULL; size_t tapfdSize = 0; @@ -1131,7 +1135,7 @@ qemuDomainAttachNetDevice(virQEMUDriverPtr driver, virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("virtio-s390 net device cannot be hotplugged.")); goto cleanup; - } else if (virDomainPCIAddressEnsureAddr(priv->pciaddrs, &net->info) < 0) { + } else if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) { goto cleanup; } @@ -1360,6 +1364,8 @@ qemuDomainAttachHostPCIDevice(virQEMUDriverPtr driver, virDomainHostdevDefPtr hostdev) { qemuDomainObjPrivatePtr priv = vm->privateData; + virDomainDeviceDef dev = { VIR_DOMAIN_DEVICE_HOSTDEV, + { .hostdev = hostdev } }; int ret; char *devstr = NULL; int configfd = -1; @@ -1430,7 +1436,7 @@ qemuDomainAttachHostPCIDevice(virQEMUDriverPtr driver, if (qemuAssignDeviceHostdevAlias(vm->def, &hostdev->info->alias, -1) < 0) goto error; - if (virDomainPCIAddressEnsureAddr(priv->pciaddrs, hostdev->info) < 0) + if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) goto error; releaseaddr = true; if (backend != VIR_DOMAIN_HOSTDEV_PCI_BACKEND_VFIO && @@ -1766,6 +1772,7 @@ qemuDomainAttachChrDeviceAssignAddr(virDomainObjPtr vm, { virDomainDefPtr def = vm->def; qemuDomainObjPrivatePtr priv = vm->privateData; + virDomainDeviceDef dev = { VIR_DOMAIN_DEVICE_CHR, { .chr = chr } }; if (chr->deviceType == VIR_DOMAIN_CHR_DEVICE_TYPE_CONSOLE && chr->targetType == VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_VIRTIO) { @@ -1775,7 +1782,7 @@ qemuDomainAttachChrDeviceAssignAddr(virDomainObjPtr vm, } else if (chr->deviceType == VIR_DOMAIN_CHR_DEVICE_TYPE_SERIAL && chr->targetType == VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_PCI) { - if (virDomainPCIAddressEnsureAddr(priv->pciaddrs, &chr->info) < 0) + if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) return -1; return 1; @@ -1931,6 +1938,7 @@ qemuDomainAttachRNGDevice(virConnectPtr conn, { virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver); qemuDomainObjPrivatePtr priv = vm->privateData; + virDomainDeviceDef dev = { VIR_DOMAIN_DEVICE_RNG, { .rng = rng } }; virErrorPtr orig_err; char *devstr = NULL; char *charAlias = NULL; @@ -1973,7 +1981,7 @@ qemuDomainAttachRNGDevice(virConnectPtr conn, if (rng->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE || rng->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) { - if (virDomainPCIAddressEnsureAddr(priv->pciaddrs, &rng->info) < 0) + if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) goto cleanup; } else if (rng->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCW) { if (!(ccwaddrs = qemuDomainCCWAddrSetCreateFromDomain(vm->def))) @@ -2486,6 +2494,7 @@ qemuDomainAttachShmemDevice(virQEMUDriverPtr driver, virErrorPtr orig_err = NULL; virJSONValuePtr props = NULL; qemuDomainObjPrivatePtr priv = vm->privateData; + virDomainDeviceDef dev = { VIR_DOMAIN_DEVICE_SHMEM, { .shmem = shmem } }; switch ((virDomainShmemModel)shmem->model) { case VIR_DOMAIN_SHMEM_MODEL_IVSHMEM_PLAIN: @@ -2512,7 +2521,7 @@ qemuDomainAttachShmemDevice(virQEMUDriverPtr driver, if ((shmem->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE || shmem->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) && - (virDomainPCIAddressEnsureAddr(priv->pciaddrs, &shmem->info) < 0)) + (qemuDomainEnsurePCIAddress(vm, &dev) < 0)) return -1; if (!(shmstr = qemuBuildShmemDevStr(vm->def, shmem, priv->qemuCaps))) -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
Before now, all the qemu hotplug functions assumed that all devices to be hotplugged were legacy PCI endpoint devices (VIR_PCI_CONNECT_TYPE_PCI_DEVICE). This worked out "okay", because all devices *are* legacy PCI endpoint devices on x86/440fx machinetypes, and hotplug didn't work properly on machinetypes using PCIe anyway (hotplugging onto a legacy PCI slot doesn't work, and until commit b87703cf any attempt to manually specify a PCIe address for a hotplugged device would be erroneously rejected). This patch makes all qemu hotplug operations honor the pciConnectFlags set by the single all-knowing function qemuDomainDeviceCalculatePCIConnectFlags(). This is done in 3 steps, but in a single commit since we would have to touch the other points at each step anyway: 1) add a flags argument to the hypervisor-agnostic virDomainPCIAddressEnsureAddr() (previously it hardcoded ..._PCI_DEVICE) 2) add a new qemu-specific function qemuDomainEnsurePCIAddress() which gets the correct pciConnectFlags for the device from qemuDomainDeviceConnectFlags(), then calls virDomainPCIAddressEnsureAddr(). 3) in qemu_hotplug.c replace all calls to virDomainPCIAddressEnsureAddr() with calls to qemuDomainEnsurePCIAddress() So in effect, we're putting a "shim" on top of all calls to virDomainPCIAddressEnsureAddr() that sets the right pciConnectFlags. --- src/conf/domain_addr.c | 10 ++-------- src/conf/domain_addr.h | 3 ++- src/qemu/qemu_domain_address.c | 27 +++++++++++++++++++++++++++ src/qemu/qemu_domain_address.h | 4 ++++ src/qemu/qemu_hotplug.c | 23 ++++++++++++++++------- 5 files changed, 51 insertions(+), 16 deletions(-)
[...]
+/** + * qemuDomainEnsurePCIAddress: + * + * if @dev should have a PCI address but doesn't, assign an address on + * a compatible PCI bus, and set it in @dev->...info. If there is an + * address already, validate that it is on a compatible bus, based on + * @dev->...info.pciConnectFlags. + * + * @obj: the virDomainObjPtr for the domain. This will include + * qemuCaps and address cache (if there is one) + * + * @dev: the device that we need to ensure has a PCI address
Please move the description of the function arguments before the description of the function itself.
+ * returns 0 on success -1 on failure. + */ +int +qemuDomainEnsurePCIAddress(virDomainObjPtr obj, + virDomainDeviceDefPtr dev) +{ + qemuDomainObjPrivatePtr priv = obj->privateData; + virDomainDeviceInfoPtr info = virDomainDeviceGetInfo(dev);
virDomainDeviceGetInfo() can return NULL...
+ qemuDomainFillDevicePCIConnectFlags(obj->def, dev, priv->qemuCaps); + + return virDomainPCIAddressEnsureAddr(priv->pciaddrs, info, + info->pciConnectFlags);
... but you don't check for that before dereferencing info here. I guess this function will never be called on a device that doesn't have a DeviceInfo, but it's probably a good idea to explicitly check anyways in case the callers change. ACK once the comments above have been taken care of. -- Andrea Bolognani / Red Hat / Virtualization
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
Before now, all the qemu hotplug functions assumed that all devices to be hotplugged were legacy PCI endpoint devices (VIR_PCI_CONNECT_TYPE_PCI_DEVICE). This worked out "okay", because all devices *are* legacy PCI endpoint devices on x86/440fx machinetypes, and hotplug didn't work properly on machinetypes using PCIe anyway (hotplugging onto a legacy PCI slot doesn't work, and until commit b87703cf any attempt to manually specify a PCIe address for a hotplugged device would be erroneously rejected). This patch makes all qemu hotplug operations honor the pciConnectFlags set by the single all-knowing function qemuDomainDeviceCalculatePCIConnectFlags(). This is done in 3 steps, but in a single commit since we would have to touch the other points at each step anyway: 1) add a flags argument to the hypervisor-agnostic virDomainPCIAddressEnsureAddr() (previously it hardcoded ..._PCI_DEVICE) 2) add a new qemu-specific function qemuDomainEnsurePCIAddress() which gets the correct pciConnectFlags for the device from qemuDomainDeviceConnectFlags(), then calls virDomainPCIAddressEnsureAddr(). 3) in qemu_hotplug.c replace all calls to virDomainPCIAddressEnsureAddr() with calls to qemuDomainEnsurePCIAddress() So in effect, we're putting a "shim" on top of all calls to virDomainPCIAddressEnsureAddr() that sets the right pciConnectFlags. --- src/conf/domain_addr.c | 10 ++-------- src/conf/domain_addr.h | 3 ++- src/qemu/qemu_domain_address.c | 27 +++++++++++++++++++++++++++ src/qemu/qemu_domain_address.h | 4 ++++ src/qemu/qemu_hotplug.c | 23 ++++++++++++++++------- 5 files changed, 51 insertions(+), 16 deletions(-)
Oh, just one more tiny detail: [...]
@@ -2512,7 +2521,7 @@ qemuDomainAttachShmemDevice(virQEMUDriverPtr driver, if ((shmem->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE || shmem->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) && - (virDomainPCIAddressEnsureAddr(priv->pciaddrs, &shmem->info) < 0)) + (qemuDomainEnsurePCIAddress(vm, &dev) < 0)) return -1;
This branch should have had curly braces even before, but since you're changing the condition anyway you might as well slip them in now. And thanks for fixing the indentation :) -- Andrea Bolognani / Red Hat / Virtualization
This patch cleans up the connect flags for certain types/models of devices that aren't PCI to return 0. In the future that may be used as an indicator to the caller about whether or not a device needs a PCI address. For now it's just ignored, except for in virDomainPCIAddressEnsureAddr() - called during device hotplug - (and in some cases actually needs to be re-set to PCI|HOTPLUGGABLE just in case someone (in some old config) has manually set a PCI address for a device that isn't PCI. --- src/conf/domain_addr.c | 6 +++++ src/qemu/qemu_domain_address.c | 54 +++++++++++++++++++++++++----------------- 2 files changed, 38 insertions(+), 22 deletions(-) diff --git a/src/conf/domain_addr.c b/src/conf/domain_addr.c index 0346471e..d3a2f6f 100644 --- a/src/conf/domain_addr.c +++ b/src/conf/domain_addr.c @@ -493,6 +493,12 @@ virDomainPCIAddressEnsureAddr(virDomainPCIAddressSetPtr addrs, int ret = -1; char *addrStr = NULL; + /* if flags is 0, the particular model of this device on this + * machinetype doesn't need a PCI address, so we're done. + */ + if (!flags) + return 0; + if (!(addrStr = virDomainPCIAddressAsString(&dev->addr.pci))) goto cleanup; diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 85e9521..657dc62 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -467,8 +467,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_CONTROLLER_MODEL_USB_QUSB2: /* xen only */ case VIR_DOMAIN_CONTROLLER_MODEL_USB_NONE: case VIR_DOMAIN_CONTROLLER_MODEL_USB_LAST: - /* should be 0 */ - return pciFlags; + return 0; } case VIR_DOMAIN_CONTROLLER_TYPE_IDE: @@ -497,8 +496,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, */ if (net->type == VIR_DOMAIN_NET_TYPE_HOSTDEV || STREQ(net->model, "usb-net")) { - /* should be 0 */ - return pciFlags; + return 0; } return pciFlags; } @@ -515,8 +513,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_SOUND_MODEL_PCSPK: case VIR_DOMAIN_SOUND_MODEL_USB: case VIR_DOMAIN_SOUND_MODEL_LAST: - /* should be 0 */ - return pciFlags; + return 0; } case VIR_DOMAIN_DEVICE_DISK: @@ -533,8 +530,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_DISK_BUS_SATA: case VIR_DOMAIN_DISK_BUS_SD: case VIR_DOMAIN_DISK_BUS_LAST: - /* should be 0 */ - return pciFlags; + return 0; } case VIR_DOMAIN_DEVICE_HOSTDEV: @@ -548,8 +544,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_MEMBALLOON_MODEL_XEN: case VIR_DOMAIN_MEMBALLOON_MODEL_NONE: case VIR_DOMAIN_MEMBALLOON_MODEL_LAST: - /* should be 0 (not PCI) */ - return pciFlags; + return 0; } case VIR_DOMAIN_DEVICE_RNG: @@ -558,8 +553,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, return pciFlags; case VIR_DOMAIN_RNG_MODEL_LAST: - /* should be 0 */ - return pciFlags; + return 0; } case VIR_DOMAIN_DEVICE_WATCHDOG: @@ -571,8 +565,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_WATCHDOG_MODEL_IB700: case VIR_DOMAIN_WATCHDOG_MODEL_DIAG288: case VIR_DOMAIN_WATCHDOG_MODEL_LAST: - /* should be 0 */ - return pciFlags; + return 0; } case VIR_DOMAIN_DEVICE_VIDEO: @@ -588,8 +581,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, return pciFlags; case VIR_DOMAIN_VIDEO_TYPE_LAST: - /* should be 0 */ - return pciFlags; + return 0; } @@ -606,8 +598,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_INPUT_BUS_XEN: case VIR_DOMAIN_INPUT_BUS_PARALLELS: case VIR_DOMAIN_INPUT_BUS_LAST: - /* should be 0 */ - return pciFlags; + return 0; } case VIR_DOMAIN_DEVICE_CHR: @@ -618,8 +609,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_ISA: case VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_USB: case VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_LAST: - /* should be 0 */ - return pciFlags; + return 0; } /* These devices don't ever connect with PCI */ @@ -636,8 +626,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_DEVICE_IOMMU: case VIR_DOMAIN_DEVICE_LAST: case VIR_DOMAIN_DEVICE_NONE: - /* should be 0 */ - return pciFlags; + return 0; } /* We can never get here, because all cases are covered in the @@ -817,6 +806,27 @@ qemuDomainCollectPCIAddress(virDomainDefPtr def ATTRIBUTE_UNUSED, return 0; } + /* If we get to here, the device has a PCI address assigned in the + * config and we should mark it as in-use. But if the + * pciConnectFlags are 0, then this device shouldn't have a PCI + * address associated with it. *BUT* since there are cases in the + * past where we've apparently allowed that, we need to pretend + * for now that it's okay, otherwise an existing domain could + * "disappear" from the list of domains due to a parse failure. We + * can fix this by just forcing the pciConnectFlags to be + * PCI_DEVICE (and then relying on validation functions to report + * inappropriate address types. + */ + if (info->pciConnectFlags == 0) { + char *addrStr = virDomainPCIAddressAsString(&info->addr.pci); + + VIR_WARN("qemuDomainDeviceCalculatePCIConnectFlags() thinks that the " + "device with PCI address %s should not have a PCI address", + addrStr ? addrStr : "(unknown)"); + VIR_FREE(addrStr); + info->pciConnectFlags = VIR_PCI_CONNECT_TYPE_PCI_DEVICE; + } + /* Ignore implicit controllers on slot 0:0:1.0: * implicit IDE controller on 0:0:1.1 (no qemu command line) * implicit USB controller on 0:0:1.2 (-usb) -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
This patch cleans up the connect flags for certain types/models of devices that aren't PCI to return 0. In the future that may be used as an indicator to the caller about whether or not a device needs a PCI address. For now it's just ignored, except for in virDomainPCIAddressEnsureAddr() - called during device hotplug - (and in some cases actually needs to be re-set to PCI|HOTPLUGGABLE just in case someone (in some old config) has manually set a PCI address for a device that isn't PCI. --- src/conf/domain_addr.c | 6 +++++ src/qemu/qemu_domain_address.c | 54 +++++++++++++++++++++++++----------------- 2 files changed, 38 insertions(+), 22 deletions(-)
[...]
@@ -817,6 +806,27 @@ qemuDomainCollectPCIAddress(virDomainDefPtr def ATTRIBUTE_UNUSED, return 0; } + /* If we get to here, the device has a PCI address assigned in the
This line is not aligned properly.
+ * config and we should mark it as in-use. But if the + * pciConnectFlags are 0, then this device shouldn't have a PCI + * address associated with it. *BUT* since there are cases in the + * past where we've apparently allowed that, we need to pretend + * for now that it's okay, otherwise an existing domain could + * "disappear" from the list of domains due to a parse failure. We + * can fix this by just forcing the pciConnectFlags to be + * PCI_DEVICE (and then relying on validation functions to report + * inappropriate address types. + */ + if (info->pciConnectFlags == 0) {
You can turn this into if (!info->pciConnectFlags) {
+ char *addrStr = virDomainPCIAddressAsString(&info->addr.pci); + + VIR_WARN("qemuDomainDeviceCalculatePCIConnectFlags() thinks that the " + "device with PCI address %s should not have a PCI address", + addrStr ? addrStr : "(unknown)"); + VIR_FREE(addrStr);
Please add an empty line here.
+ info->pciConnectFlags = VIR_PCI_CONNECT_TYPE_PCI_DEVICE; + } + /* Ignore implicit controllers on slot 0:0:1.0: * implicit IDE controller on 0:0:1.1 (no qemu command line) * implicit USB controller on 0:0:1.2 (-usb)
ACK -- Andrea Bolognani / Red Hat / Virtualization
libvirt previously assigned nearly all devices to a "hotpluggable" legacy PCI slot even on machines with a PCIe root bus (and even though most such machines don't even support hotplug on legacy PCI slots!) Forcing all devices onto legacy PCI slots means that the domain will need a dmi-to-pci-bridge (to convert from PCIe to legacy PCI) and a pci-bridge (to provide hotpluggable legacy PCI slots which, again, usually aren't hotpluggable anyway). To help reduce the need for these legacy controllers, this patch tries to assign virtio-1.0-capable devices to PCIe slots whenever possible, by setting appropriate connectFlags in virDomainCalculateDevicePCIConnectFlags(). Happily, when that function was written (just a few commits ago) it was created with a "virtioFlags" argument, set by both of its callers, which is the proper connectFlags to set for any virtio-*-pci device - depending on the arch/machinetype of the domain, and whether or not the qemu binary supports virtio-1.0, that flag will have either been set to PCI or PCIE. This patch merely enables the functionality by setting the flags for the device to whatever is in virtioFlags if the device is a virtio-*-pci device. NB: the first virtio video device will be placed directly on bus 0 slot 1 rather than on a pcie-root-port due to the override for primary video devices in qemuDomainValidateDevicePCISlotsQ35(). Whether or not to change that is a topic of discussion, but this patch doesn't change that particular behavior. NB2: since the slot must be hotpluggable, and pcie-root (the PCIe root complex) does *not* support hotplug, this means that suitable controllers must also be in the config (i.e. either pcie-root-port, or pcie-downstream-port). For now, libvirt doesn't add those automatically, so if you put virtio devices in a config for a qemu that has PCIe-capable virtio devices, you'll need to add extra pcie-root-ports yourself. That requirement will be eliminated in a future patch, but for now, it's simple to do this: <controller type='pci' model='pcie-root-port'/> <controller type='pci' model='pcie-root-port'/> <controller type='pci' model='pcie-root-port'/> ... Partially Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1330024 --- src/qemu/qemu_domain_address.c | 41 ++++-- tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args | 58 ++++++++ tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml | 61 ++++++++ .../qemuxml2argv-q35-virtio-pci.args | 58 ++++++++ .../qemuxml2argv-q35-virtio-pci.xml | 1 + tests/qemuxml2argvtest.c | 48 +++++++ .../qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml | 154 +++++++++++++++++++++ .../qemuxml2xmlout-q35-virtio-pci.xml | 154 +++++++++++++++++++++ tests/qemuxml2xmltest.c | 43 ++++++ 9 files changed, 609 insertions(+), 9 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args create mode 120000 tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 657dc62..3f3e9e4 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -430,8 +430,7 @@ static virDomainPCIConnectFlags qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, virDomainPCIConnectFlags pcieFlags ATTRIBUTE_UNUSED, - virDomainPCIConnectFlags virtioFlags - ATTRIBUTE_UNUSED) + virDomainPCIConnectFlags virtioFlags) { virDomainPCIConnectFlags pciFlags = (VIR_PCI_CONNECT_TYPE_PCI_DEVICE | VIR_PCI_CONNECT_HOTPLUGGABLE); @@ -471,9 +470,28 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, } case VIR_DOMAIN_CONTROLLER_TYPE_IDE: + return pciFlags; + case VIR_DOMAIN_CONTROLLER_TYPE_SCSI: + switch ((virDomainControllerModelSCSI)cont->model) { + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VIRTIO_SCSI: + return virtioFlags; + + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_AUTO: + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_BUSLOGIC: + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LSILOGIC: + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LSISAS1068: + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_VMPVSCSI: + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_IBMVSCSI: + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LSISAS1078: + return pciFlags; + + case VIR_DOMAIN_CONTROLLER_MODEL_SCSI_LAST: + return 0; + } + case VIR_DOMAIN_CONTROLLER_TYPE_VIRTIO_SERIAL: - return pciFlags; + return virtioFlags; case VIR_DOMAIN_CONTROLLER_TYPE_FDC: case VIR_DOMAIN_CONTROLLER_TYPE_CCID: @@ -485,7 +503,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_DEVICE_FS: /* the only type of filesystem so far is virtio-9p-pci */ - return pciFlags; + return virtioFlags; case VIR_DOMAIN_DEVICE_NET: { virDomainNetDefPtr net = dev->data.net; @@ -498,6 +516,10 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, STREQ(net->model, "usb-net")) { return 0; } + + if (STREQ(net->model, "virtio")) + return virtioFlags; + return pciFlags; } @@ -519,7 +541,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_DEVICE_DISK: switch ((virDomainDiskBus) dev->data.disk->bus) { case VIR_DOMAIN_DISK_BUS_VIRTIO: - return pciFlags; /* only virtio disks use PCI */ + return virtioFlags; /* only virtio disks use PCI */ case VIR_DOMAIN_DISK_BUS_IDE: case VIR_DOMAIN_DISK_BUS_FDC: @@ -539,7 +561,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_DEVICE_MEMBALLOON: switch ((virDomainMemballoonModel) dev->data.memballoon->model) { case VIR_DOMAIN_MEMBALLOON_MODEL_VIRTIO: - return pciFlags; + return virtioFlags; case VIR_DOMAIN_MEMBALLOON_MODEL_XEN: case VIR_DOMAIN_MEMBALLOON_MODEL_NONE: @@ -550,7 +572,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_DEVICE_RNG: switch ((virDomainRNGModel) dev->data.rng->model) { case VIR_DOMAIN_RNG_MODEL_VIRTIO: - return pciFlags; + return virtioFlags; case VIR_DOMAIN_RNG_MODEL_LAST: return 0; @@ -571,6 +593,8 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_DEVICE_VIDEO: switch ((virDomainVideoType) dev->data.video->type) { case VIR_DOMAIN_VIDEO_TYPE_VIRTIO: + return virtioFlags; + case VIR_DOMAIN_VIDEO_TYPE_VGA: case VIR_DOMAIN_VIDEO_TYPE_CIRRUS: case VIR_DOMAIN_VIDEO_TYPE_VMVGA: @@ -584,14 +608,13 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, return 0; } - case VIR_DOMAIN_DEVICE_SHMEM: return pciFlags; case VIR_DOMAIN_DEVICE_INPUT: switch ((virDomainInputBus) dev->data.input->bus) { case VIR_DOMAIN_INPUT_BUS_VIRTIO: - return pciFlags; + return virtioFlags; case VIR_DOMAIN_INPUT_BUS_PS2: case VIR_DOMAIN_INPUT_BUS_USB: diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args new file mode 100644 index 0000000..f07c085 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args @@ -0,0 +1,58 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/libexec/qemu-kvm \ +-name q35-test \ +-S \ +-M q35 \ +-m 2048 \ +-smp 2,sockets=2,cores=1,threads=1 \ +-uuid 11dbdcdd-4c3b-482b-8903-9bdb8c0a2774 \ +-nographic \ +-nodefaults \ +-monitor unix:/tmp/lib/domain--1-q35-test/monitor.sock,server,nowait \ +-no-acpi \ +-boot c \ +-device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ +-device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \ +-device ioh3420,port=0x10,chassis=3,id=pci.3,bus=pcie.0,addr=0x2 \ +-device ioh3420,port=0x18,chassis=4,id=pci.4,bus=pcie.0,addr=0x3 \ +-device ioh3420,port=0x20,chassis=5,id=pci.5,bus=pcie.0,addr=0x4 \ +-device ioh3420,port=0x28,chassis=6,id=pci.6,bus=pcie.0,addr=0x5 \ +-device ioh3420,port=0x30,chassis=7,id=pci.7,bus=pcie.0,addr=0x6 \ +-device ioh3420,port=0x38,chassis=8,id=pci.8,bus=pcie.0,addr=0x7 \ +-device ioh3420,port=0x40,chassis=9,id=pci.9,bus=pcie.0,addr=0x8 \ +-device ioh3420,port=0x48,chassis=10,id=pci.10,bus=pcie.0,addr=0x9 \ +-device ioh3420,port=0x50,chassis=11,id=pci.11,bus=pcie.0,addr=0xa \ +-device ioh3420,port=0x58,chassis=12,id=pci.12,bus=pcie.0,addr=0xb \ +-device ioh3420,port=0x60,chassis=13,id=pci.13,bus=pcie.0,addr=0xc \ +-device ioh3420,port=0x68,chassis=14,id=pci.14,bus=pcie.0,addr=0xd \ +-device ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 \ +-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,\ +addr=0x1d \ +-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1 \ +-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2 \ +-device virtio-scsi-pci,id=scsi0,bus=pci.6,addr=0x0 \ +-device virtio-serial-pci,id=virtio-serial0,bus=pci.5,addr=0x0 \ +-drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-virtio-disk1 \ +-device virtio-blk-pci,bus=pci.7,addr=0x0,drive=drive-virtio-disk1,\ +id=virtio-disk1 \ +-fsdev local,security_model=passthrough,id=fsdev-fs0,path=/export/to/guest \ +-device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=/import/from/host,\ +bus=pci.3,addr=0x0 \ +-netdev user,id=hostnet0 \ +-device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.4,\ +addr=0x0 \ +-device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.10,\ +addr=0x0 \ +-device virtio-mouse-pci,id=input1,bus=pci.11,addr=0x0 \ +-device virtio-keyboard-pci,id=input2,bus=pci.12,addr=0x0 \ +-device virtio-tablet-pci,id=input3,bus=pci.13,addr=0x0 \ +-device virtio-gpu-pci,id=video0,bus=pcie.0,addr=0x1 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.8,addr=0x0 \ +-object rng-random,id=objrng0,filename=/dev/urandom \ +-device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=123,period=1234,bus=pci.9,\ +addr=0x0 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml new file mode 100644 index 0000000..be2439e --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml @@ -0,0 +1,61 @@ +<domain type='qemu'> + <name>q35-test</name> + <uuid>11dbdcdd-4c3b-482b-8903-9bdb8c0a2774</uuid> + <memory unit='KiB'>2097152</memory> + <currentMemory unit='KiB'>2097152</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='x86_64' machine='q35'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <controller type='pci' model='pcie-root'/> + <controller type='pci' model='dmi-to-pci-bridge'/> + <controller type='pci' model='pci-bridge'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='virtio-serial'/> + <controller type='scsi' model='virtio-scsi'/> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='vdb' bus='virtio'/> + </disk> + <filesystem type='mount'> + <source dir='/export/to/guest'/> + <target dir='/import/from/host'/> + </filesystem> + <video> + <model type='virtio'/> + </video> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + </interface> + <memballoon model='virtio'/> + <rng model='virtio'> + <rate bytes='123' period='1234'/> + <backend model='random'>/dev/urandom</backend> + </rng> + <input type='passthrough' bus='virtio'> + <source evdev='/dev/input/event1234'/> + </input> + <input type='mouse' bus='virtio'/> + <input type='keyboard' bus='virtio'/> + <input type='tablet' bus='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args new file mode 100644 index 0000000..98284a2 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args @@ -0,0 +1,58 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/libexec/qemu-kvm \ +-name q35-test \ +-S \ +-M q35 \ +-m 2048 \ +-smp 2,sockets=2,cores=1,threads=1 \ +-uuid 11dbdcdd-4c3b-482b-8903-9bdb8c0a2774 \ +-nographic \ +-nodefaults \ +-monitor unix:/tmp/lib/domain--1-q35-test/monitor.sock,server,nowait \ +-no-acpi \ +-boot c \ +-device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ +-device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \ +-device ioh3420,port=0x10,chassis=3,id=pci.3,bus=pcie.0,addr=0x2 \ +-device ioh3420,port=0x18,chassis=4,id=pci.4,bus=pcie.0,addr=0x3 \ +-device ioh3420,port=0x20,chassis=5,id=pci.5,bus=pcie.0,addr=0x4 \ +-device ioh3420,port=0x28,chassis=6,id=pci.6,bus=pcie.0,addr=0x5 \ +-device ioh3420,port=0x30,chassis=7,id=pci.7,bus=pcie.0,addr=0x6 \ +-device ioh3420,port=0x38,chassis=8,id=pci.8,bus=pcie.0,addr=0x7 \ +-device ioh3420,port=0x40,chassis=9,id=pci.9,bus=pcie.0,addr=0x8 \ +-device ioh3420,port=0x48,chassis=10,id=pci.10,bus=pcie.0,addr=0x9 \ +-device ioh3420,port=0x50,chassis=11,id=pci.11,bus=pcie.0,addr=0xa \ +-device ioh3420,port=0x58,chassis=12,id=pci.12,bus=pcie.0,addr=0xb \ +-device ioh3420,port=0x60,chassis=13,id=pci.13,bus=pcie.0,addr=0xc \ +-device ioh3420,port=0x68,chassis=14,id=pci.14,bus=pcie.0,addr=0xd \ +-device ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 \ +-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,\ +addr=0x1d \ +-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1 \ +-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2 \ +-device virtio-scsi-pci,id=scsi0,bus=pci.2,addr=0x4 \ +-device virtio-serial-pci,id=virtio-serial0,bus=pci.2,addr=0x3 \ +-drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-virtio-disk1 \ +-device virtio-blk-pci,bus=pci.2,addr=0x5,drive=drive-virtio-disk1,\ +id=virtio-disk1 \ +-fsdev local,security_model=passthrough,id=fsdev-fs0,path=/export/to/guest \ +-device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=/import/from/host,\ +bus=pci.2,addr=0x1 \ +-netdev user,id=hostnet0 \ +-device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.2,\ +addr=0x2 \ +-device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.2,\ +addr=0x8 \ +-device virtio-mouse-pci,id=input1,bus=pci.2,addr=0x9 \ +-device virtio-keyboard-pci,id=input2,bus=pci.2,addr=0xa \ +-device virtio-tablet-pci,id=input3,bus=pci.2,addr=0xb \ +-device virtio-gpu-pci,id=video0,bus=pcie.0,addr=0x1 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x6 \ +-object rng-random,id=objrng0,filename=/dev/urandom \ +-device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=123,period=1234,bus=pci.2,\ +addr=0x7 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.xml b/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.xml new file mode 120000 index 0000000..fc8c0ad --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.xml @@ -0,0 +1 @@ +qemuxml2argv-q35-pcie.xml \ No newline at end of file diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index d025930..0d5afa2 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1769,6 +1769,54 @@ mymain(void) QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, QEMU_CAPS_DEVICE_QXL); + /* verify that devices with pcie capability are assigned to a pcie slot */ + DO_TEST("q35-pcie", + QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, + QEMU_CAPS_DEVICE_VIRTIO_RNG, + QEMU_CAPS_OBJECT_RNG_RANDOM, + QEMU_CAPS_NETDEV, + QEMU_CAPS_DEVICE_VIRTIO_NET, + QEMU_CAPS_DEVICE_VIRTIO_GPU, + QEMU_CAPS_VIRTIO_GPU_VIRGL, + QEMU_CAPS_VIRTIO_KEYBOARD, + QEMU_CAPS_VIRTIO_MOUSE, + QEMU_CAPS_VIRTIO_TABLET, + QEMU_CAPS_VIRTIO_INPUT_HOST, + QEMU_CAPS_VIRTIO_SCSI, + QEMU_CAPS_FSDEV, + QEMU_CAPS_FSDEV_WRITEOUT, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY); + /* same XML as q35-pcie, but don't set + * QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, so virtio devices should + * be assigned to legacy pci slots + */ + DO_TEST("q35-virtio-pci", + QEMU_CAPS_DEVICE_VIRTIO_RNG, + QEMU_CAPS_OBJECT_RNG_RANDOM, + QEMU_CAPS_NETDEV, + QEMU_CAPS_DEVICE_VIRTIO_NET, + QEMU_CAPS_DEVICE_VIRTIO_GPU, + QEMU_CAPS_VIRTIO_GPU_VIRGL, + QEMU_CAPS_VIRTIO_KEYBOARD, + QEMU_CAPS_VIRTIO_MOUSE, + QEMU_CAPS_VIRTIO_TABLET, + QEMU_CAPS_VIRTIO_INPUT_HOST, + QEMU_CAPS_VIRTIO_SCSI, + QEMU_CAPS_FSDEV, + QEMU_CAPS_FSDEV_WRITEOUT, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY); DO_TEST("pcie-root-port", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml new file mode 100644 index 0000000..48e9b02 --- /dev/null +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml @@ -0,0 +1,154 @@ +<domain type='qemu'> + <name>q35-test</name> + <uuid>11dbdcdd-4c3b-482b-8903-9bdb8c0a2774</uuid> + <memory unit='KiB'>2097152</memory> + <currentMemory unit='KiB'>2097152</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='x86_64' machine='q35'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='vdb' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> + </disk> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='dmi-to-pci-bridge'> + <model name='i82801b11-bridge'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pci-bridge'> + <model name='pci-bridge'/> + <target chassisNr='2'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='3' port='0x10'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='4' port='0x18'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </controller> + <controller type='pci' index='5' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='5' port='0x20'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </controller> + <controller type='pci' index='6' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='6' port='0x28'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </controller> + <controller type='pci' index='7' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='7' port='0x30'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> + </controller> + <controller type='pci' index='8' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='8' port='0x38'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> + </controller> + <controller type='pci' index='9' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='9' port='0x40'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> + </controller> + <controller type='pci' index='10' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='10' port='0x48'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> + </controller> + <controller type='pci' index='11' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='11' port='0x50'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> + </controller> + <controller type='pci' index='12' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='12' port='0x58'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> + </controller> + <controller type='pci' index='13' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='13' port='0x60'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/> + </controller> + <controller type='pci' index='14' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='14' port='0x68'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/> + </controller> + <controller type='virtio-serial' index='0'> + <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> + </controller> + <controller type='scsi' index='0' model='virtio-scsi'> + <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + </controller> + <controller type='usb' index='0' model='ich9-ehci1'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/> + </controller> + <controller type='usb' index='0' model='ich9-uhci1'> + <master startport='0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/> + </controller> + <controller type='usb' index='0' model='ich9-uhci2'> + <master startport='2'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/> + </controller> + <controller type='usb' index='0' model='ich9-uhci3'> + <master startport='4'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/> + </controller> + <controller type='sata' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> + </controller> + <filesystem type='mount' accessmode='passthrough'> + <source dir='/export/to/guest'/> + <target dir='/import/from/host'/> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </filesystem> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </interface> + <input type='passthrough' bus='virtio'> + <source evdev='/dev/input/event1234'/> + <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> + </input> + <input type='mouse' bus='virtio'> + <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> + </input> + <input type='keyboard' bus='virtio'> + <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> + </input> + <input type='tablet' bus='virtio'> + <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> + </input> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <video> + <model type='virtio' heads='1' primary='yes'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </video> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> + </memballoon> + <rng model='virtio'> + <rate bytes='123' period='1234'/> + <backend model='random'>/dev/urandom</backend> + <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> + </rng> + </devices> +</domain> diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml new file mode 100644 index 0000000..ae541d3 --- /dev/null +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml @@ -0,0 +1,154 @@ +<domain type='qemu'> + <name>q35-test</name> + <uuid>11dbdcdd-4c3b-482b-8903-9bdb8c0a2774</uuid> + <memory unit='KiB'>2097152</memory> + <currentMemory unit='KiB'>2097152</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='x86_64' machine='q35'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='vdb' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x05' function='0x0'/> + </disk> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='dmi-to-pci-bridge'> + <model name='i82801b11-bridge'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pci-bridge'> + <model name='pci-bridge'/> + <target chassisNr='2'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='3' port='0x10'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='4' port='0x18'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </controller> + <controller type='pci' index='5' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='5' port='0x20'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </controller> + <controller type='pci' index='6' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='6' port='0x28'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </controller> + <controller type='pci' index='7' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='7' port='0x30'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> + </controller> + <controller type='pci' index='8' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='8' port='0x38'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> + </controller> + <controller type='pci' index='9' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='9' port='0x40'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> + </controller> + <controller type='pci' index='10' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='10' port='0x48'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> + </controller> + <controller type='pci' index='11' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='11' port='0x50'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> + </controller> + <controller type='pci' index='12' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='12' port='0x58'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> + </controller> + <controller type='pci' index='13' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='13' port='0x60'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/> + </controller> + <controller type='pci' index='14' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='14' port='0x68'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/> + </controller> + <controller type='virtio-serial' index='0'> + <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/> + </controller> + <controller type='scsi' index='0' model='virtio-scsi'> + <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> + </controller> + <controller type='usb' index='0' model='ich9-ehci1'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/> + </controller> + <controller type='usb' index='0' model='ich9-uhci1'> + <master startport='0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/> + </controller> + <controller type='usb' index='0' model='ich9-uhci2'> + <master startport='2'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/> + </controller> + <controller type='usb' index='0' model='ich9-uhci3'> + <master startport='4'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/> + </controller> + <controller type='sata' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> + </controller> + <filesystem type='mount' accessmode='passthrough'> + <source dir='/export/to/guest'/> + <target dir='/import/from/host'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/> + </filesystem> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/> + </interface> + <input type='passthrough' bus='virtio'> + <source evdev='/dev/input/event1234'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x08' function='0x0'/> + </input> + <input type='mouse' bus='virtio'> + <address type='pci' domain='0x0000' bus='0x02' slot='0x09' function='0x0'/> + </input> + <input type='keyboard' bus='virtio'> + <address type='pci' domain='0x0000' bus='0x02' slot='0x0a' function='0x0'/> + </input> + <input type='tablet' bus='virtio'> + <address type='pci' domain='0x0000' bus='0x02' slot='0x0b' function='0x0'/> + </input> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <video> + <model type='virtio' heads='1' primary='yes'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </video> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x02' slot='0x06' function='0x0'/> + </memballoon> + <rng model='virtio'> + <rate bytes='123' period='1234'/> + <backend model='random'>/dev/urandom</backend> + <address type='pci' domain='0x0000' bus='0x02' slot='0x07' function='0x0'/> + </rng> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 8a2b5ff..8d5766d 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -696,6 +696,49 @@ mymain(void) QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, QEMU_CAPS_DEVICE_QXL); + DO_TEST("q35-pcie", + QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, + QEMU_CAPS_DEVICE_VIRTIO_RNG, + QEMU_CAPS_OBJECT_RNG_RANDOM, + QEMU_CAPS_DEVICE_VIRTIO_NET, + QEMU_CAPS_DEVICE_VIRTIO_GPU, + QEMU_CAPS_VIRTIO_GPU_VIRGL, + QEMU_CAPS_VIRTIO_KEYBOARD, + QEMU_CAPS_VIRTIO_MOUSE, + QEMU_CAPS_VIRTIO_TABLET, + QEMU_CAPS_VIRTIO_INPUT_HOST, + QEMU_CAPS_VIRTIO_SCSI, + QEMU_CAPS_FSDEV, + QEMU_CAPS_FSDEV_WRITEOUT, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY); + /* same XML as q35-pcie, but don't set + QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY */ + DO_TEST("q35-virtio-pci", + QEMU_CAPS_DEVICE_VIRTIO_RNG, + QEMU_CAPS_OBJECT_RNG_RANDOM, + QEMU_CAPS_DEVICE_VIRTIO_NET, + QEMU_CAPS_DEVICE_VIRTIO_GPU, + QEMU_CAPS_VIRTIO_GPU_VIRGL, + QEMU_CAPS_VIRTIO_KEYBOARD, + QEMU_CAPS_VIRTIO_MOUSE, + QEMU_CAPS_VIRTIO_TABLET, + QEMU_CAPS_VIRTIO_INPUT_HOST, + QEMU_CAPS_VIRTIO_SCSI, + QEMU_CAPS_FSDEV, + QEMU_CAPS_FSDEV_WRITEOUT, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY); DO_TEST("pcie-root", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
libvirt previously assigned nearly all devices to a "hotpluggable" legacy PCI slot even on machines with a PCIe root bus (and even though most such machines don't even support hotplug on legacy PCI slots!) Forcing all devices onto legacy PCI slots means that the domain will need a dmi-to-pci-bridge (to convert from PCIe to legacy PCI) and a pci-bridge (to provide hotpluggable legacy PCI slots which, again, usually aren't hotpluggable anyway). To help reduce the need for these legacy controllers, this patch tries to assign virtio-1.0-capable devices to PCIe slots whenever possible, by setting appropriate connectFlags in virDomainCalculateDevicePCIConnectFlags(). Happily, when that function was written (just a few commits ago) it was created with a "virtioFlags" argument, set by both of its callers, which is the proper connectFlags to set for any virtio-*-pci device - depending on the arch/machinetype of the domain, and whether or not the qemu binary supports virtio-1.0, that flag will have either been set to PCI or PCIE. This patch merely enables the functionality by setting the flags
s/PCIE/PCIe/
for the device to whatever is in virtioFlags if the device is a virtio-*-pci device. NB: the first virtio video device will be placed directly on bus 0 slot 1 rather than on a pcie-root-port due to the override for primary video devices in qemuDomainValidateDevicePCISlotsQ35(). Whether or not to change that is a topic of discussion, but this patch doesn't change that particular behavior. NB2: since the slot must be hotpluggable, and pcie-root (the PCIe root complex) does *not* support hotplug, this means that suitable controllers must also be in the config (i.e. either pcie-root-port, or pcie-downstream-port). For now, libvirt doesn't add those automatically, so if you put virtio devices in a config for a qemu that has PCIe-capable virtio devices, you'll need to add extra pcie-root-ports yourself. That requirement will be eliminated in a future patch, but for now, it's simple to do this: <controller type='pci' model='pcie-root-port'/> <controller type='pci' model='pcie-root-port'/> <controller type='pci' model='pcie-root-port'/> ... Partially Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1330024
[...]
@@ -471,9 +470,28 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, } case VIR_DOMAIN_CONTROLLER_TYPE_IDE: + return pciFlags; + case VIR_DOMAIN_CONTROLLER_TYPE_SCSI: + switch ((virDomainControllerModelSCSI)cont->model) {
Add a space before 'cont->model' for consistency.
@@ -584,14 +608,13 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, return 0; } -
Spurious change, but it's better to fix it here than to leave it as it is :) [...]
+++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml @@ -0,0 +1,61 @@ +<domain type='qemu'> + <name>q35-test</name> + <uuid>11dbdcdd-4c3b-482b-8903-9bdb8c0a2774</uuid> + <memory unit='KiB'>2097152</memory> + <currentMemory unit='KiB'>2097152</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='x86_64' machine='q35'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <controller type='pci' model='pcie-root'/> + <controller type='pci' model='dmi-to-pci-bridge'/> + <controller type='pci' model='pci-bridge'/>
These controllers are still needed at this point, but we should drop them from the input XML once that's no longer the case, eg. definitely after patch 12/17. Unless of course you've already thought of that and added more minimal use cases down the line, which I've simply not yet gotten to...
+ <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/>
... which you probably have, because we definitely need to make sure libvirt can add at least these for us :) ACK with the minor stuff mentioned above taken care of. -- Andrea Bolognani / Red Hat / Virtualization
The e1000e is an emulated network device based on the Intel 82574, present in qemu 2.7.0 and later. Among other differences from the e1000, it presents itself as a PCIe device rather than legacy PCI. In order to get it assigned to a PCIe controller, this patch updates the flags setting for network devices when the model name is "e1000e". (Note that for some reason libvirt has never validated the network device model names other than to check that there are no dangerous characters in them. That should probably change, but is the subject of another patch.) Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1343094 --- src/qemu/qemu_domain_address.c | 6 ++++-- tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args | 23 ++++++++++++---------- tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml | 4 ++++ .../qemuxml2argv-q35-virtio-pci.args | 3 +++ .../qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml | 23 +++++++++++++--------- .../qemuxml2xmlout-q35-virtio-pci.xml | 5 +++++ 6 files changed, 43 insertions(+), 21 deletions(-) diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 3f3e9e4..4c59a28 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -428,8 +428,7 @@ qemuDomainAssignARMVirtioMMIOAddresses(virDomainDefPtr def, */ static virDomainPCIConnectFlags qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, - virDomainPCIConnectFlags pcieFlags - ATTRIBUTE_UNUSED, + virDomainPCIConnectFlags pcieFlags, virDomainPCIConnectFlags virtioFlags) { virDomainPCIConnectFlags pciFlags = (VIR_PCI_CONNECT_TYPE_PCI_DEVICE | @@ -520,6 +519,9 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, if (STREQ(net->model, "virtio")) return virtioFlags; + if (STREQ(net->model, "e1000e")) + return pcieFlags; + return pciFlags; } diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args index f07c085..d92e0b8 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args @@ -35,10 +35,10 @@ QEMU_AUDIO_DRV=none \ addr=0x1d \ -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1 \ -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2 \ --device virtio-scsi-pci,id=scsi0,bus=pci.6,addr=0x0 \ --device virtio-serial-pci,id=virtio-serial0,bus=pci.5,addr=0x0 \ +-device virtio-scsi-pci,id=scsi0,bus=pci.7,addr=0x0 \ +-device virtio-serial-pci,id=virtio-serial0,bus=pci.6,addr=0x0 \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-virtio-disk1 \ --device virtio-blk-pci,bus=pci.7,addr=0x0,drive=drive-virtio-disk1,\ +-device virtio-blk-pci,bus=pci.8,addr=0x0,drive=drive-virtio-disk1,\ id=virtio-disk1 \ -fsdev local,security_model=passthrough,id=fsdev-fs0,path=/export/to/guest \ -device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=/import/from/host,\ @@ -46,13 +46,16 @@ bus=pci.3,addr=0x0 \ -netdev user,id=hostnet0 \ -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.4,\ addr=0x0 \ --device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.10,\ +-netdev user,id=hostnet1 \ +-device e1000e,netdev=hostnet1,id=net1,mac=00:11:22:33:44:66,bus=pci.5,\ addr=0x0 \ --device virtio-mouse-pci,id=input1,bus=pci.11,addr=0x0 \ --device virtio-keyboard-pci,id=input2,bus=pci.12,addr=0x0 \ --device virtio-tablet-pci,id=input3,bus=pci.13,addr=0x0 \ +-device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.11,\ +addr=0x0 \ +-device virtio-mouse-pci,id=input1,bus=pci.12,addr=0x0 \ +-device virtio-keyboard-pci,id=input2,bus=pci.13,addr=0x0 \ +-device virtio-tablet-pci,id=input3,bus=pci.14,addr=0x0 \ -device virtio-gpu-pci,id=video0,bus=pcie.0,addr=0x1 \ --device virtio-balloon-pci,id=balloon0,bus=pci.8,addr=0x0 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.9,addr=0x0 \ -object rng-random,id=objrng0,filename=/dev/urandom \ --device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=123,period=1234,bus=pci.9,\ -addr=0x0 +-device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=123,period=1234,\ +bus=pci.10,addr=0x0 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml index be2439e..39db5f0 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml @@ -46,6 +46,10 @@ <mac address='00:11:22:33:44:55'/> <model type='virtio'/> </interface> + <interface type='user'> + <mac address='00:11:22:33:44:66'/> + <model type='e1000e'/> + </interface> <memballoon model='virtio'/> <rng model='virtio'> <rate bytes='123' period='1234'/> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args index 98284a2..68d19ca 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args @@ -46,6 +46,9 @@ bus=pci.2,addr=0x1 \ -netdev user,id=hostnet0 \ -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.2,\ addr=0x2 \ +-netdev user,id=hostnet1 \ +-device e1000e,netdev=hostnet1,id=net1,mac=00:11:22:33:44:66,bus=pci.3,\ +addr=0x0 \ -device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.2,\ addr=0x8 \ -device virtio-mouse-pci,id=input1,bus=pci.2,addr=0x9 \ diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml index 48e9b02..5a23a51 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml @@ -17,7 +17,7 @@ <disk type='block' device='disk'> <source dev='/dev/HostVG/QEMUGuest1'/> <target dev='vdb' bus='virtio'/> - <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </disk> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> @@ -90,10 +90,10 @@ <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> - <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </controller> <controller type='scsi' index='0' model='virtio-scsi'> - <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </controller> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/> @@ -123,18 +123,23 @@ <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </interface> + <interface type='user'> + <mac address='00:11:22:33:44:66'/> + <model type='e1000e'/> + <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> + </interface> <input type='passthrough' bus='virtio'> <source evdev='/dev/input/event1234'/> - <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> </input> <input type='mouse' bus='virtio'> - <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> </input> <input type='keyboard' bus='virtio'> - <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> </input> <input type='tablet' bus='virtio'> - <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0e' slot='0x00' function='0x0'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> @@ -143,12 +148,12 @@ <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <memballoon model='virtio'> - <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </memballoon> <rng model='virtio'> <rate bytes='123' period='1234'/> <backend model='random'>/dev/urandom</backend> - <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> </rng> </devices> </domain> diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml index ae541d3..22c2c68 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml @@ -123,6 +123,11 @@ <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/> </interface> + <interface type='user'> + <mac address='00:11:22:33:44:66'/> + <model type='e1000e'/> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </interface> <input type='passthrough' bus='virtio'> <source evdev='/dev/input/event1234'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x08' function='0x0'/> -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
The e1000e is an emulated network device based on the Intel 82574, present in qemu 2.7.0 and later. Among other differences from the e1000, it presents itself as a PCIe device rather than legacy PCI. In order to get it assigned to a PCIe controller, this patch updates the flags setting for network devices when the model name is "e1000e".
(Note that for some reason libvirt has never validated the network device model names other than to check that there are no dangerous characters in them. That should probably change, but is the subject of another patch.)
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1343094 --- src/qemu/qemu_domain_address.c | 6 ++++-- tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args | 23 ++++++++++++---------- tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml | 4 ++++ .../qemuxml2argv-q35-virtio-pci.args | 3 +++ .../qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml | 23 +++++++++++++--------- .../qemuxml2xmlout-q35-virtio-pci.xml | 5 +++++ 6 files changed, 43 insertions(+), 21 deletions(-)
ACK -- Andrea Bolognani / Red Hat / Virtualization
The nec-usb-xhci device (which is a USB3 controller) has always presented itself as a PCI device when plugged into a legacy PCI slot, and a PCIe device when plugged into a PCIe slot, but libvirt has always auto-assigned it to a legacy PCI slot. This patch changes that behavior to auto-assign to a PCIe slot on systems that have pcie-root (e.g. Q35 and aarch64/virt). Since we don't yet auto-create pcie-*-port controllers on demand, this means a config with an nec-xhci USB controller that has no PCI address assigned will also need to have an otherwise-unused pcie-*-port controller specified: <controller type='pci' model='pcie-root-port'/> <controller type='usb' model='nec-xhci'/> (this assumes there is an otherwise-unused slot on pcie-root to accept the pcie-root-port) --- src/qemu/qemu_domain_address.c | 2 +- tests/qemuxml2argvdata/qemuxml2argv-autoindex.args | 10 +++---- tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args | 21 ++++++------- tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml | 2 ++ .../qemuxml2argv-q35-virtio-pci.args | 7 ++--- tests/qemuxml2argvtest.c | 2 ++ .../qemuxml2xmlout-autoindex.xml | 10 +++---- .../qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml | 35 +++++++++------------- .../qemuxml2xmlout-q35-virtio-pci.xml | 21 +++++-------- tests/qemuxml2xmltest.c | 2 ++ 10 files changed, 49 insertions(+), 63 deletions(-) diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 4c59a28..94f9534 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -448,7 +448,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, case VIR_DOMAIN_CONTROLLER_TYPE_USB: switch ((virDomainControllerModelUSB) cont->model) { case VIR_DOMAIN_CONTROLLER_MODEL_USB_NEC_XHCI: - return pciFlags; + return pcieFlags; case VIR_DOMAIN_CONTROLLER_MODEL_USB_EHCI: case VIR_DOMAIN_CONTROLLER_MODEL_USB_ICH9_EHCI1: diff --git a/tests/qemuxml2argvdata/qemuxml2argv-autoindex.args b/tests/qemuxml2argvdata/qemuxml2argv-autoindex.args index 43b9661..fc14dc4 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-autoindex.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-autoindex.args @@ -43,11 +43,11 @@ addr=0x1 \ -device ich9-usb-uhci2,masterbus=usb2.0,firstport=2,bus=pci.2,addr=0x1.0x1 \ -device ich9-usb-uhci3,masterbus=usb2.0,firstport=4,bus=pci.2,addr=0x1.0x2 \ -device ich9-usb-ehci1,id=usb2,bus=pci.2,addr=0x1.0x7 \ --device nec-usb-xhci,id=usb3,bus=pci.2,addr=0x2 \ +-device nec-usb-xhci,id=usb3,bus=pci.5,addr=0x0 \ -device ich9-usb-uhci1,masterbus=usb4.0,firstport=0,bus=pci.2,multifunction=on,\ -addr=0x3 \ --device ich9-usb-uhci2,masterbus=usb4.0,firstport=2,bus=pci.2,addr=0x3.0x1 \ --device ich9-usb-uhci3,masterbus=usb4.0,firstport=4,bus=pci.2,addr=0x3.0x2 \ --device ich9-usb-ehci1,id=usb4,bus=pci.2,addr=0x3.0x7 \ +addr=0x2 \ +-device ich9-usb-uhci2,masterbus=usb4.0,firstport=2,bus=pci.2,addr=0x2.0x1 \ +-device ich9-usb-uhci3,masterbus=usb4.0,firstport=4,bus=pci.2,addr=0x2.0x2 \ +-device ich9-usb-ehci1,id=usb4,bus=pci.2,addr=0x2.0x7 \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-sata0-0-0 \ -device ide-drive,bus=ide.0,drive=drive-sata0-0-0,id=sata0-0-0 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args index d92e0b8..2738749 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args @@ -30,15 +30,12 @@ QEMU_AUDIO_DRV=none \ -device ioh3420,port=0x58,chassis=12,id=pci.12,bus=pcie.0,addr=0xb \ -device ioh3420,port=0x60,chassis=13,id=pci.13,bus=pcie.0,addr=0xc \ -device ioh3420,port=0x68,chassis=14,id=pci.14,bus=pcie.0,addr=0xd \ --device ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 \ --device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,\ -addr=0x1d \ --device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1 \ --device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2 \ +-device ioh3420,port=0x70,chassis=15,id=pci.15,bus=pcie.0,addr=0xe \ +-device nec-usb-xhci,id=usb,bus=pci.8,addr=0x0 \ -device virtio-scsi-pci,id=scsi0,bus=pci.7,addr=0x0 \ -device virtio-serial-pci,id=virtio-serial0,bus=pci.6,addr=0x0 \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-virtio-disk1 \ --device virtio-blk-pci,bus=pci.8,addr=0x0,drive=drive-virtio-disk1,\ +-device virtio-blk-pci,bus=pci.9,addr=0x0,drive=drive-virtio-disk1,\ id=virtio-disk1 \ -fsdev local,security_model=passthrough,id=fsdev-fs0,path=/export/to/guest \ -device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=/import/from/host,\ @@ -49,13 +46,13 @@ addr=0x0 \ -netdev user,id=hostnet1 \ -device e1000e,netdev=hostnet1,id=net1,mac=00:11:22:33:44:66,bus=pci.5,\ addr=0x0 \ --device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.11,\ +-device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.12,\ addr=0x0 \ --device virtio-mouse-pci,id=input1,bus=pci.12,addr=0x0 \ --device virtio-keyboard-pci,id=input2,bus=pci.13,addr=0x0 \ --device virtio-tablet-pci,id=input3,bus=pci.14,addr=0x0 \ +-device virtio-mouse-pci,id=input1,bus=pci.13,addr=0x0 \ +-device virtio-keyboard-pci,id=input2,bus=pci.14,addr=0x0 \ +-device virtio-tablet-pci,id=input3,bus=pci.15,addr=0x0 \ -device virtio-gpu-pci,id=video0,bus=pcie.0,addr=0x1 \ --device virtio-balloon-pci,id=balloon0,bus=pci.9,addr=0x0 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.10,addr=0x0 \ -object rng-random,id=objrng0,filename=/dev/urandom \ -device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=123,period=1234,\ -bus=pci.10,addr=0x0 +bus=pci.11,addr=0x0 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml index 39db5f0..b6b971b 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml @@ -29,8 +29,10 @@ <controller type='pci' model='pcie-root-port'/> <controller type='pci' model='pcie-root-port'/> <controller type='pci' model='pcie-root-port'/> + <controller type='pci' model='pcie-root-port'/> <controller type='virtio-serial'/> <controller type='scsi' model='virtio-scsi'/> + <controller type='usb' model='nec-xhci'/> <disk type='block' device='disk'> <source dev='/dev/HostVG/QEMUGuest1'/> <target dev='vdb' bus='virtio'/> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args index 68d19ca..cada05e 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-virtio-pci.args @@ -30,11 +30,8 @@ QEMU_AUDIO_DRV=none \ -device ioh3420,port=0x58,chassis=12,id=pci.12,bus=pcie.0,addr=0xb \ -device ioh3420,port=0x60,chassis=13,id=pci.13,bus=pcie.0,addr=0xc \ -device ioh3420,port=0x68,chassis=14,id=pci.14,bus=pcie.0,addr=0xd \ --device ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 \ --device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,\ -addr=0x1d \ --device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1 \ --device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2 \ +-device ioh3420,port=0x70,chassis=15,id=pci.15,bus=pcie.0,addr=0xe \ +-device nec-usb-xhci,id=usb,bus=pci.4,addr=0x0 \ -device virtio-scsi-pci,id=scsi0,bus=pci.2,addr=0x4 \ -device virtio-serial-pci,id=virtio-serial0,bus=pci.2,addr=0x3 \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-virtio-disk1 \ diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 0d5afa2..bbf5ff5 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1791,6 +1791,7 @@ mymain(void) QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY); /* same XML as q35-pcie, but don't set * QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, so virtio devices should @@ -1816,6 +1817,7 @@ mymain(void) QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY); DO_TEST("pcie-root-port", QEMU_CAPS_DEVICE_PCI_BRIDGE, diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-autoindex.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-autoindex.xml index 086a1cf..2a8a44a 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-autoindex.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-autoindex.xml @@ -126,22 +126,22 @@ <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x7'/> </controller> <controller type='usb' index='3' model='nec-xhci'> - <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </controller> <controller type='usb' index='4' model='ich9-uhci1'> <master startport='0'/> - <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0' multifunction='on'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='4' model='ich9-uhci2'> <master startport='2'/> - <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x1'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x1'/> </controller> <controller type='usb' index='4' model='ich9-uhci3'> <master startport='4'/> - <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x2'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x2'/> </controller> <controller type='usb' index='4' model='ich9-ehci1'> - <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x7'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x7'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml index 5a23a51..8e727fb 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml @@ -17,7 +17,7 @@ <disk type='block' device='disk'> <source dev='/dev/HostVG/QEMUGuest1'/> <target dev='vdb' bus='virtio'/> - <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </disk> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> @@ -89,26 +89,19 @@ <target chassis='14' port='0x68'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/> </controller> + <controller type='pci' index='15' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='15' port='0x70'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0e' function='0x0'/> + </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </controller> <controller type='scsi' index='0' model='virtio-scsi'> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </controller> - <controller type='usb' index='0' model='ich9-ehci1'> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/> - </controller> - <controller type='usb' index='0' model='ich9-uhci1'> - <master startport='0'/> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/> - </controller> - <controller type='usb' index='0' model='ich9-uhci2'> - <master startport='2'/> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/> - </controller> - <controller type='usb' index='0' model='ich9-uhci3'> - <master startport='4'/> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/> + <controller type='usb' index='0' model='nec-xhci'> + <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> @@ -130,16 +123,16 @@ </interface> <input type='passthrough' bus='virtio'> <source evdev='/dev/input/event1234'/> - <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> </input> <input type='mouse' bus='virtio'> - <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> </input> <input type='keyboard' bus='virtio'> - <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0e' slot='0x00' function='0x0'/> </input> <input type='tablet' bus='virtio'> - <address type='pci' domain='0x0000' bus='0x0e' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0f' slot='0x00' function='0x0'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> @@ -148,12 +141,12 @@ <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <memballoon model='virtio'> - <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> </memballoon> <rng model='virtio'> <rate bytes='123' period='1234'/> <backend model='random'>/dev/urandom</backend> - <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> </rng> </devices> </domain> diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml index 22c2c68..c4bd357 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virtio-pci.xml @@ -89,26 +89,19 @@ <target chassis='14' port='0x68'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/> </controller> + <controller type='pci' index='15' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='15' port='0x70'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0e' function='0x0'/> + </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/> </controller> <controller type='scsi' index='0' model='virtio-scsi'> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </controller> - <controller type='usb' index='0' model='ich9-ehci1'> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/> - </controller> - <controller type='usb' index='0' model='ich9-uhci1'> - <master startport='0'/> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/> - </controller> - <controller type='usb' index='0' model='ich9-uhci2'> - <master startport='2'/> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/> - </controller> - <controller type='usb' index='0' model='ich9-uhci3'> - <master startport='4'/> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/> + <controller type='usb' index='0' model='nec-xhci'> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 8d5766d..91d6059 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -716,6 +716,7 @@ mymain(void) QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY); /* same XML as q35-pcie, but don't set QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY */ @@ -738,6 +739,7 @@ mymain(void) QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY); DO_TEST("pcie-root", -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
The nec-usb-xhci device (which is a USB3 controller) has always presented itself as a PCI device when plugged into a legacy PCI slot, and a PCIe device when plugged into a PCIe slot, but libvirt has always auto-assigned it to a legacy PCI slot. This patch changes that behavior to auto-assign to a PCIe slot on systems that have pcie-root (e.g. Q35 and aarch64/virt). Since we don't yet auto-create pcie-*-port controllers on demand, this means a config with an nec-xhci USB controller that has no PCI address assigned will also need to have an otherwise-unused pcie-*-port controller specified: <controller type='pci' model='pcie-root-port'/> <controller type='usb' model='nec-xhci'/> (this assumes there is an otherwise-unused slot on pcie-root to accept the pcie-root-port) --- src/qemu/qemu_domain_address.c | 2 +- tests/qemuxml2argvdata/qemuxml2argv-autoindex.args | 10 +++---- tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.args | 21 ++++++------- tests/qemuxml2argvdata/qemuxml2argv-q35-pcie.xml | 2 ++ .../qemuxml2argv-q35-virtio-pci.args | 7 ++--- tests/qemuxml2argvtest.c | 2 ++ .../qemuxml2xmlout-autoindex.xml | 10 +++---- .../qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie.xml | 35 +++++++++------------- .../qemuxml2xmlout-q35-virtio-pci.xml | 21 +++++-------- tests/qemuxml2xmltest.c | 2 ++ 10 files changed, 49 insertions(+), 63 deletions(-)
I see there are existing tests to make sure nec-xhci is assigned to legacy PCI slots for machine types that don't have a pcie-root, so we should be covering all grounds still. ACK -- Andrea Bolognani / Red Hat / Virtualization
Andrea had the right idea when he disabled the "reserve an extra unused slot" bit for aarch64/virt. For *any* PCI Express-based machine, it is pointless since 1) an extra legacy-PCI slot can't be used for hotplug, since hotplug into legacy PCI slots doesn't work on PCI Express machinetypes, and 2) even for "coldplug" expansion, everybody will want to expand using Express controllers, not legacy PCI. This patch eliminates the extra slot reserve unless the system has a pci-root (i.e. legacy PCI) --- src/qemu/qemu_domain_address.c | 44 ++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 17 deletions(-) diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 94f9534..81c7bce 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -1807,8 +1807,6 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, */ virDomainDeviceInfo info; - info.pciConnectFlags = VIR_PCI_CONNECT_TYPE_PCI_DEVICE; - /* 1st pass to figure out how many PCI bridges we need */ if (!(addrs = qemuDomainPCIAddressSetCreate(def, nbuses, true))) goto cleanup; @@ -1817,23 +1815,35 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, addrs) < 0) goto cleanup; - for (i = 0; i < addrs->nbuses; i++) { - if (!qemuDomainPCIBusFullyReserved(&addrs->buses[i])) - buses_reserved = false; - } - - /* Reserve 1 extra slot for a (potential) bridge only if buses - * are not fully reserved yet. + /* For domains that have pci-root, reserve 1 extra slot for a + * (potential) bridge (for future expansion) only if buses are + * not fully reserved yet (if all buses are fully reserved + * with manually/previously assigned addresses, any attempt to + * reserve an extra slot would fail anyway. But if all buses + * are *not* fully reserved, this extra reservation might push + * the config to add a new pci-bridge to plug into the final + * available slot, thus preserving the ability to expand) * - * We don't reserve the extra slot for aarch64 mach-virt guests - * either because we want to be able to have pure virtio-mmio - * guests, and reserving this slot would force us to add at least - * a dmi-to-pci-bridge to an otherwise PCI-free topology + * We only do this for those domains that have pci-root, since + * those with pcie-root will usually want to expand using PCIe + * controllers, which we will do after assigning addresses for + * all *actual* devices. */ - if (!buses_reserved && - !qemuDomainMachineIsVirt(def) && - qemuDomainPCIAddressReserveNextSlot(addrs, &info) < 0) - goto cleanup; + + if (qemuDomainMachineHasPCIRoot(def)) { + info.pciConnectFlags = (VIR_PCI_CONNECT_HOTPLUGGABLE | + VIR_PCI_CONNECT_TYPE_PCI_DEVICE); + + for (i = 0; i < addrs->nbuses; i++) { + if (!qemuDomainPCIBusFullyReserved(&addrs->buses[i])) { + buses_reserved = false; + break; + } + } + if (!buses_reserved && + qemuDomainPCIAddressReserveNextSlot(addrs, &info) < 0) + goto cleanup; + } if (qemuDomainAssignDevicePCISlots(def, qemuCaps, addrs) < 0) goto cleanup; -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
Andrea had the right idea when he disabled the "reserve an extra unused slot" bit for aarch64/virt.
You bet I did! :P
For *any* PCI Express-based machine, it is pointless since 1) an extra legacy-PCI slot can't be used for hotplug, since hotplug into legacy PCI slots doesn't work on PCI Express machinetypes, and 2) even for "coldplug" expansion, everybody will want to expand using Express controllers, not legacy PCI. This patch eliminates the extra slot reserve unless the system has a pci-root (i.e. legacy PCI)
[...]
@@ -1817,23 +1815,35 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, addrs) < 0) goto cleanup; - for (i = 0; i < addrs->nbuses; i++) { - if (!qemuDomainPCIBusFullyReserved(&addrs->buses[i])) - buses_reserved = false; - } - - /* Reserve 1 extra slot for a (potential) bridge only if buses - * are not fully reserved yet. + /* For domains that have pci-root, reserve 1 extra slot for a + * (potential) bridge (for future expansion) only if buses are + * not fully reserved yet (if all buses are fully reserved + * with manually/previously assigned addresses, any attempt to + * reserve an extra slot would fail anyway. But if all buses + * are *not* fully reserved, this extra reservation might push + * the config to add a new pci-bridge to plug into the final + * available slot, thus preserving the ability to expand) * - * We don't reserve the extra slot for aarch64 mach-virt guests - * either because we want to be able to have pure virtio-mmio - * guests, and reserving this slot would force us to add at least - * a dmi-to-pci-bridge to an otherwise PCI-free topology + * We only do this for those domains that have pci-root, since + * those with pcie-root will usually want to expand using PCIe + * controllers, which we will do after assigning addresses for + * all *actual* devices. */ - if (!buses_reserved &&
I feel like I mentioned this in a previous review, but just in case I didn't: the declaration for buses_reserved should be moved to this inner scope, as it's not used at all outside of it.
- !qemuDomainMachineIsVirt(def) && - qemuDomainPCIAddressReserveNextSlot(addrs, &info) < 0) - goto cleanup; + + if (qemuDomainMachineHasPCIRoot(def)) { + info.pciConnectFlags = (VIR_PCI_CONNECT_HOTPLUGGABLE | + VIR_PCI_CONNECT_TYPE_PCI_DEVICE);
info almost falls into the same bucket, except I see later in the series it will be reused to make sure an empty, hotplug-capable PCIe slot is availabe to newly-defined guests. I would argue that, even then, both scopes are narrow enough that it would still be nicer to define info twice close to its usages than sharing a single variable with a very wide scope. That said, I leave the final call on this one entirely up to you :) ACK with the scope(s?) adjusted. -- Andrea Bolognani / Red Hat / Virtualization
Previously libvirt would only add pci-bridge devices automatically when an address was requested for a device that required a legacy PCI slot and none was available. This patch expands that support to dmi-to-pci-bridge (which is needed in order to add a pci-bridge on a machine with a pcie-root), and pcie-root-port (which is needed to add a hotpluggable PCIe device). It does *not* automatically add pcie-switch-upstream-ports or pcie-switch-downstream-ports (and currently there are no plans for that). Given the existing code to auto-add pci-bridge devices, automatically adding pcie-root-ports is fairly straightforward. The dmi-to-pci-bridge support is a bit tricky though, for a few reasons: 1) Although the only reason to add a dmi-to-pci-bridge is so that there is a reasonable place to plug in a pci-bridge controller, most of the time it's not the presence of a pci-bridge *in the config* that triggers the requirement to add a dmi-to-pci-bridge. Rather, it is the presence of a legacy-PCI device in the config, which triggers auto-add of a pci-bridge, which triggers auto-add of a dmi-to-pci-bridge (this is handled in virDomainPCIAddressSetGrow() - if there's a request to add a pci-bridge we'll check if there is a suitable bus to plug it into; if not, we first add a dmi-to-pci-bridge). 2) Once there is already a single dmi-to-pci-bridge on the system, there won't be a need for any more, even if it's full, as long as there is a pci-bridge with an open slot - you can also plug pci-bridges into existing pci-bridges. So we have to make sure we don't add a dmi-to-pci-bridge unless there aren't any dmi-to-pci-bridges *or* any pci-bridges. 3) Although it is strongly discouraged, it is legal for a pci-bridge to be directly plugged into pcie-root, and we don't want to auto-add a dmi-to-pci-bridge if there is already a pci-bridge that's been forced directly into pcie-root. Although libvirt will now automatically create a dmi-to-pci-bridge when it's needed, the code still remains for now that forces a dmi-to-pci-bridge on all domains with pcie-root (in qemuDomainDefAddDefaultDevices()). That will be removed in the next patch. For now, the pcie-root-ports are added one to a slot, which is a bit wasteful and means it will fail after 31 total PCIe devices (30 if there are also some PCI devices), but helps keep the changeset down for this patch. A future patch will have 8 pcie-root-ports sharing the functions on a single slot. --- src/conf/domain_addr.c | 96 +++++++++++--- src/qemu/qemu_domain_address.c | 55 +++++--- .../qemuxml2argv-q35-pcie-autoadd.args | 57 ++++++++ .../qemuxml2argv-q35-pcie-autoadd.xml | 51 +++++++ tests/qemuxml2argvtest.c | 24 ++++ .../qemuxml2xmlout-q35-pcie-autoadd.xml | 147 +++++++++++++++++++++ tests/qemuxml2xmltest.c | 25 +++- 7 files changed, 421 insertions(+), 34 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml diff --git a/src/conf/domain_addr.c b/src/conf/domain_addr.c index d3a2f6f..781050c 100644 --- a/src/conf/domain_addr.c +++ b/src/conf/domain_addr.c @@ -82,6 +82,30 @@ virDomainPCIControllerModelToConnectType(virDomainControllerModelPCI model) return 0; } + +static int +virDomainPCIControllerConnectTypeToModel(virDomainPCIConnectFlags flags) +{ + if (flags & VIR_PCI_CONNECT_TYPE_PCIE_ROOT_PORT) + return VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; + if (flags & VIR_PCI_CONNECT_TYPE_PCIE_SWITCH_UPSTREAM_PORT) + return VIR_DOMAIN_CONTROLLER_MODEL_PCIE_SWITCH_UPSTREAM_PORT; + if (flags & VIR_PCI_CONNECT_TYPE_PCIE_SWITCH_DOWNSTREAM_PORT) + return VIR_DOMAIN_CONTROLLER_MODEL_PCIE_SWITCH_DOWNSTREAM_PORT; + if (flags & VIR_PCI_CONNECT_TYPE_DMI_TO_PCI_BRIDGE) + return VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE; + if (flags & VIR_PCI_CONNECT_TYPE_PCI_EXPANDER_BUS) + return VIR_DOMAIN_CONTROLLER_MODEL_PCI_EXPANDER_BUS; + if (flags & VIR_PCI_CONNECT_TYPE_PCIE_EXPANDER_BUS) + return VIR_DOMAIN_CONTROLLER_MODEL_PCIE_EXPANDER_BUS; + if (flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE) + return VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE; + + /* some connect types don't correspond to a controller model */ + return -1; +} + + bool virDomainPCIAddressFlagsCompatible(virPCIDeviceAddressPtr addr, const char *addrStr, @@ -334,10 +358,7 @@ virDomainPCIAddressBusSetModel(virDomainPCIAddressBusPtr bus, } -/* Ensure addr fits in the address set, by expanding it if needed. - * This will only grow if the flags say that we need a normal - * hot-pluggable PCI slot. If we need a different type of slot, it - * will fail. +/* Ensure addr fits in the address set, by expanding it if needed * * Return value: * -1 = OOM @@ -351,32 +372,73 @@ virDomainPCIAddressSetGrow(virDomainPCIAddressSetPtr addrs, { int add; size_t i; + int model; + bool needDMIToPCIBridge = false; add = addr->bus - addrs->nbuses + 1; - i = addrs->nbuses; if (add <= 0) return 0; - /* auto-grow only works when we're adding plain PCI devices */ - if (!(flags & VIR_PCI_CONNECT_TYPE_PCI_DEVICE)) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Cannot automatically add a new PCI bus for a " - "device requiring a slot other than standard PCI.")); + if (flags & VIR_PCI_CONNECT_TYPE_PCI_DEVICE) { + model = VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE; + + /* if there aren't yet any buses that will accept a + * pci-bridge, and the caller is asking for one, we'll need to + * add a dmi-to-pci-bridge first. + */ + needDMIToPCIBridge = true; + for (i = 0; i < addrs->nbuses; i++) { + if (addrs->buses[i].flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE) { + needDMIToPCIBridge = false; + break; + } + } + if (needDMIToPCIBridge && add == 1) { + /* we need to add at least two buses - one dmi-to-pci, + * and the other the requested pci-bridge + */ + add++; + addr->bus++; + } + } else if (flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE && + addrs->buses[0].model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) { + model = VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE; + } else if (flags & (VIR_PCI_CONNECT_TYPE_PCIE_DEVICE | + VIR_PCI_CONNECT_TYPE_PCIE_SWITCH_UPSTREAM_PORT)) { + model = VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; + } else { + int existingContModel = virDomainPCIControllerConnectTypeToModel(flags); + + if (existingContModel >= 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("a PCI slot is needed to connect a PCI controller " + "model='%s', but none is available, and it " + "cannot be automatically added"), + virDomainControllerModelPCITypeToString(existingContModel)); + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Cannot automatically add a new PCI bus for a " + "device with connect flags %.2x"), flags); + } return -1; } + i = addrs->nbuses; + if (VIR_EXPAND_N(addrs->buses, addrs->nbuses, add) < 0) return -1; - for (; i < addrs->nbuses; i++) { - /* Any time we auto-add a bus, we will want a multi-slot - * bus. Currently the only type of bus we will auto-add is a - * pci-bridge, which is hot-pluggable and provides standard - * PCI slots. + if (needDMIToPCIBridge) { + /* first of the new buses is dmi-to-pci-bridge, the + * rest are of the requested type */ - virDomainPCIAddressBusSetModel(&addrs->buses[i], - VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE); + virDomainPCIAddressBusSetModel(&addrs->buses[i++], + VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE); } + + for (; i < addrs->nbuses; i++) + virDomainPCIAddressBusSetModel(&addrs->buses[i], model); + return add; } diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 81c7bce..fee93fd 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -910,27 +910,14 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, { virDomainPCIAddressSetPtr addrs; size_t i; + bool hasPCIeRoot = false; + virDomainControllerModelPCI defaultModel; if ((addrs = virDomainPCIAddressSetAlloc(nbuses)) == NULL) return NULL; addrs->dryRun = dryRun; - /* As a safety measure, set default model='pci-root' for first pci - * controller and 'pci-bridge' for all subsequent. After setting - * those defaults, then scan the config and set the actual model - * for all addrs[idx]->bus that already have a corresponding - * controller in the config. - * - */ - if (nbuses > 0) - virDomainPCIAddressBusSetModel(&addrs->buses[0], - VIR_DOMAIN_CONTROLLER_MODEL_PCI_ROOT); - for (i = 1; i < nbuses; i++) { - virDomainPCIAddressBusSetModel(&addrs->buses[i], - VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE); - } - for (i = 0; i < def->ncontrollers; i++) { virDomainControllerDefPtr cont = def->controllers[i]; size_t idx = cont->idx; @@ -947,7 +934,43 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, if (virDomainPCIAddressBusSetModel(&addrs->buses[idx], cont->model) < 0) goto error; - } + + if (cont->model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) + hasPCIeRoot = true; + } + + if (nbuses > 0 && !addrs->buses[0].model) { + /* This is just here to replicate a safety measure already in + * an older version of this code. In practice, the root bus + * should have already been added at index 0 prior to + * assigning addresses to devices. + */ + if (virDomainPCIAddressBusSetModel(&addrs->buses[0], + VIR_DOMAIN_CONTROLLER_MODEL_PCI_ROOT) < 0) + goto error; + } + + /* Now fill in a reasonable model for all the buses in the set + * that don't yet have a corresponding controller in the domain + * config. + */ + + if (hasPCIeRoot) + defaultModel = VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; + else + defaultModel = VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE; + + for (i = 1; i < addrs->nbuses; i++) { + + if (addrs->buses[i].model) + continue; + + if (virDomainPCIAddressBusSetModel(&addrs->buses[i], defaultModel) < 0) + goto error; + + VIR_DEBUG("Auto-adding <controller type='pci' model='%s' index='%zu'/>", + virDomainControllerModelPCITypeToString(defaultModel), i); + } if (virDomainDeviceInfoIterate(def, qemuDomainCollectPCIAddress, addrs) < 0) goto error; diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args new file mode 100644 index 0000000..70c759e --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args @@ -0,0 +1,57 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/libexec/qemu-kvm \ +-name q35-test \ +-S \ +-M q35 \ +-m 2048 \ +-smp 2,sockets=2,cores=1,threads=1 \ +-uuid 11dbdcdd-4c3b-482b-8903-9bdb8c0a2774 \ +-nographic \ +-nodefaults \ +-monitor unix:/tmp/lib/domain--1-q35-test/monitor.sock,server,nowait \ +-no-acpi \ +-boot c \ +-device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ +-device ioh3420,port=0x10,chassis=2,id=pci.2,bus=pcie.0,addr=0x2 \ +-device ioh3420,port=0x18,chassis=3,id=pci.3,bus=pcie.0,addr=0x3 \ +-device ioh3420,port=0x20,chassis=4,id=pci.4,bus=pcie.0,addr=0x4 \ +-device ioh3420,port=0x28,chassis=5,id=pci.5,bus=pcie.0,addr=0x5 \ +-device ioh3420,port=0x30,chassis=6,id=pci.6,bus=pcie.0,addr=0x6 \ +-device ioh3420,port=0x38,chassis=7,id=pci.7,bus=pcie.0,addr=0x7 \ +-device ioh3420,port=0x40,chassis=8,id=pci.8,bus=pcie.0,addr=0x8 \ +-device ioh3420,port=0x48,chassis=9,id=pci.9,bus=pcie.0,addr=0x9 \ +-device ioh3420,port=0x50,chassis=10,id=pci.10,bus=pcie.0,addr=0xa \ +-device ioh3420,port=0x58,chassis=11,id=pci.11,bus=pcie.0,addr=0xb \ +-device ioh3420,port=0x60,chassis=12,id=pci.12,bus=pcie.0,addr=0xc \ +-device ioh3420,port=0x68,chassis=13,id=pci.13,bus=pcie.0,addr=0xd \ +-device ioh3420,port=0x70,chassis=14,id=pci.14,bus=pcie.0,addr=0xe \ +-device nec-usb-xhci,id=usb,bus=pci.7,addr=0x0 \ +-device virtio-scsi-pci,id=scsi0,bus=pci.6,addr=0x0 \ +-device virtio-serial-pci,id=virtio-serial0,bus=pci.5,addr=0x0 \ +-drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-virtio-disk1 \ +-device virtio-blk-pci,bus=pci.8,addr=0x0,drive=drive-virtio-disk1,\ +id=virtio-disk1 \ +-fsdev local,security_model=passthrough,id=fsdev-fs0,path=/export/to/guest \ +-device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=/import/from/host,\ +bus=pci.2,addr=0x0 \ +-netdev user,id=hostnet0 \ +-device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.3,\ +addr=0x0 \ +-netdev user,id=hostnet1 \ +-device e1000e,netdev=hostnet1,id=net1,mac=00:11:22:33:44:66,bus=pci.4,\ +addr=0x0 \ +-device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.11,\ +addr=0x0 \ +-device virtio-mouse-pci,id=input1,bus=pci.12,addr=0x0 \ +-device virtio-keyboard-pci,id=input2,bus=pci.13,addr=0x0 \ +-device virtio-tablet-pci,id=input3,bus=pci.14,addr=0x0 \ +-device virtio-gpu-pci,id=video0,bus=pcie.0,addr=0x1 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.9,addr=0x0 \ +-object rng-random,id=objrng0,filename=/dev/urandom \ +-device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=123,period=1234,\ +bus=pci.10,addr=0x0 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.xml b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.xml new file mode 100644 index 0000000..06cdf61 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.xml @@ -0,0 +1,51 @@ +<domain type='qemu'> + <name>q35-test</name> + <uuid>11dbdcdd-4c3b-482b-8903-9bdb8c0a2774</uuid> + <memory unit='KiB'>2097152</memory> + <currentMemory unit='KiB'>2097152</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='x86_64' machine='q35'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <controller type='virtio-serial'/> + <controller type='scsi' model='virtio-scsi'/> + <controller type='usb' model='nec-xhci'/> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='vdb' bus='virtio'/> + </disk> + <filesystem type='mount'> + <source dir='/export/to/guest'/> + <target dir='/import/from/host'/> + </filesystem> + <video> + <model type='virtio'/> + </video> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + </interface> + <interface type='user'> + <mac address='00:11:22:33:44:66'/> + <model type='e1000e'/> + </interface> + <memballoon model='virtio'/> + <rng model='virtio'> + <rate bytes='123' period='1234'/> + <backend model='random'>/dev/urandom</backend> + </rng> + <input type='passthrough' bus='virtio'> + <source evdev='/dev/input/event1234'/> + </input> + <input type='mouse' bus='virtio'/> + <input type='keyboard' bus='virtio'/> + <input type='tablet' bus='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index bbf5ff5..98102fe 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1819,6 +1819,30 @@ mymain(void) QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY); + /* same as q35-pcie, but all PCI controllers are added automatically */ + DO_TEST("q35-pcie-autoadd", + QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, + QEMU_CAPS_DEVICE_VIRTIO_RNG, + QEMU_CAPS_OBJECT_RNG_RANDOM, + QEMU_CAPS_NETDEV, + QEMU_CAPS_DEVICE_VIRTIO_NET, + QEMU_CAPS_DEVICE_VIRTIO_GPU, + QEMU_CAPS_VIRTIO_GPU_VIRGL, + QEMU_CAPS_VIRTIO_KEYBOARD, + QEMU_CAPS_VIRTIO_MOUSE, + QEMU_CAPS_VIRTIO_TABLET, + QEMU_CAPS_VIRTIO_INPUT_HOST, + QEMU_CAPS_VIRTIO_SCSI, + QEMU_CAPS_FSDEV, + QEMU_CAPS_FSDEV_WRITEOUT, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_NEC_USB_XHCI, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY); DO_TEST("pcie-root-port", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml new file mode 100644 index 0000000..b27dbe7 --- /dev/null +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml @@ -0,0 +1,147 @@ +<domain type='qemu'> + <name>q35-test</name> + <uuid>11dbdcdd-4c3b-482b-8903-9bdb8c0a2774</uuid> + <memory unit='KiB'>2097152</memory> + <currentMemory unit='KiB'>2097152</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='x86_64' machine='q35'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='vdb' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> + </disk> + <controller type='virtio-serial' index='0'> + <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> + </controller> + <controller type='scsi' index='0' model='virtio-scsi'> + <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + </controller> + <controller type='usb' index='0' model='nec-xhci'> + <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> + </controller> + <controller type='sata' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> + </controller> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='dmi-to-pci-bridge'> + <model name='i82801b11-bridge'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='2' port='0x10'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='3' port='0x18'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='4' port='0x20'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </controller> + <controller type='pci' index='5' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='5' port='0x28'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </controller> + <controller type='pci' index='6' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='6' port='0x30'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> + </controller> + <controller type='pci' index='7' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='7' port='0x38'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> + </controller> + <controller type='pci' index='8' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='8' port='0x40'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> + </controller> + <controller type='pci' index='9' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='9' port='0x48'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> + </controller> + <controller type='pci' index='10' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='10' port='0x50'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> + </controller> + <controller type='pci' index='11' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='11' port='0x58'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> + </controller> + <controller type='pci' index='12' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='12' port='0x60'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/> + </controller> + <controller type='pci' index='13' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='13' port='0x68'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/> + </controller> + <controller type='pci' index='14' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='14' port='0x70'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0e' function='0x0'/> + </controller> + <filesystem type='mount' accessmode='passthrough'> + <source dir='/export/to/guest'/> + <target dir='/import/from/host'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </filesystem> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </interface> + <interface type='user'> + <mac address='00:11:22:33:44:66'/> + <model type='e1000e'/> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </interface> + <input type='passthrough' bus='virtio'> + <source evdev='/dev/input/event1234'/> + <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> + </input> + <input type='mouse' bus='virtio'> + <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> + </input> + <input type='keyboard' bus='virtio'> + <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> + </input> + <input type='tablet' bus='virtio'> + <address type='pci' domain='0x0000' bus='0x0e' slot='0x00' function='0x0'/> + </input> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <video> + <model type='virtio' heads='1' primary='yes'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </video> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> + </memballoon> + <rng model='virtio'> + <rate bytes='123' period='1234'/> + <backend model='random'>/dev/urandom</backend> + <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> + </rng> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 91d6059..f8b8809 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -741,7 +741,30 @@ mymain(void) QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY); - + /* same as q35-pcie, but all PCI controllers are added automatically */ + DO_TEST("q35-pcie-autoadd", + QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, + QEMU_CAPS_DEVICE_VIRTIO_RNG, + QEMU_CAPS_OBJECT_RNG_RANDOM, + QEMU_CAPS_NETDEV, + QEMU_CAPS_DEVICE_VIRTIO_NET, + QEMU_CAPS_DEVICE_VIRTIO_GPU, + QEMU_CAPS_VIRTIO_GPU_VIRGL, + QEMU_CAPS_VIRTIO_KEYBOARD, + QEMU_CAPS_VIRTIO_MOUSE, + QEMU_CAPS_VIRTIO_TABLET, + QEMU_CAPS_VIRTIO_INPUT_HOST, + QEMU_CAPS_VIRTIO_SCSI, + QEMU_CAPS_FSDEV, + QEMU_CAPS_FSDEV_WRITEOUT, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_NEC_USB_XHCI, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY); DO_TEST("pcie-root", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
Previously libvirt would only add pci-bridge devices automatically when an address was requested for a device that required a legacy PCI slot and none was available. This patch expands that support to dmi-to-pci-bridge (which is needed in order to add a pci-bridge on a machine with a pcie-root), and pcie-root-port (which is needed to add a hotpluggable PCIe device). It does *not* automatically add pcie-switch-upstream-ports or pcie-switch-downstream-ports (and currently there are no plans for that).
[...]
Although libvirt will now automatically create a dmi-to-pci-bridge when it's needed, the code still remains for now that forces a dmi-to-pci-bridge on all domains with pcie-root (in qemuDomainDefAddDefaultDevices()). That will be removed in the next patch.
s/in the next/in a future/
For now, the pcie-root-ports are added one to a slot, which is a bit wasteful and means it will fail after 31 total PCIe devices (30 if there are also some PCI devices), but helps keep the changeset down for this patch. A future patch will have 8 pcie-root-ports sharing the functions on a single slot.
[...]
@@ -82,6 +82,30 @@ virDomainPCIControllerModelToConnectType(virDomainControllerModelPCI model) return 0; } + +static int +virDomainPCIControllerConnectTypeToModel(virDomainPCIConnectFlags flags) +{ + if (flags & VIR_PCI_CONNECT_TYPE_PCIE_ROOT_PORT) + return VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT;
Maybe adding an empty line between each branch would make this a little more readable? Just a thought. Also too bad these are flags and we can't use a switch() statement here - the compiler will not warn us if we forget to handle a VIR_PCI_CONNECT_TYPE in the future :( [...]
@@ -351,32 +372,73 @@ virDomainPCIAddressSetGrow(virDomainPCIAddressSetPtr addrs, { int add; size_t i; + int model; + bool needDMIToPCIBridge = false; add = addr->bus - addrs->nbuses + 1; - i = addrs->nbuses; if (add <= 0) return 0; - /* auto-grow only works when we're adding plain PCI devices */ - if (!(flags & VIR_PCI_CONNECT_TYPE_PCI_DEVICE)) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Cannot automatically add a new PCI bus for a " - "device requiring a slot other than standard PCI.")); + if (flags & VIR_PCI_CONNECT_TYPE_PCI_DEVICE) { + model = VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE; + + /* if there aren't yet any buses that will accept a + * pci-bridge, and the caller is asking for one, we'll need to + * add a dmi-to-pci-bridge first. + */ + needDMIToPCIBridge = true; + for (i = 0; i < addrs->nbuses; i++) { + if (addrs->buses[i].flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE) { + needDMIToPCIBridge = false; + break; + } + } + if (needDMIToPCIBridge && add == 1) { + /* we need to add at least two buses - one dmi-to-pci, + * and the other the requested pci-bridge + */ + add++; + addr->bus++; + }
This last part got me confused for a bit, so I wouldn't mind having a more detailed comment here. Something like We need to add a single pci-bridge to provide the bus our legacy PCI device will be plugged into; however, we have also determined that the pci-bridge we're about to add itself needs to be plugged into a dmi-to-pci-bridge. To accomodate both requirements, we're going to add both a dmi-to-pci-bridge and a pci-bridge, and we're going to bump the bus number of the device by one to account for the extra controller. Yeah, that's quite a mouthful :/
+ } else if (flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE && + addrs->buses[0].model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) { + model = VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE;
Mh, what about the case where we want to add a pci-bridge but the machine has a pci-root instead of a pcie-root? Shouldn't that case be handled as well?
+ } else if (flags & (VIR_PCI_CONNECT_TYPE_PCIE_DEVICE | + VIR_PCI_CONNECT_TYPE_PCIE_SWITCH_UPSTREAM_PORT)) { + model = VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; + } else { + int existingContModel = virDomainPCIControllerConnectTypeToModel(flags); + + if (existingContModel >= 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("a PCI slot is needed to connect a PCI controller " + "model='%s', but none is available, and it " + "cannot be automatically added"), + virDomainControllerModelPCITypeToString(existingContModel)); + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Cannot automatically add a new PCI bus for a " + "device with connect flags %.2x"), flags); + }
So we error out for dmi-to-pci-bridge and pci{,e}-expander-bus... Shouldn't we be able to plug either into into pcie-root or pci-root?
return -1; } + i = addrs->nbuses; + if (VIR_EXPAND_N(addrs->buses, addrs->nbuses, add) < 0) return -1; - for (; i < addrs->nbuses; i++) { - /* Any time we auto-add a bus, we will want a multi-slot - * bus. Currently the only type of bus we will auto-add is a - * pci-bridge, which is hot-pluggable and provides standard - * PCI slots. + if (needDMIToPCIBridge) { + /* first of the new buses is dmi-to-pci-bridge, the + * rest are of the requested type */ - virDomainPCIAddressBusSetModel(&addrs->buses[i], - VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE); + virDomainPCIAddressBusSetModel(&addrs->buses[i++], + VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE);
virDomainPCIAddressBusSetModel() can fail, yet you're not checking the return value neither here...
} + + for (; i < addrs->nbuses; i++) + virDomainPCIAddressBusSetModel(&addrs->buses[i], model);
... nor here. [...]
@@ -947,7 +934,43 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, if (virDomainPCIAddressBusSetModel(&addrs->buses[idx], cont->model) < 0) goto error; - } + + if (cont->model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) + hasPCIeRoot = true; + } + + if (nbuses > 0 && !addrs->buses[0].model) { + /* This is just here to replicate a safety measure already in + * an older version of this code. In practice, the root bus + * should have already been added at index 0 prior to + * assigning addresses to devices. + */ + if (virDomainPCIAddressBusSetModel(&addrs->buses[0], + VIR_DOMAIN_CONTROLLER_MODEL_PCI_ROOT) < 0) + goto error; + } + + /* Now fill in a reasonable model for all the buses in the set + * that don't yet have a corresponding controller in the domain + * config. + */ +
No empty line here. Rest looks good, but no ACK for you quite yet :) -- Andrea Bolognani / Red Hat / Virtualization
On 11/09/2016 09:19 AM, Andrea Bolognani wrote:
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
Previously libvirt would only add pci-bridge devices automatically when an address was requested for a device that required a legacy PCI slot and none was available. This patch expands that support to dmi-to-pci-bridge (which is needed in order to add a pci-bridge on a machine with a pcie-root), and pcie-root-port (which is needed to add a hotpluggable PCIe device). It does *not* automatically add pcie-switch-upstream-ports or pcie-switch-downstream-ports (and currently there are no plans for that). [...] Although libvirt will now automatically create a dmi-to-pci-bridge when it's needed, the code still remains for now that forces a dmi-to-pci-bridge on all domains with pcie-root (in qemuDomainDefAddDefaultDevices()). That will be removed in the next patch. s/in the next/in a future/
For now, the pcie-root-ports are added one to a slot, which is a bit wasteful and means it will fail after 31 total PCIe devices (30 if there are also some PCI devices), but helps keep the changeset down for this patch. A future patch will have 8 pcie-root-ports sharing the functions on a single slot. [...] @@ -82,6 +82,30 @@ virDomainPCIControllerModelToConnectType(virDomainControllerModelPCI model) return 0; }
+ +static int +virDomainPCIControllerConnectTypeToModel(virDomainPCIConnectFlags flags) +{ + if (flags & VIR_PCI_CONNECT_TYPE_PCIE_ROOT_PORT) + return VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; Maybe adding an empty line between each branch would make this a little more readable? Just a thought.
Okay.
Also too bad these are flags and we can't use a switch() statement here - the compiler will not warn us if we forget to handle a VIR_PCI_CONNECT_TYPE in the future :(
Yeah, I may rework the flags someday to separate out the hotplug and aggregate_slot flags so that the connect type can be switched on.
[...]
@@ -351,32 +372,73 @@ virDomainPCIAddressSetGrow(virDomainPCIAddressSetPtr addrs, { int add; size_t i; + int model; + bool needDMIToPCIBridge = false;
add = addr->bus - addrs->nbuses + 1; - i = addrs->nbuses; if (add <= 0) return 0;
- /* auto-grow only works when we're adding plain PCI devices */ - if (!(flags & VIR_PCI_CONNECT_TYPE_PCI_DEVICE)) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Cannot automatically add a new PCI bus for a " - "device requiring a slot other than standard PCI.")); + if (flags & VIR_PCI_CONNECT_TYPE_PCI_DEVICE) { + model = VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE; + + /* if there aren't yet any buses that will accept a + * pci-bridge, and the caller is asking for one, we'll need to + * add a dmi-to-pci-bridge first. + */ + needDMIToPCIBridge = true; + for (i = 0; i < addrs->nbuses; i++) { + if (addrs->buses[i].flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE) { + needDMIToPCIBridge = false; + break; + } + } + if (needDMIToPCIBridge && add == 1) { + /* we need to add at least two buses - one dmi-to-pci, + * and the other the requested pci-bridge + */ + add++; + addr->bus++; + } This last part got me confused for a bit, so I wouldn't mind having a more detailed comment here. Something like
We need to add a single pci-bridge to provide the bus our legacy PCI device will be plugged into; however, we have also determined that the pci-bridge we're about to add itself needs to be plugged into a dmi-to-pci-bridge. To accomodate both requirements, we're going to add both a dmi-to-pci-bridge and a pci-bridge, and we're going to bump the bus number of the device by one to account for the extra controller.
Yeah, that's quite a mouthful :/
Okay, I added a variation of that description.
+ } else if (flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE && + addrs->buses[0].model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) { + model = VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE; Mh, what about the case where we want to add a pci-bridge but the machine has a pci-root instead of a pcie-root? Shouldn't that case be handled as well?
It is handled - we'll want to add a pci-bridge if flags & VIR_PCI_TYPE_PCI_DEVICE is true (i.e. the first clause). In that case, since pci-root has the flag VIR_PCI_CONNECT_TYPE_PCI_BRIDGE set, the for loop will set neetDMIToPCIBridge = false, so we will end up adding just the one pci-bridge that we need.
+ } else if (flags & (VIR_PCI_CONNECT_TYPE_PCIE_DEVICE | + VIR_PCI_CONNECT_TYPE_PCIE_SWITCH_UPSTREAM_PORT)) { + model = VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; + } else { + int existingContModel = virDomainPCIControllerConnectTypeToModel(flags); + + if (existingContModel >= 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("a PCI slot is needed to connect a PCI controller " + "model='%s', but none is available, and it " + "cannot be automatically added"), + virDomainControllerModelPCITypeToString(existingContModel)); + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Cannot automatically add a new PCI bus for a " + "device with connect flags %.2x"), flags); + } So we error out for dmi-to-pci-bridge and pci{,e}-expander-bus... Shouldn't we be able to plug either into into pcie-root or pci-root?
Exactly. And since those buses already exist from the start, and can't be added later, there wouldn't ever be a situation where we needed to automatically grow the bus structure due to one of those devices and growing could actually do anything (i.e. you can't add a pcie-root or pci-root).
return -1; }
+ i = addrs->nbuses; + if (VIR_EXPAND_N(addrs->buses, addrs->nbuses, add) < 0) return -1;
- for (; i < addrs->nbuses; i++) { - /* Any time we auto-add a bus, we will want a multi-slot - * bus. Currently the only type of bus we will auto-add is a - * pci-bridge, which is hot-pluggable and provides standard - * PCI slots. + if (needDMIToPCIBridge) { + /* first of the new buses is dmi-to-pci-bridge, the + * rest are of the requested type */ - virDomainPCIAddressBusSetModel(&addrs->buses[i], - VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE); + virDomainPCIAddressBusSetModel(&addrs->buses[i++], + VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE);
virDomainPCIAddressBusSetModel() can fail, yet you're not checking the return value neither here...
Yes, but the only way it can fail is if an unknown controller model is sent to it, and we can see by simple physical inspection that that isn't the case. If this was calling off to a function in some other file where we didn't know just how simple it is and couldn't easily see that it would never fail, then I would agree that we should check, but in this case it's superfluous - if this function had an ATTRIBUTE_RETURN_CHECK on it, I would have put these calls inside ignore_value().
} + + for (; i < addrs->nbuses; i++) + virDomainPCIAddressBusSetModel(&addrs->buses[i], model);
... nor here.
Same for this - the controller model is one of just a couple hardcoded values set within the same function, not sent in from somewhere else, and the function we're calling is a very simple function in the same file.
[...]
@@ -947,7 +934,43 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def,
if (virDomainPCIAddressBusSetModel(&addrs->buses[idx], cont->model) < 0) goto error; - }
Since I'm sure you'll bring up the fact that I *am* checking the return value of virDomainPCIAddressBusSetModel here (and everywhere else in qemu_domain_address.c), I'll point out that these calls are made from another file in another directory, so I'm being overly paranoid, not on any real factual basis, but just because it feels right :-)
+ + if (cont->model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) + hasPCIeRoot = true; + } + + if (nbuses > 0 && !addrs->buses[0].model) { + /* This is just here to replicate a safety measure already in + * an older version of this code. In practice, the root bus + * should have already been added at index 0 prior to + * assigning addresses to devices. + */ + if (virDomainPCIAddressBusSetModel(&addrs->buses[0], + VIR_DOMAIN_CONTROLLER_MODEL_PCI_ROOT) < 0) + goto error; + } + + /* Now fill in a reasonable model for all the buses in the set + * that don't yet have a corresponding controller in the domain + * config. + */ + No empty line here.
Rest looks good, but no ACK for you quite yet :)
So far I've changed comments and whitespace. You're welcome to dispute my opinion about checking return value from the SetModel function, but otherwise there's only cosmetic differences.
-- Andrea Bolognani / Red Hat / Virtualization
On Wed, 2016-11-09 at 15:23 -0500, Laine Stump wrote:
+ } else if (flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE && + addrs->buses[0].model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) { + model = VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE; Mh, what about the case where we want to add a pci-bridge but the machine has a pci-root instead of a pcie-root? Shouldn't that case be handled as well? It is handled - we'll want to add a pci-bridge if flags & VIR_PCI_TYPE_PCI_DEVICE is true (i.e. the first clause). In that case, since pci-root has the flag VIR_PCI_CONNECT_TYPE_PCI_BRIDGE set, the for loop will set neetDMIToPCIBridge = false, so we will end up adding just the one pci-bridge that we need.
Sorry, I was not clear enough. What if the function is called like virDomainPCIAddressSetGrow(addrs, addr, VIR_PCI_CONNECT_TYPE_PCI_BRIDGE) because *the caller* is going through a virDomainDef and, after finding a pci-bridge in it, wants to make sure the address set can actually fit it? The case when flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE can clearly happen, as we're handling it above, but we error out unless the guest has a pcie-root? That's the bit that I can't quite wrap my head around.
+ } else if (flags & (VIR_PCI_CONNECT_TYPE_PCIE_DEVICE | + VIR_PCI_CONNECT_TYPE_PCIE_SWITCH_UPSTREAM_PORT)) { + model = VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; + } else { + int existingContModel = virDomainPCIControllerConnectTypeToModel(flags); + + if (existingContModel >= 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("a PCI slot is needed to connect a PCI controller " + "model='%s', but none is available, and it " + "cannot be automatically added"), + virDomainControllerModelPCITypeToString(existingContModel)); + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Cannot automatically add a new PCI bus for a " + "device with connect flags %.2x"), flags); + } So we error out for dmi-to-pci-bridge and pci{,e}-expander-bus... Shouldn't we be able to plug either into into pcie-root or pci-root? Exactly. And since those buses already exist from the start, and can't be added later, there wouldn't ever be a situation where we needed to automatically grow the bus structure due to one of those devices and growing could actually do anything (i.e. you can't add a pcie-root or pci-root).
I'm clearly missing something here :( Don't we (potentially) grow the address set using this function every time we reserve an address for an endpoint device or a controller? Including when we're going through a virDomainDef, which might contain a dmi-to-pci-bridge or a pci{,e}-expander-bus? I'm not expecting the pci{,e}-root case to be handled, of course, but I can't figure out why we seem to be skipping over a bunch of perfectly valid cases.
- virDomainPCIAddressBusSetModel(&addrs->buses[i], - VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE); + virDomainPCIAddressBusSetModel(&addrs->buses[i++], + VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE); virDomainPCIAddressBusSetModel() can fail, yet you're not checking the return value neither here... Yes, but the only way it can fail is if an unknown controller model is sent to it, and we can see by simple physical inspection that that isn't the case. If this was calling off to a function in some other file where we didn't know just how simple it is and couldn't easily see that it would never fail, then I would agree that we should check, but in this case it's superfluous - if this function had an ATTRIBUTE_RETURN_CHECK on it, I would have put these calls inside ignore_value().
[...]
} + + for (; i < addrs->nbuses; i++) + virDomainPCIAddressBusSetModel(&addrs->buses[i], model); ... nor here. Same for this - the controller model is one of just a couple hardcoded values set within the same function, not sent in from somewhere else, and the function we're calling is a very simple function in the same file.
[...]
@@ -947,7 +934,43 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, if (virDomainPCIAddressBusSetModel(&addrs->buses[idx], cont->model) < 0) goto error; - } Since I'm sure you'll bring up the fact that I *am* checking the return value of virDomainPCIAddressBusSetModel here (and everywhere else in qemu_domain_address.c), I'll point out that these calls are made from another file in another directory, so I'm being overly paranoid, not on any real factual basis, but just because it feels right :-)
I get where you're coming from, but I don't think I agree with your conclusions. Sure, virDomainPCIAddressBusSetModel() might be a trivial function *now*, but we can't guarantee the same will be true in the future. Sure, the model we pass to it from inside virDomainPCIAddressSetGrow() is one of very few known-good values, but we might eg. read it from a less reliable source somewhere down the line. My point is, by not checking the return value we're implicitly embedding some knowledge about virDomainPCIAddressSetGrow() inside virDomainPCIAddressBusSetModel(), thus creating a strong relationship between the two functions. We should avoid creating such relationships, because they make the code much more difficult to reason about and might lead to unknowingly introducing subtle bugs.
Rest looks good, but no ACK for you quite yet :) So far I've changed comments and whitespace. You're welcome to dispute my opinion about checking return value from the SetModel function, but otherwise there's only cosmetic differences.
I would never hold my ACK hostage of a purely cosmetic issue :) However, as explained above, I'm still unable to wrap my head around some details, and until I do I'm not comfortable ACKing the patch. Hopefully you'll be able to help me clear out any remaining doubts :) -- Andrea Bolognani / Red Hat / Virtualization
On 11/10/2016 09:10 AM, Andrea Bolognani wrote:
On Wed, 2016-11-09 at 15:23 -0500, Laine Stump wrote:
+ } else if (flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE && + addrs->buses[0].model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) { + model = VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE;
Mh, what about the case where we want to add a pci-bridge but the machine has a pci-root instead of a pcie-root? Shouldn't that case be handled as well?
It is handled - we'll want to add a pci-bridge if flags & VIR_PCI_TYPE_PCI_DEVICE is true (i.e. the first clause). In that case, since pci-root has the flag VIR_PCI_CONNECT_TYPE_PCI_BRIDGE set, the for loop will set neetDMIToPCIBridge = false, so we will end up adding just the one pci-bridge that we need. Sorry, I was not clear enough.
What if the function is called like
virDomainPCIAddressSetGrow(addrs, addr, VIR_PCI_CONNECT_TYPE_PCI_BRIDGE)
because *the caller* is going through a virDomainDef and, after finding a pci-bridge in it, wants to make sure the address set can actually fit it? The case when
flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE
can clearly happen, as we're handling it above, but we error out unless the guest has a pcie-root?
Okay, two different scenarios, each for either pci-root or pcie-root: 1) pci-bridge requested with no address supplied a) pci-root - if there is an open slot it will be assigned to the pci-bridge. If there isn't an open slot, then there is no chance for success anyway, so the correct error message will be correct: a PCI slot is neede to connect a PCI coontller model='pci-bridge' but none is available, and it cannot be automatically added. (because the only way to add a new open slot is to add a pci-bridge, and that's what we're already trying and failing to do). b) pcie-root - if there is an open slot, then we would never get into ...Grow(), if there isn't then we need to add a dmi-to-pci-bridge (because we can only plug into a pci-bridge or a dmi-to-pci-bridge, and we've already established that we can't directly plug in a pci-bridge. So again, behavior is correct 2) pci-bridge was requested with user-supplied address that is on a nonexistent bus, and that bus number is >1 over the current last bus (i.e. there is a gap in the controller indexes). This would happen if, for example, someone put this in their xml: <controller type='pci' index='0' model='pci-root'/> <controller type='pci' index='2' model='pci-bridge'/> (if libvirt was setting the index, it would have set the index for the pci-bridge to '1'). a) pci-root - okay, here's where what you're saying would apply - we need to add extra pci-bridges to fill in the "empty space" in the list of controllers. b) pcie-root - actually we have the same issue in this case. Personally I think that it's nonsensical for a user to do that, and trying to support "empty regions" in the PCI address space by auto-adding pci-bridges is pointless busywork, but we have supported it in the past, so I guess we need to now. Sigh.
That's the bit that I can't quite wrap my head around.
Because it's a silly thing to support. But we already do, so we have to continue :-(
+ } else if (flags & (VIR_PCI_CONNECT_TYPE_PCIE_DEVICE | + VIR_PCI_CONNECT_TYPE_PCIE_SWITCH_UPSTREAM_PORT)) { + model = VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; + } else { + int existingContModel = virDomainPCIControllerConnectTypeToModel(flags); + + if (existingContModel >= 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("a PCI slot is needed to connect a PCI controller " + "model='%s', but none is available, and it " + "cannot be automatically added"), + virDomainControllerModelPCITypeToString(existingContModel)); + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Cannot automatically add a new PCI bus for a " + "device with connect flags %.2x"), flags); + }
So we error out for dmi-to-pci-bridge and pci{,e}-expander-bus... Shouldn't we be able to plug either into into pcie-root or pci-root?
Exactly. And since those buses already exist from the start, and can't be added later, there wouldn't ever be a situation where we needed to automatically grow the bus structure due to one of those devices and growing could actually do anything (i.e. you can't add a pcie-root or pci-root). I'm clearly missing something here :(
Don't we (potentially) grow the address set using this function every time we reserve an address for an endpoint device or a controller?
No. It is only called when we look for an open slot that matches the connectFlags of the device/controller, and don't find any open slot with compatible flags.
Including when we're going through a virDomainDef, which might contain a dmi-to-pci-bridge or a pci{,e}-expander-bus?
Sure, it can contain one of those. But the only place you can plug in a pci[e]-expander-bus is pci[e]-root, and you can't add a new pci[e]-root, so if you've looked for an empty slot and there isn't one, then you have an error. Since the ...Grow() function is only called if you've already failed to find an existing appropriate slot, it's correct for it to give an error. A dmi-to-pci-bridge is *slightly* different - it could also be connected to a pcie-expander-bus. But we don't automatically add those either, since they're really intended for supporting NUMA (and anyway, if there is no free slot in pcie-root to plug in a dmi-to-pci-bridge, then there is *also* no free slot to plug in a pcie-expander-bus). So again, if you've looked for an open slot to plug in a dmi-to-pci-bridge and couldn't find one, then when you call this function there is nothing productive it can do, so it's proper for it to fail and log an error.
I'm not expecting the pci{,e}-root case to be handled, of course, but I can't figure out why we seem to be skipping over a bunch of perfectly valid cases.
Because they're not valid cases.
- virDomainPCIAddressBusSetModel(&addrs->buses[i], - VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE); + virDomainPCIAddressBusSetModel(&addrs->buses[i++], + VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE);
virDomainPCIAddressBusSetModel() can fail, yet you're not checking the return value neither here...
Yes, but the only way it can fail is if an unknown controller model is sent to it, and we can see by simple physical inspection that that isn't the case. If this was calling off to a function in some other file where we didn't know just how simple it is and couldn't easily see that it would never fail, then I would agree that we should check, but in this case it's superfluous - if this function had an ATTRIBUTE_RETURN_CHECK on it, I would have put these calls inside ignore_value(). [...]
} + + for (; i < addrs->nbuses; i++) + virDomainPCIAddressBusSetModel(&addrs->buses[i], model);
... nor here.
Same for this - the controller model is one of just a couple hardcoded values set within the same function, not sent in from somewhere else, and the function we're calling is a very simple function in the same file. [...]
@@ -947,7 +934,43 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def,
if (virDomainPCIAddressBusSetModel(&addrs->buses[idx], cont->model) < 0) goto error; - }
Since I'm sure you'll bring up the fact that I *am* checking the return value of virDomainPCIAddressBusSetModel here (and everywhere else in qemu_domain_address.c), I'll point out that these calls are made from another file in another directory, so I'm being overly paranoid, not on any real factual basis, but just because it feels right :-) I get where you're coming from, but I don't think I agree with your conclusions.
Sure, virDomainPCIAddressBusSetModel() might be a trivial function *now*, but we can't guarantee the same will be true in the future. Sure, the model we pass to it from inside virDomainPCIAddressSetGrow() is one of very few known-good values, but we might eg. read it from a less reliable source somewhere down the line.
My point is, by not checking the return value we're implicitly embedding some knowledge about virDomainPCIAddressSetGrow() inside virDomainPCIAddressBusSetModel(), thus creating a strong relationship between the two functions. We should avoid creating such relationships, because they make the code much more difficult to reason about and might lead to unknowingly introducing subtle bugs.
I think that putting in error checking/recovery code for such simple cases is just adding more code to maintain and get wrong for no gain. (similar to checking the return from virBlahTypeToString() in cases where we know the enum value has already been validated). But I'll cede your point and add return checks on all the calls for virDomainPCIAddressBusSetModel().
Rest looks good, but no ACK for you quite yet :)
So far I've changed comments and whitespace. You're welcome to dispute my opinion about checking return value from the SetModel function, but otherwise there's only cosmetic differences. I would never hold my ACK hostage of a purely cosmetic issue :)
However, as explained above, I'm still unable to wrap my head around some details, and until I do I'm not comfortable ACKing the patch. Hopefully you'll be able to help me clear out any remaining doubts :)
I'll add the return value checks, and add support for calling ...Grow() with an address > 1 over the current bus count.
On 11/11/2016 11:09 AM, Laine Stump wrote:
On 11/10/2016 09:10 AM, Andrea Bolognani wrote:
On Wed, 2016-11-09 at 15:23 -0500, Laine Stump wrote:
+ } else if (flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE && + addrs->buses[0].model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) { + model = VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE; Mh, what about the case where we want to add a pci-bridge but the machine has a pci-root instead of a pcie-root? Shouldn't that case be handled as well? It is handled - we'll want to add a pci-bridge if flags & VIR_PCI_TYPE_PCI_DEVICE is true (i.e. the first clause). In that case, since pci-root has the flag VIR_PCI_CONNECT_TYPE_PCI_BRIDGE set, the for loop will set neetDMIToPCIBridge = false, so we will end up adding just the one pci-bridge that we need. Sorry, I was not clear enough.
What if the function is called like
virDomainPCIAddressSetGrow(addrs, addr, VIR_PCI_CONNECT_TYPE_PCI_BRIDGE)
because *the caller* is going through a virDomainDef and, after finding a pci-bridge in it, wants to make sure the address set can actually fit it? The case when
flags & VIR_PCI_CONNECT_TYPE_PCI_BRIDGE
can clearly happen, as we're handling it above, but we error out unless the guest has a pcie-root?
Okay, two different scenarios, each for either pci-root or pcie-root:
1) pci-bridge requested with no address supplied
(or where the address supplied has a bus# exactly 1 greater than the current last bus)
a) pci-root - if there is an open slot it will be assigned to the pci-bridge.
(i.e. ...Grow() would not have been called in the first place)
If there isn't an open slot, then there is no chance for success anyway, so the correct error message will be correct:
Sigh. "the *current* error message will be correct.
a PCI slot is neede to connect a PCI coontller model='pci-bridge' but none is available, and it cannot be automatically added.
Double Sigh. sed -e "s/neede/needed/" -e "s/coontller/controller/"
(because the only way to add a new open slot is to add a pci-bridge, and that's what we're already trying and failing to do).
b) pcie-root - if there is an open slot, then we would never get into ...Grow(), if there isn't then we need to add a dmi-to-pci-bridge (because we can only plug into a pci-bridge or a dmi-to-pci-bridge, and we've already established that we can't directly plug in a pci-bridge. So again, behavior is correct
2) pci-bridge was requested with user-supplied address that is on a nonexistent bus, and that bus number is >1 over the current last bus (i.e. there is a gap in the controller indexes). This would happen if, for example, someone put this in their xml:
<controller type='pci' index='0' model='pci-root'/> <controller type='pci' index='2' model='pci-bridge'/>
(if libvirt was setting the index, it would have set the index for the pci-bridge to '1').
a) pci-root - okay, here's where what you're saying would apply - we need to add extra pci-bridges to fill in the "empty space" in the list of controllers.
b) pcie-root - actually we have the same issue in this case.
Personally I think that it's nonsensical for a user to do that, and trying to support "empty regions" in the PCI address space by auto-adding pci-bridges is pointless busywork, but we have supported it in the past, so I guess we need to now. Sigh.
But wait! Look at the example above - that's *exactly* the odd case that patch 11/17 (the RFC patch that we agreed wasn't worthwhile) remedies. Even without that RFC patch, if you look at the code that creates a PCIAddressSet (qemuDomainPCIAddressSetCreate()), you'll see that during the initial creation, the unused bus numbers are filled in with pcie-root-port or pci-bridge controllers, i.e. the PCIAddressSet is never "sparse", and thus virDomainPCIAddressSetGrow() is guaranteed never to encounter the case above. So you're asking for a solution to a nonexistent problem.
That's the bit that I can't quite wrap my head around.
Because it's a silly thing to support. But we already do, so we have to continue :-(
I retract that - it's not a problem during Grow, only during initial creation, and we previously decided that it was a strange corner case that is unnecessary in practice and that we don't want to support.
On Sun, 2016-11-13 at 10:55 -0500, Laine Stump wrote:
2) pci-bridge was requested with user-supplied address that is on a nonexistent bus, and that bus number is >1 over the current last bus (i.e. there is a gap in the controller indexes). This would happen if, for example, someone put this in their xml: <controller type='pci' index='0' model='pci-root'/> <controller type='pci' index='2' model='pci-bridge'/> (if libvirt was setting the index, it would have set the index for the pci-bridge to '1'). a) pci-root - okay, here's where what you're saying would apply - we need to add extra pci-bridges to fill in the "empty space" in the list of controllers. b) pcie-root - actually we have the same issue in this case. Personally I think that it's nonsensical for a user to do that, and trying to support "empty regions" in the PCI address space by auto-adding pci-bridges is pointless busywork, but we have supported it in the past, so I guess we need to now. Sigh. But wait! Look at the example above - that's *exactly* the odd case that patch 11/17 (the RFC patch that we agreed wasn't worthwhile) remedies. Even without that RFC patch, if you look at the code that creates a PCIAddressSet (qemuDomainPCIAddressSetCreate()), you'll see that during the initial creation, the unused bus numbers are filled in with pcie-root-port or pci-bridge controllers, i.e. the PCIAddressSet is never "sparse", and thus virDomainPCIAddressSetGrow() is guaranteed never to encounter the case above. So you're asking for a solution to a nonexistent problem.
Yup, I had noticed that :) I'm still a bit unconvinced that function is a rock-solid as I'd like it to be, but seeing as I've made you go over it several times and it has resisted all my attempts to break it, the only reasonable thing I can do is to ACK your changes and stop being in the way of progress. -- Andrea Bolognani / Red Hat / Virtualization
I'm undecided if it is worthwhile to add this... Up until now it has been legal to have something like this in the xml: <controller type='pci' model='pcie-root' index='0'/> <controller type='pci' model='pci-bridge' index='2'/> (for example, see the existing test case "usb-controller-default-q35"). This is handled in qemuDomainPCIAddressSetCreate() when it's adding in controllers to fill holes in the indexes.) Since we have always added the following to every config: <controller type='pci' model='dmi-to-pci-bridge' index='1'/> there has always been a proper place for the unaddressed pci-bridge to plug in. A upcoming commit will eliminate the forced adding of a dmi-to-pci-bridge to *every* Q35 domain config, though. This would mean the above-mentioned test case (and one other) would begin to fail. There are two possible solutions to this problem: 1) change the test cases, and made the above construct an error (note that in the future it will work just fine to not list *any* pci controller, and they will all be auto-added as needed). or 2) This patch, which specifically looks for the case where we have a pci-bridge (but no dmi-to-pci-bridge) in a PCIe domain config and auto-adds the necessary dmi-to-pci-bridge. This can only work in the case that the pci-bridge has been given an index such that there is an unused index with a lower value for the dmi-to-pci-bridge to occupy. This seems like a kind of odd corner case that nobody should need to do in the future anyway. On the other hand, I already wrote the code and it works, so I might as well send it and see what opinions (if any) it generates. --- src/qemu/qemu_domain_address.c | 42 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 37 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index fee93fd..ea09556 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -911,6 +911,9 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, virDomainPCIAddressSetPtr addrs; size_t i; bool hasPCIeRoot = false; + unsigned int lowestDMIToPCIBridge = nbuses; + unsigned int lowestUnaddressedPCIBridge = nbuses; + unsigned int lowestAddressedPCIBridge = nbuses; virDomainControllerModelPCI defaultModel; if ((addrs = virDomainPCIAddressSetAlloc(nbuses)) == NULL) @@ -935,8 +938,24 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, if (virDomainPCIAddressBusSetModel(&addrs->buses[idx], cont->model) < 0) goto error; - if (cont->model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) + /* we'll use all this info later to determine if we need + * to add a dmi-to-pci-bridge due to unaddressed pci-bridge controllers + */ + if (cont->model == VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT) { hasPCIeRoot = true; + } else if (cont->model == + VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE) { + if (lowestDMIToPCIBridge > idx) + lowestDMIToPCIBridge = idx; + } else if (cont->model == VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE) { + if (virDeviceInfoPCIAddressWanted(&cont->info)) { + if (lowestUnaddressedPCIBridge > idx) + lowestUnaddressedPCIBridge = idx; + } else { + if (lowestAddressedPCIBridge > idx) + lowestAddressedPCIBridge = idx; + } + } } if (nbuses > 0 && !addrs->buses[0].model) { @@ -952,13 +971,21 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, /* Now fill in a reasonable model for all the buses in the set * that don't yet have a corresponding controller in the domain - * config. + * config. In the rare (and ... strange, but still allowed) case + * that a domain has 1) a pcie-root at index 0, 2) *no* + * dmi-to-pci-bridge (or pci-bridge that was manually addressed to + * sit directly on pcie-root), and 3) does have an unaddressed + * pci-bridge at an index > 1, then we need to add a + * dmi-to-pci-bridge. */ - if (hasPCIeRoot) - defaultModel = VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; - else + if (!hasPCIeRoot) defaultModel = VIR_DOMAIN_CONTROLLER_MODEL_PCI_BRIDGE; + else if (lowestUnaddressedPCIBridge < MIN(lowestAddressedPCIBridge, + lowestDMIToPCIBridge)) + defaultModel = VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE; + else + defaultModel = VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; for (i = 1; i < addrs->nbuses; i++) { @@ -970,6 +997,11 @@ qemuDomainPCIAddressSetCreate(virDomainDefPtr def, VIR_DEBUG("Auto-adding <controller type='pci' model='%s' index='%zu'/>", virDomainControllerModelPCITypeToString(defaultModel), i); + /* only add a single dmi-to-pci-bridge, then add pcie-root-port + * for any other unspecified controller indexes. + */ + if (hasPCIeRoot) + defaultModel = VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT; } if (virDomainDeviceInfoIterate(def, qemuDomainCollectPCIAddress, addrs) < 0) -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
I'm undecided if it is worthwhile to add this... Up until now it has been legal to have something like this in the xml: <controller type='pci' model='pcie-root' index='0'/> <controller type='pci' model='pci-bridge' index='2'/> (for example, see the existing test case "usb-controller-default-q35"). This is handled in qemuDomainPCIAddressSetCreate() when it's adding in controllers to fill holes in the indexes.) Since we have always added the following to every config: <controller type='pci' model='dmi-to-pci-bridge' index='1'/> there has always been a proper place for the unaddressed pci-bridge to plug in. A upcoming commit will eliminate the forced adding of a dmi-to-pci-bridge to *every* Q35 domain config, though. This would mean the above-mentioned test case (and one other) would begin to fail. There are two possible solutions to this problem: 1) change the test cases, and made the above construct an error (note that in the future it will work just fine to not list *any* pci controller, and they will all be auto-added as needed). or 2) This patch, which specifically looks for the case where we have a pci-bridge (but no dmi-to-pci-bridge) in a PCIe domain config and auto-adds the necessary dmi-to-pci-bridge. This can only work in the case that the pci-bridge has been given an index such that there is an unused index with a lower value for the dmi-to-pci-bridge to occupy. This seems like a kind of odd corner case that nobody should need to do in the future anyway. On the other hand, I already wrote the code and it works, so I might as well send it and see what opinions (if any) it generates.
I say let's forget about this. I'm fairly certain nobody is using this ludicrous pattern outside of our test suite, and even if they did their guests configurations would already have had the dmi-to-pci-bridge added long ago, so no risks of breakage there. So, unless somebody has very compelling arguments for maintaining such a "feature", consider this a NACK -- Andrea Bolognani / Red Hat / Virtualization
On 11/09/2016 11:26 AM, Andrea Bolognani wrote:
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
I'm undecided if it is worthwhile to add this...
Up until now it has been legal to have something like this in the xml:
<controller type='pci' model='pcie-root' index='0'/> <controller type='pci' model='pci-bridge' index='2'/>
(for example, see the existing test case "usb-controller-default-q35"). This is handled in qemuDomainPCIAddressSetCreate() when it's adding in controllers to fill holes in the indexes.)
Since we have always added the following to every config:
<controller type='pci' model='dmi-to-pci-bridge' index='1'/>
there has always been a proper place for the unaddressed pci-bridge to plug in.
A upcoming commit will eliminate the forced adding of a dmi-to-pci-bridge to *every* Q35 domain config, though. This would mean the above-mentioned test case (and one other) would begin to fail. There are two possible solutions to this problem:
1) change the test cases, and made the above construct an error (note that in the future it will work just fine to not list *any* pci controller, and they will all be auto-added as needed).
or
2) This patch, which specifically looks for the case where we have a pci-bridge (but no dmi-to-pci-bridge) in a PCIe domain config and auto-adds the necessary dmi-to-pci-bridge. This can only work in the case that the pci-bridge has been given an index such that there is an unused index with a lower value for the dmi-to-pci-bridge to occupy.
This seems like a kind of odd corner case that nobody should need to do in the future anyway. On the other hand, I already wrote the code and it works, so I might as well send it and see what opinions (if any) it generates. I say let's forget about this.
Forgotten. I may need to replace this patch with one that changes the test case. Alternately I could change the test case in the same patch that breaks it. Yeah, I'll probably do the latter.
I'm fairly certain nobody is using this ludicrous pattern outside of our test suite, and even if they did their guests configurations would already have had the dmi-to-pci-bridge added long ago, so no risks of breakage there.
So, unless somebody has very compelling arguments for maintaining such a "feature", consider this a
NACK
-- Andrea Bolognani / Red Hat / Virtualization
A few of the qemu test cases assume that a dmi-to-pci-bridge will always be added at index 1, and so they omit it from the input data *even though a pci-bridge is present at index 2, e.g.: <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='2' model='pci-bridge'/> Support for this odd practice was discussed on libvir-list and we decided that the complex code required to make this continue was not worth the headache of maintaining. So instead, this patch modifies the test cases to manually add a dmi-to-pci-bridge at index 1 (since an upcoming patch is going to eliminate the unconditional adding of dmi-to-pci-bridge). Because the auto-add was placing the dmi-to-pci-bridge later in the list (even though it has a lower index) the test output is also updated to take account for the new order (which puts the pci controllers in index-order) --- .../qemuxml2argvdata/qemuxml2argv-usb-controller-default-q35.args | 2 +- .../qemuxml2argvdata/qemuxml2argv-usb-controller-default-q35.xml | 1 + .../qemuxml2argv-usb-controller-explicit-q35.args | 2 +- .../qemuxml2argvdata/qemuxml2argv-usb-controller-explicit-q35.xml | 1 + .../qemuxml2xmlout-usb-controller-default-q35.xml | 8 ++++---- .../qemuxml2xmlout-usb-controller-explicit-q35.xml | 8 ++++---- 6 files changed, 12 insertions(+), 10 deletions(-) diff --git a/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-default-q35.args b/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-default-q35.args index f98f838..f45400d 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-default-q35.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-default-q35.args @@ -16,6 +16,6 @@ QEMU_AUDIO_DRV=none \ -monitor unix:/tmp/lib/domain--1-q35-test/monitor.sock,server,nowait \ -no-acpi \ -boot c \ --device pci-bridge,chassis_nr=56,id=pci.2,bus=pci.1,addr=0x0 \ -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ +-device pci-bridge,chassis_nr=56,id=pci.2,bus=pci.1,addr=0x0 \ -device piix3-usb-uhci,id=usb,bus=pcie.0,addr=0x1 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-default-q35.xml b/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-default-q35.xml index 26eecef..5f949d6 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-default-q35.xml +++ b/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-default-q35.xml @@ -15,6 +15,7 @@ <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='dmi-to-pci-bridge'/> <controller type='pci' index='2' model='pci-bridge'> <model name='pci-bridge'/> <target chassisNr='56'/> diff --git a/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-explicit-q35.args b/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-explicit-q35.args index 5eb39ba..92a4ee3 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-explicit-q35.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-explicit-q35.args @@ -16,6 +16,6 @@ QEMU_AUDIO_DRV=none \ -monitor unix:/tmp/lib/domain--1-q35-test/monitor.sock,server,nowait \ -no-acpi \ -boot c \ --device pci-bridge,chassis_nr=56,id=pci.2,bus=pci.1,addr=0x0 \ -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ +-device pci-bridge,chassis_nr=56,id=pci.2,bus=pci.1,addr=0x0 \ -device nec-usb-xhci,id=usb,bus=pcie.0,addr=0x1 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-explicit-q35.xml b/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-explicit-q35.xml index f3b34ee..f6d95b4 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-explicit-q35.xml +++ b/tests/qemuxml2argvdata/qemuxml2argv-usb-controller-explicit-q35.xml @@ -15,6 +15,7 @@ <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='dmi-to-pci-bridge'/> <controller type='pci' index='2' model='pci-bridge'> <model name='pci-bridge'/> <target chassisNr='56'/> diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-usb-controller-default-q35.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-usb-controller-default-q35.xml index 082a92e..456fd54 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-usb-controller-default-q35.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-usb-controller-default-q35.xml @@ -15,6 +15,10 @@ <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='dmi-to-pci-bridge'> + <model name='i82801b11-bridge'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> + </controller> <controller type='pci' index='2' model='pci-bridge'> <model name='pci-bridge'/> <target chassisNr='56'/> @@ -26,10 +30,6 @@ <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> - <controller type='pci' index='1' model='dmi-to-pci-bridge'> - <model name='i82801b11-bridge'/> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> - </controller> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <memballoon model='none'/> diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-usb-controller-explicit-q35.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-usb-controller-explicit-q35.xml index 540e817..db92c22 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-usb-controller-explicit-q35.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-usb-controller-explicit-q35.xml @@ -15,6 +15,10 @@ <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='dmi-to-pci-bridge'> + <model name='i82801b11-bridge'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> + </controller> <controller type='pci' index='2' model='pci-bridge'> <model name='pci-bridge'/> <target chassisNr='56'/> @@ -26,10 +30,6 @@ <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> - <controller type='pci' index='1' model='dmi-to-pci-bridge'> - <model name='i82801b11-bridge'/> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> - </controller> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <memballoon model='none'/> -- 2.7.4
On Mon, 2016-11-14 at 11:40 -0500, Laine Stump wrote:
A few of the qemu test cases assume that a dmi-to-pci-bridge will always be added at index 1, and so they omit it from the input data *even though a pci-bridge is present at index 2, e.g.:
s/^.*even/even/
<controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='2' model='pci-bridge'/>
Support for this odd practice was discussed on libvir-list and we decided that the complex code required to make this continue was not worth the headache of maintaining. So instead, this patch modifies the test cases to manually add a dmi-to-pci-bridge at index 1 (since an upcoming patch is going to eliminate the unconditional adding of dmi-to-pci-bridge).
Because the auto-add was placing the dmi-to-pci-bridge later in the list (even though it has a lower index) the test output is also updated to take account for the new order (which puts the pci controllers in index-order)
ACK -- Andrea Bolognani / Red Hat / Virtualization
Now the a dmi-to-pci-bridge is automatically added just as it's needed (when a pci-bridge is being added), we no longer have any need to force-add one to every single Q35 domain. --- src/qemu/qemu_domain.c | 12 ---- tests/qemuxml2argvdata/qemuxml2argv-pcie-root.args | 3 +- .../qemuxml2argv-q35-pcie-autoadd.args | 55 +++++++-------- .../qemuxml2xmlout-pcie-root.xml | 4 -- .../qemuxml2xmlout-q35-pcie-autoadd.xml | 82 ++++++++++------------ 5 files changed, 67 insertions(+), 89 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index bd2d167..3821d37 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -2298,18 +2298,6 @@ qemuDomainDefAddDefaultDevices(virDomainDefPtr def, VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT)) { goto cleanup; } - /* Add a dmi-to-pci-bridge bridge if there are no PCI controllers - * other than the pcie-root. This is so that there will be hot-pluggable - * PCI slots available. - * - * We skip this step for aarch64 mach-virt guests, where we want to - * be able to have a pure virtio-mmio topology - */ - if (virDomainControllerFind(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 1) < 0 && - !qemuDomainMachineIsVirt(def) && - !virDomainDefAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 1, - VIR_DOMAIN_CONTROLLER_MODEL_DMI_TO_PCI_BRIDGE)) - goto cleanup; } if (addDefaultMemballoon && !def->memballoon) { diff --git a/tests/qemuxml2argvdata/qemuxml2argv-pcie-root.args b/tests/qemuxml2argvdata/qemuxml2argv-pcie-root.args index 7ef03d3..59a849f 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-pcie-root.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-pcie-root.args @@ -15,5 +15,4 @@ QEMU_AUDIO_DRV=none \ -nodefaults \ -monitor unix:/tmp/lib/domain--1-q35-test/monitor.sock,server,nowait \ -no-acpi \ --boot c \ --device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e +-boot c diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args index 70c759e..daecf96 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args @@ -16,42 +16,41 @@ QEMU_AUDIO_DRV=none \ -monitor unix:/tmp/lib/domain--1-q35-test/monitor.sock,server,nowait \ -no-acpi \ -boot c \ --device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ --device ioh3420,port=0x10,chassis=2,id=pci.2,bus=pcie.0,addr=0x2 \ --device ioh3420,port=0x18,chassis=3,id=pci.3,bus=pcie.0,addr=0x3 \ --device ioh3420,port=0x20,chassis=4,id=pci.4,bus=pcie.0,addr=0x4 \ --device ioh3420,port=0x28,chassis=5,id=pci.5,bus=pcie.0,addr=0x5 \ --device ioh3420,port=0x30,chassis=6,id=pci.6,bus=pcie.0,addr=0x6 \ --device ioh3420,port=0x38,chassis=7,id=pci.7,bus=pcie.0,addr=0x7 \ --device ioh3420,port=0x40,chassis=8,id=pci.8,bus=pcie.0,addr=0x8 \ --device ioh3420,port=0x48,chassis=9,id=pci.9,bus=pcie.0,addr=0x9 \ --device ioh3420,port=0x50,chassis=10,id=pci.10,bus=pcie.0,addr=0xa \ --device ioh3420,port=0x58,chassis=11,id=pci.11,bus=pcie.0,addr=0xb \ --device ioh3420,port=0x60,chassis=12,id=pci.12,bus=pcie.0,addr=0xc \ --device ioh3420,port=0x68,chassis=13,id=pci.13,bus=pcie.0,addr=0xd \ --device ioh3420,port=0x70,chassis=14,id=pci.14,bus=pcie.0,addr=0xe \ --device nec-usb-xhci,id=usb,bus=pci.7,addr=0x0 \ --device virtio-scsi-pci,id=scsi0,bus=pci.6,addr=0x0 \ --device virtio-serial-pci,id=virtio-serial0,bus=pci.5,addr=0x0 \ +-device ioh3420,port=0x10,chassis=1,id=pci.1,bus=pcie.0,addr=0x2 \ +-device ioh3420,port=0x18,chassis=2,id=pci.2,bus=pcie.0,addr=0x3 \ +-device ioh3420,port=0x20,chassis=3,id=pci.3,bus=pcie.0,addr=0x4 \ +-device ioh3420,port=0x28,chassis=4,id=pci.4,bus=pcie.0,addr=0x5 \ +-device ioh3420,port=0x30,chassis=5,id=pci.5,bus=pcie.0,addr=0x6 \ +-device ioh3420,port=0x38,chassis=6,id=pci.6,bus=pcie.0,addr=0x7 \ +-device ioh3420,port=0x40,chassis=7,id=pci.7,bus=pcie.0,addr=0x8 \ +-device ioh3420,port=0x48,chassis=8,id=pci.8,bus=pcie.0,addr=0x9 \ +-device ioh3420,port=0x50,chassis=9,id=pci.9,bus=pcie.0,addr=0xa \ +-device ioh3420,port=0x58,chassis=10,id=pci.10,bus=pcie.0,addr=0xb \ +-device ioh3420,port=0x60,chassis=11,id=pci.11,bus=pcie.0,addr=0xc \ +-device ioh3420,port=0x68,chassis=12,id=pci.12,bus=pcie.0,addr=0xd \ +-device ioh3420,port=0x70,chassis=13,id=pci.13,bus=pcie.0,addr=0xe \ +-device nec-usb-xhci,id=usb,bus=pci.6,addr=0x0 \ +-device virtio-scsi-pci,id=scsi0,bus=pci.5,addr=0x0 \ +-device virtio-serial-pci,id=virtio-serial0,bus=pci.4,addr=0x0 \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-virtio-disk1 \ --device virtio-blk-pci,bus=pci.8,addr=0x0,drive=drive-virtio-disk1,\ +-device virtio-blk-pci,bus=pci.7,addr=0x0,drive=drive-virtio-disk1,\ id=virtio-disk1 \ -fsdev local,security_model=passthrough,id=fsdev-fs0,path=/export/to/guest \ -device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=/import/from/host,\ -bus=pci.2,addr=0x0 \ +bus=pci.1,addr=0x0 \ -netdev user,id=hostnet0 \ --device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.3,\ +-device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.2,\ addr=0x0 \ -netdev user,id=hostnet1 \ --device e1000e,netdev=hostnet1,id=net1,mac=00:11:22:33:44:66,bus=pci.4,\ +-device e1000e,netdev=hostnet1,id=net1,mac=00:11:22:33:44:66,bus=pci.3,\ addr=0x0 \ --device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.11,\ +-device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.10,\ addr=0x0 \ --device virtio-mouse-pci,id=input1,bus=pci.12,addr=0x0 \ --device virtio-keyboard-pci,id=input2,bus=pci.13,addr=0x0 \ --device virtio-tablet-pci,id=input3,bus=pci.14,addr=0x0 \ +-device virtio-mouse-pci,id=input1,bus=pci.11,addr=0x0 \ +-device virtio-keyboard-pci,id=input2,bus=pci.12,addr=0x0 \ +-device virtio-tablet-pci,id=input3,bus=pci.13,addr=0x0 \ -device virtio-gpu-pci,id=video0,bus=pcie.0,addr=0x1 \ --device virtio-balloon-pci,id=balloon0,bus=pci.9,addr=0x0 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.8,addr=0x0 \ -object rng-random,id=objrng0,filename=/dev/urandom \ --device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=123,period=1234,\ -bus=pci.10,addr=0x0 +-device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=123,period=1234,bus=pci.9,\ +addr=0x0 diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-pcie-root.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-pcie-root.xml index 80dc35e..b53ce24 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-pcie-root.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-pcie-root.xml @@ -18,10 +18,6 @@ <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> - <controller type='pci' index='1' model='dmi-to-pci-bridge'> - <model name='i82801b11-bridge'/> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> - </controller> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <memballoon model='none'/> diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml index b27dbe7..d784801 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml @@ -17,117 +17,113 @@ <disk type='block' device='disk'> <source dev='/dev/HostVG/QEMUGuest1'/> <target dev='vdb' bus='virtio'/> - <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </disk> <controller type='virtio-serial' index='0'> - <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </controller> <controller type='scsi' index='0' model='virtio-scsi'> - <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </controller> <controller type='usb' index='0' model='nec-xhci'> - <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> - <controller type='pci' index='1' model='dmi-to-pci-bridge'> - <model name='i82801b11-bridge'/> - <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> - </controller> - <controller type='pci' index='2' model='pcie-root-port'> + <controller type='pci' index='1' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='2' port='0x10'/> + <target chassis='1' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </controller> - <controller type='pci' index='3' model='pcie-root-port'> + <controller type='pci' index='2' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='3' port='0x18'/> + <target chassis='2' port='0x18'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> - <controller type='pci' index='4' model='pcie-root-port'> + <controller type='pci' index='3' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='4' port='0x20'/> + <target chassis='3' port='0x20'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> - <controller type='pci' index='5' model='pcie-root-port'> + <controller type='pci' index='4' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='5' port='0x28'/> + <target chassis='4' port='0x28'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </controller> - <controller type='pci' index='6' model='pcie-root-port'> + <controller type='pci' index='5' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='6' port='0x30'/> + <target chassis='5' port='0x30'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </controller> - <controller type='pci' index='7' model='pcie-root-port'> + <controller type='pci' index='6' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='7' port='0x38'/> + <target chassis='6' port='0x38'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> - <controller type='pci' index='8' model='pcie-root-port'> + <controller type='pci' index='7' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='8' port='0x40'/> + <target chassis='7' port='0x40'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </controller> - <controller type='pci' index='9' model='pcie-root-port'> + <controller type='pci' index='8' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='9' port='0x48'/> + <target chassis='8' port='0x48'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </controller> - <controller type='pci' index='10' model='pcie-root-port'> + <controller type='pci' index='9' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='10' port='0x50'/> + <target chassis='9' port='0x50'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> </controller> - <controller type='pci' index='11' model='pcie-root-port'> + <controller type='pci' index='10' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='11' port='0x58'/> + <target chassis='10' port='0x58'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> </controller> - <controller type='pci' index='12' model='pcie-root-port'> + <controller type='pci' index='11' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='12' port='0x60'/> + <target chassis='11' port='0x60'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/> </controller> - <controller type='pci' index='13' model='pcie-root-port'> + <controller type='pci' index='12' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='13' port='0x68'/> + <target chassis='12' port='0x68'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0d' function='0x0'/> </controller> - <controller type='pci' index='14' model='pcie-root-port'> + <controller type='pci' index='13' model='pcie-root-port'> <model name='ioh3420'/> - <target chassis='14' port='0x70'/> + <target chassis='13' port='0x70'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0e' function='0x0'/> </controller> <filesystem type='mount' accessmode='passthrough'> <source dir='/export/to/guest'/> <target dir='/import/from/host'/> - <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </filesystem> <interface type='user'> <mac address='00:11:22:33:44:55'/> <model type='virtio'/> - <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </interface> <interface type='user'> <mac address='00:11:22:33:44:66'/> <model type='e1000e'/> - <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </interface> <input type='passthrough' bus='virtio'> <source evdev='/dev/input/event1234'/> - <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> </input> <input type='mouse' bus='virtio'> - <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> </input> <input type='keyboard' bus='virtio'> - <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> </input> <input type='tablet' bus='virtio'> - <address type='pci' domain='0x0000' bus='0x0e' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> @@ -136,12 +132,12 @@ <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <memballoon model='virtio'> - <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </memballoon> <rng model='virtio'> <rate bytes='123' period='1234'/> <backend model='random'>/dev/urandom</backend> - <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </rng> </devices> </domain> -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
Now the a dmi-to-pci-bridge is automatically added just as it's needed (when a pci-bridge is being added), we no longer have any need to force-add one to every single Q35 domain. --- src/qemu/qemu_domain.c | 12 ---- tests/qemuxml2argvdata/qemuxml2argv-pcie-root.args | 3 +- .../qemuxml2argv-q35-pcie-autoadd.args | 55 +++++++-------- .../qemuxml2xmlout-pcie-root.xml | 4 -- .../qemuxml2xmlout-q35-pcie-autoadd.xml | 82 ++++++++++------------ 5 files changed, 67 insertions(+), 89 deletions(-)
ACK -- Andrea Bolognani / Red Hat / Virtualization
Previously we added a set of EHCI+UHCI controllers to Q35 machines to mimic real hardware as closely as possible, but recent discussions have pointed out that the nec-usb-xhci (USB3) controller is much more virtualization-friendly (uses less CPU), so this patch switches the default for Q35 machinetypes to add an XHCI instead (if it's supported, which it of course *will* be). Since none of the existing test cases left out USB controllers in the input XML, a new Q35 test case was added which has *no* devices, so ends up with only the defaults always put in by qemu, plus those added by libvirt. --- src/qemu/qemu_domain.c | 10 ++++-- .../qemuxml2argv-q35-default-devices-only.args | 22 ++++++++++++ .../qemuxml2argv-q35-default-devices-only.xml | 18 ++++++++++ tests/qemuxml2argvtest.c | 23 +++++++++++++ .../qemuxml2xmlout-q35-default-devices-only.xml | 40 ++++++++++++++++++++++ tests/qemuxml2xmltest.c | 23 +++++++++++++ 6 files changed, 133 insertions(+), 3 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-default-devices-only.xml diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 3821d37..0f113a7 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -2190,10 +2190,14 @@ qemuDomainDefAddDefaultDevices(virDomainDefPtr def, addPCIeRoot = true; addImplicitSATA = true; - /* add a USB2 controller set, but only if the - * ich9-usb-ehci1 device is supported + /* Prefer adding USB3 controller if supported + * (nec-usb-xhci). Failing that, add a USB2 controller set + * if the ich9-usb-ehci1 device is supported. Otherwise + * don't add anything. */ - if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_ICH9_USB_EHCI1)) + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_NEC_USB_XHCI)) + usbModel = VIR_DOMAIN_CONTROLLER_MODEL_USB_NEC_XHCI; + else if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_ICH9_USB_EHCI1)) usbModel = VIR_DOMAIN_CONTROLLER_MODEL_USB_ICH9_EHCI1; else addDefaultUSB = false; diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.args new file mode 100644 index 0000000..9ac30dd --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.args @@ -0,0 +1,22 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/libexec/qemu-kvm \ +-name q35-test \ +-S \ +-M q35 \ +-m 2048 \ +-smp 2,sockets=2,cores=1,threads=1 \ +-uuid 11dbdcdd-4c3b-482b-8903-9bdb8c0a2774 \ +-nographic \ +-nodefaults \ +-monitor unix:/tmp/lib/domain--1-q35-test/monitor.sock,server,nowait \ +-no-acpi \ +-boot c \ +-device ioh3420,port=0x8,chassis=1,id=pci.1,bus=pcie.0,addr=0x1 \ +-device ioh3420,port=0x10,chassis=2,id=pci.2,bus=pcie.0,addr=0x2 \ +-device nec-usb-xhci,id=usb,bus=pci.1,addr=0x0 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x0 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.xml b/tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.xml new file mode 100644 index 0000000..7436583 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.xml @@ -0,0 +1,18 @@ +<domain type='qemu'> + <name>q35-test</name> + <uuid>11dbdcdd-4c3b-482b-8903-9bdb8c0a2774</uuid> + <memory unit='KiB'>2097152</memory> + <currentMemory unit='KiB'>2097152</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='x86_64' machine='q35'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 98102fe..254bdaa 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1843,6 +1843,29 @@ mymain(void) QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY); + DO_TEST("q35-default-devices-only", + QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, + QEMU_CAPS_DEVICE_VIRTIO_RNG, + QEMU_CAPS_OBJECT_RNG_RANDOM, + QEMU_CAPS_NETDEV, + QEMU_CAPS_DEVICE_VIRTIO_NET, + QEMU_CAPS_DEVICE_VIRTIO_GPU, + QEMU_CAPS_VIRTIO_GPU_VIRGL, + QEMU_CAPS_VIRTIO_KEYBOARD, + QEMU_CAPS_VIRTIO_MOUSE, + QEMU_CAPS_VIRTIO_TABLET, + QEMU_CAPS_VIRTIO_INPUT_HOST, + QEMU_CAPS_VIRTIO_SCSI, + QEMU_CAPS_FSDEV, + QEMU_CAPS_FSDEV_WRITEOUT, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_NEC_USB_XHCI, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY); DO_TEST("pcie-root-port", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-default-devices-only.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-default-devices-only.xml new file mode 100644 index 0000000..8d7bc9d --- /dev/null +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-default-devices-only.xml @@ -0,0 +1,40 @@ +<domain type='qemu'> + <name>q35-test</name> + <uuid>11dbdcdd-4c3b-482b-8903-9bdb8c0a2774</uuid> + <memory unit='KiB'>2097152</memory> + <currentMemory unit='KiB'>2097152</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='x86_64' machine='q35'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <controller type='usb' index='0' model='nec-xhci'> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='sata' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> + </controller> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='1' port='0x8'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='2' port='0x10'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </memballoon> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index f8b8809..24d9ab7 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -765,6 +765,29 @@ mymain(void) QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY); + DO_TEST("q35-default-devices-only", + QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, + QEMU_CAPS_DEVICE_VIRTIO_RNG, + QEMU_CAPS_OBJECT_RNG_RANDOM, + QEMU_CAPS_NETDEV, + QEMU_CAPS_DEVICE_VIRTIO_NET, + QEMU_CAPS_DEVICE_VIRTIO_GPU, + QEMU_CAPS_VIRTIO_GPU_VIRGL, + QEMU_CAPS_VIRTIO_KEYBOARD, + QEMU_CAPS_VIRTIO_MOUSE, + QEMU_CAPS_VIRTIO_TABLET, + QEMU_CAPS_VIRTIO_INPUT_HOST, + QEMU_CAPS_VIRTIO_SCSI, + QEMU_CAPS_FSDEV, + QEMU_CAPS_FSDEV_WRITEOUT, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_NEC_USB_XHCI, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY); DO_TEST("pcie-root", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
Previously we added a set of EHCI+UHCI controllers to Q35 machines to mimic real hardware as closely as possible, but recent discussions have pointed out that the nec-usb-xhci (USB3) controller is much more virtualization-friendly (uses less CPU), so this patch switches the default for Q35 machinetypes to add an XHCI instead (if it's supported, which it of course *will* be). Since none of the existing test cases left out USB controllers in the input XML, a new Q35 test case was added which has *no* devices, so ends up with only the defaults always put in by qemu, plus those added by libvirt.
We should probably have added that test case a while ago :) ACK -- Andrea Bolognani / Red Hat / Virtualization
Real Q35 hardware has an ICH9 chip that includes several integrated devices at particular addresses (see the file docs/q35-chipset.cfg in the qemu source). libvirt already attempts to put the first two sets of ich9 USB2 controllers it finds at 00:1D.* and 00:1A.* to match the real hardware. This patch does the same for the ich9 "HD audio" device. The main inspiration for this patch is that currently the *only* device in a reasonable "workstation" type virtual machine config that requires a legacy PCI slot is the audio device, Without this patch, the standard Q35 machine created by virt-manager will have a dmi-to-pci-bridge and a pci-bridge just for the sound device; with the patch (and if you change the sound device model from the default "ich6" to "ich9"), the machine definition constructed by virt-manager has absolutely no legacy PCI controllers - any legacy PCI devices (e.g. video and sound) are on pcie-root as integrated devices. --- src/qemu/qemu_domain_address.c | 26 +++++ .../qemuxml2argv-q35-virt-manager-basic.args | 55 ++++++++++ .../qemuxml2argv-q35-virt-manager-basic.xml | 76 ++++++++++++++ tests/qemuxml2argvtest.c | 34 ++++++ .../qemuxml2xmlout-q35-virt-manager-basic.xml | 116 +++++++++++++++++++++ tests/qemuxml2xmltest.c | 24 +++++ 6 files changed, 331 insertions(+) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virt-manager-basic.xml diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index ea09556..fbbcfb2 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -1334,6 +1334,32 @@ qemuDomainValidateDevicePCISlotsQ35(virDomainDefPtr def, goto cleanup; } } + + memset(&tmp_addr, 0, sizeof(tmp_addr)); + tmp_addr.slot = 0x1B; + if (!virDomainPCIAddressSlotInUse(addrs, &tmp_addr)) { + /* Since real Q35 hardware has an ICH9 chip that has an + * integrated HD audio device at 0000:00:1B.0 put any + * unaddressed ICH9 audio device at that address if it's not + * already taken. If there's something already there, let the + * normal device addressing assign something later. + */ + for (i = 0; i < def->nsounds; i++) { + virDomainSoundDefPtr sound = def->sounds[i]; + + if (sound->model != VIR_DOMAIN_SOUND_MODEL_ICH9 || + !virDeviceInfoPCIAddressWanted(&sound->info)) { + continue; + } + if (virDomainPCIAddressReserveSlot(addrs, &tmp_addr, flags) < 0) + goto cleanup; + + sound->info.type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI; + sound->info.addr.pci = tmp_addr; + break; + } + } + ret = 0; cleanup: VIR_FREE(addrStr); diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.args new file mode 100644 index 0000000..d3217d5 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.args @@ -0,0 +1,55 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=spice \ +/usr/bin/qemu-system_x86_64 \ +-name virt-manager-basic \ +-S \ +-M pc-q35-2.7 \ +-m 4096 \ +-smp 2,sockets=2,cores=1,threads=1 \ +-uuid 1b826c23-8767-47ad-a6b5-c83a88277f71 \ +-nodefaults \ +-monitor unix:/tmp/lib/domain--1-virt-manager-basic/monitor.sock,server,nowait \ +-rtc base=utc,driftfix=slew \ +-no-kvm-pit-reinjection \ +-global ICH9-LPC.disable_s3=1 \ +-global ICH9-LPC.disable_s4=1 \ +-boot c \ +-device ioh3420,port=0x10,chassis=1,id=pci.1,bus=pcie.0,addr=0x2 \ +-device ioh3420,port=0x18,chassis=2,id=pci.2,bus=pcie.0,addr=0x3 \ +-device ioh3420,port=0x20,chassis=3,id=pci.3,bus=pcie.0,addr=0x4 \ +-device ioh3420,port=0x28,chassis=4,id=pci.4,bus=pcie.0,addr=0x5 \ +-device ioh3420,port=0x30,chassis=5,id=pci.5,bus=pcie.0,addr=0x6 \ +-device nec-usb-xhci,id=usb,bus=pci.2,addr=0x0 \ +-device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 \ +-drive file=/var/lib/libvirt/images/basic.qcow2,format=qcow2,if=none,\ +id=drive-virtio-disk0 \ +-device virtio-blk-pci,bus=pci.4,addr=0x0,drive=drive-virtio-disk0,\ +id=virtio-disk0 \ +-netdev user,id=hostnet0 \ +-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:9a:e6:c6,bus=pci.1,\ +addr=0x0 \ +-serial pty \ +-chardev socket,id=charchannel0,\ +path=/tmp/channel/domain--1-virt-manager-basic/org.qemu.guest_agent.0,server,\ +nowait \ +-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,\ +id=channel0,name=org.qemu.guest_agent.0 \ +-chardev spicevmc,id=charchannel1,name=vdagent \ +-device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,\ +id=channel1,name=com.redhat.spice.0 \ +-device usb-tablet,id=input0,bus=usb.0,port=1 \ +-spice port=5901,tls-port=5902,addr=127.0.0.1,x509-dir=/etc/pki/libvirt-spice,\ +image-compression=off \ +-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,bus=pcie.0,\ +addr=0x1 \ +-device ich9-intel-hda,id=sound0,bus=pcie.0,addr=0x1b \ +-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \ +-chardev spicevmc,id=charredir0,name=usbredir \ +-device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 \ +-chardev spicevmc,id=charredir1,name=usbredir \ +-device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.5,addr=0x0 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.xml b/tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.xml new file mode 100644 index 0000000..6d212a4 --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.xml @@ -0,0 +1,76 @@ +<domain type='kvm'> + <name>virt-manager-basic</name> + <uuid>1b826c23-8767-47ad-a6b5-c83a88277f71</uuid> + <memory unit='KiB'>4194304</memory> + <currentMemory unit='KiB'>4194304</currentMemory> + <vcpu placement='static'>2</vcpu> + <os> + <type arch='x86_64' machine='pc-q35-2.7'>hvm</type> + <boot dev='hd'/> + </os> + <features> + <acpi/> + <apic/> + <vmport state='off'/> + </features> + <clock offset='utc'> + <timer name='rtc' tickpolicy='catchup'/> + <timer name='pit' tickpolicy='delay'/> + <timer name='hpet' present='no'/> + </clock> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>restart</on_crash> + <pm> + <suspend-to-mem enabled='no'/> + <suspend-to-disk enabled='no'/> + </pm> + <devices> + <emulator>/usr/bin/qemu-system_x86_64</emulator> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2'/> + <source file='/var/lib/libvirt/images/basic.qcow2'/> + <target dev='vda' bus='virtio'/> + </disk> + <controller type='usb' model='nec-xhci'/> + <controller type='sata'/> + <controller type='virtio-serial' index='0'/> + <interface type='user'> + <mac address='52:54:00:9a:e6:c6'/> + <model type='virtio'/> + </interface> + <serial type='pty'> + <target port='0'/> + </serial> + <console type='pty'> + <target type='serial' port='0'/> + </console> + <channel type='unix'> + <target type='virtio' name='org.qemu.guest_agent.0'/> + <address type='virtio-serial' controller='0' bus='0' port='1'/> + </channel> + <channel type='spicevmc'> + <target type='virtio' name='com.redhat.spice.0'/> + <address type='virtio-serial' controller='0' bus='0' port='2'/> + </channel> + <input type='tablet' bus='usb'> + <address type='usb' bus='0' port='1'/> + </input> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <graphics type='spice' autoport='yes'> + <listen type='address'/> + <image compression='off'/> + </graphics> + <sound model='ich9'/> + <video> + <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> + </video> + <redirdev bus='usb' type='spicevmc'> + <address type='usb' bus='0' port='2'/> + </redirdev> + <redirdev bus='usb' type='spicevmc'> + <address type='usb' bus='0' port='3'/> + </redirdev> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 254bdaa..b29ca23 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1866,6 +1866,40 @@ mymain(void) QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY); + DO_TEST("q35-virt-manager-basic", + QEMU_CAPS_KVM, + QEMU_CAPS_RTC, + QEMU_CAPS_NO_KVM_PIT, + QEMU_CAPS_ICH9_DISABLE_S3, + QEMU_CAPS_ICH9_DISABLE_S4, + QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, + QEMU_CAPS_DEVICE_VIRTIO_RNG, + QEMU_CAPS_OBJECT_RNG_RANDOM, + QEMU_CAPS_NETDEV, + QEMU_CAPS_DEVICE_VIRTIO_NET, + QEMU_CAPS_DEVICE_VIRTIO_GPU, + QEMU_CAPS_VIRTIO_GPU_VIRGL, + QEMU_CAPS_VIRTIO_KEYBOARD, + QEMU_CAPS_VIRTIO_MOUSE, + QEMU_CAPS_VIRTIO_TABLET, + QEMU_CAPS_VIRTIO_INPUT_HOST, + QEMU_CAPS_VIRTIO_SCSI, + QEMU_CAPS_FSDEV, + QEMU_CAPS_FSDEV_WRITEOUT, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_NEC_USB_XHCI, + QEMU_CAPS_DEVICE_ICH9_INTEL_HDA, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY, + QEMU_CAPS_SPICE, + QEMU_CAPS_CHARDEV_SPICEVMC, + QEMU_CAPS_DEVICE_QXL, + QEMU_CAPS_HDA_DUPLEX, + QEMU_CAPS_USB_REDIR); DO_TEST("pcie-root-port", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virt-manager-basic.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virt-manager-basic.xml new file mode 100644 index 0000000..fe9ea4e --- /dev/null +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virt-manager-basic.xml @@ -0,0 +1,116 @@ +<domain type='kvm'> + <name>virt-manager-basic</name> + <uuid>1b826c23-8767-47ad-a6b5-c83a88277f71</uuid> + <memory unit='KiB'>4194304</memory> + <currentMemory unit='KiB'>4194304</currentMemory> + <vcpu placement='static'>2</vcpu> + <os> + <type arch='x86_64' machine='pc-q35-2.7'>hvm</type> + <boot dev='hd'/> + </os> + <features> + <acpi/> + <apic/> + <vmport state='off'/> + </features> + <clock offset='utc'> + <timer name='rtc' tickpolicy='catchup'/> + <timer name='pit' tickpolicy='delay'/> + <timer name='hpet' present='no'/> + </clock> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>restart</on_crash> + <pm> + <suspend-to-mem enabled='no'/> + <suspend-to-disk enabled='no'/> + </pm> + <devices> + <emulator>/usr/bin/qemu-system_x86_64</emulator> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2'/> + <source file='/var/lib/libvirt/images/basic.qcow2'/> + <target dev='vda' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </disk> + <controller type='usb' index='0' model='nec-xhci'> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <controller type='sata' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> + </controller> + <controller type='virtio-serial' index='0'> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='1' port='0x10'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='2' port='0x18'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='3' port='0x20'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='4' port='0x28'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </controller> + <controller type='pci' index='5' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='5' port='0x30'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> + </controller> + <interface type='user'> + <mac address='52:54:00:9a:e6:c6'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </interface> + <serial type='pty'> + <target port='0'/> + </serial> + <console type='pty'> + <target type='serial' port='0'/> + </console> + <channel type='unix'> + <target type='virtio' name='org.qemu.guest_agent.0'/> + <address type='virtio-serial' controller='0' bus='0' port='1'/> + </channel> + <channel type='spicevmc'> + <target type='virtio' name='com.redhat.spice.0'/> + <address type='virtio-serial' controller='0' bus='0' port='2'/> + </channel> + <input type='tablet' bus='usb'> + <address type='usb' bus='0' port='1'/> + </input> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <graphics type='spice' autoport='yes'> + <listen type='address'/> + <image compression='off'/> + </graphics> + <sound model='ich9'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/> + </sound> + <video> + <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </video> + <redirdev bus='usb' type='spicevmc'> + <address type='usb' bus='0' port='2'/> + </redirdev> + <redirdev bus='usb' type='spicevmc'> + <address type='usb' bus='0' port='3'/> + </redirdev> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> + </memballoon> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 24d9ab7..eb95061 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -788,6 +788,30 @@ mymain(void) QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_NEC_USB_XHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY); + DO_TEST("q35-virt-manager-basic", + QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, + QEMU_CAPS_DEVICE_VIRTIO_RNG, + QEMU_CAPS_OBJECT_RNG_RANDOM, + QEMU_CAPS_NETDEV, + QEMU_CAPS_DEVICE_VIRTIO_NET, + QEMU_CAPS_DEVICE_VIRTIO_GPU, + QEMU_CAPS_VIRTIO_GPU_VIRGL, + QEMU_CAPS_VIRTIO_KEYBOARD, + QEMU_CAPS_VIRTIO_MOUSE, + QEMU_CAPS_VIRTIO_TABLET, + QEMU_CAPS_VIRTIO_INPUT_HOST, + QEMU_CAPS_VIRTIO_SCSI, + QEMU_CAPS_FSDEV, + QEMU_CAPS_FSDEV_WRITEOUT, + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_NEC_USB_XHCI, + QEMU_CAPS_DEVICE_ICH9_INTEL_HDA, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY); DO_TEST("pcie-root", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
Real Q35 hardware has an ICH9 chip that includes several integrated devices at particular addresses (see the file docs/q35-chipset.cfg in the qemu source). libvirt already attempts to put the first two sets of ich9 USB2 controllers it finds at 00:1D.* and 00:1A.* to match the real hardware. This patch does the same for the ich9 "HD audio" device.
The main inspiration for this patch is that currently the *only* device in a reasonable "workstation" type virtual machine config that requires a legacy PCI slot is the audio device, Without this patch, the standard Q35 machine created by virt-manager will have a dmi-to-pci-bridge and a pci-bridge just for the sound device; with the patch (and if you change the sound device model from the default "ich6" to "ich9"), the machine definition constructed by virt-manager has absolutely no legacy PCI controllers - any legacy PCI devices (e.g. video and sound) are on pcie-root as integrated devices.
ACK -- Andrea Bolognani / Red Hat / Virtualization
For machinetypes with a pci-root bus (all legacy PCI), libvirt will make a "fake" reservation for one extra slot prior to assigning addresses to unaddressed PCI endpoint devices in the domain. This will trigger auto-adding of a pci-bridge for the final device to be assigned an address *if that device would have otherwise instead been the last device on the last available pci-bridge*; thus it assures that there will always be at least one slot left open in the domain's bus topology for expansion (which is important both for hotplug (since a new pci-bridge can't be added while the guest is running) as well as for offline additions to the config (since adding a new device might otherwise in some cases require re-addressing existing devices, which we want to avoid)). It's important to note that for the above case (legacy PCI), we must check for the special case of all slots on all buses being occupied *prior to assigning any addresses*, and avoid attempting to reserve the extra address in that case, because there is no free address in the existing topology, so no place to auto-add a pci-bridge for expansion (i.e. it would always fail anyway). Since that condition can only be reached by manual intervention, this is acceptable. For machinetypes with pcie-root (Q35, aarch64 virt), libvirt's methodology for automatically expanding the bus topology is different - pcie-root-ports are plugged into slots (soon to be functions) of pcie-root as needed, and the new endpoint devices are assigned to the single slot in each pcie-root-port. This is done so that the devices are, by default, hotpluggable (the slots of pcie-root don't support hotplug, but the single slot of the pcie-root-port does). Since pcie-root-ports can only be plugged into pcie-root, and we don't auto-assign endpoint devices to the pcie-root slots, this means topology expansion doesn't compete with endpoint devices for slots, so we don't need to worry about checking for all "useful" slots being free *prior* to assigning addresses to new endpoint devices - as a matter of fact, if we attempt to reserve the open slots before the used slots, it can lead to errors. Instead this patch just reserves one slot for a "future potential" PCIe device after doing the assignment for actual devices, but only if the only PCI controller defined prior to starting address assignment was pcie-root, and only if we auto-added at least one PCI controller during address assignment. This assures two things: 1) that reserving the open slots will only be done when the domain is initially defined, never at any time after, and 2) that if the user understands enough about PCI controllers that they are adding them manually, that we don't mess up their plan by adding extras - if they know enough to add one pcie-root-port, or to manually assign addresses such that no pcie-root-ports are needed, they know enough to add extra pcie-root-ports if they want them (this could be called the "libguestfs clause", since libguestfs needs to be able to create domains with as few devices/controllers as possible). This is set to reserve a single free port for now, but could be increased in the future if public sentiment goes in that direction (it's easy to increase later, but essential impossible to decrease) --- src/qemu/qemu_domain_address.c | 30 ++++++++++++++++++++ .../qemuxml2argv-bios-nvram-secure.args | 1 + .../qemuxml2argv-machine-smm-opt.args | 1 + .../qemuxml2argv-q35-default-devices-only.args | 1 + .../qemuxml2argv-q35-pcie-autoadd.args | 1 + .../qemuxml2argv-q35-pm-disable-fallback.args | 1 + .../qemuxml2argv-q35-pm-disable.args | 1 + .../qemuxml2argv-q35-virt-manager-basic.args | 1 + tests/qemuxml2argvtest.c | 33 ++++++++++++++++++---- .../qemuxml2xmlout-q35-default-devices-only.xml | 5 ++++ .../qemuxml2xmlout-q35-pcie-autoadd.xml | 5 ++++ .../qemuxml2xmlout-q35-virt-manager-basic.xml | 5 ++++ tests/qemuxml2xmltest.c | 19 +++++++++++-- 13 files changed, 96 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index fbbcfb2..15d7c1a 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -1929,6 +1929,36 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, if (qemuDomainAssignDevicePCISlots(def, qemuCaps, addrs) < 0) goto cleanup; + /* Only for *new* domains with pcie-root (and no other + * manually specified PCI controllers in the definition): If, + * after assigning addresses/reserving slots for all devices, + * we see that any extra buses have been auto-added, we + * understand that the application has left management of PCI + * addresses and controllers up to libvirt. In order to allow + * such applications to easily support hotplug, we will do a + * "one time" reservation of one extra PCIE|HOTPLUGGABLE + * slots, which should cause su to auto-add 1 extra + * pcie-root-ports The single slot in this root-port will be + * available for hotplug, or may also be used when a device is + * added to the config offline. + */ + + if (max_idx <= 0 && + addrs->nbuses > max_idx + 1 && + qemuDomainMachineHasPCIeRoot(def)) { + info.pciConnectFlags = (VIR_PCI_CONNECT_HOTPLUGGABLE | + VIR_PCI_CONNECT_TYPE_PCIE_DEVICE); + + /* if there isn't an empty pcie-root-port, this will + * cause one to be added + */ + if (qemuDomainPCIAddressReserveNextSlot(addrs, &info) < 0) + goto cleanup; + } + + /* now reflect any controllers auto-added to addrs into the + * domain controllers list + */ for (i = 1; i < addrs->nbuses; i++) { virDomainDeviceDef dev; int contIndex; diff --git a/tests/qemuxml2argvdata/qemuxml2argv-bios-nvram-secure.args b/tests/qemuxml2argvdata/qemuxml2argv-bios-nvram-secure.args index c014254..c5fe72b 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-bios-nvram-secure.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-bios-nvram-secure.args @@ -21,6 +21,7 @@ readonly=on \ -boot c \ -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \ +-device ioh3420,port=0x10,chassis=3,id=pci.3,bus=pcie.0,addr=0x2 \ -device virtio-scsi-pci,id=scsi0,bus=pci.2,addr=0x1 \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-scsi0-0-0-0 \ -device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ diff --git a/tests/qemuxml2argvdata/qemuxml2argv-machine-smm-opt.args b/tests/qemuxml2argvdata/qemuxml2argv-machine-smm-opt.args index e49d7e9..cc9c355 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-machine-smm-opt.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-machine-smm-opt.args @@ -18,6 +18,7 @@ QEMU_AUDIO_DRV=none \ -boot c \ -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \ +-device ioh3420,port=0x10,chassis=3,id=pci.3,bus=pcie.0,addr=0x2 \ -device virtio-scsi-pci,id=scsi0,bus=pci.2,addr=0x1 \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-scsi0-0-0-0 \ -device scsi-disk,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.args index 9ac30dd..9d13466 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-default-devices-only.args @@ -18,5 +18,6 @@ QEMU_AUDIO_DRV=none \ -boot c \ -device ioh3420,port=0x8,chassis=1,id=pci.1,bus=pcie.0,addr=0x1 \ -device ioh3420,port=0x10,chassis=2,id=pci.2,bus=pcie.0,addr=0x2 \ +-device ioh3420,port=0x18,chassis=3,id=pci.3,bus=pcie.0,addr=0x3 \ -device nec-usb-xhci,id=usb,bus=pci.1,addr=0x0 \ -device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x0 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args index daecf96..ba26326 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pcie-autoadd.args @@ -29,6 +29,7 @@ QEMU_AUDIO_DRV=none \ -device ioh3420,port=0x60,chassis=11,id=pci.11,bus=pcie.0,addr=0xc \ -device ioh3420,port=0x68,chassis=12,id=pci.12,bus=pcie.0,addr=0xd \ -device ioh3420,port=0x70,chassis=13,id=pci.13,bus=pcie.0,addr=0xe \ +-device ioh3420,port=0x78,chassis=14,id=pci.14,bus=pcie.0,addr=0xf \ -device nec-usb-xhci,id=usb,bus=pci.6,addr=0x0 \ -device virtio-scsi-pci,id=scsi0,bus=pci.5,addr=0x0 \ -device virtio-serial-pci,id=virtio-serial0,bus=pci.4,addr=0x0 \ diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pm-disable-fallback.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-pm-disable-fallback.args index deae687..739c918 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-pm-disable-fallback.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pm-disable-fallback.args @@ -20,4 +20,5 @@ QEMU_AUDIO_DRV=none \ -boot n \ -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \ +-device ioh3420,port=0x10,chassis=3,id=pci.3,bus=pcie.0,addr=0x2 \ -device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x1 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-pm-disable.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-pm-disable.args index 871340f..5ee16b7 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-pm-disable.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-pm-disable.args @@ -20,4 +20,5 @@ QEMU_AUDIO_DRV=none \ -boot n \ -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \ +-device ioh3420,port=0x10,chassis=3,id=pci.3,bus=pcie.0,addr=0x2 \ -device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x1 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.args index d3217d5..60af251 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.args +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-virt-manager-basic.args @@ -23,6 +23,7 @@ QEMU_AUDIO_DRV=spice \ -device ioh3420,port=0x20,chassis=3,id=pci.3,bus=pcie.0,addr=0x4 \ -device ioh3420,port=0x28,chassis=4,id=pci.4,bus=pcie.0,addr=0x5 \ -device ioh3420,port=0x30,chassis=5,id=pci.5,bus=pcie.0,addr=0x6 \ +-device ioh3420,port=0x38,chassis=6,id=pci.6,bus=pcie.0,addr=0x7 \ -device nec-usb-xhci,id=usb,bus=pci.2,addr=0x0 \ -device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 \ -drive file=/var/lib/libvirt/images/basic.qcow2,format=qcow2,if=none,\ diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index b29ca23..2b6ce10 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -656,6 +656,7 @@ mymain(void) DO_TEST("machine-smm-opt", QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_MACHINE_OPT, QEMU_CAPS_MACHINE_SMM_OPT, @@ -673,10 +674,12 @@ mymain(void) DO_TEST("boot-floppy-q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI); DO_TEST("bootindex-floppy-q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_BOOT_MENU, QEMU_CAPS_BOOTINDEX); DO_TEST("boot-multi", QEMU_CAPS_BOOT_MENU); @@ -723,6 +726,7 @@ mymain(void) DO_TEST("bios-nvram-secure", QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_MACHINE_OPT, QEMU_CAPS_MACHINE_SMM_OPT, @@ -1336,18 +1340,22 @@ mymain(void) QEMU_CAPS_CHARDEV, QEMU_CAPS_NODEFCONFIG); DO_TEST("usb-controller-default-q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_PCI_OHCI, QEMU_CAPS_PIIX3_USB_UHCI, QEMU_CAPS_NEC_USB_XHCI); DO_TEST_FAILURE("usb-controller-default-unavailable-q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_PCI_OHCI, QEMU_CAPS_NEC_USB_XHCI); DO_TEST("usb-controller-explicit-q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_PCI_OHCI, QEMU_CAPS_PIIX3_USB_UHCI, QEMU_CAPS_NEC_USB_XHCI); DO_TEST_FAILURE("usb-controller-explicit-unavailable-q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_PCI_OHCI, QEMU_CAPS_PIIX3_USB_UHCI); DO_TEST("usb-controller-xhci", @@ -1723,10 +1731,12 @@ mymain(void) DO_TEST("pcie-root", QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_DEVICE_PCI_BRIDGE, - QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE); + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420); DO_TEST("q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, @@ -1741,16 +1751,19 @@ mymain(void) QEMU_CAPS_DEVICE_IOH3420); DO_TEST("q35-pm-disable", QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PIIX_DISABLE_S3, QEMU_CAPS_PIIX_DISABLE_S4, QEMU_CAPS_ICH9_DISABLE_S3, QEMU_CAPS_ICH9_DISABLE_S4); DO_TEST("q35-pm-disable-fallback", QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PIIX_DISABLE_S3, QEMU_CAPS_PIIX_DISABLE_S4); DO_TEST("q35-usb2", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, @@ -1758,6 +1771,7 @@ mymain(void) DO_TEST("q35-usb2-multi", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, @@ -1765,6 +1779,7 @@ mymain(void) DO_TEST("q35-usb2-reorder", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, @@ -1927,6 +1942,7 @@ mymain(void) DO_TEST_PARSE_ERROR("q35-wrong-root", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, @@ -2040,6 +2056,7 @@ mymain(void) DO_TEST("pcihole64-q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, QEMU_CAPS_DEVICE_QXL, @@ -2072,13 +2089,15 @@ mymain(void) QEMU_CAPS_DEVICE_VIRTIO_MMIO, QEMU_CAPS_DEVICE_VIRTIO_RNG, QEMU_CAPS_OBJECT_RNG_RANDOM, QEMU_CAPS_OBJECT_GPEX, QEMU_CAPS_DEVICE_PCI_BRIDGE, - QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE); + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420); DO_TEST("aarch64-virt-2.6-virtio-pci-default", QEMU_CAPS_NODEFCONFIG, QEMU_CAPS_DTB, QEMU_CAPS_DEVICE_VIRTIO_MMIO, QEMU_CAPS_DEVICE_VIRTIO_RNG, QEMU_CAPS_OBJECT_RNG_RANDOM, QEMU_CAPS_OBJECT_GPEX, QEMU_CAPS_DEVICE_PCI_BRIDGE, - QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE); + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420); /* Example of using virtio-pci with no explicit PCI controller but with manual PCI addresses */ DO_TEST("aarch64-virtio-pci-manual-addresses", @@ -2086,7 +2105,9 @@ mymain(void) QEMU_CAPS_DEVICE_VIRTIO_MMIO, QEMU_CAPS_DEVICE_VIRTIO_RNG, QEMU_CAPS_OBJECT_RNG_RANDOM, QEMU_CAPS_OBJECT_GPEX, QEMU_CAPS_DEVICE_PCI_BRIDGE, - QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_VIRTIO_SCSI); + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_VIRTIO_SCSI); DO_TEST("aarch64-video-virtio-gpu-pci", QEMU_CAPS_NODEFCONFIG, QEMU_CAPS_OBJECT_GPEX, QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_IOH3420, @@ -2345,7 +2366,9 @@ mymain(void) DO_TEST("acpi-table", NONE); DO_TEST("intel-iommu", QEMU_CAPS_DEVICE_PCI_BRIDGE, - QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_DEVICE_INTEL_IOMMU); + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_DEVICE_INTEL_IOMMU); DO_TEST("intel-iommu-machine", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_MACHINE_OPT, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_MACHINE_IOMMU); diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-default-devices-only.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-default-devices-only.xml index 8d7bc9d..e64b80c 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-default-devices-only.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-default-devices-only.xml @@ -31,6 +31,11 @@ <target chassis='2' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='3' port='0x18'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </controller> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <memballoon model='virtio'> diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml index d784801..3742e14 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-pcie-autoadd.xml @@ -97,6 +97,11 @@ <target chassis='13' port='0x70'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0e' function='0x0'/> </controller> + <controller type='pci' index='14' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='14' port='0x78'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0f' function='0x0'/> + </controller> <filesystem type='mount' accessmode='passthrough'> <source dir='/export/to/guest'/> <target dir='/import/from/host'/> diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virt-manager-basic.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virt-manager-basic.xml index fe9ea4e..236d955 100644 --- a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virt-manager-basic.xml +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-virt-manager-basic.xml @@ -68,6 +68,11 @@ <target chassis='5' port='0x30'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </controller> + <controller type='pci' index='6' model='pcie-root-port'> + <model name='ioh3420'/> + <target chassis='6' port='0x38'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> + </controller> <interface type='user'> <mac address='52:54:00:9a:e6:c6'/> <model type='virtio'/> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index eb95061..e22b63f 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -593,10 +593,12 @@ mymain(void) QEMU_CAPS_PIIX3_USB_UHCI); DO_TEST("usb-controller-default-q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_PCI_OHCI, QEMU_CAPS_PIIX3_USB_UHCI, QEMU_CAPS_NEC_USB_XHCI); DO_TEST("usb-controller-explicit-q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_PCI_OHCI, QEMU_CAPS_PIIX3_USB_UHCI, QEMU_CAPS_NEC_USB_XHCI); DO_TEST("ppc64-usb-controller", @@ -676,23 +678,27 @@ mymain(void) DO_TEST("q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, QEMU_CAPS_DEVICE_QXL); DO_TEST("q35-usb2", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, QEMU_CAPS_DEVICE_QXL); DO_TEST("q35-usb2-multi", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, QEMU_CAPS_DEVICE_QXL); DO_TEST("q35-usb2-reorder", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, QEMU_CAPS_DEVICE_QXL); @@ -906,6 +912,7 @@ mymain(void) DO_TEST("pcihole64-none", NONE); DO_TEST("pcihole64-q35", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, QEMU_CAPS_DEVICE_QXL, QEMU_CAPS_Q35_PCI_HOLE64_SIZE); @@ -950,13 +957,17 @@ mymain(void) QEMU_CAPS_DEVICE_VIRTIO_MMIO, QEMU_CAPS_DEVICE_VIRTIO_RNG, QEMU_CAPS_OBJECT_RNG_RANDOM, QEMU_CAPS_OBJECT_GPEX, QEMU_CAPS_DEVICE_PCI_BRIDGE, - QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_VIRTIO_SCSI); + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_VIRTIO_SCSI); DO_TEST("aarch64-virtio-pci-manual-addresses", QEMU_CAPS_NODEFCONFIG, QEMU_CAPS_DTB, QEMU_CAPS_DEVICE_VIRTIO_MMIO, QEMU_CAPS_DEVICE_VIRTIO_RNG, QEMU_CAPS_OBJECT_RNG_RANDOM, QEMU_CAPS_OBJECT_GPEX, QEMU_CAPS_DEVICE_PCI_BRIDGE, - QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, QEMU_CAPS_VIRTIO_SCSI); + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420, + QEMU_CAPS_VIRTIO_SCSI); DO_TEST("aarch64-video-virtio-gpu-pci", QEMU_CAPS_NODEFCONFIG, QEMU_CAPS_OBJECT_GPEX, QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_IOH3420, @@ -1004,7 +1015,9 @@ mymain(void) DO_TEST("video-virtio-gpu-secondary", NONE); DO_TEST("intel-iommu", - QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE); + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_DEVICE_IOH3420); qemuTestDriverFree(&driver); -- 2.7.4
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote: [...]
Instead this patch just reserves one slot for a "future potential" PCIe device after doing the assignment for actual devices, but only if the only PCI controller defined prior to starting address assignment was pcie-root, and only if we auto-added at least one PCI controller during address assignment. This assures two things:
Double space here. I wouldn't normally care about picking one style over the other, but you've used a single space everywhere else in the commit message :) [...]
This is set to reserve a single free port for now, but could be increased in the future if public sentiment goes in that direction (it's easy to increase later, but essential impossible to decrease)
s/essential/essentially/ [...]
diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index fbbcfb2..15d7c1a 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -1929,6 +1929,36 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, if (qemuDomainAssignDevicePCISlots(def, qemuCaps, addrs) < 0) goto cleanup; + /* Only for *new* domains with pcie-root (and no other + * manually specified PCI controllers in the definition): If, + * after assigning addresses/reserving slots for all devices, + * we see that any extra buses have been auto-added, we + * understand that the application has left management of PCI + * addresses and controllers up to libvirt. In order to allow + * such applications to easily support hotplug, we will do a + * "one time" reservation of one extra PCIE|HOTPLUGGABLE + * slots, which should cause su to auto-add 1 extra
s/cause su/cause us/
+ * pcie-root-ports The single slot in this root-port will be
s/pcie-root-ports/pcie-root-port./ ACK with the typos fixed. -- Andrea Bolognani / Red Hat / Virtualization
If libvirtd is running unprivileged, it is not able to read devices' PCI config data from sysfs, and qemuDomainDeviceCalculatePCIConnectFlags() will need to read the PCI config data in to positively determine whether or not a device is an Express device or a legacy PCI device. In order to avoid creating a parallel "quiet" version of the function that reads PCI config, this patch passes a virQEMUDriverPtr down through all the call chains that that initialize the qemuDomainFillDevicePCIConnectFlagsIterData, and saves the pointer with the rest of the iterdata so that it can be used by qemuDomainDeviceCalculatePCIConnectFlags(). This pointer isn't used yet, but will be used in an upcoming patch (that detects Express vs legacy PCI for VFIO assigned devices) to examine driver->privileged. --- src/qemu/qemu_domain.c | 2 +- src/qemu/qemu_domain_address.c | 33 +++++++++++++++++++++++---------- src/qemu/qemu_domain_address.h | 7 +++++-- src/qemu/qemu_hotplug.c | 19 ++++++++++--------- src/qemu/qemu_process.c | 13 +++++++++---- tests/qemuhotplugtest.c | 4 +++- 6 files changed, 51 insertions(+), 27 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 0f113a7..5642134 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -3123,7 +3123,7 @@ qemuDomainDefAssignAddresses(virDomainDef *def, goto cleanup; } - if (qemuDomainAssignAddresses(def, qemuCaps, NULL, newDomain) < 0) + if (qemuDomainAssignAddresses(def, qemuCaps, driver, NULL, newDomain) < 0) goto cleanup; ret = 0; diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 15d7c1a..65753c5 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -428,6 +428,7 @@ qemuDomainAssignARMVirtioMMIOAddresses(virDomainDefPtr def, */ static virDomainPCIConnectFlags qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, + virQEMUDriverPtr driver ATTRIBUTE_UNUSED, virDomainPCIConnectFlags pcieFlags, virDomainPCIConnectFlags virtioFlags) { @@ -666,6 +667,7 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, typedef struct { virDomainPCIConnectFlags virtioFlags; virDomainPCIConnectFlags pcieFlags; + virQEMUDriverPtr driver; } qemuDomainFillDevicePCIConnectFlagsIterData; @@ -678,8 +680,12 @@ typedef struct { static void qemuDomainFillDevicePCIConnectFlagsIterInit(virDomainDefPtr def, virQEMUCapsPtr qemuCaps, + virQEMUDriverPtr driver, qemuDomainFillDevicePCIConnectFlagsIterData *data) { + + data->driver = driver; + if (qemuDomainMachineHasPCIeRoot(def)) { data->pcieFlags = (VIR_PCI_CONNECT_TYPE_PCIE_DEVICE | VIR_PCI_CONNECT_HOTPLUGGABLE); @@ -719,7 +725,8 @@ qemuDomainFillDevicePCIConnectFlagsIter(virDomainDefPtr def ATTRIBUTE_UNUSED, qemuDomainFillDevicePCIConnectFlagsIterData *data = opaque; info->pciConnectFlags - = qemuDomainDeviceCalculatePCIConnectFlags(dev, data->pcieFlags, + = qemuDomainDeviceCalculatePCIConnectFlags(dev, data->driver, + data->pcieFlags, data->virtioFlags); return 0; } @@ -739,11 +746,12 @@ qemuDomainFillDevicePCIConnectFlagsIter(virDomainDefPtr def ATTRIBUTE_UNUSED, */ static int qemuDomainFillAllPCIConnectFlags(virDomainDefPtr def, - virQEMUCapsPtr qemuCaps) + virQEMUCapsPtr qemuCaps, + virQEMUDriverPtr driver) { qemuDomainFillDevicePCIConnectFlagsIterData data; - qemuDomainFillDevicePCIConnectFlagsIterInit(def, qemuCaps, &data); + qemuDomainFillDevicePCIConnectFlagsIterInit(def, qemuCaps, driver, &data); return virDomainDeviceInfoIterate(def, qemuDomainFillDevicePCIConnectFlagsIter, @@ -765,7 +773,8 @@ qemuDomainFillAllPCIConnectFlags(virDomainDefPtr def, static void qemuDomainFillDevicePCIConnectFlags(virDomainDefPtr def, virDomainDeviceDefPtr dev, - virQEMUCapsPtr qemuCaps) + virQEMUCapsPtr qemuCaps, + virQEMUDriverPtr driver) { virDomainDeviceInfoPtr info = virDomainDeviceGetInfo(dev); @@ -781,10 +790,11 @@ qemuDomainFillDevicePCIConnectFlags(virDomainDefPtr def, */ qemuDomainFillDevicePCIConnectFlagsIterData data; - qemuDomainFillDevicePCIConnectFlagsIterInit(def, qemuCaps, &data); + qemuDomainFillDevicePCIConnectFlagsIterInit(def, qemuCaps, driver, &data); info->pciConnectFlags - = qemuDomainDeviceCalculatePCIConnectFlags(dev, data.pcieFlags, + = qemuDomainDeviceCalculatePCIConnectFlags(dev, data.driver, + data.pcieFlags, data.virtioFlags); } } @@ -1851,6 +1861,7 @@ qemuDomainAddressFindNewBusNr(virDomainDefPtr def) static int qemuDomainAssignPCIAddresses(virDomainDefPtr def, virQEMUCapsPtr qemuCaps, + virQEMUDriverPtr driver, virDomainObjPtr obj) { int ret = -1; @@ -1877,7 +1888,7 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, * of all devices. This will be used to pick an appropriate * bus when assigning addresses. */ - if (qemuDomainFillAllPCIConnectFlags(def, qemuCaps) < 0) + if (qemuDomainFillAllPCIConnectFlags(def, qemuCaps, driver) < 0) goto cleanup; if (nbuses > 0 && @@ -1991,7 +2002,7 @@ qemuDomainAssignPCIAddresses(virDomainDefPtr def, dev.type = VIR_DOMAIN_DEVICE_CONTROLLER; dev.data.controller = def->controllers[contIndex]; /* set connect flags so it will be properly addressed */ - qemuDomainFillDevicePCIConnectFlags(def, &dev, qemuCaps); + qemuDomainFillDevicePCIConnectFlags(def, &dev, qemuCaps, driver); if (qemuDomainPCIAddressReserveNextSlot(addrs, &dev.data.controller->info) < 0) goto cleanup; @@ -2273,6 +2284,7 @@ qemuDomainAssignUSBAddresses(virDomainDefPtr def, int qemuDomainAssignAddresses(virDomainDefPtr def, virQEMUCapsPtr qemuCaps, + virQEMUDriverPtr driver, virDomainObjPtr obj, bool newDomain) { @@ -2287,7 +2299,7 @@ qemuDomainAssignAddresses(virDomainDefPtr def, qemuDomainAssignARMVirtioMMIOAddresses(def, qemuCaps); - if (qemuDomainAssignPCIAddresses(def, qemuCaps, obj) < 0) + if (qemuDomainAssignPCIAddresses(def, qemuCaps, driver, obj) < 0) return -1; if (qemuDomainAssignUSBAddresses(def, obj, newDomain) < 0) @@ -2313,12 +2325,13 @@ qemuDomainAssignAddresses(virDomainDefPtr def, */ int qemuDomainEnsurePCIAddress(virDomainObjPtr obj, + virQEMUDriverPtr driver, virDomainDeviceDefPtr dev) { qemuDomainObjPrivatePtr priv = obj->privateData; virDomainDeviceInfoPtr info = virDomainDeviceGetInfo(dev); - qemuDomainFillDevicePCIConnectFlags(obj->def, dev, priv->qemuCaps); + qemuDomainFillDevicePCIConnectFlags(obj->def, dev, priv->qemuCaps, driver); return virDomainPCIAddressEnsureAddr(priv->pciaddrs, info, info->pciConnectFlags); diff --git a/src/qemu/qemu_domain_address.h b/src/qemu/qemu_domain_address.h index 800859c..0cdfacb 100644 --- a/src/qemu/qemu_domain_address.h +++ b/src/qemu/qemu_domain_address.h @@ -25,6 +25,7 @@ # include "domain_addr.h" # include "domain_conf.h" +# include "qemu_conf.h" # include "qemu_capabilities.h" int qemuDomainSetSCSIControllerModel(const virDomainDef *def, @@ -33,13 +34,15 @@ int qemuDomainSetSCSIControllerModel(const virDomainDef *def, int qemuDomainAssignAddresses(virDomainDefPtr def, virQEMUCapsPtr qemuCaps, + virQEMUDriverPtr driver, virDomainObjPtr obj, bool newDomain) - ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3); int qemuDomainEnsurePCIAddress(virDomainObjPtr obj, + virQEMUDriverPtr driver, virDomainDeviceDefPtr dev) - ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2); + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3); void qemuDomainReleaseDeviceAddress(virDomainObjPtr vm, virDomainDeviceInfoPtr info, diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 41731ab..dc424d0 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -345,7 +345,7 @@ qemuDomainAttachVirtioDiskDevice(virConnectPtr conn, goto error; } else if (!disk->info.type || disk->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) { - if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) + if (qemuDomainEnsurePCIAddress(vm, driver, &dev) < 0) goto error; } releaseaddr = true; @@ -504,7 +504,7 @@ int qemuDomainAttachControllerDevice(virQEMUDriverPtr driver, if (controller->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE || controller->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) { - if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) + if (qemuDomainEnsurePCIAddress(vm, driver, &dev) < 0) goto cleanup; } else if (controller->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCW) { if (!(ccwaddrs = qemuDomainCCWAddrSetCreateFromDomain(vm->def))) @@ -1135,7 +1135,7 @@ qemuDomainAttachNetDevice(virQEMUDriverPtr driver, virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("virtio-s390 net device cannot be hotplugged.")); goto cleanup; - } else if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) { + } else if (qemuDomainEnsurePCIAddress(vm, driver, &dev) < 0) { goto cleanup; } @@ -1436,7 +1436,7 @@ qemuDomainAttachHostPCIDevice(virQEMUDriverPtr driver, if (qemuAssignDeviceHostdevAlias(vm->def, &hostdev->info->alias, -1) < 0) goto error; - if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) + if (qemuDomainEnsurePCIAddress(vm, driver, &dev) < 0) goto error; releaseaddr = true; if (backend != VIR_DOMAIN_HOSTDEV_PCI_BACKEND_VFIO && @@ -1767,7 +1767,8 @@ qemuDomainChrRemove(virDomainDefPtr vmdef, * 0 otherwise */ static int -qemuDomainAttachChrDeviceAssignAddr(virDomainObjPtr vm, +qemuDomainAttachChrDeviceAssignAddr(virQEMUDriverPtr driver, + virDomainObjPtr vm, virDomainChrDefPtr chr) { virDomainDefPtr def = vm->def; @@ -1782,7 +1783,7 @@ qemuDomainAttachChrDeviceAssignAddr(virDomainObjPtr vm, } else if (chr->deviceType == VIR_DOMAIN_CHR_DEVICE_TYPE_SERIAL && chr->targetType == VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_PCI) { - if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) + if (qemuDomainEnsurePCIAddress(vm, driver, &dev) < 0) return -1; return 1; @@ -1841,7 +1842,7 @@ int qemuDomainAttachChrDevice(virConnectPtr conn, if (qemuAssignDeviceChrAlias(vmdef, chr, -1) < 0) goto cleanup; - if ((rc = qemuDomainAttachChrDeviceAssignAddr(vm, chr)) < 0) + if ((rc = qemuDomainAttachChrDeviceAssignAddr(driver, vm, chr)) < 0) goto cleanup; if (rc == 1) need_release = true; @@ -1981,7 +1982,7 @@ qemuDomainAttachRNGDevice(virConnectPtr conn, if (rng->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE || rng->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) { - if (qemuDomainEnsurePCIAddress(vm, &dev) < 0) + if (qemuDomainEnsurePCIAddress(vm, driver, &dev) < 0) goto cleanup; } else if (rng->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCW) { if (!(ccwaddrs = qemuDomainCCWAddrSetCreateFromDomain(vm->def))) @@ -2521,7 +2522,7 @@ qemuDomainAttachShmemDevice(virQEMUDriverPtr driver, if ((shmem->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE || shmem->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) && - (qemuDomainEnsurePCIAddress(vm, &dev) < 0)) + (qemuDomainEnsurePCIAddress(vm, driver, &dev) < 0)) return -1; if (!(shmstr = qemuBuildShmemDevStr(vm->def, shmem, priv->qemuCaps))) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 1b67aee..a404471 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -3350,8 +3350,10 @@ qemuProcessReconnect(void *opaque) goto cleanup; } - if ((qemuDomainAssignAddresses(obj->def, priv->qemuCaps, obj, false)) < 0) + if ((qemuDomainAssignAddresses(obj->def, priv->qemuCaps, + driver, obj, false)) < 0) { goto error; + } /* if domain requests security driver we haven't loaded, report error, but * do not kill the domain @@ -5136,9 +5138,10 @@ qemuProcessPrepareDomain(virConnectPtr conn, * use in hotplug */ VIR_DEBUG("Assigning domain PCI addresses"); - if ((qemuDomainAssignAddresses(vm->def, priv->qemuCaps, vm, - !!(flags & VIR_QEMU_PROCESS_START_NEW))) < 0) + if ((qemuDomainAssignAddresses(vm->def, priv->qemuCaps, driver, vm, + !!(flags & VIR_QEMU_PROCESS_START_NEW))) < 0) { goto cleanup; + } if (qemuAssignDeviceAliases(vm->def, priv->qemuCaps) < 0) goto cleanup; @@ -6344,8 +6347,10 @@ int qemuProcessAttach(virConnectPtr conn ATTRIBUTE_UNUSED, * use in hotplug */ VIR_DEBUG("Assigning domain PCI addresses"); - if ((qemuDomainAssignAddresses(vm->def, priv->qemuCaps, vm, false)) < 0) + if ((qemuDomainAssignAddresses(vm->def, priv->qemuCaps, + driver, vm, false)) < 0) { goto error; + } if ((timestamp = virTimeStringNow()) == NULL) goto error; diff --git a/tests/qemuhotplugtest.c b/tests/qemuhotplugtest.c index 89e870c..f0a8453 100644 --- a/tests/qemuhotplugtest.c +++ b/tests/qemuhotplugtest.c @@ -90,8 +90,10 @@ qemuHotplugCreateObjects(virDomainXMLOptionPtr xmlopt, VIR_DOMAIN_DEF_PARSE_INACTIVE))) goto cleanup; - if (qemuDomainAssignAddresses((*vm)->def, priv->qemuCaps, *vm, true) < 0) + if (qemuDomainAssignAddresses((*vm)->def, priv->qemuCaps, + &driver, *vm, true) < 0) { goto cleanup; + } if (qemuAssignDeviceAliases((*vm)->def, priv->qemuCaps) < 0) goto cleanup; -- 2.7.4
Although nearly all host devices that are assigned to guests using vfio ("<hostdev>" devices in libvirt) are physically PCI Express devices, until now libvirt's PCI address assignment has always assigned them addresses on legacy PCI controllers. This patch tries to assign them to an address on a PCIe controller instead. First we do some preliminary checks that might allow setting the flags without doing any extra work, and if those conditions aren't met (and if libvirt is running privileged so that it has proper permissions), we perform the (relatively) time consuming task of reading the device's PCI config to see if it is an Express device. If this is successful, the connect flags are set based on the result, but if we aren't able to read the PCI config (most likely due to the device not being present on the system at the time of the check) we assume it is (or will be) and Express device, since that is almost always the case anyway. --- src/qemu/qemu_domain_address.c | 66 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 63 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 65753c5..86984fb 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -428,7 +428,7 @@ qemuDomainAssignARMVirtioMMIOAddresses(virDomainDefPtr def, */ static virDomainPCIConnectFlags qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, - virQEMUDriverPtr driver ATTRIBUTE_UNUSED, + virQEMUDriverPtr driver, virDomainPCIConnectFlags pcieFlags, virDomainPCIConnectFlags virtioFlags) { @@ -558,8 +558,68 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, return 0; } - case VIR_DOMAIN_DEVICE_HOSTDEV: - return pciFlags; + case VIR_DOMAIN_DEVICE_HOSTDEV: { + virDomainHostdevDefPtr hostdev = dev->data.hostdev; + + if (hostdev->mode == VIR_DOMAIN_HOSTDEV_MODE_SUBSYS && + hostdev->source.subsys.type == VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI) { + bool isExpress = false; + virPCIDevicePtr pciDev; + virPCIDeviceAddressPtr hostAddr = &hostdev->source.subsys.u.pci.addr; + + if (pciFlags == pcieFlags) { + /* This arch/qemu only supports legacy PCI, so there + * is no point in checking if the device is an Express + * device. + */ + return pciFlags; + } + + if (virDeviceInfoPCIAddressPresent(hostdev->info)) { + /* A guest-side address has already been assigned, so + * we can avoid reading the PCI config, and just use + * pcieFlags, since the pciConnectFlags checking is + * more relaxed when an address is already assigned + * than it is when we're looking for a new address (so + * validation will pass regardless of whether we set + * the flags to PCI or PCIE). + */ + return pcieFlags; + } + + if (!driver->privileged) { + /* unprivileged libvirtd is unable to read a device's + * PCI config, so instead of trying and failing, we + * will just assume what is by far the most likely + * version of reality: this is a PCIE device. + */ + return pcieFlags; + } + + if (!(pciDev = virPCIDeviceNew(hostAddr->domain, + hostAddr->bus, + hostAddr->slot, + hostAddr->function))) { + /* Even though libvirtd is running with privileges, we + * still couldn't read the PCI config. So either + * device doesn't currently exist on the host, or + * libvirt is running unprivileged. Since the + * overwhelming majority of assignable host devices + * are PCIe, assume that. + */ + return pcieFlags; + } + + isExpress = virPCIDeviceIsPCIExpress(pciDev); + virPCIDeviceFree(pciDev); + + if (isExpress) + return pcieFlags; + else + return pciFlags; + } + return 0; + } case VIR_DOMAIN_DEVICE_MEMBALLOON: switch ((virDomainMemballoonModel) dev->data.memballoon->model) { -- 2.7.4
On Mon, 7 Nov 2016 14:50:25 -0500 Laine Stump <laine@laine.org> wrote:
Although nearly all host devices that are assigned to guests using vfio ("<hostdev>" devices in libvirt) are physically PCI Express devices, until now libvirt's PCI address assignment has always assigned them addresses on legacy PCI controllers.
This patch tries to assign them to an address on a PCIe controller instead. First we do some preliminary checks that might allow setting the flags without doing any extra work, and if those conditions aren't met (and if libvirt is running privileged so that it has proper permissions), we perform the (relatively) time consuming task of reading the device's PCI config to see if it is an Express device. If this is successful, the connect flags are set based on the result, but if we aren't able to read the PCI config (most likely due to the device not being present on the system at the time of the check) we assume it is (or will be) and Express device, since that is almost always the case anyway. --- src/qemu/qemu_domain_address.c | 66 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 63 insertions(+), 3 deletions(-)
diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 65753c5..86984fb 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -428,7 +428,7 @@ qemuDomainAssignARMVirtioMMIOAddresses(virDomainDefPtr def, */ static virDomainPCIConnectFlags qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, - virQEMUDriverPtr driver ATTRIBUTE_UNUSED, + virQEMUDriverPtr driver, virDomainPCIConnectFlags pcieFlags, virDomainPCIConnectFlags virtioFlags) { @@ -558,8 +558,68 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, return 0; }
- case VIR_DOMAIN_DEVICE_HOSTDEV: - return pciFlags; + case VIR_DOMAIN_DEVICE_HOSTDEV: { + virDomainHostdevDefPtr hostdev = dev->data.hostdev; + + if (hostdev->mode == VIR_DOMAIN_HOSTDEV_MODE_SUBSYS && + hostdev->source.subsys.type == VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI) { + bool isExpress = false; + virPCIDevicePtr pciDev; + virPCIDeviceAddressPtr hostAddr = &hostdev->source.subsys.u.pci.addr; + + if (pciFlags == pcieFlags) { + /* This arch/qemu only supports legacy PCI, so there + * is no point in checking if the device is an Express + * device. + */ + return pciFlags; + } + + if (virDeviceInfoPCIAddressPresent(hostdev->info)) { + /* A guest-side address has already been assigned, so + * we can avoid reading the PCI config, and just use + * pcieFlags, since the pciConnectFlags checking is + * more relaxed when an address is already assigned + * than it is when we're looking for a new address (so + * validation will pass regardless of whether we set + * the flags to PCI or PCIE). + */ + return pcieFlags; + } + + if (!driver->privileged) { + /* unprivileged libvirtd is unable to read a device's + * PCI config, so instead of trying and failing, we + * will just assume what is by far the most likely + * version of reality: this is a PCIE device. + */ + return pcieFlags;
Don't you still have permission to stat the config file? You can tell pretty well by whether the config file is 4096 bytes that the device is either PCIe or PCIx (and given the rarity of PCIx in most systems assume that it's PCIe). Conventional PCI will be 256 bytes. Thanks, Alex
+ } + + if (!(pciDev = virPCIDeviceNew(hostAddr->domain, + hostAddr->bus, + hostAddr->slot, + hostAddr->function))) { + /* Even though libvirtd is running with privileges, we + * still couldn't read the PCI config. So either + * device doesn't currently exist on the host, or + * libvirt is running unprivileged. Since the + * overwhelming majority of assignable host devices + * are PCIe, assume that. + */ + return pcieFlags; + } + + isExpress = virPCIDeviceIsPCIExpress(pciDev); + virPCIDeviceFree(pciDev); + + if (isExpress) + return pcieFlags; + else + return pciFlags; + } + return 0; + }
case VIR_DOMAIN_DEVICE_MEMBALLOON: switch ((virDomainMemballoonModel) dev->data.memballoon->model) {
On 11/07/2016 03:26 PM, Alex Williamson wrote:
On Mon, 7 Nov 2016 14:50:25 -0500 Laine Stump <laine@laine.org> wrote:
Although nearly all host devices that are assigned to guests using vfio ("<hostdev>" devices in libvirt) are physically PCI Express devices, until now libvirt's PCI address assignment has always assigned them addresses on legacy PCI controllers.
This patch tries to assign them to an address on a PCIe controller instead. First we do some preliminary checks that might allow setting the flags without doing any extra work, and if those conditions aren't met (and if libvirt is running privileged so that it has proper permissions), we perform the (relatively) time consuming task of reading the device's PCI config to see if it is an Express device. If this is successful, the connect flags are set based on the result, but if we aren't able to read the PCI config (most likely due to the device not being present on the system at the time of the check) we assume it is (or will be) and Express device, since that is almost always the case anyway. --- src/qemu/qemu_domain_address.c | 66 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 63 insertions(+), 3 deletions(-)
diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 65753c5..86984fb 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -428,7 +428,7 @@ qemuDomainAssignARMVirtioMMIOAddresses(virDomainDefPtr def, */ static virDomainPCIConnectFlags qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, - virQEMUDriverPtr driver ATTRIBUTE_UNUSED, + virQEMUDriverPtr driver, virDomainPCIConnectFlags pcieFlags, virDomainPCIConnectFlags virtioFlags) { @@ -558,8 +558,68 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, return 0; }
- case VIR_DOMAIN_DEVICE_HOSTDEV: - return pciFlags; + case VIR_DOMAIN_DEVICE_HOSTDEV: { + virDomainHostdevDefPtr hostdev = dev->data.hostdev; + + if (hostdev->mode == VIR_DOMAIN_HOSTDEV_MODE_SUBSYS && + hostdev->source.subsys.type == VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI) { + bool isExpress = false; + virPCIDevicePtr pciDev; + virPCIDeviceAddressPtr hostAddr = &hostdev->source.subsys.u.pci.addr; + + if (pciFlags == pcieFlags) { + /* This arch/qemu only supports legacy PCI, so there + * is no point in checking if the device is an Express + * device. + */ + return pciFlags; + } + + if (virDeviceInfoPCIAddressPresent(hostdev->info)) { + /* A guest-side address has already been assigned, so + * we can avoid reading the PCI config, and just use + * pcieFlags, since the pciConnectFlags checking is + * more relaxed when an address is already assigned + * than it is when we're looking for a new address (so + * validation will pass regardless of whether we set + * the flags to PCI or PCIE). + */ + return pcieFlags; + } + + if (!driver->privileged) { + /* unprivileged libvirtd is unable to read a device's + * PCI config, so instead of trying and failing, we + * will just assume what is by far the most likely + * version of reality: this is a PCIE device. + */ + return pcieFlags;
Don't you still have permission to stat the config file? You can tell pretty well by whether the config file is 4096 bytes that the device is either PCIe or PCIx (and given the rarity of PCIx in most systems assume that it's PCIe). Conventional PCI will be 256 bytes. Thanks,
Interesting idea. Although still not 100%, it would reduce the possibility of false positives to "nearly 0".
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
This is a rebase of v5 (, with the addition of patches 16 & 17, which detect the type of VFIO assigned devices and assign them to either an Express or a legacy PCI slot as appropriate,
I'm afraid you're going to have to rebase again, hopefully it won't be too much work. I think that, seeing as all patches... [...]
[RFC] qemu: if pci-bridge is in PCIe config w/o dmi-to-pci-bridge, add it
... except for this one, that we've decided to drop... [...]
qemu: propagate virQEMUDriver object to qemuDomainDeviceCalculatePCIConnectFlags qemu: assign vfio devices to PCIe addresses when appropriate
... and these two, that AIUI will have to be tweaked anyway, have been ACKed with only trivial changes needed, the bulk of the series (1-15) can be pushed now. I would not hold up the entire series because of the last two patches as they're fairly self-contained, but up to you. Do you want me to look in detail at their current form (they looked okay from a quick look), or would you rather implement Alex's suggestions first and then make a separate mini-series out of them? It would also be awesome if you could try and update docs/news.html.in with a couple of sentences describing, in broad strokes, what this series changes for users, following the ideas outlined in last month's "Toward a better NEWS file" thread. You can post it as patch 18/17 and we can iterate over it, the whole thing is still very much a work in progress at this point anyway. -- Andrea Bolognani / Red Hat / Virtualization
On 11/14/2016 11:20 AM, Andrea Bolognani wrote:
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
This is a rebase of v5 (, with the addition of patches 16 & 17, which detect the type of VFIO assigned devices and assign them to either an Express or a legacy PCI slot as appropriate, I'm afraid you're going to have to rebase again, hopefully it won't be too much work.
I think that, seeing as all patches...
[...]
[RFC] qemu: if pci-bridge is in PCIe config w/o dmi-to-pci-bridge, add it
... except for this one, that we've decided to drop...
[...]
I made a separate patch to fix the test cases this affects. It will have to be pushed before 12-15 can be pushed. I just posted it as a reply to my last message on that sub-thread.
qemu: propagate virQEMUDriver object to qemuDomainDeviceCalculatePCIConnectFlags qemu: assign vfio devices to PCIe addresses when appropriate
... and these two, that AIUI will have to be tweaked anyway, have been ACKed with only trivial changes needed, the bulk of the series (1-15) can be pushed now.
I would not hold up the entire series because of the last two patches as they're fairly self-contained, but up to you. Do you want me to look in detail at their current form (they looked okay from a quick look), or would you rather implement Alex's suggestions first and then make a separate mini-series out of them?
You can skip them while I try to implement Alex's suggestion. They should be independent of the other two series that are pending.
It would also be awesome if you could try and update docs/news.html.in with a couple of sentences describing, in broad strokes, what this series changes for users, following the ideas outlined in last month's "Toward a better NEWS file" thread. You can post it as patch 18/17 and we can iterate over it, the whole thing is still very much a work in progress at this point anyway.
Yeah, I'll give that a look in a bit. Thanks!
On 11/14/2016 11:45 AM, Laine Stump wrote:
On 11/14/2016 11:20 AM, Andrea Bolognani wrote:
On Mon, 2016-11-07 at 14:50 -0500, Laine Stump wrote:
This is a rebase of v5 (, with the addition of patches 16 & 17, which detect the type of VFIO assigned devices and assign them to either an Express or a legacy PCI slot as appropriate, I'm afraid you're going to have to rebase again, hopefully it won't be too much work.
Nope. It was a single line in one patch.
I think that, seeing as all patches...
[...]
[RFC] qemu: if pci-bridge is in PCIe config w/o dmi-to-pci-bridge, add it
... except for this one, that we've decided to drop...
[...]
I made a separate patch to fix the test cases this affects. It will have to be pushed before 12-15 can be pushed. I just posted it as a reply to my last message on that sub-thread.
qemu: propagate virQEMUDriver object to qemuDomainDeviceCalculatePCIConnectFlags qemu: assign vfio devices to PCIe addresses when appropriate
... and these two, that AIUI will have to be tweaked anyway, have been ACKed with only trivial changes needed, the bulk of the series (1-15) can be pushed now.
Okay. I just pushed 1-15 (including the newly ACKed replacement for 11). Thanks for all the reviews!!
participants (3)
-
Alex Williamson -
Andrea Bolognani -
Laine Stump