[libvirt] [PATCH 0/6] Relax PCI domain limitation
As reported here [1], a PCI domain can be just any number. Well, not in case of traditional 32-bit PCI, bit in case of PCI-X (64-bit PCI; not to be confused with PCI Express), the PCI domain number can really be just anything. I wanted to download the specification from PCI SIG web but it's paid (why on earth would somebody create a standard and then make it not widely available is beyond me). 1: https://www.redhat.com/archives/libvir-list/2019-July/msg01732.html Michal Prívozník (6): qemuBuildPCIHostdevDevStr: Always format PCI domain onto cmd line virPCIDeviceNew: Prefer VIR_RETURN_PTR virPCIDevice: Make @name dynamically allocated lib: Unify PCI address formatting lib: Format PCI address differently virpci: Allow greater PCI domain value in virPCIDeviceAddressIsValid docs/schemas/basictypes.rng | 2 +- src/conf/device_conf.c | 4 +- src/conf/domain_addr.c | 10 ++-- src/conf/domain_audit.c | 3 +- src/conf/domain_conf.c | 12 ++--- src/conf/node_device_conf.c | 12 ++--- src/libxl/libxl_driver.c | 14 +++-- src/qemu/qemu_command.c | 9 ++-- src/qemu/qemu_hotplug.c | 5 +- src/util/virpci.c | 52 +++++++++---------- src/util/virpci.h | 2 + .../hostdev-pci-address-device.args | 2 +- .../qemuxml2argvdata/hostdev-pci-address.args | 2 +- .../hostdev-vfio-zpci-autogenerate.args | 2 +- .../hostdev-vfio-zpci-boundaries.args | 2 +- tests/qemuxml2argvdata/hostdev-vfio-zpci.args | 2 +- tests/qemuxml2argvdata/hostdev-vfio.args | 2 +- .../net-hostdev-bootorder.args | 3 +- tests/qemuxml2argvdata/net-hostdev-vfio.args | 2 +- tests/qemuxml2argvdata/net-hostdev.args | 2 +- tests/qemuxml2argvdata/pci-domain-invalid.xml | 2 +- tests/qemuxml2argvdata/pci-rom.args | 4 +- tools/virsh-domain.c | 4 +- 23 files changed, 81 insertions(+), 73 deletions(-) -- 2.21.0
QEMU is perfectly capable of dealing with fully expanded PCI address. Format PCI domain always as it will simplify next commits. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/qemu/qemu_command.c | 8 ++++---- tests/qemuxml2argvdata/hostdev-pci-address-device.args | 2 +- tests/qemuxml2argvdata/hostdev-pci-address.args | 2 +- .../qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args | 2 +- tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args | 2 +- tests/qemuxml2argvdata/hostdev-vfio-zpci.args | 2 +- tests/qemuxml2argvdata/hostdev-vfio.args | 2 +- tests/qemuxml2argvdata/net-hostdev-bootorder.args | 3 ++- tests/qemuxml2argvdata/net-hostdev-vfio.args | 2 +- tests/qemuxml2argvdata/net-hostdev.args | 2 +- tests/qemuxml2argvdata/pci-rom.args | 4 ++-- 11 files changed, 16 insertions(+), 15 deletions(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index fee51158a9..1871c6ef81 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -4933,10 +4933,10 @@ qemuBuildPCIHostdevDevStr(const virDomainDef *def, } virBufferAddLit(&buf, ",host="); - if (pcisrc->addr.domain) - virBufferAsprintf(&buf, "%.4x:", pcisrc->addr.domain); - virBufferAsprintf(&buf, "%.2x:%.2x.%.1x", - pcisrc->addr.bus, pcisrc->addr.slot, + virBufferAsprintf(&buf, "%.4x:%.2x:%.2x.%.1x", + pcisrc->addr.domain, + pcisrc->addr.bus, + pcisrc->addr.slot, pcisrc->addr.function); virBufferAsprintf(&buf, ",id=%s", dev->info->alias); if (dev->info->bootIndex) diff --git a/tests/qemuxml2argvdata/hostdev-pci-address-device.args b/tests/qemuxml2argvdata/hostdev-pci-address-device.args index 4763e1a147..5159b00ef1 100644 --- a/tests/qemuxml2argvdata/hostdev-pci-address-device.args +++ b/tests/qemuxml2argvdata/hostdev-pci-address-device.args @@ -27,5 +27,5 @@ server,nowait \ -usb \ -drive file=/dev/HostVG/QEMUGuest2,format=raw,if=none,id=drive-ide0-0-0 \ -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \ --device pci-assign,host=06:12.5,id=hostdev0,bus=pci.0,addr=0x3 \ +-device pci-assign,host=0000:06:12.5,id=hostdev0,bus=pci.0,addr=0x3 \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 diff --git a/tests/qemuxml2argvdata/hostdev-pci-address.args b/tests/qemuxml2argvdata/hostdev-pci-address.args index d90fb2c085..fe6acaaf62 100644 --- a/tests/qemuxml2argvdata/hostdev-pci-address.args +++ b/tests/qemuxml2argvdata/hostdev-pci-address.args @@ -27,4 +27,4 @@ server,nowait \ -usb \ -drive file=/dev/HostVG/QEMUGuest2,format=raw,if=none,id=drive-ide0-0-0 \ -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \ --device pci-assign,host=06:12.5,id=hostdev0,bus=pci.0,addr=0x3 +-device pci-assign,host=0000:06:12.5,id=hostdev0,bus=pci.0,addr=0x3 diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args b/tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args index 1861811e8b..963d508066 100644 --- a/tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate.args @@ -24,6 +24,6 @@ server,nowait \ -rtc base=utc \ -no-shutdown \ -device zpci,uid=1,fid=0,target=hostdev0,id=zpci1 \ --device vfio-pci,host=00:00.0,id=hostdev0,bus=pci.0,addr=0x1 \ +-device vfio-pci,host=0000:00:00.0,id=hostdev0,bus=pci.0,addr=0x1 \ -device zpci,uid=2,fid=1,target=balloon0,id=zpci2 \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2 diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args b/tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args index 4a58a1eb8d..2fc4200c23 100644 --- a/tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci-boundaries.args @@ -28,6 +28,6 @@ server,nowait \ -device zpci,uid=65535,fid=4294967295,target=hostdev0,id=zpci65535 \ -device vfio-pci,host=ffff:00:00.0,id=hostdev0,bus=pci.1,addr=0x1f \ -device zpci,uid=1,fid=0,target=hostdev1,id=zpci1 \ --device vfio-pci,host=00:00.0,id=hostdev1,bus=pci.0,addr=0x2 \ +-device vfio-pci,host=0000:00:00.0,id=hostdev1,bus=pci.0,addr=0x2 \ -device zpci,uid=2,fid=1,target=balloon0,id=zpci2 \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 diff --git a/tests/qemuxml2argvdata/hostdev-vfio-zpci.args b/tests/qemuxml2argvdata/hostdev-vfio-zpci.args index 07405c7c14..d3a762e5d4 100644 --- a/tests/qemuxml2argvdata/hostdev-vfio-zpci.args +++ b/tests/qemuxml2argvdata/hostdev-vfio-zpci.args @@ -24,6 +24,6 @@ server,nowait \ -rtc base=utc \ -no-shutdown \ -device zpci,uid=25,fid=31,target=hostdev0,id=zpci25 \ --device vfio-pci,host=00:00.0,id=hostdev0,bus=pci.0,addr=0x8 \ +-device vfio-pci,host=0000:00:00.0,id=hostdev0,bus=pci.0,addr=0x8 \ -device zpci,uid=1,fid=0,target=balloon0,id=zpci1 \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x1 diff --git a/tests/qemuxml2argvdata/hostdev-vfio.args b/tests/qemuxml2argvdata/hostdev-vfio.args index b86181e267..79654f44bb 100644 --- a/tests/qemuxml2argvdata/hostdev-vfio.args +++ b/tests/qemuxml2argvdata/hostdev-vfio.args @@ -27,5 +27,5 @@ server,nowait \ -usb \ -drive file=/dev/HostVG/QEMUGuest2,format=raw,if=none,id=drive-ide0-0-0 \ -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \ --device vfio-pci,host=06:12.5,id=hostdev0,bus=pci.0,addr=0x3 \ +-device vfio-pci,host=0000:06:12.5,id=hostdev0,bus=pci.0,addr=0x3 \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 diff --git a/tests/qemuxml2argvdata/net-hostdev-bootorder.args b/tests/qemuxml2argvdata/net-hostdev-bootorder.args index 700ed5366b..eefc247eed 100644 --- a/tests/qemuxml2argvdata/net-hostdev-bootorder.args +++ b/tests/qemuxml2argvdata/net-hostdev-bootorder.args @@ -27,4 +27,5 @@ server,nowait \ -usb \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-ide0-0-0 \ -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=2 \ --device pci-assign,host=03:07.1,id=hostdev0,bootindex=1,bus=pci.0,addr=0x3 +-device pci-assign,host=0000:03:07.1,id=hostdev0,bootindex=1,bus=pci.0,\ +addr=0x3 diff --git a/tests/qemuxml2argvdata/net-hostdev-vfio.args b/tests/qemuxml2argvdata/net-hostdev-vfio.args index fcc8716fdf..aa9e91db82 100644 --- a/tests/qemuxml2argvdata/net-hostdev-vfio.args +++ b/tests/qemuxml2argvdata/net-hostdev-vfio.args @@ -27,5 +27,5 @@ server,nowait \ -usb \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-ide0-0-0 \ -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \ --device vfio-pci,host=03:07.1,id=hostdev0,bus=pci.0,addr=0x3 \ +-device vfio-pci,host=0000:03:07.1,id=hostdev0,bus=pci.0,addr=0x3 \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 diff --git a/tests/qemuxml2argvdata/net-hostdev.args b/tests/qemuxml2argvdata/net-hostdev.args index 75f6934d0d..94eac176f3 100644 --- a/tests/qemuxml2argvdata/net-hostdev.args +++ b/tests/qemuxml2argvdata/net-hostdev.args @@ -27,5 +27,5 @@ server,nowait \ -usb \ -drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-ide0-0-0 \ -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \ --device pci-assign,host=03:07.1,id=hostdev0,bus=pci.0,addr=0x3 \ +-device pci-assign,host=0000:03:07.1,id=hostdev0,bus=pci.0,addr=0x3 \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 diff --git a/tests/qemuxml2argvdata/pci-rom.args b/tests/qemuxml2argvdata/pci-rom.args index 5f33b20b4e..7235642ee8 100644 --- a/tests/qemuxml2argvdata/pci-rom.args +++ b/tests/qemuxml2argvdata/pci-rom.args @@ -33,7 +33,7 @@ addr=0x3,rombar=1 \ -netdev user,id=hostnet1 \ -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:24:a5:9e,bus=pci.0,\ addr=0x4,romfile=/etc/fake/bootrom.bin \ --device pci-assign,host=06:12.5,id=hostdev0,bus=pci.0,addr=0x5,rombar=0 \ --device pci-assign,host=06:12.6,id=hostdev1,bus=pci.0,addr=0x6,rombar=1,\ +-device pci-assign,host=0000:06:12.5,id=hostdev0,bus=pci.0,addr=0x5,rombar=0 \ +-device pci-assign,host=0000:06:12.6,id=hostdev1,bus=pci.0,addr=0x6,rombar=1,\ romfile=/etc/fake/bootrom.bin \ -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -- 2.21.0
On Wed, Jul 31, 2019 at 10:32:19AM +0200, Michal Privoznik wrote:
QEMU is perfectly capable of dealing with fully expanded PCI address. Format PCI domain always as it will simplify next commits.
Are you sure about that for old QEMUs ? I have a feeling that it did not support that in the past. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Wed, Jul 31, 2019 at 09:36:47AM +0100, Daniel P. Berrangé wrote:
On Wed, Jul 31, 2019 at 10:32:19AM +0200, Michal Privoznik wrote:
QEMU is perfectly capable of dealing with fully expanded PCI address. Format PCI domain always as it will simplify next commits.
Are you sure about that for old QEMUs ? I have a feeling that it did not support that in the past.
commit 4a42ece13a53810410567323dea1719d63877e1f Author: Ján Tomko <jtomko@redhat.com> AuthorDate: 2018-03-29 12:51:55 +0200 Commit: Ján Tomko <jtomko@redhat.com> CommitDate: 2018-04-17 14:40:38 +0200 qemu: deprecate QEMU_CAPS_HOST_PCI_MULTIDOMAIN Implied by QEMU >= 1.2.0. Jano
On 7/31/19 10:36 AM, Daniel P. Berrangé wrote:
On Wed, Jul 31, 2019 at 10:32:19AM +0200, Michal Privoznik wrote:
QEMU is perfectly capable of dealing with fully expanded PCI address. Format PCI domain always as it will simplify next commits.
Are you sure about that for old QEMUs ? I have a feeling that it did not support that in the past.
It's not only the commit that Jano referred to, but also the commit that introduced PCI address parsing in QEMU supports this from the very beginning v1.2.0-rc0~242^2~4. Therefore I think we can safely assume all QEMUs we care about are capable of dealing with PCI domain. But let me record this in commit message to avoid confusion. The new commit mesage now reads: qemuBuildPCIHostdevDevStr: Always format PCI domain onto cmd line While it's true that older QEMUs were not able to deal with PCI domains, we don't support those versions anymore (see 4a42ece13a538). Therefore it is safe to always format fully expanded PCI address. Format PCI domain always as it will simplify next commits. Michal
This function declares @ret variable and then uses VIR_STEAL_PTR() to avoid freeing temporary variable @dev which is constructed. Well, as of 267f1e6da53 we have VIR_RETURN_PTR() macro so that we can avoid this pattern. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/util/virpci.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/src/util/virpci.c b/src/util/virpci.c index 6dc0a2711c..1a3d316399 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -1749,7 +1749,6 @@ virPCIDeviceNew(unsigned int domain, unsigned int slot, unsigned int function) { - virPCIDevicePtr ret = NULL; VIR_AUTOPTR(virPCIDevice) dev = NULL; VIR_AUTOFREE(char *) vendor = NULL; VIR_AUTOFREE(char *) product = NULL; @@ -1767,17 +1766,17 @@ virPCIDeviceNew(unsigned int domain, virReportError(VIR_ERR_INTERNAL_ERROR, _("dev->name buffer overflow: %.4x:%.2x:%.2x.%.1x"), domain, bus, slot, function); - goto cleanup; + return NULL; } if (virAsprintf(&dev->path, PCI_SYSFS "devices/%s/config", dev->name) < 0) - goto cleanup; + return NULL; if (!virFileExists(dev->path)) { virReportSystemError(errno, _("Device %s not found: could not access %s"), dev->name, dev->path); - goto cleanup; + return NULL; } vendor = virPCIDeviceReadID(dev, "vendor"); @@ -1787,7 +1786,7 @@ virPCIDeviceNew(unsigned int domain, virReportError(VIR_ERR_INTERNAL_ERROR, _("Failed to read product/vendor ID for %s"), dev->name); - goto cleanup; + return NULL; } /* strings contain '0x' prefix */ @@ -1796,15 +1795,12 @@ virPCIDeviceNew(unsigned int domain, virReportError(VIR_ERR_INTERNAL_ERROR, _("dev->id buffer overflow: %s %s"), &vendor[2], &product[2]); - goto cleanup; + return NULL; } VIR_DEBUG("%s %s: initialized", dev->id, dev->name); - VIR_STEAL_PTR(ret, dev); - - cleanup: - return ret; + VIR_RETURN_PTR(dev); } -- 2.21.0
In near future, the length restriction of PCI domain is going to be lifted. This means that our assumption that PCI address is 13 bytes long is no longer true. We can avoid this problem by making @name dynamically allocated and thus not bother with actual length of stringified PCI address. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/util/virpci.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/src/util/virpci.c b/src/util/virpci.c index 1a3d316399..6d8ddc3577 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -44,7 +44,6 @@ VIR_LOG_INIT("util.pci"); #define PCI_SYSFS "/sys/bus/pci/" #define PCI_ID_LEN 10 /* "XXXX XXXX" */ -#define PCI_ADDR_LEN 13 /* "XXXX:XX:XX.X" */ VIR_ENUM_IMPL(virPCIELinkSpeed, VIR_PCIE_LINK_SPEED_LAST, @@ -69,7 +68,7 @@ VIR_ENUM_IMPL(virPCIHeader, struct _virPCIDevice { virPCIDeviceAddress address; - char name[PCI_ADDR_LEN]; /* domain:bus:slot.function */ + char *name; /* domain:bus:slot.function */ char id[PCI_ID_LEN]; /* product vendor */ char *path; @@ -1761,13 +1760,11 @@ virPCIDeviceNew(unsigned int domain, dev->address.slot = slot; dev->address.function = function; - if (snprintf(dev->name, sizeof(dev->name), "%.4x:%.2x:%.2x.%.1x", - domain, bus, slot, function) >= sizeof(dev->name)) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("dev->name buffer overflow: %.4x:%.2x:%.2x.%.1x"), - domain, bus, slot, function); + if (virAsprintf(&dev->name, + "%.4x:%.2x:%.2x.%.1x", + domain, bus, slot, function) < 0) return NULL; - } + if (virAsprintf(&dev->path, PCI_SYSFS "devices/%s/config", dev->name) < 0) return NULL; @@ -1816,7 +1813,8 @@ virPCIDeviceCopy(virPCIDevicePtr dev) *copy = *dev; copy->path = NULL; copy->used_by_drvname = copy->used_by_domname = NULL; - if (VIR_STRDUP(copy->path, dev->path) < 0 || + if (VIR_STRDUP(copy->name, dev->name) < 0 || + VIR_STRDUP(copy->path, dev->path) < 0 || VIR_STRDUP(copy->used_by_drvname, dev->used_by_drvname) < 0 || VIR_STRDUP(copy->used_by_domname, dev->used_by_domname) < 0) { goto error; @@ -1835,6 +1833,7 @@ virPCIDeviceFree(virPCIDevicePtr dev) if (!dev) return; VIR_DEBUG("%s %s: freeing", dev->id, dev->name); + VIR_FREE(dev->name); VIR_FREE(dev->path); VIR_FREE(dev->used_by_drvname); VIR_FREE(dev->used_by_domname); -- 2.21.0
The format string for a PCI address is copied over and over again, often with slight adjustments. Introduce global VIR_PCI_DEVICE_ADDRESS_FMT macro that holds the formatting string and use it wherever possible. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/conf/domain_audit.c | 3 ++- src/conf/domain_conf.c | 8 ++++---- src/libxl/libxl_driver.c | 14 +++++++++----- src/qemu/qemu_command.c | 3 ++- src/qemu/qemu_hotplug.c | 5 +++-- src/util/virpci.c | 19 +++++++++++-------- src/util/virpci.h | 2 ++ 7 files changed, 33 insertions(+), 21 deletions(-) diff --git a/src/conf/domain_audit.c b/src/conf/domain_audit.c index de7386d4c6..cc5a55a635 100644 --- a/src/conf/domain_audit.c +++ b/src/conf/domain_audit.c @@ -376,7 +376,8 @@ virDomainAuditHostdev(virDomainObjPtr vm, virDomainHostdevDefPtr hostdev, case VIR_DOMAIN_HOSTDEV_MODE_SUBSYS: switch ((virDomainHostdevSubsysType) hostdev->source.subsys.type) { case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI: - if (virAsprintfQuiet(&address, "%.4x:%.2x:%.2x.%.1x", + if (virAsprintfQuiet(&address, + VIR_PCI_DEVICE_ADDRESS_FMT, pcisrc->addr.domain, pcisrc->addr.bus, pcisrc->addr.slot, diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 441eb1a5a2..b5ab2bca93 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -16992,7 +16992,7 @@ virDomainNetFindIdx(virDomainDefPtr def, virDomainNetDefPtr net) if (MACAddrSpecified && PCIAddrSpecified) { virReportError(VIR_ERR_DEVICE_MISSING, _("no device matching MAC address %s found on " - "%.4x:%.2x:%.2x.%.1x"), + VIR_PCI_DEVICE_ADDRESS_FMT), virMacAddrFormat(&net->mac, mac), net->info.addr.pci.domain, net->info.addr.pci.bus, @@ -17000,7 +17000,7 @@ virDomainNetFindIdx(virDomainDefPtr def, virDomainNetDefPtr net) net->info.addr.pci.function); } else if (PCIAddrSpecified) { virReportError(VIR_ERR_DEVICE_MISSING, - _("no device found on %.4x:%.2x:%.2x.%.1x"), + _("no device found on " VIR_PCI_DEVICE_ADDRESS_FMT), net->info.addr.pci.domain, net->info.addr.pci.bus, net->info.addr.pci.slot, @@ -21620,8 +21620,8 @@ virDomainDeviceInfoCheckABIStability(virDomainDeviceInfoPtr src, src->addr.pci.slot != dst->addr.pci.slot || src->addr.pci.function != dst->addr.pci.function) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("Target device PCI address %04x:%02x:%02x.%02x " - "does not match source %04x:%02x:%02x.%02x"), + _("Target device PCI address " VIR_PCI_DEVICE_ADDRESS_FMT + "does not match source " VIR_PCI_DEVICE_ADDRESS_FMT), dst->addr.pci.domain, dst->addr.pci.bus, dst->addr.pci.slot, dst->addr.pci.function, src->addr.pci.domain, src->addr.pci.bus, diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index 492028c487..48c23b43eb 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -3113,7 +3113,8 @@ libxlDomainAttachHostPCIDevice(libxlDriverPrivatePtr driver, if (virDomainHostdevFind(vm->def, hostdev, &found) >= 0) { virReportError(VIR_ERR_OPERATION_FAILED, - _("target pci device %.4x:%.2x:%.2x.%.1x already exists"), + _("target pci device " VIR_PCI_DEVICE_ADDRESS_FMT + " already exists"), pcisrc->addr.domain, pcisrc->addr.bus, pcisrc->addr.slot, pcisrc->addr.function); goto cleanup; @@ -3132,7 +3133,8 @@ libxlDomainAttachHostPCIDevice(libxlDriverPrivatePtr driver, if (libxl_device_pci_add(cfg->ctx, vm->def->id, &pcidev, 0) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, - _("libxenlight failed to attach pci device %.4x:%.2x:%.2x.%.1x"), + _("libxenlight failed to attach pci device " + VIR_PCI_DEVICE_ADDRESS_FMT), pcisrc->addr.domain, pcisrc->addr.bus, pcisrc->addr.slot, pcisrc->addr.function); goto error; @@ -3647,7 +3649,8 @@ libxlDomainDetachHostPCIDevice(libxlDriverPrivatePtr driver, idx = virDomainHostdevFind(vm->def, hostdev, &detach); if (idx < 0) { virReportError(VIR_ERR_OPERATION_FAILED, - _("host pci device %.4x:%.2x:%.2x.%.1x not found"), + _("host pci device " VIR_PCI_DEVICE_ADDRESS_FMT + " not found"), pcisrc->addr.domain, pcisrc->addr.bus, pcisrc->addr.slot, pcisrc->addr.function); goto cleanup; @@ -3655,7 +3658,8 @@ libxlDomainDetachHostPCIDevice(libxlDriverPrivatePtr driver, if (libxlIsMultiFunctionDevice(vm->def, detach->info)) { virReportError(VIR_ERR_OPERATION_FAILED, - _("cannot hot unplug multifunction PCI device: %.4x:%.2x:%.2x.%.1x"), + _("cannot hot unplug multifunction PCI device: " + VIR_PCI_DEVICE_ADDRESS_FMT), pcisrc->addr.domain, pcisrc->addr.bus, pcisrc->addr.slot, pcisrc->addr.function); goto error; @@ -3668,7 +3672,7 @@ libxlDomainDetachHostPCIDevice(libxlDriverPrivatePtr driver, if (libxl_device_pci_remove(cfg->ctx, vm->def->id, &pcidev, 0) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("libxenlight failed to detach pci device " - "%.4x:%.2x:%.2x.%.1x"), + VIR_PCI_DEVICE_ADDRESS_FMT), pcisrc->addr.domain, pcisrc->addr.bus, pcisrc->addr.slot, pcisrc->addr.function); goto error; diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 1871c6ef81..c8494de785 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -4933,7 +4933,8 @@ qemuBuildPCIHostdevDevStr(const virDomainDef *def, } virBufferAddLit(&buf, ",host="); - virBufferAsprintf(&buf, "%.4x:%.2x:%.2x.%.1x", + virBufferAsprintf(&buf, + VIR_PCI_DEVICE_ADDRESS_FMT, pcisrc->addr.domain, pcisrc->addr.bus, pcisrc->addr.slot, diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 7501ae5029..d8be63b71c 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -5388,7 +5388,8 @@ qemuDomainDetachPrepHostdev(virDomainObjPtr vm, switch (subsys->type) { case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI: virReportError(VIR_ERR_DEVICE_MISSING, - _("host pci device %.4x:%.2x:%.2x.%.1x not found"), + _("host pci device " VIR_PCI_DEVICE_ADDRESS_FMT + " not found"), pcisrc->addr.domain, pcisrc->addr.bus, pcisrc->addr.slot, pcisrc->addr.function); break; @@ -5874,7 +5875,7 @@ qemuDomainDetachDeviceLive(virDomainObjPtr vm, if (qemuIsMultiFunctionDevice(vm->def, info)) { virReportError(VIR_ERR_OPERATION_FAILED, _("cannot hot unplug %s device with multifunction PCI guest address: " - "%.4x:%.2x:%.2x.%.1x"), + VIR_PCI_DEVICE_ADDRESS_FMT), virDomainDeviceTypeToString(detach.type), info->addr.pci.domain, info->addr.pci.bus, info->addr.pci.slot, info->addr.pci.function); diff --git a/src/util/virpci.c b/src/util/virpci.c index 6d8ddc3577..36b7f8b424 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -1734,7 +1734,8 @@ virPCIDeviceAddressAsString(virPCIDeviceAddressPtr addr) { char *str; - ignore_value(virAsprintf(&str, "%.4x:%.2x:%.2x.%.1x", + ignore_value(virAsprintf(&str, + VIR_PCI_DEVICE_ADDRESS_FMT, addr->domain, addr->bus, addr->slot, @@ -1761,7 +1762,7 @@ virPCIDeviceNew(unsigned int domain, dev->address.function = function; if (virAsprintf(&dev->name, - "%.4x:%.2x:%.2x.%.1x", + VIR_PCI_DEVICE_ADDRESS_FMT, domain, bus, slot, function) < 0) return NULL; @@ -2120,7 +2121,7 @@ int virPCIDeviceFileIterate(virPCIDevicePtr dev, struct dirent *ent; int direrr; - if (virAsprintf(&pcidir, "/sys/bus/pci/devices/%04x:%02x:%02x.%x", + if (virAsprintf(&pcidir, "/sys/bus/pci/devices/" VIR_PCI_DEVICE_ADDRESS_FMT, dev->address.domain, dev->address.bus, dev->address.slot, dev->address.function) < 0) goto cleanup; @@ -2174,7 +2175,7 @@ virPCIDeviceAddressIOMMUGroupIterate(virPCIDeviceAddressPtr orig, int direrr; if (virAsprintf(&groupPath, - PCI_SYSFS "devices/%04x:%02x:%02x.%x/iommu_group/devices", + PCI_SYSFS "devices/" VIR_PCI_DEVICE_ADDRESS_FMT "/iommu_group/devices", orig->domain, orig->bus, orig->slot, orig->function) < 0) goto cleanup; @@ -2323,8 +2324,9 @@ virPCIDeviceAddressGetIOMMUGroupNum(virPCIDeviceAddressPtr addr) const char *groupNumStr; unsigned int groupNum; - if (virAsprintf(&devName, "%.4x:%.2x:%.2x.%.1x", addr->domain, - addr->bus, addr->slot, addr->function) < 0) + if (virAsprintf(&devName, + VIR_PCI_DEVICE_ADDRESS_FMT, + addr->domain, addr->bus, addr->slot, addr->function) < 0) return -1; if (!(devPath = virPCIFile(devName, "iommu_group"))) @@ -2648,7 +2650,8 @@ virPCIGetPhysicalFunction(const char *vf_sysfs_path, } if ((*pf = virPCIGetDeviceAddressFromSysfsLink(device_link))) { - VIR_DEBUG("PF for VF device '%s': %.4x:%.2x:%.2x.%.1x", vf_sysfs_path, + VIR_DEBUG("PF for VF device '%s': " VIR_PCI_DEVICE_ADDRESS_FMT, + vf_sysfs_path, (*pf)->domain, (*pf)->bus, (*pf)->slot, (*pf)->function); } @@ -2806,7 +2809,7 @@ virPCIDeviceAddressGetSysfsFile(virPCIDeviceAddressPtr addr, char **pci_sysfs_device_link) { if (virAsprintf(pci_sysfs_device_link, - PCI_SYSFS "devices/%04x:%02x:%02x.%x", + PCI_SYSFS "devices/" VIR_PCI_DEVICE_ADDRESS_FMT, addr->domain, addr->bus, addr->slot, addr->function) < 0) return -1; diff --git a/src/util/virpci.h b/src/util/virpci.h index 457be3c929..c13e7cd407 100644 --- a/src/util/virpci.h +++ b/src/util/virpci.h @@ -44,6 +44,8 @@ struct _virZPCIDeviceAddress { unsigned int fid; }; +#define VIR_PCI_DEVICE_ADDRESS_FMT "%.4x:%.2x:%.2x.%.1x" + struct _virPCIDeviceAddress { unsigned int domain; unsigned int bus; -- 2.21.0
Currently, the way we format PCI address is using printf-s precision, e.g. "%.4x". This works if we don't want to print any value outside of bounds (which is usually the case). However, turns out, PCI domain can be 0x10000 which doesn't work well with our format strings. However, if we change the format string to "%04x" then we still pad small values with zeroes but also we are able to print values that are larger than four digits. In fact, this format string used by kernel to print a PCI address: "%04x:%02x:%02x.%d" The other three format strings (for bus, device and function) are changed too, so that we use the same format string as kernel. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/conf/device_conf.c | 4 ++-- src/conf/domain_addr.c | 10 +++++----- src/conf/domain_conf.c | 4 ++-- src/conf/node_device_conf.c | 12 ++++++------ src/util/virpci.h | 2 +- tools/virsh-domain.c | 4 ++-- 6 files changed, 18 insertions(+), 18 deletions(-) diff --git a/src/conf/device_conf.c b/src/conf/device_conf.c index b28f30dfc5..4c57f0995f 100644 --- a/src/conf/device_conf.c +++ b/src/conf/device_conf.c @@ -293,8 +293,8 @@ virPCIDeviceAddressFormat(virBufferPtr buf, virPCIDeviceAddress addr, bool includeTypeInAddr) { - virBufferAsprintf(buf, "<address %sdomain='0x%.4x' bus='0x%.2x' " - "slot='0x%.2x' function='0x%.1x'/>\n", + virBufferAsprintf(buf, "<address %sdomain='0x%04x' bus='0x%02x' " + "slot='0x%02x' function='0x%d'/>\n", includeTypeInAddr ? "type='pci' " : "", addr.domain, addr.bus, diff --git a/src/conf/domain_addr.c b/src/conf/domain_addr.c index 04c4e6d7e1..4e02c43375 100644 --- a/src/conf/domain_addr.c +++ b/src/conf/domain_addr.c @@ -868,7 +868,7 @@ virDomainPCIAddressReserveAddrInternal(virDomainPCIAddressSetPtr addrs, /* The first device decides the isolation group for the * entire bus */ bus->isolationGroup = isolationGroup; - VIR_DEBUG("PCI bus %.4x:%.2x assigned isolation group %u because of " + VIR_DEBUG("PCI bus %04x:%02x assigned isolation group %u because of " "first device %s", addr->domain, addr->bus, isolationGroup, addrStr); } else if (bus->isolationGroup != isolationGroup && fromConfig) { @@ -879,7 +879,7 @@ virDomainPCIAddressReserveAddrInternal(virDomainPCIAddressSetPtr addrs, * back to the default (because at that point isolation can't * be guaranteed anymore) */ bus->isolationGroup = 0; - VIR_DEBUG("PCI bus %.4x:%.2x assigned isolation group %u because of " + VIR_DEBUG("PCI bus %04x:%02x assigned isolation group %u because of " "user assigned address %s", addr->domain, addr->bus, isolationGroup, addrStr); } @@ -1114,7 +1114,7 @@ virDomainPCIAddressFindUnusedFunctionOnBus(virDomainPCIAddressBusPtr bus, if (!virDomainPCIAddressFlagsCompatible(searchAddr, addrStr, bus->flags, flags, false, false)) { - VIR_DEBUG("PCI bus %.4x:%.2x is not compatible with the device", + VIR_DEBUG("PCI bus %04x:%02x is not compatible with the device", searchAddr->domain, searchAddr->bus); } else { while (searchAddr->slot <= bus->maxSlot) { @@ -1150,7 +1150,7 @@ virDomainPCIAddressFindUnusedFunctionOnBus(virDomainPCIAddressBusPtr bus, } } - VIR_DEBUG("PCI slot %.4x:%.2x:%.2x already in use", + VIR_DEBUG("PCI slot %04x:%02x:%02x already in use", searchAddr->domain, searchAddr->bus, searchAddr->slot); searchAddr->slot++; } @@ -1248,7 +1248,7 @@ virDomainPCIAddressGetNextAddr(virDomainPCIAddressSetPtr addrs, return -1; success: - VIR_DEBUG("Found free PCI slot %.4x:%.2x:%.2x", + VIR_DEBUG("Found free PCI slot %04x:%02x:%02x", a.domain, a.bus, a.slot); *next_addr = a; return 0; diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index b5ab2bca93..aadb8a9162 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -7091,8 +7091,8 @@ virDomainDeviceInfoFormat(virBufferPtr buf, switch ((virDomainDeviceAddressType) info->type) { case VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI: if (!virPCIDeviceAddressIsEmpty(&info->addr.pci)) { - virBufferAsprintf(&attrBuf, " domain='0x%.4x' bus='0x%.2x' " - "slot='0x%.2x' function='0x%.1x'", + virBufferAsprintf(&attrBuf, " domain='0x%04x' bus='0x%02x' " + "slot='0x%02x' function='0x%d'", info->addr.pci.domain, info->addr.pci.bus, info->addr.pci.slot, diff --git a/src/conf/node_device_conf.c b/src/conf/node_device_conf.c index 4ef92d5129..e51371de89 100644 --- a/src/conf/node_device_conf.c +++ b/src/conf/node_device_conf.c @@ -239,8 +239,8 @@ virNodeDeviceCapPCIDefFormat(virBufferPtr buf, virBufferAddLit(buf, "<capability type='phys_function'>\n"); virBufferAdjustIndent(buf, 2); virBufferAsprintf(buf, - "<address domain='0x%.4x' bus='0x%.2x' " - "slot='0x%.2x' function='0x%.1x'/>\n", + "<address domain='0x%04x' bus='0x%02x' " + "slot='0x%02x' function='0x%d'/>\n", data->pci_dev.physical_function->domain, data->pci_dev.physical_function->bus, data->pci_dev.physical_function->slot, @@ -260,8 +260,8 @@ virNodeDeviceCapPCIDefFormat(virBufferPtr buf, virBufferAdjustIndent(buf, 2); for (i = 0; i < data->pci_dev.num_virtual_functions; i++) { virBufferAsprintf(buf, - "<address domain='0x%.4x' bus='0x%.2x' " - "slot='0x%.2x' function='0x%.1x'/>\n", + "<address domain='0x%04x' bus='0x%02x' " + "slot='0x%02x' function='0x%d'/>\n", data->pci_dev.virtual_functions[i]->domain, data->pci_dev.virtual_functions[i]->bus, data->pci_dev.virtual_functions[i]->slot, @@ -302,8 +302,8 @@ virNodeDeviceCapPCIDefFormat(virBufferPtr buf, virBufferAdjustIndent(buf, 2); for (i = 0; i < data->pci_dev.nIommuGroupDevices; i++) { virBufferAsprintf(buf, - "<address domain='0x%.4x' bus='0x%.2x' " - "slot='0x%.2x' function='0x%.1x'/>\n", + "<address domain='0x%04x' bus='0x%02x' " + "slot='0x%02x' function='0x%d'/>\n", data->pci_dev.iommuGroupDevices[i]->domain, data->pci_dev.iommuGroupDevices[i]->bus, data->pci_dev.iommuGroupDevices[i]->slot, diff --git a/src/util/virpci.h b/src/util/virpci.h index c13e7cd407..e333146ac0 100644 --- a/src/util/virpci.h +++ b/src/util/virpci.h @@ -44,7 +44,7 @@ struct _virZPCIDeviceAddress { unsigned int fid; }; -#define VIR_PCI_DEVICE_ADDRESS_FMT "%.4x:%.2x:%.2x.%.1x" +#define VIR_PCI_DEVICE_ADDRESS_FMT "%04x:%02x:%02x.%d" struct _virPCIDeviceAddress { unsigned int domain; diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index e79dc75342..ccda71d7e0 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -1006,8 +1006,8 @@ cmdAttachInterface(vshControl *ctl, const vshCmd *cmd) virBufferAddLit(&buf, "<source>\n"); virBufferAdjustIndent(&buf, 2); - virBufferAsprintf(&buf, "<address type='pci' domain='0x%.4x'" - " bus='0x%.2x' slot='0x%.2x' function='0x%.1x'/>\n", + virBufferAsprintf(&buf, "<address type='pci' domain='0x%04x'" + " bus='0x%02x' slot='0x%02x' function='0x%d'/>\n", pciAddr.domain, pciAddr.bus, pciAddr.slot, pciAddr.function); virBufferAdjustIndent(&buf, -2); -- 2.21.0
There is no restriction on maximum value of PCI domain. In fact, Linux kernel uses plain atomic inc when assigning PCI domains: drivers/pci/pci.c:static int pci_get_new_domain_nr(void) drivers/pci/pci.c-{ drivers/pci/pci.c- return atomic_inc_return(&__domain_nr); drivers/pci/pci.c-} Of course, this function is called only if kernel was compiled without PCI domain support or ACPI did not provide PCI domain. However, QEMU still has the same restriction as us: in set_pci_host_devaddr() QEMU checks if domain isn't greater than 0xffff. But one can argue that that's a QEMU limitation. We still want to be able to cope with other hypervisors that don't have this limitation (possibly). Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- docs/schemas/basictypes.rng | 2 +- src/util/virpci.c | 2 +- tests/qemuxml2argvdata/pci-domain-invalid.xml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/schemas/basictypes.rng b/docs/schemas/basictypes.rng index 70d2101b78..81465273c8 100644 --- a/docs/schemas/basictypes.rng +++ b/docs/schemas/basictypes.rng @@ -324,7 +324,7 @@ </define> <define name="pciDomain"> - <ref name="uint16"/> + <ref name="uint32"/> </define> <define name="pciBus"> <ref name="uint8"/> diff --git a/src/util/virpci.c b/src/util/virpci.c index 36b7f8b424..bc7ff46194 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -1668,7 +1668,7 @@ bool virPCIDeviceAddressIsValid(virPCIDeviceAddressPtr addr, bool report) { - if (addr->domain > 0xFFFF) { + if (addr->domain > 0xFFFFFFFF) { if (report) virReportError(VIR_ERR_XML_ERROR, _("Invalid PCI address domain='0x%x', " diff --git a/tests/qemuxml2argvdata/pci-domain-invalid.xml b/tests/qemuxml2argvdata/pci-domain-invalid.xml index 1ac56fc703..21f1dc98af 100644 --- a/tests/qemuxml2argvdata/pci-domain-invalid.xml +++ b/tests/qemuxml2argvdata/pci-domain-invalid.xml @@ -26,7 +26,7 @@ <interface type='user'> <mac address='00:11:22:33:44:55'/> <model type='rtl8139'/> - <address type='pci' domain='0x10000' bus='0x00' slot='0x05' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x100' slot='0x05' function='0x0'/> </interface> <memballoon model='none'/> </devices> -- 2.21.0
On Wed, Jul 31, 2019 at 10:32:24AM +0200, Michal Privoznik wrote:
There is no restriction on maximum value of PCI domain. In fact, Linux kernel uses plain atomic inc when assigning PCI domains:
drivers/pci/pci.c:static int pci_get_new_domain_nr(void) drivers/pci/pci.c-{ drivers/pci/pci.c- return atomic_inc_return(&__domain_nr); drivers/pci/pci.c-}
Of course, this function is called only if kernel was compiled without PCI domain support or ACPI did not provide PCI domain.
However, QEMU still has the same restriction as us: in set_pci_host_devaddr() QEMU checks if domain isn't greater than 0xffff. But one can argue that that's a QEMU limitation. We still want to be able to cope with other hypervisors that don't have this limitation (possibly).
I would argue that by lifting this check we fail to report the error early for QEMU that (still) lacks the support for it. But I doubt that the QEMU fix will be detectable in our capability probing code, so:
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- docs/schemas/basictypes.rng | 2 +- src/util/virpci.c | 2 +- tests/qemuxml2argvdata/pci-domain-invalid.xml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano
On Wed, Jul 31, 2019 at 10:32:18AM +0200, Michal Privoznik wrote:
As reported here [1], a PCI domain can be just any number. Well, not in case of traditional 32-bit PCI, bit in case of PCI-X (64-bit PCI; not to be confused with PCI Express), the PCI domain number can really be just anything. I wanted to download the specification from PCI SIG web but it's paid (why on earth would somebody create a standard and then make it not widely available is beyond me).
1: https://www.redhat.com/archives/libvir-list/2019-July/msg01732.html
Michal Prívozník (6): qemuBuildPCIHostdevDevStr: Always format PCI domain onto cmd line virPCIDeviceNew: Prefer VIR_RETURN_PTR virPCIDevice: Make @name dynamically allocated lib: Unify PCI address formatting lib: Format PCI address differently virpci: Allow greater PCI domain value in virPCIDeviceAddressIsValid
docs/schemas/basictypes.rng | 2 +- src/conf/device_conf.c | 4 +- src/conf/domain_addr.c | 10 ++-- src/conf/domain_audit.c | 3 +- src/conf/domain_conf.c | 12 ++--- src/conf/node_device_conf.c | 12 ++--- src/libxl/libxl_driver.c | 14 +++-- src/qemu/qemu_command.c | 9 ++-- src/qemu/qemu_hotplug.c | 5 +- src/util/virpci.c | 52 +++++++++---------- src/util/virpci.h | 2 + .../hostdev-pci-address-device.args | 2 +- .../qemuxml2argvdata/hostdev-pci-address.args | 2 +- .../hostdev-vfio-zpci-autogenerate.args | 2 +- .../hostdev-vfio-zpci-boundaries.args | 2 +- tests/qemuxml2argvdata/hostdev-vfio-zpci.args | 2 +- tests/qemuxml2argvdata/hostdev-vfio.args | 2 +- .../net-hostdev-bootorder.args | 3 +- tests/qemuxml2argvdata/net-hostdev-vfio.args | 2 +- tests/qemuxml2argvdata/net-hostdev.args | 2 +- tests/qemuxml2argvdata/pci-domain-invalid.xml | 2 +- tests/qemuxml2argvdata/pci-rom.args | 4 +- tools/virsh-domain.c | 4 +- 23 files changed, 81 insertions(+), 73 deletions(-)
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano
participants (3)
-
Daniel P. Berrangé -
Ján Tomko -
Michal Privoznik