1:09 p.m.
v2:
- rebased without 3 patches from Michal that are under review
in his NVMe patch series
This is the first part of the feature discussed at [1]. These
patches are mostly cleanup and fixes, thus it is beneficial
to have them upstream right away.
The whole feature can be checked out at [2]. All patches survives
unit testing. The feature was stress tested with hundreds
of consecutive hotplug/unplugs of a Broadcom BCM5719 multifunction
network card in a guest running in a Power 8 server. Hopefully
I'll find a suitable x86 env to stress test the feature there
too.
[1] https://www.redhat.com/archives/libvir-list/2019-June/msg00703.html
[2] https://github.com/danielhb/libvirt/tree/multifunction_latest
Daniel Henrique Barboza (1):
util/virhostdev: enhance VFIO device is in use detection
Shivaprasad G Bhat (6):
tests: Fix the iommu group path in mock pci
tests: pci: Mock the iommu groups and vfio
virpcitest: Change the stub driver to vfio from pci-stub
virpcimock: Mock the SRIOV Virtual functions
tests: qemu: Add test case for pci-hostdev hotplug
tests: Add a baseline test for multifunction pci device use case
src/util/virhostdev.c | 74 +++++--
src/util/virprocess.h | 2 +-
tests/Makefile.am | 7 +
tests/qemuhotplugtest.c | 42 +++-
.../qemuhotplug-hostdev-pci.xml | 6 +
.../qemuhotplug-base-live+hostdev-pci.xml | 58 +++++
...uhotplug-pseries-base-live+hostdev-pci.xml | 51 +++++
.../qemuhotplug-pseries-base-live.xml | 43 ++++
.../hostdev-pci-multifunction.args | 35 +++
.../hostdev-pci-multifunction.xml | 59 +++++
.../hostdev-vfio-multidomain.args | 2 +-
.../hostdev-vfio-multidomain.xml | 2 +-
tests/qemuxml2argvdata/hostdev-vfio.args | 2 +-
tests/qemuxml2argvdata/hostdev-vfio.xml | 2 +-
tests/qemuxml2argvdata/net-hostdev-fail.xml | 2 +-
tests/qemuxml2argvdata/net-hostdev-vfio.args | 2 +-
tests/qemuxml2argvdata/net-hostdev-vfio.xml | 2 +-
tests/qemuxml2argvtest.c | 3 +
.../hostdev-pci-multifunction.xml | 79 +++++++
tests/qemuxml2xmloutdata/hostdev-vfio.xml | 2 +-
tests/qemuxml2xmloutdata/net-hostdev-vfio.xml | 2 +-
tests/qemuxml2xmltest.c | 1 +
tests/virhostdevtest.c | 4 +-
tests/virpcimock.c | 201 ++++++++++++++++--
tests/virpcitest.c | 12 +-
tests/virpcitestdata/0000-06-12.0.config | Bin 0 -> 256 bytes
tests/virpcitestdata/0000-06-12.1.config | Bin 0 -> 256 bytes
tests/virpcitestdata/0000-06-12.2.config | Bin 0 -> 256 bytes
tests/virprocessmock.c | 28 +++
29 files changed, 668 insertions(+), 55 deletions(-)
create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-hostdev-pci.xml
create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+hostdev-pci.xml
create mode 100644
tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+hostdev-pci.xml
create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live.xml
create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction.args
create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction.xml
create mode 100644 tests/qemuxml2xmloutdata/hostdev-pci-multifunction.xml
create mode 100644 tests/virpcitestdata/0000-06-12.0.config
create mode 100644 tests/virpcitestdata/0000-06-12.1.config
create mode 100644 tests/virpcitestdata/0000-06-12.2.config
create mode 100644 tests/virprocessmock.c
--
2.21.0
1:09 p.m.
New subject: [libvirt] [PATCH v2 1/7] tests: Fix the iommu group path in mock pci
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
The mocked path falls into /sys/bus/kernel/iommu_groups instead of
/sys/kernel/iommu_groups. Needed for adding some new test cases.
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
tests/virpcimock.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tests/virpcimock.c b/tests/virpcimock.c
index beb5e1490d..5ec9abe05f 100644
--- a/tests/virpcimock.c
+++ b/tests/virpcimock.c
@@ -399,7 +399,7 @@ pci_device_new_from_stub(const struct pciDevice *data)
make_file(devpath, "class", tmp, -1);
if (snprintf(tmp, sizeof(tmp),
- "%s/../../../kernel/iommu_groups/%d",
+ "%s/../../../../kernel/iommu_groups/%d",
devpath, dev->iommuGroup) < 0) {
ABORT("@tmp overflow");
}
@@ -407,7 +407,7 @@ pci_device_new_from_stub(const struct pciDevice *data)
ABORT("Unable to create %s", tmp);
if (snprintf(tmp, sizeof(tmp),
- "../../../kernel/iommu_groups/%d", dev->iommuGroup) < 0)
{
+ "../../../../kernel/iommu_groups/%d", dev->iommuGroup) <
0) {
ABORT("@tmp overflow");
}
make_symlink(devpath, "iommu_group", tmp);
--
2.21.0
7:44 a.m.
New subject: [libvirt] [PATCH v2 1/7] tests: Fix the iommu group path in mock pci
On 7/25/19 8:09 PM, Daniel Henrique Barboza wrote:
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
The mocked path falls into /sys/bus/kernel/iommu_groups instead of
/sys/kernel/iommu_groups. Needed for adding some new test cases.
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
tests/virpcimock.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tests/virpcimock.c b/tests/virpcimock.c
index beb5e1490d..5ec9abe05f 100644
--- a/tests/virpcimock.c
+++ b/tests/virpcimock.c
@@ -399,7 +399,7 @@ pci_device_new_from_stub(const struct pciDevice *data)
make_file(devpath, "class", tmp, -1);
if (snprintf(tmp, sizeof(tmp),
- "%s/../../../kernel/iommu_groups/%d",
+ "%s/../../../../kernel/iommu_groups/%d",
devpath, dev->iommuGroup) < 0) {
ABORT("@tmp overflow");
}
@@ -407,7 +407,7 @@ pci_device_new_from_stub(const struct pciDevice *data)
ABORT("Unable to create %s", tmp);
if (snprintf(tmp, sizeof(tmp),
- "../../../kernel/iommu_groups/%d", dev->iommuGroup) <
0) {
+ "../../../../kernel/iommu_groups/%d", dev->iommuGroup)
< 0) {
ABORT("@tmp overflow");
}
make_symlink(devpath, "iommu_group", tmp);
Huh, this is again something that I fix in my series. Well, in fact,
since I'm moving the place where PCI devices are created the symlink as
we have it is correct in fact.
Michal
1:09 p.m.
New subject: [libvirt] [PATCH v2 2/7] tests: pci: Mock the iommu groups and vfio
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
The iommu group, /dev/vfio/<iommuGroupNumber> behaviors
of the host are mocked. This patch implements support for
multifunction/multiple devices per iommu groups and emulates
the /dev/vfio/<iommuGroupNumber> file correctly.
This code helps adding necessary testcases for pci-hotplug
code.
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
tests/virpcimock.c | 177 +++++++++++++++++++++++++++++++++++++++++----
1 file changed, 162 insertions(+), 15 deletions(-)
diff --git a/tests/virpcimock.c b/tests/virpcimock.c
index 5ec9abe05f..1b44ed0a44 100644
--- a/tests/virpcimock.c
+++ b/tests/virpcimock.c
@@ -44,6 +44,8 @@ char *fakerootdir;
char *fakesysfspcidir;
# define SYSFS_PCI_PREFIX "/sys/bus/pci/"
+# define SYSFS_KERNEL_PREFIX "/sys/kernel/"
+
# define STDERR(...) \
fprintf(stderr, "%s %zu: ", __FUNCTION__, (size_t) __LINE__); \
@@ -123,6 +125,11 @@ struct pciDevice {
struct pciDriver *driver; /* Driver attached. NULL if attached to no driver */
};
+struct pciIommuGroup {
+ int iommu;
+ size_t nDevicesBoundToVFIO; /* Indicates the devices in the group */
+};
+
struct fdCallback {
int fd;
char *path;
@@ -134,6 +141,9 @@ size_t nPCIDevices = 0;
struct pciDriver **pciDrivers = NULL;
size_t nPCIDrivers = 0;
+struct pciIommuGroup **pciIommuGroups = NULL;
+size_t npciIommuGroups = 0;
+
struct fdCallback *callbacks = NULL;
size_t nCallbacks = 0;
@@ -247,6 +257,13 @@ getrealpath(char **newpath,
errno = ENOMEM;
return -1;
}
+ } else if (STRPREFIX(path, SYSFS_KERNEL_PREFIX)) {
+ if (virAsprintfQuiet(newpath, "%s/%s",
+ fakerootdir,
+ path) < 0) {
+ errno = ENOMEM;
+ return -1;
+ }
} else {
if (VIR_STRDUP_QUIET(*newpath, path) < 0)
return -1;
@@ -460,6 +477,101 @@ pci_device_autobind(struct pciDevice *dev)
return pci_driver_bind(driver, dev);
}
+static void
+pci_iommu_new(int num)
+{
+ char *iommupath, *kerneldir;
+ struct pciIommuGroup *iommuGroup;
+
+ if (VIR_ALLOC_QUIET(iommuGroup) < 0)
+ ABORT_OOM();
+
+ iommuGroup->iommu = num;
+ iommuGroup->nDevicesBoundToVFIO = 0; /* No device bound to vfio by default */
+
+ if (virAsprintfQuiet(&kerneldir, "%s%s",
+ fakerootdir, SYSFS_KERNEL_PREFIX) < 0)
+ ABORT_OOM();
+
+ if (virAsprintfQuiet(&iommupath, "%s/iommu_groups/%d/devices",
kerneldir, num) < 0)
+ ABORT_OOM();
+ VIR_FREE(kerneldir);
+
+ if (virFileMakePath(iommupath) < 0)
+ ABORT("Unable to create: %s", iommupath);
+ VIR_FREE(iommupath);
+
+ if (VIR_APPEND_ELEMENT_QUIET(pciIommuGroups, npciIommuGroups, iommuGroup) < 0)
+ ABORT_OOM();
+}
+
+static int
+pci_vfio_release_iommu(struct pciDevice *device)
+{
+ char *vfiopath = NULL;
+ int ret = -1;
+ size_t i = 0;
+
+ for (i = 0; i < npciIommuGroups; i++) {
+ if (pciIommuGroups[i]->iommu == device->iommuGroup)
+ break;
+ }
+
+ if (i != npciIommuGroups) {
+ if (pciIommuGroups[i]->nDevicesBoundToVFIO == 0) {
+ ret = 0;
+ goto cleanup;
+ }
+ pciIommuGroups[i]->nDevicesBoundToVFIO--;
+ if (!pciIommuGroups[i]->nDevicesBoundToVFIO) {
+ if (virAsprintfQuiet(&vfiopath, "%s/dev/vfio/%d",
+ fakesysfspcidir, device->iommuGroup) < 0) {
+ errno = ENOMEM;
+ goto cleanup;
+ }
+ if (unlink(vfiopath) < 0)
+ goto cleanup;
+ }
+ }
+
+ ret = 0;
+ cleanup:
+ VIR_FREE(vfiopath);
+ return ret;
+}
+
+static int
+pci_vfio_lock_iommu(struct pciDevice *device)
+{
+ char *vfiopath = NULL;
+ int ret = -1;
+ size_t i = 0;
+ int fd = -1;
+
+ for (i = 0; i < npciIommuGroups; i++) {
+ if (pciIommuGroups[i]->iommu == device->iommuGroup)
+ break;
+ }
+
+ if (i != npciIommuGroups) {
+ if (!pciIommuGroups[i]->nDevicesBoundToVFIO) {
+ if (virAsprintfQuiet(&vfiopath, "%s/dev/vfio/%d",
+ fakesysfspcidir, device->iommuGroup) < 0) {
+ errno = ENOMEM;
+ goto cleanup;
+ }
+ if ((fd = real_open(vfiopath, O_CREAT)) < 0)
+ goto cleanup;
+ }
+ pciIommuGroups[i]->nDevicesBoundToVFIO++;
+ }
+
+ ret = 0;
+ cleanup:
+ real_close(fd);
+ VIR_FREE(vfiopath);
+ return ret;
+}
/*
* PCI Driver functions
@@ -583,6 +695,10 @@ pci_driver_bind(struct pciDriver *driver,
if (symlink(devpath, driverpath) < 0)
goto cleanup;
+ if (STREQ(driver->name, "vfio-pci"))
+ if (pci_vfio_lock_iommu(dev) < 0)
+ goto cleanup;
+
dev->driver = driver;
ret = 0;
cleanup:
@@ -617,6 +733,10 @@ pci_driver_unbind(struct pciDriver *driver,
unlink(driverpath) < 0)
goto cleanup;
+ if (STREQ(driver->name, "vfio-pci"))
+ if (pci_vfio_release_iommu(dev) < 0)
+ goto cleanup;
+
dev->driver = NULL;
ret = 0;
cleanup:
@@ -813,6 +933,8 @@ init_syms(void)
static void
init_env(void)
{
+ char *devVFIO;
+
if (fakerootdir && fakesysfspcidir)
return;
@@ -828,6 +950,29 @@ init_env(void)
make_file(fakesysfspcidir, "drivers_probe", NULL, -1);
+ if (virAsprintfQuiet(&devVFIO, "%s/dev/vfio", fakesysfspcidir) < 0)
+ ABORT_OOM();
+
+ if (virFileMakePath(devVFIO) < 0)
+ ABORT("Unable to create: %s", devVFIO);
+
+ /* Create /dev/vfio/vfio file */
+ make_file(devVFIO, "vfio", NULL, -1);
+ VIR_FREE(devVFIO);
+
+ pci_iommu_new(0);
+ pci_iommu_new(1);
+ pci_iommu_new(2);
+ pci_iommu_new(3);
+ pci_iommu_new(4);
+ pci_iommu_new(5);
+ pci_iommu_new(6);
+ pci_iommu_new(7);
+ pci_iommu_new(8);
+ pci_iommu_new(9);
+ pci_iommu_new(10);
+ pci_iommu_new(11);
+
# define MAKE_PCI_DRIVER(name, ...) \
pci_driver_new(name, 0, __VA_ARGS__, -1, -1)
@@ -843,20 +988,21 @@ init_env(void)
pci_device_new_from_stub(&dev); \
} while (0)
- MAKE_PCI_DEVICE("0000:00:00.0", 0x8086, 0x0044);
- MAKE_PCI_DEVICE("0000:00:01.0", 0x8086, 0x0044);
- MAKE_PCI_DEVICE("0000:00:02.0", 0x8086, 0x0046);
- MAKE_PCI_DEVICE("0000:00:03.0", 0x8086, 0x0048);
- MAKE_PCI_DEVICE("0001:00:00.0", 0x1014, 0x03b9, .klass = 0x060400);
- MAKE_PCI_DEVICE("0001:01:00.0", 0x8086, 0x105e, .iommuGroup = 0);
- MAKE_PCI_DEVICE("0001:01:00.1", 0x8086, 0x105e, .iommuGroup = 0);
- MAKE_PCI_DEVICE("0005:80:00.0", 0x10b5, 0x8112, .klass = 0x060400);
- MAKE_PCI_DEVICE("0005:90:01.0", 0x1033, 0x0035, .iommuGroup = 1);
- MAKE_PCI_DEVICE("0005:90:01.1", 0x1033, 0x0035, .iommuGroup = 1);
- MAKE_PCI_DEVICE("0005:90:01.2", 0x1033, 0x00e0, .iommuGroup = 1);
- MAKE_PCI_DEVICE("0000:0a:01.0", 0x8086, 0x0047);
- MAKE_PCI_DEVICE("0000:0a:02.0", 0x8286, 0x0048);
- MAKE_PCI_DEVICE("0000:0a:03.0", 0x8386, 0x0048);
+ MAKE_PCI_DEVICE("0000:00:00.0", 0x8086, 0x0044, .iommuGroup = 0);
+ MAKE_PCI_DEVICE("0000:00:01.0", 0x8086, 0x0044, .iommuGroup = 1);
+ MAKE_PCI_DEVICE("0000:00:02.0", 0x8086, 0x0046, .iommuGroup = 2);
+ MAKE_PCI_DEVICE("0000:00:03.0", 0x8086, 0x0048, .iommuGroup = 3);
+ MAKE_PCI_DEVICE("0001:00:00.0", 0x1014, 0x03b9, .klass = 0x060400,
.iommuGroup = 4);
+ MAKE_PCI_DEVICE("0001:01:00.0", 0x8086, 0x105e, .iommuGroup = 5);
+ MAKE_PCI_DEVICE("0001:01:00.1", 0x8086, 0x105e, .iommuGroup = 5);
+ MAKE_PCI_DEVICE("0005:80:00.0", 0x10b5, 0x8112, .klass = 0x060400,
.iommuGroup = 6);
+ MAKE_PCI_DEVICE("0005:90:01.0", 0x1033, 0x0035, .iommuGroup = 7);
+ MAKE_PCI_DEVICE("0005:90:01.1", 0x1033, 0x0035, .iommuGroup = 7);
+ MAKE_PCI_DEVICE("0005:90:01.2", 0x1033, 0x00e0, .iommuGroup = 7);
+ MAKE_PCI_DEVICE("0005:90:01.3", 0x1033, 0x00e0, .iommuGroup = 7);
+ MAKE_PCI_DEVICE("0000:0a:01.0", 0x8086, 0x0047, .iommuGroup = 8);
+ MAKE_PCI_DEVICE("0000:0a:02.0", 0x8286, 0x0048, .iommuGroup = 8);
+ MAKE_PCI_DEVICE("0000:0a:03.0", 0x8386, 0x0048, .iommuGroup = 8);
}
@@ -940,7 +1086,8 @@ opendir(const char *path)
init_syms();
- if (STRPREFIX(path, SYSFS_PCI_PREFIX) &&
+ if ((STRPREFIX(path, SYSFS_PCI_PREFIX) ||
+ STRPREFIX(path, SYSFS_KERNEL_PREFIX)) &&
getrealpath(&newpath, path) < 0)
return NULL;
--
2.21.0
7:44 a.m.
New subject: [libvirt] [PATCH v2 2/7] tests: pci: Mock the iommu groups and vfio
On 7/25/19 8:09 PM, Daniel Henrique Barboza wrote:
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
The iommu group, /dev/vfio/<iommuGroupNumber> behaviors
of the host are mocked. This patch implements support for
multifunction/multiple devices per iommu groups and emulates
the /dev/vfio/<iommuGroupNumber> file correctly.
This code helps adding necessary testcases for pci-hotplug
code.
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
tests/virpcimock.c | 177 +++++++++++++++++++++++++++++++++++++++++----
1 file changed, 162 insertions(+), 15 deletions(-)
Looks good. We will have some merge conflicts because we fix the same issue.
diff --git a/tests/virpcimock.c b/tests/virpcimock.c
index 5ec9abe05f..1b44ed0a44 100644
--- a/tests/virpcimock.c
+++ b/tests/virpcimock.c
@@ -44,6 +44,8 @@ char *fakerootdir;
char *fakesysfspcidir;
# define SYSFS_PCI_PREFIX "/sys/bus/pci/"
+# define SYSFS_KERNEL_PREFIX "/sys/kernel/"
+
# define STDERR(...) \
fprintf(stderr, "%s %zu: ", __FUNCTION__, (size_t) __LINE__); \
@@ -123,6 +125,11 @@ struct pciDevice {
struct pciDriver *driver; /* Driver attached. NULL if attached to no driver */
};
+struct pciIommuGroup {
+ int iommu;
+ size_t nDevicesBoundToVFIO; /* Indicates the devices in the group */
+};
+
struct fdCallback {
int fd;
char *path;
@@ -134,6 +141,9 @@ size_t nPCIDevices = 0;
struct pciDriver **pciDrivers = NULL;
size_t nPCIDrivers = 0;
+struct pciIommuGroup **pciIommuGroups = NULL;
+size_t npciIommuGroups = 0;
+
struct fdCallback *callbacks = NULL;
size_t nCallbacks = 0;
@@ -247,6 +257,13 @@ getrealpath(char **newpath,
errno = ENOMEM;
return -1;
}
+ } else if (STRPREFIX(path, SYSFS_KERNEL_PREFIX)) {
+ if (virAsprintfQuiet(newpath, "%s/%s",
+ fakerootdir,
+ path) < 0) {
+ errno = ENOMEM;
+ return -1;
+ }
} else {
if (VIR_STRDUP_QUIET(*newpath, path) < 0)
return -1;
@@ -460,6 +477,101 @@ pci_device_autobind(struct pciDevice *dev)
return pci_driver_bind(driver, dev);
}
+static void
+pci_iommu_new(int num)
+{
+ char *iommupath, *kerneldir;
+ struct pciIommuGroup *iommuGroup;
+
+ if (VIR_ALLOC_QUIET(iommuGroup) < 0)
+ ABORT_OOM();
+
+ iommuGroup->iommu = num;
+ iommuGroup->nDevicesBoundToVFIO = 0; /* No device bound to vfio by default */
+
+ if (virAsprintfQuiet(&kerneldir, "%s%s",
+ fakerootdir, SYSFS_KERNEL_PREFIX) < 0)
+ ABORT_OOM();
+
+ if (virAsprintfQuiet(&iommupath, "%s/iommu_groups/%d/devices",
kerneldir, num) < 0)
+ ABORT_OOM();
+ VIR_FREE(kerneldir);
+
+ if (virFileMakePath(iommupath) < 0)
+ ABORT("Unable to create: %s", iommupath);
+ VIR_FREE(iommupath);
+
+ if (VIR_APPEND_ELEMENT_QUIET(pciIommuGroups, npciIommuGroups, iommuGroup) < 0)
+ ABORT_OOM();
+}
+
+static int
+pci_vfio_release_iommu(struct pciDevice *device)
+{
+ char *vfiopath = NULL;
+ int ret = -1;
+ size_t i = 0;
+
+ for (i = 0; i < npciIommuGroups; i++) {
+ if (pciIommuGroups[i]->iommu == device->iommuGroup)
+ break;
+ }
+
+ if (i != npciIommuGroups) {
+ if (pciIommuGroups[i]->nDevicesBoundToVFIO == 0) {
+ ret = 0;
+ goto cleanup;
+ }
+ pciIommuGroups[i]->nDevicesBoundToVFIO--;
+ if (!pciIommuGroups[i]->nDevicesBoundToVFIO) {
+ if (virAsprintfQuiet(&vfiopath, "%s/dev/vfio/%d",
+ fakesysfspcidir, device->iommuGroup) < 0) {
+ errno = ENOMEM;
+ goto cleanup;
+ }
+ if (unlink(vfiopath) < 0)
+ goto cleanup;
+ }
+ }
+
+ ret = 0;
+ cleanup:
+ VIR_FREE(vfiopath);
+ return ret;
+}
+
+static int
+pci_vfio_lock_iommu(struct pciDevice *device)
+{
+ char *vfiopath = NULL;
+ int ret = -1;
+ size_t i = 0;
+ int fd = -1;
+
+ for (i = 0; i < npciIommuGroups; i++) {
+ if (pciIommuGroups[i]->iommu == device->iommuGroup)
+ break;
+ }
+
+ if (i != npciIommuGroups) {
+ if (!pciIommuGroups[i]->nDevicesBoundToVFIO) {
+ if (virAsprintfQuiet(&vfiopath, "%s/dev/vfio/%d",
+ fakesysfspcidir, device->iommuGroup) < 0) {
+ errno = ENOMEM;
+ goto cleanup;
+ }
+ if ((fd = real_open(vfiopath, O_CREAT)) < 0)
+ goto cleanup;
+ }
+ pciIommuGroups[i]->nDevicesBoundToVFIO++;
+ }
+
+ ret = 0;
+ cleanup:
+ real_close(fd);
+ VIR_FREE(vfiopath);
+ return ret;
+}
/*
* PCI Driver functions
@@ -583,6 +695,10 @@ pci_driver_bind(struct pciDriver *driver,
if (symlink(devpath, driverpath) < 0)
goto cleanup;
+ if (STREQ(driver->name, "vfio-pci"))
+ if (pci_vfio_lock_iommu(dev) < 0)
+ goto cleanup;
+
dev->driver = driver;
ret = 0;
cleanup:
@@ -617,6 +733,10 @@ pci_driver_unbind(struct pciDriver *driver,
unlink(driverpath) < 0)
goto cleanup;
+ if (STREQ(driver->name, "vfio-pci"))
+ if (pci_vfio_release_iommu(dev) < 0)
+ goto cleanup;
+
dev->driver = NULL;
ret = 0;
cleanup:
@@ -813,6 +933,8 @@ init_syms(void)
static void
init_env(void)
{
+ char *devVFIO;
+
if (fakerootdir && fakesysfspcidir)
return;
@@ -828,6 +950,29 @@ init_env(void)
make_file(fakesysfspcidir, "drivers_probe", NULL, -1);
+ if (virAsprintfQuiet(&devVFIO, "%s/dev/vfio", fakesysfspcidir) <
0)
+ ABORT_OOM();
+
+ if (virFileMakePath(devVFIO) < 0)
+ ABORT("Unable to create: %s", devVFIO);
+
+ /* Create /dev/vfio/vfio file */
+ make_file(devVFIO, "vfio", NULL, -1);
+ VIR_FREE(devVFIO);
+
+ pci_iommu_new(0);
+ pci_iommu_new(1);
+ pci_iommu_new(2);
+ pci_iommu_new(3);
+ pci_iommu_new(4);
+ pci_iommu_new(5);
+ pci_iommu_new(6);
+ pci_iommu_new(7);
+ pci_iommu_new(8);
+ pci_iommu_new(9);
+ pci_iommu_new(10);
+ pci_iommu_new(11);
I wonder if we can make these somehow dyamic. E.g. called from
pci_device_new_from_stub() and require iommuGroup in MAKE_PCI_DEVICE()
because every device belongs to an IOMMU group.
Michal
1:09 p.m.
New subject: [libvirt] [PATCH v2 3/7] util/virhostdev: enhance VFIO device is in use detection
The current virHostdevPreparePCIDevices code fails to detect an unmanaged
VFIO device that is in the activePCIHostdevs, and active in the same
domain name as dom_name, as a device in use. Considering a call to this
function, when activePCIHostdevs has a VFIO deviceA in the list, and
deviceA is active in domain 'dom_name', this is what happens:
- At step 1, the code goes to the 'if (usesVFIO)' block. All devices in
the same IOMMU group of deviceA are used as argument of
virHostdevIsPCINodeDeviceUsed, via the IOMMUGroupIterate function;
- Inside virHostdevIsPCINodeDeviceUsed, an 'usesVFIO' verification is
made, following to an 'iommu_owner' jump that sets ret=0. This will
happen to all devices of the IOMMU group that belongs to the same domain
as dom_name, including deviceA;
- Step 2 starts, thinking that all devices inside hostdevs are available,
which is not the case.
This error was detected when changing tests/virhostdev.c to use
vfio-pci instead of pci-stub (a change that will follow this one).
The side effect observed was a test failure in
testVirHostdevPreparePCIHostdevs_unmanaged, result of the behavior
mentioned above:
Unexpected count of items in mgr->inactivePCIHostdevs: 1, expecting 0
This patch fixes virHostdevIsPCINodeDeviceUsed to consider the case of
a VFIO device that is already active in the domain. First, check if
the device is in the activePCIHostdev list and bail immediately if
true. Otherwise, in case of VFIO, check for IOMMU group ownership
of the domain. This is done by a new callback function for
IOMMUGroupIterate. If the VFIO device isn't in the active list and
its domain has ownership of the IOMMU, then it is unused.
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
src/util/virhostdev.c | 74 +++++++++++++++++++++++++++++++++----------
1 file changed, 57 insertions(+), 17 deletions(-)
diff --git a/src/util/virhostdev.c b/src/util/virhostdev.c
index a3647a6cf4..35be8fede1 100644
--- a/src/util/virhostdev.c
+++ b/src/util/virhostdev.c
@@ -55,6 +55,46 @@ struct virHostdevIsPCINodeDeviceUsedData {
const bool usesVFIO;
};
+
+static virPCIDevicePtr
+virHostdevFindActivePCIDevWithAddr(virPCIDeviceAddressPtr devAddr,
+ virHostdevManagerPtr mgr)
+{
+ virPCIDevicePtr ret = NULL;
+
+ ret = virPCIDeviceListFindByIDs(mgr->activePCIHostdevs,
+ devAddr->domain, devAddr->bus,
+ devAddr->slot, devAddr->function);
+
+ return ret;
+}
+
+/* Callback to be used inside virHostdevIsPCINodeDeviceUsed to check
+ * for IOMMU ownership of a domain given by helperData->domainName. */
+static int
+virHostdevDomainHasIOMMUOwnershipCb(virPCIDeviceAddressPtr devAddr, void *opaque)
+{
+ struct virHostdevIsPCINodeDeviceUsedData *helperData = opaque;
+ virPCIDevicePtr actual;
+ int ret = 0;
+
+ actual = virHostdevFindActivePCIDevWithAddr(devAddr, helperData->mgr);
+ if (!actual)
+ return ret;
+
+ if (helperData->usesVFIO) {
+ const char *actual_drvname = NULL;
+ const char *actual_domname = NULL;
+ virPCIDeviceGetUsedBy(actual, &actual_drvname, &actual_domname);
+
+ if ((actual_domname && helperData->domainName) &&
+ (STRNEQ(actual_domname, helperData->domainName)))
+ ret = -1;
+ }
+
+ return ret;
+}
+
/* This module makes heavy use of bookkeeping lists contained inside a
* virHostdevManager instance to keep track of the devices' status. To make
* it easy to spot potential ownership errors when moving devices from one
@@ -82,19 +122,13 @@ static int virHostdevIsPCINodeDeviceUsed(virPCIDeviceAddressPtr
devAddr, void *o
int ret = -1;
struct virHostdevIsPCINodeDeviceUsedData *helperData = opaque;
- actual = virPCIDeviceListFindByIDs(helperData->mgr->activePCIHostdevs,
- devAddr->domain, devAddr->bus,
- devAddr->slot, devAddr->function);
+ actual = virHostdevFindActivePCIDevWithAddr(devAddr, helperData->mgr);
+
if (actual) {
const char *actual_drvname = NULL;
const char *actual_domname = NULL;
virPCIDeviceGetUsedBy(actual, &actual_drvname, &actual_domname);
- if (helperData->usesVFIO &&
- (actual_domname && helperData->domainName) &&
- (STREQ(actual_domname, helperData->domainName)))
- goto iommu_owner;
-
if (actual_drvname && actual_domname)
virReportError(VIR_ERR_OPERATION_INVALID,
_("PCI device %s is in use by "
@@ -105,9 +139,20 @@ static int virHostdevIsPCINodeDeviceUsed(virPCIDeviceAddressPtr
devAddr, void *o
virReportError(VIR_ERR_OPERATION_INVALID,
_("PCI device %s is in use"),
virPCIDeviceGetName(actual));
+
goto cleanup;
}
- iommu_owner:
+
+ /* For VFIO devices, the domain helperData->domainName must have ownership
+ * of the IOMMU group that contains devAddr. Otherwise, even if the devAddr
+ * is not in use, another device of that IOMMU group is in use by anotherdomain,
+ * forbidding devAddr to be used in helperData->domainName. */
+ if (helperData->usesVFIO)
+ if (virPCIDeviceAddressIOMMUGroupIterate(devAddr,
+ virHostdevDomainHasIOMMUOwnershipCb,
+ helperData) < 0)
+ goto cleanup;
+
ret = 0;
cleanup:
return ret;
@@ -673,17 +718,12 @@ virHostdevPreparePCIDevices(virHostdevManagerPtr mgr,
/* The device is in use by other active domain if
* the dev is in list activePCIHostdevs. VFIO devices
* belonging to same iommu group can't be shared
- * across guests.
+ * across guests. virHostdevIsPCINodeDeviceUsedData handles
+ * both cases.
*/
devAddr = virPCIDeviceGetAddress(pci);
- if (usesVFIO) {
- if (virPCIDeviceAddressIOMMUGroupIterate(devAddr,
- virHostdevIsPCINodeDeviceUsed,
- &data) < 0)
- goto cleanup;
- } else if (virHostdevIsPCINodeDeviceUsed(devAddr, &data)) {
+ if (virHostdevIsPCINodeDeviceUsed(devAddr, &data))
goto cleanup;
- }
}
/* Step 1.5: For non-802.11Qbh SRIOV network devices, save the
--
2.21.0
7:43 a.m.
New subject: [libvirt] [PATCH v2 3/7] util/virhostdev: enhance VFIO device is in use detection
On 7/25/19 8:09 PM, Daniel Henrique Barboza wrote:
The current virHostdevPreparePCIDevices code fails to detect an
unmanaged
VFIO device that is in the activePCIHostdevs, and active in the same
domain name as dom_name, as a device in use. Considering a call to this
function, when activePCIHostdevs has a VFIO deviceA in the list, and
deviceA is active in domain 'dom_name', this is what happens:
- At step 1, the code goes to the 'if (usesVFIO)' block. All devices in
the same IOMMU group of deviceA are used as argument of
virHostdevIsPCINodeDeviceUsed, via the IOMMUGroupIterate function;
- Inside virHostdevIsPCINodeDeviceUsed, an 'usesVFIO' verification is
made, following to an 'iommu_owner' jump that sets ret=0. This will
happen to all devices of the IOMMU group that belongs to the same domain
as dom_name, including deviceA;
- Step 2 starts, thinking that all devices inside hostdevs are available,
which is not the case.
This error was detected when changing tests/virhostdev.c to use
vfio-pci instead of pci-stub (a change that will follow this one).
The side effect observed was a test failure in
testVirHostdevPreparePCIHostdevs_unmanaged, result of the behavior
mentioned above:
Unexpected count of items in mgr->inactivePCIHostdevs: 1, expecting 0
This patch fixes virHostdevIsPCINodeDeviceUsed to consider the case of
a VFIO device that is already active in the domain. First, check if
the device is in the activePCIHostdev list and bail immediately if
true. Otherwise, in case of VFIO, check for IOMMU group ownership
of the domain. This is done by a new callback function for
IOMMUGroupIterate. If the VFIO device isn't in the active list and
its domain has ownership of the IOMMU, then it is unused.
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
src/util/virhostdev.c | 74 +++++++++++++++++++++++++++++++++----------
1 file changed, 57 insertions(+), 17 deletions(-)
As you pointed out, I've sent a patch for the same issue. And if you
don't mind, I'd like to go with mine, because it's shorter and simpler.
Michal
1:09 p.m.
New subject: [libvirt] [PATCH v2 4/7] virpcitest: Change the stub driver to vfio from pci-stub
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
The pci-stub is obsolete for a while now. Upcoming test cases try
to test the VFIO hotplug/unplug cases.
Change the default test driver to vfio-pci instead of pci-stub,
and fail bind for pci-stub instead.
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
tests/virhostdevtest.c | 4 ++--
tests/virpcimock.c | 4 ++--
tests/virpcitest.c | 12 ++++++------
3 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/tests/virhostdevtest.c b/tests/virhostdevtest.c
index 20eaca82e0..99ee2d44ec 100644
--- a/tests/virhostdevtest.c
+++ b/tests/virhostdevtest.c
@@ -93,7 +93,7 @@ myInit(void)
subsys.u.pci.addr.bus = 0;
subsys.u.pci.addr.slot = i + 1;
subsys.u.pci.addr.function = 0;
- subsys.u.pci.backend = VIR_DOMAIN_HOSTDEV_PCI_BACKEND_KVM;
+ subsys.u.pci.backend = VIR_DOMAIN_HOSTDEV_PCI_BACKEND_VFIO;
hostdevs[i]->source.subsys = subsys;
}
@@ -101,7 +101,7 @@ myInit(void)
if (!(dev[i] = virPCIDeviceNew(0, 0, i + 1, 0)))
goto cleanup;
- virPCIDeviceSetStubDriver(dev[i], VIR_PCI_STUB_DRIVER_KVM);
+ virPCIDeviceSetStubDriver(dev[i], VIR_PCI_STUB_DRIVER_VFIO);
}
if (VIR_ALLOC(mgr) < 0)
diff --git a/tests/virpcimock.c b/tests/virpcimock.c
index 1b44ed0a44..deb802e860 100644
--- a/tests/virpcimock.c
+++ b/tests/virpcimock.c
@@ -978,8 +978,8 @@ init_env(void)
MAKE_PCI_DRIVER("iwlwifi", 0x8086, 0x0044);
MAKE_PCI_DRIVER("i915", 0x8086, 0x0046, 0x8086, 0x0047);
- MAKE_PCI_DRIVER("pci-stub", -1, -1);
- pci_driver_new("vfio-pci", PCI_ACTION_BIND, -1, -1);
+ pci_driver_new("pci-stub", PCI_ACTION_BIND, -1, -1);
+ MAKE_PCI_DRIVER("vfio-pci", -1, -1);
# define MAKE_PCI_DEVICE(Id, Vendor, Device, ...) \
do { \
diff --git a/tests/virpcitest.c b/tests/virpcitest.c
index 961a7eff1a..d120e7773a 100644
--- a/tests/virpcitest.c
+++ b/tests/virpcitest.c
@@ -106,12 +106,12 @@ testVirPCIDeviceDetach(const void *opaque ATTRIBUTE_UNUSED)
if (!(dev[i] = virPCIDeviceNew(0, 0, i + 1, 0)))
goto cleanup;
- virPCIDeviceSetStubDriver(dev[i], VIR_PCI_STUB_DRIVER_KVM);
+ virPCIDeviceSetStubDriver(dev[i], VIR_PCI_STUB_DRIVER_VFIO);
if (virPCIDeviceDetach(dev[i], activeDevs, inactiveDevs) < 0)
goto cleanup;
- if (testVirPCIDeviceCheckDriver(dev[i], "pci-stub") < 0)
+ if (testVirPCIDeviceCheckDriver(dev[i], "vfio-pci") < 0)
goto cleanup;
CHECK_LIST_COUNT(activeDevs, 0);
@@ -245,7 +245,7 @@ testVirPCIDeviceDetachSingle(const void *opaque)
if (!dev)
goto cleanup;
- virPCIDeviceSetStubDriver(dev, VIR_PCI_STUB_DRIVER_KVM);
+ virPCIDeviceSetStubDriver(dev, VIR_PCI_STUB_DRIVER_VFIO);
if (virPCIDeviceDetach(dev, NULL, NULL) < 0)
goto cleanup;
@@ -267,7 +267,7 @@ testVirPCIDeviceDetachFail(const void *opaque)
if (!dev)
goto cleanup;
- virPCIDeviceSetStubDriver(dev, VIR_PCI_STUB_DRIVER_VFIO);
+ virPCIDeviceSetStubDriver(dev, VIR_PCI_STUB_DRIVER_KVM);
if (virPCIDeviceDetach(dev, NULL, NULL) < 0) {
if (virTestGetVerbose() || virTestGetDebug())
@@ -278,7 +278,7 @@ testVirPCIDeviceDetachFail(const void *opaque)
virReportError(VIR_ERR_INTERNAL_ERROR,
"Attaching device %s to %s should have failed",
virPCIDeviceGetName(dev),
- virPCIStubDriverTypeToString(VIR_PCI_STUB_DRIVER_VFIO));
+ virPCIStubDriverTypeToString(VIR_PCI_STUB_DRIVER_KVM));
}
cleanup:
@@ -437,7 +437,7 @@ mymain(void)
/* Detach an unbound device */
DO_TEST_PCI_DRIVER(0, 0x0a, 2, 0, NULL);
DO_TEST_PCI(testVirPCIDeviceDetachSingle, 0, 0x0a, 2, 0);
- DO_TEST_PCI_DRIVER(0, 0x0a, 2, 0, "pci-stub");
+ DO_TEST_PCI_DRIVER(0, 0x0a, 2, 0, "vfio-pci");
/* Reattach an unknown unbound device */
DO_TEST_PCI_DRIVER(0, 0x0a, 3, 0, NULL);
--
2.21.0
7:43 a.m.
New subject: [libvirt] [PATCH v2 4/7] virpcitest: Change the stub driver to vfio from pci-stub
On 7/25/19 8:09 PM, Daniel Henrique Barboza wrote:
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
The pci-stub is obsolete for a while now. Upcoming test cases try
to test the VFIO hotplug/unplug cases.
Change the default test driver to vfio-pci instead of pci-stub,
and fail bind for pci-stub instead.
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
tests/virhostdevtest.c | 4 ++--
tests/virpcimock.c | 4 ++--
tests/virpcitest.c | 12 ++++++------
3 files changed, 10 insertions(+), 10 deletions(-)
Huh, again, same patches ...
diff --git a/tests/virhostdevtest.c b/tests/virhostdevtest.c
index 20eaca82e0..99ee2d44ec 100644
--- a/tests/virhostdevtest.c
+++ b/tests/virhostdevtest.c
@@ -93,7 +93,7 @@ myInit(void)
subsys.u.pci.addr.bus = 0;
subsys.u.pci.addr.slot = i + 1;
subsys.u.pci.addr.function = 0;
- subsys.u.pci.backend = VIR_DOMAIN_HOSTDEV_PCI_BACKEND_KVM;
+ subsys.u.pci.backend = VIR_DOMAIN_HOSTDEV_PCI_BACKEND_VFIO;
hostdevs[i]->source.subsys = subsys;
}
@@ -101,7 +101,7 @@ myInit(void)
if (!(dev[i] = virPCIDeviceNew(0, 0, i + 1, 0)))
goto cleanup;
- virPCIDeviceSetStubDriver(dev[i], VIR_PCI_STUB_DRIVER_KVM);
+ virPCIDeviceSetStubDriver(dev[i], VIR_PCI_STUB_DRIVER_VFIO);
}
if (VIR_ALLOC(mgr) < 0)
diff --git a/tests/virpcimock.c b/tests/virpcimock.c
index 1b44ed0a44..deb802e860 100644
--- a/tests/virpcimock.c
+++ b/tests/virpcimock.c
@@ -978,8 +978,8 @@ init_env(void)
MAKE_PCI_DRIVER("iwlwifi", 0x8086, 0x0044);
MAKE_PCI_DRIVER("i915", 0x8086, 0x0046, 0x8086, 0x0047);
- MAKE_PCI_DRIVER("pci-stub", -1, -1);
- pci_driver_new("vfio-pci", PCI_ACTION_BIND, -1, -1);
+ pci_driver_new("pci-stub", PCI_ACTION_BIND, -1, -1);
+ MAKE_PCI_DRIVER("vfio-pci", -1, -1);
.. except this one. This means I can drop two patches from mine series
if I include this change :-) Let me resping that.
Michal
1:09 p.m.
New subject: [libvirt] [PATCH v2 5/7] virpcimock: Mock the SRIOV Virtual functions
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
The softlink to physfn is the way to know if the device is
VF or not. So, the patch softlinks 'physfn' to the parent function.
The multifunction PCI devices dont have 'physfn' softlinks.
The patch adds few Virtual functions to the mock environment and
changes the existing VFIO test xmls using the VFs to use the newly
added VFs for their use case.
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
.../hostdev-vfio-multidomain.args | 2 +-
.../hostdev-vfio-multidomain.xml | 2 +-
tests/qemuxml2argvdata/hostdev-vfio.args | 2 +-
tests/qemuxml2argvdata/hostdev-vfio.xml | 2 +-
tests/qemuxml2argvdata/net-hostdev-fail.xml | 2 +-
tests/qemuxml2argvdata/net-hostdev-vfio.args | 2 +-
tests/qemuxml2argvdata/net-hostdev-vfio.xml | 2 +-
tests/qemuxml2xmloutdata/hostdev-vfio.xml | 2 +-
tests/qemuxml2xmloutdata/net-hostdev-vfio.xml | 2 +-
tests/virpcimock.c | 16 ++++++++++++++++
tests/virpcitestdata/0000-06-12.0.config | Bin 0 -> 256 bytes
tests/virpcitestdata/0000-06-12.1.config | Bin 0 -> 256 bytes
tests/virpcitestdata/0000-06-12.2.config | Bin 0 -> 256 bytes
13 files changed, 25 insertions(+), 9 deletions(-)
create mode 100644 tests/virpcitestdata/0000-06-12.0.config
create mode 100644 tests/virpcitestdata/0000-06-12.1.config
create mode 100644 tests/virpcitestdata/0000-06-12.2.config
diff --git a/tests/qemuxml2argvdata/hostdev-vfio-multidomain.args
b/tests/qemuxml2argvdata/hostdev-vfio-multidomain.args
index d098bff356..959d2a8888 100644
--- a/tests/qemuxml2argvdata/hostdev-vfio-multidomain.args
+++ b/tests/qemuxml2argvdata/hostdev-vfio-multidomain.args
@@ -27,5 +27,5 @@ server,nowait \
-usb \
-drive file=/dev/HostVG/QEMUGuest2,format=raw,if=none,id=drive-ide0-0-0 \
-device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \
--device vfio-pci,host=55aa:20:0f.3,id=hostdev0,bus=pci.0,addr=0x3 \
+-device vfio-pci,host=0021:de:1f.1,id=hostdev0,bus=pci.0,addr=0x3 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
diff --git a/tests/qemuxml2argvdata/hostdev-vfio-multidomain.xml
b/tests/qemuxml2argvdata/hostdev-vfio-multidomain.xml
index 832458125b..7c34b65c7f 100644
--- a/tests/qemuxml2argvdata/hostdev-vfio-multidomain.xml
+++ b/tests/qemuxml2argvdata/hostdev-vfio-multidomain.xml
@@ -25,7 +25,7 @@
<hostdev mode='subsystem' type='pci' managed='yes'>
<driver name='vfio'/>
<source>
- <address domain='0x55aa' bus='32' slot='15'
function='3'/>
+ <address domain='0x0021' bus='222' slot='31'
function='1'/>
</source>
</hostdev>
<memballoon model='virtio'/>
diff --git a/tests/qemuxml2argvdata/hostdev-vfio.args
b/tests/qemuxml2argvdata/hostdev-vfio.args
index b86181e267..e7456d0e49 100644
--- a/tests/qemuxml2argvdata/hostdev-vfio.args
+++ b/tests/qemuxml2argvdata/hostdev-vfio.args
@@ -27,5 +27,5 @@ server,nowait \
-usb \
-drive file=/dev/HostVG/QEMUGuest2,format=raw,if=none,id=drive-ide0-0-0 \
-device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \
--device vfio-pci,host=06:12.5,id=hostdev0,bus=pci.0,addr=0x3 \
+-device vfio-pci,host=06:12.1,id=hostdev0,bus=pci.0,addr=0x3 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
diff --git a/tests/qemuxml2argvdata/hostdev-vfio.xml
b/tests/qemuxml2argvdata/hostdev-vfio.xml
index 4d96b9f732..b2e6c62e7e 100644
--- a/tests/qemuxml2argvdata/hostdev-vfio.xml
+++ b/tests/qemuxml2argvdata/hostdev-vfio.xml
@@ -27,7 +27,7 @@
<hostdev mode='subsystem' type='pci' managed='yes'>
<driver name='vfio'/>
<source>
- <address domain='0x0000' bus='0x06' slot='0x12'
function='0x5'/>
+ <address domain='0x0000' bus='0x06' slot='0x12'
function='0x1'/>
</source>
</hostdev>
<memballoon model='virtio'/>
diff --git a/tests/qemuxml2argvdata/net-hostdev-fail.xml
b/tests/qemuxml2argvdata/net-hostdev-fail.xml
index c815d68bd9..50b102c658 100644
--- a/tests/qemuxml2argvdata/net-hostdev-fail.xml
+++ b/tests/qemuxml2argvdata/net-hostdev-fail.xml
@@ -25,7 +25,7 @@
<interface type='hostdev' managed='yes'>
<mac address='00:11:22:33:44:55'/>
<source>
- <address type='pci' domain='0x0000' bus='0x03'
slot='0x07' function='0x1'/>
+ <address type='pci' domain='0x0000' bus='0x06'
slot='0x12' function='0x1'/>
</source>
<model type='virtio'/>
<filterref filter='myfilter'/>
diff --git a/tests/qemuxml2argvdata/net-hostdev-vfio.args
b/tests/qemuxml2argvdata/net-hostdev-vfio.args
index fcc8716fdf..a7fb5819ac 100644
--- a/tests/qemuxml2argvdata/net-hostdev-vfio.args
+++ b/tests/qemuxml2argvdata/net-hostdev-vfio.args
@@ -27,5 +27,5 @@ server,nowait \
-usb \
-drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-ide0-0-0 \
-device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \
--device vfio-pci,host=03:07.1,id=hostdev0,bus=pci.0,addr=0x3 \
+-device vfio-pci,host=06:12.1,id=hostdev0,bus=pci.0,addr=0x3 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
diff --git a/tests/qemuxml2argvdata/net-hostdev-vfio.xml
b/tests/qemuxml2argvdata/net-hostdev-vfio.xml
index 24034cad8b..aff681c0fb 100644
--- a/tests/qemuxml2argvdata/net-hostdev-vfio.xml
+++ b/tests/qemuxml2argvdata/net-hostdev-vfio.xml
@@ -26,7 +26,7 @@
<mac address='00:11:22:33:44:55'/>
<driver name='vfio'/>
<source>
- <address type='pci' domain='0x0000' bus='0x03'
slot='0x07' function='0x1'/>
+ <address type='pci' domain='0x0000' bus='0x06'
slot='0x12' function='0x1'/>
</source>
<vlan>
<tag id='42'/>
diff --git a/tests/qemuxml2xmloutdata/hostdev-vfio.xml
b/tests/qemuxml2xmloutdata/hostdev-vfio.xml
index 77bd62a129..15a845f4d1 100644
--- a/tests/qemuxml2xmloutdata/hostdev-vfio.xml
+++ b/tests/qemuxml2xmloutdata/hostdev-vfio.xml
@@ -32,7 +32,7 @@
<hostdev mode='subsystem' type='pci' managed='yes'>
<driver name='vfio'/>
<source>
- <address domain='0x0000' bus='0x06' slot='0x12'
function='0x5'/>
+ <address domain='0x0000' bus='0x06' slot='0x12'
function='0x1'/>
</source>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
</hostdev>
diff --git a/tests/qemuxml2xmloutdata/net-hostdev-vfio.xml
b/tests/qemuxml2xmloutdata/net-hostdev-vfio.xml
index 0523cd8d3b..3f057a8249 100644
--- a/tests/qemuxml2xmloutdata/net-hostdev-vfio.xml
+++ b/tests/qemuxml2xmloutdata/net-hostdev-vfio.xml
@@ -31,7 +31,7 @@
<mac address='00:11:22:33:44:55'/>
<driver name='vfio'/>
<source>
- <address type='pci' domain='0x0000' bus='0x03'
slot='0x07' function='0x1'/>
+ <address type='pci' domain='0x0000' bus='0x06'
slot='0x12' function='0x1'/>
</source>
<vlan>
<tag id='42'/>
diff --git a/tests/virpcimock.c b/tests/virpcimock.c
index deb802e860..aa1858c4a6 100644
--- a/tests/virpcimock.c
+++ b/tests/virpcimock.c
@@ -122,6 +122,7 @@ struct pciDevice {
int device;
int klass;
int iommuGroup;
+ const char *physfn;
struct pciDriver *driver; /* Driver attached. NULL if attached to no driver */
};
@@ -429,6 +430,15 @@ pci_device_new_from_stub(const struct pciDevice *data)
}
make_symlink(devpath, "iommu_group", tmp);
+ if (dev->physfn) {
+ if (snprintf(tmp, sizeof(tmp),
+ "%s/devices/%s", fakesysfspcidir, dev->physfn) < 0)
{
+ ABORT("@tmp overflow");
+ }
+ make_symlink(devpath, "physfn", tmp);
+ }
+
+
if (pci_device_autobind(dev) < 0)
ABORT("Unable to bind: %s", data->id);
@@ -1003,6 +1013,12 @@ init_env(void)
MAKE_PCI_DEVICE("0000:0a:01.0", 0x8086, 0x0047, .iommuGroup = 8);
MAKE_PCI_DEVICE("0000:0a:02.0", 0x8286, 0x0048, .iommuGroup = 8);
MAKE_PCI_DEVICE("0000:0a:03.0", 0x8386, 0x0048, .iommuGroup = 8);
+ MAKE_PCI_DEVICE("0000:06:12.0", 0x8086, 0x0047, .iommuGroup = 9);
+ MAKE_PCI_DEVICE("0000:06:12.1", 0x8086, 0x0047, .iommuGroup = 10, .physfn =
"0000:06:12.0"); /* Virtual Function */
+ MAKE_PCI_DEVICE("0000:06:12.2", 0x8086, 0x0047, .iommuGroup = 11, .physfn =
"0000:06:12.0"); /* Virtual Function */
+ MAKE_PCI_DEVICE("0021:de:1f.0", 0x8086, 0x0047, .iommuGroup = 12);
+ MAKE_PCI_DEVICE("0021:de:1f.1", 0x8086, 0x0047, .iommuGroup = 13, .physfn =
"0021:de:1f.0"); /* Virtual Function */
+
}
diff --git a/tests/virpcitestdata/0000-06-12.0.config
b/tests/virpcitestdata/0000-06-12.0.config
new file mode 100644
index 0000000000000000000000000000000000000000..beee76534041a7020c08ae9ac03d9a349c6ea12e
GIT binary patch
literal 256
ycmXpOFlAt45MXi^VCG@qU|?VnU}yr8Sb;H6EeJS(Ng%<*sKv;@R0rb@MH&E<&;s)S
literal 0
HcmV?d00001
diff --git a/tests/virpcitestdata/0000-06-12.1.config
b/tests/virpcitestdata/0000-06-12.1.config
new file mode 100644
index 0000000000000000000000000000000000000000..beee76534041a7020c08ae9ac03d9a349c6ea12e
GIT binary patch
literal 256
ycmXpOFlAt45MXi^VCG@qU|?VnU}yr8Sb;H6EeJS(Ng%<*sKv;@R0rb@MH&E<&;s)S
literal 0
HcmV?d00001
diff --git a/tests/virpcitestdata/0000-06-12.2.config
b/tests/virpcitestdata/0000-06-12.2.config
new file mode 100644
index 0000000000000000000000000000000000000000..beee76534041a7020c08ae9ac03d9a349c6ea12e
GIT binary patch
literal 256
ycmXpOFlAt45MXi^VCG@qU|?VnU}yr8Sb;H6EeJS(Ng%<*sKv;@R0rb@MH&E<&;s)S
literal 0
HcmV?d00001
--
2.21.0
1:09 p.m.
New subject: [libvirt] [PATCH v2 6/7] tests: qemu: Add test case for pci-hostdev hotplug
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
src/util/virprocess.h | 2 +-
tests/Makefile.am | 7 +++
tests/qemuhotplugtest.c | 42 +++++++++++++-
.../qemuhotplug-hostdev-pci.xml | 6 ++
.../qemuhotplug-base-live+hostdev-pci.xml | 58 +++++++++++++++++++
...uhotplug-pseries-base-live+hostdev-pci.xml | 51 ++++++++++++++++
.../qemuhotplug-pseries-base-live.xml | 43 ++++++++++++++
tests/virprocessmock.c | 28 +++++++++
8 files changed, 235 insertions(+), 2 deletions(-)
create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-hostdev-pci.xml
create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+hostdev-pci.xml
create mode 100644
tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+hostdev-pci.xml
create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live.xml
create mode 100644 tests/virprocessmock.c
diff --git a/src/util/virprocess.h b/src/util/virprocess.h
index 003ba1edf4..4806c592da 100644
--- a/src/util/virprocess.h
+++ b/src/util/virprocess.h
@@ -75,7 +75,7 @@ int virProcessGetNamespaces(pid_t pid,
int virProcessSetNamespaces(size_t nfdlist,
int *fdlist);
-int virProcessSetMaxMemLock(pid_t pid, unsigned long long bytes);
+int virProcessSetMaxMemLock(pid_t pid, unsigned long long bytes) ATTRIBUTE_NOINLINE;
int virProcessSetMaxProcesses(pid_t pid, unsigned int procs);
int virProcessSetMaxFiles(pid_t pid, unsigned int files);
int virProcessSetMaxCoreSize(pid_t pid, unsigned long long bytes);
diff --git a/tests/Makefile.am b/tests/Makefile.am
index f480e68e7d..66d249a0cc 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -210,6 +210,7 @@ test_libraries = libshunload.la \
virpcimock.la \
virnetdevmock.la \
virrandommock.la \
+ virprocessmock.la \
virhostcpumock.la \
domaincapsmock.la \
virfilecachemock.la \
@@ -1182,6 +1183,12 @@ virrandommock_la_SOURCES = \
virrandommock_la_LDFLAGS = $(MOCKLIBS_LDFLAGS)
virrandommock_la_LIBADD = $(MOCKLIBS_LIBS)
+virprocessmock_la_SOURCES = \
+ virprocessmock.c
+virprocessmock_la_CFLAGS = $(AM_CFLAGS)
+virprocessmock_la_LDFLAGS = $(MOCKLIBS_LDFLAGS)
+virprocessmock_la_LIBADD = $(MOCKLIBS_LIBS)
+
virhostcpumock_la_SOURCES = \
virhostcpumock.c
virhostcpumock_la_LDFLAGS = $(MOCKLIBS_LDFLAGS)
diff --git a/tests/qemuhotplugtest.c b/tests/qemuhotplugtest.c
index 354068d748..bf95bfb4a7 100644
--- a/tests/qemuhotplugtest.c
+++ b/tests/qemuhotplugtest.c
@@ -28,6 +28,7 @@
#include "testutils.h"
#include "testutilsqemu.h"
#include "testutilsqemuschema.h"
+#include "virhostdev.h"
#include "virerror.h"
#include "virstring.h"
#include "virthread.h"
@@ -79,6 +80,8 @@ qemuHotplugCreateObjects(virDomainXMLOptionPtr xmlopt,
virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_DEVICE_IVSHMEM_PLAIN);
virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_DEVICE_IVSHMEM_DOORBELL);
virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_SCSI_DISK_WWN);
+ virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_DEVICE_VFIO_PCI);
+ virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE);
if (qemuTestCapsCacheInsert(driver.qemuCapsCache, priv->qemuCaps) < 0)
goto cleanup;
@@ -130,6 +133,9 @@ testQemuHotplugAttach(virDomainObjPtr vm,
case VIR_DOMAIN_DEVICE_WATCHDOG:
ret = qemuDomainAttachWatchdog(&driver, vm, dev->data.watchdog);
break;
+ case VIR_DOMAIN_DEVICE_HOSTDEV:
+ ret = qemuDomainAttachHostDevice(&driver, vm, dev->data.hostdev);
+ break;
default:
VIR_TEST_VERBOSE("device type '%s' cannot be attached\n",
virDomainDeviceTypeToString(dev->type));
@@ -151,6 +157,7 @@ testQemuHotplugDetach(virDomainObjPtr vm,
case VIR_DOMAIN_DEVICE_CHR:
case VIR_DOMAIN_DEVICE_SHMEM:
case VIR_DOMAIN_DEVICE_WATCHDOG:
+ case VIR_DOMAIN_DEVICE_HOSTDEV:
ret = qemuDomainDetachDeviceLive(vm, dev, &driver, async);
break;
default:
@@ -578,6 +585,7 @@ testQemuHotplugCpuIndividual(const void *opaque)
return ret;
}
+#define FAKEROOTDIRTEMPLATE abs_builddir "/fakerootdir-XXXXXX"
static int
@@ -587,6 +595,21 @@ mymain(void)
int ret = 0;
struct qemuHotplugTestData data = {0};
struct testQemuHotplugCpuParams cpudata;
+ char *fakerootdir;
+
+ if (VIR_STRDUP_QUIET(fakerootdir, FAKEROOTDIRTEMPLATE) < 0) {
+ fprintf(stderr, "Out of memory\n");
+ abort();
+ }
+
+ if (!mkdtemp(fakerootdir)) {
+ fprintf(stderr, "Cannot create fakerootdir");
+ abort();
+ }
+
+ setenv("LIBVIRT_FAKE_ROOT_DIR", fakerootdir, 1);
+ unsetenv("LD_PRELOAD");
+
#if !WITH_YAJL
fputs("libvirt not compiled with JSON support, skipping this test\n",
stderr);
@@ -621,6 +644,8 @@ mymain(void)
if (!driver.lockManager)
return EXIT_FAILURE;
+ driver.hostdevMgr = virHostdevManagerGetDefault();
+
/* wait only 100ms for DEVICE_DELETED event */
qemuDomainRemoveDeviceWaitTime = 100;
@@ -789,6 +814,15 @@ mymain(void)
"human-monitor-command", HMP("OK\\r\\n"),
"device_add", QMP_OK);
+ DO_TEST_ATTACH("base-live", "hostdev-pci", false, true,
+ "device_add", QMP_OK);
+ DO_TEST_DETACH("base-live", "hostdev-pci", false, false,
+ "device_del", QMP_DEVICE_DELETED("hostdev0")
QMP_OK);
+ DO_TEST_ATTACH("pseries-base-live", "hostdev-pci", false, true,
+ "device_add", QMP_OK);
+ DO_TEST_DETACH("pseries-base-live", "hostdev-pci", false, false,
+ "device_del", QMP_DEVICE_DELETED("hostdev0")
QMP_OK);
+
DO_TEST_ATTACH("base-live", "watchdog", false, true,
"watchdog-set-action", QMP_OK,
"device_add", QMP_OK);
@@ -846,9 +880,15 @@ mymain(void)
DO_TEST_CPU_INDIVIDUAL("ppc64-modern-individual", "16-22", true,
true, true);
DO_TEST_CPU_INDIVIDUAL("ppc64-modern-individual", "17", true,
true, true);
+ if (getenv("LIBVIRT_SKIP_CLEANUP") == NULL)
+ virFileDeleteTree(fakerootdir);
+ VIR_FREE(fakerootdir);
+
qemuTestDriverFree(&driver);
virObjectUnref(data.vm);
return (ret == 0) ? EXIT_SUCCESS : EXIT_FAILURE;
}
-VIR_TEST_MAIN(mymain)
+VIR_TEST_MAIN_PRELOAD(mymain,
+ abs_builddir "/.libs/virpcimock.so",
+ abs_builddir "/.libs/virprocessmock.so");
diff --git a/tests/qemuhotplugtestdevices/qemuhotplug-hostdev-pci.xml
b/tests/qemuhotplugtestdevices/qemuhotplug-hostdev-pci.xml
new file mode 100644
index 0000000000..6f7c99c943
--- /dev/null
+++ b/tests/qemuhotplugtestdevices/qemuhotplug-hostdev-pci.xml
@@ -0,0 +1,6 @@
+<hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0000' bus='0x06' slot='0x12'
function='0x2'/>
+ </source>
+</hostdev>
diff --git a/tests/qemuhotplugtestdomains/qemuhotplug-base-live+hostdev-pci.xml
b/tests/qemuhotplugtestdomains/qemuhotplug-base-live+hostdev-pci.xml
new file mode 100644
index 0000000000..40dcc33595
--- /dev/null
+++ b/tests/qemuhotplugtestdomains/qemuhotplug-base-live+hostdev-pci.xml
@@ -0,0 +1,58 @@
+<domain type='kvm' id='7'>
+ <name>hotplug</name>
+ <uuid>d091ea82-29e6-2e34-3005-f02617b36e87</uuid>
+ <memory unit='KiB'>4194304</memory>
+ <currentMemory unit='KiB'>4194304</currentMemory>
+ <vcpu placement='static'>4</vcpu>
+ <os>
+ <type arch='x86_64' machine='pc'>hvm</type>
+ <boot dev='hd'/>
+ </os>
+ <features>
+ <acpi/>
+ <apic/>
+ <pae/>
+ </features>
+ <clock offset='utc'/>
+ <on_poweroff>destroy</on_poweroff>
+ <on_reboot>restart</on_reboot>
+ <on_crash>restart</on_crash>
+ <devices>
+ <emulator>/usr/bin/qemu-system-x86_64</emulator>
+ <controller type='usb' index='0'>
+ <alias name='usb'/>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x2'/>
+ </controller>
+ <controller type='ide' index='0'>
+ <alias name='ide'/>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x1'/>
+ </controller>
+ <controller type='scsi' index='0' model='virtio-scsi'>
+ <alias name='scsi0'/>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
+ </controller>
+ <controller type='pci' index='0' model='pci-root'>
+ <alias name='pci'/>
+ </controller>
+ <controller type='virtio-serial' index='0'>
+ <alias name='virtio-serial0'/>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x04' function='0x0'/>
+ </controller>
+ <input type='mouse' bus='ps2'>
+ <alias name='input0'/>
+ </input>
+ <input type='keyboard' bus='ps2'>
+ <alias name='input1'/>
+ </input>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0000' bus='0x06' slot='0x12'
function='0x2'/>
+ </source>
+ <alias name='hostdev0'/>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x05' function='0x0'/>
+ </hostdev>
+ <memballoon model='none'/>
+ </devices>
+ <seclabel type='none' model='none'/>
+</domain>
diff --git a/tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+hostdev-pci.xml
b/tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+hostdev-pci.xml
new file mode 100644
index 0000000000..483cb5d8f0
--- /dev/null
+++ b/tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+hostdev-pci.xml
@@ -0,0 +1,51 @@
+<domain type='kvm' id='7'>
+ <name>hotplug</name>
+ <uuid>d091ea82-29e6-2e34-3005-f02617b36e87</uuid>
+ <memory unit='KiB'>4194304</memory>
+ <currentMemory unit='KiB'>4194304</currentMemory>
+ <vcpu placement='static'>4</vcpu>
+ <os>
+ <type arch='ppc64' machine='pseries'>hvm</type>
+ <boot dev='hd'/>
+ </os>
+ <clock offset='utc'/>
+ <on_poweroff>destroy</on_poweroff>
+ <on_reboot>restart</on_reboot>
+ <on_crash>destroy</on_crash>
+ <devices>
+ <emulator>/usr/bin/qemu-system-ppc64</emulator>
+ <controller type='pci' index='0' model='pci-root'>
+ <model name='spapr-pci-host-bridge'/>
+ <target index='0'/>
+ <alias name='pci.0'/>
+ </controller>
+ <controller type='pci' index='1' model='pci-root'>
+ <model name='spapr-pci-host-bridge'/>
+ <target index='1'/>
+ <alias name='pci.1'/>
+ </controller>
+ <controller type='usb' index='0'>
+ <alias name='usb'/>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x0'/>
+ </controller>
+ <input type='keyboard' bus='usb'>
+ <alias name='input0'/>
+ <address type='usb' bus='0' port='1'/>
+ </input>
+ <input type='mouse' bus='usb'>
+ <alias name='input1'/>
+ <address type='usb' bus='0' port='2'/>
+ </input>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0000' bus='0x06' slot='0x12'
function='0x2'/>
+ </source>
+ <alias name='hostdev0'/>
+ <address type='pci' domain='0x0000' bus='0x01'
slot='0x01' function='0x0'/>
+ </hostdev>
+ <memballoon model='none'/>
+ <panic model='pseries'/>
+ </devices>
+ <seclabel type='none' model='none'/>
+</domain>
diff --git a/tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live.xml
b/tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live.xml
new file mode 100644
index 0000000000..1dcc35f626
--- /dev/null
+++ b/tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live.xml
@@ -0,0 +1,43 @@
+<domain type='kvm' id='7'>
+ <name>hotplug</name>
+ <uuid>d091ea82-29e6-2e34-3005-f02617b36e87</uuid>
+ <memory unit='KiB'>4194304</memory>
+ <currentMemory unit='KiB'>4194304</currentMemory>
+ <vcpu placement='static'>4</vcpu>
+ <os>
+ <type arch='ppc64' machine='pseries'>hvm</type>
+ <boot dev='hd'/>
+ </os>
+ <clock offset='utc'/>
+ <on_poweroff>destroy</on_poweroff>
+ <on_reboot>restart</on_reboot>
+ <on_crash>destroy</on_crash>
+ <devices>
+ <emulator>/usr/bin/qemu-system-ppc64</emulator>
+ <controller type='pci' index='0' model='pci-root'>
+ <model name='spapr-pci-host-bridge'/>
+ <target index='0'/>
+ <alias name='pci.0'/>
+ </controller>
+ <controller type='pci' index='1' model='pci-root'>
+ <model name='spapr-pci-host-bridge'/>
+ <target index='1'/>
+ <alias name='pci.1'/>
+ </controller>
+ <controller type='usb' index='0'>
+ <alias name='usb'/>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x0'/>
+ </controller>
+ <input type='keyboard' bus='usb'>
+ <alias name='input0'/>
+ <address type='usb' bus='0' port='1'/>
+ </input>
+ <input type='mouse' bus='usb'>
+ <alias name='input1'/>
+ <address type='usb' bus='0' port='2'/>
+ </input>
+ <memballoon model='none'/>
+ <panic model='pseries'/>
+ </devices>
+ <seclabel type='none' model='none'/>
+</domain>
diff --git a/tests/virprocessmock.c b/tests/virprocessmock.c
new file mode 100644
index 0000000000..985562fee1
--- /dev/null
+++ b/tests/virprocessmock.c
@@ -0,0 +1,28 @@
+/*
+ * Copyright (C) 2018 Red Hat, Inc.
+ * Copyright (C) 2018 IBM Corp.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library. If not, see
+ * <http://www.gnu.org/licenses/>;.
+ *
+ */
+
+#include <config.h>
+#include "virprocess.h"
+
+int
+virProcessSetMaxMemLock(pid_t pid ATTRIBUTE_UNUSED, unsigned long long bytes
ATTRIBUTE_UNUSED)
+{
+ return 0;
+}
--
2.21.0
7:43 a.m.
New subject: [libvirt] [PATCH v2 6/7] tests: qemu: Add test case for pci-hostdev hotplug
On 7/25/19 8:09 PM, Daniel Henrique Barboza wrote:
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
src/util/virprocess.h | 2 +-
tests/Makefile.am | 7 +++
tests/qemuhotplugtest.c | 42 +++++++++++++-
.../qemuhotplug-hostdev-pci.xml | 6 ++
.../qemuhotplug-base-live+hostdev-pci.xml | 58 +++++++++++++++++++
...uhotplug-pseries-base-live+hostdev-pci.xml | 51 ++++++++++++++++
.../qemuhotplug-pseries-base-live.xml | 43 ++++++++++++++
tests/virprocessmock.c | 28 +++++++++
8 files changed, 235 insertions(+), 2 deletions(-)
create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-hostdev-pci.xml
create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+hostdev-pci.xml
create mode 100644
tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live+hostdev-pci.xml
create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-pseries-base-live.xml
create mode 100644 tests/virprocessmock.c
diff --git a/src/util/virprocess.h b/src/util/virprocess.h
index 003ba1edf4..4806c592da 100644
--- a/src/util/virprocess.h
+++ b/src/util/virprocess.h
@@ -75,7 +75,7 @@ int virProcessGetNamespaces(pid_t pid,
int virProcessSetNamespaces(size_t nfdlist,
int *fdlist);
-int virProcessSetMaxMemLock(pid_t pid, unsigned long long bytes);
+int virProcessSetMaxMemLock(pid_t pid, unsigned long long bytes) ATTRIBUTE_NOINLINE;
int virProcessSetMaxProcesses(pid_t pid, unsigned int procs);
int virProcessSetMaxFiles(pid_t pid, unsigned int files);
int virProcessSetMaxCoreSize(pid_t pid, unsigned long long bytes);
diff --git a/tests/Makefile.am b/tests/Makefile.am
index f480e68e7d..66d249a0cc 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -210,6 +210,7 @@ test_libraries = libshunload.la \
virpcimock.la \
virnetdevmock.la \
virrandommock.la \
+ virprocessmock.la \
virhostcpumock.la \
domaincapsmock.la \
virfilecachemock.la \
@@ -1182,6 +1183,12 @@ virrandommock_la_SOURCES = \
virrandommock_la_LDFLAGS = $(MOCKLIBS_LDFLAGS)
virrandommock_la_LIBADD = $(MOCKLIBS_LIBS)
+virprocessmock_la_SOURCES = \
+ virprocessmock.c
+virprocessmock_la_CFLAGS = $(AM_CFLAGS)
+virprocessmock_la_LDFLAGS = $(MOCKLIBS_LDFLAGS)
+virprocessmock_la_LIBADD = $(MOCKLIBS_LIBS)
+
virhostcpumock_la_SOURCES = \
virhostcpumock.c
virhostcpumock_la_LDFLAGS = $(MOCKLIBS_LDFLAGS)
diff --git a/tests/qemuhotplugtest.c b/tests/qemuhotplugtest.c
index 354068d748..bf95bfb4a7 100644
--- a/tests/qemuhotplugtest.c
+++ b/tests/qemuhotplugtest.c
@@ -28,6 +28,7 @@
#include "testutils.h"
#include "testutilsqemu.h"
#include "testutilsqemuschema.h"
+#include "virhostdev.h"
#include "virerror.h"
#include "virstring.h"
#include "virthread.h"
@@ -79,6 +80,8 @@ qemuHotplugCreateObjects(virDomainXMLOptionPtr xmlopt,
virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_DEVICE_IVSHMEM_PLAIN);
virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_DEVICE_IVSHMEM_DOORBELL);
virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_SCSI_DISK_WWN);
+ virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_DEVICE_VFIO_PCI);
+ virQEMUCapsSet(priv->qemuCaps, QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE);
if (qemuTestCapsCacheInsert(driver.qemuCapsCache, priv->qemuCaps) < 0)
goto cleanup;
@@ -130,6 +133,9 @@ testQemuHotplugAttach(virDomainObjPtr vm,
case VIR_DOMAIN_DEVICE_WATCHDOG:
ret = qemuDomainAttachWatchdog(&driver, vm, dev->data.watchdog);
break;
+ case VIR_DOMAIN_DEVICE_HOSTDEV:
+ ret = qemuDomainAttachHostDevice(&driver, vm, dev->data.hostdev);
+ break;
default:
VIR_TEST_VERBOSE("device type '%s' cannot be attached\n",
virDomainDeviceTypeToString(dev->type));
@@ -151,6 +157,7 @@ testQemuHotplugDetach(virDomainObjPtr vm,
case VIR_DOMAIN_DEVICE_CHR:
case VIR_DOMAIN_DEVICE_SHMEM:
case VIR_DOMAIN_DEVICE_WATCHDOG:
+ case VIR_DOMAIN_DEVICE_HOSTDEV:
ret = qemuDomainDetachDeviceLive(vm, dev, &driver, async);
break;
default:
@@ -578,6 +585,7 @@ testQemuHotplugCpuIndividual(const void *opaque)
return ret;
}
+#define FAKEROOTDIRTEMPLATE abs_builddir "/fakerootdir-XXXXXX"
static int
@@ -587,6 +595,21 @@ mymain(void)
int ret = 0;
struct qemuHotplugTestData data = {0};
struct testQemuHotplugCpuParams cpudata;
+ char *fakerootdir;
+
+ if (VIR_STRDUP_QUIET(fakerootdir, FAKEROOTDIRTEMPLATE) < 0) {
+ fprintf(stderr, "Out of memory\n");
+ abort();
+ }
+
+ if (!mkdtemp(fakerootdir)) {
+ fprintf(stderr, "Cannot create fakerootdir");
+ abort();
+ }
+
+ setenv("LIBVIRT_FAKE_ROOT_DIR", fakerootdir, 1);
+ unsetenv("LD_PRELOAD");
This unsetenv is not necessary, isn't it?
Reviewed-by: Michal Privoznik <mprivozn(a)redhat.com>
Michal
1:09 p.m.
New subject: [libvirt] [PATCH v2 7/7] tests: Add a baseline test for multifunction pci device use case
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
There are already good number of test cases with hostdevices,
few have multifunction devices but none having more than one
than one multifunction cards.
This patch adds a case where there are two multifunction cards
and two Virtual functions part of the same XML.
0001:01:00.X & 0005:09:00.X - are Multifunction PCI cards.
0000:06:12.[5|6] - are SRIOV Virtual functions
The next few commits improve on automatically detecting the multifunction
cards and auto-assinging the addresses appropriately.
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
.../hostdev-pci-multifunction.args | 35 ++++++++
.../hostdev-pci-multifunction.xml | 59 ++++++++++++++
tests/qemuxml2argvtest.c | 3 +
.../hostdev-pci-multifunction.xml | 79 +++++++++++++++++++
tests/qemuxml2xmltest.c | 1 +
5 files changed, 177 insertions(+)
create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction.args
create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction.xml
create mode 100644 tests/qemuxml2xmloutdata/hostdev-pci-multifunction.xml
diff --git a/tests/qemuxml2argvdata/hostdev-pci-multifunction.args
b/tests/qemuxml2argvdata/hostdev-pci-multifunction.args
new file mode 100644
index 0000000000..7747cca442
--- /dev/null
+++ b/tests/qemuxml2argvdata/hostdev-pci-multifunction.args
@@ -0,0 +1,35 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/tmp/lib/domain--1-delete \
+USER=test \
+LOGNAME=test \
+XDG_DATA_HOME=/tmp/lib/domain--1-delete/.local/share \
+XDG_CACHE_HOME=/tmp/lib/domain--1-delete/.cache \
+XDG_CONFIG_HOME=/tmp/lib/domain--1-delete/.config \
+QEMU_AUDIO_DRV=none \
+/usr/bin/qemu-system-x86_64 \
+-name delete \
+-S \
+-machine pc,accel=kvm,usb=off,dump-guest-core=off \
+-m 256 \
+-realtime mlock=off \
+-smp 4,sockets=4,cores=1,threads=1 \
+-uuid 583a8e8e-f0ce-4f53-89ab-092862148b25 \
+-display none \
+-no-user-config \
+-nodefaults \
+-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-delete/monitor.sock,\
+server,nowait \
+-mon chardev=charmonitor,id=monitor,mode=control \
+-rtc base=utc \
+-no-shutdown \
+-no-acpi \
+-usb \
+-device vfio-pci,host=0005:90:01.0,id=hostdev0,bus=pci.0,addr=0x3 \
+-device vfio-pci,host=0001:01:00.1,id=hostdev1,bus=pci.0,addr=0x4 \
+-device vfio-pci,host=0001:01:00.0,id=hostdev2,bus=pci.0,addr=0x5 \
+-device vfio-pci,host=0005:90:01.2,id=hostdev3,bus=pci.0,addr=0x6 \
+-device vfio-pci,host=0005:90:01.3,id=hostdev4,bus=pci.0,addr=0x7 \
+-device vfio-pci,host=06:12.1,id=hostdev5,bus=pci.0,addr=0x8 \
+-device vfio-pci,host=06:12.2,id=hostdev6,bus=pci.0,addr=0x9 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0xa
diff --git a/tests/qemuxml2argvdata/hostdev-pci-multifunction.xml
b/tests/qemuxml2argvdata/hostdev-pci-multifunction.xml
new file mode 100644
index 0000000000..06c889c64d
--- /dev/null
+++ b/tests/qemuxml2argvdata/hostdev-pci-multifunction.xml
@@ -0,0 +1,59 @@
+<domain type='kvm'>
+ <name>delete</name>
+ <uuid>583a8e8e-f0ce-4f53-89ab-092862148b25</uuid>
+ <memory unit='KiB'>262144</memory>
+ <vcpu placement='static'>4</vcpu>
+ <os>
+ <type arch='x86_64' machine='pc'>hvm</type>
+ </os>
+ <devices>
+ <emulator>/usr/bin/qemu-system-x86_64</emulator>
+ <controller type='usb' index='0'/>
+ <controller type='ide' index='0'/>
+ <controller type='pci' index='0' model='pci-root'/>
+ <input type='mouse' bus='ps2'/>
+ <input type='keyboard' bus='ps2'/>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0005' bus='0x90' slot='0x01'
function='0x0'/>
+ </source>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0001' bus='0x01' slot='0x00'
function='0x1'/>
+ </source>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0001' bus='0x01' slot='0x00'
function='0x0'/>
+ </source>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0005' bus='0x90' slot='0x01'
function='0x2'/>
+ </source>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0005' bus='0x90' slot='0x01'
function='0x3'/>
+ </source>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0000' bus='0x06' slot='0x12'
function='0x1'/>
+ </source>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0000' bus='0x06' slot='0x12'
function='0x2'/>
+ </source>
+ </hostdev>
+ </devices>
+</domain>
diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c
index d6e6272518..5feb305f41 100644
--- a/tests/qemuxml2argvtest.c
+++ b/tests/qemuxml2argvtest.c
@@ -1313,6 +1313,9 @@ mymain(void)
DO_TEST_FAILURE("net-hostdev-fail",
QEMU_CAPS_DEVICE_VFIO_PCI);
+ DO_TEST("hostdev-pci-multifunction",
+ QEMU_CAPS_KVM,
+ QEMU_CAPS_DEVICE_VFIO_PCI);
DO_TEST("serial-file-log",
QEMU_CAPS_CHARDEV_FILE_APPEND,
diff --git a/tests/qemuxml2xmloutdata/hostdev-pci-multifunction.xml
b/tests/qemuxml2xmloutdata/hostdev-pci-multifunction.xml
new file mode 100644
index 0000000000..52ed86e305
--- /dev/null
+++ b/tests/qemuxml2xmloutdata/hostdev-pci-multifunction.xml
@@ -0,0 +1,79 @@
+<domain type='kvm'>
+ <name>delete</name>
+ <uuid>583a8e8e-f0ce-4f53-89ab-092862148b25</uuid>
+ <memory unit='KiB'>262144</memory>
+ <currentMemory unit='KiB'>262144</currentMemory>
+ <vcpu placement='static'>4</vcpu>
+ <os>
+ <type arch='x86_64' machine='pc'>hvm</type>
+ <boot dev='hd'/>
+ </os>
+ <clock offset='utc'/>
+ <on_poweroff>destroy</on_poweroff>
+ <on_reboot>restart</on_reboot>
+ <on_crash>destroy</on_crash>
+ <devices>
+ <emulator>/usr/bin/qemu-system-x86_64</emulator>
+ <controller type='usb' index='0'>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x2'/>
+ </controller>
+ <controller type='ide' index='0'>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x1'/>
+ </controller>
+ <controller type='pci' index='0' model='pci-root'/>
+ <input type='mouse' bus='ps2'/>
+ <input type='keyboard' bus='ps2'/>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0005' bus='0x90' slot='0x01'
function='0x0'/>
+ </source>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0001' bus='0x01' slot='0x00'
function='0x1'/>
+ </source>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x04' function='0x0'/>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0001' bus='0x01' slot='0x00'
function='0x0'/>
+ </source>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x05' function='0x0'/>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0005' bus='0x90' slot='0x01'
function='0x2'/>
+ </source>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x06' function='0x0'/>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0005' bus='0x90' slot='0x01'
function='0x3'/>
+ </source>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x07' function='0x0'/>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0000' bus='0x06' slot='0x12'
function='0x1'/>
+ </source>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x08' function='0x0'/>
+ </hostdev>
+ <hostdev mode='subsystem' type='pci' managed='yes'>
+ <driver name='vfio'/>
+ <source>
+ <address domain='0x0000' bus='0x06' slot='0x12'
function='0x2'/>
+ </source>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x09' function='0x0'/>
+ </hostdev>
+ <memballoon model='virtio'>
+ <address type='pci' domain='0x0000' bus='0x00'
slot='0x0a' function='0x0'/>
+ </memballoon>
+ </devices>
+</domain>
diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c
index 09c86eda2a..b7e1b58887 100644
--- a/tests/qemuxml2xmltest.c
+++ b/tests/qemuxml2xmltest.c
@@ -427,6 +427,7 @@ mymain(void)
DO_TEST("hostdev-usb-address", NONE);
DO_TEST("hostdev-pci-address", NONE);
+ DO_TEST("hostdev-pci-multifunction", NONE);
DO_TEST("hostdev-vfio", NONE);
DO_TEST("hostdev-vfio-zpci",
QEMU_CAPS_DEVICE_ZPCI,
--
2.21.0
7:43 a.m.
New subject: [libvirt] [PATCH v2 7/7] tests: Add a baseline test for multifunction pci device use case
On 7/25/19 8:09 PM, Daniel Henrique Barboza wrote:
From: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
There are already good number of test cases with hostdevices,
few have multifunction devices but none having more than one
than one multifunction cards.
This patch adds a case where there are two multifunction cards
and two Virtual functions part of the same XML.
0001:01:00.X & 0005:09:00.X - are Multifunction PCI cards.
0000:06:12.[5|6] - are SRIOV Virtual functions
The next few commits improve on automatically detecting the multifunction
cards and auto-assinging the addresses appropriately.
Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com>
---
.../hostdev-pci-multifunction.args | 35 ++++++++
.../hostdev-pci-multifunction.xml | 59 ++++++++++++++
tests/qemuxml2argvtest.c | 3 +
.../hostdev-pci-multifunction.xml | 79 +++++++++++++++++++
tests/qemuxml2xmltest.c | 1 +
5 files changed, 177 insertions(+)
create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction.args
create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction.xml
create mode 100644 tests/qemuxml2xmloutdata/hostdev-pci-multifunction.xml
Reviewed-by: Michal Privoznik <mprivozn(a)redhat.com>
Michal
7:46 a.m.
On 7/25/19 8:09 PM, Daniel Henrique Barboza wrote:
Huh, looks like we have came across same bugs recently. Well, great
minds think alike :D
So I guess the resolution might be this: I'll merge my patches as soon
as I get review and then you can repost your patches (it's going to be
shorter series that way).
Michal
2116
days inactive
2138
days old
14 comments
2 participants
participants (2)
-
Daniel Henrique Barboza -
Michal Privoznik