Re: [libvirt PATCH v3 0/8] Support for VFIO variant drivers, Part 1

On 21/08/2023 20:32, Laine Stump wrote: Don't we have a: <interface type='hostdev' managed='yes'> ... <driver name='vfio'> ... </interface/ ... that could be re-used somehow to specify the VFIO variant driver of choice? Somewhat similar to the nodedev-detach equivalent? OTOH, it would only cover networking hostdevs... :/ Interesting, e1000e and igbvf have quiesce/resume/dirty-tracking commands? Sounds too old to have :D unless of course this is Qemu-created commands and not actual IGB/E1000e hardware doing those Nice Provided the initial use of VFIO variant drivers have been to support live migration, should the hostdev blocker be lifted too *just in case it is simple addition of course* ? In vfio 8.1 it is enabled by default, so there isn't much else to do other than unblock migration with hostdevs. Something like below is <interface> specific; which works but it is likely too specific: diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index d21551ab07d9..6edc8583c539 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1316,13 +1316,15 @@ qemuMigrationSrcIsAllowedHostdev(const virDomainDef *def) continue; } - /* all other PCI hostdevs can't be migrated */ + /* some PCI hostdevs can't be migrated */ if (hostdev->parentnet) { - virDomainNetType actualType = virDomainNetGetActualType(hostdev->parentnet); - - virReportError(VIR_ERR_OPERATION_UNSUPPORTED, - _("cannot migrate a domain with <interface type='%1$s'>"), - virDomainNetTypeToString(actualType)); + /* + * Some VFIO network devices can be migrated if vendor + * VFIO driver is used e.g. via mlx5-vfio-pci. + * We leave it up to the live migration blocker in Qemu to + * stop us from proceeding. + */ + continue; } else { virReportError(VIR_ERR_OPERATION_UNSUPPORTED, _("cannot migrate a domain with <hostdev mode='subsystem' type='%1$s'>"), Or even something like this generic to the hostdev as long as it's assigned to a variant driver. For any off shot case that isn't a migration vfio driver then Qemu will fail for us as it will fail to probe the right things e.g. (below is totally untested, it is just for illustration purposes, we can probably do away with just VIR_DOMAIN_HOSTDEV_PCI_BACKEND_VFIO and let Qemu block migration for us?) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index d21551ab07d9..1f29493aa0c2 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1301,6 +1301,8 @@ qemuMigrationSrcIsAllowedHostdev(const virDomainDef *def) return false; case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI: + g_autoptr(virPCIDevice) pci = NULL; + /* * if the device has a <teaming type='transient'> * element, then migration *is* allowed because the @@ -1316,6 +1318,22 @@ qemuMigrationSrcIsAllowedHostdev(const virDomainDef *def) continue; } + if ((hostdev->source.subsys.u.pci.backend == VIR_DOMAIN_HOSTDEV_PCI_BACKEND_VFIO) && + !virHostdevGetPCIHostDevice(hostdev, &pci) && pci) { + g_autofree char *drvPath = NULL; + g_autofree char *drvName = NULL; + + /* + * Some VFIO network devices can be migrated if non standard + * VFIO driver is used. We leave it up to the live migration + * blocker in Qemu to stop us from proceeding. + */ + if (!virPCIDeviceGetCurrentDriverPathAndName(pci, &drvPath, + &drvName) && + !STREQ(drvName, "vfio-pci")) + continue; + } + /* all other PCI hostdevs can't be migrated */ if (hostdev->parentnet) { virDomainNetType actualType = virDomainNetGetActualType(hostdev->parentnet);