May 2023 - Devel - libvirt List Archives

[libvirt PATCH 00/20] ci: Move GitLab build recipes to a standalone script
by Erik Skultety 24 Aug '23

24 Aug '23

This is a follow up to: https://listman.redhat.com/archives/libvir-list/2023-January/237201.html The effort here is to unify the way builds/tests are executed in GitLab CI vs local container executions and make another step forward in terms of reproducibility of (specifically) GitLab environments. Even though code to run all but one (coverity) jobs from GitLab via the build.sh script is added with this series, local behavior remains the same as before this series. The reason for that is that that will require more patches ridding of the Makefile which is currently used and instead integrate usage of lcitool with the ci/helper Python script which is currently the entry point for local container executions. Pipeline: https://gitlab.com/eskultety/libvirt/-/pipelines/768645158 Ubuntu is having some repo connection issues today, so the one failed ^job can be ignored Erik Skultety (20): gitlab-ci.yml: Replace all explicit calls to ninja with meson commands gitlab-ci.yml: potfile: Consolidate the meson compile calls gitlab-ci.yml: Use $HOME for rpmbuild's topdir instead of PWD ci: build.sh: Drop the commentary about CI_BUILD_SCRIPT ci: build.sh: Use 'meson setup' explicitly ci: build.sh: Always assume -Dsystem=true ci: build.sh: Drop the CI prefix from the CI_{MESON,NINJA}_ARGS vars ci: build.sh: Move off of ninja command to directly calling meson ci: build.sh: Join MESON_ARGS and MESON_OPTS ci: build.sh: Break the script functionality into helper functions ci: build.sh: Move the necessary env variables to build.sh ci: build.sh: Add support for individual GitLab jobs ci: build.sh: Wire up the individual job functions to the CLI ci: build.sh: Document CI_CONT_SRCDIR ci: build.sh: Make the build script fail ASAP with 'set -e' ci: build.sh: Update git index in local container environments on 'dist' ci: build.sh: Make the script executable gitlab-ci.yml: Add 'after_script' stage to prep for artifact collection gitlab-ci.yml: Adopt job execution via a Bash script gitlab-ci.yml: Drop the usage of script variables reference .gitlab-ci.yml | 56 ++++++++++------------- ci/Makefile | 16 ++++--- ci/build.sh | 121 +++++++++++++++++++++++++++++++++++++++++++------ ci/helper | 21 ++++++--- 4 files changed, 155 insertions(+), 59 deletions(-) mode change 100644 => 100755 ci/build.sh -- 2.39.1

3 40

[PATCH] spec: Drop numad usage on fedora 39+
by Cole Robinson 22 Jun '23

22 Jun '23

numad is removed from Fedora 39. Upstream is dead https://src.fedoraproject.org/rpms/numad/c/a6bb891e8447e3b2a4c63774da94ad0d… https://pagure.io/releng/failed-composes/issue/4990#comment-857670 Signed-off-by: Cole Robinson <crobinso(a)redhat.com> --- This powers vcpu and numatune placement='auto'. Anyone know if that's being actively used in kubevirt, openstack, etc? libvirt.spec.in | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/libvirt.spec.in b/libvirt.spec.in index 1f77cd90b7..5f267a086b 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -173,6 +173,10 @@ %define with_numad 0%{!?_without_numad:1} %endif %endif +%if 0%{?fedora} >= 39 + # numad is retired in fedora 39+, upstream is dead + %define with_numad 0 +%endif %ifarch %{arches_dmidecode} %define with_dmidecode 0%{!?_without_dmidecode:1} -- 2.40.1

4 4

[PATCH v4] vfio/pci: Propagate ACPI notifications to user-space via eventfd
by Grzegorz Jaszczyk 09 Jun '23

09 Jun '23

To allow pass-through devices receiving ACPI notifications, permit to register ACPI notify handler (via VFIO_DEVICE_SET_IRQS) for a given device. The handler role is to receive and propagate such ACPI notifications to the user-space through the user provided eventfd. This allows VMM to receive and propagate them further to the VM, where the actual driver for pass-through device resides and can react to device specific notifications accordingly. The eventfd usage ensures VMM and device isolation: it allows to use a dedicated channel associated with the device for such events, such that the VMM has direct access. Since the eventfd counter is used as ACPI notification value placeholder, the eventfd signaling needs to be serialized in order to not end up with notification values being coalesced. Therefore ACPI notification values are buffered and signalized one by one, when the previous notification value has been consumed. Signed-off-by: Grzegorz Jaszczyk <jaz(a)semihalf.com> --- Changelog v3..v4 Address Alex Williamson feedback: - Instead of introducing new ioctl used for eventfd registration, take advantage of VFIO_DEVICE_SET_IRQS which already supports virtual IRQs for things like error notification and device release requests. - Introduced mechanism preventing creation of large queues. Other: - Move the implementation into the newly introduced VFIO_ACPI_NOTIFY helper module. It is actually not bound to VFIO_PCI but VFIO_PCI enables it whenever ACPI support is enabled. This change is introduced since ACPI notifications are not limited to PCI devices, making it PCI independent will allow to re-use it also for other VFIO_* like supports: e.g. VFIO_PLATFORM in the future if needed. Moving it out of drivers/vfio/pci/ was also suggested offline. - s/notify_val_next/node - v3: https://patchwork.kernel.org/project/kvm/patch/20230502132700.654528-1-jasz… Changelog v2..v3: - Fix compilation warnings when building with "W=1" Changelog v1..v2: - The v2 implementation is actually completely different then v1: instead of using acpi netlink events for propagating ACPI notifications to the user space take advantage of eventfd, which can provide better VMM and device isolation: it allows to use a dedicated channel associated with the device for such events, such that the VMM has direct access. - Using eventfd counter as notification value placeholder was suggested in v1 and requires additional serialization logic introduced in v2. - Since the vfio-pci supports non-ACPI platforms address !CONFIG_ACPI case. - v1 discussion: https://patchwork.kernel.org/project/kvm/patch/20230307220553.631069-1-jaz@… --- --- drivers/vfio/Kconfig | 5 + drivers/vfio/Makefile | 1 + drivers/vfio/pci/Kconfig | 1 + drivers/vfio/pci/vfio_pci_core.c | 9 ++ drivers/vfio/pci/vfio_pci_intrs.c | 73 ++++++++++ drivers/vfio/vfio_acpi_notify.c | 219 ++++++++++++++++++++++++++++++ include/linux/vfio_acpi_notify.h | 40 ++++++ include/linux/vfio_pci_core.h | 1 + include/uapi/linux/vfio.h | 1 + 9 files changed, 350 insertions(+) create mode 100644 drivers/vfio/vfio_acpi_notify.c create mode 100644 include/linux/vfio_acpi_notify.h diff --git a/drivers/vfio/Kconfig b/drivers/vfio/Kconfig index 89e06c981e43..7822b0d8e7b1 100644 --- a/drivers/vfio/Kconfig +++ b/drivers/vfio/Kconfig @@ -12,6 +12,11 @@ menuconfig VFIO If you don't know what to do here, say N. if VFIO +config VFIO_ACPI_NOTIFY + tristate + depends on ACPI + default n + config VFIO_CONTAINER bool "Support for the VFIO container /dev/vfio/vfio" select VFIO_IOMMU_TYPE1 if MMU && (X86 || S390 || ARM || ARM64) diff --git a/drivers/vfio/Makefile b/drivers/vfio/Makefile index 70e7dcb302ef..129c121b503d 100644 --- a/drivers/vfio/Makefile +++ b/drivers/vfio/Makefile @@ -14,3 +14,4 @@ obj-$(CONFIG_VFIO_PCI) += pci/ obj-$(CONFIG_VFIO_PLATFORM) += platform/ obj-$(CONFIG_VFIO_MDEV) += mdev/ obj-$(CONFIG_VFIO_FSL_MC) += fsl-mc/ +obj-$(CONFIG_VFIO_ACPI_NOTIFY) += vfio_acpi_notify.o diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index f9d0c908e738..5d229dbd074c 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -14,6 +14,7 @@ config VFIO_PCI_INTX config VFIO_PCI tristate "Generic VFIO support for any PCI device" select VFIO_PCI_CORE + select VFIO_ACPI_NOTIFY if ACPI help Support for the generic PCI VFIO bus driver which can connect any PCI device to the VFIO framework. diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index a5ab416cf476..b42299396d81 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -27,6 +27,7 @@ #include <linux/vgaarb.h> #include <linux/nospec.h> #include <linux/sched/mm.h> +#include <linux/vfio_acpi_notify.h> #if IS_ENABLED(CONFIG_EEH) #include <asm/eeh.h> #endif @@ -683,6 +684,7 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev) { struct vfio_pci_core_device *vdev = container_of(core_vdev, struct vfio_pci_core_device, vdev); + struct acpi_device *adev = ACPI_COMPANION(&vdev->pdev->dev); if (vdev->sriov_pf_core_dev) { mutex_lock(&vdev->sriov_pf_core_dev->vf_token->lock); @@ -705,6 +707,11 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev) vdev->req_trigger = NULL; } mutex_unlock(&vdev->igate); + + if (adev) { + vfio_acpi_notify_cleanup(vdev->acpi_notification, adev); + vdev->acpi_notification = NULL; + } } EXPORT_SYMBOL_GPL(vfio_pci_core_close_device); @@ -761,6 +768,8 @@ static int vfio_pci_get_irq_count(struct vfio_pci_core_device *vdev, int irq_typ return 1; } else if (irq_type == VFIO_PCI_REQ_IRQ_INDEX) { return 1; + } else if (irq_type == VFIO_PCI_ACPI_NTFY_IRQ_INDEX) { + return 1; } return 0; diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index bffb0741518b..e28f70c213ca 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -10,6 +10,7 @@ * Author: Tom Lyon, pugs(a)cisco.com */ +#include <linux/acpi.h> #include <linux/device.h> #include <linux/interrupt.h> #include <linux/eventfd.h> @@ -19,6 +20,7 @@ #include <linux/vfio.h> #include <linux/wait.h> #include <linux/slab.h> +#include <linux/vfio_acpi_notify.h> #include "vfio_pci_priv.h" @@ -667,6 +669,63 @@ static int vfio_pci_set_req_trigger(struct vfio_pci_core_device *vdev, count, flags, data); } +static int +vfio_pci_set_acpi_ntfy_trigger(struct vfio_pci_core_device *vdev, + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, void *data) +{ + struct acpi_device *adev = ACPI_COMPANION(&vdev->pdev->dev); + + if (index != VFIO_PCI_ACPI_NTFY_IRQ_INDEX || start != 0 || count > 1) + return -EINVAL; + + if (!vdev->acpi_notification) + return -EINVAL; + + /* + * Disable notifications: flags = (DATA_NONE|ACTION_TRIGGER), count = 0 + * Enable loopback testing: (DATA_BOOL|ACTION_TRIGGER) + */ + if (flags & VFIO_IRQ_SET_DATA_NONE) { + if (!count) { + vfio_acpi_notify_cleanup(vdev->acpi_notification, adev); + vdev->acpi_notification = NULL; + return 0; + } + } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { + u32 notification_val; + + if (!count) + return -EINVAL; + + notification_val = *(u32 *)data; + vfio_acpi_notify(NULL, notification_val, vdev->acpi_notification); + + return 0; + } + + return -EINVAL; +} + +static int +vfio_pci_set_acpi_ntfy_eventfd_trigger(struct vfio_pci_core_device *vdev, + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, void *data) +{ + struct acpi_device *adev = ACPI_COMPANION(&vdev->pdev->dev); + int32_t fd; + + if (index != VFIO_PCI_ACPI_NTFY_IRQ_INDEX || start != 0 || count != 1) + return -EINVAL; + + if (!adev) + return -ENODEV; + + fd = *(int32_t *)data; + + return vfio_register_acpi_notify_handler(&vdev->acpi_notification, adev, fd); +} + int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device *vdev, uint32_t flags, unsigned index, unsigned start, unsigned count, void *data) @@ -716,6 +775,20 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device *vdev, uint32_t flags, break; } break; + case VFIO_PCI_ACPI_NTFY_IRQ_INDEX: + switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { + case VFIO_IRQ_SET_ACTION_TRIGGER: + switch (flags & VFIO_IRQ_SET_DATA_TYPE_MASK) { + case VFIO_IRQ_SET_DATA_BOOL: + case VFIO_IRQ_SET_DATA_NONE: + func = vfio_pci_set_acpi_ntfy_trigger; + break; + case VFIO_IRQ_SET_DATA_EVENTFD: + func = vfio_pci_set_acpi_ntfy_eventfd_trigger; + break; + } + } + break; } if (!func) diff --git a/drivers/vfio/vfio_acpi_notify.c b/drivers/vfio/vfio_acpi_notify.c new file mode 100644 index 000000000000..8ef4db4b43b3 --- /dev/null +++ b/drivers/vfio/vfio_acpi_notify.c @@ -0,0 +1,219 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * VFIO ACPI notification propagation + * + * Author: Grzegorz Jaszczyk <jaz(a)semihalf.com> + */ +#include <linux/vfio_acpi_notify.h> + +#define DRIVER_AUTHOR "Grzegorz Jaszczyk <jaz(a)semihalf.com>" +#define DRIVER_DESC "ACPI notification propagation helper module for VFIO based devices" + +#define NOTIFICATION_QUEUE_SIZE 20 + +struct notification_queue { + int notification_val; + struct list_head node; +}; + +static int vfio_eventfd_wakeup(wait_queue_entry_t *wait, unsigned int mode, + int sync, void *key) +{ + struct vfio_acpi_notification *acpi_notify = + container_of(wait, struct vfio_acpi_notification, wait); + __poll_t flags = key_to_poll(key); + + /* + * eventfd_read signalize EPOLLOUT at the end of its function - this + * means previous eventfd with its notification value was consumed so + * the next notification can be signalized now if pending - schedule + * proper work. + */ + if (flags & EPOLLOUT) { + mutex_unlock(&acpi_notify->notification_lock); + schedule_work(&acpi_notify->acpi_notification_work); + } + + return 0; +} + +static void vfio_ptable_queue_proc(struct file *file, + wait_queue_head_t *wqh, poll_table *pt) +{ + struct vfio_acpi_notification *acpi_notify = + container_of(pt, struct vfio_acpi_notification, pt); + + add_wait_queue(wqh, &acpi_notify->wait); +} + +static void acpi_notification_work_fn(struct work_struct *work) +{ + struct vfio_acpi_notification *acpi_notify; + struct notification_queue *entry; + + acpi_notify = container_of(work, struct vfio_acpi_notification, + acpi_notification_work); + + mutex_lock(&acpi_notify->notification_list_lock); + if (list_empty(&acpi_notify->notification_list) || !acpi_notify->acpi_notify_trigger) + goto out; + + /* + * If the previous eventfd was not yet consumed by user-space lets hold + * on and exit. The notification function will be rescheduled when + * signaling eventfd will be possible (when the EPOLLOUT will be + * signalized and unlocks notify_events). + */ + if (!mutex_trylock(&acpi_notify->notification_lock)) + goto out; + + entry = list_first_entry(&acpi_notify->notification_list, + struct notification_queue, node); + + list_del(&entry->node); + acpi_notify->notification_queue_count--; + mutex_unlock(&acpi_notify->notification_list_lock); + + eventfd_signal(acpi_notify->acpi_notify_trigger, entry->notification_val); + + kfree(entry); + + return; +out: + mutex_unlock(&acpi_notify->notification_list_lock); +} + +void vfio_acpi_notify(acpi_handle handle, u32 event, void *data) +{ + struct vfio_acpi_notification *acpi_notify = (struct vfio_acpi_notification *)data; + struct notification_queue *entry; + + entry = kmalloc(sizeof(*entry), GFP_KERNEL); + if (!entry) + return; + + entry->notification_val = event; + INIT_LIST_HEAD(&entry->node); + + mutex_lock(&acpi_notify->notification_list_lock); + if (acpi_notify->notification_queue_count > NOTIFICATION_QUEUE_SIZE) { + struct notification_queue *oldest_entry; + + oldest_entry = list_first_entry(&acpi_notify->notification_list, + struct notification_queue, + node); + list_del(&oldest_entry->node); + acpi_notify->notification_queue_count--; + kfree(oldest_entry); + + } + list_add_tail(&entry->node, &acpi_notify->notification_list); + acpi_notify->notification_queue_count++; + mutex_unlock(&acpi_notify->notification_list_lock); + + schedule_work(&acpi_notify->acpi_notification_work); +} +EXPORT_SYMBOL_GPL(vfio_acpi_notify); + +void vfio_acpi_notify_cleanup(struct vfio_acpi_notification *acpi_notify, + struct acpi_device *adev) +{ + struct notification_queue *entry, *entry_tmp; + u64 cnt; + + if (!acpi_notify || !acpi_notify->acpi_notify_trigger) + return; + + acpi_remove_notify_handler(adev->handle, ACPI_DEVICE_NOTIFY, + vfio_acpi_notify); + + eventfd_ctx_remove_wait_queue(acpi_notify->acpi_notify_trigger, + &acpi_notify->wait, &cnt); + + flush_work(&acpi_notify->acpi_notification_work); + + mutex_lock(&acpi_notify->notification_list_lock); + list_for_each_entry_safe(entry, entry_tmp, + &acpi_notify->notification_list, + node) { + list_del(&entry->node); + kfree(entry); + } + mutex_unlock(&acpi_notify->notification_list_lock); + + eventfd_ctx_put(acpi_notify->acpi_notify_trigger); + + kfree(acpi_notify); +} +EXPORT_SYMBOL_GPL(vfio_acpi_notify_cleanup); + +int vfio_register_acpi_notify_handler(struct vfio_acpi_notification **acpi_notify_ptr, + struct acpi_device *adev, int32_t fd) +{ + struct vfio_acpi_notification *acpi_notify = *acpi_notify_ptr; + struct file *acpi_notify_trigger_file; + struct eventfd_ctx *efdctx; + acpi_status status; + + if (fd < -1) + return -EINVAL; + else if (fd == -1) + vfio_acpi_notify_cleanup(acpi_notify, adev); + + if (acpi_notify && acpi_notify->acpi_notify_trigger) + return -EBUSY; + + efdctx = eventfd_ctx_fdget(fd); + if (IS_ERR(efdctx)) + return PTR_ERR(efdctx); + + acpi_notify = kzalloc(sizeof(*acpi_notify), GFP_KERNEL); + if (!acpi_notify) + return -ENOMEM; + + *acpi_notify_ptr = acpi_notify; + + INIT_WORK(&acpi_notify->acpi_notification_work, acpi_notification_work_fn); + INIT_LIST_HEAD(&acpi_notify->notification_list); + + acpi_notify->acpi_notify_trigger = efdctx; + + mutex_init(&acpi_notify->notification_lock); + + /* + * Install custom wake-up handler to be notified whenever underlying + * eventfd is consumed by the user-space. + */ + init_waitqueue_func_entry(&acpi_notify->wait, vfio_eventfd_wakeup); + init_poll_funcptr(&acpi_notify->pt, vfio_ptable_queue_proc); + + acpi_notify_trigger_file = eventfd_fget(fd); + vfs_poll(acpi_notify_trigger_file, &acpi_notify->pt); + + status = acpi_install_notify_handler(adev->handle, ACPI_DEVICE_NOTIFY, + vfio_acpi_notify, (void *)acpi_notify); + if (ACPI_FAILURE(status)) { + u64 cnt; + + dev_err(&adev->dev, "Failed to install notify handler: %s", + acpi_format_exception(status)); + + eventfd_ctx_remove_wait_queue(acpi_notify->acpi_notify_trigger, + &acpi_notify->wait, &cnt); + + flush_work(&acpi_notify->acpi_notification_work); + + eventfd_ctx_put(acpi_notify->acpi_notify_trigger); + + kfree(acpi_notify); + + return -ENODEV; + } + + return 0; +} +EXPORT_SYMBOL_GPL(vfio_register_acpi_notify_handler); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR(DRIVER_AUTHOR); +MODULE_DESCRIPTION(DRIVER_DESC); diff --git a/include/linux/vfio_acpi_notify.h b/include/linux/vfio_acpi_notify.h new file mode 100644 index 000000000000..2722ad24d8e3 --- /dev/null +++ b/include/linux/vfio_acpi_notify.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * VFIO ACPI notification replication + * + * Author: Grzegorz Jaszczyk <jaz(a)semihalf.com> + */ +#include <linux/acpi.h> +#include <linux/eventfd.h> +#include <linux/poll.h> + +struct vfio_acpi_notification { + struct eventfd_ctx *acpi_notify_trigger; + struct work_struct acpi_notification_work; + struct list_head notification_list; + struct mutex notification_list_lock; + struct mutex notification_lock; + int notification_queue_count; + poll_table pt; + wait_queue_entry_t wait; +}; + +#if IS_ENABLED(CONFIG_ACPI) +void vfio_acpi_notify(acpi_handle handle, u32 event, void *data); +int vfio_register_acpi_notify_handler(struct vfio_acpi_notification **acpi_notify, + struct acpi_device *adev, int32_t fd); +void vfio_acpi_notify_cleanup(struct vfio_acpi_notification *acpi_notify, + struct acpi_device *adev); +#else +static inline void vfio_acpi_notify(acpi_handle handle, u32 event, void *data) {} +static inline int +vfio_register_acpi_notify_handler(struct vfio_acpi_notification **acpi_notify, + struct acpi_device *adev, int32_t fd) +{ + return -ENODEV; +} + +static inline void +vfio_acpi_notify_cleanup(struct vfio_acpi_notification *acpi_notify, + struct acpi_device *adev) {} +#endif /* CONFIG_ACPI */ diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h index 367fd79226a3..a4491b3d8064 100644 --- a/include/linux/vfio_pci_core.h +++ b/include/linux/vfio_pci_core.h @@ -96,6 +96,7 @@ struct vfio_pci_core_device { struct mutex vma_lock; struct list_head vma_list; struct rw_semaphore memory_lock; + struct vfio_acpi_notification *acpi_notification; }; /* Will be exported for vfio pci drivers usage */ diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 0552e8dcf0cb..b2619fd16cc4 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -625,6 +625,7 @@ enum { VFIO_PCI_MSIX_IRQ_INDEX, VFIO_PCI_ERR_IRQ_INDEX, VFIO_PCI_REQ_IRQ_INDEX, + VFIO_PCI_ACPI_NTFY_IRQ_INDEX, VFIO_PCI_NUM_IRQS }; -- 2.40.1.698.g37aff9b760-goog

2 5

[PATCH 00/18] QMP schema validation fixes
by Peter Krempa 02 Jun '23

02 Jun '23

Turns out we still had wired up use of 'device' argument of block_set_io_throttle which got deprecated and our valiator didn't catch it due to multiple reasons. Address all of that. Peter Krempa (18): qemuMonitorTestAddItemVerbatim: Simplify cleanup qemuMonitorTestAddHandler: Remove return value virRaiseErrorLog: Don't skip error printing when enabling debug logging env variable testQemuMonitorJSONAttachChardev: Move all setup code under virTestRun qemuMonitorJSONTestAttachOneChardev: Rewrite using qemuMonitorTestAddItemVerbatim qemuMonitorTestAddItemExpect: Remove unused helper testQemuAgentCPU: Rewrite using qemuMonitorTestAddItemVerbatim testQemuAgentFSTrim: Rewrite using qemuMonitorTestAddItemVerbatim testQemuMonitorJSONqemuMonitorJSONSendKeyHoldtime: Rewrite using qemuMonitorTestAddItemVerbatim testQemuMonitorJSONqemuMonitorJSONSetBlockIoThrottle: Rewrite using qemuMonitorTestAddItemVerbatim Drop unused qemuMonitorTestAddItemParams qemumonitorjsontest: Drop 'schema-meta' case qemuDiskConfigBlkdeviotuneEnabled: Make 'disk' argument const qemu: Refuse setting <iotune> for 'SD' disks qemumonitorjsontest: Use 'id' instead of deprecated 'device' argument of 'block_set_io_throttle' qemuMonitorGetBlockIoThrottle: Drop 'diskalias' argument qemuMonitorSetBlockIoThrottle: Drop 'diskalias' argument testQEMUSchemaValidateObjectMember: validate QMP object member deprecation src/qemu/qemu_command.c | 2 +- src/qemu/qemu_command.h | 2 +- src/qemu/qemu_driver.c | 40 ++---- src/qemu/qemu_hotplug.c | 4 +- src/qemu/qemu_monitor.c | 12 +- src/qemu/qemu_monitor.h | 2 - src/qemu/qemu_monitor_json.c | 14 +- src/qemu/qemu_monitor_json.h | 2 - src/qemu/qemu_process.c | 12 +- src/qemu/qemu_validate.c | 8 ++ src/util/virerror.c | 4 +- tests/qemuagenttest.c | 92 ++++++------ tests/qemumonitorjsontest.c | 153 ++++++++++---------- tests/qemumonitortestutils.c | 266 +++-------------------------------- tests/qemumonitortestutils.h | 16 +-- tests/testutilsqemuschema.c | 4 + 16 files changed, 191 insertions(+), 442 deletions(-) -- 2.40.1

3 36

[PATCH v2] util: basic support for VFIO variant drivers
by Laine Stump 01 Jun '23

01 Jun '23

Before a PCI device can be assigned to a guest with VFIO, that device must be bound to the vfio-pci driver rather than to the device's normal driver. The vfio-pci driver provides APIs that permit QEMU to perform all the necessary operations to make the device accessible to the guest. There has been kernel work recently to support vendor/device-specific VFIO variant drivers that provide the basic vfio-pci driver functionality while adding support for device-specific operations (for example these device-specific drivers are planned to support live migration of certain devices). All that will be needed to make this functionality available will be to bind the new vendor-specific driver to the device (rather than the generic vfio-pci driver, which will continue to work just without the extra functionality). But until now libvirt has required that all PCI devices being assigned to a guest with VFIO specifically have the "vfio-pci" driver bound to the device. So even if the user manually binds a shiny new vendor-specific vfio variant driver to the device (and puts "managed='no'" in the config to prevent libvirt from changing the binding), libvirt will just fail during startup of the guest (or during hotplug) because the driver bound to the device isn't exactly "vfio-pci". This patch loosens that restriction a bit - rather than requiring that the device be bound to "vfio-pci", it also checks if the drivername contains the string "vfio" at all, and in this case allows the operation to continue. If the driver is in fact a VFIO variant, then the assignment will succeed, but if it is not a VFIO variant then QEMU will fail (and report the error back to libvirt). In the near future (possibly by kernel 6.0) there will be a formal method of identifying a VFIO variant driver by looking in sysfs; in the meantime the inexact, but simple, method in this patch will allow users of the few existing VFIO variant drivers (and developers of new VFIO variant drivers) to use their new drivers without needing to remove libvirt from their setup - they can simply pre-bind the device to the new driver, then use "managed='no'" in their libvirt config. NB: this patch does *not* handle automatically determining the proper vendor-specific driver and binding to it in the case of "managed='yes'". This will be implemented later when there is a widely available driver / device combo we can use for testing. Signed-off-by: Laine Stump <laine(a)redhat.com> --- V1 here: https://listman.redhat.com/archives/libvir-list/2022-August/233327.html Change in V2: V1 used the output of modinfo to look for "vfio_pci" as an alias for a driver to see if it was a VFIO variant driver. As a result of discussion of V1, V2 is much simpler - it just assumes that any driver with "vfio" in the name is a VFIO variant. This is okay because 1) QEMU will still catch it and libvirt will properly log the error if the driver isn't actually a VFIO variant, and 2) it's a temporary situation, just to enable use of VFIO variant drivers with libvirt until a standard method of detecting this is added to sysfs (which, according to the discussion of V1, is coming in the near future). (NB: I did implement checking of /lib/modules/`uname -r`/modules.alias as suggested by Erik, but it turned out that this caused the unit tests to call uname(3) and open the modules.alias file on the test host - for a proper unit test I would have also needed to mock these two functions, and it seemed like too much complexity for a temporary workaround. I've implemented Jason's suggestion here (accept any driver with "vfio" in the name), which is similar to danpb's suggestion (accept specifically the two drivers that are already in the upstream kernel), but will also allow for new drivers that may be under development.) src/hypervisor/virhostdev.c | 26 ++++--------- src/util/virpci.c | 76 ++++++++++++++++++++++++++++++++++--- src/util/virpci.h | 3 ++ 3 files changed, 82 insertions(+), 23 deletions(-) diff --git a/src/hypervisor/virhostdev.c b/src/hypervisor/virhostdev.c index c0ce867596..15b35fa75e 100644 --- a/src/hypervisor/virhostdev.c +++ b/src/hypervisor/virhostdev.c @@ -747,9 +747,8 @@ virHostdevPreparePCIDevicesImpl(virHostdevManager *mgr, mgr->inactivePCIHostdevs) < 0) goto reattachdevs; } else { - g_autofree char *driverPath = NULL; - g_autofree char *driverName = NULL; - int stub; + g_autofree char *drvName = NULL; + virPCIStubDriver drvType; /* Unmanaged devices should already have been marked as * inactive: if that's the case, we can simply move on */ @@ -769,18 +768,14 @@ virHostdevPreparePCIDevicesImpl(virHostdevManager *mgr, * information about active / inactive device across * daemon restarts has been implemented */ - if (virPCIDeviceGetDriverPathAndName(pci, - &driverPath, &driverName) < 0) + if (virPCIDeviceGetDriverNameAndType(pci, &drvName, &drvType) < 0) goto reattachdevs; - stub = virPCIStubDriverTypeFromString(driverName); - - if (stub > VIR_PCI_STUB_DRIVER_NONE && - stub < VIR_PCI_STUB_DRIVER_LAST) { + if (drvType > VIR_PCI_STUB_DRIVER_NONE) { /* The device is bound to a known stub driver: store this * information and add a copy to the inactive list */ - virPCIDeviceSetStubDriver(pci, stub); + virPCIDeviceSetStubDriver(pci, drvType); VIR_DEBUG("Adding PCI device %s to inactive list", virPCIDeviceGetName(pci)); @@ -2292,18 +2287,13 @@ virHostdevPrepareOneNVMeDevice(virHostdevManager *hostdev_mgr, /* Let's check if all PCI devices are NVMe disks. */ for (i = 0; i < virPCIDeviceListCount(pciDevices); i++) { virPCIDevice *pci = virPCIDeviceListGet(pciDevices, i); - g_autofree char *drvPath = NULL; g_autofree char *drvName = NULL; - int stub = VIR_PCI_STUB_DRIVER_NONE; + virPCIStubDriver drvType; - if (virPCIDeviceGetDriverPathAndName(pci, &drvPath, &drvName) < 0) + if (virPCIDeviceGetDriverNameAndType(pci, &drvName, &drvType) < 0) goto cleanup; - if (drvName) - stub = virPCIStubDriverTypeFromString(drvName); - - if (stub == VIR_PCI_STUB_DRIVER_VFIO || - STREQ_NULLABLE(drvName, "nvme")) + if (drvType == VIR_PCI_STUB_DRIVER_VFIO || STREQ_NULLABLE(drvName, "nvme")) continue; VIR_WARN("Suspicious NVMe disk assignment. PCI device " diff --git a/src/util/virpci.c b/src/util/virpci.c index 7800966963..51ccf4d9fd 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -277,6 +277,71 @@ virPCIDeviceGetDriverPathAndName(virPCIDevice *dev, char **path, char **name) } +/** + * virPCIDeviceGetDriverNameAndType: + * @dev: virPCIDevice object to examine + * @drvName: returns name of driver bound to this device (if any) + * @drvType: returns type of driver if it is a known stub driver type + * + * Find the name of the driver bound to @dev (if any) and the type of + * the driver if it is a known/recognized "stub" driver (based on the + * driver name). + * + * There are vfio "variant" drivers that provide all the basic + * functionality of the standard vfio-pci driver as well as additional + * stuff. There is a plan to add info to sysfs that will allow easily + * determining if a driver is a vfio variant driver, but that sysfs + * entry isn't yet available. In the meantime as a workaround so that + * the few existing vfio variant drivers can be used with libvirt, and + * so that driver developers can test their new vfio variant drivers + * without needing to bypass libvirt, we also check if the driver name + * contains the string "vfio"; if it does, then we consider this drier + * as type VFIO. This can lead to false positives, but that isn't a + * horrible thing, because the problem will still be caught by QEMU as + * soon as libvirt makes the request to attach the device. + * + * Return 0 on success, -1 on failure. If -1 is returned, then an error + * message has been logged. + */ +int +virPCIDeviceGetDriverNameAndType(virPCIDevice *dev, + char **drvName, + virPCIStubDriver *drvType) +{ + g_autofree char *drvPath = NULL; + int tmpType; + + if (virPCIDeviceGetDriverPathAndName(dev, &drvPath, drvName) < 0) + return -1; + + if (!*drvName) { + *drvType = VIR_PCI_STUB_DRIVER_NONE; + return 0; + } + + tmpType = virPCIStubDriverTypeFromString(*drvName); + + if (tmpType > VIR_PCI_STUB_DRIVER_NONE) { + *drvType = tmpType; + return 0; /* exact match of a known driver name (or no name) */ + } + + /* Check if the drivername contains "vfio" and count as a VFIO + * driver if so - see above for explanation. + */ + + if (strstr(*drvName, "vfio")) { + VIR_DEBUG("Driver %s is a vfio_pci driver", *drvName); + *drvType = VIR_PCI_STUB_DRIVER_VFIO; + } else { + VIR_DEBUG("Driver %s is NOT a vfio_pci driver", *drvName); + *drvType = VIR_PCI_STUB_DRIVER_NONE; + } + + return 0; +} + + static int virPCIDeviceConfigOpenInternal(virPCIDevice *dev, bool readonly, bool fatal) { @@ -1004,8 +1069,8 @@ virPCIDeviceReset(virPCIDevice *dev, virPCIDeviceList *activeDevs, virPCIDeviceList *inactiveDevs) { - g_autofree char *drvPath = NULL; g_autofree char *drvName = NULL; + virPCIStubDriver drvType; int ret = -1; int fd = -1; int hdrType = -1; @@ -1032,15 +1097,16 @@ virPCIDeviceReset(virPCIDevice *dev, * reset it whenever appropriate, so doing it ourselves would just * be redundant. */ - if (virPCIDeviceGetDriverPathAndName(dev, &drvPath, &drvName) < 0) + if (virPCIDeviceGetDriverNameAndType(dev, &drvName, &drvType) < 0) goto cleanup; - if (virPCIStubDriverTypeFromString(drvName) == VIR_PCI_STUB_DRIVER_VFIO) { - VIR_DEBUG("Device %s is bound to vfio-pci - skip reset", - dev->name); + if (drvType == VIR_PCI_STUB_DRIVER_VFIO) { + + VIR_DEBUG("Device %s is bound to %s - skip reset", dev->name, drvName); ret = 0; goto cleanup; } + VIR_DEBUG("Resetting device %s", dev->name); if ((fd = virPCIDeviceConfigOpenWrite(dev)) < 0) diff --git a/src/util/virpci.h b/src/util/virpci.h index 4d9193f24e..0532b90f90 100644 --- a/src/util/virpci.h +++ b/src/util/virpci.h @@ -280,6 +280,9 @@ int virPCIDeviceRebind(virPCIDevice *dev); int virPCIDeviceGetDriverPathAndName(virPCIDevice *dev, char **path, char **name); +int virPCIDeviceGetDriverNameAndType(virPCIDevice *dev, + char **drvName, + virPCIStubDriver *drvType); int virPCIDeviceIsPCIExpress(virPCIDevice *dev); int virPCIDeviceHasPCIExpressLink(virPCIDevice *dev); -- 2.37.1

5 13

[PATCH 0/4] docs: Move 'golang' metadata headers into rST files and further XSL cleanups
by Peter Krempa 01 Jun '23

01 Jun '23

Pipeline: https://gitlab.com/pipo.sk/libvirt/-/jobs/4385932520 Impacted HTML files from pipeline: https://pipo.sk.gitlab.io/-/libvirt/-/jobs/4385932520/artifacts/website/lib… https://pipo.sk.gitlab.io/-/libvirt/-/jobs/4385932520/artifacts/website/lib… https://pipo.sk.gitlab.io/-/libvirt/-/jobs/4385932520/artifacts/website/go/… https://pipo.sk.gitlab.io/-/libvirt/-/jobs/4385932520/artifacts/website/go/… Peter Krempa (4): docs: newapi: Remove unused 'docomponents' template docs: page.xsl: Drop passhrough of <script> elements into header docs: go: Add 'go-import' metadata via rST docs: Use '--exit-status=' option for 'rst2html5' docs/go/libvirt.rst | 3 +++ docs/go/libvirtxml.rst | 3 +++ docs/libvirt-go-xml.rst | 3 +++ docs/libvirt-go.rst | 3 +++ docs/meson.build | 2 +- docs/newapi.xsl | 18 ------------------ docs/page.xsl | 14 ++------------ 7 files changed, 15 insertions(+), 31 deletions(-) -- 2.40.1

2 6

[libvirt PATCHv3 for 9.4.0] conf: node_device: use separate variables for parsing integers
by Ján Tomko 01 Jun '23

01 Jun '23

In virNodeDeviceGetSCSIHostCaps, there is a pattern of reusing a tmp value and stealing the pointer. But in two case it is not stolen. Use separate variables for them to avoid mixing autofree with manual free() calls. This fixes the memory leak of the "max_npiv_vports" string. (The other gets freed anyway because it was the last use of "tmp" in the function") Fixes: 8a0cb5f73ade3900546718eabe70cb064c6bd22c Signed-off-by: Ján Tomko <jtomko(a)redhat.com> --- v3: fix "declaration after statement" warning src/conf/node_device_conf.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/src/conf/node_device_conf.c b/src/conf/node_device_conf.c index fcee9c027c..172223225f 100644 --- a/src/conf/node_device_conf.c +++ b/src/conf/node_device_conf.c @@ -2857,29 +2857,32 @@ virNodeDeviceGetSCSIHostCaps(virNodeDevCapSCSIHost *scsi_host) } if (virVHBAIsVportCapable(NULL, scsi_host->host)) { + g_autofree char *max_vports = NULL; + g_autofree char *vports = NULL; + scsi_host->flags |= VIR_NODE_DEV_CAP_FLAG_HBA_VPORT_OPS; - if (!(tmp = virVHBAGetConfig(NULL, scsi_host->host, + if (!(max_vports = virVHBAGetConfig(NULL, scsi_host->host, "max_npiv_vports"))) { VIR_WARN("Failed to read max_npiv_vports for host%d", scsi_host->host); goto cleanup; } - if (virStrToLong_i(tmp, NULL, 10, &scsi_host->max_vports) < 0) { - VIR_WARN("Failed to parse value of max_npiv_vports '%s'", tmp); + if (virStrToLong_i(max_vports, NULL, 10, &scsi_host->max_vports) < 0) { + VIR_WARN("Failed to parse value of max_npiv_vports '%s'", max_vports); goto cleanup; } - if (!(tmp = virVHBAGetConfig(NULL, scsi_host->host, + if (!(vports = virVHBAGetConfig(NULL, scsi_host->host, "npiv_vports_inuse"))) { VIR_WARN("Failed to read npiv_vports_inuse for host%d", scsi_host->host); goto cleanup; } - if (virStrToLong_i(tmp, NULL, 10, &scsi_host->vports) < 0) { - VIR_WARN("Failed to parse value of npiv_vports_inuse '%s'", tmp); + if (virStrToLong_i(vports, NULL, 10, &scsi_host->vports) < 0) { + VIR_WARN("Failed to parse value of npiv_vports_inuse '%s'", vports); goto cleanup; } } -- 2.40.1

2 1

[PATCH v6 00/11] tracing: remove dynamic vcpu state
by Alex Bennée 31 May '23

31 May '23

Hi Stefan, The references dynamic vcpu tracing support was removed when the original TCG trace points where removed. However there was still a legacy of dynamic trace state to track this in cpu.h and extra hash variables to track TBs. While the removed vcpu tracepoints are not in generated code (or helpers) they still bring in a bunch of machinery to manage the state so I've pulled them out. We keep and rename one (cpu_reset) to a static trace points which dump vcpu->index as it is useful to f4bug. v6 new patch to shuffle deprecated, added rth's rb, qapi doc cleanups Please queue into your tree. Alex Bennée (11): *-user: remove the guest_user_syscall tracepoints trace-events: remove the remaining vcpu trace events trace: remove vcpu_id from the TraceEvent structure scripts/qapi: document the tool that generated the file docs/deprecated: move QMP events bellow QMP command section qapi: make the vcpu parameters deprecated for 8.1 trace: remove code that depends on setting vcpu trace: remove control-vcpu.h tcg: remove the final vestiges of dstate hw/9pfs: use qemu_xxhash4 accel/tcg: include cs_base in our hash calculations docs/about/deprecated.rst | 25 +++++--- qapi/trace.json | 40 ++++++------- accel/tcg/tb-hash.h | 6 +- include/exec/exec-all.h | 3 - include/hw/core/cpu.h | 5 -- include/qemu/xxhash.h | 23 +++++-- include/user/syscall-trace.h | 4 -- trace/control-internal.h | 10 ---- trace/control-vcpu.h | 63 -------------------- trace/control.h | 48 --------------- trace/event-internal.h | 2 - accel/tcg/cpu-exec.c | 7 +-- accel/tcg/tb-maint.c | 5 +- accel/tcg/translate-all.c | 6 -- bsd-user/freebsd/os-syscall.c | 2 - hw/9pfs/9p.c | 5 +- hw/core/cpu-common.c | 6 +- stubs/trace-control.c | 13 ---- trace/control-target.c | 109 +++------------------------------- trace/control.c | 28 --------- trace/qmp.c | 76 +++--------------------- trace/trace-hmp-cmds.c | 18 +----- util/qsp.c | 2 +- hw/core/trace-events | 3 + scripts/qapi/gen.py | 9 ++- scripts/tracetool/format/c.py | 6 -- scripts/tracetool/format/h.py | 16 +---- trace-events | 50 ---------------- 28 files changed, 94 insertions(+), 496 deletions(-) delete mode 100644 trace/control-vcpu.h -- 2.39.2

3 14

[PULL 00/11] Tracing patches
by Stefan Hajnoczi 31 May '23

31 May '23

The following changes since commit 51bdb0b57a2d9e84d6915fbae7b5d76c8820cf3c: Merge tag 'pull-tcg-20230530' of https://gitlab.com/rth7680/qemu into staging (2023-05-30 13:25:18 -0700) are available in the Git repository at: https://gitlab.com/stefanha/qemu.git tags/tracing-pull-request for you to fetch changes up to 2e7eda52e5ea2f066b37ecd9fdb5983851eabbbf: accel/tcg: include cs_base in our hash calculations (2023-05-31 13:21:53 -0400) ---------------------------------------------------------------- Pull request ---------------------------------------------------------------- Alex Bennée (11): *-user: remove the guest_user_syscall tracepoints trace-events: remove the remaining vcpu trace events trace: remove vcpu_id from the TraceEvent structure scripts/qapi: document the tool that generated the file docs/deprecated: move QMP events bellow QMP command section qapi: make the vcpu parameters deprecated for 8.1 trace: remove code that depends on setting vcpu trace: remove control-vcpu.h tcg: remove the final vestiges of dstate hw/9pfs: use qemu_xxhash4 accel/tcg: include cs_base in our hash calculations docs/about/deprecated.rst | 25 +++++--- qapi/trace.json | 40 ++++++------- accel/tcg/tb-hash.h | 6 +- include/exec/exec-all.h | 3 - include/hw/core/cpu.h | 5 -- include/qemu/xxhash.h | 23 +++++-- include/user/syscall-trace.h | 4 -- trace/control-internal.h | 10 ---- trace/control-vcpu.h | 63 -------------------- trace/control.h | 48 --------------- trace/event-internal.h | 2 - accel/tcg/cpu-exec.c | 7 +-- accel/tcg/tb-maint.c | 5 +- accel/tcg/translate-all.c | 6 -- bsd-user/freebsd/os-syscall.c | 2 - hw/9pfs/9p.c | 5 +- hw/core/cpu-common.c | 6 +- stubs/trace-control.c | 13 ---- trace/control-target.c | 109 +++------------------------------- trace/control.c | 28 --------- trace/qmp.c | 76 +++--------------------- trace/trace-hmp-cmds.c | 18 +----- util/qsp.c | 2 +- hw/core/trace-events | 3 + scripts/qapi/gen.py | 9 ++- scripts/tracetool/format/c.py | 6 -- scripts/tracetool/format/h.py | 16 +---- trace-events | 50 ---------------- 28 files changed, 94 insertions(+), 496 deletions(-) delete mode 100644 trace/control-vcpu.h -- 2.40.1

1 11

[PATCH for 9.4.0] include: Fix 'Since' for new VIR_MIGRATE_PARAM_COMPRESSION_* macros
by Michal Privoznik 31 May '23

31 May '23

In v9.3.0-98-g150ae3e62b two new macros were introduced: VIR_MIGRATE_PARAM_COMPRESSION_ZLIB_LEVEL and VIR_MIGRATE_PARAM_COMPRESSION_ZSTD_LEVEL. But both list 9.1.0 as the version they were introduced in (this is because the patch was sent in that release time frame). Change the version to the current release. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- include/libvirt/libvirt-domain.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 20467849a4..a1902546bb 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1329,7 +1329,7 @@ typedef enum { * VIR_TYPED_PARAM_INT. Accepted values are in range 0-9. 0 is no compression, * 1 is maximum speed and 9 is maximum compression. * - * Since: 9.1.0 + * Since: 9.4.0 */ # define VIR_MIGRATE_PARAM_COMPRESSION_ZLIB_LEVEL "compression.zlib.level" @@ -1340,7 +1340,7 @@ typedef enum { * VIR_TYPED_PARAM_INT. Accepted values are in range 0-20. 0 is no compression, * 1 is maximum speed and 20 is maximum compression. * - * Since: 9.1.0 + * Since: 9.4.0 */ # define VIR_MIGRATE_PARAM_COMPRESSION_ZSTD_LEVEL "compression.zstd.level" -- 2.39.3

2 1