May 2020 - Devel - libvirt List Archives

[PATCH libvirt-python 0/5] Fixes from adding type annotation
by Philipp Hahn 27 Jul '20

27 Jul '20

Hello, as announced a long time ago with <https://www.redhat.com/archives/libvir-list/2018-November/msg00291.html> and recently refreshed with <https://www.redhat.com/archives/libvir-list/2020-April/msg00892.html> I'm working on adding PEP 484 type hints <https://www.python.org/dev/peps/pep-0484/> to the Python binding of libvirt. I have finished this work now and have a working version at <https://github.com/univention/libvirt-python/tree/typing> which consists of 90 patches in total as I has to go over evry file to understand and fix all things. As that patch bomb is quiet large I'm going to submit them in smaller chunks to make them more reviewable. Today I start with the first round consisting of "real" bugs in the current code: Philipp Hahn (5): generator: Fix undefined variables file generator: Fix string formatting generator: Fix domainSnapshot.listAllChildren() qemu-api: Fix return type libvirtaio: Fix return types of callback generator.py | 6 +++--- libvirt-override-virDomainSnapshot.py | 2 +- libvirt-qemu-override-api.xml | 4 ++-- libvirtaio.py | 18 ++++++++++++------ 4 files changed, 18 insertions(+), 12 deletions(-) After that I plan to continue with: 2. fix examples/ to work with Python 3 3. Cleanup code tree-wide 4. Cleanup generator.py 5. Cleanup sanitytest.py 6. Teach generator.py to add PEP 484 annotation 7. Assorted cleanups (the order and chunking is not final yet) > examples/domipaddrs: Convert to python 3 print() > examples/domipaddrs: Fix Python 2 dict.iteritems() > examples/*: Remove stray semicolon > example/dhcp*: Fix None comparison > examples/event-test: Remove unneeded global statement > examples/event-test: Work with old version of python-libvirt > examples/event-test: Use atexit for Python 3 > examples/esxlist: Fix Python 2 raw_input() > examples/consolecallback: Add var to save callback > examples/consolecallback: Fix assorted errors > examples: Add missing return values > libvirtaio: Drop object(*args, **kwargs) > libvirtaio: Fix return type > libvirtaio: assert callback type > Do not use bare except > Cleanup imports > Fix white space > Remove legacy libvirtError arguments > stream: Fix exception traceback handling > override: Simplify exception handling > generator: Simplify exception handling > generator: Change type of quiet to bool > generator: Remove unneeded line continuation > generator: Convert to 'not in' and 'is not' > generator: Remove dead variable assignments > generator: Remove skipped_modules > generator: Remove useless sort key > generator: Fix return type on failure > generator: Merge now identical if-elif-else cases > generator: Use more string formatting > generator: Simplify string concatentaion > generator: Use enumerate() > generator: Use increment assignment > generator: Use string concatenation > generator: Remove global declarations > generator: Initialize function_classes directly > generator: Check contained in hash > generator: Use dict.item() to walk keys and values > generator: Walk only the values > generator: Directly get dict length > generator: Just walk the dict > generator: Use splitlines() > generator: Open file with context manager > generator: Refactor parser creation > generator: Remove unused SAX content handler methods > generator: Use SAX method names > generator: Use string formatting > generator: Convert in_function to boolean > generator: Simplify XML attribute fetching > generator: Initialize with empty strings > generator: Expand tuple to names in for loop > generator: Store arguments and return as tuple > generator: Fixed writing cached=None > generator: Simplify sorting > generator: Simplify loop break > generator: Simplify boolean condition > generator: Convert dict() to set() > generator: Converto to defaultdict() > generator: Add PEP 484 type annotations > override: Add manual PEP 484 type annotations > sanitytest: Skip type annotations > stream: Simplify boolean condition > domain: Fix None comparison > stream: no type change > stream: Convert type() to isinstance() > stream: Return None from callback > connect: Just clear all event handlers > override: no type change > sanitytest: Do not re-declare set > sanitytest: Drop else:pass > sanitytest: Drop Python 2 compatibility > sanitytest: Add PEP 484 type annotations > sanitytest: Use 3-tuple for basicklassmap > sanitytest: Use 3-tuple for finalklassmap > sanitytest: Use set for tracking used functions > sanitytest: Use str.startswith() instead of str[0] > generator: Generate PEP 484 type annotation > override: Catch type error > generator: Special handling for virStoragePool.listAllVolumes > generator: Merge code for __init__ genration > generator: Use empty string instead of None > generator: break lines in generated code > generator: Expand tuple to names in for loop > generator: Work around type change > generator: use pointer wrapper for all objects > examples: Add/fix PEP 484 type annotation -- 2.20.1

3 10

kvm-hint-dedicated requires host CPU passthrough?
by Stefan Hajnoczi 16 Jul '20

16 Jul '20

Hi, libvirt refuses to set KVM_HINTS_DEDICATED when the CPU model is not host-passthrough. Is there a reason for this limitation? My understanding is that KVM_HINTS_DEDICATED means the vCPU is pinned to a host CPU that is not shared with other tasks. Any KVM vCPU should be able to support this feature, regardless of whether host-passthrough is used or not. Stefan

3 5

[libvirt-jenkins-ci PATCH 0/7] lcitool: Create and expose ccache wrappers
by Andrea Bolognani 15 Jul '20

15 Jul '20

This series makes ccache work out of the box for container-based builds. Most of it is refactoring. Applies on top of https://www.redhat.com/archives/libvir-list/2020-March/msg01263.html Andrea Bolognani (7): lcitool: Improve ccache symlinks creation lcitool: Configure ccache using environment variables lcitool: Create ccache wrappers outside of $HOME lcitool: Refactor cross_arch handling a bit lcitool: Use commands[] for deb-based distros lcitool: Use cross_commands[] for all distros lcitool: Create and expose ccache wrappers guests/host_vars/libvirt-centos-7/main.yml | 1 + guests/host_vars/libvirt-centos-8/main.yml | 1 + guests/host_vars/libvirt-debian-10/main.yml | 1 + guests/host_vars/libvirt-debian-9/main.yml | 1 + guests/host_vars/libvirt-debian-sid/main.yml | 1 + guests/host_vars/libvirt-fedora-30/main.yml | 1 + guests/host_vars/libvirt-fedora-31/main.yml | 1 + .../host_vars/libvirt-fedora-rawhide/main.yml | 1 + guests/host_vars/libvirt-freebsd-11/main.yml | 1 + guests/host_vars/libvirt-freebsd-12/main.yml | 1 + .../libvirt-freebsd-current/main.yml | 1 + .../host_vars/libvirt-opensuse-151/main.yml | 1 + guests/host_vars/libvirt-ubuntu-1604/main.yml | 1 + guests/host_vars/libvirt-ubuntu-1804/main.yml | 1 + guests/lcitool | 92 +++++++++++-------- guests/playbooks/update/main.yml | 1 + guests/playbooks/update/tasks/global.yml | 14 +++ guests/playbooks/update/tasks/users.yml | 45 --------- guests/playbooks/update/templates/bashrc.j2 | 3 +- .../playbooks/update/templates/ccache.conf.j2 | 1 - 20 files changed, 87 insertions(+), 83 deletions(-) create mode 100644 guests/playbooks/update/tasks/global.yml delete mode 100644 guests/playbooks/update/templates/ccache.conf.j2 -- 2.25.1

3 26

[PATCH v5 for-5.0? 0/7] Tighten qemu-img rules on missing backing format
by Eric Blake 23 Jun '20

23 Jun '20

3 15

CFP: KVM Forum 2020
by Paolo Bonzini 22 Jun '20

22 Jun '20

================================================================ KVM Forum 2020: Call For Participation October 28-30, 2020 - Convention Centre Dublin - Dublin, Ireland (All submissions must be received before June 15, 2020 at 23:59 PST) ================================================================= KVM Forum is an annual event that presents a rare opportunity for developers and users to meet, discuss the state of Linux virtualization technology, and plan for the challenges ahead. This highly technical conference unites the developers who drive KVM development and the users who depend on KVM as part of their offerings, or to power their data centers and clouds. Sessions include updates on the state of the KVM virtualization stack, planning for the future, and many opportunities for attendees to collaborate. After more than ten years in the mainline kernel, KVM continues to be a critical part of the FOSS cloud infrastructure. Come join us in continuing to improve the KVM ecosystem. We understand that these are uncertain times and we are continuously monitoring the COVID-19/Novel Coronavirus situation. Our attendees' safety is of the utmost importance. If we feel it is not safe to meet in person, we will turn the event into a virtual experience. We hope to announce this at the same time as the speaker notification. Speakers will still be expected to attend if a physical event goes ahead, but we understand that exceptional circumstances might arise after speakers are accepted and we will do our best to accommodate such circumstances. Based on these factors, we encourage you to submit and reach out to us should you have any questions. The program committee may be contacted as a group via email: kvm-forum-2020-pc(a)redhat.com. SUGGESTED TOPICS ---------------- * Scaling, latency optimizations, performance tuning * Hardening and security * New features * Testing KVM and the Linux Kernel: * Nested virtualization * Resource management (CPU, I/O, memory) and scheduling * VFIO: IOMMU, SR-IOV, virtual GPU, etc. * Networking: Open vSwitch, XDP, etc. * virtio and vhost * Architecture ports and new processor features QEMU: * Management interfaces: QOM and QMP * New devices, new boards, new architectures * New storage features * High availability, live migration and fault tolerance * Emulation and TCG * Firmware: ACPI, UEFI, coreboot, U-Boot, etc. Management & Infrastructure * Managing KVM: Libvirt, OpenStack, oVirt, KubeVirt, etc. * Storage: Ceph, Gluster, SPDK, etc. * Network Function Virtualization: DPDK, OPNFV, OVN, etc. * Provisioning SUBMITTING YOUR PROPOSAL ------------------------ Abstracts due: June 15, 2020 Please submit a short abstract (~150 words) describing your presentation proposal. Slots vary in length up to 45 minutes. Submit your proposal here: https://events.linuxfoundation.org/kvm-forum/program/cfp/ Please only use the categories "presentation" and "panel discussion" You will receive a notification whether or not your presentation proposal was accepted by August 17, 2020. Speakers will receive a complimentary pass for the event. In case your submission has multiple presenters, only the primary speaker for a proposal will receive a complimentary event pass. For panel discussions, all panelists will receive a complimentary event pass. TECHNICAL TALKS A good technical talk should not just report on what has happened over the last year; it should present a concrete problem and how it impacts the user and/or developer community. Whenever applicable, focus on work that needs to be done, difficulties that haven't yet been solved, and on decisions that other developers should be aware of. Summarizing recent developments is okay but it should not be more than a small portion of the overall talk. END-USER TALKS One of the big challenges as developers is to know what, where and how people actually use our software. We will reserve a few slots for end users talking about their deployment challenges and achievements. If you are using KVM in production you are encouraged submit a speaking proposal. Simply mark it as an end-user talk. As an end user, this is a unique opportunity to get your input to developers. PANEL DISCUSSIONS If you are proposing a panel discussion, please make sure that you list all of your potential panelists in your the abstract. We will request full biographies if a panel is accepted. HANDS-ON / BOF SESSIONS We will reserve some time for people to get together and discuss strategic decisions as well as other topics that are best solved within smaller groups. These sessions will be announced during the event. If you are interested in organizing such a session, please add it to the list at http://www.linux-kvm.org/page/KVM_Forum_2020_BOF Let people you think who might be interested know about your BOF, and encourage them to add their names to the wiki page as well. Please add your ideas to the list before KVM Forum starts. HOTEL / TRAVEL -------------- This year's event will take place at the Conference Center Dublin. For information on hotels close to the conference, please visit https://events.linuxfoundation.org/kvm-forum/attend/venue-travel/. Information on conference hotel blocks will be available later in Spring 2020. DATES TO REMEMBER ----------------- * CFP Opens: Monday, April 27, 2020 * CFP Closes: Monday, June 15 at 11:59 PM PST * CFP Notifications: Monday, August 17 * Schedule Announcement: Thursday, August 20 * Slide Due Date: Monday, October 26 * Event Dates: Wednesday, October 28 – Friday, October 30 Thank you for your interest in KVM. We're looking forward to your submissions and, if the conditions will permit it, to seeing you at the KVM Forum 2020 in October! -your KVM Forum 2020 Program Committee Please contact us with any questions or comments at kvm-forum-2020-pc(a)redhat.com

2 3

[PATCH v5 0/4] introduction of migration_version attribute for VFIO live migration
by Yan Zhao 22 Jun '20

22 Jun '20

This patchset introduces a migration_version attribute under sysfs of VFIO Mediated devices. This migration_version attribute is used to check migration compatibility between two mdev devices. Currently, it has two locations: (1) under mdev_type node, which can be used even before device creation, but only for mdev devices of the same mdev type. (2) under mdev device node, which can only be used after the mdev devices are created, but the src and target mdev devices are not necessarily be of the same mdev type (The second location is newly added in v5, in order to keep consistent with the migration_version node for migratable pass-though devices) Patch 1 defines migration_version attribute for the first location in Documentation/vfio-mediated-device.txt Patch 2 uses GVT as an example for patch 1 to show how to expose migration_version attribute and check migration compatibility in vendor driver. Patch 3 defines migration_version attribute for the second location in Documentation/vfio-mediated-device.txt Patch 4 uses GVT as an example for patch 3 to show how to expose migration_version attribute and check migration compatibility in vendor driver. (The previous "Reviewed-by" and "Acked-by" for patch 1 and patch 2 are kept in v5, as there are only small changes to commit messages of the two patches.) v5: added patch 2 and 4 for mdev device part of migration_version attribute. v4: 1. fixed indentation/spell errors, reworded several error messages 2. added a missing memory free for error handling in patch 2 v3: 1. renamed version to migration_version 2. let errno to be freely defined by vendor driver 3. let checking mdev_type be prerequisite of migration compatibility check 4. reworded most part of patch 1 5. print detailed error log in patch 2 and generate migration_version string at init time v2: 1. renamed patched 1 2. made definition of device version string completely private to vendor driver 3. reverted changes to sample mdev drivers 4. described intent and usage of version attribute more clearly. Yan Zhao (4): vfio/mdev: add migration_version attribute for mdev (under mdev_type node) drm/i915/gvt: export migration_version to mdev sysfs (under mdev_type node) vfio/mdev: add migration_version attribute for mdev (under mdev device node) drm/i915/gvt: export migration_version to mdev sysfs (under mdev device node) .../driver-api/vfio-mediated-device.rst | 183 ++++++++++++++++++ drivers/gpu/drm/i915/gvt/Makefile | 2 +- drivers/gpu/drm/i915/gvt/gvt.c | 39 ++++ drivers/gpu/drm/i915/gvt/gvt.h | 7 + drivers/gpu/drm/i915/gvt/kvmgt.c | 55 ++++++ drivers/gpu/drm/i915/gvt/migration_version.c | 170 ++++++++++++++++ drivers/gpu/drm/i915/gvt/vgpu.c | 13 +- 7 files changed, 466 insertions(+), 3 deletions(-) create mode 100644 drivers/gpu/drm/i915/gvt/migration_version.c -- 2.17.1

7 39

[libvirt] Request for a new release of libvirt-java in Maven
by Slavka Peleva 17 Jun '20

17 Jun '20

Hello, The Apache CloudStack project is using Maven libvirt java bindings in KVM code. In libvirt-java project were added qemu methods that will prevent the usage of bash scripts or Process class methods. It will be great to use those features in CloudStack. >From what I see, the previous release was requested in this list, and there doesn’t seem to be a formal procedure for this, so how should I proceed? Best regards and thank you in advance, Slavka

1 1

[PATCH libvirt v1 0/6] Fix ZPCI address auto-generation on s390
by Shalini Chellathurai Saroja 16 Jun '20

16 Jun '20

The ZPCI address validation or autogeneration does not work as expected in the following scenarios 1. uid = 0 and fid = 0 2. uid = 0 and fid > 0 3. uid = 0 and fid not specified 4. uid not specified and fid > 0 5. 2 ZPCI devices with uid > 0 and fid not specified. This is because of the following reasons 1. If uid = 0 or fid = 0 the code assumes that user has not specified the corresponding address 2. If either uid or fid is provided, the code assumes that both uid and fid addresses are specified by the user. This patch fixes these issues. Shalini Chellathurai Saroja (6): conf: fix ZPCI address validation on s390 tests: qemu: add tests for ZPCI on s390 conf: fix ZPCI address auto-generation on s390 qemu: move ZPCI uid validation into device validation tests: qemu: add more tests for ZPCI on S390 tests: add test with PCI and CCW device src/conf/device_conf.c | 45 ++++++-------- src/conf/domain_addr.c | 59 +++++++++++++------ src/conf/domain_conf.c | 2 +- src/libvirt_private.syms | 4 +- src/qemu/qemu_validate.c | 26 +++++++- src/util/virpci.c | 23 ++------ src/util/virpci.h | 6 +- .../hostdev-vfio-zpci-autogenerate-fids.args | 31 ++++++++++ .../hostdev-vfio-zpci-autogenerate-fids.xml | 29 +++++++++ .../hostdev-vfio-zpci-autogenerate-uids.args | 31 ++++++++++ .../hostdev-vfio-zpci-autogenerate-uids.xml | 29 +++++++++ .../hostdev-vfio-zpci-ccw-memballoon.args | 28 +++++++++ .../hostdev-vfio-zpci-ccw-memballoon.xml | 17 ++++++ .../hostdev-vfio-zpci-duplicate.xml | 30 ++++++++++ ...ostdev-vfio-zpci-invalid-uid-valid-fid.xml | 21 +++++++ .../hostdev-vfio-zpci-multidomain-many.args | 6 +- .../hostdev-vfio-zpci-set-zero.xml | 21 +++++++ .../hostdev-vfio-zpci-uid-set-zero.xml | 20 +++++++ tests/qemuxml2argvtest.c | 25 ++++++++ .../hostdev-vfio-zpci-multidomain-many.xml | 6 +- 20 files changed, 387 insertions(+), 72 deletions(-) create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate-fids.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate-fids.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate-uids.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate-uids.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-ccw-memballoon.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-ccw-memballoon.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-duplicate.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-invalid-uid-valid-fid.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-set-zero.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-uid-set-zero.xml -- 2.21.1

5 35

[PATCH v2 0/7] Add Security Guest doc and check for capabilities cache validation
by Paulo de Rezende Pinatti 12 Jun '20

12 Jun '20

This series introduces the concept of a 'Secure Guest' feature which covers on s390 IBM Secure Execution and on x86 AMD Secure Encrypted Virtualization. Besides adding documentation for IBM Secure Execution it also adds checks during validation of the qemu capabilities cache. These checks per architecture can be performed for IBM Secure Execution on s390 and AMD Secure Encrypted Virtualization on AMD x86 CPUs (both checks implemented in this series). For s390 the verification consists of: - checking if /sys/firmware/uv is available: meaning the HW facility is available and the host OS supports it; - checking if the kernel cmdline contains 'prot_virt=1': meaning the host OS wants to use the feature. For AMD Secure Encrypted Virtualization the verification consists of: - checking if /sys/module/kvm_amd/parameters/sev contains the value '1': meaning SEV is enabled in the host kernel; - checking if /dev/sev exists Whenever the availability of the feature does not match the secure guest flag in the cache then libvirt will re-build it in order to pick up the new set of capabilities available. Additionally, this series adds the same aforementioned checks to the virt-host-validate tool to facilitate the manual verification process for users. Changes in v2: [Patch 1] Reworked kernel cmdline parser into a parameter based processing. [Patch 2] Added missing value "on" to kvalue list [Patch 3] Changed AMD SEV support check to module parameter is set and /dev/sev exists. Moved doc changes to a new standalone patch 6. [Patch 4] Added missing value "on" to kvalue list [Patch 5] Changed AMD SEV support check to align with patch 3. Moved doc changes to a new standalone patch 6. [Patch 6] Summarized AMD SEV doc changes from patches 3 and 5. Adjusted libvirt version number [Patch 7 (v1: Patch 6)] Adjusted libvirt version number link to v1: https://www.redhat.com/archives/libvir-list/2020-May/msg00416.html Boris Fiuczynski (3): tools: secure guest check on s390 in virt-host-validate tools: secure guest check for AMD in virt-host-validate docs: update AMD launch secure description Paulo de Rezende Pinatti (3): util: introduce a parser for kernel cmdline arguments qemu: check if s390 secure guest support is enabled qemu: check if AMD secure guest support is enabled Viktor Mihajlovski (1): docs: Describe protected virtualization guest setup docs/kbase.html.in | 3 + docs/kbase/launch_security_sev.rst | 9 +- docs/kbase/s390_protected_virt.rst | 189 +++++++++++++++++++++++++++++ src/libvirt_private.syms | 2 + src/qemu/qemu_capabilities.c | 76 ++++++++++++ src/util/virutil.c | 169 ++++++++++++++++++++++++++ src/util/virutil.h | 17 +++ tests/utiltest.c | 141 +++++++++++++++++++++ tools/virt-host-validate-common.c | 83 ++++++++++++- tools/virt-host-validate-common.h | 5 + tools/virt-host-validate-qemu.c | 4 + 11 files changed, 693 insertions(+), 5 deletions(-) create mode 100644 docs/kbase/s390_protected_virt.rst -- 2.25.4

3 19

[PATCH 0/5] qemu: Prefer -numa cpu over -numa node,cpus=
by Michal Privoznik 05 Jun '20

05 Jun '20

Another round of patches to catch up with the latest command line building recommendations. This time, the winner is: -numa cpu Thing is, the way vCPUs are assigned to NUMA nodes is not very deterministic right now. QEMU relies on this weird mapping where firstly threads are iterated over, then cores, then (possibly) dies and then sockets. So they came up with a better schema: -numa cpu,node-id=N,socket-id=S,die-id=D,core-id=C,thread-id=T where everything is specified on the command line. Long story short, some QEMU code is copied over to keep status quo (or, whatever you want). It was buried down down (include/hw/i386/topology.h). I wanted to allow users configure the vCPU topology too, but then decided not to because neither Libvirt nor QEMU can really guarantee the topology will look the same in the guest (at least in my testing with cpu mode='host-passthrough' it didn't). Or, what you're proposing? I can post them as a follow up if needed. Michal Prívozník (5): qemuBuildNumaArgStr: Move vars into loops qemuBuildNumaArgStr: Use g_autofree on @nodeBackends qemuBuildNumaArgStr: Separate out old style of building CPU list qemu: Prefer -numa cpu over -numa node,cpus= virCPUDefParseXML: Parse uint using virXPathUInt src/conf/cpu_conf.c | 14 +- src/qemu/qemu_command.c | 168 +++++++++++++++--- .../hugepages-nvdimm.x86_64-latest.args | 4 +- ...memory-default-hugepage.x86_64-latest.args | 10 +- .../memfd-memory-numa.x86_64-latest.args | 10 +- ...y-hotplug-nvdimm-access.x86_64-latest.args | 4 +- ...ry-hotplug-nvdimm-align.x86_64-latest.args | 4 +- ...ry-hotplug-nvdimm-label.x86_64-latest.args | 4 +- ...ory-hotplug-nvdimm-pmem.x86_64-latest.args | 4 +- ...ory-hotplug-nvdimm-ppc64.ppc64-latest.args | 4 +- ...hotplug-nvdimm-readonly.x86_64-latest.args | 4 +- .../memory-hotplug-nvdimm.x86_64-latest.args | 4 +- ...vhost-user-fs-fd-memory.x86_64-latest.args | 4 +- ...vhost-user-fs-hugepages.x86_64-latest.args | 4 +- ...host-user-gpu-secondary.x86_64-latest.args | 3 +- .../vhost-user-vga.x86_64-latest.args | 3 +- 16 files changed, 195 insertions(+), 53 deletions(-) -- 2.26.2

2 14