November 2020 - Devel - libvirt List Archives

[PATCH 00/24] qemu: Prepare for QAPIfied 'object-add'
by Peter Krempa 30 Nov '20

30 Nov '20

Prepare libvirt for the changes proposed for qemu: https://www.redhat.com/archives/libvir-list/2020-November/msg01580.html Patches 1-12/24 are pure refactors and can be merged right away. Patches 13-15/24 can be merged, but the justification will be weak without the rest. Patches 16-22/24 are RFC until the qemu bits are merged. Patch 23/24 adds capabilities and needs to be udpated later Patch 24/24 is not to be merged! Note that patch 23/24 is intentionally missing from the mailing list posting as it's just a dump of QEMU capabilities from newest qemu. No need to litter the list with it. Also note that after the last patch tests will fail. See that patch for the reason. Fetch the full series at: git fetch https://gitlab.com/pipo.sk/libvirt.git object-add-qapi-3 Peter Krempa (24): qemuMonitorJSONSetMigrationParams: Take double pointer for @params qemuMonitorSetMigrationCapabilities: Take double pointer for @caps qemuMonitorJSONSetMigrationCapabilities: Refactor cleanup testQemuMonitorJSONqemuMonitorJSONGetMigrationCapabilities: refactor cleanup qemuMonitorJSONAddObject: Take double pointer for @props qemuMonitorJSONMakeCommandInternal: Clear @arguments when stolen qemuMonitorAddObject: Fix semantics of @alias qemuMonitorAddObject: Refactor cleanup util: json: Replace virJSONValueObjectSteal by virJSONValueObjectRemoveKey tests: qemuxml2argv: Don't check whether -netdev was QAPIfied repeatedly qemuBuildChrChardevStr: Rename 'flags' to 'cdevflags' testCompareXMLToArgvValidateSchema: Populate autoNodeset qemu: command: Generate commandline of 'masterKey0' secret via JSON qemu: command: Generate commandline of 'sev0' sev-guest object via JSON qemu: command: Generate commandline of iothread objects JSON qemu: capabilities: Introduce QEMU_CAPS_OBJECT_QAPIFIED tests: qemuxml2argv: Validate generation of JSON props for object-add qemu: command: Introduce raw JSON passthrough for '-object' for testing qemu: monitor: Make wrapping of 'props' of 'object-add' optional qemuMonitorCreateObjectPropsWrap: Open-code in qemuBuildMemoryBackendProps qemu: monitor: Don't add 'props' wrapper if qemu has QEMU_CAPS_OBJECT_QAPIFIED qemumonitorjsontest: Remove bomb guarding object-add qemu: capabilities: Add 6.0 capabilities with qapified object-add [DONTMERGE] qemuxml2argvtest: Force QMP validation with latest caps src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 332 +- src/qemu/qemu_migration_params.c | 22 +- src/qemu/qemu_monitor.c | 121 +- src/qemu/qemu_monitor.h | 8 +- src/qemu/qemu_monitor_json.c | 57 +- src/qemu/qemu_monitor_json.h | 6 +- src/util/virjson.c | 29 +- src/util/virqemu.c | 48 +- src/util/virqemu.h | 3 +- .../domaincapsdata/qemu_6.0.0-q35.x86_64.xml | 197 + .../domaincapsdata/qemu_6.0.0-tcg.x86_64.xml | 203 + tests/domaincapsdata/qemu_6.0.0.x86_64.xml | 197 + .../caps_6.0.0.x86_64.replies | 32021 ++++++++++++++++ .../caps_6.0.0.x86_64.xml | 3205 ++ tests/qemumonitorjsontest.c | 40 +- ...v-missing-platform-info.x86_64-2.12.0.args | 2 +- .../launch-security-sev.x86_64-2.12.0.args | 2 +- tests/qemuxml2argvtest.c | 44 +- 20 files changed, 36202 insertions(+), 338 deletions(-) create mode 100644 tests/domaincapsdata/qemu_6.0.0-q35.x86_64.xml create mode 100644 tests/domaincapsdata/qemu_6.0.0-tcg.x86_64.xml create mode 100644 tests/domaincapsdata/qemu_6.0.0.x86_64.xml create mode 100644 tests/qemucapabilitiesdata/caps_6.0.0.x86_64.replies create mode 100644 tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml -- 2.28.0

1 23

[PATCH 0/3] qemu: Remove caching in handler for query-command-line-options
by Peter Krempa 30 Nov '20

30 Nov '20

Shuffle around where we extract the data, so that we don't have to call the monitor multiple times which necessitates caching. Peter Krempa (3): qemu: monitor: Implement new handlers for 'query-command-line-options' virQEMUCapsProbeQMPCommandLine: Rewrite using qemuMonitorGetCommandLineOptions qemuMonitorGetCommandLineOptionParameters: remove the unused function and helpers src/qemu/qemu_capabilities.c | 34 +++++---- src/qemu/qemu_monitor.c | 34 +-------- src/qemu/qemu_monitor.h | 9 +-- src/qemu/qemu_monitor_json.c | 138 +++++++++-------------------------- src/qemu/qemu_monitor_json.h | 6 +- tests/qemumonitorjsontest.c | 117 ----------------------------- 6 files changed, 58 insertions(+), 280 deletions(-) -- 2.28.0

2 4

Re: [PATCH for-6.0 6/6] qapi: Deprecate 'query-kvm'
by Daniel P. Berrangé 30 Nov '20

30 Nov '20

Copying libvir-list for the deprecation warning. On Mon, Nov 16, 2020 at 04:10:11PM +0300, Roman Bolshakov wrote: > 'query-accel' QMP command should be used instead. > > Signed-off-by: Roman Bolshakov <r.bolshakov(a)yadro.com> > --- > qapi/machine.json | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) docs/system/deprecated.rst needs to be updated for any deprecation to be visible to consumers of QEMU. > > diff --git a/qapi/machine.json b/qapi/machine.json > index 5648d8d24d..130b0dbebc 100644 > --- a/qapi/machine.json > +++ b/qapi/machine.json > @@ -579,6 +579,9 @@ > # > # Returns information about KVM acceleration > # > +# Features: > +# @deprecated: This command is deprecated, use 'query-accel' instead. > +# > # Returns: @AccelInfo > # > # Since: 0.14.0 > @@ -589,7 +592,8 @@ > # <- { "return": { "enabled": true, "present": true } } > # > ## > -{ 'command': 'query-kvm', 'returns': 'AccelInfo' } > +{ 'command': 'query-kvm', 'returns': 'AccelInfo', > + 'features': [ 'deprecated' ] } > > ## > # @query-accel: > -- > 2.29.2 > > Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

5 11

[libvirt PATCH] vircgroupv2: fix virCgroupV2DenyDevice
by Pavel Hrdina 30 Nov '20

30 Nov '20

The original logic is incorrect. We would delete the device entry from eBPF map only if the newval would be same as current val in the map. In case that the device was allowed only as read-only but later we remove all permissions for that device it would remain in the table with empty values. The old code would still deny the device but it's not working as intended. Instead we will update the value in advance. If the updated value is 0 it means that we are removing all permissions so it should be removed from the map, otherwise we will update the value in map. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1810356 Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com> --- src/util/vircgroupv2.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/util/vircgroupv2.c b/src/util/vircgroupv2.c index 2b32f614e4..40cfa20376 100644 --- a/src/util/vircgroupv2.c +++ b/src/util/vircgroupv2.c @@ -1796,7 +1796,9 @@ virCgroupV2DenyDevice(virCgroupPtr group, return 0; } - if (newval == val) { + val = val & ~newval; + + if (val == 0) { if (virBPFDeleteElem(group->unified.devices.mapfd, &key) < 0) { virReportSystemError(errno, "%s", _("failed to remove device from BPF cgroup map")); @@ -1804,7 +1806,6 @@ virCgroupV2DenyDevice(virCgroupPtr group, } group->unified.devices.count--; } else { - val ^= val & newval; if (virBPFUpdateElem(group->unified.devices.mapfd, &key, &val) < 0) { virReportSystemError(errno, "%s", _("failed to update device in BPF cgroup map")); -- 2.28.0

2 1

[PATCH 0/3] gitlab: Add issue templates
by Peter Krempa 30 Nov '20

30 Nov '20

Gitlab allows projects to define templates for filing issues [1]. Libvirt didn't have them yet. We can use them to encourage users to file better bug reports and feature requests. You can test-drive the proposed templates in my test repo: https://gitlab.com/pipo.sk/test/-/issues/new?issue%5Bassignee_id%5D=&issue%… Any suggestions on how to improve them further are very welcome. [1] https://docs.gitlab.com/ee/user/project/description_templates.html Peter Krempa (3): gitlab: Add issue template for reporting a generic bug gitlab: Add issue template for a feature request gitlab: Add issue template for filing a bug in libvirt's qemu driver .gitlab/issue_templates/bug-qemu.md | 56 +++++++++++++++++++++++++++++ .gitlab/issue_templates/bug.md | 55 ++++++++++++++++++++++++++++ .gitlab/issue_templates/feature.md | 26 ++++++++++++++ 3 files changed, 137 insertions(+) create mode 100644 .gitlab/issue_templates/bug-qemu.md create mode 100644 .gitlab/issue_templates/bug.md create mode 100644 .gitlab/issue_templates/feature.md -- 2.28.0

4 10

[PATCH v2 0/6] Final changes to fix test suite on macOS
by Roman Bolshakov 27 Nov '20

27 Nov '20

The set of changes fixes qemuxml2argvtest and closes issue #58. Changes since v1: - Simplified lookup of symbols in main executable (Michal) - Reduced timeout to 90s for qemuxml2argvtest (Andrea) - Two new patches to fix qemucapsprobe Pipeline results: https://gitlab.com/roolebo/libvirt/-/pipelines/220225729 Roman Bolshakov (6): tests: Fix opendir mocks on macOS tests: Fix mock chaining on macOS qemuxml2argvtest: Increase timeout on macOS ci: Run test suite on macOS tests: Delay mock creation qemucapsprobemock: Fix lookup of qemu functions ci/cirrus/build.yml | 3 +-- tests/meson.build | 49 +++++++++++++++++++++++++----------------- tests/virfilewrapper.c | 4 ++++ tests/virmock.h | 13 +++++++++++ tests/virpcimock.c | 5 +++++ 5 files changed, 52 insertions(+), 22 deletions(-) -- 2.29.2

2 14

Entering freeze for libvirt-6.10.0
by Jiri Denemark 27 Nov '20

27 Nov '20

I have just tagged v6.10.0-rc1 in the repository and pushed signed tarballs and source RPMs to https://libvirt.org/sources/ Please give the release candidate some testing and in case you find a serious issue which should have a fix in the upcoming release, feel free to reply to this thread to make sure the issue is more visible. If you have not done so yet, please update NEWS.rst to document any significant change you made since the last release. Thanks, Jirka

1 1

QEMU Advent Calendar 2020 Call for Images
by Eldon Stegall 27 Nov '20

27 Nov '20

Hi, QEMU Advent Calendar 2020 is around the corner and we are looking for volunteers to contribute disk images that showcase something cool, bring back retro computing memories, or simply entertain with a puzzle or game. QEMU Advent Calendar publishes a QEMU disk image each day from December 1-24. Each image is a surprise designed to delight an audience consisting of the QEMU community and beyond. You can see previous years here: https://www.qemu-advent-calendar.org/ You can help us make this year's calendar awesome by: * Sending disk images ( or links to larger images ) * Replying with ideas for disk images (reply off-list to avoid spoilers!) If you have an idea after the start of the advent, go ahead and send it. We may find space to include it, or go ahead and get a jump on 2021! Here are the requirements for disk images: * Content must be freely redistributable (i.e. no proprietary license that prevents distribution). For GPL based software, you need to provide the source code, too. * Provide a name and a short description of the disk image (e.g. with hints on what to try) * Provide a ./run shell script that prints out the name and description/hints and launches QEMU * Provide a 320x240 screenshot/image/logo for the website * Size should be ideally under 100 MB per disk image (but if some few images are bigger, that should be OK, too) Check out this disk image as an example of how to distribute an image: https://www.qemu-advent-calendar.org/2018/download/day24.tar.xz PS: QEMU Advent Calendar is a secular calendar (not religious). The idea is to create a fun experience for the QEMU community which can be shared with everyone. You don't need to celebrate Christmas or another religious festival to participate! Thanks, and best wishes! Eldon

1 0

[libvirt PATCH v4] ci: Switch to meson build system
by Erik Skultety 27 Nov '20

27 Nov '20

Add meson required bits to the ci logic in the repo to be able to run a meson build in a container. This patch also drops several environment variables we don't need with meson anymore. Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- v3: https://www.redhat.com/archives/libvir-list/2020-November/msg01293.html Since v3: - further review comments - introduced CI_MESON_ARGS variable ci/Makefile | 38 +++++++------------------------------- ci/build.sh | 29 ++++++----------------------- 2 files changed, 13 insertions(+), 54 deletions(-) diff --git a/ci/Makefile b/ci/Makefile index c7c8eb9a45..d9d14f82c2 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -20,27 +20,6 @@ CI_HOST_SRCDIR = $(CI_SCRATCHDIR)/src # the $(CI_HOST_SRCDIR) directory from the host CI_CONT_SRCDIR = $(CI_USER_HOME)/libvirt -# Relative directory to perform the build in. This -# defaults to using a separate build dir, but can be -# set to empty string for an in-source tree build. -CI_VPATH = build - -# The directory holding the build output inside the -# container. -CI_CONT_BUILDDIR = $(CI_CONT_SRCDIR)/$(CI_VPATH) - -# Can be overridden with mingw{32,64}-configure if desired -CI_CONFIGURE = $(CI_CONT_SRCDIR)/configure - -# Default to using all possible CPUs -CI_SMP = $(shell getconf _NPROCESSORS_ONLN) - -# Any extra arguments to pass to make -CI_MAKE_ARGS = - -# Any extra arguments to pass to configure -CI_CONFIGURE_ARGS = - # Script containing environment preparation steps CI_PREPARE_SCRIPT = $(CI_ROOTDIR)/prepare.sh @@ -220,13 +199,10 @@ ci-run-command@%: ci-prepare-tree --login \ --user="#$(CI_UID)" \ --group="#$(CI_GID)" \ - CONFIGURE_OPTS="$$CONFIGURE_OPTS" \ + MESON_OPTS="$$MESON_OPTS" \ CI_CONT_SRCDIR="$(CI_CONT_SRCDIR)" \ - CI_CONT_BUILDDIR="$(CI_CONT_BUILDDIR)" \ - CI_SMP="$(CI_SMP)" \ - CI_CONFIGURE="$(CI_CONFIGURE)" \ - CI_CONFIGURE_ARGS="$(CI_CONFIGURE_ARGS)" \ - CI_MAKE_ARGS="$(CI_MAKE_ARGS)" \ + CI_MESON_ARGS="$(CI_MESON_ARGS)" \ + CI_NINJA_ARGS="$(CI_NINJA_ARGS)" \ $(CI_COMMAND) || exit 1' @test "$(CI_CLEAN)" = "1" && rm -rf $(CI_SCRATCHDIR) || : @@ -236,8 +212,8 @@ ci-shell@%: ci-build@%: $(MAKE) -C $(CI_ROOTDIR) ci-run-command@$* CI_COMMAND="$(CI_USER_HOME)/build" -ci-check@%: - $(MAKE) -C $(CI_ROOTDIR) ci-build@$* CI_MAKE_ARGS="check" +ci-test@%: + $(MAKE) -C $(CI_ROOTDIR) ci-build@$* CI_NINJA_ARGS+=test ci-list-images: @echo @@ -266,6 +242,6 @@ ci-help: @echo " CI_CLEAN=0 - do not delete '$(CI_SCRATCHDIR)' after completion" @echo " CI_REUSE=1 - re-use existing '$(CI_SCRATCHDIR)' content" @echo " CI_ENGINE=auto - container engine to use (podman, docker)" - @echo " CI_CONFIGURE_ARGS= - extra arguments passed to configure" - @echo " CI_MAKE_ARGS= - extra arguments passed to make, e.g. space delimited list of targets" + @echo " CI_MESON_ARGS= - extra arguments passed to meson" + @echo " CI_NINJA_ARGS= - extra arguments passed to ninja" @echo diff --git a/ci/build.sh b/ci/build.sh index 2da84c080a..740b46a935 100644 --- a/ci/build.sh +++ b/ci/build.sh @@ -7,32 +7,15 @@ # # to make. -mkdir -p "$CI_CONT_BUILDDIR" || exit 1 -cd "$CI_CONT_BUILDDIR" +cd "$CI_CONT_SRCDIR" export VIR_TEST_DEBUG=1 -NOCONFIGURE=1 "$CI_CONT_SRCDIR/autogen.sh" || exit 1 -# $CONFIGURE_OPTS is a env that can optionally be set in the container, +# $MESON_OPTS is an env that can optionally be set in the container, # populated at build time from the Dockerfile. A typical use case would -# be to pass --host/--target args to trigger cross-compilation -# -# This can be augmented by make local args in $CI_CONFIGURE_ARGS -"$CI_CONFIGURE" $CONFIGURE_OPTS $CI_CONFIGURE_ARGS -if test $? != 0; then - test -f config.log && cat config.log - exit 1 -fi -find -name test-suite.log -delete +# be to pass options to trigger cross-compilation -make -j"$CI_SMP" $CI_MAKE_ARGS +meson build --werror $MESON_OPTS $CI_MESON_ARGS || \ +(cat build/meson-logs/meson-log.txt && exit 1) -if test $? != 0; then \ - LOGS=$(find -name test-suite.log) - if test "$LOGS"; then - echo "=== LOG FILE(S) START ===" - cat $LOGS - echo "=== LOG FILE(S) END ===" - fi - exit 1 -fi +ninja -C build $CI_NINJA_ARGS -- 2.26.2

2 4

[PATCH] NEWS: add note about virt-ssh-helper perf improvements
by Daniel P. Berrangé 27 Nov '20

27 Nov '20

Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- NEWS.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index aa8a217eb6..8245e34efa 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -44,6 +44,13 @@ v6.10.0 (unreleased) * **Bug fixes** + * remote: fixed performance regression in SSH tunnelling + + The virt-ssh-helper binary introduced in 6.9.0 had very + poor scalability which impacted libvirt tunnelled migration + and storage volume upload/download in particular. It has been + updated and now has performance on a par with netcat. + * **Removed features** * hyperv: removed support for the Hyper-V V1 WMI API -- 2.28.0

2 1