May 2022 - Devel - libvirt List Archives

Entering freeze for libvirt-8.4.0
by Jiri Denemark 30 May '22

30 May '22

I have just tagged v8.4.0-rc1 in the repository and pushed signed tarballs and source RPMs to https://libvirt.org/sources/ Please give the release candidate some testing and in case you find a serious issue which should have a fix in the upcoming release, feel free to reply to this thread to make sure the issue is more visible. If you have not done so yet, please update NEWS.rst to document any significant change you made since the last release. Thanks, Jirka

1 1

[PATCH] qemuProcessStop: Don't try to remove QoS on already removed TAP
by Michal Privoznik 30 May '22

30 May '22

When cleaning up after stopped domain, one of the things we do is attempt to clear QoS settings on OVS type interfaces. Well, this is needless because they were removed just a couple of lines above. As a result, the attempt fails and a warning is printed into logs, polluting them needlessly. Closes: https://gitlab.com/libvirt/libvirt/-/issues/313 Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_process.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 07acb1c427..c6d965d720 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -8166,12 +8166,6 @@ void qemuProcessStop(virQEMUDriver *driver, else VIR_WARN("Unable to release network device '%s'", NULLSTR(net->ifname)); } - - if (virDomainNetDefIsOvsport(net) && - virNetDevOpenvswitchInterfaceClearQos(net->ifname, vm->def->uuid) < 0) { - VIR_WARN("cannot clear bandwidth setting for ovs device : %s", - net->ifname); - } } retry: -- 2.35.1

2 1

[PATCH] docs: patches: Add a note about reviews and contacting developers
by Peter Krempa 30 May '22

30 May '22

Add a note outling best practices around review and responding to it. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- docs/submitting-patches.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/submitting-patches.rst b/docs/submitting-patches.rst index 7cb5c2e172..d6bc9b623c 100644 --- a/docs/submitting-patches.rst +++ b/docs/submitting-patches.rst @@ -89,3 +89,18 @@ Moreover, such patch needs to be prefixed correctly with ``--subject-prefix=PATCHv2`` appended to ``git send-email`` (substitute ``v2`` with the correct version if needed though). + +Reviews +------- + +Please note that reviewing may take a lot of effort so be patient. Try to react +to questions from reviewers. There's no need to acknowledge individual points +before addressing them in next version. Make sure to address all of reviewers +requests, or start a discussion if you don't agree with reviewers points. + +Feel free to respond to any flaw in any patch on the list, do not feel obliged +to review the rest of the series in such case. + +If your patches don't get attention after some time (~week) feel free to +remind us on list that your patches still need review. **Do not** contact +individual maintainers directly to review your patches. -- 2.35.3

2 1

[PATCH 0/2] 2 release notes for v8.4.0
by Han Han 30 May '22

30 May '22

Han Han (2): news: Add qemu-vdagent news: Add APIs virDomainSaveParams virDomainRestoreParams NEWS.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) -- 2.36.1

1 2

[libvirt PATCH] tools: add virt-qmp-proxy for proxying QMP clients to libvirt QEMU guests
by Daniel P. Berrangé 27 May '22

27 May '22

Libvirt provides QMP passthrough APIs for the QEMU driver and these are exposed in virsh. It is not especially pleasant, however, using the raw QMP JSON syntax. QEMU has a tool 'qmp-shell' which can speak QMP and exposes a human friendly interactive shell. It is not possible to use this with libvirt managed guest, however, since only one client can attach to he QMP socket at any point in time. The virt-qmp-proxy tool aims to solve this problem. It opens a UNIX socket and listens for incoming client connections, speaking QMP on the connected socket. It will forward any QMP commands received onto the running libvirt QEMU guest, and forward any replies back to the QMP client. $ virsh start demo $ virt-qmp-proxy demo demo.qmp & $ qmp-shell demo.qmp Welcome to the QMP low-level shell! Connected to QEMU 6.2.0 (QEMU) query-kvm { "return": { "enabled": true, "present": true } } Note this tool of course has the same risks as the raw libvirt QMP passthrough. It is safe to run query commands to fetch information but commands which change the QEMU state risk disrupting libvirt's management of QEMU, potentially resulting in data loss/corruption in the worst case. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- CC'ing QEMU since this is likely of interest to maintainers and users who work with QEMU and libvirt Note this impl is fairly crude in that it assumes it is receiving the QMP commands linewise one at a time. None the less it is good enough to work with qmp-shell already, so I figured it was worth exposing to the world. It also lacks support for forwarding events back to the QMP client. docs/manpages/meson.build | 1 + docs/manpages/virt-qmp-proxy.rst | 123 ++++++++++++++++++++++++++++ tools/meson.build | 5 ++ tools/virt-qmp-proxy | 133 +++++++++++++++++++++++++++++++ 4 files changed, 262 insertions(+) create mode 100644 docs/manpages/virt-qmp-proxy.rst create mode 100755 tools/virt-qmp-proxy diff --git a/docs/manpages/meson.build b/docs/manpages/meson.build index ba673cf472..4162a9969a 100644 --- a/docs/manpages/meson.build +++ b/docs/manpages/meson.build @@ -18,6 +18,7 @@ docs_man_files = [ { 'name': 'virt-pki-query-dn', 'section': '1', 'install': true }, { 'name': 'virt-pki-validate', 'section': '1', 'install': true }, { 'name': 'virt-qemu-run', 'section': '1', 'install': conf.has('WITH_QEMU') }, + { 'name': 'virt-qmp-proxy', 'section': '1', 'install': conf.has('WITH_QEMU') }, { 'name': 'virt-xml-validate', 'section': '1', 'install': true }, { 'name': 'libvirt-guests', 'section': '8', 'install': conf.has('WITH_LIBVIRTD') }, diff --git a/docs/manpages/virt-qmp-proxy.rst b/docs/manpages/virt-qmp-proxy.rst new file mode 100644 index 0000000000..94679406ab --- /dev/null +++ b/docs/manpages/virt-qmp-proxy.rst @@ -0,0 +1,123 @@ +============== +virt-qmp-proxy +============== + +-------------------------------------------------- +Expose a QMP proxy server for a libvirt QEMU guest +-------------------------------------------------- + +:Manual section: 1 +:Manual group: Virtualization Support + +.. contents:: + + +SYNOPSIS +======== + +``virt-qmp-proxy`` [*OPTION*]... *DOMAIN* *QMP-SOCKET-PATH* + + +DESCRIPTION +=========== + +This tool provides a way to expose a QMP proxy server that communicates +with a QEMU guest managed by libvirt. This enables standard QMP client +tools to interact with libvirt managed guests. + +**NOTE: use of this tool will result in the running QEMU guest being +marked as tainted.** It is strongly recommended that this tool *only be +used to send commands which query information* about the running guest. +If this tool is used to make changes to the state of the guest, this +may have negative interactions with the QEMU driver, resulting in an +inability to manage the guest operation thereafter, and in the worst +case **potentially lead to data loss or corruption**. + +The ``virt-qmp-proxy`` program will listen on a UNIX socket for incoming +client connections, and run the QMP protocol over the connection. Any +commands received will be sent to the running libvirt guest, and replies +sent back. + +The ``virt-qemu-proxy`` program may be interrupted (eg Ctrl-C) when it +is no longer required. The libvirt QEMU guest will continue running. + + +OPTIONS +======= + +*DOMAIN* + +The ID or UUID or Name of the libvirt QEMU guest. + +*QMP-SOCKET-PATH* + +The filesystem path at which to run the QMP server, listening for +incoming connections. + +``-c`` *CONNECTION-URI* +``--connect``\ =\ *CONNECTION-URI* + +The URI for the connection to the libvirt QEMU driver. If omitted, +a URI will be auto-detected. + +``-v``, ``--verbose`` + +Run in verbose mode, printing all QMP commands and replies that +are handled. + +``-h``, ``--help`` + +Display the command line help. + + +EXIT STATUS +=========== + +Upon successful shutdown, an exit status of 0 will be set. Upon +failure a non-zero status will be set. + + +AUTHOR +====== + +Daniel P. Berrangé + + +BUGS +==== + +Please report all bugs you discover. This should be done via either: + +#. the mailing list + + `https://libvirt.org/contact.html <https://libvirt.org/contact.html>`_ + +#. the bug tracker + + `https://libvirt.org/bugs.html <https://libvirt.org/bugs.html>`_ + +Alternatively, you may report bugs to your software distributor / vendor. + +NOTE: at this time there is no support for forwarding QMP events back +to the clients + +COPYRIGHT +========= + +Copyright (C) 2022 by Red Hat, Inc. + + +LICENSE +======= + +``virt-qemu-proxy`` is distributed under the terms of the GNU LGPL v2+. +This is free software; see the source for copying conditions. There +is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR +PURPOSE + + +SEE ALSO +======== + +virsh(1), `https://libvirt.org/ <https://libvirt.org/>`_, +`QMP reference <https://www.qemu.org/docs/master/interop/qemu-qmp-ref.html>`_ diff --git a/tools/meson.build b/tools/meson.build index bb28a904dc..4e959ecf0b 100644 --- a/tools/meson.build +++ b/tools/meson.build @@ -320,6 +320,11 @@ if conf.has('WITH_LIBVIRTD') endif endif +if conf.has('WITH_QEMU') + install_data('virt-qmp-proxy', + install_dir: bindir) +endif + if bash_completion_dep.found() subdir('bash-completion') endif diff --git a/tools/virt-qmp-proxy b/tools/virt-qmp-proxy new file mode 100755 index 0000000000..57f9759fab --- /dev/null +++ b/tools/virt-qmp-proxy @@ -0,0 +1,133 @@ +#!/usr/bin/env python3 + +import argparse +import libvirt +import libvirt_qemu +import os +import re +import socket +import sys +import json + + +def get_domain(uri, domstr): + conn = libvirt.open(uri) + + dom = None + if re.match(r'^\d+$', domstr): + dom = conn.lookupByID(int(domstr)) + elif re.match(r'^[+a-f0-9]+$', domstr): + dom = conn.lookupByUUIDString(domstr) + else: + dom = conn.lookupByName(domstr) + + if not dom.isActive(): + raise Exception( + "Domain must be running to validate measurement") + + return conn, dom + + +def qmp_server(conn, dom, client, verbose): + ver = conn.getVersion() + major = int(ver / 1000000) % 1000 + minor = int(ver / 1000) % 1000 + micro = ver % 1000 + + greetingobj = { + "QMP": { + "version": { + "qemu": { + "major": major, + "minor": minor, + "micro": micro, + }, + "package": f"qemu-{major}.{minor}.{micro}", + }, + "capabilities": [ + "oob" + ], + } + } + greeting = json.dumps(greetingobj) + "\r\n" + if verbose: + print(greeting, end='') + client.send(greeting.encode("utf-8")) + + while True: + # XXX shouldn't blindly assume this one read + # will fully capture one-and-only-one cmd + cmd = client.recv(1024).decode('utf8') + if verbose: + print(cmd) + + if cmd == "": + break + + if "qmp_capabilities" in cmd: + capabilitiesobj = { + "return": {}, + } + capabilities = json.dumps(capabilitiesobj) + "\r\n" + if verbose: + print(capabilities, end='') + client.send(capabilities.encode("utf-8")) + continue + + id = None + if "id" in cmd: + id = cmd[id] + + res = libvirt_qemu.qemuMonitorCommand(dom, cmd, 0) + + resobj = json.loads(res) + del resobj["id"] + if id is not None: + resobj["id"] = id + res = json.dumps(resobj) + "\r\n" + if verbose: + print(res, end='') + + client.send(res.encode('utf8')) + + +def parse_commandline(): + parser = argparse.ArgumentParser(description="Libvirt QMP proxy") + parser.add_argument("--connect", "-c", + help="Libvirt QEMU driver connection URI") + parser.add_argument("--verbose", "-v", action='store_true', + help="Display QMP traffic") + parser.add_argument("domain", metavar="DOMAIN", + help="Libvirt guest domain ID/UUID/Name") + parser.add_argument("sockpath", metavar="QMP-SOCK-PATH", + help="UNIX socket path for QMP server") + + return parser.parse_args() + + +def main(): + args = parse_commandline() + + conn, dom = get_domain(args.connect, args.domain) + + if conn.getType() != "QEMU": + raise Exception("QMP proxy requires a QEMU driver connection not %s" % + conn.getType()) + + sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + if os.path.exists(args.sockpath): + os.unlink(args.sockpath) + sock.bind(args.sockpath) + sock.listen(1) + + while True: + client, peeraddr = sock.accept() + qmp_server(conn, dom, client, args.verbose) + + +try: + main() + sys.exit(0) +except Exception as e: + print("%s: %s" % (sys.argv[0], str(e))) + sys.exit(1) -- 2.36.1

4 8

Upstream issue triage
by Peter Krempa 27 May '22

27 May '22

Hi, over the last week I've (re)triaged all of the upstream open issues. Now there are still some issues which per the classification are unconfirmed. That means I'm not sure whether the bug or feature request makes sense. I'd highly appreciate for assistance in looking at those. Please refer to the links on the new page I'm adding in the patch as the gitlab links are quite verbose. https://pipo.sk.gitlab.io/-/libvirt/-/jobs/2515052302/artifacts/website/iss… or once it gets merged at https://libvirt.org/issue-handling.html#issues-needing-attention Peter Krempa (1): docs: Add article about handling upstream issues docs/docs.rst | 3 + docs/issue-handling.rst | 164 ++++++++++++++++++++++++++++++++++++++++ docs/meson.build | 1 + 3 files changed, 168 insertions(+) create mode 100644 docs/issue-handling.rst -- 2.35.3

1 1

[PATCH] ci: Refresh generated files
by Martin Kletzander 27 May '22

27 May '22

Notable changes: * 'lcitool manifest' now generates absolute include paths Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- Pushed under the 'build-breaker' and 'just-ci-update' rules (just in case the latter also exists). ci/gitlab.yml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ci/gitlab.yml b/ci/gitlab.yml index 379a41111cab..cab4e6074c21 100644 --- a/ci/gitlab.yml +++ b/ci/gitlab.yml @@ -5,8 +5,8 @@ # https://gitlab.com/libvirt/libvirt-ci include: - - local: 'ci/gitlab/container-templates.yml' - - local: 'ci/gitlab/build-templates.yml' - - local: 'ci/gitlab/sanity-checks.yml' - - local: 'ci/gitlab/containers.yml' - - local: 'ci/gitlab/builds.yml' + - local: '/ci/gitlab/container-templates.yml' + - local: '/ci/gitlab/build-templates.yml' + - local: '/ci/gitlab/sanity-checks.yml' + - local: '/ci/gitlab/containers.yml' + - local: '/ci/gitlab/builds.yml' -- 2.35.1

2 3

[PATCH] NEWS: Mention 'absolute' clock offset
by Peter Krempa 27 May '22

27 May '22

Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- NEWS.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index b0d4297d53..e50010f9b2 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -26,6 +26,13 @@ v8.4.0 (unreleased) Support for the recently released IBM Power10 processor was added. + * qemu: Introduce ``absolute`` clock offset + + The ``absolute`` clock offset type allows to set the guest clock to an + arbitrary epoch timestamp at each start. This is useful if some VM needs + to be kept set to an arbitrary time for e.g. testing or working around + broken software. + * **Improvements** * **Bug fixes** -- 2.35.3

2 1

[libvirt PATCH] gitlab: mark job dependencies as optional
by Daniel P. Berrangé 27 May '22

27 May '22

The container jobs are all optional in upstream now, only running if dockerfile changes were made. Thus any build jobs which reference the container jobs need to mark their dependency as optional too. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- .gitlab-ci.yml | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 0fbdd82aae..013f821881 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -62,7 +62,8 @@ website: stage: builds image: $CI_REGISTRY_IMAGE/ci-almalinux-8:latest needs: - - x86_64-almalinux-8-container + - job: x86_64-almalinux-8-container + optional: true before_script: - *script_variables script: @@ -82,7 +83,8 @@ codestyle: stage: sanity_checks image: $CI_REGISTRY_IMAGE/ci-opensuse-leap-153:latest needs: - - x86_64-opensuse-leap-153-container + - job: x86_64-opensuse-leap-153-container + optional: true before_script: - *script_variables script: @@ -98,7 +100,8 @@ potfile: stage: builds image: $CI_REGISTRY_IMAGE/ci-almalinux-8:latest needs: - - x86_64-almalinux-8-container + - job: x86_64-almalinux-8-container + optional: true rules: - if: "$CI_COMMIT_BRANCH == 'master'" before_script: -- 2.36.1

2 1

[PATCH] nss: Compare hostname case insensitive
by Michal Privoznik 27 May '22

27 May '22

There are some tools that convert hostname to lowercase before resolving it (e.g. ssh). In a way it makes sense because DNS is case insensitive and in case of ssh the lowercase version is then used to find matching record in its config file. However, our NSS module performs case sensitive comparison, which makes it useless with ssh. Just consider a machine named FooBar. Therefore, switch to case insensitive string comparison. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1777873 Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- build-aux/syntax-check.mk | 2 +- tests/nsstest.c | 1 + tools/nss/libvirt_nss_leases.c | 2 +- tools/nss/libvirt_nss_macs.c | 2 +- 4 files changed, 4 insertions(+), 3 deletions(-) diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 73f8a21c1b..f9f60c934c 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -1337,7 +1337,7 @@ sc_prohibit_enum_impl_with_vir_prefix_in_virsh: ## Exceptions ## ## ---------- ## -exclude_file_name_regexp--sc_avoid_strcase = ^tools/vsh\.h$$ +exclude_file_name_regexp--sc_avoid_strcase = ^tools/(vsh\.h|nss/libvirt_nss_(leases|macs)\.c)$$ exclude_file_name_regexp--sc_avoid_write = ^src/libvirt-stream\.c$$ diff --git a/tests/nsstest.c b/tests/nsstest.c index 2c2e57769b..a03f230452 100644 --- a/tests/nsstest.c +++ b/tests/nsstest.c @@ -173,6 +173,7 @@ mymain(void) # if !defined(LIBVIRT_NSS_GUEST) DO_TEST("fedora", AF_INET, "192.168.122.197", "192.168.122.198", "192.168.122.199", "192.168.122.3"); DO_TEST("gentoo", AF_INET, "192.168.122.254"); + DO_TEST("Gentoo", AF_INET, "192.168.122.254"); DO_TEST("gentoo", AF_INET6, "2001:1234:dead:beef::2"); DO_TEST("gentoo", AF_UNSPEC, "192.168.122.254"); DO_TEST("non-existent", AF_UNSPEC, NULL); diff --git a/tools/nss/libvirt_nss_leases.c b/tools/nss/libvirt_nss_leases.c index 536fcf8608..770dae8625 100644 --- a/tools/nss/libvirt_nss_leases.c +++ b/tools/nss/libvirt_nss_leases.c @@ -272,7 +272,7 @@ findLeasesParserEndMap(void *ctx) } } else { DEBUG("Check name '%s' vs '%s'", parser->name, NULLSTR(parser->entry.hostname)); - if (parser->entry.hostname && !strcmp(parser->name, parser->entry.hostname)) + if (parser->entry.hostname && !strcasecmp(parser->name, parser->entry.hostname)) found = true; } DEBUG("Found %d", found); diff --git a/tools/nss/libvirt_nss_macs.c b/tools/nss/libvirt_nss_macs.c index d4b165eef6..f45d149793 100644 --- a/tools/nss/libvirt_nss_macs.c +++ b/tools/nss/libvirt_nss_macs.c @@ -142,7 +142,7 @@ findMACsParserEndMap(void *ctx) if (parser->state != FIND_MACS_STATE_ENTRY) return 0; - if (!strcmp(parser->entry.name, parser->name)) { + if (!strcasecmp(parser->entry.name, parser->name)) { char **macs = realloc(*parser->macs, sizeof(char *) * ((*parser->nmacs) + parser->entry.nmacs)); if (!macs) -- 2.35.1

3 2