April 2020 - Devel - libvirt List Archives

[libvirt PATCH 0/2] Improve entry point for new contributors
by Andrea Bolognani 14 Apr '20

14 Apr '20

Use the path most people will look for, reduce building instructions to the bare minimum necessary to locally bootstrap the learning process, and ensure duplication doesn't creep in over time by pointing to the full documentation. Andrea Bolognani (2): CONTRIBUTING: Add entry point for new contributors README-hacking: Drop from the git repository CONTRIBUTING.md | 18 ++++++++++++++++ README-hacking | 56 ------------------------------------------------- 2 files changed, 18 insertions(+), 56 deletions(-) create mode 100644 CONTRIBUTING.md delete mode 100644 README-hacking -- 2.25.1

2 6

[libvirt-jenkins-ci PATCH v3 0/5] Introduce the new 'gitlab' flavor
by Erik Skultety 14 Apr '20

14 Apr '20

since v1: - .bash_logout is removed from the user home environment (causes builds to fail on Debian) - gitlab-runner is now downloaded to and executed from user's bin/ directory instead of using /usr/local/bin - both RC and systemd service files now define the user explicitly that the init system changes to when spawning the gitlab-runner service - the gitlab flavour tasks file now relies on shell's 'creates' module argument so that the agent registration is idempotent rather than touching the dirs/files explicitly - both gitlab URL and runner token are now files editable under ~/.config/lcitool since v2: - download location changed to /usr/local/bin - config location changed to /etc/gitlab-runner (readable to the gitlab user) - machine tagging changed to 'os_version-distro_major_number' - fixed the regression caused by the gitlab code which was checking the gitlab config files unconditionally for all flavours - cosmetic syntactic changes to the Python code as requested by the reviewer Both the config.toml improvement in terms of replacing the ~/.config/lcitool files as discussed during the v1 review process as well as cloudinit usage is planned for a separate respective series, so for now, we'll have to live with 2 new config files and cloud images will need to be built manually. Erik Skultety (5): guests: users: Discard the .bash_logout skeleton file guests: templates: Introduce a gitlab-runner systemd service template guests: templates: Introduce a gitlab-runner RC init service template guests: Introduce the new 'gitlab' flavor guests: lcitool: Enable the new 'gitlab' flavor in the lcitool script guests/lcitool | 46 ++++++++++++++- guests/playbooks/update/main.yml | 5 ++ guests/playbooks/update/tasks/gitlab.yml | 58 +++++++++++++++++++ guests/playbooks/update/tasks/users.yml | 3 +- .../update/templates/gitlab-runner.j2 | 31 ++++++++++ .../update/templates/gitlab-runner.service.j2 | 14 +++++ 6 files changed, 154 insertions(+), 3 deletions(-) create mode 100644 guests/playbooks/update/tasks/gitlab.yml create mode 100644 guests/playbooks/update/templates/gitlab-runner.j2 create mode 100644 guests/playbooks/update/templates/gitlab-runner.service.j2 -- 2.25.1

2 12

[PATCH ocaml 0/1] RFC: add/switch to dune build system
by Pino Toscano 14 Apr '20

14 Apr '20

ocaml-libvirt is currently built using manually written makefiles, and an autoconf script to detect libvirt (using pkg-config). While this seems OK, it has a lot of downsides: - all the OCaml rules are manually written - there is some form of duplication in the OCaml rules - the bytecode vs native build is fragile Initially named jhbuild, dune is a native OCaml build system which plans to eventually replace the others (omake, oasis, ocamlbuild), with a growing user base in the OCaml community. This is a very simple patch adding support for dune. The idea is to replace the current autoconf+makefile build system, which can be removed more or less easily. I'd prefer to not maintain two build systems at once, so this patch is also a way to start a discussion about the proposed approach. Things not yet done (pending the discussion): - rewrite/simplify the release bits, currently interwonen in the makefiles - update the documentation - update libvirt-ci - remove the autoconf+makefile build system Pino Toscano (1): Add dune build system .gitignore | 3 +++ dune | 18 +++++++++++++ dune-project | 1 + dune.inc | 22 ++++++++++++++++ examples/dune | 50 ++++++++++++++++++++++++++++++++++++ libvirt/discover.ml | 40 +++++++++++++++++++++++++++++ libvirt/dune | 57 ++++++++++++++++++++++++++++++++++++++++++ libvirt/dune-project | 2 ++ libvirt/mllibvirt.opam | 33 ++++++++++++++++++++++++ 9 files changed, 226 insertions(+) create mode 100644 dune create mode 100644 dune-project create mode 100644 dune.inc create mode 100644 examples/dune create mode 100644 libvirt/discover.ml create mode 100644 libvirt/dune create mode 100644 libvirt/dune-project create mode 100644 libvirt/mllibvirt.opam -- 2.25.2

2 2

[PATCH] driver.c: Use g_autofree
by Seeteena Thoufeek 14 Apr '20

14 Apr '20

This is the only instance of g_autofree change applicable for driver.c Signed-off-by: Seeteena Thoufeek <s1seetee(a)linux.vnet.ibm.com> --- src/driver.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/src/driver.c b/src/driver.c index a2047be..e745210 100644 --- a/src/driver.c +++ b/src/driver.c @@ -48,7 +48,7 @@ virDriverLoadModule(const char *name, const char *regfunc, bool required) { - char *modfile = NULL; + g_autofree char *modfile = NULL; int ret; VIR_DEBUG("Module load %s", name); @@ -62,9 +62,6 @@ virDriverLoadModule(const char *name, return -1; ret = virModuleLoad(modfile, regfunc, required); - - VIR_FREE(modfile); - return ret; } -- 1.8.3.1

2 1

FYI: Report about libvirt failing to compile on MSYS2 with an XDR error
by Richard W.M. Jones 14 Apr '20

14 Apr '20

.. is here: https://github.com/msys2/MINGW-packages/issues/6384 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v

1 0

[PATCH 0/4] libxl: add e820_host option to the libvirt config
by Marek Marczykowski-Górecki 13 Apr '20

13 Apr '20

Context: - old thread on xen-devel: http://xen.markmail.org/thread/awcswnywzei4s65e - new thread here: https://www.redhat.com/archives/libvir-list/2020-April/msg00447.html Marek Marczykowski-Górecki (4): conf: add xen specific feature: e820_host libxl: make use of e820_host feature xenconfig: parse e820_host option tests: check e820_host option handling docs/schemas/domaincommon.rng | 16 +++- src/conf/domain_conf.c | 106 ++++++++++++++++++- src/conf/domain_conf.h | 9 ++- src/libxl/libxl_conf.c | 14 ++- src/libxl/xen_common.c | 14 ++- src/qemu/qemu_validate.c | 1 +- tests/xlconfigdata/test-paravirt-e820_host.cfg | 13 ++- tests/xlconfigdata/test-paravirt-e820_host.xml | 37 ++++++- tests/xlconfigtest.c | 1 +- 9 files changed, 211 insertions(+) create mode 100644 tests/xlconfigdata/test-paravirt-e820_host.cfg create mode 100644 tests/xlconfigdata/test-paravirt-e820_host.xml base-commit: 7118bdee1550b6022e7362402ca8204add4cf80b -- git-series 0.9.1

2 8

[PATCH] util: fix iteration in virSocketAddrResolveService
by Nikolay Shirokovskiy 13 Apr '20

13 Apr '20

getaddrinfo returns linked list. Fix iteration accordingly. Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy(a)virtuozzo.com> --- src/util/virsocketaddr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/util/virsocketaddr.c b/src/util/virsocketaddr.c index 4cad329..4c9f124 100644 --- a/src/util/virsocketaddr.c +++ b/src/util/virsocketaddr.c @@ -271,7 +271,7 @@ int virSocketAddrResolveService(const char *service) port = ntohs(in.sin6_port); goto cleanup; } - tmp++; + tmp = tmp->ai_next; } virReportError(VIR_ERR_SYSTEM_ERROR, -- 1.8.3.1

2 1

[PATCH] formatdomain.html.in: document emulator/vcpu pin delay
by Daniel Henrique Barboza 10 Apr '20

10 Apr '20

In a guest with only one vcpu, when pinning the emulator in say CPU184 and the vcpu0 in CPU0 of the host, the user might expect that only CPU0 and CPU184 of the host will be used by the guest. The reality is that Libvirt takes some time to honor the emulator and vcpu pinning, taking care of NUMA constraints first. This will result in other CPUs of the host being potentially used by the QEMU thread until the emulator/vcpu pinning is done. The user then might be confused by the output of 'virsh cpu-stats' in this scenario, showing around 200 microseconds of cycles being spent in other CPUs. Let's document this behavior, which is explained in detail in Libvirt commit v5.0.0-199-gf136b83139, in the cputune section of formatdomain.html.in. Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com> --- docs/formatdomain.html.in | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 47e10a836c..932bf8841f 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -808,6 +808,13 @@ <dd> The optional <code>cputune</code> element provides details regarding the CPU tunable parameters for the domain. + + Note: for the qemu driver, the optional <code>vcpupin</code> + and <code>emulatorpin</code> pinning settings are honored after + the emulator is launched and NUMA constraints considered. This + means that it is expected that other physical CPUs of the host + will be used during this time by the domain, which will be + reflected by the output of <code>virsh cpu-stats</code>. <span class="since">Since 0.9.0</span> </dd> <dt><code>vcpupin</code></dt> -- 2.25.1

2 1

[libvirt-ci PATCH v2] dco: build and publish a container image for DCO check
by Daniel P. Berrangé 09 Apr '20

09 Apr '20

This builds a container image containing the DCO checking script. It is then published at: registry.gitlab.com/libvirt/libvirt-jenkins-ci/check-dco:master from where it can be used in all other project repositories. This avoids having to copy the check-dco.py script into every repo when we want to make changes to it. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- In v2: - Drop -f arg to docker - Drop .py suffix .gitlab-ci.yml | 39 +++++++++++++++++ check-dco/Dockerfile | 6 +++ check-dco/check-dco.py | 99 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 144 insertions(+) create mode 100644 .gitlab-ci.yml create mode 100644 check-dco/Dockerfile create mode 100755 check-dco/check-dco.py diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml new file mode 100644 index 0000000..7735bb3 --- /dev/null +++ b/.gitlab-ci.yml @@ -0,0 +1,39 @@ + +stages: + - containers + - postbuild + +.build_container_template: &build_container_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - docker info + - docker login registry.gitlab.com -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} + script: + - docker build --tag ${CI_REGISTRY_IMAGE}/${NAME}:${CI_COMMIT_REF_SLUG} ${NAME} + - docker push ${CI_REGISTRY_IMAGE}/${NAME}:${CI_COMMIT_REF_SLUG} + after_script: + - docker logout + +build-dco: + <<: *build_container_definition + variables: + NAME: check-dco + +# Check that all commits are signed-off for the DCO. Skip +# on master branch and -maint branches, since we only need +# to test developer's personal branches. +check-dco: + # In most projects we'd put this in a "prebuild", but this + # repo is a special case as we need to actually build the + # container first! + stage: postbuild + image: ${CI_REGISTRY_IMAGE}/check-dco:${CI_COMMIT_REF_SLUG} + script: + - /check-dco + only: + - branches + except: + - master diff --git a/check-dco/Dockerfile b/check-dco/Dockerfile new file mode 100644 index 0000000..366f72b --- /dev/null +++ b/check-dco/Dockerfile @@ -0,0 +1,6 @@ +FROM centos:8 + +RUN dnf -y install python3 git && \ + dnf -y clean all + +COPY check-dco.py /check-dco diff --git a/check-dco/check-dco.py b/check-dco/check-dco.py new file mode 100755 index 0000000..f8204ca --- /dev/null +++ b/check-dco/check-dco.py @@ -0,0 +1,99 @@ +#!/usr/bin/env python3 +# +# ci-job.py: validate all commits are signed off +# +# Copyright (C) 2020 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# <http://www.gnu.org/licenses/>. + +import os +import os.path +import sys +import subprocess + +cwd = os.getcwd() +reponame = os.path.basename(cwd) +repourl = "https://gitlab.com/libvirt/%s.git" % reponame + +subprocess.check_call(["git", "remote", "add", "dcocheck", repourl]) +subprocess.check_call(["git", "fetch", "dcocheck", "master"], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL) + +ancestor = subprocess.check_output(["git", "merge-base", "dcocheck/master", "HEAD"], + universal_newlines=True) + +ancestor = ancestor.strip() + +subprocess.check_call(["git", "remote", "rm", "dcocheck"]) + +errors = False + +print("\nChecking for 'Signed-off-by: NAME <EMAIL>' on all commits since %s...\n" % ancestor) + +log = subprocess.check_output(["git", "log", "--format=%H %s", ancestor + "..."], + universal_newlines=True) + +if log == "": + commits = [] +else: + commits = [[c[0:40], c[41:]] for c in log.strip().split("\n")] + +for sha, subject in commits: + + msg = subprocess.check_output(["git", "show", "-s", sha], + universal_newlines=True) + lines = msg.strip().split("\n") + + print("🔍 %s %s" % (sha, subject)) + sob = False + for line in lines: + if "Signed-off-by:" in line: + sob = True + if "localhost" in line: + print(" ❌ FAIL: bad email in %s" % line) + errors = True + + if not sob: + print(" ❌ FAIL missing Signed-off-by tag") + errors = True + +if errors: + print(""" + +❌ ERROR: One or more commits are missing a valid Signed-off-By tag. + + +This project requires all contributors to assert that their contributions +are provided in compliance with the terms of the Developer's Certificate +of Origin 1.1 (DCO): + + https://developercertificate.org/ + +To indicate acceptance of the DCO every commit must have a tag + + Signed-off-by: REAL NAME <EMAIL> + +This can be achieved by passing the "-s" flag to the "git commit" command. + +To bulk update all commits on current branch "git rebase" can be used: + + git rebase -i master -x 'git commit --amend --no-edit -s' + +""") + + sys.exit(1) + +sys.exit(0) -- 2.24.1

2 1

[libvirt-jenkins-ci PATCH] dco: build and publish a container image for DCO check
by Daniel P. Berrangé 09 Apr '20

09 Apr '20

This builds a container image containing the DCO checking script. It is then published at: registry.gitlab.com/libvirt/libvirt-jenkins-ci/check-dco:master from where it can be used in all other project repositories. This avoids having to copy the check-dco.py script into every repo when we want to make changes to it. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- .gitlab-ci.yml | 39 +++++++++++++++++ check-dco/Dockerfile | 6 +++ check-dco/check-dco.py | 99 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 144 insertions(+) create mode 100644 .gitlab-ci.yml create mode 100644 check-dco/Dockerfile create mode 100755 check-dco/check-dco.py diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml new file mode 100644 index 0000000..83b0d64 --- /dev/null +++ b/.gitlab-ci.yml @@ -0,0 +1,39 @@ + +stages: + - containers + - postbuild + +.build_container_template: &build_container_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - docker info + - docker login registry.gitlab.com -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} + script: + - docker build --tag ${CI_REGISTRY_IMAGE}/${NAME}:${CI_COMMIT_REF_SLUG} -f ${NAME}/Dockerfile ${NAME} + - docker push ${CI_REGISTRY_IMAGE}/${NAME}:${CI_COMMIT_REF_SLUG} + after_script: + - docker logout + +build-dco: + <<: *build_container_definition + variables: + NAME: check-dco + +# Check that all commits are signed-off for the DCO. Skip +# on master branch and -maint branches, since we only need +# to test developer's personal branches. +check-dco: + # In most projects we'd put this in a "prebuild", but this + # repo is a special case as we need to actually build the + # container first! + stage: postbuild + image: ${CI_REGISTRY_IMAGE}/check-dco:${CI_COMMIT_REF_SLUG} + script: + - /check-dco.py + only: + - branches + except: + - master diff --git a/check-dco/Dockerfile b/check-dco/Dockerfile new file mode 100644 index 0000000..86cada7 --- /dev/null +++ b/check-dco/Dockerfile @@ -0,0 +1,6 @@ +FROM centos:8 + +RUN dnf -y install python3 git && \ + dnf -y clean all + +COPY check-dco.py check-dco.py diff --git a/check-dco/check-dco.py b/check-dco/check-dco.py new file mode 100755 index 0000000..f8204ca --- /dev/null +++ b/check-dco/check-dco.py @@ -0,0 +1,99 @@ +#!/usr/bin/env python3 +# +# ci-job.py: validate all commits are signed off +# +# Copyright (C) 2020 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# <http://www.gnu.org/licenses/>. + +import os +import os.path +import sys +import subprocess + +cwd = os.getcwd() +reponame = os.path.basename(cwd) +repourl = "https://gitlab.com/libvirt/%s.git" % reponame + +subprocess.check_call(["git", "remote", "add", "dcocheck", repourl]) +subprocess.check_call(["git", "fetch", "dcocheck", "master"], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL) + +ancestor = subprocess.check_output(["git", "merge-base", "dcocheck/master", "HEAD"], + universal_newlines=True) + +ancestor = ancestor.strip() + +subprocess.check_call(["git", "remote", "rm", "dcocheck"]) + +errors = False + +print("\nChecking for 'Signed-off-by: NAME <EMAIL>' on all commits since %s...\n" % ancestor) + +log = subprocess.check_output(["git", "log", "--format=%H %s", ancestor + "..."], + universal_newlines=True) + +if log == "": + commits = [] +else: + commits = [[c[0:40], c[41:]] for c in log.strip().split("\n")] + +for sha, subject in commits: + + msg = subprocess.check_output(["git", "show", "-s", sha], + universal_newlines=True) + lines = msg.strip().split("\n") + + print("🔍 %s %s" % (sha, subject)) + sob = False + for line in lines: + if "Signed-off-by:" in line: + sob = True + if "localhost" in line: + print(" ❌ FAIL: bad email in %s" % line) + errors = True + + if not sob: + print(" ❌ FAIL missing Signed-off-by tag") + errors = True + +if errors: + print(""" + +❌ ERROR: One or more commits are missing a valid Signed-off-By tag. + + +This project requires all contributors to assert that their contributions +are provided in compliance with the terms of the Developer's Certificate +of Origin 1.1 (DCO): + + https://developercertificate.org/ + +To indicate acceptance of the DCO every commit must have a tag + + Signed-off-by: REAL NAME <EMAIL> + +This can be achieved by passing the "-s" flag to the "git commit" command. + +To bulk update all commits on current branch "git rebase" can be used: + + git rebase -i master -x 'git commit --amend --no-edit -s' + +""") + + sys.exit(1) + +sys.exit(0) -- 2.24.1

2 7