- Devel - libvirt List Archives

[PATCH 0/4] libxl: add e820_host option to the libvirt config
by Marek Marczykowski-Górecki 13 Apr '20

13 Apr '20

Context: - old thread on xen-devel: http://xen.markmail.org/thread/awcswnywzei4s65e - new thread here: https://www.redhat.com/archives/libvir-list/2020-April/msg00447.html Marek Marczykowski-Górecki (4): conf: add xen specific feature: e820_host libxl: make use of e820_host feature xenconfig: parse e820_host option tests: check e820_host option handling docs/schemas/domaincommon.rng | 16 +++- src/conf/domain_conf.c | 106 ++++++++++++++++++- src/conf/domain_conf.h | 9 ++- src/libxl/libxl_conf.c | 14 ++- src/libxl/xen_common.c | 14 ++- src/qemu/qemu_validate.c | 1 +- tests/xlconfigdata/test-paravirt-e820_host.cfg | 13 ++- tests/xlconfigdata/test-paravirt-e820_host.xml | 37 ++++++- tests/xlconfigtest.c | 1 +- 9 files changed, 211 insertions(+) create mode 100644 tests/xlconfigdata/test-paravirt-e820_host.cfg create mode 100644 tests/xlconfigdata/test-paravirt-e820_host.xml base-commit: 7118bdee1550b6022e7362402ca8204add4cf80b -- git-series 0.9.1

2 8

[PATCH] util: fix iteration in virSocketAddrResolveService
by Nikolay Shirokovskiy 13 Apr '20

13 Apr '20

getaddrinfo returns linked list. Fix iteration accordingly. Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy(a)virtuozzo.com> --- src/util/virsocketaddr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/util/virsocketaddr.c b/src/util/virsocketaddr.c index 4cad329..4c9f124 100644 --- a/src/util/virsocketaddr.c +++ b/src/util/virsocketaddr.c @@ -271,7 +271,7 @@ int virSocketAddrResolveService(const char *service) port = ntohs(in.sin6_port); goto cleanup; } - tmp++; + tmp = tmp->ai_next; } virReportError(VIR_ERR_SYSTEM_ERROR, -- 1.8.3.1

2 1

[PATCH] formatdomain.html.in: document emulator/vcpu pin delay
by Daniel Henrique Barboza 10 Apr '20

10 Apr '20

In a guest with only one vcpu, when pinning the emulator in say CPU184 and the vcpu0 in CPU0 of the host, the user might expect that only CPU0 and CPU184 of the host will be used by the guest. The reality is that Libvirt takes some time to honor the emulator and vcpu pinning, taking care of NUMA constraints first. This will result in other CPUs of the host being potentially used by the QEMU thread until the emulator/vcpu pinning is done. The user then might be confused by the output of 'virsh cpu-stats' in this scenario, showing around 200 microseconds of cycles being spent in other CPUs. Let's document this behavior, which is explained in detail in Libvirt commit v5.0.0-199-gf136b83139, in the cputune section of formatdomain.html.in. Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com> --- docs/formatdomain.html.in | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 47e10a836c..932bf8841f 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -808,6 +808,13 @@ <dd> The optional <code>cputune</code> element provides details regarding the CPU tunable parameters for the domain. + + Note: for the qemu driver, the optional <code>vcpupin</code> + and <code>emulatorpin</code> pinning settings are honored after + the emulator is launched and NUMA constraints considered. This + means that it is expected that other physical CPUs of the host + will be used during this time by the domain, which will be + reflected by the output of <code>virsh cpu-stats</code>. <span class="since">Since 0.9.0</span> </dd> <dt><code>vcpupin</code></dt> -- 2.25.1

2 1

[libvirt-ci PATCH v2] dco: build and publish a container image for DCO check
by Daniel P. Berrangé 09 Apr '20

09 Apr '20

This builds a container image containing the DCO checking script. It is then published at: registry.gitlab.com/libvirt/libvirt-jenkins-ci/check-dco:master from where it can be used in all other project repositories. This avoids having to copy the check-dco.py script into every repo when we want to make changes to it. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- In v2: - Drop -f arg to docker - Drop .py suffix .gitlab-ci.yml | 39 +++++++++++++++++ check-dco/Dockerfile | 6 +++ check-dco/check-dco.py | 99 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 144 insertions(+) create mode 100644 .gitlab-ci.yml create mode 100644 check-dco/Dockerfile create mode 100755 check-dco/check-dco.py diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml new file mode 100644 index 0000000..7735bb3 --- /dev/null +++ b/.gitlab-ci.yml @@ -0,0 +1,39 @@ + +stages: + - containers + - postbuild + +.build_container_template: &build_container_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - docker info + - docker login registry.gitlab.com -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} + script: + - docker build --tag ${CI_REGISTRY_IMAGE}/${NAME}:${CI_COMMIT_REF_SLUG} ${NAME} + - docker push ${CI_REGISTRY_IMAGE}/${NAME}:${CI_COMMIT_REF_SLUG} + after_script: + - docker logout + +build-dco: + <<: *build_container_definition + variables: + NAME: check-dco + +# Check that all commits are signed-off for the DCO. Skip +# on master branch and -maint branches, since we only need +# to test developer's personal branches. +check-dco: + # In most projects we'd put this in a "prebuild", but this + # repo is a special case as we need to actually build the + # container first! + stage: postbuild + image: ${CI_REGISTRY_IMAGE}/check-dco:${CI_COMMIT_REF_SLUG} + script: + - /check-dco + only: + - branches + except: + - master diff --git a/check-dco/Dockerfile b/check-dco/Dockerfile new file mode 100644 index 0000000..366f72b --- /dev/null +++ b/check-dco/Dockerfile @@ -0,0 +1,6 @@ +FROM centos:8 + +RUN dnf -y install python3 git && \ + dnf -y clean all + +COPY check-dco.py /check-dco diff --git a/check-dco/check-dco.py b/check-dco/check-dco.py new file mode 100755 index 0000000..f8204ca --- /dev/null +++ b/check-dco/check-dco.py @@ -0,0 +1,99 @@ +#!/usr/bin/env python3 +# +# ci-job.py: validate all commits are signed off +# +# Copyright (C) 2020 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# <http://www.gnu.org/licenses/>. + +import os +import os.path +import sys +import subprocess + +cwd = os.getcwd() +reponame = os.path.basename(cwd) +repourl = "https://gitlab.com/libvirt/%s.git" % reponame + +subprocess.check_call(["git", "remote", "add", "dcocheck", repourl]) +subprocess.check_call(["git", "fetch", "dcocheck", "master"], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL) + +ancestor = subprocess.check_output(["git", "merge-base", "dcocheck/master", "HEAD"], + universal_newlines=True) + +ancestor = ancestor.strip() + +subprocess.check_call(["git", "remote", "rm", "dcocheck"]) + +errors = False + +print("\nChecking for 'Signed-off-by: NAME <EMAIL>' on all commits since %s...\n" % ancestor) + +log = subprocess.check_output(["git", "log", "--format=%H %s", ancestor + "..."], + universal_newlines=True) + +if log == "": + commits = [] +else: + commits = [[c[0:40], c[41:]] for c in log.strip().split("\n")] + +for sha, subject in commits: + + msg = subprocess.check_output(["git", "show", "-s", sha], + universal_newlines=True) + lines = msg.strip().split("\n") + + print("🔍 %s %s" % (sha, subject)) + sob = False + for line in lines: + if "Signed-off-by:" in line: + sob = True + if "localhost" in line: + print(" ❌ FAIL: bad email in %s" % line) + errors = True + + if not sob: + print(" ❌ FAIL missing Signed-off-by tag") + errors = True + +if errors: + print(""" + +❌ ERROR: One or more commits are missing a valid Signed-off-By tag. + + +This project requires all contributors to assert that their contributions +are provided in compliance with the terms of the Developer's Certificate +of Origin 1.1 (DCO): + + https://developercertificate.org/ + +To indicate acceptance of the DCO every commit must have a tag + + Signed-off-by: REAL NAME <EMAIL> + +This can be achieved by passing the "-s" flag to the "git commit" command. + +To bulk update all commits on current branch "git rebase" can be used: + + git rebase -i master -x 'git commit --amend --no-edit -s' + +""") + + sys.exit(1) + +sys.exit(0) -- 2.24.1

2 1

[libvirt-jenkins-ci PATCH] dco: build and publish a container image for DCO check
by Daniel P. Berrangé 09 Apr '20

09 Apr '20

This builds a container image containing the DCO checking script. It is then published at: registry.gitlab.com/libvirt/libvirt-jenkins-ci/check-dco:master from where it can be used in all other project repositories. This avoids having to copy the check-dco.py script into every repo when we want to make changes to it. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- .gitlab-ci.yml | 39 +++++++++++++++++ check-dco/Dockerfile | 6 +++ check-dco/check-dco.py | 99 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 144 insertions(+) create mode 100644 .gitlab-ci.yml create mode 100644 check-dco/Dockerfile create mode 100755 check-dco/check-dco.py diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml new file mode 100644 index 0000000..83b0d64 --- /dev/null +++ b/.gitlab-ci.yml @@ -0,0 +1,39 @@ + +stages: + - containers + - postbuild + +.build_container_template: &build_container_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - docker info + - docker login registry.gitlab.com -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} + script: + - docker build --tag ${CI_REGISTRY_IMAGE}/${NAME}:${CI_COMMIT_REF_SLUG} -f ${NAME}/Dockerfile ${NAME} + - docker push ${CI_REGISTRY_IMAGE}/${NAME}:${CI_COMMIT_REF_SLUG} + after_script: + - docker logout + +build-dco: + <<: *build_container_definition + variables: + NAME: check-dco + +# Check that all commits are signed-off for the DCO. Skip +# on master branch and -maint branches, since we only need +# to test developer's personal branches. +check-dco: + # In most projects we'd put this in a "prebuild", but this + # repo is a special case as we need to actually build the + # container first! + stage: postbuild + image: ${CI_REGISTRY_IMAGE}/check-dco:${CI_COMMIT_REF_SLUG} + script: + - /check-dco.py + only: + - branches + except: + - master diff --git a/check-dco/Dockerfile b/check-dco/Dockerfile new file mode 100644 index 0000000..86cada7 --- /dev/null +++ b/check-dco/Dockerfile @@ -0,0 +1,6 @@ +FROM centos:8 + +RUN dnf -y install python3 git && \ + dnf -y clean all + +COPY check-dco.py check-dco.py diff --git a/check-dco/check-dco.py b/check-dco/check-dco.py new file mode 100755 index 0000000..f8204ca --- /dev/null +++ b/check-dco/check-dco.py @@ -0,0 +1,99 @@ +#!/usr/bin/env python3 +# +# ci-job.py: validate all commits are signed off +# +# Copyright (C) 2020 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# <http://www.gnu.org/licenses/>. + +import os +import os.path +import sys +import subprocess + +cwd = os.getcwd() +reponame = os.path.basename(cwd) +repourl = "https://gitlab.com/libvirt/%s.git" % reponame + +subprocess.check_call(["git", "remote", "add", "dcocheck", repourl]) +subprocess.check_call(["git", "fetch", "dcocheck", "master"], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL) + +ancestor = subprocess.check_output(["git", "merge-base", "dcocheck/master", "HEAD"], + universal_newlines=True) + +ancestor = ancestor.strip() + +subprocess.check_call(["git", "remote", "rm", "dcocheck"]) + +errors = False + +print("\nChecking for 'Signed-off-by: NAME <EMAIL>' on all commits since %s...\n" % ancestor) + +log = subprocess.check_output(["git", "log", "--format=%H %s", ancestor + "..."], + universal_newlines=True) + +if log == "": + commits = [] +else: + commits = [[c[0:40], c[41:]] for c in log.strip().split("\n")] + +for sha, subject in commits: + + msg = subprocess.check_output(["git", "show", "-s", sha], + universal_newlines=True) + lines = msg.strip().split("\n") + + print("🔍 %s %s" % (sha, subject)) + sob = False + for line in lines: + if "Signed-off-by:" in line: + sob = True + if "localhost" in line: + print(" ❌ FAIL: bad email in %s" % line) + errors = True + + if not sob: + print(" ❌ FAIL missing Signed-off-by tag") + errors = True + +if errors: + print(""" + +❌ ERROR: One or more commits are missing a valid Signed-off-By tag. + + +This project requires all contributors to assert that their contributions +are provided in compliance with the terms of the Developer's Certificate +of Origin 1.1 (DCO): + + https://developercertificate.org/ + +To indicate acceptance of the DCO every commit must have a tag + + Signed-off-by: REAL NAME <EMAIL> + +This can be achieved by passing the "-s" flag to the "git commit" command. + +To bulk update all commits on current branch "git rebase" can be used: + + git rebase -i master -x 'git commit --amend --no-edit -s' + +""") + + sys.exit(1) + +sys.exit(0) -- 2.24.1

2 7

[libvirt PATCH] gitdm: Add missing entries
by Andrea Bolognani 09 Apr '20

09 Apr '20

One new company has contributed to libvirt since the last time the gitdm configuration was updated. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- docs/gitdm/companies/others | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/gitdm/companies/others b/docs/gitdm/companies/others index f459163dd2..37d590a6b9 100644 --- a/docs/gitdm/companies/others +++ b/docs/gitdm/companies/others @@ -16,6 +16,7 @@ citrix.com Citrix cloudwatt.com Cloudwatt cmss.chinamobile.com China Mobile codethink.co.uk Codethink +crudebyte.com Crudebyte cumulusnetworks.com Cumulus Networks dataductus.se Data Ductus datagravity.com DataGravity -- 2.25.1

2 2

[libvirt PATCH 0/6] docs: misc updates for gitlab changes
by Daniel P. Berrangé 09 Apr '20

09 Apr '20

Daniel P. Berrangé (6): docs: point users to gitlab for issue tracking docs: point to gitlab as primary git repo host docs: style git mirror links less prominently docs: update for rename of libvirt-jenkins.ci repository docs: add link to bug tracker against each download docs: replace site name with link type in download table docs/best-practices.rst | 5 +- docs/bugs.html.in | 27 +++--- docs/downloads.html.in | 187 ++++++++++++++++++++++++++++------------ docs/libvirt.css | 9 ++ 4 files changed, 159 insertions(+), 69 deletions(-) -- 2.24.1

2 14

[libvirt PATCH v2] docs: list settings required in creating a new git repo
by Daniel P. Berrangé 09 Apr '20

09 Apr '20

The libvirt project has alot of git repositories, and they must all be configured in the same way, more or less. This page documents the settings changes that I have made in GitLab and GitHub when configuring projects, both as a reminder for myself, and to help anyone else doing the same in future. Also included is info about the repo mirroring on the libvirt.org server. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- In v2: - Apply all Andrea's suggestions docs/docs.html.in | 3 + docs/newreposetup.rst | 137 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 140 insertions(+) create mode 100644 docs/newreposetup.rst diff --git a/docs/docs.html.in b/docs/docs.html.in index 142c79bfa9..6bdf0c32b8 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -172,6 +172,9 @@ <dt><a href="testsuites.html">Functional testing</a></dt> <dd>Testing libvirt with <a href="testtck.html">TCK test suite</a> and <a href="testapi.html">Libvirt-test-API</a></dd> + + <dt><a href="newreposetup.html">New repo setup</a></dt> + <dd>Procedure for configuring new git repositories for libvirt</dd> </dl> </div> diff --git a/docs/newreposetup.rst b/docs/newreposetup.rst new file mode 100644 index 0000000000..7da13830c0 --- /dev/null +++ b/docs/newreposetup.rst @@ -0,0 +1,137 @@ +=============================== +Repository infrastructure setup +=============================== + +GitLab Configuration +==================== + +The `GitLab organization <https://gitlab.com/libvirt>`_ hosts the master copy +of all the libvirt Git repositories. + +When creating a new repository the following changes to the defaults are +required under the **Settings** page: + +* **General** + + * **Naming, topics, avatar** + + * *Project avatar*: upload ``docs/logos/logo-square-256.png`` + + * **Visibility, project features, permissions** + + * *Packages*: disabled + + * *Wiki*: disabled + + * *Snippets*: disabled + + * **Merge Requests** + + * *Merge method*: Fast-forward merge + + * *Merge options*: Enable 'delete source branch' option by default + + * *Merge checks*: Pipelines must succeed + + * **Merge request approvals** + + * *Any eligible user*: Num approvals required == 1 + +* **Integrations** + + * **Pipelines emails** + + * *Recipients*: ``libvirt-ci(a)redhat.com`` + +* **Repository** + + * **Push rules** + + * *Do not allow users to remove git tags with git push*: enabled + + * *Commit message*: ``Signed-off-by:`` + + * *Branch name*: ``^(master|v.*-maint)$`` + + * **Mirroring repositories** + + * *Git repository URL*: ``https://libvirtmirror@github.com/libvirt/$repo.git`` + + * *Mirror direction*: push + + * *Password*: see ``/root/libvirt-mirror-github-api-token.txt`` on ``libvirt.org`` + + * **Protected branches** + + * *Branch*: ``master`` + + * *Allowed to merge*: Developers + Maintainers + + * *Allowed to push*: None (or Developers + Maintainers if MRs not used) + + * *Require approval from code owners*: disabled + +GitHub configuration +==================== + +The `GitHub organization <https://github.com/libvirt>`_ hosts read-only +mirrors of all the libvirt Git repositories. + +When creating a new repository the following changes to the defaults are +required under the **Settings** page: + +* **Options** + + * **Features** + + * *Wikis*: disabled + + * *Sponsorships*: disabled + + * *Projects*: disabled + + * **Manage access** + + * Add the ``@committers`` team with the role "Write", which + grants ``libvirtmirror`` access to sync from gitlab. + + * **Integrations** + + * Check for *Repo Lockdown* (should be set automatically for all projects) + +In the master git repository create a file `.github/lockdown.yml` to restrict +use of issue tracker and pull requests. + + +libvirt.org +=========== + +The `libvirt project server <https://libvirt.org>`_ hosts read-only mirrors of +all the libvirt Git repositories in the directory ``/data/git``. + +When creating a new repository the following steps are required: + +* Create repo with + :: + + $ sudo su - + # cd /data/git + # mkdir $repo.git + # cd $repo.git + # git init --bare + # touch export + # touch git-daemon-export-ok + # cd .. + # chown -R gitmirror.gitmirror $repo.git + # chmod -R g+w $repo.git + # find -type d $repo.git | xargs chmod g+s + +* Set the ``description`` and ``config`` files following other repos' example + +* Setup mirroring + :: + + $ sudo su - gitmirror + # ./newrepo.sh /data/git/$repo.git + # cd mirrors + # $HOME/sync-one.sh $repo.git -- 2.24.1

2 1

[PATCH] virsh: cmdUndefine: Properly extract delete-storage-volume-snapshots flag
by Peter Krempa 09 Apr '20

09 Apr '20

Commit 86608f787ee added the above flag as an alias for ambiguous 'delete-snapshots' flag, but forgot to actually change the code that extracts it, thus the new version actually doesn't work. https://bugzilla.redhat.com/show_bug.cgi?id=1821988 Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- tools/virsh-domain.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 2dc7c38b55..a203b34996 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -3658,7 +3658,7 @@ cmdUndefine(vshControl *ctl, const vshCmd *cmd) bool checkpoints_metadata = vshCommandOptBool(cmd, "checkpoints-metadata"); bool wipe_storage = vshCommandOptBool(cmd, "wipe-storage"); bool remove_all_storage = vshCommandOptBool(cmd, "remove-all-storage"); - bool delete_snapshots = vshCommandOptBool(cmd, "delete-snapshots"); + bool delete_snapshots = vshCommandOptBool(cmd, "delete-storage-volume-snapshots"); bool nvram = vshCommandOptBool(cmd, "nvram"); bool keep_nvram = vshCommandOptBool(cmd, "keep-nvram"); /* Positive if these items exist. */ -- 2.26.0

2 1

[libvirt-dockerfiles PATCH 0/5] Update after libvirt-ci repo rename
by Andrea Bolognani 09 Apr '20

09 Apr '20

All pushed as trivial. Andrea Bolognani (5): README: Update git URLs to point to GitLab README: Update after libvirt-ci repo rename refresh: Update after libvirt-ci repo rename trigger: Update after libvirt-ci repo rename monitor: Update after libvirt-ci repo rename README.md | 5 ++--- monitor | 2 +- refresh | 4 ++-- trigger | 2 +- 4 files changed, 6 insertions(+), 7 deletions(-) -- 2.25.2

1 5