- Devel - libvirt List Archives

[libvirt] [PATCH] qemu: Fix regression with undefine --snapshots-metadata
by Eric Blake 10 May '19

10 May '19

In refactoring the snapshot code to prepare for checkpoints, I changed qemuDomainMomentDiscardAll to take a callback that would handle the cleanup of either a snapshot or a checkpoint, but failed to set the callback on one of the two snapshot callers. As a result, 'virsh undefine $dom --snapshots-metadata' crashed on a NULL function dereference. Fixes: a487890d371b8cc3662c1717dfe07eea3f1ef1c0 Signed-off-by: Eric Blake <eblake(a)redhat.com> --- src/qemu/qemu_domain.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 2af17816c6..3290c5d490 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -8664,7 +8664,8 @@ qemuDomainSnapshotDiscardAllMetadata(virQEMUDriverPtr driver, virQEMUMomentRemove rem = { .driver = driver, .vm = vm, - .metadata_only = true + .metadata_only = true, + .momentDiscard = qemuDomainSnapshotDiscard, }; virDomainSnapshotForEach(vm->snapshots, qemuDomainMomentDiscardAll, &rem); -- 2.20.1

2 1

[libvirt] Running libvirtd out of source directory connection reset error
by Peter P. 10 May '19

10 May '19

Hi all, I'm getting started with hacking around with libvirt and am trying to familiarize myself with launching and running an instance of libvirtd I built from source on Centos 7.6. Following the instructions from https://libvirt.org/compiling.html to launch my built versions of libvirtd and virsh, I get the following error with no other context when trying to start a domain using "virsh start mydomain": error: Cannot recv data: Connection reset by peer Despite this error, I am able to run commands list virsh list. Are there additional parameters needed to launch libvirtd or additional services I need to start up alongside it? Thanks, Peter

2 2

[libvirt] [PATCH RFC] iohelper: Introduces a small sleep do avoid hunging other tasks
by Leonardo Bras 10 May '19

10 May '19

While dumping very large VMs (over 128GB), iohelper seems to cause very intense IO usage on the disk, and it causes some processes (like journald) to hung, and depending on kernel configuration, to panic. This change creates a time window, after every 10GB written, so this processes can write to the disk, and avoid hunging. Signed-off-by: Leonardo Bras <leonardo(a)linux.ibm.com> --- src/util/iohelper.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index ddc338b7c7..164c1e2085 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -52,6 +52,8 @@ runIO(const char *path, int fd, int oflags) unsigned long long total = 0; bool direct = O_DIRECT && ((oflags & O_DIRECT) != 0); off_t end = 0; + const unsigned long long sleep_step = (long long)10*1024*1024*1024; + unsigned long long next_sleep = sleep_step; #if HAVE_POSIX_MEMALIGN if (posix_memalign(&base, alignMask + 1, buflen)) { @@ -128,6 +130,12 @@ runIO(const char *path, int fd, int oflags) total += got; + /* sleeps for a while to avoid hunging other tasks */ + if (total > next_sleep) { + next_sleep += sleep_step; + usleep(100*1000); + } + /* handle last write size align in direct case */ if (got < buflen && direct && fdout == fd) { ssize_t aligned_got = (got + alignMask) & ~alignMask; -- 2.20.1

3 3

[libvirt] [PATCH v2] Add support for podman in Makefile.ci
by Martin Kletzander 10 May '19

10 May '19

This way more users can run our CI builds locally. Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- Makefile.ci | 91 ++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 70 insertions(+), 21 deletions(-) diff --git a/Makefile.ci b/Makefile.ci index 12a62167cc67..241c58d2d4e9 100644 --- a/Makefile.ci +++ b/Makefile.ci @@ -17,7 +17,7 @@ CI_GIT_ROOT = $(shell git rev-parse --show-toplevel) CI_HOST_SRCDIR = $(CI_SCRATCHDIR)/src # The directory holding the source inside the -# container. ie where we told Docker to expose +# container, i.e. where we want to expose # the $(CI_HOST_SRCDIR) directory from the host CI_CONT_SRCDIR = /src @@ -46,14 +46,13 @@ CI_CONFIGURE_ARGS = # cloning them CI_SUBMODULES = $(shell git submodule | awk '{ print $$2 }') -# Location of the Docker images we're going to pull +# Location of the container images we're going to pull # Can be useful to overridde to use a locally built # image instead CI_IMAGE_PREFIX = quay.io/libvirt/buildenv- -# Docker defaults to pulling the ':latest' tag but -# if the Docker repo above uses different conventions -# this can override it +# The default tag is ':latest' but if the container +# repo above uses different conventions this can override it CI_IMAGE_TAG = :master # We delete the virtual root after completion, set @@ -71,15 +70,23 @@ CI_REUSE = 0 CI_UID = $(shell id -u) CI_GID = $(shell id -g) -# Docker doesn't require the IDs you run as to exist in +CI_ENGINE = auto +# Container engine we are going to use, can be overridden per make +# invocation, if it is not we try podman and then default to docker. +ifeq ($(CI_ENGINE),auto) + override CI_ENGINE = $(shell podman version >/dev/null 2>&1 && echo podman || echo docker) +endif + +# IDs you run as do not need to exist in # the container's /etc/passwd & /etc/group files, but -# if they do not, then libvirt's 'make check' will fail +# if they do not, then libvirt's 'make check' will fail # many tests. -# + # We do not directly mount /etc/{passwd,group} as Docker # is liable to mess with SELinux labelling which will -# then prevent the host accessing them. Copying them -# first is safer. +# then prevent the host accessing them. And podman cannot +# relabel the files due to it running rootless. So +# copying them first is safer and error-prone. CI_PWDB_MOUNTS = \ --volume $(CI_SCRATCHDIR)/group:/etc/group:ro,z \ --volume $(CI_SCRATCHDIR)/passwd:/etc/passwd:ro,z \ @@ -90,6 +97,46 @@ CI_PWDB_MOUNTS = \ # libvirt very slow at exec'ing programs. CI_ULIMIT_FILES = 1024 +ifeq ($(CI_ENGINE),podman) + # Podman cannot reuse host namespace when running non-root containers. Until + # support for --keep-uid is added we can just create another mapping that will + # do that for us. Beware, that in {uid,git}map=container_id:host_id:range, + # the host_id does actually refer to the uid in the first mapping where 0 + # (root) is mapped to the current user and rest is offset. + + # In order to set up this mapping, we need to keep all the user IDs to prevent + # possible errors as some images might expect UIDs up to 90000 (looking at you + # fedora), so we don't want the overflowuid to be used for them. For mapping + # all the other users properly ther eneeds to be some math done. Don't worry, + # it's just addition and subtraction. + + # 65536 ought to be enough (tm), but for really rare cases the maximums might + # need to be higher, but that only happens when your /etc/sub{u,g}id allow + # users to have more IDs. Unless --keep-uid is supported, let's do this in a + # way that should work for everyone. + CI_MAX_UID = $(shell sed -n "s/^$USER:[^:]\+://p" /etc/subuid) + CI_MAX_GID = $(shell sed -n "s/^$USER:[^:]\+://p" /etc/subgid) + ifeq ($(CI_MAX_UID),) + CI_MAX_UID = 65536 + endif + ifeq ($(CI_MAX_GID),) + CI_MAX_GID = 65536 + endif + CI_UID_OTHER = $(shell echo $$(($(CI_UID)+1))) + CI_GID_OTHER = $(shell echo $$(($(CI_GID)+1))) + CI_UID_OTHER_RANGE = $(shell echo $$(($(CI_MAX_UID)-$(CI_UID)))) + CI_GID_OTHER_RANGE = $(shell echo $$(($(CI_MAX_GID)-$(CI_GID)))) + + CI_PODMAN_ARGS = \ + --uidmap 0:1:$(CI_UID) \ + --uidmap $(CI_UID):0:1 \ + --uidmap $(CI_UID_OTHER):$(CI_UID_OTHER):$(CI_UID_OTHER_RANGE) \ + --gidmap 0:1:$(CI_GID) \ + --gidmap $(CI_GID):0:1 \ + --gidmap $(CI_GID_OTHER):$(CI_GID_OTHER):$(CI_GID_OTHER_RANGE) \ + $(NULL) +endif + # Args to use when cloning a git repo. # -c stop it complaining about checking out a random hash # -q stop it displaying progress info for local clone @@ -100,7 +147,7 @@ CI_GIT_ARGS = \ --local \ $(NULL) -# Args to use when running the Docker env +# Args to use when running the container # --rm stop inactive containers getting left behind # --user we execute as the same user & group account # as dev so that file ownership matches host @@ -110,22 +157,23 @@ CI_GIT_ARGS = \ # --ulimit lower files limit for performance reasons # --interactive # --tty Ensure we have ability to Ctrl-C the build -CI_DOCKER_ARGS = \ +CI_ENGINE_ARGS = \ --rm \ --user $(CI_UID):$(CI_GID) \ --interactive \ --tty \ + $(CI_PODMAN_ARGS) \ $(CI_PWDB_MOUNTS) \ --volume $(CI_HOST_SRCDIR):$(CI_CONT_SRCDIR):z \ --workdir $(CI_CONT_SRCDIR) \ --ulimit nofile=$(CI_ULIMIT_FILES):$(CI_ULIMIT_FILES) \ $(NULL) -ci-check-docker: - @echo -n "Checking if Docker is available and running..." && \ - docker version 1>/dev/null && echo "yes" +ci-check-engine: + @echo -n "Checking if $(CI_ENGINE) is available..." && \ + $(CI_ENGINE) version 1>/dev/null && echo "yes" -ci-prepare-tree: ci-check-docker +ci-prepare-tree: ci-check-engine @test "$(CI_REUSE)" != "1" && rm -rf $(CI_SCRATCHDIR) || : @if ! test -d $(CI_SCRATCHDIR) ; then \ mkdir -p $(CI_SCRATCHDIR); \ @@ -150,7 +198,7 @@ ci-prepare-tree: ci-check-docker # gl_public_submodule_commit= to disable gnulib's submodule check # which breaks due to way we clone the submodules ci-build@%: ci-prepare-tree - docker run $(CI_DOCKER_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) \ + $(CI_ENGINE) run $(CI_ENGINE_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) \ /bin/bash -c '\ mkdir -p $(CI_CONT_BUILDDIR) || exit 1 ; \ cd $(CI_CONT_BUILDDIR) ; \ @@ -179,11 +227,11 @@ ci-check@%: $(MAKE) -f $(CI_MAKEFILE) ci-build@$* CI_MAKE_ARGS="check" ci-shell@%: ci-prepare-tree - docker run $(CI_DOCKER_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) /bin/bash + $(CI_ENGINE) run $(CI_ENGINE_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) /bin/bash @test "$(CI_CLEAN)" = "1" && rm -rf $(CI_SCRATCHDIR) || : ci-help: - @echo "Build libvirt inside Docker containers used for CI" + @echo "Build libvirt inside containers used for CI" @echo @echo "Available targets:" @echo @@ -215,6 +263,7 @@ ci-help: @echo @echo "Available make variables:" @echo - @echo " CI_CLEAN=0 - do not delete '$(CI_SCRATCHDIR)' after completion" - @echo " CI_REUSE=1 - re-use existing '$(CI_SCRATCHDIR)' content" + @echo " CI_CLEAN=0 - do not delete '$(CI_SCRATCHDIR)' after completion" + @echo " CI_REUSE=1 - re-use existing '$(CI_SCRATCHDIR)' content" + @echo " CI_ENGINE=auto - container engine to use (podman, docker)" @echo -- 2.21.0

2 1

[libvirt] [jenkins-ci PATCH 0/6] Add support for gtk-vnc builds
by Daniel P. Berrangé 10 May '19

10 May '19

Support gtk-vnc and make virt-viewer depend on it Daniel P. Berrangé (6): mappings: add libgcrypt native & mingw packages mappings: add pulseaudio libs for native packages projects: add gtk-vnc project projects: make virt-viewer depend on gtk-vnc jobs mappings: remove gtk-vnc2 native and mingw packages jobs: allow test-suite.log to exist anywhere guests/host_vars/libvirt-centos-7/main.yml | 1 + guests/host_vars/libvirt-debian-9/main.yml | 1 + guests/host_vars/libvirt-debian-sid/main.yml | 1 + guests/host_vars/libvirt-fedora-29/main.yml | 1 + guests/host_vars/libvirt-fedora-30/main.yml | 1 + .../host_vars/libvirt-fedora-rawhide/main.yml | 3 +++ guests/host_vars/libvirt-freebsd-11/main.yml | 1 + guests/host_vars/libvirt-freebsd-12/main.yml | 1 + .../libvirt-freebsd-current/main.yml | 1 + guests/host_vars/libvirt-ubuntu-18/main.yml | 1 + .../build/jobs/autotools-check-job.yml | 2 +- guests/playbooks/build/jobs/defaults.yml | 3 +++ .../build/projects/gtk-vnc+mingw32.yml | 12 +++++++++ .../build/projects/gtk-vnc+mingw64.yml | 12 +++++++++ guests/playbooks/build/projects/gtk-vnc.yml | 12 +++++++++ guests/vars/mappings.yml | 27 ++++++++++--------- guests/vars/projects/gtk-vnc+mingw32.yml | 6 +++++ guests/vars/projects/gtk-vnc+mingw64.yml | 6 +++++ guests/vars/projects/gtk-vnc.yml | 11 ++++++++ guests/vars/projects/virt-viewer+mingw32.yml | 1 - guests/vars/projects/virt-viewer+mingw64.yml | 1 - guests/vars/projects/virt-viewer.yml | 1 - jenkins/jobs/autotools.yaml | 2 +- jenkins/jobs/defaults.yaml | 3 +++ jenkins/projects/gtk-vnc+mingw32.yaml | 11 ++++++++ jenkins/projects/gtk-vnc+mingw64.yaml | 11 ++++++++ jenkins/projects/gtk-vnc.yaml | 15 +++++++++++ jenkins/projects/virt-viewer+mingw32.yaml | 4 ++- jenkins/projects/virt-viewer+mingw64.yaml | 4 ++- jenkins/projects/virt-viewer.yaml | 4 ++- 30 files changed, 140 insertions(+), 20 deletions(-) create mode 100644 guests/playbooks/build/projects/gtk-vnc+mingw32.yml create mode 100644 guests/playbooks/build/projects/gtk-vnc+mingw64.yml create mode 100644 guests/playbooks/build/projects/gtk-vnc.yml create mode 100644 guests/vars/projects/gtk-vnc+mingw32.yml create mode 100644 guests/vars/projects/gtk-vnc+mingw64.yml create mode 100644 guests/vars/projects/gtk-vnc.yml create mode 100644 jenkins/projects/gtk-vnc+mingw32.yaml create mode 100644 jenkins/projects/gtk-vnc+mingw64.yaml create mode 100644 jenkins/projects/gtk-vnc.yaml -- 2.21.0

2 16

[libvirt] [PATCHv2] build: restore support for libyajl 2.0.1
by Ján Tomko 10 May '19

10 May '19

Commit 105756660f944e7db02de3b55b98bb7c11cd03bf was too eager and did not consider SLE 12 which still has 2.0.1 that does not ship a pkg-config file. Similar to how we check for readline, prefer pkg-config if available and fall back to the old detection code if not found. NB: this is not a clean revert because we're not reintroducing support for YAJL 1. Signed-off-by: Ján Tomko <jtomko(a)redhat.com> Reported-by: Olaf Hering <olaf(a)aepfle.de> --- m4/virt-yajl.m4 | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/m4/virt-yajl.m4 b/m4/virt-yajl.m4 index 494e722963..3caf0c5d07 100644 --- a/m4/virt-yajl.m4 +++ b/m4/virt-yajl.m4 @@ -24,7 +24,19 @@ AC_DEFUN([LIBVIRT_ARG_YAJL],[ AC_DEFUN([LIBVIRT_CHECK_YAJL],[ dnl YAJL JSON library http://lloyd.github.com/yajl/ - LIBVIRT_CHECK_PKG([YAJL], [yajl], [2.0.3]) + PKG_CHECK_EXISTS([readline], [use_pkgconfig=1], [use_pkgconfig=0]) + + if test $use_pkgconfig = 1; then + dnl 2.0.3 was the version where the pkg-config file was first added + LIBVIRT_CHECK_PKG([YAJL], [yajl], [2.0.3]) + else + dnl SUSE SLE 12 and OpenSUSE Leap 42.3 still use 2.0.1 + dnl TODO: delete this in July 2020 + LIBVIRT_CHECK_LIB([YAJL], [yajl], + [yajl_tree_parse], [yajl/yajl_common.h]) + + fi + ]) AC_DEFUN([LIBVIRT_RESULT_YAJL],[ -- 2.19.2

3 3

[libvirt] [PATCH 0/2] Support more container engines in Makefile.ci
by Martin Kletzander 10 May '19

10 May '19

Maybe someone can add support for libvirt-lxc later on =) Martin Kletzander (2): Don't include Makefile.ci in Makefile.am Add support for podman in Makefile.ci Makefile.am | 4 +- Makefile.ci | 125 ++++++++++++++++++++++++++++++++++++++-------------- 2 files changed, 96 insertions(+), 33 deletions(-) -- 2.21.0

3 9

[libvirt] [PATCH 0/3] virSysinfo: Try dmidecode on aarch64
by Michal Privoznik 10 May '19

10 May '19

Turns out some aarch64 machines provide SMBIOS and thus we can use dmidecode to get the sysinfo. Currently, we're parsing /proc/cpuinfo which is very sparse on ARMs. Yes, ideally everything would be exposed somewhere in sysfs, and libvirt would ditch dmidecode in favor of sysfs, but how realistic is that? Michal Prívozník (3): virSysinfoRead: Simplify #ifdef underbush virSysinfoParseX86BaseBoard: Free memory upfront if no board detected virSysinfoReadARM: Try reading DMI table src/util/virsysinfo.c | 43 +++++++++++++++++++++++++++++++------------ 1 file changed, 31 insertions(+), 12 deletions(-) -- 2.21.0

3 8

[libvirt] [PATCH] test_driver: implement virDomainGetDiskErrors
by Ilias Stamatis 09 May '19

09 May '19

Return the number of disks present in the configuration of the fake driver when called with @errors as NULL and @maxerrors as 0. Otherwise return 0 as the number of errors encountered. Signed-off-by: Ilias Stamatis <stamatis.iliass(a)gmail.com> --- src/test/test_driver.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/src/test/test_driver.c b/src/test/test_driver.c index 460c896ef6..5fa9ab30f1 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -3046,6 +3046,32 @@ static int testDomainSetAutostart(virDomainPtr domain, return 0; } +static int testDomainGetDiskErrors(virDomainPtr dom, + virDomainDiskErrorPtr errors, + unsigned int maxerrors ATTRIBUTE_UNUSED, + unsigned int flags) +{ + int ret = -1; + virDomainObjPtr vm = NULL; + + virCheckFlags(0, -1); + + if (!(vm = testDomObjFromDomain(dom))) + goto cleanup; + + if (virDomainObjCheckActive(vm) < 0) + goto cleanup; + + if (!errors) + ret = vm->def->ndisks; + else + ret = 0; + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + static char *testDomainGetSchedulerType(virDomainPtr domain ATTRIBUTE_UNUSED, int *nparams) { @@ -6833,6 +6859,7 @@ static virHypervisorDriver testHypervisorDriver = { .domainUndefineFlags = testDomainUndefineFlags, /* 0.9.4 */ .domainGetAutostart = testDomainGetAutostart, /* 0.3.2 */ .domainSetAutostart = testDomainSetAutostart, /* 0.3.2 */ + .domainGetDiskErrors = testDomainGetDiskErrors, /* 5.4.0 */ .domainGetSchedulerType = testDomainGetSchedulerType, /* 0.3.2 */ .domainGetSchedulerParameters = testDomainGetSchedulerParameters, /* 0.3.2 */ .domainGetSchedulerParametersFlags = testDomainGetSchedulerParametersFlags, /* 0.9.2 */ -- 2.21.0

2 3

[libvirt] [PATCH] build: restore support for libyajl 2.0.1
by Ján Tomko 09 May '19

09 May '19

Commit 105756660f944e7db02de3b55b98bb7c11cd03bf was too eager and did not consider SLE 12 which still has 2.0.1 that does not ship a pkg-config file. Similar to how we check for readline, prefer pkg-config if available and fall back to the old detection code if not found. NB: this is not a clean revert because we're not reintroducing support for YAJL 1. Signed-off-by: Ján Tomko <jtomko(a)redhat.com> Reported-by: Olaf Hering <olaf(a)aepfle.de> --- m4/virt-yajl.m4 | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/m4/virt-yajl.m4 b/m4/virt-yajl.m4 index 494e722963..05a8f8b9bd 100644 --- a/m4/virt-yajl.m4 +++ b/m4/virt-yajl.m4 @@ -24,7 +24,19 @@ AC_DEFUN([LIBVIRT_ARG_YAJL],[ AC_DEFUN([LIBVIRT_CHECK_YAJL],[ dnl YAJL JSON library http://lloyd.github.com/yajl/ - LIBVIRT_CHECK_PKG([YAJL], [yajl], [2.0.3]) + PKG_CHECK_EXISTS([readline], [use_pkgconfig=1], [use_pkgconfig=0]) + + if test $use_pkgconfig = 1; then + dnl 2.0.3 was the version where the pkg-config file was first added + LIBVIRT_CHECK_PKG([YAJL], [yajl], [2.0.3]) + else + dnl SUSE SLE 12 and OpenSUSE Leap 42.3 still use 2.0.1 + dnl TODO: delete this in July 2020 + LIBVIRT_CHECK_LIB_ALT([YAJL], [yajl], + [yajl_tree_parse], [yajl/yajl_common.h]) + + fi + ]) AC_DEFUN([LIBVIRT_RESULT_YAJL],[ -- 2.19.2

4 3