April 2020 - Devel - libvirt List Archives

[virttools-web PATCH 0/3] Switch www.virt-tools.org over to use GitLab Pages
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

This introduces use of GitLab CI + Pages to replace the current OpenShift application which is only admin accessible by myself. It also has automatic integration with LetsEncrypt guaranteeing that we'll never have expired certificates. Daniel P. Berrangé (3): Convert README to markdown format Introduce use of GitLab CI for publishing to GitLab Pages Remove obsolete openshift hosting configuration .gitlab-ci.yml | 9 + README | 19 -- README.md | 15 ++ openshift/templates/.gitignore | 2 - openshift/templates/update-tls.sh | 16 -- openshift/templates/virttools-web-tls.json | 82 ------- openshift/templates/virttools-web.json | 272 --------------------- {web => public}/header-bg.png | Bin {web => public}/index.css | 0 {web => public}/index.html | 4 +- {web => public}/logo-kvm.png | Bin {web => public}/logo-libguestfs.png | Bin {web => public}/logo-libosinfo.png | Bin {web => public}/logo-libvirt.png | Bin {web => public}/logo-qemu.png | Bin {web => public}/logo-virt-manager.png | Bin {web => public}/logo.png | Bin {web => public}/logo.xcf | Bin 18 files changed, 25 insertions(+), 394 deletions(-) create mode 100644 .gitlab-ci.yml delete mode 100644 README create mode 100644 README.md delete mode 100644 openshift/templates/.gitignore delete mode 100755 openshift/templates/update-tls.sh delete mode 100644 openshift/templates/virttools-web-tls.json delete mode 100644 openshift/templates/virttools-web.json rename {web => public}/header-bg.png (100%) rename {web => public}/index.css (100%) rename {web => public}/index.html (93%) rename {web => public}/logo-kvm.png (100%) rename {web => public}/logo-libguestfs.png (100%) rename {web => public}/logo-libosinfo.png (100%) rename {web => public}/logo-libvirt.png (100%) rename {web => public}/logo-qemu.png (100%) rename {web => public}/logo-virt-manager.png (100%) rename {web => public}/logo.png (100%) rename {web => public}/logo.xcf (100%) -- 2.24.1

2 4

[PATCHv2 0/1] Use g_autoptr and g_autofree in cpu.c
by Seeteena Thoufeek 07 Apr '20

07 Apr '20

return NULL for all the failure cases for the function virCPUDataParse(). Previous discussion can be found at https://www.redhat.com/archives/libvir-list/2020-April/msg00191.html Seeteena Thoufeek (1): cpu: Use g_autoptr and g_autofree in cpu.c src/cpu/cpu.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) -- 1.8.3.1

2 2

[libvirt PATCH 00/15] RFC: basic CGroup support with qemu:///session
by marcandre.lureau＠redhat.com 07 Apr '20

07 Apr '20

From: Marc-André Lureau <marcandre.lureau(a)redhat.com> Hi, This is a small series that allows basic QEMU VM CGroup support with the help of machined --user: https://github.com/systemd/systemd/pull/15312 The first few patches are fixes to register dbus and slirp-helper correctly with the VM cgroup. A few changes are done to the machined support, adding session support, and registering the VM to get a systemd scope cgroup under user machine.slice. Marc-André Lureau (15): slirp: leave the dbus daemon running on error slirp: add helper to VM cgroup qemu-dbus: prevent double start of the bus daemon qemu-dbus: remove unused variable qemu-dbus: register DBus bus to the VM cgroup qemu-cgroup: remove unnecessary include dbus: rename virDBusIs* -> virDBusSystemIs* dbus: add virDBusIsService{Enabled,Registered} with @conn argument systemd: check org.freedesktop.machine1 registration systemd: only check for activitable machine1 service systemd: register machines against session service cgroup: return directly if there is nothing to remove qemu-cgroup: register ext devices when cpu/cpuset controller are missing qemu: create cgroup regardless of controller support qemu-cgroup: drop the need for privileges to use cgroup src/qemu/qemu_cgroup.c | 13 ------- src/qemu/qemu_dbus.c | 29 +++++++++++++- src/qemu/qemu_dbus.h | 3 ++ src/qemu/qemu_extdevice.c | 12 ++++++ src/qemu/qemu_process.c | 6 +-- src/qemu/qemu_slirp.c | 16 +++++++- src/qemu/qemu_slirp.h | 3 ++ src/util/vircgroup.c | 5 +++ src/util/vircgroupv2devices.c | 7 ++-- src/util/virdbus.c | 71 ++++++++++++++++++++++++++--------- src/util/virdbus.h | 7 +++- src/util/virfirewalld.c | 2 +- src/util/virsystemd.c | 27 +++++-------- tests/virsystemdtest.c | 35 ----------------- 14 files changed, 140 insertions(+), 96 deletions(-) -- 2.26.0.rc2.42.g98cedd0233

5 23

ANNOUNCE: primary git repos switched to https://gitlab.com/libvirt/
by Daniel P. Berrangé 07 Apr '20

07 Apr '20

Hello All, As of this email, the primary host for libvirt project git repositories has been switched to gitlab.com with all projects under the following group: https://gitlab.com/libvirt The development process remains the same for now, with patches still to be submitted to this mailing list as before. The repositories on libvirt.org will continue to exist, but as an automated read-only mirror of the primary gitlab.com repositories, so may be delayed 30-60 minutes vs the primary repos. To update your current checkouts to point to the new primary repos use the following: git remote set-url origin https://gitlab.com/libvirt/$REPO.git eg git remote set-url origin https://gitlab.com/libvirt/libvirt.git (if you are a person with commit rights you'll need to use the ssh URL instead which I've emailed you about directly) Switching over to require merge requests is a separate task that will be done incrementally for repos over a period of time, with further announcements made on this at a later date. As long as a ".gitpublish" config file exists in a repo, then patches for that repo should continue to be sent to this mailing list. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

1 0

backport new changes to 4.0.0
by Jiatong Shen 07 Apr '20

07 Apr '20

Hello, Sorry in advance if I am asking questions at the wrong place. I would like know if we have plans to backport new features to old releases, for example block copy remote disks. If the answer is no, then where can I find some guidance on building a libvirt debian packages? Thank you very much for the help. -- Best Regards, Jiatong Shen

3 2

[PATCH v3 00/10] Another round of qemu:///embed fixes
by Michal Privoznik 06 Apr '20

06 Apr '20

v3 of: https://www.redhat.com/archives/libvir-list/2020-March/msg01112.html diff to v2: - Patch 2/10 is new; after discussion to v2 we can access driver->privileged and driver->embeddedRoot directly - Patch 3/10 is slightly reworked; it still drops unnecessary layers of nesting in the default case (/var/lib/libvirt/qemu/ram) but it doesn't do that if user provides a path via qemu.conf Michal Prívozník (10): tests: Fix virQEMUDriverConfigNew() calling with respect to @root qemu: Drop virQEMUDriverIsPrivileged() qemu: Drop two layers of nesting of memoryBackingDir conf: Move virDomainGenerateMachineName to hypervisor/ virDomainDriverGenerateMachineName: Factor out embed path hashing qemuDomainGetMachineName: Access embeddedRoot from driver rather than cfg Revert "qemu_conf: Track embed root dir" qemu: Make hugepages path generation embed driver aware qemu: Make memory path generation embed driver aware qemu: Make auto dump path generation embed driver aware src/conf/domain_conf.c | 72 --------------- src/conf/domain_conf.h | 7 -- src/hypervisor/domain_driver.c | 88 +++++++++++++++++++ src/hypervisor/domain_driver.h | 11 +++ src/libvirt_private.syms | 3 +- src/lxc/lxc_domain.c | 3 +- src/qemu/qemu_cgroup.c | 4 +- src/qemu/qemu_command.c | 23 +++-- src/qemu/qemu_conf.c | 76 +++++++++------- src/qemu/qemu_conf.h | 23 +++-- src/qemu/qemu_domain.c | 19 ++-- src/qemu/qemu_driver.c | 42 ++++----- src/qemu/qemu_interface.c | 6 +- src/qemu/qemu_process.c | 7 +- tests/domaincapstest.c | 2 +- .../qemuxml2argvdata/cpu-numa-memshared.args | 8 +- .../fd-memory-no-numa-topology.args | 2 +- .../fd-memory-numa-topology.args | 4 +- .../fd-memory-numa-topology2.args | 8 +- .../fd-memory-numa-topology3.args | 12 +-- .../hugepages-memaccess2.args | 12 +-- .../qemuxml2argvdata/pages-dimm-discard.args | 4 +- ...vhost-user-fs-fd-memory.x86_64-latest.args | 2 +- tests/testutilsqemu.c | 2 +- tests/virsystemdtest.c | 5 +- 25 files changed, 235 insertions(+), 210 deletions(-) -- 2.24.1

2 11

[libvirt-jenkins-ci PATCH 0/5] Introduce the new 'gitlab' flavor
by Erik Skultety 06 Apr '20

06 Apr '20

Erik Skultety (5): guests: templates: Introduce a gitlab-runner systemd service template guests: templates: Introduce a gitlab-runner RC init service template guests: Introduce the new 'gitlab' flavor playbooks: gitlab: Force a random password for the root account guests: lcitool: Enable the new 'gitlab' flavor in the lcitool script guests/group_vars/all/main.yml | 3 + guests/lcitool | 29 ++++++-- guests/playbooks/update/main.yml | 5 ++ guests/playbooks/update/tasks/gitlab.yml | 68 +++++++++++++++++++ .../update/templates/gitlab-runner.j2 | 32 +++++++++ .../update/templates/gitlab-runner.service.j2 | 12 ++++ 6 files changed, 144 insertions(+), 5 deletions(-) create mode 100644 guests/playbooks/update/tasks/gitlab.yml create mode 100644 guests/playbooks/update/templates/gitlab-runner.j2 create mode 100644 guests/playbooks/update/templates/gitlab-runner.service.j2 -- 2.25.1

2 26

[PATCH] libxl: fix crash when initializing driver
by Jim Fehlig 06 Apr '20

06 Apr '20

Commit 54a401af478 split out DriverConfigInit from DriverConfigNew, but then called it a bit late from libxlStateInitialize. The cfg is used in libxlDriverConfigLoadFile and when uninitialized results in a crash. Calling DriverConfigInit immediately after DriverConfigNew fixes the crash. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- src/libxl/libxl_driver.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index 7ec4fcc3d1..980984b199 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -702,14 +702,14 @@ libxlStateInitialize(bool privileged, if (!(cfg = libxlDriverConfigNew())) goto error; + if (libxlDriverConfigInit(cfg) < 0) + goto error; + driverConf = g_strdup_printf("%s/libxl.conf", cfg->configBaseDir); if (libxlDriverConfigLoadFile(cfg, driverConf) < 0) goto error; - if (libxlDriverConfigInit(cfg) < 0) - goto error; - /* Register the callbacks providing access to libvirt's event loop */ libxl_osevent_register_hooks(cfg->ctx, &libxl_osevent_callbacks, cfg->ctx); -- 2.26.0

3 3

[PATCH] formatdomain.html.in: fix 'sockets' info in topology element
by Daniel Henrique Barboza 06 Apr '20

06 Apr '20

In the 'topology' element it is mentioned, regarding the sockets value, "They refer to the total number of CPU sockets". This is not accurate. What we're doing is calculating the number of sockets per NUMA node, which can be checked in the current implementation of virHostCPUGetInfoPopulateLinux(). Calculating the total number of sockets would break the topology sanity check nodes*sockets*cores*threads=online_cpus. This documentation fix is important to avoid user confusion when seeing the output of 'virsh capabilities' and expecting it to be equal to the output of 'lscpu'. E.g in a Power 9 host this 'lscpu' output: Architecture: ppc64le Byte Order: Little Endian CPU(s): 160 On-line CPU(s) list: 0-159 Thread(s) per core: 4 Core(s) per socket: 20 Socket(s): 2 NUMA node(s): 2 Model: 2.2 (pvr 004e 1202) Model name: POWER9, altivec supported And this XML output from virsh capabilities: <cpu> <arch>ppc64le</arch> <model>POWER9</model> <vendor>IBM</vendor> <topology sockets='1' dies='1' cores='20' threads='4'/> (...) </cpu> Both are correct, as long as we mention in the Libvirt documentation that 'sockets' in the topology element represents the number of sockets per NUMA node. Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com> --- docs/formatdomain.html.in | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index aaeb05961f..7940d6ace5 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -1675,9 +1675,9 @@ <dd>The <code>topology</code> element specifies requested topology of virtual CPU provided to the guest. Four attributes, <code>sockets</code>, <code>dies</code>, <code>cores</code>, and <code>threads</code>, - accept non-zero positive integer values. They refer to the total number - of CPU sockets, number of dies per socket, number of cores per die, and - number of threads per core, respectively. The <code>dies</code> + accept non-zero positive integer values. They refer to the number of + CPU sockets per NUMA node, number of dies per socket, number of cores + per die, and number of threads per core, respectively. The <code>dies</code> attribute is optional and will default to 1 if omitted, while the other attributes are all mandatory. Hypervisors may require that the maximum number of vCPUs specified by the <code>cpus</code> element equals to -- 2.25.1

2 1

[PATCH] tools: variables clean-up in libvirt-guests script
by Prathamesh Chavan 06 Apr '20

06 Apr '20

Redeclared variables in script functions marked as local. Variables `guest_running` and `guests_shutting_down` in the functions 'guest_is_on` and `check_guests_shutdown` were untouched, as the functions returned values in these variables. Signed-off-by: Prathamesh Chavan <pc44800(a)gmail.com> --- I'm submitting this as my BiteSizedTask for GSoC'20 I've cc'd Martin Kletzander, as he is mentor for this task, and to ensure that the needful was done by me. After running `make check`, I received "PASS: 98 and SKIP: 7" as the result. Travis-ci build report can be found at [1]. For my project, 'Introducing job control to the storage driver', right now I'm going through the `/src/qemu/THREADS.txt` as well as previous work done by Tucker DiNapoli during GSoC'14, as well as the proposal of Taowei Luo for GSoC'15. Their email threads are surely helping me get a better picture of the project. Apart from this, please let me know if there is anything else, which could help me with this project. Thansk! [1]: https://travis-ci.org/github/pratham-pc/libvirt/builds/668209157 tools/libvirt-guests.sh.in | 104 +++++++++++++++++++------------------ 1 file changed, 54 insertions(+), 50 deletions(-) diff --git a/tools/libvirt-guests.sh.in b/tools/libvirt-guests.sh.in index a881f6266e..90a18fee49 100644 --- a/tools/libvirt-guests.sh.in +++ b/tools/libvirt-guests.sh.in @@ -65,7 +65,7 @@ retval() { # If URI is "default" virsh is called without the "-c" argument # (using libvirt's default connection) run_virsh() { - uri=$1 + local uri=$1 shift if [ "x$uri" = xdefault ]; then @@ -86,7 +86,7 @@ run_virsh_c() { # check if URI is reachable test_connect() { - uri=$1 + local uri=$1 if run_virsh "$uri" connect 2>/dev/null; then return 0; @@ -103,8 +103,8 @@ test_connect() # --transient: list only transient guests # [none]: list both persistent and transient guests list_guests() { - uri=$1 - persistent=$2 + local uri=$1 + local persistent=$2 list=$(run_virsh_c "$uri" list --uuid $persistent) if [ $? -ne 0 ]; then @@ -118,8 +118,8 @@ list_guests() { # guest_name URI UUID # return name of guest UUID on URI guest_name() { - uri=$1 - uuid=$2 + local uri=$1 + local uuid=$2 run_virsh "$uri" domname "$uuid" 2>/dev/null } @@ -128,8 +128,8 @@ guest_name() { # check if guest UUID on URI is running # Result is returned by variable "guest_running" guest_is_on() { - uri=$1 - uuid=$2 + local uri=$1 + local uuid=$2 guest_running=false id=$(run_virsh "$uri" domid "$uuid") @@ -161,13 +161,13 @@ start() { return 0 fi - isfirst=true - bypass= - sync_time=false + local isfirst=true + local bypass= + local sync_time=false test "x$BYPASS_CACHE" = x0 || bypass=--bypass-cache test "x$SYNC_TIME" = x0 || sync_time=true while read uri list; do - configured=false + local configured=false set -f for confuri in $URIS; do set +f @@ -186,7 +186,7 @@ start() { eval_gettext "Resuming guests on \$uri URI..."; echo for guest in $list; do - name=$(guest_name "$uri" "$guest") + local name=$(guest_name "$uri" "$guest") eval_gettext "Resuming guest \$name: " if guest_is_on "$uri" "$guest"; then if "$guest_running"; then @@ -217,17 +217,17 @@ start() { # was saved. suspend_guest() { - uri=$1 - guest=$2 + local uri=$1 + local guest=$2 - name=$(guest_name "$uri" "$guest") - label=$(eval_gettext "Suspending \$name: ") - bypass= - slept=0 + local name=$(guest_name "$uri" "$guest") + local label=$(eval_gettext "Suspending \$name: ") + local bypass= + local slept=0 test "x$BYPASS_CACHE" = x0 || bypass=--bypass-cache printf '%s...\n' "$label" run_virsh "$uri" managedsave $bypass "$guest" >/dev/null & - virsh_pid=$! + local virsh_pid=$! while true; do sleep 1 kill -0 "$virsh_pid" >/dev/null 2>&1 || break @@ -251,15 +251,17 @@ suspend_guest() # was successfully shutdown or the timeout defined by $SHUTDOWN_TIMEOUT expired. shutdown_guest() { - uri=$1 - guest=$2 + local uri=$1 + local guest=$2 - name=$(guest_name "$uri" "$guest") + local name=$(guest_name "$uri" "$guest") eval_gettext "Starting shutdown on guest: \$name" echo retval run_virsh "$uri" shutdown "$guest" >/dev/null || return - timeout=$SHUTDOWN_TIMEOUT - check_timeout=false + local timeout=$SHUTDOWN_TIMEOUT + local check_timeout=false + local format= + local slept= if [ $timeout -gt 0 ]; then check_timeout=true format=$(eval_gettext "Waiting for guest %s to shut down, %d seconds left\n") @@ -300,8 +302,8 @@ shutdown_guest() # was issued to libvirt to allow parallel shutdown. shutdown_guest_async() { - uri=$1 - guest=$2 + local uri=$1 + local guest=$2 name=$(guest_name "$uri" "$guest") eval_gettext "Starting shutdown on guest: \$name" @@ -323,8 +325,8 @@ guest_count() # Result is returned in "guests_shutting_down" check_guests_shutdown() { - uri=$1 - guests_to_check=$2 + local uri=$1 + local guests_to_check=$2 guests_shutting_down= for guest in $guests_to_check; do @@ -344,16 +346,16 @@ check_guests_shutdown() # a shutdown complete notice for guests that have finished print_guests_shutdown() { - uri=$1 - before=$2 - after=$3 + local uri=$1 + local before=$2 + local after=$3 for guest in $before; do case " $after " in *" $guest "*) continue;; esac - name=$(guest_name "$uri" "$guest") + local name=$(guest_name "$uri" "$guest") if [ -n "$name" ]; then eval_gettext "Shutdown of guest \$name complete." echo @@ -365,12 +367,14 @@ print_guests_shutdown() # Shutdown guests GUESTS on machine URI in parallel shutdown_guests_parallel() { - uri=$1 - guests=$2 - - on_shutdown= - check_timeout=false - timeout=$SHUTDOWN_TIMEOUT + local uri=$1 + local guests=$2 + + local on_shutdown= + local check_timeout=false + local timeout=$SHUTDOWN_TIMEOUT + local slept= + local format= if [ $timeout -gt 0 ]; then check_timeout=true format=$(eval_gettext "Waiting for %d guests to shut down, %d seconds left\n") @@ -382,7 +386,7 @@ shutdown_guests_parallel() while [ -n "$guests" ] && [ $(guest_count "$on_shutdown") -lt "$PARALLEL_SHUTDOWN" ]; do set -- $guests - guest=$1 + local guest=$1 shift guests=$* if [ -z "$(echo $on_shutdown | grep $guest)" ] && @@ -394,9 +398,9 @@ shutdown_guests_parallel() sleep 1 set -- $guests - guestcount=$# + local guestcount=$# set -- $on_shutdown - shutdowncount=$# + local shutdowncount=$# if $check_timeout; then if [ $(($timeout % 5)) -eq 0 ]; then @@ -415,7 +419,7 @@ shutdown_guests_parallel() fi fi - on_shutdown_prev=$on_shutdown + local on_shutdown_prev=$on_shutdown check_guests_shutdown "$uri" "$on_shutdown" on_shutdown="$guests_shutting_down" print_guests_shutdown "$uri" "$on_shutdown_prev" "$on_shutdown" @@ -428,8 +432,8 @@ stop() { # last stop was not followed by start [ -f "$LISTFILE" ] && return 0 - suspending=true - if [ "x$ON_SHUTDOWN" = xshutdown ]; then + local suspending=true + if [ "/x$ON_SHUTDOWN" = xshutdown ]; then suspending=false if [ $SHUTDOWN_TIMEOUT -lt 0 ]; then gettext "SHUTDOWN_TIMEOUT must be equal or greater than 0" @@ -448,9 +452,9 @@ stop() { eval_gettext "Running guests on \$uri URI: " - list=$(list_guests "$uri") + local list=$(list_guests "$uri") if [ $? -eq 0 ]; then - empty=true + local empty=true for uuid in $list; do "$empty" || printf ", " printf %s "$(guest_name "$uri" "$uuid")" @@ -464,9 +468,9 @@ stop() { fi if "$suspending"; then - transient=$(list_guests "$uri" "--transient") + local transient=$(list_guests "$uri" "--transient") if [ $? -eq 0 ]; then - empty=true + local empty=true for uuid in $transient; do if "$empty"; then eval_gettext "Not suspending transient guests on URI: \$uri: " @@ -563,7 +567,7 @@ rh_status() { # usage [val] # Display usage string, then exit with VAL (defaults to 2). usage() { - program_name=$0 + local program_name=$0 eval_gettext "Usage: \$program_name {start|stop|status|restart|"\ "condrestart|try-restart|reload|force-reload|gueststatus|shutdown}"; echo exit ${1-2} -- 2.17.1

3 4