July 2019 - Devel - libvirt List Archives

[libvirt] [PATCH] virsh emulatorpin, vcpupin: omit offline CPUs from affinity map
by Scott Cheloha 11 Jul '19

11 Jul '19

Each host CPU is not necessarily online. Including CPUs that are known to be offline in the default affinity map doesn't make much sense. We can try to omit those CPUs if the host supports CPU bitmaps, i.e. virHostCPUHasBitmap() is true. Otherwise we can return a full map as we do now. For example, given the following lscpu(1): Architecture: ppc64le Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120 Off-line CPU(s) list: 1-7,9-15,17-23,25-31,33-39,41-47,49-55,57-63,65-71,73-79,81-87,89-95,97-103,105-111,113-119,121-127 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 4 NUMA node(s): 4 Model: 2.1 (pvr 004b 0201) Model name: POWER8E (raw), altivec supported CPU max MHz: 4322.0000 CPU min MHz: 2061.0000 L1d cache: 64K L1i cache: 32K L2 cache: 512K L3 cache: 8192K NUMA node0 CPU(s): 0,8,16,24 NUMA node1 CPU(s): 32,40,48,56 NUMA node16 CPU(s): 64,72,80,88 NUMA node17 CPU(s): 96,104,112,120 the current behavior for a guest with no VCPU configuration is: $ virsh vcpupin myvm ---------------------------------- 0: 0-127 but this patch instead you get: VCPU CPU Affinity ---------------------------------- 0 0,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120 which is more consistent with the lscpu(1) output. Fixes: ibm bz174632 (rhbz#1434276) Signed-off-by: Scott Cheloha <cheloha(a)linux.vnet.ibm.com> --- I'm unsure whether it's better to automatically fall back to the full map if virHostCPUGetOnlineBitmap() fails, or to fail loudly as I do in this patch. Preferences? src/conf/domain_conf.c | 8 ++++++++ src/qemu/qemu_driver.c | 4 ++++ 2 files changed, 12 insertions(+) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 3323c9a5b1..0ea6f69574 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -1989,6 +1989,7 @@ virDomainDefGetVcpuPinInfoHelper(virDomainDefPtr def, int maxvcpus = virDomainDefGetVcpusMax(def); size_t i; VIR_AUTOPTR(virBitmap) allcpumap = NULL; + VIR_AUTOPTR(virBitmap) onlinemap = NULL; if (hostcpus < 0) return -1; @@ -1998,6 +1999,11 @@ virDomainDefGetVcpuPinInfoHelper(virDomainDefPtr def, virBitmapSetAll(allcpumap); + if (virHostCPUHasBitmap()) { + if (!(onlinemap = virHostCPUGetOnlineBitmap())) + return -1; + } + for (i = 0; i < maxvcpus && i < ncpumaps; i++) { virDomainVcpuDefPtr vcpu = virDomainDefGetVcpu(def, i); virBitmapPtr bitmap = NULL; @@ -2009,6 +2015,8 @@ virDomainDefGetVcpuPinInfoHelper(virDomainDefPtr def, bitmap = autoCpuset; else if (def->cpumask) bitmap = def->cpumask; + else if (onlinemap) + bitmap = onlinemap; else bitmap = allcpumap; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 5a75f23981..2c59513929 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5377,6 +5377,10 @@ qemuDomainGetEmulatorPinInfo(virDomainPtr dom, } else if (vm->def->placement_mode == VIR_DOMAIN_CPU_PLACEMENT_MODE_AUTO && autoCpuset) { cpumask = autoCpuset; + } else if (virHostCPUHasBitmap()) { + if (!(bitmap = virHostCPUGetOnlineBitmap())) + goto cleanup; + cpumask = bitmap; } else { if (!(bitmap = virBitmapNew(hostcpus))) goto cleanup; -- 2.20.1

1 0

[libvirt] [jenkins-ci PATCH 0/3] Enable building of the VZ driver on CentOS 7
by Daniel P. Berrangé 11 Jul '19

11 Jul '19

The VZ driver in libvirt periodically gets broken by refactoring in libvirt. This is not noticed either before or after merge because none of our CI tests nor common developer build hosts includ the deps needed for the VZ driver. The OpenVZ project, however, does provide builds of the required packages for RHEL-7. We can use these packages in our CentOS 7 CI VMs to enable build testing of the VZ driver. This closes the only hole we have in driver build coverage for CI. Daniel P. Berrangé (3): guests: add openvz repository on CentOS 7 guests: define mapping for the libprlsdk package guests: add libprlsdk package to libvirt project guests/playbooks/update/tasks/base.yml | 10 ++++++++++ guests/playbooks/update/templates/openvz.repo.j2 | 9 +++++++++ guests/vars/mappings.yml | 4 ++++ guests/vars/projects/libvirt.yml | 1 + 4 files changed, 24 insertions(+) create mode 100644 guests/playbooks/update/templates/openvz.repo.j2 -- 2.21.0

2 6

[libvirt] [jenkins-ci PATCH] guests: Make Java symlink creation conditional
by Andrea Bolognani 11 Jul '19

11 Jul '19

We only install Java for the "jenkins" flavor, which means we need to avoid creating the Java symlink for other flavors because the operation can't succeed there. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- Pushed as trivial. guests/playbooks/update/tasks/kludges.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/guests/playbooks/update/tasks/kludges.yml b/guests/playbooks/update/tasks/kludges.yml index 22d8ce5..8af36ab 100644 --- a/guests/playbooks/update/tasks/kludges.yml +++ b/guests/playbooks/update/tasks/kludges.yml @@ -12,6 +12,7 @@ group: wheel when: - os_name == 'FreeBSD' + - flavor == "jenkins" # FreeBSD compiles bash without defining SSH_SOURCE_BASHRC, which means # it won't try to detect when it's spawned by ssh and source ~/.bashrc -- 2.21.0

1 0

[libvirt] [jenkins-ci PATCH] guests: Create java symlink on FreeBSD
by Andrea Bolognani 11 Jul '19

11 Jul '19

This kludge is necessary because the OpenJDK 11 packages, unlike the OpenJDK 8 packages, install all binaries outside of $PATH. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- Pushed so that I could bring the CI environment back online; if people have objections to the approach, we can make the necessary changes as a follow-up. guests/playbooks/update/main.yml | 2 +- guests/playbooks/update/tasks/kludges.yml | 14 ++++++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/guests/playbooks/update/main.yml b/guests/playbooks/update/main.yml index 753bac4..e82055b 100644 --- a/guests/playbooks/update/main.yml +++ b/guests/playbooks/update/main.yml @@ -48,10 +48,10 @@ - flavor == "jenkins" # Configure environment. Needs to happen after installing packages + - include: '{{ playbook_base }}/tasks/kludges.yml' - include: '{{ playbook_base }}/tasks/paths.yml' - include: '{{ playbook_base }}/tasks/bootloader.yml' - include: '{{ playbook_base }}/tasks/services.yml' - - include: '{{ playbook_base }}/tasks/kludges.yml' - include: '{{ playbook_base }}/tasks/users.yml' # Configure the Jenkins agent diff --git a/guests/playbooks/update/tasks/kludges.yml b/guests/playbooks/update/tasks/kludges.yml index 3648514..22d8ce5 100644 --- a/guests/playbooks/update/tasks/kludges.yml +++ b/guests/playbooks/update/tasks/kludges.yml @@ -1,4 +1,18 @@ --- +# FreeBSD allows multiple versions of OpenJDK to be installed and used +# in parallel, which is neat but also means they must all live outside +# of $PATH. Create a symlink so that we can still find the main Java +# executable. +- name: Create java symlink + file: + src: /usr/local/openjdk11/bin/java + dest: /usr/local/bin/java + state: link + owner: root + group: wheel + when: + - os_name == 'FreeBSD' + # FreeBSD compiles bash without defining SSH_SOURCE_BASHRC, which means # it won't try to detect when it's spawned by ssh and source ~/.bashrc # when that's the case. Our workaround is setting $BASH_ENV globally -- 2.21.0

1 0

[libvirt] [PATCH] test_driver: consider flags in testDomainSetMemoryFlags
by Ilias Stamatis 11 Jul '19

11 Jul '19

Update the current or max memory, on the persistent or live definition depending on the flags which are currently ignored. Signed-off-by: Ilias Stamatis <stamatis.iliass(a)gmail.com> --- src/test/test_driver.c | 51 +++++++++++++++++++++++++++++++++++------- 1 file changed, 43 insertions(+), 8 deletions(-) diff --git a/src/test/test_driver.c b/src/test/test_driver.c index c10344f6cd..90910060ed 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -2473,24 +2473,59 @@ static int testDomainSetMemoryFlags(virDomainPtr domain, unsigned long memory, unsigned int flags) { - virDomainObjPtr privdom; + virDomainObjPtr vm; + virDomainDefPtr def; int ret = -1; - virCheckFlags(0, -1); + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG | + VIR_DOMAIN_MEM_MAXIMUM, -1); - if (!(privdom = testDomObjFromDomain(domain))) + if (!(vm = testDomObjFromDomain(domain))) return -1; - if (memory > virDomainDefGetMemoryTotal(privdom->def)) { - virReportError(VIR_ERR_INVALID_ARG, __FUNCTION__); + if (!(def = virDomainObjGetOneDef(vm, flags))) goto cleanup; + + if (flags & VIR_DOMAIN_MEM_MAXIMUM) { + if (virDomainObjCheckActive(vm)) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("cannot resize the maximum memory on an " + "active domain")); + goto cleanup; + } + + if (virDomainNumaGetNodeCount(def->numa) > 0) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("initial memory size of a domain with NUMA " + "nodes cannot be modified with this API")); + goto cleanup; + } + + if (def->mem.max_memory && def->mem.max_memory < memory) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("cannot set initial memory size greater than " + "the maximum memory size")); + goto cleanup; + } + + virDomainDefSetMemoryTotal(def, memory); + + if (def->mem.cur_balloon > memory) + def->mem.cur_balloon = memory; + } else { + if (memory > virDomainDefGetMemoryTotal(def)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("cannot set memory higher than max memory")); + goto cleanup; + } + + def->mem.cur_balloon = memory; } - privdom->def->mem.cur_balloon = memory; ret = 0; - cleanup: - virDomainObjEndAPI(&privdom); + virDomainObjEndAPI(&vm); return ret; } -- 2.22.0

2 3

[libvirt] [PATCH 00/10] qemu: remove json fields from domain and monitor objects
by Ján Tomko 11 Jul '19

11 Jul '19

Applies on top of Peter's removal of commandline parsing https://www.redhat.com/archives/libvir-list/2019-June/msg00405.html After removal of qemuProcessAttach there is no codepath that could possibly set monJSON to false (not that libvirt would be able to much with such domain), so we can finish cleaning it up. Ján Tomko (10): qemu: also delete qemuProcessAttach qemu: assume monJSON is always true qemu: domain: remove monJSON field qemu: stop formatting json='1' in status XML qemu: remove json argument from qemuMonitorOpen qemu: monitor: assume JSON in QEMU_CHECK_MONITOR macro qemu: monitor: remove mon->json checks qemu: monitor: remove the json field qemu: monitor: use VIR_AUTOFREE in qemuMonitor*VideoSize qemu: monitor: s/ret/rc/ in UpdateVideoSize functions src/qemu/qemu_command.c | 8 +- src/qemu/qemu_domain.c | 12 +- src/qemu/qemu_domain.h | 1 - src/qemu/qemu_driver.c | 3 +- src/qemu/qemu_migration.c | 21 +- src/qemu/qemu_monitor.c | 109 ++------ src/qemu/qemu_monitor.h | 2 - src/qemu/qemu_process.c | 247 +----------------- src/qemu/qemu_process.h | 8 - tests/qemuhotplugtest.c | 2 - tests/qemumonitortestutils.c | 1 - .../blockjob-mirror-in.xml | 2 +- .../disk-secinfo-upgrade-out.xml | 2 +- .../migration-in-params-in.xml | 2 +- .../migration-out-nbd-out.xml | 2 +- .../migration-out-nbd-tls-out.xml | 2 +- .../migration-out-params-in.xml | 2 +- tests/qemustatusxml2xmldata/modern-in.xml | 2 +- .../qemustatusxml2xmldata/vcpus-multi-in.xml | 2 +- 19 files changed, 52 insertions(+), 378 deletions(-) -- 2.20.1

2 21

[libvirt] [PATCH 00/23] Enable proper use of systemd socket activation with libvirtd
by Daniel P. Berrangé 11 Jul '19

11 Jul '19

The libvirtd daemon has some support for systemd socket activation from: commit 27a7081c2968ca0d7fbd590629b5a5303851f4a3 Author: Martin Kletzander <mkletzan(a)redhat.com> Date: Tue Jul 15 15:28:53 2014 +0200 daemon: support passing FDs from the calling process First FD is the RW unix socket to listen on, second one (if applicable) is the RO unix socket. This was originally intended for use by the libvirt client when doing auto-spawning of libvirtd, but we later deleted that client side code in commit be78814ae07f092d9c4e71fd82dd1947aba2f029 Author: Michal Privoznik <mprivozn(a)redhat.com> Date: Thu Apr 2 14:41:17 2015 +0200 virNetSocketNewConnectUNIX: Use flocks when spawning a daemon We never added systemd socket units before as we need libvirtd to start on boot to perform autostart. It was recently pointed out by Lennart that these two features are not mutually exclusive though. Libvirtd can be set to start on boot, and also have socket unit files. The idea is that we start libvirtd on boot, perform autostart, and then libvirtd can exit if nothing is running. The socket unit files are then there to start it again when a mgmt app connects. This series implements that strategy. In doing so the current socket activation support was rewritten to be more flexible, able to cope with the admin socket and the TCP/TLS sockets, all passed in any order. NB, I don't believe I have got the RPM upgrade procedure right yet. As there are alot of scenario to test for upgrades, I need more validation of that. The series is long enough now though, that it would benefit from code review already This socket activation is also going to be important when we split out the daemons, as we will use the same libvirtd codebase for these new daemons, simply compiled with different options. Daniel P. Berrangé (23): locking,logging: put a strong dep from admin socket to main socket util: add helper API for getting UNIX path from socket address rpc: add helper API for getting UNIX path from socket object util: add VIR_AUTOSTRUCT for directly calling a struct free function util: add API for resolving socket service names util: add APIs for facilitating use of systemd activation FDs rpc: ensure all sockets bind to same port when service is NULL rpc: refactor RPC service constructors to share more code rpc: allow creating RPC service from an array of FDs rpc: avoid unlinking sockets passed in from systemd rpc: add helper APIs for adding services with systemd activation rpc: add API for checking whether an auth scheme is in use on a server remote: simplify libvirtd code for deciding if SASL auth is needed remote: fix handling of systemd activation wrt socket ordering rpc: remove unused API for creating services from FDs remote: add systemd socket units for UNIX/TCP sockets remote: make system libvirtd exit when idle via timeout remote: update config files to note usage wrt systemd socket activation util: remove code spawning with systemd activation env vars locking: convert lock daemon to use systemd activation APIs logging: convert log daemon to use systemd activation APIs util: move code for getting listen FDs into systemd module util: remove unused helper for getting UNIX socket path libvirt.spec.in | 24 +- src/libvirt_private.syms | 10 +- src/libvirt_remote.syms | 7 +- src/locking/lock_daemon.c | 121 +++---- src/locking/virtlockd-admin.socket.in | 2 + src/logging/log_daemon.c | 121 +++---- src/logging/virtlogd-admin.socket.in | 2 + src/remote/Makefile.inc.am | 35 +++ src/remote/libvirtd-admin.socket.in | 15 + src/remote/libvirtd-ro.socket.in | 15 + src/remote/libvirtd-tcp.socket.in | 14 + src/remote/libvirtd-tls.socket.in | 14 + src/remote/libvirtd.conf | 31 ++ src/remote/libvirtd.service.in | 16 +- src/remote/libvirtd.socket.in | 13 + src/remote/libvirtd.sysconf | 3 +- src/remote/remote_daemon.c | 255 +++++++-------- src/rpc/virnetserver.c | 162 ++++++++++ src/rpc/virnetserver.h | 26 ++ src/rpc/virnetserverservice.c | 238 ++++++-------- src/rpc/virnetserverservice.h | 24 +- src/rpc/virnetsocket.c | 93 ++++-- src/rpc/virnetsocket.h | 2 + src/util/viralloc.h | 13 + src/util/vircommand.c | 99 ------ src/util/vircommand.h | 2 - src/util/virsocketaddr.c | 93 ++++++ src/util/virsocketaddr.h | 4 + src/util/virsystemd.c | 434 ++++++++++++++++++++++++++ src/util/virsystemd.h | 30 ++ src/util/virutil.c | 116 ------- src/util/virutil.h | 3 - tests/commanddata/test24.log | 8 - tests/commandtest.c | 58 ---- tests/virsystemdtest.c | 169 ++++++++++ 35 files changed, 1490 insertions(+), 782 deletions(-) create mode 100644 src/remote/libvirtd-admin.socket.in create mode 100644 src/remote/libvirtd-ro.socket.in create mode 100644 src/remote/libvirtd-tcp.socket.in create mode 100644 src/remote/libvirtd-tls.socket.in create mode 100644 src/remote/libvirtd.socket.in delete mode 100644 tests/commanddata/test24.log -- 2.21.0

3 35

[libvirt] [PATCH 0/5] Implement qemu-ga 'guest-get-users'
by Jonathon Jongsma 11 Jul '19

11 Jul '19

This set of patches adds new API and an implementation for getting the active users for a domain via the guest agent. There is only an implementation for the qemu driver. I've implemented the remote protocol and added support in virsh (new command 'guestusers'). A lot of the implementation was modelled on the implementation of guest-get-fsinfo, which also returns an array of info structures. Since this is my first time adding API or dealing with libvirt RPC stuff, I anticipate several rounds of the series. This series addresses https://bugzilla.redhat.com/show_bug.cgi?id=1704761 Jonathon Jongsma (5): lib: add API to query info about logged-in users remote: implement remote protocol for virDomainGetGuestUsers virsh: add command 'guestusers' implementing VirDomainGetGuestUsers qemu_agent: add helper for getting guest users qemu: implement virDomainGetGuestUsers include/libvirt/libvirt-domain.h | 18 ++++ src/driver-hypervisor.h | 6 ++ src/libvirt-domain.c | 62 ++++++++++++++ src/libvirt_public.syms | 5 ++ src/qemu/qemu_agent.c | 92 ++++++++++++++++++++ src/qemu/qemu_agent.h | 2 + src/qemu/qemu_driver.c | 38 +++++++++ src/remote/remote_daemon_dispatch.c | 89 ++++++++++++++++++++ src/remote/remote_driver.c | 82 +++++++++++++++++- src/remote/remote_protocol.x | 26 +++++- src/remote_protocol-structs | 17 ++++ tests/qemuagenttest.c | 125 ++++++++++++++++++++++++++++ tools/virsh-domain.c | 76 +++++++++++++++++ tools/virsh.pod | 4 + 14 files changed, 640 insertions(+), 2 deletions(-) -- 2.20.1

3 8

[libvirt] [PATCH] nodedev: add missing include for virFileMakePathWithMode
by Daniel P. Berrangé 11 Jul '19

11 Jul '19

Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- Pushed as a build fix for BSD src/node_device/node_device_hal.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/node_device/node_device_hal.c b/src/node_device/node_device_hal.c index 876e808dce..1920f4566e 100644 --- a/src/node_device/node_device_hal.c +++ b/src/node_device/node_device_hal.c @@ -32,6 +32,7 @@ #include "datatypes.h" #include "viralloc.h" #include "viruuid.h" +#include "virfile.h" #include "virpci.h" #include "virpidfile.h" #include "virlog.h" -- 2.21.0

1 0

[libvirt] [jenkins-ci PATCH 0/2] guests: Use OpenJDK 11 where possible
by Andrea Bolognani 11 Jul '19

11 Jul '19

See commit 2/2 for more information. Andrea Bolognani (2): guests: Add mapping for OpenJDK on Ubuntu guests: Use OpenJDK 11 where possible guests/vars/mappings.yml | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) -- 2.21.0

2 4