August 2018 - Devel - libvirt List Archives

[libvirt] [PATCH v3 0/2] Fix detection of slow guest shutdown
by Christian Ehrhardt 21 Aug '18

21 Aug '18

Hi, after a good discussion a few days ago in https://www.redhat.com/archives/libvir-list/2018-August/msg00122.html and a short lived but back then untested v2 in https://www.redhat.com/archives/libvir-list/2018-August/msg00199.html I finally get access to the right HW again and completed the series. Being finally retested and working I finally feel safe to submit without a RFC prefix. I think this would be a great addition for a better handling of guests with plenty of host devices passed through. With the new code in place I can shutdown systems that have 12, 16 or even more hostdevs attached without getting into the "zombie" mode where libvirt will forever consider the guest as "in shutdown" as it gave up waiting too early because the signal zero still was able to reach it. Scaling examples (extracted with gdb): 16 Devices: virProcessKillPainfullyDelay (pid=67096, force=true, extradelay=32) 12 Devices: virProcessKillPainfullyDelay (pid=68251, force=true, extradelay=24) *Updates in v3* - fixup some issues found in testing and code checks *Updates in v2* - removed the "accept the lack of /proc/<pid> as valid process removal" approach due to valid concerns about reusing ressources. - added a dynamic extra wait scaling with the amount of hostdevs Christian Ehrhardt (2): process: wait longer on kill per assigned Hostdev process: wait longer 5->30s on hard shutdown src/libvirt_private.syms | 1 + src/qemu/qemu_process.c | 5 +++-- src/util/virprocess.c | 20 ++++++++++++++++---- src/util/virprocess.h | 1 + 4 files changed, 21 insertions(+), 6 deletions(-) -- 2.17.1

3 10

[libvirt] [PATCH] conf: Parse guestfwd channel device info again
by Michal Privoznik 21 Aug '18

21 Aug '18

https://bugzilla.redhat.com/show_bug.cgi?id=1610072 Due to historical reasons we were not parsing device info on guestfwd channel. Sure, it doesn't make much sense to parse <address/> but it surely makes sense to parse its alias (which might be an user alias). This reverts commit 47a3dd46ead20e6fdc30bcdc1b8e707e250d33da which fixed https://bugzilla.redhat.com/show_bug.cgi?id=1172526. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/conf/domain_conf.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 3c254801cd..123abec1ba 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -12807,14 +12807,8 @@ virDomainChrDefParseXML(virDomainXMLOptionPtr xmlopt, } } - if (def->deviceType == VIR_DOMAIN_CHR_DEVICE_TYPE_CHANNEL && - def->targetType == VIR_DOMAIN_CHR_CHANNEL_TARGET_TYPE_GUESTFWD) { - VIR_DEBUG("Ignoring device address for gustfwd channel"); - } else if (virDomainDeviceInfoParseXML(xmlopt, node, - &def->info, flags) < 0) { + if (virDomainDeviceInfoParseXML(xmlopt, node, &def->info, flags) < 0) goto error; - } - if (def->deviceType == VIR_DOMAIN_CHR_DEVICE_TYPE_SERIAL && def->targetType == VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_USB && -- 2.16.4

2 3

[libvirt] [PATCH v2 0/2] Parse guestfwd channel device info again
by Michal Privoznik 21 Aug '18

21 Aug '18

v2 of: https://www.redhat.com/archives/libvir-list/2018-August/msg01208.html diff to v1: - did what Andrea suggested Michal Prívozník (2): device_conf: Split virDomainDeviceInfoClear conf: Parse guestfwd channel device info again src/conf/device_conf.c | 10 ++++++++-- src/conf/device_conf.h | 1 + src/conf/domain_conf.c | 13 ++++++------- 3 files changed, 15 insertions(+), 9 deletions(-) -- 2.16.4

2 6

[libvirt] [PULL 00/12] Ui 20180821 patches
by Gerd Hoffmann 21 Aug '18

21 Aug '18

The following changes since commit d0092d90eb546a8bbe9e9120426c189474123797: Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20180820' into staging (2018-08-20 17:41:18 +0100) are available in the git repository at: git://git.kraxel.org/qemu tags/ui-20180821-pull-request for you to fetch changes up to 8a674d96a572c7ec1219a24ef223160dc6a27ca8: util: promote qemu_egl_rendernode_open() to libqemuutil (2018-08-21 08:24:30 +0200) ---------------------------------------------------------------- ui: misc ui fixed which piled up during 3.0 release freeze. ---------------------------------------------------------------- Daniel P. Berrangé (2): doc: switch to modern syntax for VNC TLS setup vnc: remove support for deprecated tls, x509, x509verify options Marc-André Lureau (3): ui: use enum to string helpers dmabuf: add y0_top, pass it to spice util: promote qemu_egl_rendernode_open() to libqemuutil Paolo Bonzini (2): spice-display: access ptr_x/ptr_y under Mutex spice-display: fix qemu_spice_cursor_refresh_bh locking Peter Wu (1): vnc: fix memleak of the "vnc-worker-output" name Philippe Mathieu-Daudé (1): ui/vnc: Remove useless parenthesis around DIV_ROUND_UP macro Tao Wu via Qemu-devel (1): sdl2: redraw correctly when scanout_mode enabled. Thomas Huth (2): ui/sdl2: Remove the obsolete SDL_INIT_NOPARACHUTE flag ui/sdl2: Fix broken -full-screen CLI option include/qemu/drm.h | 6 ++++ include/ui/console.h | 1 + qemu-keymap.c | 2 +- ui/console.c | 6 ++-- ui/egl-helpers.c | 51 ++-------------------------- ui/sdl2-gl.c | 5 +++ ui/sdl2.c | 13 +++----- ui/spice-display.c | 40 ++++++++++++++-------- ui/vnc-enc-tight.c | 2 +- ui/vnc-jobs.c | 3 +- ui/vnc.c | 94 ++-------------------------------------------------- util/drm.c | 66 ++++++++++++++++++++++++++++++++++++ MAINTAINERS | 1 + qemu-deprecated.texi | 20 ----------- qemu-doc.texi | 20 ++++++++--- qemu-options.hx | 43 ------------------------ util/Makefile.objs | 1 + 17 files changed, 138 insertions(+), 236 deletions(-) create mode 100644 include/qemu/drm.h create mode 100644 util/drm.c -- 2.9.3

2 14

[libvirt] [PATCH] util: Don't delete the original file for truncation
by Marc Hartmayer 21 Aug '18

21 Aug '18

Truncate means that if a file exists it's length will be truncated to 0, but the mode and the owner shall be unchanged. The current behavior is that the original file is deleted and a new file is created. Let's fix this by using O_TRUNC. The function virRotatingFileWriterDelete is now unused but may be used in the future and is therefore still defined. Signed-off-by: Marc Hartmayer <mhartmay(a)linux.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy(a)linux.ibm.com> --- Note: This change has the (potentially unwanted) security effect that the owner/group of the log file does not change. Before this patch the old log file was deleted and the newly created log file was owned by the virtlogd user. Now, if a user has created the log file before, he can read the logs. If we don't wanna have this effect we can either adjust/add a virtlogd API or do a chown within the calling driver (e.g. QEMU driver). Side note: the original behavior has changed with the patch series "qemu: use FD passing for chardev UNIX sockets". Example where it the behavior has changed: <console type='file'> <source path='/tmp/console.log'/> <target type='serial'/> </console> --- src/util/virrotatingfile.c | 49 ++++++++++++++++++++++++++++++++-------------- 1 file changed, 34 insertions(+), 15 deletions(-) diff --git a/src/util/virrotatingfile.c b/src/util/virrotatingfile.c index ca62a8e02641..d6ab3780aae3 100644 --- a/src/util/virrotatingfile.c +++ b/src/util/virrotatingfile.c @@ -98,17 +98,25 @@ virRotatingFileReaderEntryFree(virRotatingFileReaderEntryPtr entry) static virRotatingFileWriterEntryPtr virRotatingFileWriterEntryNew(const char *path, - mode_t mode) + mode_t mode, + bool truncate) { virRotatingFileWriterEntryPtr entry; struct stat sb; + /* O_APPEND is also useful in combination with O_TRUNC since it + * guarantees the atomicity of a write operation (at least for + * POSIX systems) */ + int oflag = O_CREAT|O_APPEND|O_WRONLY|O_CLOEXEC; VIR_DEBUG("Opening %s mode=0%02o", path, mode); if (VIR_ALLOC(entry) < 0) return NULL; - if ((entry->fd = open(path, O_CREAT|O_APPEND|O_WRONLY|O_CLOEXEC, mode)) < 0) { + if (truncate) + oflag |= O_TRUNC; + + if ((entry->fd = open(path, oflag, mode)) < 0) { virReportSystemError(errno, _("Unable to open file: %s"), path); goto error; @@ -182,18 +190,10 @@ virRotatingFileReaderEntryNew(const char *path) static int -virRotatingFileWriterDelete(virRotatingFileWriterPtr file) +virRotatingFileWriterDeleteBackup(virRotatingFileWriterPtr file) { size_t i; - if (unlink(file->basepath) < 0 && - errno != ENOENT) { - virReportSystemError(errno, - _("Unable to delete file %s"), - file->basepath); - return -1; - } - for (i = 0; i < file->maxbackup; i++) { char *oldpath; if (virAsprintf(&oldpath, "%s.%zu", file->basepath, i) < 0) @@ -214,6 +214,24 @@ virRotatingFileWriterDelete(virRotatingFileWriterPtr file) } +static int ATTRIBUTE_UNUSED +virRotatingFileWriterDelete(virRotatingFileWriterPtr file) +{ + if (unlink(file->basepath) < 0 && + errno != ENOENT) { + virReportSystemError(errno, + _("Unable to delete file %s"), + file->basepath); + return -1; + } + + if (virRotatingFileWriterDeleteBackup(file) < 0) + return -1; + + return 0; +} + + /** * virRotatingFileWriterNew * @path: the base path for files @@ -257,12 +275,12 @@ virRotatingFileWriterNew(const char *path, file->maxbackup = maxbackup; file->maxlen = maxlen; - if (trunc && - virRotatingFileWriterDelete(file) < 0) + if (trunc && virRotatingFileWriterDeleteBackup(file) < 0) goto error; if (!(file->entry = virRotatingFileWriterEntryNew(file->basepath, - mode))) + mode, + trunc))) goto error; return file; @@ -491,7 +509,8 @@ virRotatingFileWriterAppend(virRotatingFileWriterPtr file, return -1; if (!(tmp = virRotatingFileWriterEntryNew(file->basepath, - file->mode))) + file->mode, + false))) return -1; virRotatingFileWriterEntryFree(file->entry); -- 2.13.4

2 3

[libvirt] [PATCH 0/2] vnc: remove deprecated TLS related features
by Daniel P. Berrangé 21 Aug '18

21 Aug '18

Daniel P. Berrangé (2): doc: switch to modern syntax for VNC TLS setup vnc: remove support for deprecated tls, x509, x509verify options qemu-deprecated.texi | 20 ---------- qemu-doc.texi | 20 +++++++--- qemu-options.hx | 43 --------------------- ui/vnc.c | 91 -------------------------------------------- 4 files changed, 15 insertions(+), 159 deletions(-) -- 2.17.1

2 3

[libvirt] [RFC v3] x86 RDT Cache Monitoring Technology (CMT)
by Huaqiang,Wang 21 Aug '18

21 Aug '18

For the supporting of Intel x86 Cache Monitoring Technology (CMT) in libvirt, we have already discussed a couple of times, and I have raised two RFCs, you can find the discussion from the following links, and thanks to the people who participated in the discussion online and offline. RFCv2 links https://www.redhat.com/archives/libvir-list/2018-July/msg00409.html https://www.redhat.com/archives/libvir-list/2018-July/msg01241.html RFCv1 https://www.redhat.com/archives/libvir-list/2018-June/msg00674.html Nearly one month has passed since last online discussion, in this month, MBA (memory bandwidth allocation) feature has been introduced by Niu Bin, and some fundamental code for CMT feature has been changed. This change is not that significant, and I wrote my POC code for this RFC again upon the new virresctrl framework, I also made some changes for the CMT feature. I think it is better to summarize my thought about the enabling of CMT here before I submit my code. I'll try to keep consistent with the previous discussion. 1. What is the purpose of this RFC and later patches? Short answer: Introducing Kernel resctrlfs based x86 CMT feature to libvirt. Detail explanation: Latest kernel has removed the "cmt, mbmt, mbml" perf event, and libvirt relies on these kernel perf events to report the result of CMT/MBM through virperf interface. Libvirt will no longer support CMT/MBM feature for Linux distribution with the latest kernel, while there CPU features are vital to some software such as Openstack Nova. About the deprecation of cmt/mbm perf events, refer to the following link for details: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git /commit/?id=c39a0e2c8850f08249383f2425dbd8dbe4baad69 Latest kernel introduces the RDT feature, including CAT MBA CDP CMT and MBM, through resource control file system (resctrl fs), and libvirt already implemented the cache and memory bandwidth allocation through kernel resctrlfs interface. I'd like to add the cache and memory bandwidth monitoring feature to libvirt through kernel resctrlfs interface. Since CMT and MBM shares a very similar kernel interface, for simplicity, only discuss CMT here. MBM is the followup that will be implemented after CMT, which should be very straightforward once CMT is ready. It is planned to output the cache monitoring result through 'domstats' command, and create or delete monitoring groups with either domain's statical XML configuration file or a new 'virsh' runtime command. 2. How many stages scheduled to implement the libvirt CMT(MBM) feature? Short answer: 4 stages. Detail explanation: stage 1: Statical CMT feature. Implementing the libvirt interface for creating and deleting CMT groups for statically through domain's XML configuration file, changes only takes effect after a reboot. Stage 2: Statical MBM feature. Very similar to stage1 CMT. Stage 3: Dynamical CMT feature. In this stage, it is aimed to implement interfaces to change the CMT groups dynamically at runtime, with no requirement for a reboot. My basic thought is implementing a new 'virsh' sub-command providing all interfaces. I also hope to implement the functionality of dynamical cache allocation (CAT) with this command. Stage 4: Dynamical MBM feature. Depending on the implementation of stage3, should share many interfaces created at stage 3. This RFC mainly covers the discussion of stage1. 3. What is the interface for statical CMT feature (the output and input)? Short answer: output through command 'domstats'; input through XML file. Detail explanation: Output: Either CMT or MBM is Intel x86 CPU feature, so I put the statistical message under the result of 'cpu-total' subcommand. This is different with RFCv2, in RFCv2 a separate 'domstats' sub-command has been created, with the name 'cpu-resource', Martin and I were not very satisfied with the the naming. The output would be: virsh domstats --cpu--total Domain: 'ubuntu16.04-base' cpu.time=3143918715755 cpu.user=15320000000 cpu.system=235280000000 cpu.cache.monitor.count=3 cpu.cache.0.name=vcpus_2 cpu.cache.0.vcpus=2 cpu.cache.0.bank.count=2 cpu.cache.0.bank.0.bytes=5488 cpu.cache.0.bank.1.bytes =4410000 cpu.cache.1.name=vcpus_1 cpu.cache.1.vcpus=1 cpu.cache.1.bank.count=2 cpu.cache.1.bank.0.bytes =7839744 cpu.cache.1.bank.1.bytes =0 cpu.cache.2.name=vcpu_0,3 cpu.cache.2.vcpus=0,3 cpu.cache.2.bank.count=2 cpu.cache.2.bank.0.bytes=53796864 cpu.cache.2.bank.1.bytes=0 Comparing with RFCv2, the CMT information is slightly changed. Input: Comparing with RFC v2, this part has been changed. Now the resource monitoring group is considered as a resource monitor toward specific allocation. The main interface for creating monitoring group is through XML file. The proposed configuration is like: <cputune> <cachetune vcpus='0-1'> <cache id='0' level='3' type='code' size='7680' unit='KiB'/> <cache id='1' level='3' type='data' size='3840' unit='KiB'/> + <monitor vcpus='0-1'/> + <monitor vcpus='0'/> </cachetune> <cachetune vcpus='3'> + <monitor vcpus='3'/> </cachetune> </cputune> In above XML, created 2 cache resctrl allocation groups and 3 resctrl monitoring groups. For cache allocation group with vcpu 0 and vcpu 1, there are two monitors: one is monitoring the last level cache occupancy of two vcpus, while the other one only monitors the cache occupancy of vcpu 0. In another cache allocation, there is no explicit cache resource allocation, this cache allocation uses the resource of 'default' cache allocation group, and a cache monitor is created to monitor the cache occupancy. 4. Have emulator and io threads been considered for CMT? Short answer: No. Unsupport to allocate dedicated cache or memory bandwidth for emulator and io threads.

1 0

[libvirt] [jenkins-ci PATCH 0/5] Drop last use of ~/rpmbuild, other cleanups
by Andrea Bolognani 20 Aug '18

20 Aug '18

Patch 1/5 ensures ~/rpmbuild is no longer used, while the remaining ones increase consistency in the various rules used to build RPMs. Andrea Bolognani (5): jobs: Call rpmbuild explicitly for Python projects jobs: Call rpmbuild and sed consistently jobs: Introduce strip_buildrequires jobs: Minimize strip_buildrequires jobs: Don't remove non-existing archives jobs/autotools.yaml | 7 ++----- jobs/defaults.yaml | 5 +++++ jobs/perl-modulebuild.yaml | 5 +---- jobs/python-distutils.yaml | 5 +++-- projects/osinfo-db.yaml | 2 +- 5 files changed, 12 insertions(+), 12 deletions(-) -- 2.17.1

2 14

[libvirt] [PATCH] spec: Add support for RISC-V (riscv64)
by Richard W.M. Jones 20 Aug '18

20 Aug '18

From: David Abdurachmanov <david.abdurachmanov(a)gmail.com> --- libvirt.spec.in | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libvirt.spec.in b/libvirt.spec.in index e7196b7d3b..5cb995a6a4 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -101,7 +101,7 @@ %endif # Numactl is not available on s390[x] and ARM -%ifarch s390 s390x %{arm} +%ifarch s390 s390x %{arm} riscv64 %define with_numactl 0 %endif @@ -120,7 +120,7 @@ %endif # zfs-fuse is not available on some architectures -%ifarch s390 s390x aarch64 +%ifarch s390 s390x aarch64 riscv64 %define with_storage_zfs 0 %endif @@ -195,7 +195,7 @@ %if %{with_qemu} || %{with_lxc} || %{with_uml} # numad is used to manage the CPU and memory placement dynamically, # it's not available on s390[x] and ARM. - %ifnarch s390 s390x %{arm} + %ifnarch s390 s390x %{arm} riscv64 %define with_numad 0%{!?_without_numad:1} %endif %endif -- 2.18.0

2 1

[libvirt] [sandbox PATCH 0/2] Remove all uses of g_type_class_add_private()
by Andrea Bolognani 20 Aug '18

20 Aug '18

This fixes build on Fedora Rawhide. Andrea Bolognani (2): configure: Use a single required version for GLib Remove all uses of g_type_class_add_private() configure.ac | 7 +++---- libvirt-sandbox/libvirt-sandbox-builder-container.c | 4 +--- libvirt-sandbox/libvirt-sandbox-builder-initrd.c | 4 +--- libvirt-sandbox/libvirt-sandbox-builder-machine.c | 4 +--- libvirt-sandbox/libvirt-sandbox-builder.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-disk.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-initrd.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-interactive.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-mount-file.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-mount-guest-bind.c | 5 ++--- libvirt-sandbox/libvirt-sandbox-config-mount-host-bind.c | 5 ++--- libvirt-sandbox/libvirt-sandbox-config-mount-host-image.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-mount-ram.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-mount.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-network-address.c | 4 +--- .../libvirt-sandbox-config-network-filterref-parameter.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-network-filterref.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-network-route.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-network.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-service-generic.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-service-systemd.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config-service.c | 4 +--- libvirt-sandbox/libvirt-sandbox-config.c | 4 +--- libvirt-sandbox/libvirt-sandbox-console-raw.c | 4 +--- libvirt-sandbox/libvirt-sandbox-console-rpc.c | 4 +--- libvirt-sandbox/libvirt-sandbox-console.c | 4 +--- libvirt-sandbox/libvirt-sandbox-context-interactive.c | 4 +--- libvirt-sandbox/libvirt-sandbox-context-service.c | 4 +--- libvirt-sandbox/libvirt-sandbox-context.c | 4 +--- 29 files changed, 33 insertions(+), 88 deletions(-) -- 2.17.1

2 4