- Devel - libvirt List Archives

[libvirt] [PATCH 0/7] introduce support for an embedded driver mode
by Daniel P. Berrangé 20 Dec '19

20 Dec '19

This is a followup to: https://www.redhat.com/archives/libvir-list/2019-May/msg00467.html This series implements support for an embedded driver mode for libvirt, with initial support in the QEMU and secrets drivers. In this mode of operation, the driver stores all its config and state under a private directory tree. See the individual patches for the illustrated directory hierarchy used. The intent of this embedded mode is to suit cases where the application is using virtualization as a building block for some functionality, as opposed to running traditional "full OS" builds. The long time posterchild example would be libguestfs, while a more recent example could be Kata containers. The general principal in enabling this embedded mode is that the functionality available should be identical to that seen when the driver is running inside libvirtd. This is achieved by loading the exact same driver .so module as libvirtd would load, and simply configuring it with a different directory layout. The result of this is that when running in embedded mode, the driver can still talk to other secondary drivers running inside libvirtd if desired. This is useful, for example, to connect a VM to the default virtual network. The secondary drivers can be made to operate in embedded mode as well, however, this will require some careful consideration for each driver to ensure they don't clash with each other. Thus in this series only the secret driver is enabled for embedded mode. This is required to enable use of VMs with encrypted disks, or authenticated network block storage. In this series we introduce a new command line tool 'virt-qemu-run' which is a really simple tool for launching a VM in embedded mode. I'm not entirely sure whether we should provide this as an official supported tool in this way, or merely put it into the 'examples' directory as demo-ware. With testing of the virt-qemu-run tool we can immediately see what the next important thing to tackle is: performance. We have not really cared too much about the startup performance of libvirtd as this is a one time cost when the mgmt application connects. We did none the less cache capabilities because probing caps for 30 QEMU binaries takes a long time. Even with this caching it takes an unacceptably long time to start a VM in embedded mode. About 100 ms to open the embedded QEMU driver, assuming pre-cached capabilies - ~2 seconds if not cached and all 30 QEMU targets are present. Then about 300 ms to actually start the QEMU guest. IOW, about 400 ms to get QEMU running. NB this is measuring time from launching the virt-run-qemu program, to the point at which the API call 'virDomainCreate' returns control. This has both libvirt & QEMU overhead in & I don't have clear figures to distinguish, but I can see a 40 ms delay between issuing the 'qmp_capabilities' call and getting a reply, which is QEMU startup overead. This is a i440fx based QEMU with a general purpose virtio-pci config (disk, net, etc) tyupical for running a full OS. I've not tried any kind of optimized QEMU config with microvm. I've already started on measuring & optimizing & identified several key areas that can be addressed, but it is all ultimately about not doing work before we need the answers from that work (which often means we will never do the work at all). For example, we shouldn't probe all 30 QEMU's upfront. If the app is only going to create an x86_64 KVM guest we should only care about that 1 QEMU. This is painful because parsing any guest XML requires a virCapsPtr which in turn causes probing of every QEMU binary. I've got in progress patches to eliminate virCapsPtr almost entirely and work directly with the virQEMUCapsPtr instead. It is possible we'll want to use a different file format for storing the cached QEMU capabilities, and the CPU feature/model info. Parsing this XML is a non-negligible time sink. A binary format is likely way quicker, especially if its designed to be just mmap'able for direct read. To be investigated... We shouldn't probe for whether host PM suspend is possible unless someone wants that info, or tries to issue that API call. After starting QEMU we spend 150-200 ms issuing a massive number of qom-get calls to check whether QEMU enabled each individual CPU feature flag. We only need this info if someone asks for the live XML or we intend to live migrate etc. So we shouldn't issue these qom-get calls in the "hot path" of QEMU startup. It can be done later in a non-time critical point. Also the QEMU API for this is horribly inefficient to require so many qom-get calls. There's more but I won't talk about it now. Suffice to say that I think we can get libvirt overhead down to less than 100 ms fairly easily and probably even down to less than 50 ms without much effort. The exact figure will depend on what libvirt features you want enabled, and how much work we want/need to put into optimization. We'll want to fix the really gross mistakes & slow downs, but we'll want guidance from likely users as to their VM startup targets to decide how much work needs investing. This optimization will ultimately help non-embedded QEMU mode too, making it faster to respond & start. Changed in v2: - Use a simplified directory layout for embedded mode. Previously we just put a dir prefix onto the normal paths. This has the downside that the embedded drivers paths are needlessly different for privileged vs unprivileged user. It also results in very long paths which can be a problem for the UNIX socket name length limits. - Also ported the secret driver to support embedded mode - Check to validate that the event loop is registered. - Add virt-qemu-run tool for embedded usage. - Added docs for the qemu & secret driver explaining embedded mode Daniel P. Berrangé (7): access: report an error if no access manager is present libvirt: pass a directory path into drivers for embedded usage event: add API for requiring an event loop impl to be registered libvirt: support an "embed" URI path selector for opening drivers qemu: add support for running QEMU driver in embedded mode secrets: add support for running secret driver in embedded mode qemu: introduce a new "virt-qemu-run" program build-aux/syntax-check.mk | 2 +- docs/drivers.html.in | 1 + docs/drvqemu.html.in | 84 +++++++ docs/drvsecret.html.in | 82 +++++++ libvirt.spec.in | 2 + po/POTFILES.in | 1 + src/Makefile.am | 9 + src/access/viraccessmanager.c | 5 + src/driver-state.h | 2 + src/driver.h | 2 + src/interface/interface_backend_netcf.c | 7 + src/interface/interface_backend_udev.c | 7 + src/libvirt.c | 93 ++++++- src/libvirt_internal.h | 4 +- src/libxl/libxl_driver.c | 7 + src/lxc/lxc_driver.c | 8 + src/network/bridge_driver.c | 7 + src/node_device/node_device_hal.c | 7 + src/node_device/node_device_udev.c | 7 + src/nwfilter/nwfilter_driver.c | 7 + src/qemu/Makefile.inc.am | 26 ++ src/qemu/qemu_conf.c | 38 ++- src/qemu/qemu_conf.h | 6 +- src/qemu/qemu_driver.c | 21 +- src/qemu/qemu_process.c | 15 +- src/qemu/qemu_shim.c | 313 ++++++++++++++++++++++++ src/qemu/qemu_shim.pod | 94 +++++++ src/remote/remote_daemon.c | 1 + src/remote/remote_driver.c | 1 + src/secret/secret_driver.c | 41 +++- src/storage/storage_driver.c | 7 + src/util/virevent.c | 25 ++ src/util/virevent.h | 2 + src/vz/vz_driver.c | 7 + tests/domaincapstest.c | 2 +- tests/testutilsqemu.c | 2 +- 36 files changed, 920 insertions(+), 25 deletions(-) create mode 100644 docs/drvsecret.html.in create mode 100644 src/qemu/qemu_shim.c create mode 100644 src/qemu/qemu_shim.pod -- 2.23.0

3 16

[libvirt] [PATCH 00/42] Cleanups after adopting g_get_*_dir()
by Fabiano Fidêncio 20 Dec '19

20 Dec '19

After adopting g_get_*_dir(), internally, when implementing virGetUser*Dir(), the libvirt functions changed their behaviour as NULL is never ever returned by the GLib functions. Knowing that, let's cleanup the callers' code of those functions, removing the unnecessary checks. While doing the cleanup mentioned above, I've noticed that some other cleanups could be done, when touching the very same function, as using g_autofree. It's done as part of this series as well. phyp driver was not touched as there's some plan to just drop its code entirely: https://www.redhat.com/archives/libvir-list/2019-December/msg01162.html Fabiano Fidêncio (42): tools: Use g_autofree on cmdCd() vbox: Don't leak virGetUserDirectory()'s output rpc: Use g_autofree on virNetClientNewLibSSH2() rpc: Use g_autofree on virNetClientNewLibssh() rpc: Don't check the output of virGetUserDirectory() qemu: Don't check the output of virGetUserDirectory() util: Don't check the output of virGetUserConfigDirectory() storage: Don't check the output of virGetUserConfigDirectory() secret: Don't check the output of virGetUserConfigDirectory() tools: Don't check the output of virGetUserCacheDirectory() vbox: Use g_autofree on vboxDomainScreenshot() vbox: Don't check the output of virGetUserCacheDirectory() util: Use g_autofree on virLogSetDefaultOutputToFile() util: Don't check the output of virGetUserCacheDirectory() qemu: Don't check the output of virGetUserCacheDirectory() rpc: Don't check the output of virGetUserConfigDirectory() remote: Don't check the output of virGetUserConfigDirectory() qemu: Don't check the output of virGetUserConfigDirectory() network: Don't check the output of virGetUserConfigDirectory() logging: Use g_autofree on virLogDaemonConfigFilePath() logging: Don't check the output of virGetUserConfigDirectory() locking: Use g_autofree on virLockDaemonConfigFilePath() locking: Don't check the output of virGetUserConfigDirectory() util: Don't check the output of virGetUserRuntimeDirectory() storage: Don't check the output of virGetUserRuntimeDirectory() secret: Don't check the output of virGetUserRuntimeDirectory() rpc: Don't check the output of virGetUserRuntimeDirectory() remote: Don't check the output of virGetUserRuntimeDirectory() qemu: Don't check the output of virGetUserRuntimeDirectory() node_device: Don't check the output of virGetUserRuntimeDirectory() network: Don't check the output of virGetUserRuntimeDirectory() logging: Use g_autofree on virLogManagerDaemonPath() logging: Use g_autofree on virLogDaemonUnixSocketPaths() logging: Use g_autofree on virLogDaemonExecRestartStatePath() logging: Don't check the output of virGetUserRuntimeDirectory() locking: Use g_autofree on virLockManagerLockDaemonPath() locking: Use g_autofree on virLockDaemonUnixSocketPaths() locking: Use g_autofree on virLockDaemonExecRestartStatePath() locking: Don't check the output of virGetUserRuntimeDirectory() interface: Don't check the output of virGetUserRuntimeDirectory() admin: Use g_autofree on getSocketPath() admin: Don't check the output of virGetUserRuntimeDirectory() src/admin/libvirt-admin.c | 6 +-- src/interface/interface_backend_netcf.c | 3 +- src/interface/interface_backend_udev.c | 3 +- src/locking/lock_daemon.c | 31 +++---------- src/locking/lock_daemon_config.c | 9 +--- src/locking/lock_driver_lockd.c | 7 +-- src/logging/log_daemon.c | 31 +++---------- src/logging/log_daemon_config.c | 9 +--- src/logging/log_manager.c | 7 +-- src/network/bridge_driver.c | 2 - src/node_device/node_device_hal.c | 3 +- src/node_device/node_device_udev.c | 3 +- src/qemu/qemu_conf.c | 7 +-- src/qemu/qemu_interop_config.c | 3 -- src/remote/remote_daemon.c | 8 +--- src/remote/remote_daemon_config.c | 6 +-- src/remote/remote_driver.c | 3 +- src/rpc/virnetclient.c | 59 +++++++++---------------- src/rpc/virnetsocket.c | 3 +- src/rpc/virnettlscontext.c | 12 ----- src/secret/secret_driver.c | 6 +-- src/storage/storage_driver.c | 2 - src/util/virauth.c | 3 +- src/util/virconf.c | 2 - src/util/virhostdev.c | 3 +- src/util/virlog.c | 13 ++---- src/util/virpidfile.c | 3 +- src/vbox/vbox_common.c | 12 ++--- src/vbox/vbox_storage.c | 7 ++- tools/vsh.c | 13 ++---- 30 files changed, 73 insertions(+), 206 deletions(-) -- 2.24.1

5 100

[libvirt] [PATCH] cpu: add CLZERO CPUID support for AMD platforms
by Ani Sinha 20 Dec '19

20 Dec '19

Qemu commit e900135dcfb67 ("i386: Add CPUID bit for CLZERO and XSAVEERPTR") adds support for CLZERO CPUID bit. This commit extends support for this CPUID bit into libvirt. Signed-off-by: Ani Sinha <ani.sinha(a)nutanix.com> --- src/cpu_map/x86_EPYC-IBPB.xml | 1 + src/cpu_map/x86_EPYC.xml | 1 + src/cpu_map/x86_features.xml | 3 +++ 3 files changed, 5 insertions(+) diff --git a/src/cpu_map/x86_EPYC-IBPB.xml b/src/cpu_map/x86_EPYC-IBPB.xml index 283697e..a70fbd8 100644 --- a/src/cpu_map/x86_EPYC-IBPB.xml +++ b/src/cpu_map/x86_EPYC-IBPB.xml @@ -14,6 +14,7 @@ <feature name='bmi2'/> <feature name='clflush'/> <feature name='clflushopt'/> + <feature name='clzero'/> <feature name='cmov'/> <feature name='cr8legacy'/> <feature name='cx16'/> diff --git a/src/cpu_map/x86_EPYC.xml b/src/cpu_map/x86_EPYC.xml index f060139..6c11d82 100644 --- a/src/cpu_map/x86_EPYC.xml +++ b/src/cpu_map/x86_EPYC.xml @@ -14,6 +14,7 @@ <feature name='bmi2'/> <feature name='clflush'/> <feature name='clflushopt'/> + <feature name='clzero'/> <feature name='cmov'/> <feature name='cr8legacy'/> <feature name='cx16'/> diff --git a/src/cpu_map/x86_features.xml b/src/cpu_map/x86_features.xml index 2bed1e0..dd62755 100644 --- a/src/cpu_map/x86_features.xml +++ b/src/cpu_map/x86_features.xml @@ -473,6 +473,9 @@ <feature name='ibpb'> <cpuid eax_in='0x80000008' ebx='0x00001000'/> </feature> + <feature name='clzero'> + <cpuid eax_in='0x80000008' ebx='0x00000001'/> + </feature> <feature name='amd-ssbd'> <cpuid eax_in='0x80000008' ebx='0x01000000'/> </feature> -- 1.9.4

2 2

[libvirt] [PATCH] AUTHORS: Add Fabiano Fidêncio
by Ján Tomko 19 Dec '19

19 Dec '19

$ git log --committer=fidencio --pretty=oneline | wc -l 12 Signed-off-by: Ján Tomko <jtomko(a)redhat.com> --- Déjà vu from c379576dbc80a66820e256f9ce27595270d95ac2 except the mailmap entry is already set up. AUTHORS.in | 1 + 1 file changed, 1 insertion(+) diff --git a/AUTHORS.in b/AUTHORS.in index fbac54c48d..4ed8ceb858 100644 --- a/AUTHORS.in +++ b/AUTHORS.in @@ -19,6 +19,7 @@ Daniel Veillard <veillard(a)redhat.com> Doug Goldstein <cardoe(a)gentoo.org> Eric Blake <eblake(a)redhat.com> Erik Skultety <eskultet(a)redhat.com> +Fabiano Fidêncio <fidencio(a)redhat.com> Gao Feng <gaofeng(a)cn.fujitsu.com> Guido Günther <agx(a)sigxcpu.org> Ján Tomko <jtomko(a)redhat.com> -- 2.21.0

3 2

[libvirt] CfP VHPC20: HPC Containers-Kubernetes
by VHPC 20 19 Dec '19

19 Dec '19

==================================================================== CALL FOR PAPERSa 15th Workshop on Virtualization in High-Performance Cloud Computing (VHPC 20) held in conjunction with the International Supercomputing Conference - High Performance, June 21-25, 2020, Frankfurt, Germany. (Springer LNCS Proceedings) ==================================================================== Date: June 25, 2020 Workshop URL: vhpc[dot]org Abstract registration Deadline: Jan 31st, 2020 Paper Submission Deadline: Apr 05th, 2020 Springer LNCS Call for Papers Containers and virtualization technologies constitute key enabling factors for flexible resource management in modern data centers, and particularly in cloud environments. Cloud providers need to manage complex infrastructures in a seamless fashion to support the highly dynamic and heterogeneous workloads and hosted applications customers deploy. Similarly, HPC environments have been increasingly adopting techniques that enable flexible management of vast computing and networking resources, close to marginal provisioning cost, which is unprecedented in the history of scientific and commercial computing. Most recently, Function as a Service (Faas) and Serverless computing, utilizing lightweight VMs-containers widens the spectrum of applications that can be deployed in a cloud environment, especially in an HPC context. Here, HPC-provided services can be become accessible to distributed workloads outside of large cluster environments. Various virtualization-containerization technologies contribute to the overall picture in different ways: machine virtualization, with its capability to enable consolidation of multiple underutilized servers with heterogeneous software and operating systems (OSes), and its capability to live-migrate a fully operating virtual machine (VM) with a very short downtime, enables novel and dynamic ways to manage physical servers; OS-level virtualization (i.e., containerization), with its capability to isolate multiple user-space environments and to allow for their coexistence within the same OS kernel, promises to provide many of the advantages of machine virtualization with high levels of responsiveness and performance; lastly, unikernels provide for many virtualization benefits with a minimized OS/library surface. I/O Virtualization in turn allows physical network interfaces to take traffic from multiple VMs or containers; network virtualization, with its capability to create logical network overlays that are independent of the underlying physical topology is furthermore enabling virtualization of HPC infrastructures. Publication Accepted papers will be published in a Springer LNCS proceedings volume. Topics of Interest The VHPC program committee solicits original, high-quality submissions related to virtualization across the entire software stack with a special focus on the intersection of HPC, containers-virtualization and the cloud. Major Topics: - HPC workload orchestration (Kubernetes) - Kubernetes HPC batch - HPC Container Environments Landscape - HW Heterogeneity - Container ecosystem (Docker alternatives) - Networking - Lightweight Virtualization - Unikernels / LibOS - State-of-the-art processor virtualization (RISC-V, EPI) - Containerizing HPC Stacks/Apps/Codes: Climate model containers each major topic encompassing design/architecture, management, performance management, modeling and configuration/tooling. Specifically, we invite papers that deal with the following topics: - HPC orchestration (Kubernetes) - Virtualizing Kubernetes for HPC - Deployment paradigms - Multitenancy - Serverless - Declerative data center integration - Network provisioning - Storage - OCI i.a. images - Isolation/security - HW Accelerators, including GPUs, FPGAs, AI, and others - State-of-practice/art, including transition to cloud - Frameworks, system software - Programming models, runtime systems, and APIs to facilitate cloud adoption - Edge use-cases - Application adaptation, success stories - Kubernetes Batch - Scheduling, job management - Execution paradigm - workflow - Data management - Deployment paradigm - Multi-cluster/scalability - Performance improvement - Workflow / execution paradigm - Podman: end-to-end Docker alternative container environment & use-cases - Creating, Running containers as non-root (rootless) - Running rootless containers with MPI - Container live migration - Running containers in restricted environments without setuid - Networking - Software defined networks and network virtualization - New virtualization NICs/Nitro alike ASICs for the data center? - Kubernetes SDN policy (Calico i.a.) - Kubernetes network provisioning (Flannel i.a.) - Lightweight Virtualization - Micro VMMs (Rust-VMM, Firecracker, solo5) - Xen - Nitro hypervisor (KVM) - RVirt - Cloud Hypervisor - Unikernels / LibOS - HPC Storage in Virtualization - HPC container storage - Cloud-native storage - Hypervisors in storage virtualization - Processor Virtualization - RISC-V hypervisor extensions - RISC-V Hypervisor ports - EPI - Composable HPC microservices - Containerizing Scientific Codes - Building - Deploying - Securing - Storage - Monitoring - Use case for containerizing HPC codes: Climate model containers for portability, reproducibility, traceability, immutability, provenance, data & software preservation The Workshop on Virtualization in High-Performance Cloud Computing (VHPC) aims to bring together researchers and industrial practitioners facing the challenges posed by virtualization in order to foster discussion, collaboration, mutual exchange of knowledge and experience, enabling research to ultimately provide novel solutions for virtualized computing systems of tomorrow. The workshop will be one day in length, composed of 20 min paper presentations, each followed by 10 min discussion sections, plus lightning talks that are limited to 5 minutes. Presentations may be accompanied by interactive demonstrations. Important Dates Jan 31st, 2020 - Abstract Apr 5th, 2020 - Paper submission deadline (Springer LNCS) Apr 26th, 2020 - Acceptance notification June 25th, 2020 - Workshop Day July 10th, 2020 - Camera-ready version due Chair Michael Alexander (chair), BOKU, Vienna, Austria Anastassios Nanos (co-chair), Sunlight.io, UK Program committee Stergios Anastasiadis, University of Ioannina, Greece Paolo Bonzini, Redhat, Italy Jakob Blomer, CERN, Europe Eduardo César, Universidad Autonoma de Barcelona, Spain Taylor Childers, Argonne National Laboratory, USA Stephen Crago, USC ISI, USA Tommaso Cucinotta, St. Anna School of Advanced Studies, Italy François Diakhaté CEA DAM Ile de France, France Kyle Hale, Northwestern University, USA Brian Kocoloski, Washington University, USA John Lange, University of Pittsburgh, USA Giuseppe Lettieri, University of Pisa, Italy Klaus Ma, Huawei, China Alberto Madonna, Swiss National Supercomputing Center, Switzerland Nikos Parlavantzas, IRISA, France Anup Patel, Western Digital, USA Kevin Pedretti, Sandia National Laboratories, USA Amer Qouneh, Western New England University, USA Carlos Reaño, Queen’s University Belfast, UK Adrian Reber, Redhat, Germany Riccardo Rocha, CERN, Europe Borja Sotomayor, University of Chicago, USA Jonathan Sparks, Cray, USA Kurt Tutschku, Blekinge Institute of Technology, Sweden John Walters, USC ISI, USA Yasuhiro Watashiba, Osaka University, Japan Chao-Tung Yang, Tunghai University, Taiwan Paper Submission-Publication Papers submitted to the workshop will be reviewed by at least two members of the program committee and external reviewers. Submissions should include abstract, keywords, the e-mail address of the corresponding author, and must not exceed 10 pages, including tables and figures at a main font size no smaller than 11 point. Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the conference to present the work. Accepted papers will be published in a Springer LNCS volume. The format must be according to the Springer LNCS Style. Initial submissions are in PDF; authors of accepted papers will be requested to provide source files. Abstract, Paper Submission Link: edas[dot]info/newPaper.php?c=26973 Lightning Talks Lightning Talks are non-paper track, synoptical in nature and are strictly limited to 5 minutes. They can be used to gain early feedback on ongoing research, for demonstrations, to present research results, early research ideas, perspectives and positions of interest to the community. Submit abstract via the main submission link. General Information The workshop is one day in length and will be held in conjunction with the International Supercomputing Conference - High Performance (ISC) 2019, June 21-25, Frankfurt, Germany.

1 0

[libvirt] [libvirt-tck PATCH v2] Add cases for nvram
by dzheng＠redhat.com 19 Dec '19

19 Dec '19

From: Dan Zheng <dzheng(a)redhat.com> This is to add the tests for below flags: - Sys::Virt::Domain::UNDEFINE_KEEP_NVRAM - Sys::Virt::Domain::UNDEFINE_NVRAM v1: https://www.redhat.com/archives/libvir-list/2019-December/msg00932.html Signed-off-by: Dan Zheng <dzheng(a)redhat.com> --- scripts/domain/401-ovmf-nvram.t | 144 ++++++++++++++++++++++++++++++++ 1 file changed, 144 insertions(+) create mode 100644 scripts/domain/401-ovmf-nvram.t diff --git a/scripts/domain/401-ovmf-nvram.t b/scripts/domain/401-ovmf-nvram.t new file mode 100644 index 0000000..4af2117 --- /dev/null +++ b/scripts/domain/401-ovmf-nvram.t @@ -0,0 +1,144 @@ +# -*- perl -*- +# +# Copyright (C) 2009 Red Hat, Inc. +# Copyright (C) 2018 Dan Zheng (dzheng(a)redhat.com) +# +# This program is free software; You can redistribute it and/or modify +# it under the GNU General Public License as published by the Free +# Software Foundation; either version 2, or (at your option) any +# later version +# +# The file "LICENSE" distributed along with this file provides full +# details of the terms and conditions +# + +=pod + +=head1 NAME + +domain/401-ovmf-nvram.t - Test OVMF related functions and flags + +=head1 DESCRIPTION + +The test cases validates OVMF related APIs and flags + +Sys::Virt::Domain::UNDEFINE_KEEP_NVRAM +Sys::Virt::Domain::UNDEFINE_NVRAM + +=cut + +use strict; +use warnings; + +use Test::More tests => 6; + +use Sys::Virt::TCK; +use File::stat; +use File::Copy; + + +my $tck = Sys::Virt::TCK->new(); +my $conn = eval { $tck->setup(); }; +BAIL_OUT "failed to setup test harness: $@" if $@; +END { $tck->cleanup if $tck; } + + +sub setup_nvram { + + my $loader_path = shift; + my $nvram_template = shift; + my $nvram_path = shift; + + # Check below two files should exist + # - /usr/share/OVMF/OVMF_CODE.secboot.fd + # - /usr/share/OVMF/OVMF_VARS.fd + if (!stat($loader_path) or !stat($nvram_template)) { + return undef; + } + + # Ensure the sample nvram file exists + copy($nvram_template, $nvram_path) or die "Copy failed: $!"; + + # Use 'q35' as machine type and add below lines to guest xml + # <loader readonly='yes' secure='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.secboot.fd</loader> + # <nvram template='/usr/share/OVMF/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/test_VARS.fd</nvram> + + my $xml = $tck->generic_domain(name => "tck")->as_xml; + my $xp = XML::XPath->new($xml); + + if ($xp->getNodeText("/domain/os/type/\@machine") ne 'q35') { + diag "Changing guest machine type to q35"; + $xp->setNodeText("/domain/os/type/\@machine", "q35"); + } + + my $loader_node = XML::XPath::Node::Element->new('loader'); + my $loader_text = XML::XPath::Node::Text->new($loader_path); + my $attr_ro = XML::XPath::Node::Attribute->new('readonly'); + my $attr_secure = XML::XPath::Node::Attribute->new('secure'); + my $attr_type = XML::XPath::Node::Attribute->new('type'); + + $attr_ro -> setNodeValue('yes'); + $attr_secure -> setNodeValue('yes'); + $attr_type -> setNodeValue('pflash'); + + $loader_node->appendChild($loader_text); + $loader_node->appendAttribute($attr_ro); + $loader_node->appendAttribute($attr_secure); + $loader_node->appendAttribute($attr_type); + + my $nvram_node = XML::XPath::Node::Element->new('nvram'); + my $nvram_text = XML::XPath::Node::Text->new($nvram_path); + my $attr_template = XML::XPath::Node::Attribute->new('template'); + + $attr_template -> setNodeValue($nvram_template); + + $nvram_node->appendChild($nvram_text); + $nvram_node->appendAttribute($attr_template); + + my $smm_node = XML::XPath::Node::Element->new('smm'); + my $attr_state = XML::XPath::Node::Attribute->new('state'); + $attr_state -> setNodeValue("on"); + $smm_node -> appendAttribute($attr_state); + + my ($root) = $xp->findnodes('/domain/os'); + $root->appendChild($loader_node); + $root->appendChild($nvram_node); + ($root) = $xp->findnodes('/domain/features'); + $root->appendChild($smm_node); + + $xml = $xp->findnodes_as_string('/'); + diag $xml; + return $xml; +} + +diag "Defining an inactive domain config with nvram"; +my $loader_file_path = '/usr/share/OVMF/OVMF_CODE.secboot.fd'; +my $nvram_file_template = '/usr/share/OVMF/OVMF_VARS.fd'; +my $nvram_file_path = '/var/lib/libvirt/qemu/nvram/test_VARS.fd'; + +my $xml = setup_nvram($loader_file_path, $nvram_file_template, $nvram_file_path); + +SKIP: { + diag "Require files ($loader_file_path, $nvram_file_template) for testing"; + skip "Please install OVMF and ensure necessary files exist", 5 if !defined($xml); + my $dom; + + diag "Test Sys::Virt::Domain::UNDEFINE_KEEP_NVRAM"; + ok_domain(sub { $dom = $conn->define_domain($xml) }, "defined domain with nvram configure"); + diag "Checking nvram file already exists"; + my $st = stat($nvram_file_path); + ok($st, "File '$nvram_file_path' exists as expected"); + $dom->undefine(Sys::Virt::Domain::UNDEFINE_KEEP_NVRAM); + diag "Checking nvram file still exists"; + $st = stat($nvram_file_path); + ok($st, "File '$nvram_file_path' still exists as expected"); + + diag "Test Sys::Virt::Domain::UNDEFINE_NVRAM"; + ok_domain(sub { $dom = $conn->define_domain($xml) }, "defined domain with nvram configure"); + $dom->undefine(Sys::Virt::Domain::UNDEFINE_NVRAM); + diag "Checking nvram file removed"; + $st = stat($nvram_file_path); + ok(!$st, "File '$nvram_file_path' is removed"); +} +ok_error(sub { $conn->get_domain_by_name("tck") }, "NO_DOMAIN error raised from missing domain", + Sys::Virt::Error::ERR_NO_DOMAIN); -- 2.18.1

2 1

[libvirt] [PATCH] remote_daemon: Log host boot time
by Michal Privoznik 19 Dec '19

19 Dec '19

This is not strictly needed, but it makes sure we initialize the @bootTime global variable. Thing is, in order to validate XATTRs and prune those set in some previous runs of the host, a timestamp is recorded in XATTRs. The host boot time was unique enough so it was chosen as the timestamp value. And to avoid querying and parsing /proc/uptime every time, the query function does that only once and stores the boot time in a global variable. However, the only time the query function is called is in a child process that does lock files and changes seclabels. So effectively, we are doing exactly what we wanted to prevent from happening. The fix is simple, call the query function whilst the daemon is initializing. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Since this will be used by security driver only, I was tempted to put this somewhere under src/security/, but especially because of split daemon I didn't. Placing this into remote_daemon.c makes sure all sub-daemons will have the variable initialized and won't suffer the problem. src/remote/remote_daemon.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/remote/remote_daemon.c b/src/remote/remote_daemon.c index b400b1dd10..5debb6ce97 100644 --- a/src/remote/remote_daemon.c +++ b/src/remote/remote_daemon.c @@ -57,6 +57,7 @@ #include "virgettext.h" #include "util/virnetdevopenvswitch.h" #include "virsystemd.h" +#include "virhostuptime.h" #include "driver.h" @@ -1020,6 +1021,7 @@ int main(int argc, char **argv) { bool implicit_conf = false; char *run_dir = NULL; mode_t old_umask; + unsigned long long bootTime; struct option opts[] = { { "verbose", no_argument, &verbose, 'v'}, @@ -1151,6 +1153,12 @@ int main(int argc, char **argv) { exit(EXIT_FAILURE); } + if (virHostGetBootTime(&bootTime) < 0) { + VIR_DEBUG("Unable to get host boot time"); + } else { + VIR_DEBUG("host boot time: %lld", bootTime); + } + daemonSetupNetDevOpenvswitch(config); if (daemonSetupAccessManager(config) < 0) { -- 2.24.1

3 2

[libvirt] [PATCH] cpu: add CLZERO CPUID support for AMD platforms (libvirt 4.5)
by Ani Sinha 19 Dec '19

19 Dec '19

Qemu commit e900135dcfb67 ("i386: Add CPUID bit for CLZERO and XSAVEERPTR") adds support for CLZERO CPUID bit. This commit extends support for this bit in libvirt. This patch is based off of libvirt-4.5 tree. Signed-off-by: Ani Sinha <ani.sinha(a)nutanix.com> --- src/cpu/cpu_map.xml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/cpu/cpu_map.xml b/src/cpu/cpu_map.xml index 088c820..c251cb7 100644 --- a/src/cpu/cpu_map.xml +++ b/src/cpu/cpu_map.xml @@ -431,6 +431,9 @@ <feature name='virt-ssbd'> <cpuid eax_in='0x80000008' ebx='0x02000000'/> </feature> + <feature name='clzero'> + <cpuid eax_in='0x80000008' ebx='0x00000001'/> + </feature>  <model name='486'> @@ -1685,6 +1688,7 @@ <feature name='bmi2'/> <feature name='clflush'/> <feature name='clflushopt'/> + <feature name='clzero'/> <feature name='cmov'/> <feature name='cr8legacy'/> <feature name='cx16'/> @@ -1756,6 +1760,7 @@ <feature name='bmi2'/> <feature name='clflush'/> <feature name='clflushopt'/> + <feature name='clzero'/> <feature name='cmov'/> <feature name='cr8legacy'/> <feature name='cx16'/> -- 1.9.4

2 3

[libvirt] [PATCH 1/1] qemu: hide details of fake reboot
by Nikolay Shirokovskiy 19 Dec '19

19 Dec '19

If we use fake reboot then domain goes thru running->shutdown->running state changes with shutdown state only for short period of time. At least this is implementation details leaking into API. And also there is one real case when this is not convinient. I'm doing a backup with the help of temporary block snapshot (with the help of qemu's API which is used in the newly created libvirt's backup API). If guest is shutdowned I want to continue to backup so I don't kill the process and domain is in shutdown state. Later when backup is finished I want to destroy qemu process. So I check if it is in shutdowned state and destroy it if it is. Now if instead of shutdown domain got fake reboot then I can destroy process in the middle of fake reboot process. After shutdown event we also get stop event and now as domain state is running it will be transitioned to paused state and back to running later. Though this is not critical for the described case I guess it is better not to leak these details to user too. So let's leave domain in running state on stop event if fake reboot is in process. As we don't know when stop event really arrive let's move resetting flag after starting CPUs - at this point stop event should be handled. Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy(a)virtuozzo.com> --- src/qemu/qemu_process.c | 57 ++++++++++++++++++++++------------------- 1 file changed, 31 insertions(+), 26 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index ed8666e9d1..2d37f92724 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -500,6 +500,8 @@ qemuProcessFakeReboot(void *opaque) goto endjob; } + qemuDomainSetFakeReboot(driver, vm, false); + if (virDomainSaveStatus(driver->xmlopt, cfg->stateDir, vm, driver->caps) < 0) { VIR_WARN("Unable to save status on vm %s after state change", vm->def->name); @@ -525,7 +527,6 @@ qemuProcessShutdownOrReboot(virQEMUDriverPtr driver, qemuDomainObjPrivatePtr priv = vm->privateData; if (priv->fakeReboot) { - qemuDomainSetFakeReboot(driver, vm, false); virObjectRef(vm); virThread th; if (virThreadCreate(&th, @@ -534,6 +535,7 @@ qemuProcessShutdownOrReboot(virQEMUDriverPtr driver, vm) < 0) { VIR_ERROR(_("Failed to create reboot thread, killing domain")); ignore_value(qemuProcessKill(vm, VIR_QEMU_PROCESS_KILL_NOWAIT)); + qemuDomainSetFakeReboot(driver, vm, false); virObjectUnref(vm); } } else { @@ -595,35 +597,37 @@ qemuProcessHandleShutdown(qemuMonitorPtr mon G_GNUC_UNUSED, goto unlock; } - VIR_DEBUG("Transitioned guest %s to shutdown state", - vm->def->name); - virDomainObjSetState(vm, - VIR_DOMAIN_SHUTDOWN, - VIR_DOMAIN_SHUTDOWN_UNKNOWN); + if (!priv->fakeReboot) { + VIR_DEBUG("Transitioned guest %s to shutdown state", + vm->def->name); + virDomainObjSetState(vm, + VIR_DOMAIN_SHUTDOWN, + VIR_DOMAIN_SHUTDOWN_UNKNOWN); - switch (guest_initiated) { - case VIR_TRISTATE_BOOL_YES: - detail = VIR_DOMAIN_EVENT_SHUTDOWN_GUEST; - break; + switch (guest_initiated) { + case VIR_TRISTATE_BOOL_YES: + detail = VIR_DOMAIN_EVENT_SHUTDOWN_GUEST; + break; - case VIR_TRISTATE_BOOL_NO: - detail = VIR_DOMAIN_EVENT_SHUTDOWN_HOST; - break; + case VIR_TRISTATE_BOOL_NO: + detail = VIR_DOMAIN_EVENT_SHUTDOWN_HOST; + break; - case VIR_TRISTATE_BOOL_ABSENT: - case VIR_TRISTATE_BOOL_LAST: - default: - detail = VIR_DOMAIN_EVENT_SHUTDOWN_FINISHED; - break; - } + case VIR_TRISTATE_BOOL_ABSENT: + case VIR_TRISTATE_BOOL_LAST: + default: + detail = VIR_DOMAIN_EVENT_SHUTDOWN_FINISHED; + break; + } - event = virDomainEventLifecycleNewFromObj(vm, - VIR_DOMAIN_EVENT_SHUTDOWN, - detail); + event = virDomainEventLifecycleNewFromObj(vm, + VIR_DOMAIN_EVENT_SHUTDOWN, + detail); - if (virDomainSaveStatus(driver->xmlopt, cfg->stateDir, vm, driver->caps) < 0) { - VIR_WARN("Unable to save status on vm %s after state change", - vm->def->name); + if (virDomainSaveStatus(driver->xmlopt, cfg->stateDir, vm, driver->caps) < 0) { + VIR_WARN("Unable to save status on vm %s after state change", + vm->def->name); + } } if (priv->agent) @@ -657,7 +661,8 @@ qemuProcessHandleStop(qemuMonitorPtr mon G_GNUC_UNUSED, reason = priv->pausedReason; priv->pausedReason = VIR_DOMAIN_PAUSED_UNKNOWN; - if (virDomainObjGetState(vm, NULL) == VIR_DOMAIN_RUNNING) { + if (virDomainObjGetState(vm, NULL) == VIR_DOMAIN_RUNNING && + !priv->fakeReboot) { if (priv->job.asyncJob == QEMU_ASYNC_JOB_MIGRATION_OUT) { if (priv->job.current->status == QEMU_DOMAIN_JOB_STATUS_POSTCOPY) reason = VIR_DOMAIN_PAUSED_POSTCOPY; -- 2.23.0

2 2

[libvirt] [PATCH 0/2] Remove VIR_FILE_*_SEPARATOR_*
by Fabiano Fidêncio 18 Dec '19

18 Dec '19

Let's use the defines provided by GLib, instead of keeping our own. Fabiano Fidêncio (2): util: Use G_DIR_SEPARATOR instead of VIR_FILE_DIR_SEPARATOR util: Remove VIR_FILE_*_SEPARATOR* src/util/virfile.c | 2 +- src/util/virfile.h | 19 ------------------- 2 files changed, 1 insertion(+), 20 deletions(-) -- 2.23.0

2 3