- Devel - libvirt List Archives

[libvirt] [PATCH 0/6] util: bitmap: Various fixes and improvements
by Peter Krempa 05 Feb '18

05 Feb '18

Peter Krempa (6): util: bitmap: Rename 'max_bit' to 'nbits' util: bitmap: Fix function formatting and spacing util: bitmap: Add comments for functions which don't have them util: bitmap: Fix value of 'map_alloc' when shrinking bitmap util: bitmap: Use VIR_SHRINK_N in virBitmapShrink util: bitmap: Note that shrinking the bitmap requires clearing of unused bits src/conf/domain_conf.c | 3 +- src/util/virbitmap.c | 213 +++++++++++++++++++++++++++++++++++-------------- src/util/virbitmap.h | 2 +- src/util/virresctrl.c | 3 +- tests/virbitmaptest.c | 6 +- 5 files changed, 157 insertions(+), 70 deletions(-) -- 2.15.0

2 7

[libvirt] [PATCH 0/6] port allocator: make used port bitmap global etc
by Nikolay Shirokovskiy 05 Feb '18

05 Feb '18

This patch set addresses issue described in [1] and the core of changes go to the first patch. The others are cleanups and refactorings. [1] https://www.redhat.com/archives/libvir-list/2017-December/msg00600.html Nikolay Shirokovskiy (6): port allocator: make used port bitmap global port allocator: remove range on manual port reserving port allocator: remove range check in release function port allocator: drop skip flag port allocator: remove release functionality from set used port allocator: make port range constant object src/bhyve/bhyve_command.c | 4 +- src/bhyve/bhyve_driver.c | 4 +- src/bhyve/bhyve_process.c | 7 +- src/bhyve/bhyve_utils.h | 2 +- src/libvirt_private.syms | 3 +- src/libxl/libxl_conf.c | 8 +-- src/libxl/libxl_conf.h | 12 ++-- src/libxl/libxl_domain.c | 3 +- src/libxl/libxl_driver.c | 17 +++-- src/libxl/libxl_migration.c | 4 +- src/qemu/qemu_conf.h | 12 ++-- src/qemu/qemu_driver.c | 27 ++++---- src/qemu/qemu_migration.c | 12 ++-- src/qemu/qemu_process.c | 55 +++++---------- src/util/virportallocator.c | 148 +++++++++++++++++++++++------------------ src/util/virportallocator.h | 24 +++---- tests/bhyvexml2argvtest.c | 5 +- tests/libxlxml2domconfigtest.c | 7 +- tests/virportallocatortest.c | 49 ++++++++------ 19 files changed, 196 insertions(+), 207 deletions(-) -- 1.8.3.1

4 23

[libvirt] [PATCH v3] virt-aa-helper: Skip feature support checks while parsing
by Shivaprasad G Bhat 05 Feb '18

05 Feb '18

virt-aa-helper need not verify the feature support as libvirt does it during domain creation anyway. Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com> --- src/conf/domain_conf.c | 3 +++ src/conf/domain_conf.h | 1 + src/security/virt-aa-helper.c | 6 +++++- 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index e827b2a81..92929a289 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -4736,6 +4736,9 @@ static int virDomainDefPostParseCheckFeatures(virDomainDefPtr def, virDomainXMLOptionPtr xmlopt) { + if (!UNSUPPORTED(VIR_DOMAIN_DEF_FEATURE_SKIP_VALIDATE)) + return 0; + if (UNSUPPORTED(VIR_DOMAIN_DEF_FEATURE_MEMORY_HOTPLUG) && virDomainDefCheckUnsupportedMemoryHotplug(def) < 0) return -1; diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 21e004515..c36d29fd9 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -2539,6 +2539,7 @@ typedef enum { VIR_DOMAIN_DEF_FEATURE_NAME_SLASH = (1 << 3), VIR_DOMAIN_DEF_FEATURE_INDIVIDUAL_VCPUS = (1 << 4), VIR_DOMAIN_DEF_FEATURE_USER_ALIAS = (1 << 5), + VIR_DOMAIN_DEF_FEATURE_SKIP_VALIDATE = (1 << 6), } virDomainDefFeatures; diff --git a/src/security/virt-aa-helper.c b/src/security/virt-aa-helper.c index f7ccae0b0..b15fea7eb 100644 --- a/src/security/virt-aa-helper.c +++ b/src/security/virt-aa-helper.c @@ -654,6 +654,9 @@ caps_mockup(vahControl * ctl, const char *xmlStr) return rc; } +virDomainDefParserConfig virAAHelperDomainDefParserConfig = { + .features = VIR_DOMAIN_DEF_FEATURE_SKIP_VALIDATE, +}; static int get_definition(vahControl * ctl, const char *xmlStr) @@ -673,7 +676,8 @@ get_definition(vahControl * ctl, const char *xmlStr) goto exit; } - if (!(ctl->xmlopt = virDomainXMLOptionNew(NULL, NULL, NULL, NULL, NULL))) { + if (!(ctl->xmlopt = virDomainXMLOptionNew(&virAAHelperDomainDefParserConfig, + NULL, NULL, NULL, NULL))) { vah_error(ctl, 0, _("Failed to create XML config object")); goto exit; }

2 1

[libvirt] [PATCH v2 0/2] Fix possible crash when resctrl gets remounted
by Martin Kletzander 05 Feb '18

05 Feb '18

First one just fixes the crash, but the second one makes sure we get consistent info with at least higher probability. Martin Kletzander (2): util: Check if kernel-provided info is consistent with itself qemu: Refresh capabilities when creating resctrl allocation src/qemu/qemu_process.c | 8 +++++++- src/util/virresctrl.c | 11 +++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) -- 2.16.1

2 8

[libvirt] PATCH add q35 support ide
by Paul Schlacter 04 Feb '18

04 Feb '18

hello everyone: In q35 motherboard use ide, Currently, the qemu has supported q35 Motherboard support ide bus diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index cc7596b..2dbade8 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -7188,6 +7188,7 @@ bool qemuDomainMachineHasBuiltinIDE(const char *machine) { return qemuDomainMachineIsI440FX(machine) || + qemuDomainMachineIsQ35(machine) || STREQ(machine, "malta") || STREQ(machine, "sun4u") || STREQ(machine, "g3beige"); [root@kvm ~]# virsh dumpxml instance-00000004 | grep machine= <type arch='x86_64' machine='pc-q35-rhel7.3.0'>hvm</type> [root@kvm~]# [root@kvm~]# virsh dumpxml instance-00000004 | grep "'disk'" -A 13 <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/lib/nova/instances/288271ce-69eb-4629-b98c- 779036661294/disk'/> <backingStore type='file' index='1'> <format type='raw'/> <source file='/var/lib/nova/instances/_base/ 8d383eef2e628adfc197a6e40e656916de566ab1'/> <backingStore/> </backingStore> <target dev='vda' bus='ide'/> <alias name='ide0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk>

3 3

[libvirt] CfP 13th Virtualization in High-Performance Cloud Computing Workshop (VHPC '18)
by VHPC 18 02 Feb '18

02 Feb '18

*====================================================================CALL FOR PAPERS 13th Workshop on Virtualization in High-Performance Cloud Computing (VHPC '18)held in conjunction with the International Supercomputing Conference - High Performance,June 24-28, 2018, Frankfurt, Germany.(Springer LNCS Proceedings) ====================================================================Date: June 28, 2018Workshop URL: http://vhpc.org <http://vhpc.org>Paper Submission Deadline: April 23, 2018, Springer LNCS, rolling abstract submissionAbstract/Paper Submission Link: https://edas.info/newPaper.php?c=24355 <https://edas.info/newPaper.php?c=24355>Special Track: GPU - Accelerator Virtualization Call for PapersVirtualization technologies constitute a key enabling factor for flexible resource managementin modern data centers, and particularly in cloud environments. Cloud providers need tomanage complex infrastructures in a seamless fashion to support the highly dynamic andheterogeneous workloads and hosted applications customers deploy. Similarly, HPCenvironments have been increasingly adopting techniques that enable flexible managementof vast computing and networking resources, close to marginal provisioning cost, which isunprecedented in the history of scientific and commercial computing.Various virtualization technologies contribute to the overall picture in different ways: machinevirtualization, with its capability to enable consolidation of multiple underutilized servers withheterogeneous software and operating systems (OSes), and its capability to live-migrate afully operating virtual machine (VM) with a very short downtime, enables novel and dynamicways to manage physical servers; OS-level virtualization (i.e., containerization), with itscapability to isolate multiple user-space environments and to allow for their coexistencewithin the same OS kernel, promises to provide many of the advantages of machine virtualization with high levels of responsiveness and performance; I/O Virtualization allows physical network interfaces to take traffic from multiple VMs or containers; network virtualization, with its capability to create logical network overlays that are independent of theunderlying physical topology is furthermore enabling virtualization of HPC infrastructures. PublicationAccepted papers will be published in a Springer LNCS proceedings volume.Topics of InterestThe VHPC program committee solicits original, high-quality submissions related tovirtualization across the entire software stack with a special focus on the intersection of HPCand the cloud.Major Topics- Virtualization in supercomputing environments, HPC clusters, HPC in the cloud and grids- OS-level virtualization and containers (LXC, Docker, rkt, Singularity, Shifter, i.a.)- Lightweight/specialized operating systems in conjunction with virtual machines- Novel unikernels and use cases for virtualized HPC environments- Performance improvements for or driven by unikernels- Tool support for unikernels: configuration/build environments, debuggers, profilers- Hypervisor extensions to mitigate side-channel attacks ([micro-]architectural timing attacks, privilege escalation)- VM & Container trust and security- Containers inside VMs with hypervisor isolation- GPU virtualization operationalization- Approaches to GPGPU virtualization including API remoting and hypervisor abstraction- Optimizations of virtual machine monitor platforms and hypervisors- Hypervisor support for heterogeneous resources (GPUs, co-processors, FPGAs, etc.)- Virtualization support for emerging memory technologies- Virtualization in enterprise HPC and microvisors- Software defined networks and network virtualization- Management, deployment of virtualized environments and orchestration (Kubernetes i.a.)- Workflow-pipeline container-based composability - Checkpointing facilitation utilizing containers and VMs - Emerging topics including multi-kernel approaches and NUMA in hypervisors- Operating MPI in containers/VMs and Unikernels - Virtualization in data intensive computing (big data) - HPC convergence- Adaptation of HPC technologies in the cloud (high performance networks, RDMA, etc.)- Performance measurement, modelling and monitoring of virtualized/cloud workloads- Latency-and jitter sensitive workloads in virtualized/containerized environments- I/O virtualization (including applications, SR-IOV, i.a.) - Hybrid local facility + cloud compute and based storage systems, cloudbursting- FPGA and many-core accelerator virtualization- Job scheduling/control/policy and container placement in virtualized environments- Cloud reliability, fault-tolerance and high-availability- QoS and SLA in virtualized environments- IaaS platforms, cloud frameworks and APIs- Energy-efficient and power-aware virtualization- Configuration management tools for containers (including in OpenStack, Ansible, i.a.)- ARM-based hypervisors, ARM virtualization extensionsSpecial Track: GPU - Accelerator VirtualizationGPU virtualization technologies, performance and benchmarking, integration withworkflow scheduling systems, integration to cluster managers.GPUs are taking on many HPC workload areas, especially in deep learning withinmachine learning. In addition, a lot of workload is being pushed to elastic environments utilizing various virtualization technologies on different levels like hypervisors (e.g. VMWare, Xen, KVM), kernel (Docker, Kubernetes) or on the resource managerlevel (YARN, Mesos). In this track we invite submissions addressing these problems. Suggested Themes and Topics:Technology - What technologies and best practices exist for GPU - hardware accelerator virtualization and usage of hardware accelerators in virtual environments on the hypervisor, kernel or resource manager levelDevelopers - Real-life experience when addressing HPC/ML/DL problems with GPUs or hardware accelerators in virtual environmentsPerformance - Performance comparisons between different technologies / solutionsThe Workshop on Virtualization in High-Performance Cloud Computing (VHPC) aims tobring together researchers and industrial practitioners facing the challenges posed by virtualization in order to foster discussion, collaboration, mutual exchangeof knowledge and experience, enabling research to ultimately provide novelsolutions for virtualized computing systems of tomorrow.The workshop will be one day in length, composed of 20 min paper presentations, eachfollowed by 10 min discussion sections, plus lightning talks that are limited to 5 minutes.Presentations may be accompanied by interactive demonstrations.Important DatesFebruary 23, 2018 (AoE) - Abstract SubmissionApril 23, 2018 (AoE) - Paper submission deadline (Springer LNCS)May 30, 2018 - Acceptance notification June 28, 2018 - Workshop DayJuly 12, 2018 - Camera-ready version dueChairMichael Alexander (chair), Institute of Science and Technology, AustriaAnastassios Nanos (co-chair), OnApp, UKRomeo Kienzler (co-chair), IBM, SwitzerlandProgram committeeStergios Anastasiadis, University of Ioannina, Greece Jakob Blomer, CERN, Europe Eduardo César, Universidad Autonoma de Barcelona, SpainStephen Crago, USC ISI, USATommaso Cucinotta, St. Anna School of Advanced Studies, ItalyChristoffer Dall, Columbia University, USAFrançois Diakhaté, CEA, FrancePatrick Dreher, MIT, USA Kyle Hale, Northwestern University, USA Brian Kocoloski, University of Pittsburgh, USAUday Kurkure, VMware, USAJohn Lange, University of Pittsburgh, USAGiuseppe Lettieri, University of Pisa, ItalyQing Liu, Oak Ridge National Laboratory, USANikos Parlavantzas, IRISA, FranceKevin Pedretti, Sandia National Laboratories, USAAmer Qouneh, Western New England University, USA Carlos Reaño, Technical University of Valencia, SpainBorja Sotomayor, University of Chicago, USA Anata Tiwari, San Diego Supercomputer Center, USAKurt Tutschku, Blekinge Institute of Technology, Sweden Yasuhiro Watashiba, Osaka University, JapanChao-Tung Yang, Tunghai University, Taiwan Andrew Younge, Sandia National Laboratory, USANa Zhang, VMware, USAPaper Submission-PublicationPapers submitted to the workshop will be reviewed by at least twomembers of the program committee and external reviewers. Submissionsshould include abstract, keywords, the e-mail address of thecorresponding author, and must not exceed 10 pages, including tablesand figures at a main font size no smaller than 11 point. Submissionof a paper should be regarded as a commitment that, should the paperbe accepted, at least one of the authors will register and attend theconference to present the work. Accepted papers will be published in aSpringer LNCS volume. . The format must be according to the Springer LNCS Style. Initialsubmissions are in PDF; authors of accepted papers will be requestedto provide source files.Format Guidelines:ftp://ftp.springer.de/pub/tex/latex/llncs/latex2e/llncs2e.zip <ftp://ftp.springer.de/pub/tex/latex/llncs/latex2e/llncs2e.zip>Abstract, Paper Submission Link:https://edas.info/newPaper.php?c=24355 <https://edas.info/newPaper.php?c=24355>Lightning Talks Lightning Talks are non-paper track, synoptical in nature and are strictly limited to 5 minutes.They can be used to gain early feedback on ongoing research, for demonstrations, to present research results, early research ideas, perspectives and positions of interest to the community. Submit abstract via the main submission link. General InformationThe workshop is one day in length and will be held in conjunction with the InternationalSupercomputing Conference - High Performance (ISC) 2018, June 18-22, Frankfurt, Germany.*

1 0

[libvirt] [PATCH v4 00/12] Implement query-dump command
by John Ferlan 02 Feb '18

02 Feb '18

v3: https://www.redhat.com/archives/libvir-list/2018-January/msg01047.html Changes since v3: * Reorder patches, moved the "setup" patches to the first few patches and the meat-n-potatoes starting at patch 4 which is "new"-ish having been extracted from other patches to set up the buffers that the event will need. * Rather than allocating a dump buffer, use the stack variables and the jobinfo stats buffer. "Force tested" the failure to parse the stats error just to be sure it got propagated. John Ferlan (12): qemu: Introduce qemuDomainGetJobInfoMigrationStats qemu: Convert jobInfo stats into a union qemu: Introduce QEMU_DOMAIN_JOB_STATS_TYPE_SAVEDUMP qemu: Introduce QEMU_DOMAIN_JOB_STATS_TYPE_MEMDUMP qemu: Add support for DUMP_COMPLETED event qemu: Introduce qemuProcessHandleDumpCompleted qemu: Introduce qemuMonitor[JSON]QueryDump qemu: Introduce qemuDomainGetJobInfoDumpStats qemu: Add dump completed event to the capabilities qemu: Add new parameter to qemuMonitorDumpToFd qemu: Allow showing the dump progress for memory only dump docs: Add news article for query memory-only dump processing percentage docs/news.xml | 11 ++ src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_domain.c | 127 ++++++++++++-- src/qemu/qemu_domain.h | 17 +- src/qemu/qemu_driver.c | 183 ++++++++++++++++++--- src/qemu/qemu_migration.c | 13 +- src/qemu/qemu_migration_cookie.c | 4 +- src/qemu/qemu_monitor.c | 38 ++++- src/qemu/qemu_monitor.h | 37 ++++- src/qemu/qemu_monitor_json.c | 103 +++++++++++- src/qemu/qemu_monitor_json.h | 6 +- src/qemu/qemu_process.c | 42 ++++- .../caps_2.10.0-gicv2.aarch64.xml | 1 + .../caps_2.10.0-gicv3.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml | 1 + .../caps_2.6.0-gicv2.aarch64.xml | 1 + .../caps_2.6.0-gicv3.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.x86_64.xml | 1 + tests/qemumonitorjsontest.c | 3 +- 30 files changed, 548 insertions(+), 55 deletions(-) -- 2.13.6

2 26

[libvirt] [PATCH] util: Remove unnecessary initialization
by John Ferlan 02 Feb '18

02 Feb '18

VIR_ALLOC will already initialize, so no need to do it again. Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- This is something Coverity noted a while ago that I found sitting in one of my branches that I'd forgotten about. Pushed as trivial. src/util/virsysinfo.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/util/virsysinfo.c b/src/util/virsysinfo.c index 855580d44..e8d337191 100644 --- a/src/util/virsysinfo.c +++ b/src/util/virsysinfo.c @@ -597,8 +597,6 @@ virSysinfoReadS390(void) goto no_memory; } - ret->nprocessor = 0; - ret->processor = NULL; if (virSysinfoParseS390Processor(outbuf, ret) < 0) goto no_memory; -- 2.13.6

1 0

[libvirt] [PATCH] qemu: Refresh capabilities when creating resctrl allocation
by Martin Kletzander 02 Feb '18

02 Feb '18

Since one of the things in capabilities (info from resctrl updated with data about caches) can be change on the system by remounting the /sys/fs/resctrl with different options, the capabilities need to be refreshed. There is a better fix in the works, but it's going to be way bigger than this (hence the XXX note there), so for the time being let's workaround this. And in order not to slow down the domain starting, only get the capabilities if there are any cachetunes. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1540780 Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- src/qemu/qemu_process.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index bcd4ac8ad699..6cdb9ebc7fce 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -2514,9 +2514,15 @@ qemuProcessResctrlCreate(virQEMUDriverPtr driver, { int ret = -1; size_t i = 0; - virCapsPtr caps = virQEMUDriverGetCapabilities(driver, false); + virCapsPtr caps = NULL; qemuDomainObjPrivatePtr priv = vm->privateData; + if (!vm->def->ncachetunes) + return 0; + + /* Force capability refresh since resctrl info can change + * XXX: move cache info into virresctrl so caps are not needed */ + caps = virQEMUDriverGetCapabilities(driver, true); if (!caps) return -1; -- 2.16.1

2 3

[libvirt] [PATCH 0/3] Misc util/ fixes plus tests
by Martin Kletzander 02 Feb '18

02 Feb '18

Yay!!! \o/ Patches! Martin Kletzander (3): util: Fix possible leak in virResctrlAllocMasksAssign util: Clear unused part of the map in virBitmapShrink tests: Add test for properly removing cachetune entries src/util/virbitmap.c | 13 +++++++++ src/util/virresctrl.c | 4 +-- .../genericxml2xmlindata/cachetune-extra-tunes.xml | 33 ++++++++++++++++++++++ .../cachetune-extra-tunes.xml | 30 ++++++++++++++++++++ tests/genericxml2xmltest.c | 1 + tests/virbitmaptest.c | 5 ++++ 6 files changed, 84 insertions(+), 2 deletions(-) create mode 100644 tests/genericxml2xmlindata/cachetune-extra-tunes.xml create mode 100644 tests/genericxml2xmloutdata/cachetune-extra-tunes.xml -- 2.16.1

2 14

[libvirt] [PATCH v2 00/15] Misc build refactoring / isolation work
by Daniel P. Berrangé 02 Feb '18

02 Feb '18

This was triggered by the recent Fedora change to add '-z defs' to RPM builds by default which breaks libvirt. Various make rule changes can fix much of the problem, but it also requires source refactoring to get rid of places where virt drivers directly call into the storage/network drivers. Co-incidentally this work will also be useful in allowing us to separate out drivers to distinct daemons. In v2: - Fixed header file name comment - Resolve conflicts - Fix unit tests - Fix bisectable build by moving libvirt_lxc build patch earlier - Update syntax check header include rule Daniel P. Berrangé (15): storage: extract storage file backend from main storage driver backend storage: move storage file backend framework into util directory rpc: don't link in second copy of RPC code to libvirtd & lockd plugin build: link libvirt_lxc against libvirt.so conf: introduce callback registration for domain net device allocation conf: expand network device callbacks to cover bandwidth updates qemu: replace networkGetNetworkAddress with public API calls conf: expand network device callbacks to cover resolving NIC type network: remove conditional declarations conf: move virStorageTranslateDiskSourcePool into domain conf storage: export virStoragePoolLookupByTargetPath as a public API build: explicitly link all modules with libvirt.so build: provide a AM_FLAGS_MOD for loadable modules build: passing the "-z defs" linker flag to prevent undefined symbols cfg: forbid includes of headers in network and storage drivers again cfg.mk | 2 +- configure.ac | 1 + daemon/Makefile.am | 3 +- include/libvirt/libvirt-storage.h | 2 + m4/virt-linker-no-undefined.m4 | 32 ++ po/POTFILES.in | 2 +- src/Makefile.am | 142 ++++---- src/bhyve/bhyve_command.c | 7 +- src/conf/domain_conf.c | 355 +++++++++++++++++++ src/conf/domain_conf.h | 71 ++++ src/driver-storage.h | 5 + src/libvirt-storage.c | 40 +++ src/libvirt_private.syms | 29 ++ src/libvirt_public.syms | 6 + src/libvirt_remote.syms | 11 +- src/libxl/libxl_domain.c | 5 +- src/libxl/libxl_driver.c | 7 +- src/lxc/lxc_driver.c | 5 +- src/lxc/lxc_process.c | 7 +- src/network/bridge_driver.c | 144 ++------ src/network/bridge_driver.h | 72 ---- src/qemu/qemu_alias.c | 3 +- src/qemu/qemu_command.c | 1 - src/qemu/qemu_domain.c | 3 - src/qemu/qemu_domain_address.c | 3 +- src/qemu/qemu_driver.c | 15 +- src/qemu/qemu_hotplug.c | 18 +- src/qemu/qemu_migration.c | 3 +- src/qemu/qemu_process.c | 115 +++++- src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 17 +- src/remote_protocol-structs | 7 + src/security/virt-aa-helper.c | 2 - src/storage/storage_backend.c | 66 ---- src/storage/storage_backend.h | 75 ---- src/storage/storage_backend_fs.c | 8 +- src/storage/storage_backend_gluster.c | 4 +- src/storage/storage_driver.c | 256 +------------- src/storage/storage_driver.h | 3 - src/storage/storage_source.c | 645 ---------------------------------- src/storage/storage_source.h | 59 ---- src/util/virstoragefile.c | 609 +++++++++++++++++++++++++++++++- src/util/virstoragefile.h | 32 ++ src/util/virstoragefilebackend.c | 108 ++++++ src/util/virstoragefilebackend.h | 104 ++++++ src/vz/vz_sdk.c | 1 - tests/Makefile.am | 4 +- tests/qemuxml2argvtest.c | 4 + tests/virstoragetest.c | 1 - tools/Makefile.am | 1 + 50 files changed, 1675 insertions(+), 1441 deletions(-) create mode 100644 m4/virt-linker-no-undefined.m4 delete mode 100644 src/storage/storage_source.c delete mode 100644 src/storage/storage_source.h create mode 100644 src/util/virstoragefilebackend.c create mode 100644 src/util/virstoragefilebackend.h -- 2.14.3

2 17

[libvirt] [PATCH] qemuDomainRemoveMemoryDevice: unlink() memory backing file
by Michal Privoznik 02 Feb '18

02 Feb '18

https://bugzilla.redhat.com/show_bug.cgi?id=1461214 Since fec8f9c49af we try to use predictable file names for 'memory-backend-file' objects. But that made us provide full path to qemu when hot plugging the object while previously we provided merely a directory. But this makes qemu behave differently. If qemu sees a path terminated with a directory it calls mkstemp() and unlinks the file immediately. But if it sees full path it just calls open(path, O_CREAT ..); and never unlinks the file. Therefore it's up to libvirt to unlink the file and not leave it behind. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Zack, can you please check if this patch is suitable for your use cases? src/qemu/qemu_hotplug.c | 3 +++ src/qemu/qemu_process.c | 26 ++++++++++++++++++++++++++ src/qemu/qemu_process.h | 4 ++++ 3 files changed, 33 insertions(+) diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 6dc16a105..f26e2ca60 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -3894,6 +3894,9 @@ qemuDomainRemoveMemoryDevice(virQEMUDriverPtr driver, if (qemuDomainNamespaceTeardownMemory(vm, mem) < 0) VIR_WARN("Unable to remove memory device from /dev"); + if (qemuProcessDestroyMemoryBackingPath(driver, vm, mem) < 0) + VIR_WARN("Unable to destroy memory backing path"); + virDomainMemoryDefFree(mem); /* fix the balloon size */ diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 1a0923af3..73624eefe 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -3467,6 +3467,32 @@ qemuProcessBuildDestroyMemoryPaths(virQEMUDriverPtr driver, } +int +qemuProcessDestroyMemoryBackingPath(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryDefPtr mem) +{ + virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver); + char *path = NULL; + int ret = -1; + + if (qemuGetMemoryBackingPath(vm->def, cfg, mem->info.alias, &path) < 0) + goto cleanup; + + if (unlink(path) < 0 && + errno != ENOENT) { + virReportSystemError(errno, _("Unable to remove %s"), path); + goto cleanup; + } + + ret = 0; + cleanup: + VIR_FREE(path); + virObjectUnref(cfg); + return ret; +} + + static int qemuProcessVNCAllocatePorts(virQEMUDriverPtr driver, virDomainGraphicsDefPtr graphics, diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index cd9a72031..3fc7d6c85 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -43,6 +43,10 @@ int qemuProcessBuildDestroyMemoryPaths(virQEMUDriverPtr driver, virDomainMemoryDefPtr mem, bool build); +int qemuProcessDestroyMemoryBackingPath(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryDefPtr mem); + void qemuProcessAutostartAll(virQEMUDriverPtr driver); void qemuProcessReconnectAll(virConnectPtr conn, virQEMUDriverPtr driver); -- 2.13.6

5 8

[libvirt] [PATCH] qemu: migration: Refresh device information after transferring state
by Peter Krempa 02 Feb '18

02 Feb '18

In my first approach in 4b480d10768c I overlooked the comment in qemuMigrationRunIncoming stating that during actual migration the qemuMigrationRunIncoming does not wait until the migration is complete but rather offloads that to the Finish phase of migration. This means that during actual migration qemuProcessRefreshState was called prior to qemu actually transferring the full state and thus the queries did not get the correct information. The approach worked only for restore, where we wait for the migration to finish during qemu startup. Fix the issue by calling qemuProcessRefreshState both from qemuProcessStart if there's no incomming migration and from qemuMigrationFinish so that the code actually works as expected. --- src/qemu/qemu_migration.c | 9 +++++++++ src/qemu/qemu_process.c | 17 ++++++++++------- src/qemu/qemu_process.h | 4 ++++ 3 files changed, 23 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 1854900c9a..ea8b275601 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -5373,6 +5373,15 @@ qemuMigrationFinish(virQEMUDriverPtr driver, goto endjob; } + /* Now that the state data was transferred we can refresh the actual state + * of the devices */ + if (qemuProcessRefreshState(driver, vm, QEMU_ASYNC_JOB_MIGRATION_IN) < 0) { + /* Similarly to the case above v2 protocol will not be able to recover + * from this. Let's ignore this and perhaps stuff will not break. */ + if (v3proto) + goto endjob; + } + if (priv->job.current->status == QEMU_DOMAIN_JOB_STATUS_POSTCOPY) inPostCopy = true; diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index bcd4ac8ad6..fe12d94d37 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -6084,7 +6084,7 @@ qemuProcessLaunch(virConnectPtr conn, * function is called after a deferred migration finishes so that we can update * state influenced by the migration stream. */ -static int +int qemuProcessRefreshState(virQEMUDriverPtr driver, virDomainObjPtr vm, qemuDomainAsyncJob asyncJob) @@ -6125,9 +6125,6 @@ qemuProcessFinishStartup(virConnectPtr conn, virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver); int ret = -1; - if (qemuProcessRefreshState(driver, vm, asyncJob) < 0) - goto cleanup; - if (startCPUs) { VIR_DEBUG("Starting domain CPUs"); if (qemuProcessStartCPUs(driver, vm, conn, @@ -6231,11 +6228,17 @@ qemuProcessStart(virConnectPtr conn, VIR_DOMAIN_PAUSED_USER) < 0) goto stop; - /* Keep watching qemu log for errors during incoming migration, otherwise - * unset reporting errors from qemu log. */ - if (!incoming) + if (!incoming) { + /* Keep watching qemu log for errors during incoming migration, otherwise + * unset reporting errors from qemu log. */ qemuMonitorSetDomainLog(priv->mon, NULL, NULL, NULL); + /* Refresh state of devices from qemu. During migration this needs to + * happen after the state information is fully transferred. */ + if (qemuProcessRefreshState(driver, vm, asyncJob) < 0) + goto stop; + } + ret = 0; cleanup: diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index cd9a720313..b383ff309b 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -129,6 +129,10 @@ int qemuProcessFinishStartup(virConnectPtr conn, bool startCPUs, virDomainPausedReason pausedReason); +int qemuProcessRefreshState(virQEMUDriverPtr driver, + virDomainObjPtr vm, + qemuDomainAsyncJob asyncJob); + typedef enum { VIR_QEMU_PROCESS_STOP_MIGRATED = 1 << 0, VIR_QEMU_PROCESS_STOP_NO_RELABEL = 1 << 1, -- 2.15.0

2 1

[libvirt] [PATCH v3 00/11] Implement query-dump command
by John Ferlan 02 Feb '18

02 Feb '18

v2: https://www.redhat.com/archives/libvir-list/2018-January/msg00636.html Summary of changes since v2: * Generate a dump stats extraction helper in order to share with the DUMP_COMPLETED event and the query-dump command. Additionally add the error string from Qemu to be processed later * When processing DUMP_COMPLETED event implementation, use the returned stats buffer and copy any error message for the waiting completion to process properly. NB: Was able to test this by altering the guest memory size larger than the space to store the dump memory and got the following error: error: operation failed: memory-only dump failed: dump: failed to save memory * As suggested during review - alter the jobInfo @stats to be part of a union. Took the liberty to rename the fields as well. * The point raised in patch 5 regarding mirrorStats being collect for a non-migration (e.g. save and dump) was handled by creating a new type which will use the migStats only to collect data and avoid the mirrorStats. When converting to a JobInfo, only the migStats will then be used. * Patches 6 and 7 are new. One for the union and one to separate migrate and save/dump style migration jobs. * Former patch 6 (now 8) is altered to handle the union separation * Former patch 8 (now 10) is altered mainly to handle the data in buffers from the DUMP_COMPLETED event (either data or error). NB: Comment from former patch 8: "We should update job.completed at this point." I believe that's handled because qemuDomainObjResetAsyncJob will reset the dumpCompleted back to false. Unless there's a specific reason to do it there... NB: Formerly ACK'd patches 4 and 7 (now 9) did not change. John Ferlan (11): qemu: Add support for DUMP_COMPLETED event qemu: Introduce qemuProcessHandleDumpCompleted qemu: Introduce qemuMonitor[JSON]QueryDump qemu: Add new parameter to qemuMonitorDumpToFd qemu: Introduce qemuDomainGetJobInfoMigrationStats qemu: Convert jobInfo stats into a union qemu: Introduce QEMU_DOMAIN_JOB_STATS_TYPE_SAVEDUMP qemu: Introduce qemuDomainGetJobInfoDumpStats qemu: Add dump completed event to the capabilities qemu: Allow showing the dump progress for memory only dump docs: Add news article for query memory-only dump processing percentage docs/news.xml | 11 ++ src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_domain.c | 130 +++++++++++++-- src/qemu/qemu_domain.h | 18 +- src/qemu/qemu_driver.c | 183 ++++++++++++++++++--- src/qemu/qemu_migration.c | 13 +- src/qemu/qemu_migration_cookie.c | 4 +- src/qemu/qemu_monitor.c | 45 ++++- src/qemu/qemu_monitor.h | 36 +++- src/qemu/qemu_monitor_json.c | 106 +++++++++++- src/qemu/qemu_monitor_json.h | 6 +- src/qemu/qemu_process.c | 34 +++- .../caps_2.10.0-gicv2.aarch64.xml | 1 + .../caps_2.10.0-gicv3.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml | 1 + .../caps_2.6.0-gicv2.aarch64.xml | 1 + .../caps_2.6.0-gicv3.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.x86_64.xml | 1 + tests/qemumonitorjsontest.c | 3 +- 30 files changed, 553 insertions(+), 55 deletions(-) -- 2.13.6

2 17

[libvirt] [PATCH v3 0/5] Introduce VIR_ERR_DEVICE_MISSING
by Chen Hanxiao 01 Feb '18

01 Feb '18

Use Introduce VIR_ERR_DEVICE_MISSING for hotplug and detach device error message. Chen Hanxiao (5): qemu: Add some more details for hotplug errors when device not found qemu: Introduce VIR_ERR_DEVICE_MISSING qemu: Use VIR_ERR_DEVICE_MISSING for various hotplug messages qemu: Use VIR_ERR_DEVICE_MISSING for various DetachDeviceConfig messages news: Add VIR_ERR_DEVICE_MISSING change as improvements docs/news.xml | 5 +++++ include/libvirt/virterror.h | 1 + src/conf/domain_conf.c | 8 ++++---- src/libvirt_private.syms | 2 ++ src/qemu/qemu_driver.c | 20 +++++++++---------- src/qemu/qemu_hotplug.c | 47 ++++++++++++++++++++++++++++----------------- src/util/virerror.c | 6 ++++++ 7 files changed, 57 insertions(+), 32 deletions(-) -- 2.14.3

2 13

[libvirt] [PATCH 0/3] Misc RPC fixes
by Daniel P. Berrangé 01 Feb '18

01 Feb '18

Misc fixes pointed out by John after coverity had a fit :-) Daniel P. Berrangé (3): tests: validate private data / pre / post exec hooks for RPC APIs rpc: assume private data callbacks are always non-NULL rpc: fix non-NULL annotations when GNUTLS is disabled src/rpc/virnetserverclient.c | 47 +++++++------- src/rpc/virnetserverclient.h | 4 ++ .../virnetdaemondata/input-data-admin-nomdns.json | 12 ++-- .../input-data-admin-server-names.json | 12 ++-- .../virnetdaemondata/input-data-anon-clients.json | 6 +- .../input-data-client-auth-pending-failure.json | 3 +- .../input-data-client-auth-pending.json | 6 +- tests/virnetdaemondata/input-data-client-ids.json | 6 +- .../input-data-client-timestamp.json | 6 +- .../input-data-initial-nomdns.json | 6 +- tests/virnetdaemondata/input-data-initial.json | 6 +- .../input-data-no-keepalive-required.json | 12 ++-- .../virnetdaemondata/output-data-admin-nomdns.json | 12 ++-- .../output-data-admin-server-names.json | 12 ++-- .../virnetdaemondata/output-data-anon-clients.json | 6 +- .../output-data-client-auth-pending.json | 6 +- tests/virnetdaemondata/output-data-client-ids.json | 6 +- .../output-data-client-timestamp.json | 6 +- .../output-data-initial-nomdns.json | 6 +- tests/virnetdaemondata/output-data-initial.json | 6 +- .../output-data-no-keepalive-required.json | 12 ++-- tests/virnetdaemontest.c | 71 +++++++++++++++++++--- tests/virnetserverclienttest.c | 22 ++++++- 23 files changed, 208 insertions(+), 83 deletions(-) -- 2.14.3

2 6

[libvirt] [PATCH] qemu: Add USB and memory balloon by default for aarch64/virt guests
by Andrea Bolognani 01 Feb '18

01 Feb '18

Basically all existing guest types, regardless of the architectur, get both a USB controller and a virtio memory balloon by default. s390 guests are an exception, for the very good reason that they don't support USB at all; the other exception is aarch64/virt guests, but in the latter case isn't a compelling reason for them to deviate from the widely adopted convention, especially since: * x86/q35 guests, which aarch64/virt guests are for the most part identical to, add these devices by default; * it's trivial to opt out of both default devices by setting model='none'; * higher level applications such as Nova expect at least the USB controller to be present. So add it by default for newly-defined guests. Existing guests will, of course, be left unchanged. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1538637 Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- src/qemu/qemu_domain.c | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 6b4bd3cca..f1c3b3d1e 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -2630,7 +2630,8 @@ qemuDomainDefAddImplicitInputDevice(virDomainDef *def) static int qemuDomainDefAddDefaultDevices(virDomainDefPtr def, - virQEMUCapsPtr qemuCaps) + virQEMUCapsPtr qemuCaps, + unsigned int parseFlags) { bool addDefaultUSB = true; int usbModel = -1; /* "default for machinetype" */ @@ -2680,10 +2681,33 @@ qemuDomainDefAddDefaultDevices(virDomainDefPtr def, case VIR_ARCH_ARMV7L: case VIR_ARCH_AARCH64: - addDefaultUSB = false; - addDefaultMemballoon = false; - if (qemuDomainIsVirt(def)) + if (qemuDomainIsVirt(def)) { + /* All mach-virt guests get a PCIe Root, if supported by + * the QEMU binary */ addPCIeRoot = virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_GPEX); + } + + if (qemuDomainIsVirt(def) && + parseFlags & VIR_DOMAIN_DEF_PARSE_ABI_UPDATE) { + /* In addition to PCIe Root, newly-defined mach-virt guests + * also get a couple more devices so that they're more similar + * to guests on other architectures, notably x86/q35: + * + * 1) a USB3 controller, if supported by the QEMU binary; + * 2) a virtio memory balloon, as per the defaults defined + * above. + */ + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_QEMU_XHCI)) + usbModel = VIR_DOMAIN_CONTROLLER_MODEL_USB_QEMU_XHCI; + else + addDefaultUSB = false; + } else { + /* Other ARM guests (and existing mach-virt guests, in order + * to preserve guest ABI compatibility) don't get a PCIe Root, + * a USB controller or a memory balloon */ + addDefaultUSB = false; + addDefaultMemballoon = false; + } break; case VIR_ARCH_PPC64: @@ -3187,7 +3211,7 @@ qemuDomainDefPostParse(virDomainDefPtr def, goto cleanup; } - if (qemuDomainDefAddDefaultDevices(def, qemuCaps) < 0) + if (qemuDomainDefAddDefaultDevices(def, qemuCaps, parseFlags) < 0) goto cleanup; if (qemuCanonicalizeMachine(def, qemuCaps) < 0) -- 2.14.3

4 11

[libvirt] [PATCH v6 00/14] Move qemu command line controller checks to qemuDomainDeviceDefValidateController* checks
by John Ferlan 01 Feb '18

01 Feb '18

v5: https://www.redhat.com/archives/libvir-list/2018-January/msg00188.html Although each was ACK'd in v5, that was before alter the post parse callbacks to ensure SCSI controller model's were filled in. I post again mainly because some things do change slightly and well for sanity and being "correct". Unless there's an objection, I'll push some time on Friday. Andrea Bolognani (1): qemu: Add missing checks for pcie-root-port options John Ferlan (12): qemu: Introduce qemuDomainDeviceDefValidateControllerAttributes qemu: Move and rename qemuBuildCheckSCSIControllerModel qemu: Adjust SCSI controller switch in qemuBuildControllerDevStr qemu: Add check for iothread attribute in validate controller qemu: Introduce qemuDomainDeviceDefValidateControllerSCSI qemu: Introduce qemuDomainDeviceDefValidateControllerPCI qemu: Use virDomainPCIControllerOpts in qemuBuildControllerDevStr qemu: Move PCI command modelName check to controller def validate qemu: Move PCI command modelName TypeToString to controller def validate qemu: Move PCI more command checks to controller def validate qemu: Complete PCI command checks to controller def validate qemu: Introduce qemuDomainDeviceDefValidateControllerSATA Marc Hartmayer (1): qemu: Use switch statement for address types in qemuBuildControllerDevStr src/qemu/qemu_command.c | 505 +++++------------------------------------------- src/qemu/qemu_domain.c | 478 ++++++++++++++++++++++++++++++++++++++++++++- tests/qemumemlocktest.c | 14 ++ tests/qemuxml2xmltest.c | 5 +- 4 files changed, 545 insertions(+), 457 deletions(-) -- 2.13.6

2 16

[libvirt] [PATCH v2 00/11] Add admin protocol support for virtlogd/virtlockd
by Daniel P. Berrange 01 Feb '18

01 Feb '18

The initial admin protocol support was only integrated into libvirtd. This series extracts that code so that it is reusable with all the daemons we have (and more than we'll get). In v2: - Fixed completely broken post-exec restart support for admin servers - Misc fixes from v1 review Daniel P. Berrange (11): admin: move admins server impl/dispatch into src/admin directory libvirtd: rename virNetServerClient callback impls to match type names admin: add support for post-exec restart callbacks util: add virGetUNIXSocketPath helper rpc: clarify "void *" values passed to client callbacks rpc: pass virNetServer to post-exec restart callback in typesafe manner rpc: annotate various parameters as being required to be non-NULL rpc: add method for checking if a named server exists rpc: refactor virNetServer setup for post-exec restarts logd: add support for admin protocol in virtlogd lockd: add support for admin protocol in virtlockd .gitignore | 1 + cfg.mk | 10 +- daemon/Makefile.am | 33 +--- daemon/libvirtd.c | 10 +- daemon/libvirtd.h | 10 -- daemon/remote.c | 6 +- daemon/remote.h | 6 +- po/POTFILES.in | 6 +- src/Makefile.am | 33 +++- {daemon => src/admin}/admin_server.c | 4 +- {daemon => src/admin}/admin_server.h | 6 +- .../admin.c => src/admin/admin_server_dispatch.c | 48 ++++-- .../admin.h => src/admin/admin_server_dispatch.h | 18 +- src/libvirt-admin.c | 23 ++- src/libvirt_private.syms | 1 + src/libvirt_remote.syms | 2 +- src/locking/lock_daemon.c | 186 +++++++++++++++----- src/locking/lock_daemon_config.c | 3 + src/locking/lock_daemon_config.h | 1 + src/locking/test_virtlockd.aug.in | 4 + src/locking/virtlockd-admin.socket.in | 10 ++ src/locking/virtlockd.aug | 1 + src/locking/virtlockd.conf | 6 + src/locking/virtlockd.service.in | 1 + src/logging/log_daemon.c | 188 ++++++++++++++++----- src/logging/log_daemon_config.c | 3 + src/logging/log_daemon_config.h | 1 + src/logging/test_virtlogd.aug.in | 4 + src/logging/virtlogd-admin.socket.in | 10 ++ src/logging/virtlogd.aug | 1 + src/logging/virtlogd.service.in | 1 + src/rpc/virnetdaemon.c | 164 ++++++++++-------- src/rpc/virnetdaemon.h | 20 ++- src/rpc/virnetserver.c | 6 +- src/rpc/virnetserver.h | 7 +- src/rpc/virnetserverclient.c | 14 +- src/rpc/virnetserverclient.h | 23 ++- src/rpc/virnetserverprogram.h | 3 - src/util/virutil.c | 45 +++++ src/util/virutil.h | 1 + tests/virnetdaemontest.c | 37 +++- 41 files changed, 670 insertions(+), 287 deletions(-) rename {daemon => src/admin}/admin_server.c (99%) rename {daemon => src/admin}/admin_server.h (96%) rename daemon/admin.c => src/admin/admin_server_dispatch.c (92%) rename daemon/admin.h => src/admin/admin_server_dispatch.h (59%) create mode 100644 src/locking/virtlockd-admin.socket.in create mode 100644 src/logging/virtlogd-admin.socket.in -- 2.14.3

5 22

[libvirt] [PATCH] qemu: Expose rx/tx_queue_size in qemu.conf too
by Michal Privoznik 01 Feb '18

01 Feb '18

In 2074ef6cd4a2 and c56cdf259 (and friends) we've added two attributes to virtio NICs: rx_queue_size and tx_queue_size. However, sysadmins might want to set these on per-host basis but don't necessarily have an access to domain XML (e.g. because they are generated by some other app). So let's expose them under qemu.conf (the settings from domain XML still take precedence as they are more specific ones). Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- docs/formatdomain.html.in | 12 +++++++++-- src/qemu/libvirtd_qemu.aug | 4 ++++ src/qemu/qemu.conf | 7 +++++++ src/qemu/qemu_command.c | 42 ++++++++++++++++++++++++++++++-------- src/qemu/qemu_command.h | 3 ++- src/qemu/qemu_conf.c | 4 ++++ src/qemu/qemu_conf.h | 3 +++ src/qemu/qemu_hotplug.c | 2 +- src/qemu/test_libvirtd_qemu.aug.in | 2 ++ 9 files changed, 66 insertions(+), 13 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index d272cc1ba..c0107ab4b 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -5373,7 +5373,11 @@ qemu-kvm -net nic,model=? /dev/null some restrictions on actual value. For instance, latest QEMU (as of 2016-09-01) requires value to be a power of two from [256, 1024] range. - Since 2.3.0 (QEMU and KVM only) + Since 2.3.0 (QEMU and KVM only) + Then, Since 4.1.0 the default value can be + set in <code>qemu.conf</code> file and thus overrides hypervisor + default. + In general you should leave this option alone, unless you are very certain you know what you are doing. @@ -5389,7 +5393,11 @@ qemu-kvm -net nic,model=? /dev/null range. In addition to that, this may work only for a subset of interface types, e.g. aforementioned QEMU enables this option only for <code>vhostuser</code> type. - Since 3.7.0 (QEMU and KVM only) + Since 3.7.0 (QEMU and KVM only) + Then, Since 4.1.0 the default value can be + set in <code>qemu.conf</code> file and thus overrides hypervisor + default. + In general you should leave this option alone, unless you are very certain you know what you are doing. diff --git a/src/qemu/libvirtd_qemu.aug b/src/qemu/libvirtd_qemu.aug index c19bf3a43..084290296 100644 --- a/src/qemu/libvirtd_qemu.aug +++ b/src/qemu/libvirtd_qemu.aug @@ -118,6 +118,9 @@ module Libvirtd_qemu = let vxhs_entry = bool_entry "vxhs_tls" | str_entry "vxhs_tls_x509_cert_dir" + let virtio_entry = int_entry "rx_queue_size" + | int_entry "tx_queue_size" + (* Each entry in the config is one of the following ... *) let entry = default_tls_entry | vnc_entry @@ -137,6 +140,7 @@ module Libvirtd_qemu = | gluster_debug_level_entry | memory_entry | vxhs_entry + | virtio_entry let comment = [ label "#comment" . del /#[ \t]*/ "# " . store /([^ \t\n][^\n]*)?/ . del /\n/ "\n" ] let empty = [ label "#empty" . eol ] diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index 43dd561cc..a945ebdd5 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -775,3 +775,10 @@ # This directory is used for memoryBacking source if configured as file. # NOTE: big files will be stored here #memory_backing_dir = "/var/lib/libvirt/qemu/ram" + +# The following two values set the default RX/TX ring buffer size for virtio +# interfaces. These values are taken unless overridden in domain XML. Please +# note that QEMU accepts 256, 512 and 1024 only. These values correspond to +# those from domain XML. +#rx_queue_size = 1024 +#tx_queue_size = 1024 diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index b8aede32d..771c12445 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3693,7 +3693,8 @@ qemuBuildNicStr(virDomainNetDefPtr net, char * -qemuBuildNicDevStr(virDomainDefPtr def, +qemuBuildNicDevStr(virQEMUDriverConfigPtr cfg, + virDomainDefPtr def, virDomainNetDefPtr net, int vlan, unsigned int bootindex, @@ -3813,21 +3814,40 @@ qemuBuildNicDevStr(virDomainDefPtr def, virBufferAsprintf(&buf, ",mq=on,vectors=%zu", 2 * vhostfdSize + 2); } } - if (usingVirtio && net->driver.virtio.rx_queue_size) { - if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_VIRTIO_NET_RX_QUEUE_SIZE)) { + if (usingVirtio) { + unsigned int rx_queue_size = net->driver.virtio.rx_queue_size; + + if (rx_queue_size == 0 && + virQEMUCapsGet(qemuCaps, QEMU_CAPS_VIRTIO_NET_RX_QUEUE_SIZE)) + rx_queue_size = cfg->rx_queue_size; + + + if (rx_queue_size && + virQEMUCapsGet(qemuCaps, QEMU_CAPS_VIRTIO_NET_RX_QUEUE_SIZE)) { + net->driver.virtio.rx_queue_size = rx_queue_size; + virBufferAsprintf(&buf, ",rx_queue_size=%u", rx_queue_size); + } else if (rx_queue_size) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("virtio rx_queue_size option is not supported with this QEMU binary")); goto error; } - virBufferAsprintf(&buf, ",rx_queue_size=%u", net->driver.virtio.rx_queue_size); } - if (usingVirtio && net->driver.virtio.tx_queue_size) { - if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_VIRTIO_NET_TX_QUEUE_SIZE)) { + if (usingVirtio) { + unsigned int tx_queue_size = net->driver.virtio.tx_queue_size; + + if (tx_queue_size == 0 && + virQEMUCapsGet(qemuCaps, QEMU_CAPS_VIRTIO_NET_TX_QUEUE_SIZE)) + tx_queue_size = cfg->tx_queue_size; + + if (tx_queue_size && + virQEMUCapsGet(qemuCaps, QEMU_CAPS_VIRTIO_NET_TX_QUEUE_SIZE)) { + net->driver.virtio.tx_queue_size = tx_queue_size; + virBufferAsprintf(&buf, ",tx_queue_size=%u", tx_queue_size); + } else if (tx_queue_size) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("virtio tx_queue_size option is not supported with this QEMU binary")); goto error; } - virBufferAsprintf(&buf, ",tx_queue_size=%u", net->driver.virtio.tx_queue_size); } if (usingVirtio && net->mtu) { @@ -8489,7 +8509,7 @@ qemuBuildVhostuserCommandLine(virQEMUDriverPtr driver, virCommandAddArg(cmd, netdev); VIR_FREE(netdev); - if (!(nic = qemuBuildNicDevStr(def, net, -1, bootindex, + if (!(nic = qemuBuildNicDevStr(cfg, def, net, -1, bootindex, queues, qemuCaps))) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Error generating NIC -device string")); @@ -8526,6 +8546,7 @@ qemuBuildInterfaceCommandLine(virQEMUDriverPtr driver, int **nicindexes, bool chardevStdioLogd) { + virQEMUDriverConfigPtr cfg; int ret = -1; char *nic = NULL, *host = NULL; int *tapfd = NULL; @@ -8587,6 +8608,8 @@ qemuBuildInterfaceCommandLine(virQEMUDriverPtr driver, return -1; } + cfg = virQEMUDriverGetConfig(driver); + switch (actualType) { case VIR_DOMAIN_NET_TYPE_NETWORK: case VIR_DOMAIN_NET_TYPE_BRIDGE: @@ -8782,7 +8805,7 @@ qemuBuildInterfaceCommandLine(virQEMUDriverPtr driver, virCommandAddArgList(cmd, "-netdev", host, NULL); } if (qemuDomainSupportsNicdev(def, net)) { - if (!(nic = qemuBuildNicDevStr(def, net, vlan, bootindex, + if (!(nic = qemuBuildNicDevStr(cfg, def, net, vlan, bootindex, vhostfdSize, qemuCaps))) goto cleanup; virCommandAddArgList(cmd, "-device", nic, NULL); @@ -8826,6 +8849,7 @@ qemuBuildInterfaceCommandLine(virQEMUDriverPtr driver, VIR_FREE(host); VIR_FREE(tapfdName); VIR_FREE(vhostfdName); + virObjectUnref(cfg); return ret; } diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index bdde6f918..85f7bd1b4 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -90,7 +90,8 @@ char *qemuBuildNicStr(virDomainNetDefPtr net, int vlan); /* Current, best practice */ -char *qemuBuildNicDevStr(virDomainDefPtr def, +char *qemuBuildNicDevStr(virQEMUDriverConfigPtr cfg, + virDomainDefPtr def, virDomainNetDefPtr net, int vlan, unsigned int bootindex, diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index af503d31c..2fa96431f 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -912,6 +912,10 @@ int virQEMUDriverConfigLoadFile(virQEMUDriverConfigPtr cfg, if (virConfGetValueString(conf, "memory_backing_dir", &cfg->memoryBackingDir) < 0) goto cleanup; + if (virConfGetValueUInt(conf, "rx_queue_size", &cfg->rx_queue_size) < 0 || + virConfGetValueUInt(conf, "tx_queue_size", &cfg->tx_queue_size) < 0) + goto cleanup; + ret = 0; cleanup: diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index a553e30e2..3f38a76c2 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -206,6 +206,9 @@ struct _virQEMUDriverConfig { bool vxhsTLS; char *vxhsTLSx509certdir; + + unsigned int rx_queue_size; + unsigned int tx_queue_size; }; /* Main driver state */ diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 6b245bd6a..18b460704 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -1116,7 +1116,7 @@ qemuDomainAttachNetDevice(virQEMUDriverPtr driver, for (i = 0; i < vhostfdSize; i++) VIR_FORCE_CLOSE(vhostfd[i]); - if (!(nicstr = qemuBuildNicDevStr(vm->def, net, vlan, 0, + if (!(nicstr = qemuBuildNicDevStr(cfg, vm->def, net, vlan, 0, queueSize, priv->qemuCaps))) goto try_remove; diff --git a/src/qemu/test_libvirtd_qemu.aug.in b/src/qemu/test_libvirtd_qemu.aug.in index 688e5b9fd..4fc4e2f4e 100644 --- a/src/qemu/test_libvirtd_qemu.aug.in +++ b/src/qemu/test_libvirtd_qemu.aug.in @@ -100,3 +100,5 @@ module Test_libvirtd_qemu = { "1" = "mount" } } { "memory_backing_dir" = "/var/lib/libvirt/qemu/ram" } +{ "rx_queue_size" = "1024" } +{ "tx_queue_size" = "1024" } -- 2.13.6

2 2

[libvirt] [PATCH v2 0/6] Last CAT fixups v2
by Martin Kletzander 01 Feb '18

01 Feb '18

Few finishing touches for the Cache Allocation Technology. This series copies the default group's mask to newly created groups, fixes a possible wrong memory access, and cleans up after domains even if daemon was restarted. Martin Kletzander (6): util: Add helpers for getting resctrl group allocs util: Use default group's mask for unspecified resctrl allocations util: Don't overwrite mask in virResctrlAllocFindUnused qemu: Restore machinename even without cgroups util: Extract path formatting into virResctrlAllocDeterminePath qemu: Restore resctrl alloc data after restart src/libvirt_private.syms | 1 + src/qemu/qemu_cgroup.c | 4 - src/qemu/qemu_process.c | 11 +++ src/util/virresctrl.c | 219 +++++++++++++++++++++++++++++------------------ src/util/virresctrl.h | 4 + 5 files changed, 152 insertions(+), 87 deletions(-) -- 2.15.1

4 15

[libvirt] [Question] nodedev-list --cap return nothing after creating a mdev
by Wuzongyong (Euler Dept) 01 Feb '18

01 Feb '18

Hi, I meet a problem that libvirt handle new mediated devices wrongly. Command `virsh nodedev-list -cap mdev` return none after I create a mdev, and the root cause I found is: // kernel code int mdev_device_create(struct kobject *kobj, struct device *dev, uuid_le uuid) { ... ret = device_register(&mdev->dev); // send udev event here --- 1 if (ret) { put_device(&mdev->dev); goto create_err; } ret = mdev_device_create_ops(kobj, mdev); if (ret) goto create_failed; ret = mdev_create_sysfs_files(&mdev->dev, type); // create mdev_type symbol link here --- 2 if (ret) { mdev_device_remove_ops(mdev, true); goto create_failed; } mdev->type_kobj = kobj; dev_dbg(&mdev->dev, "MDEV: created\n"); ... } At point 1, kernel send a udev event(add). If libvirt receive this event and handle it before kernel code reach point 2, libvirt just fail to resolve link such as '/sys/devices/pci0000:80/0000:80:02.0/0000:81:00.0/36b79575-ada8-4406-893d-8cfd8d10a984/mdev_type' because the link haven't been created yet. Then libvirt thinks this devices doesn't exsit and stop tracking info of this device. Finally, command `vish nodedev-list -cap` return nothing though the device exist at host indeed. To make it easy to reproduce this problem, I just add a `udelay(1)` before point 2 by kprobe. So I think maybe we should add 'kobject_uevent(..., KOBJ_ADD)' after point 2 instead of point 1 ? Or is there any other method to solve this problem? Thanks, Zongyong Wu

2 7

[libvirt] [PATCH] tools: unlink bash completion files before symlinking
by Daniel P. Berrangé 01 Feb '18

01 Feb '18

"ln" will not replace an existing symlink, so if you run 'make install' twice, the second time will get an error: ln: failed to create symbolic link 'virsh': File exists We must always remove the symlink target first. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- Pushed as a build fix tools/Makefile.am | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/Makefile.am b/tools/Makefile.am index e173f56347..4c33e78a1d 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -425,6 +425,7 @@ install-bash-completion: $(INSTALL_SCRIPT) $(srcdir)/bash-completion/vsh \ "$(DESTDIR)$(BASH_COMPLETIONS_DIR)/vsh" ( cd $(DESTDIR)$(BASH_COMPLETIONS_DIR) && \ + rm -f virsh virt-admin && \ $(LN_S) vsh virsh && \ $(LN_S) vsh virt-admin ) -- 2.14.3

2 2

[libvirt] [PATCH] cpu: Add support for al57 Intel features
by Shaohe Feng 01 Feb '18

01 Feb '18

We can start qemu with a "cpu,+la57" to set 57-bit vitrual address space. So VM can be aware that it need to enable 5-level paging. Corresponding QEMU commits: al57 6c7c3c21f95dd9af8a0691c0dd29b07247984122 --- src/cpu/cpu_map.xml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/cpu/cpu_map.xml b/src/cpu/cpu_map.xml index e5da7a8..922a195 100644 --- a/src/cpu/cpu_map.xml +++ b/src/cpu/cpu_map.xml @@ -285,6 +285,9 @@ <feature name='ospke'> <cpuid eax_in='0x07' ecx_in='0x00' ecx='0x00000010'/> </feature> + <feature name='la57'> + <cpuid eax_in='0x07' ecx_in='0x00' ecx='0x00010000'/> + </feature> <feature name='avx512-4vnniw'> <cpuid eax_in='0x07' ecx_in='0x00' edx='0x00000004'/> -- 2.7.4

4 6

[libvirt] [PATCH go] go: Add ERR_DEVICE_MISSING constants
by Chen Hanxiao 01 Feb '18

01 Feb '18

From: Chen Hanxiao <chenhanxiao(a)gmail.com> Signed-off-by: Chen Hanxiao <chenhanxiao(a)gmail.com> --- error.go | 3 +++ error_compat.h | 6 ++++++ 2 files changed, 9 insertions(+) diff --git a/error.go b/error.go index fb218f7..9de277c 100644 --- a/error.go +++ b/error.go @@ -353,6 +353,9 @@ const ( // error in libssh transport driver ERR_LIBSSH = ErrorNumber(C.VIR_ERR_LIBSSH) + + // libvirt fail to find the desired device + ERR_DEVICE_MISSING = ErrorNumber(C.VIR_ERR_DEVICE_MISSING) ) type ErrorDomain int diff --git a/error_compat.h b/error_compat.h index f26dc09..64ed991 100644 --- a/error_compat.h +++ b/error_compat.h @@ -147,4 +147,10 @@ #define VIR_FROM_RESCTRL 67 #endif +/* 4.1.0 */ + +#ifndef VIR_ERR_DEVICE_MISSING +#define VIR_ERR_DEVICE_MISSING 99 +#endif + #endif /* LIBVIRT_GO_ERROR_COMPAT_H__ */ -- 2.14.3

2 1