February 2016 - Devel - libvirt List Archives

[libvirt] [PATCH 0/2] Avoid infinite loop if vCPUs can't be resumed in qemu snapshot code
by Peter Krempa 08 Feb '16

08 Feb '16

Messed up refactor :(. Bunny ears of shame were applied. Peter Krempa (2): qemu: snapshot: Don't overwrite existing errors when thawing filesystems qemu: snapshot: Avoid infinite loop if vCPUs can't be resumed src/qemu/qemu_driver.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) -- 2.6.2

2 4

[libvirt] Fwd: Virtualization in High-Performance Cloud Computing (VHPC '16)
by VHPC 16 07 Feb '16

07 Feb '16

==================================================================== CALL FOR PAPERS 11th Workshop on Virtualization in High-Performance Cloud Computing (VHPC '16) held in conjunction with the International Supercomputing Conference - High Performance, June 19-23, 2016, Frankfurt, Germany. ==================================================================== Date: June 23, 2016 Workshop URL: http://vhpc.org Lightning talk abstract registration deadline: February 29, 2016 Paper/publishing track abstract registration deadline: March 21, 2016 Paper Submission Deadline: April 25, 2016 Call for Papers Virtualization technologies constitute a key enabling factor for flexible resource management in modern data centers, and particularly in cloud environments. Cloud providers need to manage complex infrastructures in a seamless fashion to support the highly dynamic and heterogeneous workloads and hosted applications customers deploy. Similarly, HPC environments have been increasingly adopting techniques that enable flexible management of vast computing and networking resources, close to marginal provisioning cost, which is unprecedented in the history of scientific and commercial computing. Various virtualization technologies contribute to the overall picture in different ways: machine virtualization, with its capability to enable consolidation of multiple underutilized servers with heterogeneous software and operating systems (OSes), and its capability to live-migrate a fully operating virtual machine (VM) with a very short downtime, enables novel and dynamic ways to manage physical servers; OS-level virtualization (i.e., containerization), with its capability to isolate multiple user-space environments and to allow for their coexistence within the same OS kernel, promises to provide many of the advantages of machine virtualization with high levels of responsiveness and performance; I/O Virtualization allows physical NICs/HBAs to take traffic from multiple VMs or containers; network virtualization, with its capability to create logical network overlays that are independent of the underlying physical topology and IP addressing, provides the fundamental ground on top of which evolved network services can be realized with an unprecedented level of dynamicity and flexibility; the increasingly adopted paradigm of Software-Defined Networking (SDN) promises to extend this flexibility to the control and data planes of network paths. Topics of Interest The VHPC program committee solicits original, high-quality submissions related to virtualization across the entire software stack with a special focus on the intersection of HPC and the cloud. Topics include, but are not limited to: - Virtualization in supercomputing environments, HPC clusters, cloud HPC and grids - OS-level virtualization including container runtimes (Docker, rkt et al.) - Lightweight compute node operating systems/VMMs - Optimizations of virtual machine monitor platforms, hypervisors - QoS and SLA in hypervisors and network virtualization - Cloud based network and system management for SDN and NFV - Management, deployment and monitoring of virtualized environments - Virtual per job / on-demand clusters and cloud bursting - Performance measurement, modelling and monitoring of virtualized/cloud workloads - Programming models for virtualized environments - Virtualization in data intensive computing and Big Data processing - Cloud reliability, fault-tolerance, high-availability and security - Heterogeneous virtualized environments, virtualized accelerators, GPUs and co-processors - Optimized communication libraries/protocols in the cloud and for HPC in the cloud - Topology management and optimization for distributed virtualized applications - Adaptation of emerging HPC technologies (high performance networks, RDMA, etc..) - I/O and storage virtualization, virtualization aware file systems - Job scheduling/control/policy in virtualized environments - Checkpointing and migration of VM-based large compute jobs - Cloud frameworks and APIs - Energy-efficient / power-aware virtualization The Workshop on Virtualization in High-Performance Cloud Computing (VHPC) aims to bring together researchers and industrial practitioners facing the challenges posed by virtualization in order to foster discussion, collaboration, mutual exchange of knowledge and experience, enabling research to ultimately provide novel solutions for virtualized computing systems of tomorrow. The workshop will be one day in length, composed of 20 min paper presentations, each followed by 10 min discussion sections, plus lightning talks that are limited to 5 minutes. Presentations may be accompanied by interactive demonstrations. Important Dates February 29, 2016 - Lightning talk abstract registration March 21, 2016 - Paper/publishing track abstract registration April 25, 2016 - Full paper submission May 30, 2016 Acceptance notification June 23, 2016 - Workshop Day July 25, 2016 - Camera-ready version due Chair Michael Alexander (chair), TU Wien, Austria Anastassios Nanos (co-chair), NTUA, Greece Balazs Gerofi (co-chair), RIKEN Advanced Institute for Computational Science, Japan Program committee Stergios Anastasiadis, University of Ioannina, Greece Costas Bekas, IBM Research, Switzerland Jakob Blomer, CERN Ron Brightwell, Sandia National Laboratories, USA Roberto Canonico, University of Napoli Federico II, Italy Julian Chesterfield, OnApp, UK Stephen Crago, USC ISI, USA Christoffer Dall, Columbia University, USA Patrick Dreher, MIT, USA Robert Futrick, Cycle Computing, USA Robert Gardner, University of Chicago, USA William Gardner, University of Guelph, Canada Wolfgang Gentzsch, UberCloud, USA Kyle Hale, Northwestern University, USA Marcus Hardt, Karlsruhe Institute of Technology, Germany Krishna Kant, Templte University, USA Romeo Kinzler, IBM, Switzerland Brian Kocoloski, University of Pittsburgh, USA Kornilios Kourtis, IBM Research, Switzerland Nectarios Koziris, National Technical University of Athens, Greece John Lange, University of Pittsburgh, USA Nikos Parlavantzas, IRISA, France Kevin Pendretti, Sandia National Laboratories, USA Che-Rung Roger Lee, National Tsing Hua University, Taiwan Giuseppe Lettieri, University of Pisa, Italy Qing Liu, Oak Ridge National Laboratory, USA Paul Mundt, Adaptant, Germany Amer Qouneh, University of Florida, USA Carlos Reaño, Technical University of Valencia, Spain Seetharami Seelam, IBM Research, USA Josh Simons, VMWare, USA Borja Sotomayor, University of Chicago, USA Dieter Suess, TU Wien, Austria Craig Stewart, Indiana University, USA Anata Tiwari, San Diego Supercomputer Center, USA Kurt Tutschku, Blekinge Institute of Technology, Sweden Amit Vadudevan, Carnegie Mellon University, USA Yasuhiro Watashiba, Osaka University, Japan Nicholas Wright, Lawrence Berkeley National Laboratory, USA Chao-Tung Yang, Tunghai University, Taiwan Gianluigi Zanetti, CRS4, Italy Paper Submission-Publication Papers submitted to the workshop will be reviewed by at least two members of the program committee and external reviewers. Submissions should include abstract, key words, the e-mail address of the corresponding author, and must not exceed 10 pages, including tables and figures at a main font size no smaller than 11 point. Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the conference to present the work. The format must be according to the Springer LNCS Style. Initial submissions are in PDF; authors of accepted papers will be requested to provide source files. Format Guidelines: ftp://ftp.springer.de/pub/tex/latex/llncs/latex2e/llncs2e.zip Abstract, Paper Submission Link: https://edas.info/newPaper.php?c=21801 Lightning Talks Lightning Talks are non-paper track, synoptical in nature and are strictly limited to 5 minutes. They can be used to gain early feedback on ongoing research, for demonstrations, to present research results, early research ideas, perspectives and positions of interest to the community. Submit abstract via the main submission link. General Information The workshop is one day in length and will be held in conjunction with the International Supercomputing Conference - High Performance (ISC) 2016, June 19-23, Frankfurt, Germany.

1 0

[libvirt] [PATCH] nodedev: stub nodeDeviceSysfsGetPCIRelatedDevCaps
by Roman Bogorodskiy 06 Feb '16

06 Feb '16

Add a stub for nodeDeviceSysfsGetPCIRelatedDevCaps() for non-Linux platforms. It allows nodedev driver to work on non-Linux platoforms that, however, have HAL. --- src/node_device/node_device_linux_sysfs.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/node_device/node_device_linux_sysfs.c b/src/node_device/node_device_linux_sysfs.c index 431f471..24a6a2e 100644 --- a/src/node_device/node_device_linux_sysfs.c +++ b/src/node_device/node_device_linux_sysfs.c @@ -238,4 +238,11 @@ nodeDeviceSysfsGetSCSIHostCaps(virNodeDevCapDataPtr d ATTRIBUTE_UNUSED) return -1; } +int +nodeDeviceSysfsGetPCIRelatedDevCaps(const char *sysfsPath ATTRIBUTE_UNUSED, + virNodeDevCapDataPtr data ATTRIBUTE_UNUSED) +{ + return -1; +} + #endif /* __linux__ */ -- 2.4.6

2 2

[libvirt] set a different default gateway in an isolated network
by pichon 06 Feb '16

06 Feb '16

Hello, I would like to set a specify default gateway for my VMs. However I’m not succeeding Here after is the isolated lan, where I added the statement <route address='10.0.0.0' prefix='8' gateway='10.10.0.7’/> where my aim is to route to the 10.10.0.7 What should I do , to make it works ? Thanks <network> <name>prd-private-lan</name> <uuid>255083f4-05a1-4a24-bc66-d3a3a9ab0724</uuid> <bridge name='virbr3' stp='off' delay='0'/> <mac address='52:54:00:08:1e:d8'/> <domain name='prd.pipiche.net'/> <dns> <forwarder addr='8.8.4.4'/> <forwarder addr='8.8.8.8'/> </dns> <ip address='10.10.0.1' netmask='255.255.255.0'> <dhcp> <range start='10.10.0.128' end='10.10.0.254'/> <host mac='52:54:10:b6:35:ca' ip='10.10.0.3'/> <host mac='52:54:10:5f:92:05' ip='10.10.0.7'/> <host mac='52:54:10:7d:3a:3f' ip='10.10.0.10'/> <host mac='52:54:10:30:93:56' ip='10.10.0.21'/> <host mac='52:54:10:78:c1:c2' ip='10.10.0.22'/> <host mac='52:54:10:64:00:21' ip='10.10.0.25'/> <host mac='52:54:10:5a:5f:63' ip='10.10.0.26'/> <host mac='52:54:10:4c:b2:61' ip='10.10.0.27'/> <host mac='52:54:10:62:de:5f' ' ip='10.10.0.62'/> </dhcp> </ip> <route address='10.0.0.0' prefix='8' gateway='10.10.0.7'/> </network> When looking on the VM (guest) the route is not correctly set route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.10.0.0 0.0.0.0 255.255.255.0 U 100 0 0 ens4 arp -a ? (10.10.0.1) at 52:54:00:08:1e:d8 [ether] on ensue ifconfig ens4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.10.0.27 netmask 255.255.255.0 broadcast 10.10.0.255 inet6 fe80::5054:10ff:fe4c:b261 prefixlen 64 scopeid 0x20<link> ether 52:54:10:4c:b2:61 txqueuelen 1000 (Ethernet) RX packets 1805 bytes 153644 (150.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1745 bytes 136075 (132.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

1 0

[libvirt] [PATCH v4 0/9] add close callback for drivers with persistent connection
by Nikolay Shirokovskiy 06 Feb '16

06 Feb '16

Currently close callback API can only inform us of closing the connection between remote driver and daemon. But what if a driver running in the daemon itself can have another persistent connection? In this case we want to be informed of that connection changes state too. This patch series extends meaning of current close callback API so that now it notifies of closing of any internal persistent connection. The overall approach is to move close callback support to drivers. Changes from v3: ================ Add patch [3] "close callback: make unregister clean after connect close event." Make register/unregister methods of connection close callback object return void. This solves the problem of patch [8] "daemon: add connection close rpc" ([7] in previous version) of consistent unregistering. All checks are moved outside of the methods. I hesitate whether to add or not means that track connection close callback object consinstency and finally decided to add (checks and warnings inside methods). The reason is that without these checks we get memory leaks which are rather difficult to find out. Unfortunately this change touch a number of patches as the first change is done in the first patch of the series. Changes from v2: ================ Split patches further to make it more comprehensible. Nikolay Shirokovskiy (9): factor out virConnectCloseCallbackDataPtr methods virConnectCloseCallbackData: fix connection object refcount close callback: make unregister clean after connect close event virConnectCloseCallbackData: factor out callback disarming close callback API: remove unnecessary locks virConnectCloseCallbackDataDispose: remove unnecessary locks close callback: move it to driver daemon: add connection close rpc vz: implement connection close notification daemon/libvirtd.h | 1 + daemon/remote.c | 84 ++++++++++++++++++++++++++++++ src/datatypes.c | 118 ++++++++++++++++++++++++++++++++++--------- src/datatypes.h | 16 ++++-- src/driver-hypervisor.h | 12 +++++ src/libvirt-host.c | 46 ++--------------- src/remote/remote_driver.c | 114 ++++++++++++++++++++++++++++++++++------- src/remote/remote_protocol.x | 24 ++++++++- src/remote_protocol-structs | 6 +++ src/vz/vz_driver.c | 59 ++++++++++++++++++++++ src/vz/vz_sdk.c | 4 ++ src/vz/vz_utils.h | 3 ++ 12 files changed, 397 insertions(+), 90 deletions(-) -- 1.8.3.1

1 10

[libvirt] [PATCH] rbd: Allow shrinking and deltas when resizing RBD volumes
by Wido den Hollander 06 Feb '16

06 Feb '16

Modify virCheckFlags that it accepts both VIR_STORAGE_VOL_RESIZE_DELTA and VIR_STORAGE_VOL_RESIZE_SHRINK as valid flags for resizing RBD volumes. This still does not solve the problem where RBD volumes can't be shrinked using libvirt. The allocation and capacity of RBD volumes are equal for a RBD volume and this makes libvirt think the volume is fully allocated and therefor can't be shrinked. Signed-off-by: Wido den Hollander <wido(a)widodh.nl> --- src/storage/storage_backend_rbd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/storage/storage_backend_rbd.c b/src/storage/storage_backend_rbd.c index 80a1d33..88b613a 100644 --- a/src/storage/storage_backend_rbd.c +++ b/src/storage/storage_backend_rbd.c @@ -1041,7 +1041,8 @@ static int virStorageBackendRBDResizeVol(virConnectPtr conn ATTRIBUTE_UNUSED, int ret = -1; int r = 0; - virCheckFlags(0, -1); + virCheckFlags(VIR_STORAGE_VOL_RESIZE_DELTA | + VIR_STORAGE_VOL_RESIZE_SHRINK, -1); if (virStorageBackendRBDOpenRADOSConn(&ptr, conn, &pool->def->source) < 0) goto cleanup; -- 1.9.1

3 6

[libvirt] [PATCH] util: Fix virCgroupNewMachine ATTRIBUTE_NONNULL args
by John Ferlan 06 Feb '16

06 Feb '16

Commit id 'c3bd0019c0' removed arg3, but forgot to adjust the numbers for NONNULL - caused build failure for coverity Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- Pushed as build breaker src/util/vircgroup.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/util/vircgroup.h b/src/util/vircgroup.h index bec88dc..46d3ace 100644 --- a/src/util/vircgroup.h +++ b/src/util/vircgroup.h @@ -110,7 +110,7 @@ int virCgroupNewMachine(const char *name, int controllers, virCgroupPtr *group) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) - ATTRIBUTE_NONNULL(4); + ATTRIBUTE_NONNULL(3); int virCgroupTerminateMachine(const char *name) ATTRIBUTE_NONNULL(1); -- 2.5.0

1 0

[libvirt] [PATCH] bhyve: fix preprocessor indentation
by Roman Bogorodskiy 06 Feb '16

06 Feb '16

Syntax-check fails with: cppi: src/bhyve/bhyve_driver.h: line 26: not properly indented cppi: src/bhyve/bhyve_driver.h: line 27: not properly indented maint.mk: incorrect preprocessor indentation Fix by properly indenting '#include's. Pushed as trivial. --- src/bhyve/bhyve_driver.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/bhyve/bhyve_driver.h b/src/bhyve/bhyve_driver.h index a290c4a..fb57d28 100644 --- a/src/bhyve/bhyve_driver.h +++ b/src/bhyve/bhyve_driver.h @@ -23,8 +23,8 @@ #ifndef __BHYVE_DRIVER_H__ # define __BHYVE_DRIVER_H__ -#include "capabilities.h" -#include "bhyve_utils.h" +# include "capabilities.h" +# include "bhyve_utils.h" int bhyveRegister(void); -- 2.4.6

1 0

[libvirt] [PATCH] bhyve: Fix the build
by Michal Privoznik 06 Feb '16

06 Feb '16

After 1036ddadb276e we use bhyveDriverGetCapabilities from other sources too, not only from bhyve_driver.c. However, the function was static so not properly expose to other files. In order to expose it, we need to move couple of #include-s too. Then, there has been a copy paste error in virBhyveProcessReconnect: s/privconn/data->driver/. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Pushed under build breaker rule. src/bhyve/bhyve_driver.c | 4 +--- src/bhyve/bhyve_driver.h | 5 +++++ src/bhyve/bhyve_process.c | 3 ++- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/src/bhyve/bhyve_driver.c b/src/bhyve/bhyve_driver.c index f486f86..9219890 100644 --- a/src/bhyve/bhyve_driver.c +++ b/src/bhyve/bhyve_driver.c @@ -30,7 +30,6 @@ #include "datatypes.h" #include "virbuffer.h" #include "viruuid.h" -#include "capabilities.h" #include "configmake.h" #include "viralloc.h" #include "network_conf.h" @@ -58,7 +57,6 @@ #include "bhyve_command.h" #include "bhyve_domain.h" #include "bhyve_process.h" -#include "bhyve_utils.h" #include "bhyve_capabilities.h" #define VIR_FROM_THIS VIR_FROM_BHYVE @@ -127,7 +125,7 @@ bhyveAutostartDomains(bhyveConnPtr driver) * * Returns: a reference to a virCapsPtr instance or NULL */ -static virCapsPtr ATTRIBUTE_NONNULL(1) +virCapsPtr ATTRIBUTE_NONNULL(1) bhyveDriverGetCapabilities(bhyveConnPtr driver) { diff --git a/src/bhyve/bhyve_driver.h b/src/bhyve/bhyve_driver.h index 221d5a0..a290c4a 100644 --- a/src/bhyve/bhyve_driver.h +++ b/src/bhyve/bhyve_driver.h @@ -23,10 +23,15 @@ #ifndef __BHYVE_DRIVER_H__ # define __BHYVE_DRIVER_H__ +#include "capabilities.h" +#include "bhyve_utils.h" + int bhyveRegister(void); unsigned bhyveDriverGetCaps(virConnectPtr conn); unsigned bhyveDriverGetGrubCaps(virConnectPtr conn); +virCapsPtr bhyveDriverGetCapabilities(bhyveConnPtr driver); + #endif /* __BHYVE_DRIVER_H__ */ diff --git a/src/bhyve/bhyve_process.c b/src/bhyve/bhyve_process.c index 9763d71..14588a9 100644 --- a/src/bhyve/bhyve_process.c +++ b/src/bhyve/bhyve_process.c @@ -32,6 +32,7 @@ #include <net/if_tap.h> #include "bhyve_device.h" +#include "bhyve_driver.h" #include "bhyve_command.h" #include "bhyve_monitor.h" #include "bhyve_process.h" @@ -375,7 +376,7 @@ virBhyveProcessReconnect(virDomainObjPtr vm, if (!vm->pid) return 0; - caps = bhyveDriverGetCapabilities(privconn); + caps = bhyveDriverGetCapabilities(data->driver); if (!caps) return -1; -- 2.4.10

2 1

[libvirt] [PATCH v3 00/13] vcpu info storage refactors - part 2
by Peter Krempa 05 Feb '16

05 Feb '16

Rest of the series was pushed. Patches with no explicit note were already ACKed but depend on the rest of the series. Peter Krempa (13): cgroup: Prepare for sparse vCPU topologies in virCgroupGetPercpuStats qemu: Differentiate error codes when VM exits in qemuDomainDetectVcpuPids qemu: Reuse qemuDomainDetectVcpuPids in cpu hot(un)plug conf: Don't copy def->cpumask into cpu pinning info conf: Store cpu pinning data in def->vcpus conf: remove unused cpu pinning helpers and data structures conf: Extract code that formats <cputune> conf: Don't store vcpusched orthogonally to other vcpu info conf: Fix how iothread scheduler info is stored qemu: vcpu: Aggregate code to set vCPU tuning qemu: vcpu: Reuse qemuProcessSetupVcpu in vcpu hotplug qemu: iothread: Aggregate code to set IOThread tuning qemu: iothread: Reuse qemuProcessSetupIOThread in iothread hotplug src/conf/domain_conf.c | 835 ++++++++++----------- src/conf/domain_conf.h | 41 +- src/libvirt_private.syms | 7 - src/libxl/libxl_domain.c | 20 +- src/libxl/libxl_driver.c | 38 +- src/lxc/lxc_driver.c | 2 +- src/qemu/qemu_cgroup.c | 195 ----- src/qemu/qemu_cgroup.h | 2 - src/qemu/qemu_domain.c | 4 +- src/qemu/qemu_driver.c | 380 +++------- src/qemu/qemu_process.c | 401 ++++++---- src/qemu/qemu_process.h | 6 + src/test/test_driver.c | 45 +- src/util/vircgroup.c | 16 +- src/util/vircgroup.h | 3 +- src/vz/vz_sdk.c | 16 +- .../qemuxml2xmlout-cputune-iothreadsched.xml | 3 +- tests/vircgrouptest.c | 2 +- 18 files changed, 823 insertions(+), 1193 deletions(-) -- 2.6.2

2 16