[libvirt] CfP VHPC19: HPC Virtualization-Containers: Paper due May 1, 2019 (extended)
by VHPC 19
====================================================================
CALL FOR PAPERS
14th Workshop on Virtualization in High-Performance Cloud Computing
(VHPC '19) held in conjunction with the International Supercomputing
Conference - High Performance, June 16-20, 2019, Frankfurt, Germany.
(Springer LNCS Proceedings)
====================================================================
Date: June 20, 2019
Workshop URL: http://vhpc.org
Paper Submission Deadline: May 1, 2019 (extended)
Springer LNCS, rolling abstract submission
Abstract/Paper Submission Link: https://edas.info/newPaper.php?c=25685
Call for Papers
Containers and virtualization technologies constitute key enabling
factors for flexible resource management in modern data centers, and
particularly in cloud environments. Cloud providers need to manage
complex infrastructures in a seamless fashion to support the highly
dynamic and heterogeneous workloads and hosted applications customers
deploy. Similarly, HPC environments have been increasingly adopting
techniques that enable flexible management of vast computing and
networking resources, close to marginal provisioning cost, which is
unprecedented in the history of scientific and commercial computing.
Various virtualization-containerization technologies contribute to the
overall picture in different ways: machine virtualization, with its
capability to enable consolidation of multiple underutilized servers
with heterogeneous software and operating systems (OSes), and its
capability to live-migrate a fully operating virtual machine (VM)
with a very short downtime, enables novel and dynamic ways to manage
physical servers; OS-level virtualization (i.e., containerization),
with its capability to isolate multiple user-space environments and
to allow for their coexistence within the same OS kernel, promises to
provide many of the advantages of machine virtualization with high
levels of responsiveness and performance; lastly, unikernels provide
for many virtualization benefits with a minimized OS/library surface.
I/O Virtualization in turn allows physical network interfaces to take
traffic from multiple VMs or containers; network virtualization, with
its capability to create logical network overlays that are independent
of the underlying physical topology is furthermore enabling
virtualization of HPC infrastructures.
Publication
Accepted papers will be published in a Springer LNCS proceedings volume.
Topics of Interest
The VHPC program committee solicits original, high-quality submissions
related to virtualization across the entire software stack with a
special focus on the intersection of HPC, containers-virtualization
and the cloud.
Major Topics:
- HPC on Containers and VMs
- Containerized applications with OS-level virtualization
- Lightweight applications with Unikernels
- HP-as-a-Service
each major topic encompassing design/architecture, management,
performance management, modeling and configuration/tooling:
Design / Architecture:
- Containers and OS-level virtualization (LXC, Docker, rkt,
Singularity, Shifter, i.a.)
- Hypervisor support for heterogeneous resources (GPUs, co-processors,
FPGAs, etc.)
- Hypervisor extensions to mitigate side-channel attacks
([micro-]architectural timing attacks, privilege escalation)
- VM & Container trust and security models
- Multi-environment coupling, system software supporting in-situ
analysis with HPC simulation
- Cloud reliability, fault-tolerance and high-availability
- Energy-efficient and power-aware virtualization
- Containers inside VMs with hypervisor isolation
- Virtualization support for emerging memory technologies
- Lightweight/specialized operating systems in conjunction with
virtual machines
- Hypervisor support for heterogeneous resources (GPUs, co-processors,
FPGAs, etc.)
- Novel unikernels and use cases for virtualized HPC environments
- ARM-based hypervisors, ARM virtualization extensions
Management:
- Container and VM management for HPC and cloud environments
- HPC services integration, services to support HPC
- Service and on-demand scheduling & resource management
- Dedicated workload management with VMs or containers
- Workflow coupling with VMs and containers
- Unikernel, lightweight VM application management
- Environments and tools for operating containerized environments
(batch, orchestration)
- Novel models for non-HPC workload provisioning on HPC resources
Performance Measurements and Modeling:
- Performance improvements for or driven by unikernels
- Optimizations of virtual machine monitor platforms and hypervisors
- Scalability analysis of VMs and/or containers at large scale
- Performance measurement, modeling and monitoring of
virtualized/cloud workloads
- Virtualization in supercomputing environments, HPC clusters, HPC in
the cloud
Configuration / Tooling:
- Tool support for unikernels: configuration/build environments,
debuggers, profilers
- Job scheduling/control/policy and container placement in virtualized
environments
- Operating MPI in containers/VMs and Unikernels
- Software defined networks and network virtualization
- GPU virtualization operationalization
The Workshop on Virtualization in High-Performance Cloud Computing
(VHPC) aims to bring together researchers and industrial practitioners
facing the challenges posed by virtualization in order to foster
discussion, collaboration, mutual exchange of knowledge and
experience, enabling research to ultimately provide novel solutions
for virtualized computing systems of tomorrow.
The workshop will be one day in length, composed of 20 min paper
presentations, each followed by 10 min discussion sections, plus
lightning talks that are limited to 5 minutes. Presentations may be
accompanied by interactive demonstrations.
Important Dates
May 1, 2019 - Abstract/Paper extended submission deadline
(Springer LNCS)
May 20, 2019 - Acceptance notification
June 20th, 2019 - Workshop Day
July 10th, 2019 - Camera-ready version due
Chair
Michael Alexander (chair), University of Vienna, Austria
Anastassios Nanos (co-chair), SunLight.io, UK
Andrew Younge (co-chair), Sandia National Laboratories
Program committee
Stergios Anastasiadis, University of Ioannina, Greece
Jakob Blomer, CERN, Europe
Eduardo César, Universidad Autonoma de Barcelona, Spain
Taylor Childers, Argonne National Laboratory, USA
Stephen Crago, USC ISI, USA
Tommaso Cucinotta, St. Anna School of Advanced Studies, Italy
Christoffer Dall, Columbia University, USA
François Diakhaté, CEA, France
Patrick Dreher, MIT, USA
Kyle Hale, Northwestern University, USA
Bob Killen, University of Michigan, USA
Brian Kocoloski, Washington University, USA
John Lange, University of Pittsburgh, USA
Giuseppe Lettieri, University of Pisa, Italy
Qing Liu, Oak Ridge National Laboratory, USA
Nikos Parlavantzas, IRISA, France
Kevin Pedretti, Sandia National Laboratories, USA
Amer Qouneh, Western New England University, USA
Carlos Reaño, Queen’s University Belfast, UK
Borja Sotomayor, University of Chicago, USA
Jonathan Sparks, Cray, USA
Joe Stubbs, Texas Advanced Computing Center, USA
Anata Tiwari, San Diego Supercomputer Center, USA
Kurt Tutschku, Blekinge Institute of Technology, Sweden
John Walters, USC ISI, USA
Yasuhiro Watashiba, Osaka University, Japan
Chao-Tung Yang, Tunghai University, Taiwan
Na Zhang, VMware, USA
Paper Submission-Publication
Papers submitted to the workshop will be reviewed by at least two members of
the program committee and external reviewers. Submissions should include
abstract, keywords, the e-mail address of the corresponding author, and must
not exceed 10 pages, including tables and figures at a main font size no
smaller than 11 point. Submission of a paper should be regarded as a commitment
that, should the paper be accepted, at least one of the authors will register
and attend the conference to present the work. Accepted papers will be
published in a Springer LNCS volume.
The format must be according to the Springer LNCS Style. Initial submissions
are in PDF; authors of accepted papers will be requested to provide source
files.
Format Guidelines:
ftp://ftp.springernature.com/cs-proceeding/llncs/llncs2e.zip
Abstract, Paper Submission Link:
https://edas.info/newPaper.php?c=25685
Lightning Talks
Lightning Talks are non-paper track, synoptical in nature and are strictly
limited to 5 minutes. They can be used to gain early feedback on ongoing
research, for demonstrations, to present research results, early research
ideas, perspectives and positions of interest to the community. Submit abstract
via the main submission link.
General Information
The workshop is one day in length and will be held in conjunction with the
International Supercomputing Conference - High Performance (ISC) 2019, June
16-20, Frankfurt, Germany.
6 years
[libvirt] Release of libvirt-5.2.0
by Daniel Veillard
It's out, a bit late but the signed tarball and rpms are at the usual place
and it's tagged in git:
ftp://libvirt.org/libvirt/
I also pushed the python bindings, with only a single patch since 5.1.0 at:
ftp://libvirt.org/libvirt/python/
Note that it's a release with rather large changes, new features and
removal of some deprecated init scripts, and as usual a large amount
of improvement and bug fixes:
New features:
- Add Storage Pool Capabilities output
Add support to list an enumerated list of supported Storage Pools via
the virConnectGetCapabilities API when connected via a Storage Driver.
Add support to get a more detailed list XML output Storage Pool
Capabilities vis the virConnectGetStoragePoolCapabilites API.
- qemu: Support virtio-{non-}transitional device models
virtio-transitional and virtio-non-transitional model values were added
to the QEMU driver for the following devices: disk, interface,
filesystem, rng, vsock, memballoon, controller type scsi, controller
type virtio-serial, input bus virtio type passthrough, hostdev type
scsi_host. These new models can be used to give fine grained control
over what virtio device version is presented to the guest.
- qemu: Enable firmware autoselection
Libvirt allows users to provide loader path for some time now. However,
this puts some burden on users because they need to know what firmware
meets their requirements. Now that QEMU ships firmware description
files this burden can be moved onto libvirt. It is as easy as setting
the firmware attribute in the os element (accepted values are bios and
efi). Moreover, libvirt automatically enables domain features needed
for firmware it chooses.
- snapshots: Add support for topological listings
A new flag VIR_DOMAIN_SNAPSHOT_LIST_TOPOLOGICAL is available for the
various snapshot listing APIs such as virDomainListAllSnapshots(). For
drivers that support the flag, the listed snapshots are guaranteed to
be sorted such that parents occur before children.
- Xen: Add support for max grant frames setting
Add support for Xen's max_grant_frames setting by adding a new xenbus
controller type with a maxGrantFrames attribute. E.g. <controller
type='xenbus' maxGrantFrames='64'/>
- qemu: Add support for parallel migration
With QEMU 4.0.0 libvirt can enable parallel migration which causes the
memory pages to be processed in parallel by several threads and sent to
the destination host using several connections at the same time. This
may increase migration speed in case a single thread is unable to
saturate the network link.
Removed features:
- Drop support for Upstart and "Red Hat" init scripts
Not a single one of the platforms we target still uses Upstart, and the
Upstart project itself has been abandoned for several years now; the
same is true for the "Red Hat" (really System V) init scripts, since
RHEL 7 and later releases use systemd.
Improvements:
- Report class information for PCI node device capability.
- Split setup of IPv4 and IPv6 top level chain
The requirement resulting from private chains improvement done in
v5.1.0 was refined so that only tables from corresponding IP version
are required. This means that if a network doesn't have IPv6 enabled
then those tables are not required.
- Don't default to building the QEMU driver
Historically, the QEMU driver has been special in that it was enabled
by default, with the option to explicitly opt-out of it; starting now,
we're enabling it opportunistically if we detect that all requirements
are available, just like we do with other drivers.
Bug fixes:
- virt-host-validate: Fix IOMMU check on s390x
- qemu: Allow creating pSeries guests with graphics and no USB mouse
It's now possible to prevent libvirt from automatically adding a USB
mouse to pSeries guests by including a USB tablet in the input XML:
doing so is desiderable as using a tablet results in a much better user
experience when working with GUIs.
- qemu: Set $HOME and XGD variables for qemu:///system guests
This avoids files being accidentally created under / or the guests not
being able to start because they lack the necessary permissions to
write to that location.
Thanks everybody for your contributions for this release, be it
with bug reports, patches, reviews, documentation, etc...
Enjoy the release !
Daniel
--
Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/
veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | virtualization library http://libvirt.org/
6 years
[libvirt] [PATCH 0/5] util: Remove some obsolete and dead code
by Peter Krempa
Peter Krempa (5):
util: Remove virPipeReadUntilEOF
vmx: Remove unused variable in virVMXParseConfig
vmx: Refactor number parsing in virVMXParseConfig
rpc: ssh: Use virStrToLong_i instead of virParseNumber
util: Remove virParseNumber
src/libvirt_private.syms | 2 -
src/rpc/virnetlibsshsession.c | 6 +-
src/util/virutil.c | 113 ----------------------------------
src/util/virutil.h | 4 --
src/vmx/vmx.c | 62 ++++++-------------
5 files changed, 23 insertions(+), 164 deletions(-)
--
2.20.1
6 years
[libvirt] [PATCH] mingw: Bump minimum supported Fedora version to 28
by Andrea Bolognani
Should have been part of 2569ba133822, but clearly wasn't.
Signed-off-by: Andrea Bolognani <abologna(a)redhat.com>
---
mingw-libvirt.spec.in | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in
index bea822fd73..8756a56a23 100644
--- a/mingw-libvirt.spec.in
+++ b/mingw-libvirt.spec.in
@@ -3,7 +3,7 @@
# This spec file assumes you are building on a Fedora version
# that's still supported by the vendor. It may work on other distros
# or versions, but no effort will be made to ensure that going forward.
-%define min_fedora 26
+%define min_fedora 28
%if 0%{?fedora} && 0%{?fedora} >= %{min_fedora}
%define supported_platform 1
--
2.20.1
6 years
[libvirt] deadlock in remoteDispatchDomainUndefine vs daemonStreamHandleAbort
by Christian Ehrhardt
Hi,
I happened to analyze a bug [1] report I got from a friend and for
quite a while it was rather elusive. But I now finally got it
reproducible [2] enough to share it with the community.
The TL;DR of what I see is:
- an automation with python-libvirt gets a SIGINT
- cleanup runs destroy and further undefine
- the guest closes FDs due to SIGINT and/or destroy which triggers
daemonStreamHandleAbort
- those two fight over the lock
There I get libvirtd into a deadlock which ends up with all threads
dead [4] and two of them fighting [3] (details) in particular.
The to related stacks summarized are like:
daemonStreamHandleWrite (failing to write)
-> daemonStreamHandleAbort (closing things and cleaning up)
-> ... virChrdevFDStreamCloseCb
virMutexLock(&priv->devs->lock);
# there is code meant to avoid such issues emitting "Unable to close"
if a lock is held
# but the log doesn't show this triggering with debug enabled
#10 seems triggered via an "undefine" call
remoteDispatchDomainUndefine
... -> virChrdevFree
... -> virFDStreamSetInternalCloseCb
-> virObjectLock(virFDStreamDataPtr fdst)
-> virMutexLock(&obj->lock);
# closing all streams of a guest (requiring the same locks)
While that already feels quite close I struggle to see where exactly
we'd want to fix it.
But finally having a repro-script [2] I hope that someone else here
might be able to help me with that.
After all it is a race - on my s390x system it triggers usually <5
tries, while on x86 I have needed up to 18 runs of the test to hang.
Given different system configs it might be better or worse for you.
FYI we hit this with libvirt 4.0 initially but libvirt 5.0 was just the same.
I haven't built 5.1 or a recent master, but the commits since 5.0
didn't mention any issue that seems related. OTOH I'm willing and able
to build and try suggestions if anyone comes up with ideas.
[1]: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1822096
[2]: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1822096/+attachmen...
[3]: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1822096/comments/3
[4]: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1822096/comments/17
--
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd
6 years
[libvirt] [PATCH] mingw: unconditionally enable libssh2 in RPM
by Daniel P. Berrangé
The libssh2 support in libvirt is not solely for phyp, it is used by the
remote driver too.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
mingw-libvirt.spec.in | 2 --
1 file changed, 2 deletions(-)
diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in
index 9983a5013f..913aaa8751 100644
--- a/mingw-libvirt.spec.in
+++ b/mingw-libvirt.spec.in
@@ -85,10 +85,8 @@ BuildRequires: gettext-devel
BuildRequires: libtool
%endif
-%if %{with_phyp}
BuildRequires: mingw32-libssh2
BuildRequires: mingw64-libssh2
-%endif
%if %{with_esx}
BuildRequires: mingw32-curl
BuildRequires: mingw64-curl
--
2.20.1
6 years
[libvirt] [PATCH] mingw: remove obsolete Group tag in RPM spec
by Daniel P. Berrangé
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
mingw-libvirt.spec.in | 1 -
1 file changed, 1 deletion(-)
diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in
index bea822fd73..9983a5013f 100644
--- a/mingw-libvirt.spec.in
+++ b/mingw-libvirt.spec.in
@@ -39,7 +39,6 @@ Release: 1%{?dist}
Summary: MinGW Windows libvirt virtualization library
License: LGPLv2+
-Group: Development/Libraries
URL: https://libvirt.org/
%if %(echo %{version} | grep -q "\.0$"; echo $?) == 1
--
2.20.1
6 years
[libvirt] [PATCH v3 0/6] Facilitate running libvirt builds via docker containers
by Daniel P. Berrangé
For a while QEMU has provided simple make rules for building QEMU inside
standard docker container environments. This provides an equivalent
mechanism for libvirt inspired by QEMU's.
QEMU actually builds the container images on developer's machines
locally. Libvirt already hosts pre-built images on quay.io, so that is
used directly as it is quicker to download them than to build them
locally. This also ensures the container contents match what the live
CI system is using, as opposed to building an image with newer packages.
NB, there is a currently a test failure with this change to travis
that we don't currently see. Current travis runs a privileged
docker container, but with this change we now run unprivileged
containers. This shows that the "virsh-snapshot" test is trying
to create a file under $HOME, and docker blocks this. Obviously
needs fixing before pushing.
Changed in v3:
- Lots of new variables & variable renames
- Rename to top level Makefile.ci
- Rename make targets to have @ separator for the image
- Refactor macOS travis config
Changed in v2:
- Drop cross-compilation patches temporarily. The cross-compiler
images are not yet built on quay.io and rather than creating and
bulding many more images manually, I'm working on script to
automate setup of quay.io images
- Many changes to the make rules to make it possible to use them
from Travis CI to match its currently testing setup
- Modify Travis CI config to use the new make rules for consistency
Daniel P. Berrangé (6):
tests: don't abort in fopen(/proc/mounts)
tests: add targets for building libvirt inside docker containers
travis: convert ubuntu, centos & mingw builds to use new make rules
travis: use declarative syntax for homebrew packages
travis: remove display of test-suite.log from macOS
travis: put macOS script inline in the macOS matrix entry
.gitignore | 1 +
.travis.yml | 78 +++++-----------
Makefile.am | 2 +
Makefile.ci | 206 ++++++++++++++++++++++++++++++++++++++++++
tests/vircgroupmock.c | 6 +-
5 files changed, 235 insertions(+), 58 deletions(-)
create mode 100644 Makefile.ci
--
2.20.1
6 years
[libvirt] [PATCH] virjson: drop compatibility macros
by Ján Tomko
Since commit 66460e3 dropped support for YAJL 1, we no longer need
these.
Signed-off-by: Ján Tomko <jtomko(a)redhat.com>
---
src/util/virjson.c | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/src/util/virjson.c b/src/util/virjson.c
index bb4052ba81..1dceb746b9 100644
--- a/src/util/virjson.c
+++ b/src/util/virjson.c
@@ -34,8 +34,6 @@
# include <yajl/yajl_gen.h>
# include <yajl/yajl_parse.h>
-# define yajl_size_t size_t
-# define VIR_YAJL_STATUS_OK(status) ((status) == yajl_status_ok)
#endif
/* XXX fixme */
@@ -1615,7 +1613,7 @@ virJSONParserHandleBoolean(void *ctx,
static int
virJSONParserHandleNumber(void *ctx,
const char *s,
- yajl_size_t l)
+ size_t l)
{
virJSONParserPtr parser = ctx;
char *str;
@@ -1643,7 +1641,7 @@ virJSONParserHandleNumber(void *ctx,
static int
virJSONParserHandleString(void *ctx,
const unsigned char *stringVal,
- yajl_size_t stringLen)
+ size_t stringLen)
{
virJSONParserPtr parser = ctx;
virJSONValuePtr value = virJSONValueNewStringLen((const char *)stringVal,
@@ -1666,7 +1664,7 @@ virJSONParserHandleString(void *ctx,
static int
virJSONParserHandleMapKey(void *ctx,
const unsigned char *stringVal,
- yajl_size_t stringLen)
+ size_t stringLen)
{
virJSONParserPtr parser = ctx;
virJSONParserStatePtr state;
@@ -1824,7 +1822,7 @@ virJSONValueFromString(const char *jsonstring)
/* Yajl 2 is nice enough to default to rejecting trailing garbage. */
rc = yajl_parse(hand, (const unsigned char *)jsonstring, len);
- if (!VIR_YAJL_STATUS_OK(rc) ||
+ if (rc != yajl_status_ok ||
yajl_complete_parse(hand) != yajl_status_ok) {
unsigned char *errstr = yajl_get_error(hand, 1,
(const unsigned char*)jsonstring,
@@ -1935,7 +1933,7 @@ virJSONValueToBuffer(virJSONValuePtr object,
{
yajl_gen g;
const unsigned char *str;
- yajl_size_t len;
+ size_t len;
int ret = -1;
VIR_DEBUG("object=%p", object);
--
2.19.2
6 years
