[libvirt] [PATCH 0/6] Add disk streaming API to libvirt
by Adam Litke
I've been working with Anthony Liguori and Stefan Hajnoczi to enable data
streaming to copy-on-read disk images in qemu. This work is working its way
through review and I expect it to be upstream soon as part of the support for
the new QED disk image format.
Disk streaming is extremely useful when provisioning domains from a central
repository of template images. Currently the domain must be provisioned by
either: 1) copying the template image to local storage before the VM can be
started or, 2) creating a qcow2 image that backs to a base image in the remote
repository. Option 1 can introduce a significant delay when provisioning large
disks. Option 2 introduces a permanent dependency on a remote service and
increased network load to satisfy disk reads.
Device streaming provides the "instant-on" benefits of option 2 without
introducing a permanent dependency to the image repository. Once the VM is
started, the contents of the disk can be streamed to the local image in
parallel. Once streaming is finished, the domain has a complete and coherent
copy of the image and no longer depends on the central image repository.
Qemu will support two streaming modes: full device and single sector. Full
device streaming is the easiest to use because one command will cause the whole
device to be streamed as fast as possible. Single sector mode can be used if
one wants to throttle streaming to reduce I/O pressure. In this mode, a
management tool issues individual commands to stream single sectors.
To enable this support in libvirt, I propose the following API...
virDomainStreamDisk() will start or stop a full device stream or stream a
single sector of a device. The behavior is controlled by setting
virDomainStreamDiskFlags. When either starting or stopping a full device
stream, the return value is either 0 or -1 to indicate whether the operation
succeeded. For a single sector stream, a device offset is returned (or -1 on
failure). This value can be used to continue streaming with a subsequent call
to virDomainStreamDisk().
virDomainStreamDiskInfo() returns information about active full device streams
(the device alias, current streaming position, and total size).
Adam Litke (6):
Add new API virDomainStreamDisk[Info] to header and drivers
virDomainStreamDisk: Add public symbols to libvirt API
Implement disk streaming in the qemu driver
Add disk streaming support to the remote driver
Add new disk streaming commands to virsh
python: Add python bindings for virDomainStreamDisk[Info]
b/daemon/remote.c | 96 ++++++++++++++++++++
b/daemon/remote_dispatch_args.h | 2
b/daemon/remote_dispatch_prototypes.h | 16 +++
b/daemon/remote_dispatch_ret.h | 2
b/daemon/remote_dispatch_table.h | 10 ++
b/include/libvirt/libvirt.h.in | 34 +++++++
b/python/generator.py | 4
b/python/libvirt-override-api.xml | 5 +
b/python/libvirt-override.c | 46 +++++++++
b/src/driver.h | 11 ++
b/src/esx/esx_driver.c | 2
b/src/libvirt.c | 115 ++++++++++++++++++++++++
b/src/libvirt_public.syms | 5 +
b/src/lxc/lxc_driver.c | 2
b/src/openvz/openvz_driver.c | 2
b/src/phyp/phyp_driver.c | 2
b/src/qemu/qemu_driver.c | 77 +++++++++++++++-
b/src/qemu/qemu_monitor.c | 42 ++++++++
b/src/qemu/qemu_monitor.h | 6 +
b/src/qemu/qemu_monitor_json.c | 108 ++++++++++++++++++++++
b/src/qemu/qemu_monitor_json.h | 7 +
b/src/qemu/qemu_monitor_text.c | 162 ++++++++++++++++++++++++++++++++++
b/src/qemu/qemu_monitor_text.h | 8 +
b/src/remote/remote_driver.c | 87 +++++++++++++++++-
b/src/remote/remote_protocol.c | 63 +++++++++++++
b/src/remote/remote_protocol.h | 51 ++++++++++
b/src/remote/remote_protocol.x | 37 +++++++
b/src/test/test_driver.c | 2
b/src/uml/uml_driver.c | 2
b/src/vbox/vbox_tmpl.c | 2
b/src/vmware/vmware_driver.c | 2
b/src/xen/xen_driver.c | 2
b/tools/virsh.c | 134 +++++++++++++++++++++++++++-
python/generator.py | 3
src/qemu/qemu_driver.c | 2
src/remote/remote_driver.c | 2
36 files changed, 1144 insertions(+), 9 deletions(-)
13 years, 11 months
[libvirt] [PATCH] build: Fix problem of building Python bindings
by Osier Yang
If one specify "--with-python=yes" but no python-devel package
is installed, we ignore it with just a notice message, which
doesn't give clear guide to user.
---
configure.ac | 6 ++----
1 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/configure.ac b/configure.ac
index 190bf40..758c893 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1917,15 +1917,13 @@ if test "$with_python" != "no" ; then
then
PYTHON_INCLUDES=-I/usr/include/python$PYTHON_VERSION
else
- AC_MSG_NOTICE([Could not find python$PYTHON_VERSION/Python.h, disabling bindings])
- with_python=no
+ AC_MSG_ERROR([You must install python-devel to build Python bindings])
fi
fi
fi
fi
else
- AC_MSG_NOTICE([Could not find python interpreter, disabling bindings])
- with_python=no
+ AC_MSG_ERROR([You must install python to build Python bindings])
fi
else
AC_MSG_NOTICE([Could not find python in $with_python, disabling bindings])
--
1.7.4
13 years, 11 months
[libvirt] [PATCH 00/22] Extend remote generator to generate function bodies too
by Matthias Bolte
Richard W.M. Jones suggested [1] that the code that directly deals with the
XDR protocol should be generated. The remote_generate_stubs.pl script
already generates all the headers, just the bodies in the daemon and remote
driver are manually written. But most of the functions just follow simple
patterns. So I extended the generator to exploit this patterns and move
11 kLOC code from manually written to generated code.
During this I came a cross many small variations and problems in the XDR
protocol. For example, NWFilterDefineXML has a flags parameter in the public
API, but it's not transferred in the XDR protocol. Another things is the
variations in the usage of unsigned VS signed types. This comes in two forms.
public API VS XDR procotol and in between different functions. For example,
some functions use int for the flags paramater and some use unsigned int.
This results in quite a lot of special case handling in the generator.
cfg.mk | 10 +-
daemon/Makefile.am | 46 +-
daemon/qemu_dispatch_args.h | 2 +-
daemon/qemu_dispatch_bodies.c | 6 +
daemon/qemu_dispatch_prototypes.h | 2 +-
daemon/qemu_dispatch_ret.h | 2 +-
daemon/qemu_dispatch_table.h | 2 +-
daemon/remote.c | 5765 +----------------------------------
daemon/remote_dispatch_args.h | 2 +-
daemon/remote_dispatch_bodies.c | 5933 +++++++++++++++++++++++++++++++++++
daemon/remote_dispatch_prototypes.h | 80 +-
daemon/remote_dispatch_ret.h | 2 +-
daemon/remote_dispatch_table.h | 158 +-
daemon/remote_generate_stubs.pl | 195 --
daemon/remote_generator.pl | 1198 +++++++
po/POTFILES.in | 1 +
src/Makefile.am | 13 +-
src/remote/qemu_client_bodies.c | 4 +
src/remote/qemu_protocol.c | 2 +-
src/remote/qemu_protocol.h | 2 +-
src/remote/qemu_protocol.x | 2 +-
src/remote/remote_client_bodies.c | 4664 +++++++++++++++++++++++++++
src/remote/remote_driver.c | 4907 +----------------------------
src/remote/remote_protocol.c | 26 +-
src/remote/remote_protocol.h | 26 +-
src/remote/remote_protocol.x | 34 +-
src/remote_protocol-structs | 26 +-
27 files changed, 12093 insertions(+), 11017 deletions(-)
[1] https://www.redhat.com/archives/libvir-list/2011-April/msg00884.html
Matthias
13 years, 11 months
[libvirt] [PATCH] virsh: fix regression in log to file
by Supriya Kannery
Commit 36deff04 introduced a regression due to which virsh is not able
to log to a file - msg_buf was changed from an array to a pointer
without corresponding change to usage of "sizeof()".
Fix regression in virsh logging
Signed-off-by: Supriya Kannery <supriyak(a)in.ibm.com>
---
tools/virsh.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
Index: libvirt/tools/virsh.c
===================================================================
--- libvirt.orig/tools/virsh.c
+++ libvirt/tools/virsh.c
@@ -12257,7 +12257,7 @@ vshOutputLogFile(vshControl *ctl, int lo
*/
gettimeofday(&stTimeval, NULL);
stTm = localtime(&stTimeval.tv_sec);
- snprintf(msg_buf, sizeof(msg_buf),
+ snprintf(msg_buf, MSG_BUFFER,
"[%d.%02d.%02d %02d:%02d:%02d ",
(1900 + stTm->tm_year),
(1 + stTm->tm_mon),
@@ -12265,7 +12265,7 @@ vshOutputLogFile(vshControl *ctl, int lo
(stTm->tm_hour),
(stTm->tm_min),
(stTm->tm_sec));
- snprintf(msg_buf + strlen(msg_buf), sizeof(msg_buf) - strlen(msg_buf),
+ snprintf(msg_buf + strlen(msg_buf), MSG_BUFFER - strlen(msg_buf),
"%s] ", SIGN_NAME);
switch (log_level) {
case VSH_ERR_DEBUG:
@@ -12287,13 +12287,13 @@ vshOutputLogFile(vshControl *ctl, int lo
lvl = LVL_DEBUG;
break;
}
- snprintf(msg_buf + strlen(msg_buf), sizeof(msg_buf) - strlen(msg_buf),
+ snprintf(msg_buf + strlen(msg_buf), MSG_BUFFER - strlen(msg_buf),
"%s ", lvl);
- vsnprintf(msg_buf + strlen(msg_buf), sizeof(msg_buf) - strlen(msg_buf),
+ vsnprintf(msg_buf + strlen(msg_buf), MSG_BUFFER - strlen(msg_buf),
msg_format, ap);
if (msg_buf[strlen(msg_buf) - 1] != '\n')
- snprintf(msg_buf + strlen(msg_buf), sizeof(msg_buf) -
strlen(msg_buf), "\n");
+ snprintf(msg_buf + strlen(msg_buf), MSG_BUFFER -
strlen(msg_buf), "\n");
/* write log */
if (safewrite(ctl->log_fd, msg_buf, strlen(msg_buf)) < 0) {
13 years, 11 months
[libvirt] [PATCH 0/6] Introduce a new migration protocol to QEMU driver
by Daniel P. Berrange
The current migration protocol has several flaws
- No initial hook on the source host to do work before
the dst VM is launched
- No ability to restart src VM if dst fails to recv all
migration data, but src successfully sent it all
This introduces a new 5 step migration process to address
this limitation. To support features such as seemless
migration of SPICE clients, and lock driver state passing
this now makes use of the migration cookie feature too
13 years, 11 months
[libvirt] [PATCH] lxc: Do not try to reconnect inactive domain when do lxcStartup
by Osier Yang
Otherwise if there are inactive lxc domains, lxcStartup will
try to reconnect to sockets of these domains, which results in
errors in libvirtd log.
---
src/lxc/lxc_driver.c | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c
index e905302..ef7827b 100644
--- a/src/lxc/lxc_driver.c
+++ b/src/lxc/lxc_driver.c
@@ -1992,6 +1992,9 @@ lxcReconnectVM(void *payload, const void *name ATTRIBUTE_UNUSED, void *opaque)
virDomainObjLock(vm);
+ if (!virDomainObjIsActive(vm))
+ goto cleanup;
+
priv = vm->privateData;
if ((priv->monitor = lxcMonitorClient(driver, vm)) < 0) {
goto cleanup;
--
1.7.4
13 years, 11 months
[libvirt] [PATCH] tests: Update valgrind suppressions file
by Matthias Bolte
---
tests/.valgrind.supp | 58 +++++++++++++++++++++++++++++++++++++++++++------
1 files changed, 51 insertions(+), 7 deletions(-)
diff --git a/tests/.valgrind.supp b/tests/.valgrind.supp
index 4af10b1..68cfa0c 100644
--- a/tests/.valgrind.supp
+++ b/tests/.valgrind.supp
@@ -258,14 +258,58 @@
Memcheck:Param
capget(data)
fun:capget
+ fun:*
fun:capng_clear
+ fun:virClearCapabilities
fun:__virExec
fun:virExecWithHook
- fun:virExec
- fun:qemudProbeMachineTypes
- fun:qemudCapsInitGuest
- fun:qemudCapsInit
- fun:qemudStartup
- fun:virStateInitialize
- fun:main
+}
+{
+ libnlMemoryLeak1
+ Memcheck:Leak
+ fun:malloc
+ fun:strdup
+ obj:/usr/lib/libnl.so.1.1
+}
+{
+ libnlMemoryLeak2
+ Memcheck:Leak
+ fun:calloc
+ obj:/usr/lib/libnl.so.1.1
+}
+{
+ libselinuxMemoryLeak1
+ Memcheck:Leak
+ fun:malloc
+ fun:getdelim
+ obj:/lib/libselinux.so.1
+}
+{
+ dashMemoryLeak1
+ Memcheck:Leak
+ fun:malloc
+ obj:/bin/dash
+}
+{
+ dashMemoryLeak2
+ Memcheck:Leak
+ fun:malloc
+ fun:strdup
+ obj:/bin/dash
+}
+{
+ vboxMemoryLeak1
+ Memcheck:Leak
+ ...
+ fun:VBoxNsxpNS_InitXPCOM2
+}
+{
+ libnetcfMemoryLeak1
+ fun:malloc
+ fun:xmlStrndup
+ fun:xmlHashUpdateEntry3
+ fun:*
+ fun:xsltRegisterAllExtras
+ fun:drv_init
+ fun:interfaceOpenInterface
}
--
1.7.0.4
13 years, 11 months
[libvirt] [PATCH v2] Fix disability to run on systems with no PCI bus
by Michal Privoznik
The patch which moved libpciaccess initialization to one place caused
regression - we were not able to run on system with no PCI bus, like
s390(x).
---
src/node_device/node_device_udev.c | 9 +++++++++
1 files changed, 9 insertions(+), 0 deletions(-)
diff --git a/src/node_device/node_device_udev.c b/src/node_device/node_device_udev.c
index 2139ef3..fcff252 100644
--- a/src/node_device/node_device_udev.c
+++ b/src/node_device/node_device_udev.c
@@ -1421,8 +1421,12 @@ static int udevDeviceMonitorShutdown(void)
ret = -1;
}
+#if defined __s390__ || defined __s390x_
+ /* Nothing was initialized, nothing needs to be cleaned up */
+#else
/* pci_system_cleanup returns void */
pci_system_cleanup();
+#endif
return ret;
}
@@ -1595,6 +1599,10 @@ static int udevDeviceMonitorStartup(int privileged)
int ret = 0;
int pciret;
+#if defined __s390__ || defined __s390x_
+ /* On x390(x) system there is no PCI bus.
+ * Therefore there is nothing to initialize here. */
+#else
if ((pciret = pci_system_init()) != 0) {
/* Ignore failure as non-root; udev is not as helpful in that
* situation, but a non-privileged user won't benefit much
@@ -1607,6 +1615,7 @@ static int udevDeviceMonitorStartup(int privileged)
goto out;
}
}
+#endif
if (VIR_ALLOC(priv) < 0) {
virReportOOMError();
--
1.7.4.4
13 years, 11 months
[libvirt] Libvirt and IPSec (was: What about Trusted Virtual Domains???)
by Paolo Smiraglia
Hi to everyone! First of all, sorry for the thread subject change.
Due to the several issues of the Libvirt implementation of the Trusted
Virtual Domains (TVD), I decided to approach the topic in a modular manner.
I think that the first step should be to define the IPSec support or,
more in general, the secure tunnel support for Libvirt. I see the
implementation divided in two step:
1. define a new driver called 'sectunnel' which describes a generic
secure tunnel that will be established using several
technologies (for now using only ipsec)
2. modify the existing 'network' driver by adding the possibility to
specify the 'sectunnel' that
the network have to use in the virtual network definition
As an example, you can see below a possible XML definition of the
network which use a secure tunnel and the corresponding 'sectunnel' XML
definition:
NETWORK DEFINITION
==================
<network>
<name>sec-net</name>
<uuid>3e3fce45-4f53-4fa7-bb32-11f34168b82b</uuid>
<bridge name='virbr0' />
<domain name='example' />
...
<sectunnel name='sec-tun' /> <--(specify the 'sectunnel' to use)
</network>
SECTUNNEL DEFINITION
====================
<sectunnel type='ipsec'>
<name>sec-tun</name>
<uuid>8b7fd1b0-4463-43b7-8b6e-8006344aeb66</uuid>
<!-- Security Association definitions -->
<sa>
<secret uuid='...' /> <--(specify the 'secret' which
contains the pre-shared key)
</sa>
<!-- Security Policy definitions -->
<sp>
<src_range address='10.0.0.1' prefixlen='30' port='5000' />
<dst_range address='10.0.0.2' prefixlen='30' port='5000' />
<upperspec protocol='any' />
<policy direction='out' action='ipsec'>
<rule protocol='esp' mode='tunnel' level='require'>
<src address='192.168.0.1' port='55055' />
<dst address='192.168.0.2' port='55055' />
</rule>
</policy>
</sp>
<sp>
<src_range address='10.0.0.2' prefixlen='30' port='5000' />
<dst_range address='10.0.0.1' prefixlen='30' port='5000' />
<upperspec protocol='any' />
<policy direction='in' action='ipsec'>
<rule protocol='esp' mode='tunnel' level='require'>
<src address='192.168.0.2' port='55055' />
<dst address='192.168.0.1' port='55055' />
</rule>
</policy>
</sp>
</sectunnel>
As you can see in the 'sectunnel' XML definition, I use a 'secret'
element. This element is a Libvirt secret [1] and it stores the
pre-shared key used by IPSec to establish the Security Associations
(SA). Obviously this feature requires to define a new usage category in
the 'secret' driver definition.
Another possible way to establish the SA is to use the X.509
certificates. To this purpose, I think that the certificates already
used by Libvirt to setup SSL/TLS remote connections, might be used.
That's all! :-)
What do you think about this possible IPSec implementation?
Thanks in advance for the replies!
Best regards,
PAOLO
LINK LIST
---------
[1] http://libvirt.org/formatsecret.html
--
PAOLO SMIRAGLIA
Department of Control and Computer Engineering
Mobile: +39 (333) 527 3593
Email: paolo.smiraglia(a)polito.it
13 years, 11 months
