- Devel - libvirt List Archives

[libvirt] [RFC] Proposed API to support block device streaming
by Adam Litke 15 Nov '10

15 Nov '10

I've been working with Anthony Liguori and Stefan Hajnoczi to enable data streaming to copy-on-read disk images in qemu. This work is working its way through peer review and I expect it to be upstream soon as part of the support for the new QED disk image format. I would like to enable these commands in libvirt in order to support at least two compelling use cases: 1) Rapid deployment of domains: Creating a new domain from a central repository of images can be time consuming since a local copy of the image must be made before the domain can be started. With copy-on-read and streaming, up-front copy time is eliminated and the domain can be started immediately. Streaming can run while the domain runs to fully populate the disk image. 2) Post-copy live block migration: A qemu-nbd server is started on the source host and serves the domain's block device to the destination host. A QED image is created on the destination host with backing to the nbd server. The domain is migrated as normal. When migration completes, a stream command is executed to fully populate the destination QED image. After streaming completes, the qemu-nbd server can be shut down and the domain (including local storage) is fully independent of the source host. Qemu will support two streaming modes: full device and single sector. Full device streaming is the easiest to use because one command will cause the whole device to be streamed as fast as possible. Single sector mode can be used if one wants to throttle streaming to reduce I/O pressure. In this mode, the user issues individual commands to stream single sectors. To enable this support in libvirt, I propose the following API... virDomainStreamDisk() initiates either a full device stream or a single sector stream (depending on virDomainStreamDiskFlags). For a full device stream, it returns either 0 or -1. For a single sector stream, it returns an offset that can be used to continue streaming with a subsequent call to virDomainStreamDisk(). virDomainStreamDiskInfo() returns the status of a currently-running full device stream (the device name, current streaming position, and total size). Comments on this design would be greatly appreciated. Thanks! diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index 81db3a2..d80a8b5 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -1046,6 +1046,39 @@ int virDomainUpdateDeviceFlags(virDomainPtr domain, const char *xml, unsigned int flags); /* + * Disk Streaming + */ +typedef enum { + VIR_STREAM_DISK_FULL = 1, /* Stream the entire disk */ + VIR_STREAM_DISK_ONE = 2, /* Stream a single disk unit */ +} virDomainStreamDiskFlags; + +#define VIR_STREAM_PATH_BUFLEN 100 +#define VIR_STREAM_DISK_MAX_STREAMS 10 + +typedef struct _virStreamDiskState virStreamDiskState; +struct _virStreamDiskState { + char path[VIR_STREAM_PATH_BUFLEN]; + /* + * The unit of measure for size and offset is unspecified. These fields + * are meant to indicate the progress of a continuous streaming operation. + */ + unsigned long long offset; /* Current offset of active streaming */ + unsigned long long size; /* Disk size */ +}; +typedef virStreamDiskState *virStreamDiskStatePtr; + +unsigned long long virDomainStreamDisk(virDomainPtr dom, + const char *path, + unsigned long long offset, + unsigned int flags); + +int virDomainStreamDiskInfo(virDomainPtr dom, + virStreamDiskStatePtr infos, + unsigned int nr_infos, + unsigned int flags); + +/* * NUMA support */ -- Thanks, Adam

5 8

[libvirt] Ask for help about libvirt error: this function is not supported by the connection driver..
by Feng Dan Liu 15 Nov '10

15 Nov '10

Hi, I'm a libvirt user, I encountered a problem when I use the libvirt java bindings, could you please help me take a look？ Thanks in advance! Any suggestions from anyone of you will be very appreciated. Here is my scenario: Remote access machine A is a RHEL 5.4 workstation with the command "virsh version" results: Compiled against library: libvir 0.8.1 Using library: libvir 0.8.1 Using API: Xen 3.0.1 Running hypervisor: Xen 3.1.0 Hypervisor machine B is a Fedora Core 14 Linux workstation with the command "virsh version" results: Compiled against library: libvir 0.8.3 Using library: libvir 0.8.3 Using API: QEMU 0.8.3 Running hypervisor: QEMU 0.13.0 I finished the code using libvirt java bindings and run it on machine A, try to remote connect to the hypervisor on machine B. It can be connected succussfully, but when the code calls the function of conn.listDefinedStoragePools() or conn.listStoragePools, the libvirt on machine B will get the errors below: 04:19:10.467: error : phypStorageOpen:3884 : invalid argument in phypStorageOpen: unsupported flags (0x1) 04:19:11.354: warning : qemudParsePCIDeviceStrs:1422 : Unexpected exit status '1', qemu probably failed 04:19:12.128: error : virLibConnError:448 : this function is not supported by the connection driver: virConnectNumOfStoragePools With the errors above, my code can't run the functions of conn.listDefinedStoragePools() or conn.listStoragePools successfully. Could you please give me some suggestions? By the way, I try to use the libvirt java bindings 0.3.2 and 0.4.6, both of them have this problem. Best Regards, Fengdan Liu, Casey AdTech, Tivoli China Development, China Software Development Lab, IBM Greater China Group Phone: 86-10-82453004 Email: liufengd(a)cn.ibm.com Addr: No.28 Building, Zhong Guan Cun Software Park, No. 8 Dong Bei Wang West Road, ShangDi, Haidian District, Beijing 100193, P.R.China

2 2

[libvirt] [PATCH] Use python discovered through env instead of hardcoding a path
by Matthias Bolte 14 Nov '10

14 Nov '10

This is more flexible regarding the location of the python binary but doesn't allow to pass the -u flag. The -i flag can be passed from inside the script using the PYTHONINSPECT env variable. This fixes a problem with the esx_vi_generator.py on FreeBSD. --- examples/python/dominfo.py | 2 +- examples/python/domrestore.py | 2 +- examples/python/domsave.py | 2 +- examples/python/domstart.py | 2 +- examples/python/esxlist.py | 2 +- python/generator.py | 2 +- python/libvirt-override.py | 5 +++++ src/esx/esx_vi_generator.py | 2 +- 8 files changed, 12 insertions(+), 7 deletions(-) diff --git a/examples/python/dominfo.py b/examples/python/dominfo.py index e41709c..2f250e7 100755 --- a/examples/python/dominfo.py +++ b/examples/python/dominfo.py @@ -1,4 +1,4 @@ -#! /usr/bin/python +#!/usr/bin/env python # dominfo - print some information about a domain import libvirt diff --git a/examples/python/domrestore.py b/examples/python/domrestore.py index b032138..ccc82f6 100755 --- a/examples/python/domrestore.py +++ b/examples/python/domrestore.py @@ -1,4 +1,4 @@ -#! /usr/bin/python +#!/usr/bin/env python # domstart - make sure a given domU is running, if not start it import libvirt diff --git a/examples/python/domsave.py b/examples/python/domsave.py index 35e2c8a..12ffac1 100755 --- a/examples/python/domsave.py +++ b/examples/python/domsave.py @@ -1,4 +1,4 @@ -#! /usr/bin/python +#!/usr/bin/env python # domstart - make sure a given domU is running, if not start it import libvirt diff --git a/examples/python/domstart.py b/examples/python/domstart.py index 52fb79e..b1e89b8 100755 --- a/examples/python/domstart.py +++ b/examples/python/domstart.py @@ -1,4 +1,4 @@ -#! /usr/bin/python +#!/usr/bin/env python # domstart - make sure a given domU is running, if not start it import libvirt diff --git a/examples/python/esxlist.py b/examples/python/esxlist.py index a0602e8..c55424f 100755 --- a/examples/python/esxlist.py +++ b/examples/python/esxlist.py @@ -1,4 +1,4 @@ -#! /usr/bin/python +#!/usr/bin/env python # esxlist - list active domains of an ESX host and print some info. # also demonstrates how to use the libvirt.openAuth() method diff --git a/python/generator.py b/python/generator.py index 68009b9..15751bd 100755 --- a/python/generator.py +++ b/python/generator.py @@ -1010,7 +1010,7 @@ def buildWrappers(): classes = open("libvirt.py", "w") extra = open(os.path.join(srcPref,"libvirt-override.py"), "r") - classes.write("#!/usr/bin/python -i\n") + classes.write("#!/usr/bin/env python\n") classes.write("#\n") classes.write("# WARNING WARNING WARNING WARNING\n") classes.write("#\n") diff --git a/python/libvirt-override.py b/python/libvirt-override.py index d544a0e..909ebe3 100644 --- a/python/libvirt-override.py +++ b/python/libvirt-override.py @@ -2,6 +2,11 @@ # Manually written part of python bindings for libvirt # +# Specify -i commandline option after python was started +if __name__ == "__main__": + import os + os.environ["PYTHONINSPECT"] = "1" + # On cygwin, the DLL is called cygvirtmod.dll try: import libvirtmod diff --git a/src/esx/esx_vi_generator.py b/src/esx/esx_vi_generator.py index 01636aa..4593379 100755 --- a/src/esx/esx_vi_generator.py +++ b/src/esx/esx_vi_generator.py @@ -1,4 +1,4 @@ -#!/usr/bin/python +#!/usr/bin/env python # # esx_vi_generator.py: generates most of the SOAP type mapping code -- 1.7.0.4

2 2

[libvirt] [PATCH] esx: Avoid warnings about breaking strict-aliasing rules on FreeBSD
by Matthias Bolte 14 Nov '10

14 Nov '10

--- src/esx/esx_vi_generator.py | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/src/esx/esx_vi_generator.py b/src/esx/esx_vi_generator.py index c2f7044..01636aa 100755 --- a/src/esx/esx_vi_generator.py +++ b/src/esx/esx_vi_generator.py @@ -699,7 +699,10 @@ class Object: if self.features & Object.FEATURE__LIST: if self.extends is not None: - source += " esxVI_%s_Free((esxVI_%s **)&item->_next);\n\n" % (self.extends, self.extends) + # avoid "dereferencing type-punned pointer will break strict-aliasing rules" warnings + source += " esxVI_%s *next = (esxVI_%s *)item->_next;\n\n" % (self.extends, self.extends) + source += " esxVI_%s_Free(&next);\n" % self.extends + source += " item->_next = (esxVI_%s *)next;\n\n" % self.name else: source += " esxVI_%s_Free(&item->_next);\n\n" % self.name @@ -719,7 +722,10 @@ class Object: if self.features & Object.FEATURE__LIST: if self.extends is not None: - source += " esxVI_%s_Free((esxVI_%s **)&item->_next);\n\n" % (self.extends, self.extends) + # avoid "dereferencing type-punned pointer will break strict-aliasing rules" warnings + source += " esxVI_%s *next = (esxVI_%s *)item->_next;\n\n" % (self.extends, self.extends) + source += " esxVI_%s_Free(&next);\n" % self.extends + source += " item->_next = (esxVI_%s *)next;\n\n" % self.name else: source += " esxVI_%s_Free(&item->_next);\n\n" % self.name -- 1.7.0.4

2 2

[libvirt] [PATCH] Make sure struct sockaddr_in is defined on FreeBSD
by Matthias Bolte 14 Nov '10

14 Nov '10

--- src/util/network.h | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/src/util/network.h b/src/util/network.h index 0900e29..724a5f8 100644 --- a/src/util/network.h +++ b/src/util/network.h @@ -21,6 +21,10 @@ # include <netdb.h> # include <stdbool.h> +# ifdef __FreeBSD__ +# include <netinet/in.h> +# endif + typedef struct { union { struct sockaddr sa; -- 1.7.0.4

2 2

[libvirt] [PATCH] configure: Disable FS storage driver if mntent.h is not available
by Matthias Bolte 14 Nov '10

14 Nov '10

This is the case on FreeBSD. --- configure.ac | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/configure.ac b/configure.ac index 66237e1..d21d558 100644 --- a/configure.ac +++ b/configure.ac @@ -1497,6 +1497,18 @@ if test "$with_osx" = "yes"; then fi if test "$with_storage_fs" = "yes" || test "$with_storage_fs" = "check"; then + AC_CHECK_HEADER([mntent.h],, + [ + if test "$with_storage_fs" = "check"; then + with_storage_fs=no + AC_MSG_NOTICE([<mntent.h> is required for the FS storage driver, disabling it]) + else + AC_MSG_ERROR([<mntent.h> is required for the FS storage driver]) + fi + ]) +fi + +if test "$with_storage_fs" = "yes" || test "$with_storage_fs" = "check"; then AC_PATH_PROG([MOUNT], [mount], [], [$PATH:/sbin:/usr/sbin]) AC_PATH_PROG([UMOUNT], [umount], [], [$PATH:/sbin:/usr/sbin]) if test "$with_storage_fs" = "yes" ; then -- 1.7.0.4

2 2

[libvirt] [PATCH] daemon: Include stdlib.h in dispatch.c
by Matthias Bolte 14 Nov '10

14 Nov '10

Otherwise GCC complains about malloc being unknown on FreeBSD. --- daemon/dispatch.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/daemon/dispatch.c b/daemon/dispatch.c index 9731614..bf2ac73 100644 --- a/daemon/dispatch.c +++ b/daemon/dispatch.c @@ -24,6 +24,7 @@ #include <config.h> #include <stdio.h> +#include <stdlib.h> #include <stdarg.h> #include <stdbool.h> -- 1.7.0.4

2 2

Re: [libvirt] Is there an API for clone?
by Alex Jia 14 Nov '10

14 Nov '10

Hi lancer, As Osier said, libvirt hasn't a independent API for cloning an existing guest, instead of using virStorageVolGetXMLDesc API to get a guest volume xml description and then calling virStorageVolCreateXMLFrom API to clone a new volume. For python-virtinst component, you may use virt-clone to implement this, for libvirt component, you may directly use python binding API or virsh tool 'vol-clone' to implement it. I hope it's helpful for you. - Alex ----- Original Message ----- From: "黄亮" <lancerhuang(a)163.com> To: "libvir-list" <libvir-list(a)redhat.com> Sent: Wednesday, November 10, 2010 9:30:28 AM GMT +08:00 Beijing / Chongqing / Hong Kong / Urumqi Subject: [libvirt] Is there an API for clone? Hi, guys I know there is a command "virt-clone", but I couldn't find the equivalent API here : http://libvirt.org/html/libvirt-libvirt.html So, is there an API to clone an existing VM or not? Thank you! Regards 2010-11-10 Lancer -- libvir-list mailing list libvir-list(a)redhat.com https://www.redhat.com/mailman/listinfo/libvir-list

3 3

[libvirt] CSharp bindings patch
by arnaud.champion＠devatom.fr 14 Nov '10

14 Nov '10

?Hi, here is a patch because I have forgotten to include comments on sample Regards, Arnaud

1 0

[libvirt] Fw: CSharp bindings patch
by arnaud.champion＠devatom.fr 14 Nov '10

14 Nov '10

?Hi, here is a 3rd patch that add a sample named "virDomainStats" Regards, Arnaud

1 0