October 2015 - Devel - libvirt List Archives

[libvirt] [PATCH RFC 0/7] libxl: domain statistics support
by Joao Martins 14 Oct '15

14 Oct '15

Hey Jim, This series bring support for various statistics about domains regarding CPU, Memory, Network Interfaces and BlockStats. Not all of the statistics are implemented: qdisk support is missing in this series and some of the memory statistics aren't available. With this series we further implement 7 more functions of libvirt APIs. It is organized as follows: * Patch 1, 2: implements cpu/memory statistics. * Patch 3, 4: implements (netback) network statistics and VBD block statistics. QDisk will follow up in a separate series regarding QEMU monitor integration. * Patch 5: implement fetching all domain statistics * Patch 6, 7: implements Job information. Overall it looks big but 70% of the patch is due to #4 and #5 but doesn't add necessarily more complexity to the driver I believe. Patch #6 and #7 are of special importance because GetJobInfo and GetJobStats are now used in Openstack Kilo to monitor live-migration progress. This two patches together with an earlier series [0] I sent before let us sucessfully live-migrate with Openstack Kilo. Further with this series we get to support nova diagnostics. Tested this series on 4.4.3 and 4.5 setups plus Openstack Kilo. Any comments or suggestions are welcome, Thanks! Joao Joao Martins (7): libxl: implement virDomainGetCPUStats libxl: implement virDomainMemorystats libxl: implement virDomainInterfaceStats libxl: implement virDomainBlockStats libxl: implement virConnectGetAllDomainStats libxl: implement virDomainGetJobInfo libxl: implement virDomainGetJobStats configure.ac | 2 +- src/libxl/libxl_domain.c | 29 ++ src/libxl/libxl_domain.h | 6 + src/libxl/libxl_driver.c | 1010 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 1046 insertions(+), 1 deletion(-) -- 2.1.4

3 27

[libvirt] libvirt-php and xpath function from libxml2
by Vasiliy Tolstov 14 Oct '15

14 Oct '15

Hi! I'm test under high load some libvirt-php xpath based function and big memory leak under each function call. I'm try to patch libvirt-php to free some values via xmlFree , but this is not solve leaks. Do we realy need this function? As i see when i'm use plain libvirt function and use returned xml with internal php functions that works with xml and xpath, i'm not get memory leaks. So does we really need it? -- Vasiliy Tolstov, e-mail: v.tolstov(a)selfip.ru

2 3

[libvirt] [PATCH] build: get further on cygwin 'make check'
by Eric Blake 14 Oct '15

14 Oct '15

Based on how we generate protocol-structs files for checking that our protocol changes are backwards-compatible, we end up compiling lxc and lockd protocol files on all platforms during 'make check', even though we may never link those files into a final library or executable. For this to work on cygwin, the CFLAGS addition of $(XDR_CFLAGS) must be unconditional. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- I'll wait for review rather than invoking build-breaker rule, as 'make check' is a bit stronger than 'make'. src/Makefile.am | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 99b4993..e2c3e47 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -1305,6 +1305,19 @@ EXTRA_DIST += qemu/qemu.conf qemu/libvirtd_qemu.aug \ qemu/test_libvirtd_qemu.aug.in qemu/THREADS.txt +libvirt_driver_lxc_impl_la_CFLAGS = \ + $(LIBNL_CFLAGS) \ + $(FUSE_CFLAGS) \ + $(XDR_CFLAGS) \ + -I$(srcdir)/access \ + -I$(srcdir)/conf \ + $(AM_CFLAGS) +libvirt_driver_lxc_impl_la_LIBADD = \ + $(CAPNG_LIBS) \ + $(LIBNL_LIBS) \ + $(LIBXML_LIBS) \ + $(FUSE_LIBS) + if WITH_LXC noinst_LTLIBRARIES += libvirt_driver_lxc_impl.la libvirt_driver_lxc_la_SOURCES = @@ -1319,19 +1332,6 @@ noinst_LTLIBRARIES += libvirt_driver_lxc.la #libvirt_la_BUILT_LIBADD += libvirt_driver_lxc.la endif ! WITH_DRIVER_MODULES -libvirt_driver_lxc_impl_la_CFLAGS = \ - $(LIBNL_CFLAGS) \ - $(FUSE_CFLAGS) \ - $(XDR_CFLAGS) \ - -I$(srcdir)/access \ - -I$(srcdir)/conf \ - $(AM_CFLAGS) -libvirt_driver_lxc_impl_la_LIBADD = \ - $(CAPNG_LIBS) \ - $(LIBNL_LIBS) \ - $(LIBXML_LIBS) \ - $(FUSE_LIBS) - if WITH_BLKID libvirt_driver_lxc_impl_la_CFLAGS += $(BLKID_CFLAGS) libvirt_driver_lxc_impl_la_LIBADD += $(BLKID_LIBS) @@ -2307,15 +2307,16 @@ endif WITH_LXC lockdriverdir = $(libdir)/libvirt/lock-driver lockdriver_LTLIBRARIES = +lockd_la_CFLAGS = -I$(srcdir)/conf \ + $(XDR_CFLAGS) \ + $(AM_CFLAGS) + if WITH_LIBVIRTD lockdriver_LTLIBRARIES += lockd.la lockd_la_SOURCES = \ $(LOCK_DRIVER_LOCKD_SOURCES) \ $(LOCK_PROTOCOL_GENERATED) \ $(NULL) -lockd_la_CFLAGS = -I$(srcdir)/conf \ - $(XDR_CFLAGS) \ - $(AM_CFLAGS) lockd_la_LDFLAGS = -module -avoid-version lockd_la_LIBADD = ../gnulib/lib/libgnu.la \ libvirt-net-rpc.la \ -- 2.4.3

2 1

[libvirt] [PATCH] build: include xdr cflags for libvirt-admin
by Eric Blake 14 Oct '15

14 Oct '15

Without this, building on cygwin fails with: CC libvirt_admin_la-libvirt-admin.lo libvirt-admin.c:25:21: fatal error: rpc/rpc.h: No such file or directory #include <rpc/rpc.h> ^ Reported by Yaakov Selkowitz <yselkowi(a)redhat.com> Signed-off-by: Eric Blake <eblake(a)redhat.com> --- Pushing under the build-breaker rule. src/Makefile.am | 1 + 1 file changed, 1 insertion(+) diff --git a/src/Makefile.am b/src/Makefile.am index d5dd9fb..99b4993 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -2087,6 +2087,7 @@ libvirt_admin_la_CFLAGS = \ -I$(srcdir)/admin libvirt_admin_la_CFLAGS += \ + $(XDR_CFLAGS) \ $(CAPNG_CFLAGS) \ $(YAJL_CFLAGS) \ $(SSH2_CFLAGS) \ -- 2.4.3

1 0

[libvirt] [RFC] Automatically sync time on domain resume?
by Michal Privoznik 14 Oct '15

14 Oct '15

So I came across this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1156194 There's a request for us to automatically sync guest time on domain resume (e.g. bare virDomainResume(), or virDomainRestore() or after migration). I'd like to explore our options here and what would be acceptable upstream. I can think of two possibilities: 1) Invent new flag to all APIs in question and let mgmt applications call it. If the flag is set we would sync the guest time too. However, this masks two different operations under single API. What should be reported if resuming succeeded but syncing time did not? Success or failure? Mgmt applications will end up calling two APIs to distinguish error states anyway. 2) Do this as best effort. Unconditionally, whenever guest agent is available just issue the time sync command (possibly without waiting for reply). This, however, takes away control from users - they will no longer have possibility to just resume domain without time sync. There's of course the obvious solution - not change anything and have mgmt apps calling two separate APIs - like they oughtta be doing today. What's your view? Michal

4 4

[libvirt] [PATCH 1/3] storage: add gluster backend initialization support for multiple servers
by Prasanna Kumar Kalever 14 Oct '15

14 Oct '15

This patch adds support for initialization of gluster backend with multiple servers which acts as gluster volfile servers for the gluster storage backend. This will help in achieving high availability of gluster backend connectivity via libgfapi i.e. when the first volfile server fails, then other servers specified are used as volfile server to re-establish the backend gluster connectivity. Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever(a)redhat.com> --- src/storage/storage_backend_gluster.c | 77 +++++++++++++++++++---------------- 1 file changed, 41 insertions(+), 36 deletions(-) diff --git a/src/storage/storage_backend_gluster.c b/src/storage/storage_backend_gluster.c index d2e79bc..53474c1 100644 --- a/src/storage/storage_backend_gluster.c +++ b/src/storage/storage_backend_gluster.c @@ -32,6 +32,9 @@ #include "virstring.h" #include "viruri.h" +#define QEMU_DEFAULT_GLUSTER_PORT 24007 +#define QEMU_DEFAULT_GLUSTER_TRANSPORT "tcp" + #define VIR_FROM_THIS VIR_FROM_STORAGE VIR_LOG_INIT("storage.storage_backend_gluster"); @@ -570,62 +573,64 @@ static int virStorageFileBackendGlusterInit(virStorageSourcePtr src) { virStorageFileBackendGlusterPrivPtr priv = NULL; - virStorageNetHostDefPtr host = &(src->hosts[0]); - const char *hostname; + const char *transport = NULL; int port = 0; + size_t i = 0; - if (src->nhosts != 1) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Expected exactly 1 host for the gluster volume")); - return -1; - } - - hostname = host->name; - - VIR_DEBUG("initializing gluster storage file %p (gluster://%s:%s/%s%s)[%u:%u]", - src, hostname, host->port ? host->port : "0", - NULLSTR(src->volume), src->path, - (unsigned int)src->drv->uid, (unsigned int)src->drv->gid); + VIR_DEBUG("Initializing gluster volume=%s image-path=%s with uid=%u gid=%u", + NULLSTR(src->volume), src->path, + (unsigned int)src->drv->uid, (unsigned int)src->drv->gid); if (!src->volume) { virReportError(VIR_ERR_INTERNAL_ERROR, - _("missing gluster volume name for path '%s'"), - src->path); + _("missing gluster volume name for path '%s'"), + src->path); return -1; } if (VIR_ALLOC(priv) < 0) return -1; - if (host->port && - virStrToLong_i(host->port, NULL, 10, &port) < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("failed to parse port number '%s'"), - host->port); - goto error; - } - - if (host->transport == VIR_STORAGE_NET_HOST_TRANS_UNIX) - hostname = host->socket; - if (!(priv->vol = glfs_new(src->volume))) { virReportOOMError(); goto error; } - if (glfs_set_volfile_server(priv->vol, - virStorageNetHostTransportTypeToString(host->transport), - hostname, port) < 0) { - virReportSystemError(errno, - _("failed to set gluster volfile server '%s'"), - hostname); - goto error; + for (i = 0; i < src->nhosts; i++) { + if (!src->hosts[i].port) { + port = QEMU_DEFAULT_GLUSTER_PORT; + } else if (virStrToLong_i(src->hosts[i].port, NULL, 10, &port) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("failed to parse port number '%s'"), src->hosts[i].port); + goto error; + } + + if (src->hosts[i].transport == VIR_STORAGE_NET_HOST_TRANS_UNIX) { + transport = src->hosts[i].socket; + } else if (!src->hosts[i].transport) { + transport = QEMU_DEFAULT_GLUSTER_TRANSPORT; + } else { + transport = virStorageNetHostTransportTypeToString(src->hosts[i].transport); + } + + if (glfs_set_volfile_server(priv->vol, transport, src->hosts[i].name, + port) < 0) { + virReportSystemError(errno, + _("failed to set gluster volfile server '%s'"), + src->hosts[i].name); + goto error; + } else { + VIR_DEBUG("Successfully set volfile server with server:'%s' " + "port:'%d' transport:'%s'", src->hosts[i].name, + port, transport); + } } if (glfs_init(priv->vol) < 0) { virReportSystemError(errno, - _("failed to initialize gluster connection to " - "server: '%s'"), hostname); + _("failed to initialize gluster connection with given hosts " + "volume : '%s' image-path : '%s'"), + NULLSTR(src->volume), src->path); goto error; } -- 2.1.0

5 9

[libvirt] [RFC PATCH 0/8] Make loading domains with invalid XML possible
by Martin Kletzander 13 Oct '15

13 Oct '15

We always had to deal with new parsing errors in a weird way. All of them needed to go into functions starting the domains. That messes up the code, it's confusing to newcomers and so on. I once had an idea that we can handle this stuff. We know what failed, we have the XML that failed parsing and if the problem is not in the domain name nor UUID, we can create a domain object out of that as well. This way we are able to do something we weren't with this series applied. Example follows. Assume "dummy" is a domain with invalid XML (I just modified the file). Now, just for the purpose of this silly example, let's say we allowed domains with archtecture x86_*, but now we've realized that x86_64 is the only one we want to allow, but there already is a domain defined that has <type arch='x86_256' .../>. With the current master, the domain would be lost, so we would need to modify the funstion starting the domain (e.g. qemuProcessStart for qemu domain). However, with this series, it behaves like this: # virsh list --all Id Name State ---------------------------------------------------- - dummy shut off # virsh domstate --reason dummy shut off (invalid XML) # virsh start dummy error: Failed to start domain dummy error: XML error: domain 'dummy' was not loaded due to an XML error (unsupported configuration: Unknown architecture x86_256), please redefine it # VISUAL='sed -i s/x86_256/x86_64/' virsh edit dummy Domain dummy XML configuration edited. # virsh domstate --reason dummy shut off (unknown) # virsh start dummy Domain dummy started This is a logical next step for us to clean and separate parsing and starting, getting rid of some old code without sacrifying compatibility and maybe generating parser code in the future. Why is this an RFC? - I can certainly easily imagine some people who might not like this - It doesn't work for all drivers (yet) - It does handle status XMLs, but only slightly. All the handling is done in a commit message that says something along the lines of "beware, we should still be careful, but not as much as before". - The error type used for the message is one that is already used for something else and we might want a new type of error for exactly this one error message, although it might be enough for some to have the possibility of checking this by querying the domain state and checking the reason. - Just so you know I came up with something new for which there's no BZ (or at least yet) =) Martin Kletzander (8): conf, virsh: Add new domain shutoff reason qemu: Few whitespace cleanups conf: Extract name-parsing into its own function conf: Extract UUID parsing into its own function conf: Optionally keep domains with invalid XML qemu: Don't lookup invalid domains unless specified otherwise qemu: Prepare basic APIs to handle invalid defs qemu: Load domains with invalid XML on start include/libvirt/libvirt-domain.h | 2 + src/bhyve/bhyve_driver.c | 2 + src/conf/domain_conf.c | 180 +++++++++++++++++++++++++++++++++------ src/conf/domain_conf.h | 5 ++ src/libxl/libxl_driver.c | 3 + src/lxc/lxc_driver.c | 3 + src/qemu/qemu_driver.c | 64 +++++++++++--- src/uml/uml_driver.c | 2 + tools/virsh-domain-monitor.c | 3 +- 9 files changed, 223 insertions(+), 41 deletions(-) -- 2.5.3

4 24

[libvirt] [PATCH] libvirt-domain: Drop virDomainMigrateCheckNotLocal attribute
by Michal Privoznik 13 Oct '15

13 Oct '15

Our apibuild.py script does not cope with ATTRIBUTE_NONNULL: Parse Error: parsing function type, ')' expected Got token ('name', 'char') Last token: ('name', 'char') Token queue: [('op', '*'), ('name', 'dconnuri'), ('sep', ')')] Line 3297 end: Makefile:2441: recipe for target '../../docs/apibuild.py.stamp' failed Let's drop it. Moreover, up until e17ae3ccc2dbc1400 where it was introduced we did not really care about NULL-ity of dconnuri. And moreover the ATTRIBUTE_NONNULL merely checks for static calls over NULL, it won't catch the dynamic ones, where a NULL is passed by a variable at runtime. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/libvirt-domain.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index e6c8604..6e1aacd 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -3293,7 +3293,7 @@ virDomainMigrateVersion3Params(virDomainPtr domain, } -static int ATTRIBUTE_NONNULL(1) +static int virDomainMigrateCheckNotLocal(const char *dconnuri) { virURIPtr tempuri = NULL; -- 2.4.9

4 4

[libvirt] virsh can't support VM offline blockcommit
by 陈明 13 Oct '15

13 Oct '15

Hi everyone! I use the libvirt(version: 1.2.2) and QEMU(version: 2.2.1) to test qemu snapshot features: I tried virsh blockcommit when VM offline, the virsh blockcommit failed: the error messase as below: * error: Requested operation is not valid: domain is not running* when I start the VM, the virsh blockcommit work fine! my question is : we need virsh support blockcommit when VM offline。how do？ we test qemu snapshot as follows： we need create snapshot of the cycle，and by virsh blockcommit merge snapshot chain，but whenVM offline， the virsh blockcommit will fail， someone suggest me：when VM offline，may use qemu-img commit to submit snapshot turn，but qemu-img can't modify snapshot chain backing relation，it need we use qemu-img rebase amend snapshot‘s backingfile，* the main question* is when we use qemu-img rebase modify snapshot's backingfile, if someone start the VM, it will may lead to VM filesystem readonly。 we should how to solve the question? thanks!

1 0

[libvirt] [PATCH v2] virt-host-validate: check for IOMMU support
by Daniel P. Berrange 13 Oct '15

13 Oct '15

This looks for existance of DMAR (Intel) and IVRS (AMD) files under /sys/firmware/acpi/tables/, as a sign that the platform has IOMMU present & enabled in the BIOS. If these are present and /sys/kernel/iommu_groups does not contain any entries this is taken as a sign that the kernel has not enabled the IOMMU currently. If no ACPI tables are found we can't distinguish between disabled in BIOS and not present in the hardware, so we have to give the user a generic hint. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- Changed in v2: - Completely refactor the check to remove false positives and cope with iommu being enabled by default - Add AMD support tools/virt-host-validate-common.c | 64 +++++++++++++++++++++++++++++++++++++++ tools/virt-host-validate-common.h | 3 ++ tools/virt-host-validate-qemu.c | 6 ++++ 3 files changed, 73 insertions(+) diff --git a/tools/virt-host-validate-common.c b/tools/virt-host-validate-common.c index a88fc56..38128b1 100644 --- a/tools/virt-host-validate-common.c +++ b/tools/virt-host-validate-common.c @@ -26,6 +26,7 @@ #include <stdio.h> #include <unistd.h> #include <sys/utsname.h> +#include <sys/stat.h> #include <mntent.h> #include "virutil.h" @@ -341,3 +342,66 @@ int virHostValidateCGroupController(const char *hvname, return -1; return 0; } + +int virHostValidateIOMMU(const char *hvname, + virHostValidateLevel level) +{ + struct stat sb; + const char *bootarg = NULL; + bool isAMD = false, isIntel = false; + + if (virHostValidateHasCPUFlag("vmx")) + isIntel = true; + else if (virHostValidateHasCPUFlag("svm")) + isAMD = true; + else + /* XXX PPC/ARM/etc support */ + return 0; + + virHostMsgCheck(hvname, "%s", _("for device assignment IOMMU support")); + + if (isIntel) { + if (access("/sys/firmware/acpi/tables/DMAR", F_OK) == 0) { + virHostMsgPass(); + bootarg = "intel_iommu=on"; + } else { + virHostMsgFail(level, + "No ACPI DMAR table found, IOMMU either " + "disabled in BIOS or not supported by this " + "hardware platform"); + return -1; + } + } else if (isAMD) { + if (access("/sys/firmware/acpi/tables/IVRS", F_OK) == 0) { + virHostMsgPass(); + bootarg = "iommu=pt iommu=1"; + } else { + virHostMsgFail(level, + "No ACPI IVRS table found, IOMMU either " + "disabled in BIOS or not supported by this " + "hardware platform"); + return -1; + } + } else { + virHostMsgFail(level, + "Unknown if this platform has IOMMU support"); + return -1; + } + + + /* We can only check on newer kernels with iommu groups & vfio */ + if (stat("/sys/kernel/iommu_groups", &sb) < 0) + return 0; + + if (!S_ISDIR(sb.st_mode)) + return 0; + + virHostMsgCheck(hvname, "%s", _("if IOMMU is enabled by kernel")); + if (sb.st_nlink <= 2) { + virHostMsgFail(level, + "IOMMU appears to be disabled in kernel. " + "Add %s to kernel cmdline arguments", bootarg); + return -1; + } + return 0; +} diff --git a/tools/virt-host-validate-common.h b/tools/virt-host-validate-common.h index 1547e22..d4c4759 100644 --- a/tools/virt-host-validate-common.h +++ b/tools/virt-host-validate-common.h @@ -70,4 +70,7 @@ extern int virHostValidateCGroupController(const char *hvname, virHostValidateLevel level, const char *config_name); +extern int virHostValidateIOMMU(const char *hvname, + virHostValidateLevel level); + #endif /* __VIRT_HOST_VALIDATE_COMMON_H__ */ diff --git a/tools/virt-host-validate-qemu.c b/tools/virt-host-validate-qemu.c index b0ae293..044df65 100644 --- a/tools/virt-host-validate-qemu.c +++ b/tools/virt-host-validate-qemu.c @@ -20,6 +20,8 @@ */ #include <config.h> +#include <unistd.h> + #include "virt-host-validate-qemu.h" #include "virt-host-validate-common.h" @@ -87,5 +89,9 @@ int virHostValidateQEMU(void) "BLK_CGROUP") < 0) ret = -1; + if (virHostValidateIOMMU("QEMU", + VIR_HOST_VALIDATE_WARN) < 0) + ret = -1; + return ret; } -- 2.4.3

1 1