[libvirt] [RFC] New libvirt API for domain memory statistics reporting (V2)
by Adam Litke
Thanks for the review and comments on V1. This series tries to address most of
the feedback I received. I did not add any additional memory stats yet (I feel
that with my new method of expanding the API, that can be handled separately).
I am not convinced that every memory stat that any hypervisor exports should
necessarily be included in this API. I tried to focus on statistics that are
useful in the context of managing domains at the hypervisor level.
Changes since V1:
* New system for maintaining ABI compatibility and API extensibility:
Rather than passing around a fixed-size stats structure, work with arrays
of stats. An enum of known statistic tags (SWAP_IN, SWAP_OUT, TOTAL_MEM,
etc) is defined. A stat is defined as a tag/value pair. When making a
call to the API, the caller provides an array of stats and the size of the
array. That array is filled with up to the requested number of stats
(depending on hypervisor and guest support). The number of stats provided
is returned.
* Miscellaneous changes:
Changed the API function from virDomainMemStats to virDomainMemoryStats
Added documentation for each memory stat
--
When using ballooning to manage overcommitted memory on a host, a system for
guests to communicate their memory usage to the host can provide information
that will minimize the impact of ballooning on the guests while maximizing
efficient use of host memory.
The design of such a communication channel was recently added to version 0.8.2
of the VirtIO Spec (See Appendix G):
- http://ozlabs.org/~rusty/virtio-spec/virtio-spec-0.8.2.pdf
Host-side and guest-side implementations have been accepted for inclusion in
their respective projects:
- Guest: http://lkml.org/lkml/2009/11/30/274
- Host: http://lists.gnu.org/archive/html/qemu-devel/2009-12/msg00380.html
The following patch series implements a new libvirt interface to expose these
memory statistics. Thank you for your review and comments.
[PATCH 1/6] domMemStats: Add domainMemoryStats method to struct _virDriver
[PATCH 2/6] domMemoryStats: Add public symbol to libvirt API
[PATCH 3/6] qemu-driver: Enable domainMemStats in the qemu driver
[PATCH 4/6] remote-driver: Add domainMemoryStats support
[PATCH 5/6] virsh: Enable virDomainMemoryStats as a new command
[PATCH 6/6] python: Add python bindings for virDomainMemoryStats
14 years, 11 months
[libvirt] Licensing of the schemas?
by Diego Elio “Flameeyes” Pettenò
Hi,
I've finally found the time to backport the schema fixes and I'm going
to add the nxml-libvirt-schemas package to Portage so that those like me
who like to edit the XML files with Emacs can do so with some support.
But I didn't pay enough attention before that the .rng files lack a
license header. I'd assume that the license of the package (LGPL-2.1)
apply to the schema, but it would be nice if somebody could confirm that
for me (and eventually somebody, me even, could add the proper headers
to them).
Thanks,
--
Diego Elio Pettenò — “Flameeyes”
http://blog.flameeyes.eu/
If you found a .asc file in this mail and know not what it is,
it's a GnuPG digital signature: http://www.gnupg.org/
14 years, 11 months
[libvirt] [PATCH v3 00/13] CPU selection patches
by Jiri Denemark
Hi,
This is a third version of CPU selection patchset.
Changes in version 3:
- fix build and segfault on i386
Changes in version 2:
- virConnectGetHostCPU() API call was completely removed
- 'CPU driver implementation' (11/14) patch was dropped
- virConnectCompareCPU() API call is implemented directly by hypervisor
drivers
- new cpuCompareXML() internal function to make virConnectCompareCPU()
simpler
Jirka
Jiri Denemark (13):
XML schema for CPU flags
XML parsing/formating code for CPU flags
Public API
Internal driver API
Public API implementation
Wire protocol format
Remote driver
Server side dispatcher
CPU selection infrastructure
CPU map for models and features
Set cpuCompare handler in drivers
Implement CPU selection in QEMU driver
Implement virsh commands
configure.in | 20 +
daemon/remote.c | 22 +
docs/schemas/capability.rng | 46 ++-
docs/schemas/domain.rng | 62 +++
include/libvirt/libvirt.h.in | 13 +
include/libvirt/virterror.h | 1 +
src/Makefile.am | 23 +-
src/conf/capabilities.c | 31 ++-
src/conf/capabilities.h | 6 +
src/conf/cpu_conf.c | 366 +++++++++++++
src/conf/cpu_conf.h | 116 +++++
src/conf/domain_conf.c | 15 +
src/conf/domain_conf.h | 2 +
src/cpu/cpu.c | 242 +++++++++
src/cpu/cpu.h | 129 +++++
src/cpu/cpu_generic.c | 123 +++++
src/cpu/cpu_generic.h | 32 ++
src/cpu/cpu_map.c | 130 +++++
src/cpu/cpu_map.h | 41 ++
src/cpu/cpu_map.xml | 304 +++++++++++
src/cpu/cpu_x86.c | 1159 ++++++++++++++++++++++++++++++++++++++++++
src/cpu/cpu_x86.h | 31 ++
src/cpu/cpu_x86_data.h | 45 ++
src/driver.h | 5 +
src/esx/esx_driver.c | 1 +
src/libvirt.c | 41 ++
src/libvirt_private.syms | 19 +
src/libvirt_public.syms | 5 +
src/lxc/lxc_driver.c | 1 +
src/openvz/openvz_driver.c | 1 +
src/qemu/qemu_conf.c | 401 ++++++++++++++-
src/qemu/qemu_conf.h | 7 +
src/qemu/qemu_driver.c | 43 ++-
src/remote/remote_driver.c | 27 +
src/remote/remote_protocol.x | 12 +-
src/test/test_driver.c | 1 +
src/uml/uml_driver.c | 1 +
src/util/virterror.c | 3 +
src/vbox/vbox_tmpl.c | 1 +
tools/virsh.c | 65 +++
40 files changed, 3558 insertions(+), 35 deletions(-)
create mode 100644 src/conf/cpu_conf.c
create mode 100644 src/conf/cpu_conf.h
create mode 100644 src/cpu/cpu.c
create mode 100644 src/cpu/cpu.h
create mode 100644 src/cpu/cpu_generic.c
create mode 100644 src/cpu/cpu_generic.h
create mode 100644 src/cpu/cpu_map.c
create mode 100644 src/cpu/cpu_map.h
create mode 100644 src/cpu/cpu_map.xml
create mode 100644 src/cpu/cpu_x86.c
create mode 100644 src/cpu/cpu_x86.h
create mode 100644 src/cpu/cpu_x86_data.h
14 years, 11 months
[libvirt] [PATCH] Initialize gcrypt threading
by Daniel P. Berrange
GNUTLS uses gcrypt for its crypto functions. gcrypt requires
that the app/library initializes threading before using it.
We don't want to force apps using libvirt to know about
gcrypt, so we make virInitialize init threading on their
behalf. This location also ensures libvirtd has initialized
it correctly. This initialization is required even if libvirt
itself were only using one thread, since another non-libvirt
library (eg GTK-VNC) could also be using gcrypt from another
thread
* src/libvirt.c: Register thread functions for gcrypt
* configure.in: Add -lgcrypt to linker flags
---
build-aux/.gitignore | 2 +
configure.in | 4 ++-
src/libvirt.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 58 insertions(+), 1 deletions(-)
diff --git a/build-aux/.gitignore b/build-aux/.gitignore
index 72e8ffc..a1b5d3b 100644
--- a/build-aux/.gitignore
+++ b/build-aux/.gitignore
@@ -1 +1,3 @@
*
+/link-warning.h
+/mktempd
diff --git a/configure.in b/configure.in
index 6ed2efd..c86ee97 100644
--- a/configure.in
+++ b/configure.in
@@ -547,7 +547,9 @@ if test "$GNUTLS_FOUND" = "no"; then
test $fail = 1 &&
AC_MSG_ERROR([You must install the GnuTLS library in order to compile and run libvirt])
- GNUTLS_LIBS=$LIBS
+ dnl Not all versions of gnutls include -lgcrypt, and so we add
+ dnl it explicitly for the calls to gcry_control/check_version
+ GNUTLS_LIBS="$LIBS -lgcrypt"
LIBS="$old_libs"
fi
diff --git a/src/libvirt.c b/src/libvirt.c
index 103b331..cad33c2 100644
--- a/src/libvirt.c
+++ b/src/libvirt.c
@@ -22,6 +22,7 @@
#include <sys/wait.h>
#endif
#include <time.h>
+#include <gcrypt.h>
#include <libxml/parser.h>
#include <libxml/xpath.h>
@@ -251,6 +252,55 @@ winsock_init (void)
}
#endif
+static int virTLSMutexInit (void **priv)
+{ \
+ virMutexPtr lock = NULL;
+
+ if (VIR_ALLOC(lock) < 0)
+ return ENOMEM;
+
+ if (virMutexInit(lock) < 0) {
+ VIR_FREE(lock);
+ return errno;
+ }
+
+ *priv = lock;
+ return 0;
+}
+
+static int virTLSMutexDestroy(void **priv)
+{
+ virMutexPtr lock = *priv;
+ virMutexDestroy(lock);
+ VIR_FREE(lock);
+ return 0;
+}
+
+static int virTLSMutexLock(void **priv)
+{
+ virMutexPtr lock = *priv;
+ virMutexLock(lock);
+ return 0;
+}
+
+static int virTLSMutexUnlock(void **priv)
+{
+ virMutexPtr lock = *priv;
+ virMutexUnlock(lock);
+ return 0;
+}
+
+static struct gcry_thread_cbs virTLSThreadImpl = {
+ (GCRY_THREAD_OPTION_PTHREAD | (GCRY_THREAD_OPTION_VERSION << 8)),
+ NULL,
+ virTLSMutexInit,
+ virTLSMutexDestroy,
+ virTLSMutexLock,
+ virTLSMutexUnlock,
+ NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL
+};
+
+
/**
* virInitialize:
*
@@ -273,6 +323,9 @@ virInitialize(void)
virRandomInitialize(time(NULL) ^ getpid()))
return -1;
+ gcry_control(GCRYCTL_SET_THREAD_CBS, &virTLSThreadImpl);
+ gcry_check_version(NULL);
+
virLogSetFromEnv();
DEBUG0("register drivers");
--
1.6.5.2
14 years, 11 months
[libvirt] [PATCH 0/10] Convert over to use new -device args everywhere
by Daniel P. Berrange
QEMU has introduced a new command line arg called -device for
specifying the guest device config. This allows separation from
the host device config. libvirt needs to use this new syntax so
that we can specify static PCI addresses for devices (the old
syntax did not allow this).
This patch series is the quick conversion I've done so far. The
most obvious thing still lacking here is new test cases. I want
to wait until this series merges with the device addressing series
before adding those, since one set of tests can cover both patch
series.
In addition the handling of disk drivers is not quite correct. We
need the disk controller patches myself/Wolfgang have worked on
in order to specify the disk bus/unit instead of index.
14 years, 11 months
[libvirt] Cannot start domain w/xenner emulator
by Gerry Reno
# virsh start Ubuntu-domU-x86-1
Connecting to uri: qemu:///system
error: Failed to start domain Ubuntu-domU-x86-1
error: internal error unable to start guest: libvir: error : cannot
execute binary /usr/bin/xenner: Permission denied
# ls -l /usr/bin/xenner
-rwxr-xr-x 1 root root 118936 2009-12-14 17:05 /usr/bin/xenner
Sure looks executable. And there's no SELinux on the box.
libvirt is 0.7.0
What the..?
-Gerry
14 years, 11 months
[libvirt] ANNOUNCE: Starting a libvirt application developer guide effort
by Daniel P. Berrange
This is a quick mail to announce that we have started an effort to write
a libvirt application developer guide. The guide is intended to introduce
programmers to the libvirt API and its concepts and demonstrate best
practice along the way. The initial focus is on the C API, but once that's
matured, further info on language bindings can be added.
As is common with all work done in the libvirt project, the guide is being
made available under an open license, specifically the
Creative Commons AttributionâShare Alike 3.0 Unported license ("CC-BY-SA")
The reason for this choice is that it has become the defacto license choice
for documentation under Fedora, GNOME, and content from Wikipedia, etc.[1]
The application developer guide is written in DocBook and is processed using
the Publican toolset to generate the HTML and PDF output. Again this choice
was made because it is the same toolset used in Fedora & Red Hat docs
NB, this is using Publican 1.x series of releases which is available starting
in Fedora 12 or later. The build config/styling is not compatible with the
older Public 0.x series from earlier Fedora.
There are two new GIT repositories for this work
- A repository containing the libvirt brand. This is essentially CSS and
graphics required to make the docs use the same style as the libvirt
website does
http://libvirt.org/git/?p=libvirt-publican.git
- A repository containing the application developer doc. This is the
actual interesting content.
http://libvirt.org/git/?p=libvirt-appdev-guide.git
Assuming you have Publican >= 1.0 installed, then the first step is to
build and install the libvirt brand. This involves getting a checkout of
the libvirt-publican.git repo, typing 'make rpm' and then installing the
RPM it generates.
Once the brand is installed, then you can get a checkout of the actual
docs from 'libvirt-appdev-guide.git'. The content is all under the
directory en-US/ in docbook format. To generate html, from the top level
directory just type 'make html', likewise pdfs with 'make pdf'. If you
just type 'make' you get html and pdfs.
I have put up example copies generated from current available content
http://berrange.fedorapeople.org/libvirt/appdev-guide/html/
http://berrange.fedorapeople.org/libvirt/appdev-guide/pdf/
If you look at these, you'll notice of course that while there is a nice
big table of contents, most of the actual content is missing. We are
continuing to write content and will update it periodically. If anyone is
interested in contributing content, then simply send patches against the
docbook XML in the libvirt-appdev-guide.git to this list as per the normal
review & commit process.
Regards,
Daniel
[1] http://iquaid.org/2009/07/06/why-relicense-fedora-documentation-and-wiki-...
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
14 years, 11 months
[libvirt] [PATCH] esx_util.c: avoid NULL deref for invalid inputs
by Jim Meyering
The trouble here is that "goto failure" would run this code:
failure:
VIR_FREE(*datastoreName);
VIR_FREE(*directoryName);
VIR_FREE(*fileName);
result = -1;
goto cleanup;
And thus if any of those 3 input variables was NULL,
that deref would probably provoke a segfault.
By the way, that interface should be changed,
because it encourages risky behavior:
int
esxUtil_ParseDatastoreRelatedPath(virConnectPtr conn,
const char *datastoreRelatedPath,
char **datastoreName,
char **directoryName, char **fileName)
Note how it takes a buffer, datastoreRelatedPath,
but with no length parameter. Then this function
writes into the buffer with this code:
if (sscanf(datastoreRelatedPath, "[%a[^]%]] %a[^\n]", datastoreName,
&directoryAndFileName) != 2) {
which cannot even check for buffer overrun because it doesn't
know the length.
You might want to add a new parameter, buf_len:
int
esxUtil_ParseDatastoreRelatedPath(virConnectPtr conn,
const char *datastoreRelatedPath,
size_t buf_len,
char **datastoreName,
char **directoryName, char **fileName)
>From aba304e07835c123bd63f1d5d6777a898916349f Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering(a)redhat.com>
Date: Tue, 15 Dec 2009 19:08:49 +0100
Subject: [PATCH] esx_util.c: avoid NULL deref for invalid inputs
* src/esx/esx_util.c (esxUtil_ParseDatastoreRelatedPath): Return
right away, rather than trying to clean up and dereference NULL
pointers, if any input is invalid.
---
src/esx/esx_util.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/esx/esx_util.c b/src/esx/esx_util.c
index 3e53921..8703ac2 100644
--- a/src/esx/esx_util.c
+++ b/src/esx/esx_util.c
@@ -277,7 +277,7 @@ esxUtil_ParseDatastoreRelatedPath(virConnectPtr conn,
directoryName == NULL || *directoryName != NULL ||
fileName == NULL || *fileName != NULL) {
ESX_ERROR(conn, VIR_ERR_INTERNAL_ERROR, "Invalid argument");
- goto failure;
+ return -1;
}
/*
@@ -299,7 +299,7 @@ esxUtil_ParseDatastoreRelatedPath(virConnectPtr conn,
ESX_ERROR(conn, VIR_ERR_INTERNAL_ERROR,
"Datastore related path '%s' doesn't have expected format "
"'[<datastore>] <path>'", datastoreRelatedPath);
- goto failure;
+ return -1;
}
/* Split <path> into <directory>/<file>, where <directory> is optional */
--
1.6.6.rc2.275.g51e2d
14 years, 11 months
[libvirt] [PATCH] esx_vmx.c: don't dereference a NULL virtualDev
by Jim Meyering
The offending code starts here:
int
esxVMX_ParseSCSIController(virConnectPtr conn, virConfPtr conf, int controller,
int *present, char **virtualDev)
{
char present_name[32];
char virtualDev_name[32];
if (virtualDev == NULL || *virtualDev != NULL) {
ESX_ERROR(conn, VIR_ERR_INTERNAL_ERROR, "Invalid argument");
goto failure;
}
If the virtualDev parameter is NULL, then we'd issue the
diagnostic and take the "goto", and (below), dereference NULL.
>From 79283ba1d667534175d4c48079e6b500feba6480 Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering(a)redhat.com>
Date: Tue, 15 Dec 2009 16:07:10 +0100
Subject: [PATCH] esx_vmx.c: don't dereference NULL for a NULL virtualDev
* src/esx/esx_vmx.c (esxVMX_ParseSCSIController): Don't deref
"virtualDev" when it is NULL.
---
src/esx/esx_vmx.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/src/esx/esx_vmx.c b/src/esx/esx_vmx.c
index f5b4544..404617e 100644
--- a/src/esx/esx_vmx.c
+++ b/src/esx/esx_vmx.c
@@ -1204,7 +1204,8 @@ esxVMX_ParseSCSIController(virConnectPtr conn, virConfPtr conf, int controller,
return 0;
failure:
- VIR_FREE(*virtualDev);
+ if (virtualDev)
+ VIR_FREE(*virtualDev);
return -1;
}
--
1.6.6.rc2.275.g51e2d
14 years, 11 months
[libvirt] [PATCH] Fix memory leak in qemudBuildCommandLine
by Matthias Bolte
---
src/qemu/qemu_conf.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c
index 86172c6..2ee2042 100644
--- a/src/qemu/qemu_conf.c
+++ b/src/qemu/qemu_conf.c
@@ -1893,7 +1893,7 @@ int qemudBuildCommandLine(virConnectPtr conn,
}
ADD_ARG_LIT("-monitor");
- ADD_ARG_LIT(virBufferContentAndReset(&buf));
+ ADD_ARG(virBufferContentAndReset(&buf));
}
if (def->localtime)
--
1.6.0.4
14 years, 11 months
