July 2011 - Devel - libvirt List Archives

[libvirt] [PATCH RESEND RFC v4 0/6] support cpu bandwidth in libvirt
by Wen Congyang 22 Jul '11

22 Jul '11

Resend this patchset TODO: 1. We create sub directory for each vcpu in cpu subsystem. So we should recalculate cpu.shares for each vcpu. Changelog: v4: address Adam Litke's comment v3: fix some small bugs implement the simple way v2: almost rewrite the patchset to support to control each vcpu's bandwidth. Limit quota to [-1, 2^64/1000] at the schemas level. We will check it at cgroup level. Wen Congyang (6): Introduce the function virCgroupForVcpu cgroup: Implement cpu.cfs_period_us and cpu.cfs_quota_us tuning API Update XML Schema for new entries qemu: Implement period and quota tunable XML configuration and parsing qemu: Implement cfs_period and cfs_quota's modification doc: Add documentation for new cputune elements period and quota docs/formatdomain.html.in | 19 ++ docs/schemas/domain.rng | 26 ++- src/conf/domain_conf.c | 20 ++- src/conf/domain_conf.h | 2 + src/libvirt_private.syms | 5 + src/qemu/qemu_cgroup.c | 137 ++++++++++ src/qemu/qemu_cgroup.h | 4 + src/qemu/qemu_driver.c | 312 +++++++++++++++++++++-- src/qemu/qemu_process.c | 4 + src/util/cgroup.c | 153 +++++++++++- src/util/cgroup.h | 11 + tests/qemuxml2argvdata/qemuxml2argv-cputune.xml | 2 + 12 files changed, 659 insertions(+), 36 deletions(-)

6 35

[libvirt] [PATCH 0/3] More work on TLS certificate code
by Daniel P. Berrange 22 Jul '11

22 Jul '11

This series started off with an attempt to refactor the TLS certificate validation code to remove duplication. I then realized I ought to be testing such sensitive code too. So the big interesting part of this patch is a test case todo validation using very certificates. Rather than add a bunch of certs to GIT, I have written (nasty) GNUTLS code to generate certs on the fly with a variety of good & bad data in them.

2 7

[libvirt] [PATCHv2 0/9] network: physical device abstraction aka 'virtual switch'
by Laine Stump 22 Jul '11

22 Jul '11

This series deprecates patches 04/10 - 10/10 of the previous series of the same name, posted here: https://www.redhat.com/archives/libvir-list/2011-July/msg00149.html This patch is in response to the following bug reports: https://bugzilla.redhat.com/show_bug.cgi?id=643947 (RHEL) https://bugzilla.redhat.com/show_bug.cgi?id=636106 (upstream) The first 4 patches of the earlier series were reasonable cleanups in their own right, and were ACKed, so I pushed them earlier. I reorganized the remaining 6 patches accord to advice from Dan Berrange and Eric Blake. Some notes about correlation between v1 and v2: [PATCHv2 1/9] util: define MAX A new patch [PATCHv2 2/9] conf: put virtPortProfile struct / functions in a common location Was 05/10 in v1: https://www.redhat.com/archives/libvir-list/2011-July/msg00154.html [PATCHv2 3/9] conf: virDomainNetDef points to (rather than contains) virtPortProfile This was also part of 05/10 in v1. I separated it out to make review of pure code movement (in 2/9) easier. [PATCHv2 4/9] conf: support abstracted interface info in domain interface XML [PATCHv2 5/9] conf: support abstracted interface info in network XML These two were 06/10 and 07/10 in v1: https://www.redhat.com/archives/libvir-list/2011-July/msg00155.html https://www.redhat.com/archives/libvir-list/2011-July/msg00156.html However, they were split differently. In v1 it was split into RNG changes vs. parser changes with network and domain combined in each patch. In v2, I reorganized (based on Dan's advice) to have one patch containing all domain XML changes, from RNG up through parser, test cases, and documentation, and one patch for network XML changes. [PATCHv2 6/9] network: separate Start/Shutdown functions for new network types This was patch 08/10 in v1: https://www.redhat.com/archives/libvir-list/2011-July/msg00157.html It is unchanged from the original. [PATCHv2 7/9] qemu: use virDomainNetGetActual*() functions where appropriate [PATCHv2 8/9] qemu: use virDomainNetGetActual*() in qemuDomainXMLToNative These two were patch 09/10 in v1: https://www.redhat.com/archives/libvir-list/2011-July/msg00158.html In v2, I just split out one piece of code that was more complicated than just replacing direct data references with helper functions, and modified the commit message to be more clear about the "NOP" nature of the change. [PATCHv2 9/9] network: internal API functions to manage assignment of physdev to guest This was 10/10 in v1: https://www.redhat.com/archives/libvir-list/2011-July/msg00159.html The main modification here was to remove all of the #if WITH_NETWORK from qemu_*.c, and instead define static inline NOP functions for the functions I'd previously eliminated references to with #if WITH_NETWORK (in the case that libvirt is build --without-network). This works out very well, except that make syntax-check doesn't like ATTRIBUTE_UNUSED in the inline static functions (I think we should change syntax-check rather than going back to the old #if WITH_NETWORK). Comments about other changes I've made *within* each patch are in the responses to the original patches, as well as in the annotations to the new patches.

3 24

[libvirt] [PATCHv2 00/16] bypass-cache in save/restore
by Eric Blake 22 Jul '11

22 Jul '11

Incorporating review comments from v1: https://www.redhat.com/archives/libvir-list/2011-July/msg00842.html Still to be written: new API for managing save state images in isolation: https://www.redhat.com/archives/libvir-list/2011-July/msg00855.html virConnectDomainSaveGetXMLDesc virConnectDomainSaveModify as well as virsh commands to wrap them: virsh managedsave-dumpxml file virsh managedsave-define file xml virsh managedsave-edit file Eric Blake (16): command: move all docs into .c file command: avoid leaking fds across fork build: rename files.h to virfile.h qemu: fix error message with migrate2 xml save: document new public API save: wire up remote protocol save: wire up trivial save/restore flags implementations save: add --bypass-cache flag to virsh save/restore operations save: support --xml to virsh save/restore save: let iohelper handle inherited fd save: let iohelper work on O_DIRECT fds save: add virFileDirectFd wrapper type save: support BYPASS_CACHE during qemu save/restore save: support bypass-cache flag in qemu.conf save: support bypass-cache flag in libvirt-guests init script save: support qemu modifying xml on domain save/restore HACKING | 2 +- cfg.mk | 3 +- configure.ac | 6 +- daemon/libvirtd.c | 2 +- docs/hacking.html.in | 2 +- include/libvirt/libvirt.h.in | 30 ++- po/POTFILES.in | 1 + src/Makefile.am | 2 +- src/conf/domain_conf.c | 2 +- src/conf/network_conf.c | 2 +- src/conf/nwfilter_conf.c | 2 +- src/conf/storage_conf.c | 2 +- src/conf/storage_encryption_conf.c | 2 +- src/driver.h | 12 + src/fdstream.c | 36 +-- src/libvirt.c | 181 ++++++++++++- src/libvirt_private.syms | 16 +- src/libvirt_public.syms | 2 + src/libxl/libxl_conf.c | 2 +- src/libxl/libxl_driver.c | 36 +++- src/locking/lock_driver_sanlock.c | 2 +- src/lxc/lxc_container.c | 2 +- src/lxc/lxc_controller.c | 2 +- src/lxc/lxc_driver.c | 2 +- src/node_device/node_device_linux_sysfs.c | 2 +- src/nodeinfo.c | 2 +- src/nwfilter/nwfilter_ebiptables_driver.c | 2 +- src/openvz/openvz_conf.c | 2 +- src/openvz/openvz_driver.c | 2 +- src/phyp/phyp_driver.c | 2 +- src/qemu/libvirtd_qemu.aug | 2 + src/qemu/qemu.conf | 16 + src/qemu/qemu_capabilities.c | 2 +- src/qemu/qemu_command.c | 18 +-- src/qemu/qemu_conf.c | 12 +- src/qemu/qemu_conf.h | 5 +- src/qemu/qemu_domain.c | 2 +- src/qemu/qemu_driver.c | 229 ++++++++++++---- src/qemu/qemu_hotplug.c | 2 +- src/qemu/qemu_migration.c | 4 +- src/qemu/qemu_monitor.c | 2 +- src/qemu/qemu_process.c | 2 +- src/remote/remote_driver.c | 4 +- src/remote/remote_protocol.x | 17 +- src/remote_protocol-structs | 13 + src/rpc/virnetclient.c | 2 +- src/rpc/virnetserver.c | 2 +- src/rpc/virnetsocket.c | 2 +- src/secret/secret_driver.c | 2 +- src/security/security_apparmor.c | 2 +- src/security/security_selinux.c | 2 +- src/security/virt-aa-helper.c | 2 +- src/storage/storage_backend.c | 2 +- src/storage/storage_backend_fs.c | 2 +- src/storage/storage_backend_iscsi.c | 2 +- src/storage/storage_backend_logical.c | 2 +- src/storage/storage_backend_mpath.c | 2 +- src/storage/storage_backend_scsi.c | 2 +- src/storage/storage_driver.c | 2 +- src/test/test_driver.c | 44 +++- src/uml/uml_conf.c | 2 +- src/uml/uml_driver.c | 2 +- src/util/bridge.c | 2 +- src/util/cgroup.c | 2 +- src/util/command.c | 430 +++++++++++++++++++++-------- src/util/command.h | 223 +-------------- src/util/conf.c | 2 +- src/util/dnsmasq.c | 2 +- src/util/event_poll.c | 2 +- src/util/files.c | 80 ------ src/util/hooks.c | 2 +- src/util/interface.c | 2 +- src/util/iohelper.c | 194 ++++++++++--- src/util/logging.c | 2 +- src/util/macvtap.c | 2 +- src/util/pci.c | 2 +- src/util/stats_linux.c | 2 +- src/util/storage_file.c | 2 +- src/util/util.c | 2 +- src/util/uuid.c | 2 +- src/util/viraudit.c | 2 +- src/util/virfile.c | 253 +++++++++++++++++ src/util/{files.h => virfile.h} | 33 ++- src/vbox/vbox_tmpl.c | 2 +- src/vmware/vmware_conf.c | 2 +- src/vmware/vmware_driver.c | 2 +- src/xen/block_stats.c | 2 +- src/xen/xen_driver.c | 36 +++- src/xen/xen_hypervisor.c | 2 +- src/xen/xen_inotify.c | 2 +- src/xen/xend_internal.c | 2 +- tests/commandhelper.c | 2 +- tests/commandtest.c | 2 +- tests/nodeinfotest.c | 2 +- tests/testutils.c | 2 +- tests/virnetsockettest.c | 2 +- tests/xencapstest.c | 2 +- tools/console.c | 2 +- tools/libvirt-guests.init.sh | 10 +- tools/libvirt-guests.sysconf | 5 + tools/virsh.c | 107 ++++++-- tools/virsh.pod | 47 +++- 102 files changed, 1560 insertions(+), 685 deletions(-) delete mode 100644 src/util/files.c create mode 100644 src/util/virfile.c rename src/util/{files.h => virfile.h} (57%) -- 1.7.4.4

2 45

[libvirt] [PATCH] Ensure that libvirtd shuts down if initialization fails
by Daniel P. Berrange 22 Jul '11

22 Jul '11

From: "Daniel P. Berrange" <berrange(a)redhat.com> If the virStateInitialize call fails we must shutdown libvirtd since drivers will not be available. Just free'ing the virNetServer is not sufficient, we must send a SIGTERM to ourselves so that we interrupt the event loop and trigger a orderly shutdown * daemon/libvirtd.c: Kill ourselves if state init fails * src/rpc/virnetserver.c: Add some debugging to event loop --- daemon/libvirtd.c | 2 ++ src/rpc/virnetserver.c | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/daemon/libvirtd.c b/daemon/libvirtd.c index 8f8b8b1..4a5f671 100644 --- a/daemon/libvirtd.c +++ b/daemon/libvirtd.c @@ -1160,6 +1160,8 @@ static void daemonRunStateInit(void *opaque) * seriously delay OS bootup process */ if (virStateInitialize(virNetServerIsPrivileged(srv)) < 0) { VIR_ERROR(_("Driver state initialization failed")); + /* Ensure the main event loop quits */ + kill(getpid(), SIGTERM); virNetServerFree(srv); return; } diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index 35f18b9..8f3ac4b 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -671,6 +671,7 @@ void virNetServerRun(virNetServerPtr srv) goto cleanup; } + VIR_DEBUG("srv=%p quit=%d", srv, srv->quit); while (!srv->quit) { /* A shutdown timeout is specified, so check * if any drivers have active state, if not @@ -731,6 +732,7 @@ void virNetServerQuit(virNetServerPtr srv) { virNetServerLock(srv); + VIR_DEBUG("Quit requested %p", srv); srv->quit = 1; virNetServerUnlock(srv); -- 1.7.6

2 1

[libvirt] [PATCH] Remove call to deprecated gnutls_certificate_type_set_priority (again)
by Daniel P. Berrange 22 Jul '11

22 Jul '11

From: "Daniel P. Berrange" <berrange(a)redhat.com> The gnutls_certificate_type_set_priority method is deprecated. Since we already set the default gnutls priority, it was not serving any useful purpose and can be removed * src/rpc/virnettlscontext.c: Remove gnutls_certificate_type_set_priority call --- src/rpc/virnettlscontext.c | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-) diff --git a/src/rpc/virnettlscontext.c b/src/rpc/virnettlscontext.c index 9279ded..ae16dbb 100644 --- a/src/rpc/virnettlscontext.c +++ b/src/rpc/virnettlscontext.c @@ -1115,7 +1115,6 @@ virNetTLSSessionPtr virNetTLSSessionNew(virNetTLSContextPtr ctxt, { virNetTLSSessionPtr sess; int err; - static const int cert_type_priority[] = { GNUTLS_CRT_X509, 0 }; VIR_DEBUG("ctxt=%p hostname=%s isServer=%d", ctxt, NULLSTR(hostname), ctxt->isServer); @@ -1142,9 +1141,7 @@ virNetTLSSessionPtr virNetTLSSessionNew(virNetTLSContextPtr ctxt, /* avoid calling all the priority functions, since the defaults * are adequate. */ - if ((err = gnutls_set_default_priority(sess->session)) != 0 || - (err = gnutls_certificate_type_set_priority(sess->session, - cert_type_priority))) { + if ((err = gnutls_set_default_priority(sess->session)) != 0) { virNetError(VIR_ERR_SYSTEM_ERROR, _("Failed to set TLS session priority %s"), gnutls_strerror(err)); -- 1.7.6

2 1

[libvirt] [PATCH] error: add new error type for reflecting partial API support
by Eric Blake 22 Jul '11

22 Jul '11

VIR_ERR_INVALID_ARG implies that an argument cannot possibly be correct, given the current state of the API. VIR_ERR_CONFIG_UNSUPPORTED implies that a configuration is wrong, but arguments aren't configuration. VIR_ERR_NO_SUPPORT implies that a function is completely unimplemented. But in the case of a function that is partially implemented, yet the full power of the API is not available for that driver, none of the above messages make sense. Hence a new error message, implying that the argument is known to comply with the current state of the API, and that while the driver supports aspects of the function, it does not support that particular use of the argument. A good use case for this is a driver that supports virDomainSaveFlags, but not the dxml argument of that API. It might be feasible to also use this new error for all functions that check flags, and which accept fewer flags than what is possible in the public API. But doing so would get complicated, since neither libvirt.c nor the remote driver may do flag filtering, and every other driver would have to do a two-part check, first using virCheckFlags on all public flags (which gives VIR_ERR_INVALID_ARG for an impossible flag), followed by a particular mask check for VIR_ERR_ARGUMENT_UNSUPPORTED (for a possible public flag but unsupported by this driver). * include/libvirt/virterror.h (VIR_ERR_ARGUMENT_UNSUPPORTED): New error. * src/util/virterror.c (virErrorMsg): Give it a message. Suggested by Daniel P. Berrange. --- Needed before v3 of 7/16 of the bypass-cache series: https://www.redhat.com/archives/libvir-list/2011-July/msg01470.html include/libvirt/virterror.h | 4 +++- src/util/virterror.c | 6 ++++++ 2 files changed, 9 insertions(+), 1 deletions(-) diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h index efa4796..9cac437 100644 --- a/include/libvirt/virterror.h +++ b/include/libvirt/virterror.h @@ -4,7 +4,7 @@ * Description: Provides the interfaces of the libvirt library to handle * errors raised while using the library. * - * Copy: Copyright (C) 2006, 2010 Red Hat, Inc. + * Copy: Copyright (C) 2006, 2010-2011 Red Hat, Inc. * * See COPYING.LIB for the License of this software * @@ -232,6 +232,8 @@ typedef enum { VIR_ERR_INVALID_DOMAIN_SNAPSHOT = 71,/* invalid domain snapshot */ VIR_ERR_NO_DOMAIN_SNAPSHOT = 72, /* domain snapshot not found */ VIR_ERR_INVALID_STREAM = 73, /* stream pointer not valid */ + VIR_ERR_ARGUMENT_UNSUPPORTED = 74, /* valid API use but unsupported by + the given driver */ } virErrorNumber; /** diff --git a/src/util/virterror.c b/src/util/virterror.c index 07f8b45..9a27feb 100644 --- a/src/util/virterror.c +++ b/src/util/virterror.c @@ -1183,6 +1183,12 @@ virErrorMsg(virErrorNumber error, const char *info) else errmsg = _("invalid stream pointer in %s"); break; + case VIR_ERR_ARGUMENT_UNSUPPORTED: + if (info == NULL) + errmsg = _("argument unsupported"); + else + errmsg = _("argument unsupported: %s"); + break; } return (errmsg); } -- 1.7.4.4

2 2

[libvirt] [PATCH] remote: Fix forgotten comma
by Michal Privoznik 21 Jul '11

21 Jul '11

during conflict resolving --- Pushing under trivial & build-breaker rules src/remote/remote_protocol.x | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 1ada5f9..f300204 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -2410,7 +2410,7 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_UNDEFINE_FLAGS = 231, /* autogen autogen */ REMOTE_PROC_DOMAIN_SAVE_FLAGS = 232, /* autogen autogen */ - REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233 /* autogen autogen */ + REMOTE_PROC_DOMAIN_RESTORE_FLAGS = 233, /* autogen autogen */ REMOTE_PROC_DOMAIN_DESTROY_FLAGS = 234 /* autogen autogen */ /* -- 1.7.5.rc3

2 1

[libvirt] [PATCHv2 00/13] New API: virDominDestroyFlags
by Michal Privoznik 21 Jul '11

21 Jul '11

This series creates new API which adds flags to virDomainDestroy. As mentioned here many times, it is a bad luck all APIs don't have flags parameter. Parameter for this might be needed to select different ways of destroying a domain. For example, qemu has 'quit' command, which force it to flush internal caches a thus preventing from data loss. Or vice versa - in cases where we are doing hypervisor assisted destroy we might want to send signals, because hypervisor is not responding. Currently, no new functionality is implemented, but this paves the way for creating multiple ways of domain destroying. Therefore, calling this new API is the same as calling its predecessor. diff to v1: -rename virDominDestroyWithFlags() to virDominDestroyFlags() -rename enum virDominDestroyFlags to virDominDestroyFlagsValues Michal Privoznik (13): destroy: Define new public API virDomainDestroyFlags destroy: Wire up the remote protocol destroy: Implement internal API for qemu driver destroy: Implement internal API for ESX driver destroy: Implement internal API for libxl driver destroy: Implement internal API for lxc driver destroy: Implement internal API for openvz driver destroy: Implement internal API for phyp driver destroy: Implement internal API for uml driver destroy: Implement internal API for vbox driver destroy: Implement internal API for vmware driver destroy: Implement internal API for xen driver destroy: Implement internal API for xenapi driver include/libvirt/libvirt.h.in | 7 +++++ src/driver.h | 4 +++ src/esx/esx_driver.c | 12 +++++++++- src/libvirt.c | 52 ++++++++++++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 1 + src/libxl/libxl_driver.c | 12 +++++++++- src/lxc/lxc_driver.c | 23 +++++++++++++++++- src/openvz/openvz_driver.c | 13 +++++++++- src/phyp/phyp_driver.c | 12 +++++++++- src/qemu/qemu_driver.c | 13 +++++++++- src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 8 +++++- src/remote_protocol-structs | 5 ++++ src/uml/uml_driver.c | 14 ++++++++++- src/vbox/vbox_tmpl.c | 14 ++++++++++- src/vmware/vmware_driver.c | 12 +++++++++- src/xen/xen_driver.c | 28 ++++++++++++++++++++++ src/xen/xen_driver.h | 1 + src/xen/xen_hypervisor.c | 24 +++++++++++++++++- src/xen/xen_hypervisor.h | 3 ++ src/xen/xen_inotify.c | 1 + src/xen/xend_internal.c | 24 +++++++++++++++++- src/xen/xm_internal.c | 1 + src/xen/xs_internal.c | 1 + src/xenapi/xenapi_driver.c | 22 ++++++++++++++++- 25 files changed, 291 insertions(+), 17 deletions(-) -- 1.7.5.rc3

3 18

[libvirt] [PATCH RFC 0/3] undefine: Interface for undefinig storage devices along with domains
by Peter Krempa 21 Jul '11

21 Jul '11

This extension of flags used while undefining domains grants the user ability to remove virtual storage devices while undefining a domain. This is only a demonstration, implemented only for the QEMU/KVM emulators. I'd like to know your opinions and comments and soon i'll provide patches for other emulators as well. Thanks for your precious time :) Peter Peter Krempa (3): undefine: Add flags for removing storage files when undefining a domain. undefine: QEMU driver - remove storage associated with a domain while undefining undefine: Add coresponding switches to virsh, for removing storage devices associated with domain include/libvirt/libvirt.h.in | 2 ++ src/qemu/qemu_driver.c | 32 +++++++++++++++++++++++++++++++- tools/virsh.c | 17 ++++++++++++++++- 3 files changed, 49 insertions(+), 2 deletions(-) -- 1.7.6

3 8