January 2017 - Devel - libvirt List Archives

[libvirt] [PATCH python 0/2] Add support for new events
by Daniel P. Berrange 10 Jan '17

10 Jan '17

Daniel P. Berrange (2): Add support for secret event APIs Add support for domain metadata change event generator.py | 2 + libvirt-override-virConnect.py | 52 +++++++++ libvirt-override.c | 260 +++++++++++++++++++++++++++++++++++++++++ sanitytest.py | 3 + 4 files changed, 317 insertions(+) -- 2.9.3

2 5

[libvirt] [libvirt-perl PATCH] Add perf cache_l1d event
by John Ferlan 10 Jan '17

10 Jan '17

Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- Daniel - feel free to add to the pile of changes you posted if it's easier. Changes | 1 + Virt.xs | 1 + lib/Sys/Virt/Domain.pm | 6 ++++++ 3 files changed, 8 insertions(+) diff --git a/Changes b/Changes index 672b566..2a13f8f 100644 --- a/Changes +++ b/Changes @@ -10,6 +10,7 @@ Revision history for perl module Sys::Virt - Add PERF_PARAM_STALLED_CYCLES_BACKEND constant - Add PERF_PARAM_REF_CPU_CYCLES constant - Add virStorageVolGetInfoFlags & associated constants + - Add PERF_PARAM_CACHE_L1D constant 2.5.0 2016-12-05 diff --git a/Virt.xs b/Virt.xs index 1eb94a8..8fbbc4a 100644 --- a/Virt.xs +++ b/Virt.xs @@ -8313,6 +8313,7 @@ BOOT: REGISTER_CONSTANT_STR(VIR_PERF_PARAM_STALLED_CYCLES_FRONTEND, PERF_PARAM_STALLED_CYCLES_FRONTEND); REGISTER_CONSTANT_STR(VIR_PERF_PARAM_STALLED_CYCLES_BACKEND, PERF_PARAM_STALLED_CYCLES_BACKEND); REGISTER_CONSTANT_STR(VIR_PERF_PARAM_REF_CPU_CYCLES, PERF_PARAM_REF_CPU_CYCLES); + REGISTER_CONSTANT_STR(VIR_PERF_PARAM_CACHE_L1D, PERF_PARAM_CACHE_L1D); REGISTER_CONSTANT_STR(VIR_DOMAIN_BANDWIDTH_IN_AVERAGE, BANDWIDTH_IN_AVERAGE); REGISTER_CONSTANT_STR(VIR_DOMAIN_BANDWIDTH_IN_PEAK, BANDWIDTH_IN_PEAK); diff --git a/lib/Sys/Virt/Domain.pm b/lib/Sys/Virt/Domain.pm index bd59139..2133a30 100644 --- a/lib/Sys/Virt/Domain.pm +++ b/lib/Sys/Virt/Domain.pm @@ -2809,6 +2809,12 @@ frequency scaling by applications running on the platform. It corresponds to the "perf.ref_cpu_cycles" field in the *Stats APIs. +=item Sys::Virt::Domain::PERF_PARAM_CACHE_L1D +The cache_l1d perf event counter which can be used to +measure the count of total level 1 data cache by applications +running on the platform. It corresponds to the "perf.cache_l1d" +field in the *Stats APIs. + =back =head2 VCPU FLAGS -- 2.7.4

2 1

[libvirt] [V2]RFC for support cache tune in libvirt
by Qiao, Liyong 10 Jan '17

10 Jan '17

Hi folks I’v send the V1 version RFC early this week, but no reply yet. Thanks Qiaowei for the comment, I’v change the RFC much more libvirt specify, please help to comments on. ##Propose Changes #Libvirtd configure changes Add a new configure option cache_allocation_ratio to libvirtd, which let libvirt to allocate how many cache to domains. Default is 0.5 Eg. On a host which has 55M cache, libvirt can allocate 55M * cache_allocation_ratio cache to domains ## Virsh command line changes: NAME cachetune - control or query domain cache allocation SYNOPSIS cachetune <domain> [--enabled true/false] [--type <type>][--size <number>] [--config] [--live] [--current] DESCRIPTION Allocate cache usage for domain. OPTIONS [--domain] <string> domain name, id or uuid --enabled <true/false> enable cache allocation --type <string> cache allocations type, support l3/l2 etc --size <number> the cache size in KB --config affect next boot --live affect running domain --current affect current domain This will allow libvirt to allocate specify type l3 cache for a domain Domain xml changes: <cachetune> <enabled=’yes’, size=4096, actual_size=4680,type=’l3’> <enabled=’no’, size=256, actual_size=0, type=’l2’> </cachetune> For more information about the detail design, please refer https://www.redhat.com/archives/libvir-list/2016-December/msg01011.html CAT intro: https://software.intel.com/en-us/articles/software-enabling-for-cache-alloc… Best Regards Eli Qiao(乔立勇）OpenStack Core team OTC Intel. --

2 2

[libvirt] [PATCH] util: fix domain object leaks on closecallbacks
by Wang King 10 Jan '17

10 Jan '17

From: wangjing <king.wang(a)huawei.com> The virCloseCallbacksSet method increase object reference for VM, and decrease object reference in virCloseCallbacksUnset. But VM UUID will be deleted from closecallbacks list in virCloseCallbacksRun when connection disconnected, and then object reference cannot be decreased by virCloseCallbacksUnset in callback functions. Signed-off-by: Wang King <king.wang(a)huawei.com> --- src/util/virclosecallbacks.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/src/util/virclosecallbacks.c b/src/util/virclosecallbacks.c index 891a92b..26d5075 100644 --- a/src/util/virclosecallbacks.c +++ b/src/util/virclosecallbacks.c @@ -300,7 +300,9 @@ virCloseCallbacksGetForConn(virCloseCallbacksPtr closeCallbacks, data.list = list; data.oom = false; + virObjectLock(closeCallbacks); virHashForEach(closeCallbacks->list, virCloseCallbacksGetOne, &data); + virObjectUnlock(closeCallbacks); if (data.oom) { VIR_FREE(list->entries); @@ -329,22 +331,15 @@ virCloseCallbacksRun(virCloseCallbacksPtr closeCallbacks, * them all from the hash. At that point we can release * the lock and run the callbacks safely. */ - virObjectLock(closeCallbacks); list = virCloseCallbacksGetForConn(closeCallbacks, conn); if (!list) return; for (i = 0; i < list->nentries; i++) { - char uuidstr[VIR_UUID_STRING_BUFLEN]; - virUUIDFormat(list->entries[i].uuid, uuidstr); - virHashRemoveEntry(closeCallbacks->list, uuidstr); - } - virObjectUnlock(closeCallbacks); - - for (i = 0; i < list->nentries; i++) { virDomainObjPtr vm; + virDomainObjPtr dom; - if (!(vm = virDomainObjListFindByUUID(domains, + if (!(vm = virDomainObjListFindByUUIDRef(domains, list->entries[i].uuid))) { char uuidstr[VIR_UUID_STRING_BUFLEN]; virUUIDFormat(list->entries[i].uuid, uuidstr); @@ -352,10 +347,20 @@ virCloseCallbacksRun(virCloseCallbacksPtr closeCallbacks, continue; } - vm = list->entries[i].callback(vm, conn, opaque); - if (vm) - virObjectUnlock(vm); + dom = list->entries[i].callback(vm, conn, opaque); + if (dom) + virObjectUnlock(dom); + virObjectUnref(vm); } + + virObjectLock(closeCallbacks); + for (i = 0; i < list->nentries; i++) { + char uuidstr[VIR_UUID_STRING_BUFLEN]; + virUUIDFormat(list->entries[i].uuid, uuidstr); + virHashRemoveEntry(closeCallbacks->list, uuidstr); + } + virObjectUnlock(closeCallbacks); + VIR_FREE(list->entries); VIR_FREE(list); } -- 2.8.3

2 2

[libvirt] [PATCH v3] libxl: define a per-domain logger.
by Cédric Bosdonnat 10 Jan '17

10 Jan '17

libxl doesn't provide a way to write one log for each domain. Thus we need to demux the messages. If our logger doesn't know to which domain to attribute a message, then it will write it to the default log file. Starting with Xen 4.9 (commit f9858025 and following), libxl will write the domain ID in an easy to grab manner. The logger introduced by this commit will use it to demux the libxl log messages. Thanks to the default log file, this logger will also work with older versions of Xen. --- * v3: add JSON-formatted libxl_domain_config to the newly opened file when creating a guest. * v2: addressed Jim's comments src/Makefile.am | 1 + src/libxl/libxl_conf.c | 38 +------- src/libxl/libxl_conf.h | 4 +- src/libxl/libxl_domain.c | 7 ++ src/libxl/libxl_driver.c | 2 + src/libxl/libxl_logger.c | 231 +++++++++++++++++++++++++++++++++++++++++++++++ src/libxl/libxl_logger.h | 40 ++++++++ 7 files changed, 286 insertions(+), 37 deletions(-) create mode 100644 src/libxl/libxl_logger.c create mode 100644 src/libxl/libxl_logger.h diff --git a/src/Makefile.am b/src/Makefile.am index 0dd2faaf6..58e3108a6 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -851,6 +851,7 @@ LIBXL_DRIVER_SOURCES = \ libxl/libxl_capabilities.c libxl/libxl_capabilities.h \ libxl/libxl_domain.c libxl/libxl_domain.h \ libxl/libxl_driver.c libxl/libxl_driver.h \ + libxl/libxl_logger.c libxl/libxl_logger.h \ libxl/libxl_migration.c libxl/libxl_migration.h UML_DRIVER_SOURCES = \ diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index b569ddad8..ac83b51c7 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -76,9 +76,7 @@ libxlDriverConfigDispose(void *obj) virObjectUnref(cfg->caps); libxl_ctx_free(cfg->ctx); - xtl_logger_destroy(cfg->logger); - if (cfg->logger_file) - VIR_FORCE_FCLOSE(cfg->logger_file); + libxlLoggerFree(cfg->logger); VIR_FREE(cfg->configDir); VIR_FREE(cfg->autostartDir); @@ -1356,8 +1354,6 @@ libxlDriverConfigPtr libxlDriverConfigNew(void) { libxlDriverConfigPtr cfg; - char *log_file = NULL; - xentoollog_level log_level = XTL_DEBUG; char ebuf[1024]; unsigned int free_mem; @@ -1386,9 +1382,6 @@ libxlDriverConfigNew(void) if (VIR_STRDUP(cfg->channelDir, LIBXL_CHANNEL_DIR) < 0) goto error; - if (virAsprintf(&log_file, "%s/libxl-driver.log", cfg->logDir) < 0) - goto error; - if (virFileMakePath(cfg->logDir) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("failed to create log dir '%s': %s"), @@ -1397,37 +1390,13 @@ libxlDriverConfigNew(void) goto error; } - if ((cfg->logger_file = fopen(log_file, "a")) == NULL) { - VIR_ERROR(_("Failed to create log file '%s': %s"), - log_file, virStrerror(errno, ebuf, sizeof(ebuf))); - goto error; - } - VIR_FREE(log_file); - - switch (virLogGetDefaultPriority()) { - case VIR_LOG_DEBUG: - log_level = XTL_DEBUG; - break; - case VIR_LOG_INFO: - log_level = XTL_INFO; - break; - case VIR_LOG_WARN: - log_level = XTL_WARN; - break; - case VIR_LOG_ERROR: - log_level = XTL_ERROR; - break; - } - - cfg->logger = - (xentoollog_logger *)xtl_createlogger_stdiostream(cfg->logger_file, - log_level, XTL_STDIOSTREAM_SHOW_DATE); + cfg->logger = libxlLoggerNew(cfg->logDir, virLogGetDefaultPriority()); if (!cfg->logger) { VIR_ERROR(_("cannot create logger for libxenlight, disabling driver")); goto error; } - if (libxl_ctx_alloc(&cfg->ctx, LIBXL_VERSION, 0, cfg->logger)) { + if (libxl_ctx_alloc(&cfg->ctx, LIBXL_VERSION, 0, (xentoollog_logger *)cfg->logger)) { VIR_ERROR(_("cannot initialize libxenlight context, probably not " "running in a Xen Dom0, disabling driver")); goto error; @@ -1478,7 +1447,6 @@ libxlDriverConfigNew(void) return cfg; error: - VIR_FREE(log_file); virObjectUnref(cfg); return NULL; } diff --git a/src/libxl/libxl_conf.h b/src/libxl/libxl_conf.h index 851f3afb4..69d78851a 100644 --- a/src/libxl/libxl_conf.h +++ b/src/libxl/libxl_conf.h @@ -41,6 +41,7 @@ # include "locking/lock_manager.h" # include "virfirmware.h" # include "libxl_capabilities.h" +# include "libxl_logger.h" # define LIBXL_DRIVER_NAME "xenlight" # define LIBXL_VNC_PORT_MIN 5900 @@ -74,8 +75,7 @@ struct _libxlDriverConfig { unsigned int version; /* log stream for driver-wide libxl ctx */ - FILE *logger_file; - xentoollog_logger *logger; + libxlLoggerPtr logger; /* libxl ctx for driver wide ops; getVersion, getNodeInfo, ... */ libxl_ctx *ctx; diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 5cde576ef..fbe7ee5ff 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -809,6 +809,8 @@ libxlDomainCleanup(libxlDriverPrivatePtr driver, VIR_FREE(xml); } + libxlLoggerCloseFile(cfg->logger, vm->def->id); + virDomainObjRemoveTransientDef(vm); virObjectUnref(cfg); } @@ -1127,6 +1129,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, libxl_asyncprogress_how aop_console_how; libxl_domain_restore_params params; unsigned int hostdev_flags = VIR_HOSTDEV_SP_PCI; + char *config_json = NULL; #ifdef LIBXL_HAVE_PVUSB hostdev_flags |= VIR_HOSTDEV_SP_USB; @@ -1290,6 +1293,9 @@ libxlDomainStart(libxlDriverPrivatePtr driver, * be cleaned up if there are any subsequent failures. */ vm->def->id = domid; + config_json = libxl_domain_config_to_json(cfg->ctx, &d_config); + + libxlLoggerOpenFile(cfg->logger, domid, vm->def->name, config_json); /* Always enable domain death events */ if (libxl_evenable_domain_death(cfg->ctx, vm->def->id, 0, &priv->deathW)) @@ -1366,6 +1372,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, cleanup: libxl_domain_config_dispose(&d_config); + VIR_FREE(config_json); VIR_FREE(dom_xml); VIR_FREE(managed_save_path); virDomainDefFree(def); diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c index 7e5d9b69e..325636c9a 100644 --- a/src/libxl/libxl_driver.c +++ b/src/libxl/libxl_driver.c @@ -406,6 +406,8 @@ libxlReconnectDomain(virDomainObjPtr vm, /* Update domid in case it changed (e.g. reboot) while we were gone? */ vm->def->id = d_info.domid; + libxlLoggerOpenFile(cfg->logger, vm->def->id, vm->def->name, NULL); + /* Update hostdev state */ if (virHostdevUpdateActiveDomainDevices(hostdev_mgr, LIBXL_DRIVER_NAME, vm->def, hostdev_flags) < 0) diff --git a/src/libxl/libxl_logger.c b/src/libxl/libxl_logger.c new file mode 100644 index 000000000..9ca745a93 --- /dev/null +++ b/src/libxl/libxl_logger.c @@ -0,0 +1,231 @@ +/* + * libxl_logger.c: libxl logger implementation + * + * Copyright (c) 2016 SUSE LINUX Products GmbH, Nuernberg, Germany. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Authors: + * Cédric Bosdonnat <cbosdonnat(a)suse.com> + */ +#include <config.h> + +#include <string.h> +#include <libxl.h> + +#include "internal.h" +#include "libxl_logger.h" +#include "util/viralloc.h" +#include "util/virerror.h" +#include "util/virfile.h" +#include "util/virhash.h" +#include "util/virstring.h" +#include "util/virtime.h" + +#define VIR_FROM_THIS VIR_FROM_LIBXL + +VIR_LOG_INIT("libxl.libxl_logger"); + +typedef struct xentoollog_logger_libvirt xentoollog_logger_libvirt; + +struct xentoollog_logger_libvirt { + xentoollog_logger vtable; + xentoollog_level minLevel; + const char *logDir; + + /* map storing the opened fds: "domid" -> FILE* */ + virHashTablePtr files; + FILE *defaultLogFile; +}; + +static void +libxlLoggerFileFree(void *payload, const void *key ATTRIBUTE_UNUSED) +{ + FILE *file = payload; + VIR_FORCE_FCLOSE(file); + file = NULL; +} + +ATTRIBUTE_FMT_PRINTF(5, 0) static void +libvirt_vmessage(xentoollog_logger *logger_in, + xentoollog_level level, + int errnoval, + const char *context, + const char *format, + va_list args) +{ + xentoollog_logger_libvirt *lg = (xentoollog_logger_libvirt *)logger_in; + FILE *logFile = lg->defaultLogFile; + char timestamp[VIR_TIME_STRING_BUFLEN]; + char *message = NULL; + char *start, *end; + char ebuf[1024]; + + VIR_DEBUG("libvirt_vmessage: context='%s' format='%s'", context, format); + + if (level < lg->minLevel) + return; + + if (virVasprintf(&message, format, args) < 0) + return; + + /* Should we print to a domain-specific log file? */ + if ((start = strstr(message, ": Domain ")) && + (end = strstr(start + 9, ":"))) { + FILE *domainLogFile; + + VIR_DEBUG("Found domain log message"); + + start = start + 9; + *end = '\0'; + + domainLogFile = virHashLookup(lg->files, start); + if (domainLogFile) + logFile = domainLogFile; + + *end = ':'; + } + + /* Do the actual print to the log file */ + if (virTimeStringNowRaw(timestamp) < 0) + timestamp[0] = '\0'; + + fprintf(logFile, "%s: ", timestamp); + if (context) + fprintf(logFile, "%s: ", context); + + fprintf(logFile, "%s", message); + + if (errnoval >= 0) + fprintf(logFile, ": %s", virStrerror(errnoval, ebuf, sizeof(ebuf))); + + fputc('\n', logFile); + fflush(logFile); + + VIR_FREE(message); +} + +static void +libvirt_progress(xentoollog_logger *logger_in ATTRIBUTE_UNUSED, + const char *context ATTRIBUTE_UNUSED, + const char *doingwhat ATTRIBUTE_UNUSED, + int percent ATTRIBUTE_UNUSED, + unsigned long done ATTRIBUTE_UNUSED, + unsigned long total ATTRIBUTE_UNUSED) +{ + /* This function purposedly does nothing: it's no logging info */ +} + +static void +libvirt_destroy(xentoollog_logger *logger_in) +{ + xentoollog_logger_libvirt *lg = (xentoollog_logger_libvirt*)logger_in; + VIR_FREE(lg); +} + + +libxlLoggerPtr libxlLoggerNew(const char *logDir, + virLogPriority minLevel) +{ + xentoollog_logger_libvirt logger; + libxlLoggerPtr logger_out = NULL; + char *path = NULL; + + switch (minLevel) { + case VIR_LOG_DEBUG: + logger.minLevel = XTL_DEBUG; + break; + case VIR_LOG_INFO: + logger.minLevel = XTL_INFO; + break; + case VIR_LOG_WARN: + logger.minLevel = XTL_WARN; + break; + case VIR_LOG_ERROR: + logger.minLevel = XTL_ERROR; + break; + } + logger.logDir = logDir; + + if ((logger.files = virHashCreate(3, libxlLoggerFileFree)) == NULL) + return NULL; + + if (virAsprintf(&path, "%s/libxl-driver.log", logDir) < 0) + goto error; + + if ((logger.defaultLogFile = fopen(path, "a")) == NULL) + goto error; + + logger_out = XTL_NEW_LOGGER(libvirt, logger); + + cleanup: + VIR_FREE(path); + return logger_out; + + error: + virHashFree(logger.files); + goto cleanup; +} + +void libxlLoggerFree(libxlLoggerPtr logger) +{ + xentoollog_logger *xtl_logger = (xentoollog_logger*)logger; + if (logger->defaultLogFile) + VIR_FORCE_FCLOSE(logger->defaultLogFile); + virHashFree(logger->files); + xtl_logger_destroy(xtl_logger); +} + +void libxlLoggerOpenFile(libxlLoggerPtr logger, int id, + const char *name, + const char *domain_config) +{ + char *path = NULL; + FILE *logFile = NULL; + char *domidstr = NULL; + char ebuf[1024]; + + if (virAsprintf(&path, "%s/%s.log", logger->logDir, name) < 0 || + virAsprintf(&domidstr, "%d", id) < 0) + goto cleanup; + + if (!(logFile = fopen(path, "a"))) { + VIR_WARN("Failed to open log file %s: %s", + path, virStrerror(errno, ebuf, sizeof(ebuf))); + goto cleanup; + } + ignore_value(virHashAddEntry(logger->files, domidstr, logFile)); + + /* domain_config is non NULL only when starting a new domain */ + if (domain_config) { + fprintf(logFile, "Domain start: %s\n", domain_config); + fflush(logFile); + } + + cleanup: + VIR_FREE(path); + VIR_FREE(domidstr); +} + +void libxlLoggerCloseFile(libxlLoggerPtr logger, int id) +{ + char *domidstr = NULL; + if (virAsprintf(&domidstr, "%d", id) < 0) + return; + + ignore_value(virHashRemoveEntry(logger->files, domidstr)); + + VIR_FREE(domidstr); +} diff --git a/src/libxl/libxl_logger.h b/src/libxl/libxl_logger.h new file mode 100644 index 000000000..92b222578 --- /dev/null +++ b/src/libxl/libxl_logger.h @@ -0,0 +1,40 @@ +/* + * libxl_logger.h: libxl logger implementation + * + * Copyright (c) 2016 SUSE LINUX Products GmbH, Nuernberg, Germany. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Authors: + * Cédric Bosdonnat <cbosdonnat(a)suse.com> + */ + +#ifndef __LIBXL_LOGGER_H +# define __LIBXL_LOGGER_H + +# include "util/virlog.h" + +typedef struct xentoollog_logger_libvirt libxlLogger; +typedef libxlLogger *libxlLoggerPtr; + +libxlLoggerPtr libxlLoggerNew(const char *logDir, + virLogPriority minLevel); +void libxlLoggerFree(libxlLoggerPtr logger); + +void libxlLoggerOpenFile(libxlLoggerPtr logger, int id, const char *name, + const char *domain_config); +void libxlLoggerCloseFile(libxlLoggerPtr logger, int id); + +#endif /* __LIBXL_LOGGER_H */ -- 2.11.0

2 1

[libvirt] [PATCH 1/2] perf: add cache_l1d perf event support
by Nitesh Konkar 10 Jan '17

10 Jan '17

This patch adds support and documentation for a generalized hardware cache event called cache_l1d perf event. Signed-off-by: Nitesh Konkar <nitkon12(a)linux.vnet.ibm.com> --- docs/formatdomain.html.in | 7 +++++++ docs/schemas/domaincommon.rng | 1 + include/libvirt/libvirt-domain.h | 11 +++++++++++ src/libvirt-domain.c | 2 ++ src/qemu/qemu_driver.c | 1 + src/util/virperf.c | 6 +++++- src/util/virperf.h | 1 + tests/genericxml2xmlindata/generic-perf.xml | 1 + tools/virsh.pod | 5 ++++- 9 files changed, 33 insertions(+), 2 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index f7bef51..e192b77 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -1937,6 +1937,7 @@ <event name='stalled_cycles_frontend' enabled='no'/> <event name='stalled_cycles_backend' enabled='no'/> <event name='ref_cpu_cycles' enabled='no'/> + <event name='cache_l1d' enabled='no'/> </perf> ... </pre> @@ -2015,6 +2016,12 @@ by applications running on the platform</td> <td><code>perf.ref_cpu_cycles</code></td> </tr> + <tr> + <td><code>cache_l1d</code></td> + <td>the count of total level 1 data cache by applications running on + the platform</td> + <td><code>perf.cache_l1d</code></td> + </tr> </table> <h3><a name="elementsDevices">Devices</a></h3> diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 4d76315..be0a609 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -433,6 +433,7 @@ <value>stalled_cycles_frontend</value> <value>stalled_cycles_backend</value> <value>ref_cpu_cycles</value> + <value>cache_l1d</value> </choice> </attribute> <attribute name="enabled"> diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 6362947..479d417 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -2188,6 +2188,17 @@ void virDomainStatsRecordListFree(virDomainStatsRecordPtr *stats); */ # define VIR_PERF_PARAM_REF_CPU_CYCLES "ref_cpu_cycles" +/** + * VIR_PERF_PARAM_CACHE_L1D: + * + * Macro for typed parameter name that represents cache_l1d + * perf event which can be used to measure the count of total + * level 1 data cache by applications running on the platform. + * It corresponds to the "perf.cache_l1d" field in the + * *Stats APIs. + */ +# define VIR_PERF_PARAM_CACHE_L1D "cache_l1d" + int virDomainGetPerfEvents(virDomainPtr dom, virTypedParameterPtr *params, int *nparams, diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 5b3e842..3023f30 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -11250,6 +11250,8 @@ virConnectGetDomainCapabilities(virConnectPtr conn, * CPU frequency scaling by applications running * as unsigned long long. It is produced by the * ref_cpu_cycles perf event. + * "perf.cache_l1d" - The count of total level 1 data cache as unsigned + * long long. It is produced by cache_l1d perf event. * * Note that entire stats groups or individual stat fields may be missing from * the output in case they are not supported by the given hypervisor, are not diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index b359e77..80db495 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -9859,6 +9859,7 @@ qemuDomainSetPerfEvents(virDomainPtr dom, VIR_PERF_PARAM_STALLED_CYCLES_FRONTEND, VIR_TYPED_PARAM_BOOLEAN, VIR_PERF_PARAM_STALLED_CYCLES_BACKEND, VIR_TYPED_PARAM_BOOLEAN, VIR_PERF_PARAM_REF_CPU_CYCLES, VIR_TYPED_PARAM_BOOLEAN, + VIR_PERF_PARAM_CACHE_L1D, VIR_TYPED_PARAM_BOOLEAN, NULL) < 0) return -1; diff --git a/src/util/virperf.c b/src/util/virperf.c index f64692b..8554723 100644 --- a/src/util/virperf.c +++ b/src/util/virperf.c @@ -43,7 +43,8 @@ VIR_ENUM_IMPL(virPerfEvent, VIR_PERF_EVENT_LAST, "cache_references", "cache_misses", "branch_instructions", "branch_misses", "bus_cycles", "stalled_cycles_frontend", - "stalled_cycles_backend", "ref_cpu_cycles"); + "stalled_cycles_backend", "ref_cpu_cycles", + "cache_l1d"); struct virPerfEvent { int type; @@ -112,6 +113,9 @@ static struct virPerfEventAttr attrs[] = { .attrConfig = 0, # endif }, + {.type = VIR_PERF_EVENT_CACHE_L1D, + .attrType = PERF_TYPE_HW_CACHE, + .attrConfig = PERF_COUNT_HW_CACHE_L1D}, }; typedef struct virPerfEventAttr *virPerfEventAttrPtr; diff --git a/src/util/virperf.h b/src/util/virperf.h index 1f43c92..4c562af 100644 --- a/src/util/virperf.h +++ b/src/util/virperf.h @@ -47,6 +47,7 @@ typedef enum { the backend of the instruction processor pipeline */ VIR_PERF_EVENT_REF_CPU_CYCLES, /* Count of ref cpu cycles */ + VIR_PERF_EVENT_CACHE_L1D, /* Count of level 1 data cache*/ VIR_PERF_EVENT_LAST } virPerfEventType; diff --git a/tests/genericxml2xmlindata/generic-perf.xml b/tests/genericxml2xmlindata/generic-perf.xml index 437cd65..d1418d0 100644 --- a/tests/genericxml2xmlindata/generic-perf.xml +++ b/tests/genericxml2xmlindata/generic-perf.xml @@ -26,6 +26,7 @@ <event name='stalled_cycles_frontend' enabled='yes'/> <event name='stalled_cycles_backend' enabled='yes'/> <event name='ref_cpu_cycles' enabled='yes'/> + <event name='cache_l1d' enabled='yes'/> </perf> <devices> </devices> diff --git a/tools/virsh.pod b/tools/virsh.pod index ef91223..e039db5 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -951,7 +951,8 @@ I<--perf> returns the statistics of all enabled perf events: "perf.bus_cycles" - the count of bus cycles, "perf.stalled_cycles_frontend" - the count of stalled frontend cpu cycles, "perf.stalled_cycles_backend" - the count of stalled backend cpu cycles, -"perf.ref_cpu_cycles" - the count of ref cpu cycles +"perf.ref_cpu_cycles" - the count of ref cpu cycles, +"perf.cache_l1d" - the count of level 1 data cache See the B<perf> command for more details about each event. @@ -2316,6 +2317,8 @@ B<Valid perf event names> ref_cpu_cycles - Provides the count of total cpu cycles not affected by CPU frequency scaling by applications running on the platform. + cache_l1d - Provides the count of total level 1 data cache + by applications running on the platform. B<Note>: The statistics can be retrieved using the B<domstats> command using the I<--perf> flag. -- 1.9.3

2 2

[libvirt] [PATCH] docs: Reword virsh manpage for --uuid --name --table options
by Nitesh Konkar 10 Jan '17

10 Jan '17

The virsh manpage lists options --uuid and --name as mutually exclusive along option --table when actually the optoin --table is mutually exclusive and can't go with options --uuid and/or --name. This patch rewords the virsh manpage to state the correct meaning. Signed-off-by: Nitesh Konkar <nitkon12(a)linux.vnet.ibm.com> --- tools/virsh.pod | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/tools/virsh.pod b/tools/virsh.pod index ef91223..cc99da1 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -529,9 +529,8 @@ output should be used. This is the default. If both I<--name> and I<--uuid> are specified, domain UUID's and names are printed side by side without any header. Flag I<--table> specifies that the legacy table-formatted output should be used. This is the -default if neither I<--name> nor I<--uuid> are specified. Options -I<--uuid> and I<--name> are mutually exclusive if option I<--table> is -specified. +default if neither I<--name> nor I<--uuid> are specified. Option +I<--table> is mutually exclusive with options I<--uuid> and I<--name>. If I<--title> is specified, then the short domain description (title) is printed in an extra column. This flag is usable only with the default -- 1.9.3

2 1

[libvirt] [PATCH v2] cmdPerf: Display enabled/disabled message on perf event enable/disable
by Nitesh Konkar 10 Jan '17

10 Jan '17

Currently no message is displayed on successful perf event enable or disable. This patch gives the enabled/disabled status message on successful enabling/disabling of the perf event. Eg:virsh perf Domain --enable instructions --disable cache_misses instructions : enabled cache_misses : disabled Signed-off-by: Nitesh Konkar <nitkon12(a)linux.vnet.ibm.com> --- tools/virsh-domain.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 3a6fa5c..287ca28 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -8848,13 +8848,27 @@ virshParseEventStr(const char *event, return ret; } +static void +virshPrintPerfStatus(vshControl *ctl, virTypedParameterPtr params, int nparams) +{ + size_t i; + + for (i = 0; i < nparams; i++) { + if (params[i].type == VIR_TYPED_PARAM_BOOLEAN && + params[i].value.b) { + vshPrint(ctl, "%-15s: %s\n", params[i].field, _("enabled")); + } else { + vshPrint(ctl, "%-15s: %s\n", params[i].field, _("disabled")); + } + } +} + static bool cmdPerf(vshControl *ctl, const vshCmd *cmd) { virDomainPtr dom; int nparams = 0; int maxparams = 0; - size_t i; virTypedParameterPtr params = NULL; bool ret = false; const char *enable = NULL, *disable = NULL; @@ -8891,18 +8905,13 @@ cmdPerf(vshControl *ctl, const vshCmd *cmd) vshError(ctl, "%s", _("Unable to get perf events")); goto cleanup; } - for (i = 0; i < nparams; i++) { - if (params[i].type == VIR_TYPED_PARAM_BOOLEAN && - params[i].value.b) { - vshPrint(ctl, "%-15s: %s\n", params[i].field, _("enabled")); - } else { - vshPrint(ctl, "%-15s: %s\n", params[i].field, _("disabled")); - } - } + virshPrintPerfStatus(ctl, params, nparams); } else { if (virDomainSetPerfEvents(dom, params, nparams, flags) != 0) { vshError(ctl, "%s", _("Unable to enable/disable perf events")); goto cleanup; + } else { + virshPrintPerfStatus(ctl, params, nparams); } } -- 1.9.3

2 1

[libvirt] [PATCH 1/2] perf: Refactor perf code
by Nitesh Konkar 10 Jan '17

10 Jan '17

Avoid unnecessary calling of function vshCommandOptStringReq. In the current code the function vshCommandOptStringReq is called irrespective of whether --enable and/or --disable is present in the command line. Eg: 'virsh perf domainName' also results in calling this function twice. This patch fixes this. Signed-off-by: Nitesh Konkar <nitkon12(a)linux.vnet.ibm.com> --- tools/virsh-domain.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 3a6fa5c..91de532 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -8862,6 +8862,8 @@ cmdPerf(vshControl *ctl, const vshCmd *cmd) bool current = vshCommandOptBool(cmd, "current"); bool config = vshCommandOptBool(cmd, "config"); bool live = vshCommandOptBool(cmd, "live"); + bool shouldEnable = vshCommandOptBool(cmd, "enable"); + bool shouldDisable = vshCommandOptBool(cmd, "disable"); VSH_EXCLUSIVE_OPTIONS_VAR(current, live); VSH_EXCLUSIVE_OPTIONS_VAR(current, config); @@ -8871,9 +8873,15 @@ cmdPerf(vshControl *ctl, const vshCmd *cmd) if (live) flags |= VIR_DOMAIN_AFFECT_LIVE; - if (vshCommandOptStringReq(ctl, cmd, "enable", &enable) < 0 || - vshCommandOptStringReq(ctl, cmd, "disable", &disable) < 0) - return false; + if (shouldEnable) { + if (vshCommandOptStringReq(ctl, cmd, "enable", &enable) < 0) + return false; + } + + if (shouldDisable) { + if (vshCommandOptStringReq(ctl, cmd, "disable", &disable) < 0) + return false; + } if (!(dom = virshCommandOptDomain(ctl, cmd, NULL))) return false; -- 1.9.3

3 3

[libvirt] [PATCH] lxc: ensure libvirt_lxc and qemu-nbd move into systemd machine slice
by Daniel P. Berrange 09 Jan '17

09 Jan '17

Currently when spawning containers with systemd, the container PID 1 will get moved into the systemd machine slice. Libvirt then manually moves the libvirt_lxc and qemu-nbd processes into the cgroups associated with the slice, but skips the systemd controller cgroup. This means that from systemd's POV, libvirt_lxc and qemu-nbd are still part of the libvirtd.service unit. On systemctl daemon-reload, it will notice that libvirt_lxc & qemu-nbd are in the libvirtd.service unit for the systemd controller, but in the machine cgroups for resources. Systemd will thus move them back into the libvirtd.service resource cgroups next time libvirtd is restarted. This causes libvirtd to kill off the container due to incorrect cgroup placement. The solution is to ensure that when moving libvirt_lxc & qemu-nbd, we also move the systemd cgroup controller placement. Normally this is not something we ever want todo, but this is a special case as we are intentionally wanting to move them to a different systemd unit. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- src/libvirt_private.syms | 1 + src/lxc/lxc_controller.c | 4 ++-- src/util/vircgroup.c | 52 +++++++++++++++++++++++++++++++++++++----------- src/util/vircgroup.h | 1 + 4 files changed, 44 insertions(+), 14 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 2d23e46..2dd5809 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1287,6 +1287,7 @@ virBufferVasprintf; # util/vircgroup.h +virCgroupAddMachineTask; virCgroupAddTask; virCgroupAddTaskController; virCgroupAllowAllDevices; diff --git a/src/lxc/lxc_controller.c b/src/lxc/lxc_controller.c index 2170b0a..88ba8aa 100644 --- a/src/lxc/lxc_controller.c +++ b/src/lxc/lxc_controller.c @@ -869,12 +869,12 @@ static int virLXCControllerSetupCgroupLimits(virLXCControllerPtr ctrl) ctrl->nicindexes))) goto cleanup; - if (virCgroupAddTask(ctrl->cgroup, getpid()) < 0) + if (virCgroupAddMachineTask(ctrl->cgroup, getpid()) < 0) goto cleanup; /* Add all qemu-nbd tasks to the cgroup */ for (i = 0; i < ctrl->nnbdpids; i++) { - if (virCgroupAddTask(ctrl->cgroup, ctrl->nbdpids[i]) < 0) + if (virCgroupAddMachineTask(ctrl->cgroup, ctrl->nbdpids[i]) < 0) goto cleanup; } diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index 80ce43c..ddf19e9 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -1184,16 +1184,8 @@ virCgroupNew(pid_t pid, } -/** - * virCgroupAddTask: - * - * @group: The cgroup to add a task to - * @pid: The pid of the task to add - * - * Returns: 0 on success, -1 on error - */ -int -virCgroupAddTask(virCgroupPtr group, pid_t pid) +static int +virCgroupAddTaskInternal(virCgroupPtr group, pid_t pid, bool withSystemd) { int ret = -1; size_t i; @@ -1203,8 +1195,10 @@ virCgroupAddTask(virCgroupPtr group, pid_t pid) if (!group->controllers[i].mountPoint) continue; - /* We must never add tasks in systemd's hierarchy */ - if (i == VIR_CGROUP_CONTROLLER_SYSTEMD) + /* We must never add tasks in systemd's hierarchy + * unless we're intentionally trying to move a + * task into a systemd machine scope */ + if (i == VIR_CGROUP_CONTROLLER_SYSTEMD && !withSystemd) continue; if (virCgroupAddTaskController(group, pid, i) < 0) @@ -1216,6 +1210,40 @@ virCgroupAddTask(virCgroupPtr group, pid_t pid) return ret; } +/** + * virCgroupAddTask: + * + * @group: The cgroup to add a task to + * @pid: The pid of the task to add + * + * Will add the task to all controllers, except the + * systemd unit controller. + * + * Returns: 0 on success, -1 on error + */ +int +virCgroupAddTask(virCgroupPtr group, pid_t pid) +{ + return virCgroupAddTaskInternal(group, pid, false); +} + +/** + * virCgroupAddMachineTask: + * + * @group: The cgroup to add a task to + * @pid: The pid of the task to add + * + * Will add the task to all controllers, including the + * systemd unit controller. + * + * Returns: 0 on success, -1 on error + */ +int +virCgroupAddMachineTask(virCgroupPtr group, pid_t pid) +{ + return virCgroupAddTaskInternal(group, pid, true); +} + /** * virCgroupAddTaskController: diff --git a/src/util/vircgroup.h b/src/util/vircgroup.h index 4b8f3ff..2de1bf2 100644 --- a/src/util/vircgroup.h +++ b/src/util/vircgroup.h @@ -131,6 +131,7 @@ int virCgroupPathOfController(virCgroupPtr group, char **path); int virCgroupAddTask(virCgroupPtr group, pid_t pid); +int virCgroupAddMachineTask(virCgroupPtr group, pid_t pid); int virCgroupAddTaskController(virCgroupPtr group, pid_t pid, -- 2.9.3

3 3