[libvirt] [PATCH v4 0/8] Add perf and Intel CMT feature support
The series mainly adds Intel CMT feature support into libvirt. CMT is new introduced PQos (Platform Qos) feature to monitor the usage of cache by applications running on the platform. Currently CMT patches has been merged into Linux kernel mainline. The CMT implementation in Linux kernel is based on perf mechanism and there is no support for perf in libvirt, and so this series firstly add perf support into libvirt, including two public API and a set of util interfaces. And based on these APIs and interfaces, thie series implements CMT perf event support. Current state: * 1/8 - perf: add new public APIs for perf event - ACKed by Daniel * 2/8 - perf: implement the remote protocol for perf event - ACKed by Daniel * 3/8 - perf: implement a set of util functions for perf event - ACKed by Daniel * 4/8 - qemu_driver: add support to perf event - restart perf in qemuProcessReconnect and qemuProcessAttach in patch 6/8 * 5/8 - perf: add new xml element - ACKed by Daniel * 6/8 - perf: reenable perf events when libvirtd restart - add qemuDomainPerfRestart() into qemuProcessReconnect() and qemuProcessAttach() * 7/8 - virsh: implement new command to support perf - ACKed by Daniel * 8/8 - virsh: extend domstats command - ACKed by Daniel TODO: 1. add support for new APIs into libvirt-python library. Changes since v1: * change perf APIs implementation to match new style of the API. * add new xml element * reenable all perf events previously enabled when libvirtd daemon restart. * redesign perf util functions. Changes since v2: * add an example XML file to the test suite. * add virPerfReadEvent(). * change 'perf' xml element to new style. * change 'perf' command to new stype. Changes since v3: * move qemuDomainPerfRestart() into qemuProcessReconnect() and qemuProcessAttach(). Qiaowei Ren (8): perf: add new public APIs for perf event perf: implement the remote protocol for perf event perf: implement a set of util functions for perf event qemu_driver: add support to perf event perf: add new xml element perf: reenable perf events when libvirtd restart virsh: implement new command to support perf virsh: extend domstats command daemon/remote.c | 47 ++++ docs/schemas/domaincommon.rng | 27 +++ include/libvirt/libvirt-domain.h | 19 ++ include/libvirt/virterror.h | 2 + src/Makefile.am | 1 + src/conf/domain_conf.c | 111 ++++++++++ src/conf/domain_conf.h | 10 + src/driver-hypervisor.h | 12 + src/libvirt-domain.c | 93 ++++++++ src/libvirt_private.syms | 12 + src/libvirt_public.syms | 2 + src/qemu/qemu_domain.h | 3 + src/qemu/qemu_driver.c | 159 +++++++++++++ src/qemu/qemu_process.c | 44 ++++ src/remote/remote_driver.c | 39 ++++ src/remote/remote_protocol.x | 30 ++- src/remote_protocol-structs | 18 ++ src/util/virerror.c | 2 + src/util/virperf.c | 307 ++++++++++++++++++++++++++ src/util/virperf.h | 63 ++++++ tests/domainschemadata/domain-perf-simple.xml | 20 ++ tools/virsh-domain-monitor.c | 7 + tools/virsh-domain.c | 128 +++++++++++ tools/virsh.pod | 27 ++- 24 files changed, 1180 insertions(+), 3 deletions(-) create mode 100644 src/util/virperf.c create mode 100644 src/util/virperf.h create mode 100644 tests/domainschemadata/domain-perf-simple.xml -- 1.9.1
API agreed on in https://www.redhat.com/archives/libvir-list/2015-October/msg00872.html * include/libvirt/libvirt-domain.h (virDomainGetPerfEvents, virDomainSetPerfEvents): New declarations. * src/libvirt_public.syms: Export new symbols. * src/driver-hypervisor.h (virDrvDomainGetPerfEvents, virDrvDomainSetPerfEvents): New typedefs. * src/libvirt-domain.c: Implement virDomainGetPerfEvents and virDomainSetPerfEvents. Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- include/libvirt/libvirt-domain.h | 18 ++++++++ src/driver-hypervisor.h | 12 ++++++ src/libvirt-domain.c | 93 ++++++++++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 2 + 4 files changed, 125 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 7e06796..b171cdf 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1835,6 +1835,24 @@ int virDomainListGetStats(virDomainPtr *doms, void virDomainStatsRecordListFree(virDomainStatsRecordPtr *stats); /* + * Perf Event API + */ + +/** + * VIR_PERF_PARAM_CMT: + * + * Macro for typed parameter name that represents CMT perf event. + */ +# define VIR_PERF_PARAM_CMT "cmt" + +int virDomainGetPerfEvents(virDomainPtr dom, + virTypedParameterPtr *params, + int *nparams); +int virDomainSetPerfEvents(virDomainPtr dom, + virTypedParameterPtr params, + int nparams); + +/* * BlockJob API */ diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index 9bc3211..d0e7298 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -962,6 +962,16 @@ typedef int unsigned int flags); typedef int +(*virDrvDomainGetPerfEvents)(virDomainPtr dom, + virTypedParameterPtr *params, + int *nparams); + +typedef int +(*virDrvDomainSetPerfEvents)(virDomainPtr dom, + virTypedParameterPtr params, + int nparams); + +typedef int (*virDrvDomainBlockJobAbort)(virDomainPtr dom, const char *path, unsigned int flags); @@ -1427,6 +1437,8 @@ struct _virHypervisorDriver { virDrvConnectSetKeepAlive connectSetKeepAlive; virDrvConnectIsAlive connectIsAlive; virDrvNodeSuspendForDuration nodeSuspendForDuration; + virDrvDomainGetPerfEvents domainGetPerfEvents; + virDrvDomainSetPerfEvents domainSetPerfEvents; virDrvDomainSetBlockIoTune domainSetBlockIoTune; virDrvDomainGetBlockIoTune domainGetBlockIoTune; virDrvDomainGetCPUStats domainGetCPUStats; diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index caa81c0..42031bc 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -9691,6 +9691,99 @@ virDomainOpenChannel(virDomainPtr dom, /** + * virDomainGetPerfEvents: + * @domain: a domain object + * @params: where to store perf events setting + * @nparams: number of items in @params + * + * Get all perf events setting. Possible fields returned in @params are + * defined by VIR_DOMAIN_PERF_* macros and new fields will likely be + * introduced in the future. + * + * Returns -1 in case of failure, 0 in case of success. + */ +int virDomainGetPerfEvents(virDomainPtr domain, + virTypedParameterPtr *params, + int *nparams) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "params=%p, nparams=%p", + params, nparams); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + virCheckNonNullArgGoto(params, error); + virCheckNonNullArgGoto(nparams, error); + + conn = domain->conn; + + if (conn->driver->domainGetPerfEvents) { + int ret; + ret = conn->driver->domainGetPerfEvents(domain, params, nparams); + if (ret < 0) + goto error; + return ret; + } + virReportUnsupportedError(); + + error: + virDispatchError(domain->conn); + return -1; +} + + +/** + * virDomainSetPerfEvents: + * @domain: a domain object + * @params: pointer to perf events parameter object + * @nparams: number of perf event parameters (this value can be the same + * less than the number of parameters supported) + * + * Enable or disable the particular list of perf events you care about. + * + * Returns -1 in case of error, 0 in case of success. + */ +int virDomainSetPerfEvents(virDomainPtr domain, + virTypedParameterPtr params, + int nparams) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "params=%p, nparams=%d", + params, nparams); + VIR_TYPED_PARAMS_DEBUG(params, nparams); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + conn = domain->conn; + + virCheckReadOnlyGoto(conn->flags, error); + virCheckNonNullArgGoto(params, error); + virCheckPositiveArgGoto(nparams, error); + + if (virTypedParameterValidateSet(conn, params, nparams) < 0) + goto error; + + if (conn->driver->domainSetPerfEvents) { + int ret; + ret = conn->driver->domainSetPerfEvents(domain, params, nparams); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(domain->conn); + return -1; +} + + +/** * virDomainBlockJobAbort: * @dom: pointer to domain object * @disk: path to the block device, or device shorthand diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 221743a..1e920d6 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -728,6 +728,8 @@ LIBVIRT_1.2.19 { LIBVIRT_1.3.3 { global: virDomainMigrateStartPostCopy; + virDomainGetPerfEvents; + virDomainSetPerfEvents; } LIBVIRT_1.2.19; # .... define new API here using predicted next version number .... -- 1.9.1
Add remote support for perf event. Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- daemon/remote.c | 47 ++++++++++++++++++++++++++++++++++++++++++++ src/remote/remote_driver.c | 39 ++++++++++++++++++++++++++++++++++++ src/remote/remote_protocol.x | 30 +++++++++++++++++++++++++++- src/remote_protocol-structs | 18 +++++++++++++++++ 4 files changed, 133 insertions(+), 1 deletion(-) diff --git a/daemon/remote.c b/daemon/remote.c index 3bf94fa..bbe22b0 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -2672,6 +2672,53 @@ remoteDispatchNodeGetMemoryStats(virNetServerPtr server ATTRIBUTE_UNUSED, } static int +remoteDispatchDomainGetPerfEvents(virNetServerPtr server ATTRIBUTE_UNUSED, + virNetServerClientPtr client ATTRIBUTE_UNUSED, + virNetMessagePtr msg ATTRIBUTE_UNUSED, + virNetMessageErrorPtr rerr, + remote_domain_get_perf_events_args *args, + remote_domain_get_perf_events_ret *ret) +{ + virDomainPtr dom = NULL; + virTypedParameterPtr params = NULL; + int nparams = 0; + int rv = -1; + struct daemonClientPrivate *priv = + virNetServerClientGetPrivateData(client); + + if (!priv->conn) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("connection not open")); + goto cleanup; + } + + if (!(dom = get_nonnull_domain(priv->conn, args->dom))) + goto cleanup; + + if (virDomainGetPerfEvents(dom, ¶ms, &nparams) < 0) + goto cleanup; + + if (nparams > REMOTE_DOMAIN_MEMORY_PARAMETERS_MAX) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("nparams too large")); + goto cleanup; + } + + if (virTypedParamsSerialize(params, nparams, + (virTypedParameterRemotePtr *) &ret->params.params_val, + &ret->params.params_len, + 0) < 0) + goto cleanup; + + rv = 0; + + cleanup: + if (rv < 0) + virNetMessageSaveError(rerr); + virTypedParamsFree(params, nparams); + virObjectUnref(dom); + return rv; +} + +static int remoteDispatchDomainGetBlockJobInfo(virNetServerPtr server ATTRIBUTE_UNUSED, virNetServerClientPtr client ATTRIBUTE_UNUSED, virNetMessagePtr msg ATTRIBUTE_UNUSED, diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index be7b2f4..b3c7b9c 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -1817,6 +1817,43 @@ remoteDomainGetNumaParameters(virDomainPtr domain, } static int +remoteDomainGetPerfEvents(virDomainPtr domain, + virTypedParameterPtr *params, + int *nparams) +{ + int rv = -1; + remote_domain_get_perf_events_args args; + remote_domain_get_perf_events_ret ret; + struct private_data *priv = domain->conn->privateData; + + remoteDriverLock(priv); + + make_nonnull_domain(&args.dom, domain); + + memset(&ret, 0, sizeof(ret)); + if (call(domain->conn, priv, 0, REMOTE_PROC_DOMAIN_GET_PERF_EVENTS, + (xdrproc_t) xdr_remote_domain_get_perf_events_args, (char *) &args, + (xdrproc_t) xdr_remote_domain_get_perf_events_ret, (char *) &ret) == -1) + goto done; + + if (virTypedParamsDeserialize((virTypedParameterRemotePtr) ret.params.params_val, + ret.params.params_len, + REMOTE_DOMAIN_PERF_EVENTS_MAX, + params, + nparams) < 0) + goto cleanup; + + rv = 0; + + cleanup: + xdr_free((xdrproc_t) xdr_remote_domain_get_perf_events_ret, + (char *) &ret); + done: + remoteDriverUnlock(priv); + return rv; +} + +static int remoteDomainGetBlkioParameters(virDomainPtr domain, virTypedParameterPtr params, int *nparams, unsigned int flags) @@ -7585,6 +7622,8 @@ static virHypervisorDriver hypervisor_driver = { .domainGetMemoryParameters = remoteDomainGetMemoryParameters, /* 0.8.5 */ .domainSetBlkioParameters = remoteDomainSetBlkioParameters, /* 0.9.0 */ .domainGetBlkioParameters = remoteDomainGetBlkioParameters, /* 0.9.0 */ + .domainGetPerfEvents = remoteDomainGetPerfEvents, /* 1.3.3 */ + .domainSetPerfEvents = remoteDomainSetPerfEvents, /* 1.3.3 */ .domainGetInfo = remoteDomainGetInfo, /* 0.3.0 */ .domainGetState = remoteDomainGetState, /* 0.9.2 */ .domainGetControlInfo = remoteDomainGetControlInfo, /* 0.9.3 */ diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 017bdbb..cb19bc0 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -109,6 +109,9 @@ const REMOTE_DOMAIN_BLOCK_IO_TUNE_PARAMETERS_MAX = 16; /* Upper limit on list of numa parameters. */ const REMOTE_DOMAIN_NUMA_PARAMETERS_MAX = 16; +/* Upper limit on list of perf events. */ +const REMOTE_DOMAIN_PERF_EVENTS_MAX = 64; + /* Upper limit on block copy tunable parameters. */ const REMOTE_DOMAIN_BLOCK_COPY_PARAMETERS_MAX = 16; @@ -628,6 +631,19 @@ struct remote_domain_get_numa_parameters_ret { int nparams; }; +struct remote_domain_set_perf_events_args { + remote_nonnull_domain dom; + remote_typed_param params<REMOTE_DOMAIN_PERF_EVENTS_MAX>; +}; + +struct remote_domain_get_perf_events_args { + remote_nonnull_domain dom; +}; + +struct remote_domain_get_perf_events_ret { + remote_typed_param params<REMOTE_DOMAIN_PERF_EVENTS_MAX>; +}; + struct remote_domain_block_stats_args { remote_nonnull_domain dom; remote_nonnull_string path; @@ -5751,5 +5767,17 @@ enum remote_procedure { * @generate: both * @acl: domain:migrate */ - REMOTE_PROC_DOMAIN_MIGRATE_START_POST_COPY = 364 + REMOTE_PROC_DOMAIN_MIGRATE_START_POST_COPY = 364, + + /** + * @generate: none + * @acl: domain:read + */ + REMOTE_PROC_DOMAIN_GET_PERF_EVENTS = 365, + + /** + * @generate: both + * @acl: domain:write + */ + REMOTE_PROC_DOMAIN_SET_PERF_EVENTS = 366 }; diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index 9d59e0e..1257aac 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -304,6 +304,22 @@ struct remote_domain_get_numa_parameters_ret { } params; int nparams; }; +struct remote_domain_set_perf_events_args { + remote_nonnull_domain dom; + struct { + u_int params_len; + remote_typed_param * params_val; + } params; +}; +struct remote_domain_get_perf_events_args { + remote_nonnull_domain dom; +}; +struct remote_domain_get_perf_events_ret { + struct { + u_int params_len; + remote_typed_param * params_val; + } params; +}; struct remote_domain_block_stats_args { remote_nonnull_domain dom; remote_nonnull_string path; @@ -3077,4 +3093,6 @@ enum remote_procedure { REMOTE_PROC_CONNECT_EVENT_CONNECTION_CLOSED = 362, REMOTE_PROC_DOMAIN_EVENT_CALLBACK_JOB_COMPLETED = 363, REMOTE_PROC_DOMAIN_MIGRATE_START_POST_COPY = 364, + REMOTE_PROC_DOMAIN_GET_PERF_EVENTS = 365, + REMOTE_PROC_DOMAIN_SET_PERF_EVENTS = 366, }; -- 1.9.1
This patch implement a set of interfaces for perf event. Based on these interfaces, we can implement internal driver API for perf, and get the results of perf conuter you care about. Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- include/libvirt/virterror.h | 2 + src/Makefile.am | 1 + src/libvirt_private.syms | 12 ++ src/util/virerror.c | 2 + src/util/virperf.c | 307 ++++++++++++++++++++++++++++++++++++++++++++ src/util/virperf.h | 63 +++++++++ 6 files changed, 387 insertions(+) create mode 100644 src/util/virperf.c create mode 100644 src/util/virperf.h diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h index 1f0702b..c6abdf7 100644 --- a/include/libvirt/virterror.h +++ b/include/libvirt/virterror.h @@ -130,6 +130,8 @@ typedef enum { VIR_FROM_LOGGING = 63, /* Error from log manager */ VIR_FROM_XENXL = 64, /* Error from Xen xl config code */ + VIR_FROM_PERF = 65, /* Error from perf */ + # ifdef VIR_ENUM_SENTINELS VIR_ERR_DOMAIN_LAST # endif diff --git a/src/Makefile.am b/src/Makefile.am index dad7bab..1726d06 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -98,6 +98,7 @@ UTIL_SOURCES = \ util/virauthconfig.c util/virauthconfig.h \ util/virbitmap.c util/virbitmap.h \ util/virbuffer.c util/virbuffer.h \ + util/virperf.c util/virperf.h \ util/vircgroup.c util/vircgroup.h util/vircgrouppriv.h \ util/virclosecallbacks.c util/virclosecallbacks.h \ util/vircommand.c util/vircommand.h util/vircommandpriv.h \ diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index af133c5..7c44047 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2023,6 +2023,18 @@ virPCIStubDriverTypeFromString; virPCIStubDriverTypeToString; +# util/virperf.h +virPerfEventDisable; +virPerfEventEnable; +virPerfEventIsEnabled; +virPerfEventTypeFromString; +virPerfEventTypeToString; +virPerfFree; +virPerfGetEventFd; +virPerfNew; +virPerfReadEvent; + + # util/virpidfile.h virPidFileAcquire; virPidFileAcquirePath; diff --git a/src/util/virerror.c b/src/util/virerror.c index 61b9ea0..4766e13 100644 --- a/src/util/virerror.c +++ b/src/util/virerror.c @@ -136,6 +136,8 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST, "Admin Interface", "Log Manager", "Xen XL Config", + + "Perf", ) diff --git a/src/util/virperf.c b/src/util/virperf.c new file mode 100644 index 0000000..42eee85 --- /dev/null +++ b/src/util/virperf.c @@ -0,0 +1,307 @@ +/* + * virperf.c: methods for managing perf events + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Authors: + * Ren Qiaowei <qiaowei.ren@intel.com> + */ +#include <config.h> + +#include <sys/ioctl.h> +#if defined HAVE_SYS_SYSCALL_H +# include <sys/syscall.h> +#endif + +#include "virperf.h" +#include "viralloc.h" +#include "virerror.h" +#include "virlog.h" +#include "virfile.h" +#include "virstring.h" +#include "virtypedparam.h" + +VIR_LOG_INIT("util.perf"); + +#define VIR_FROM_THIS VIR_FROM_PERF + +VIR_ENUM_IMPL(virPerfEvent, VIR_PERF_EVENT_LAST, + "cmt"); + +struct virPerfEvent { + int type; + int fd; + bool enabled; + union { + /* cmt */ + struct { + int scale; + } cmt; + } efields; +}; +typedef struct virPerfEvent *virPerfEventPtr; + +struct virPerf { + struct virPerfEvent events[VIR_PERF_EVENT_LAST]; +}; + +virPerfPtr +virPerfNew(void) +{ + size_t i; + virPerfPtr perf; + + if (VIR_ALLOC(perf) < 0) + return NULL; + + for (i = 0; i < VIR_PERF_EVENT_LAST; i++) { + perf->events[i].type = i; + perf->events[i].fd = -1; + perf->events[i].enabled = false; + } + + return perf; +} + +void +virPerfFree(virPerfPtr perf) +{ + size_t i; + + if (perf == NULL) + return; + + for (i = 0; i < VIR_PERF_EVENT_LAST; i++) { + if (perf->events[i].enabled) + virPerfEventDisable(perf, i); + } + + VIR_FREE(perf); +} + +#if defined(__linux__) && defined(HAVE_SYS_SYSCALL_H) + +# include <linux/perf_event.h> + +static virPerfEventPtr +virPerfGetEvent(virPerfPtr perf, + virPerfEventType type) +{ + if (type >= VIR_PERF_EVENT_LAST) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Event '%d' is not supported"), + type); + return NULL; + } + + return perf->events + type; +} + +static int +virPerfCmtEnable(virPerfEventPtr event, + pid_t pid) +{ + struct perf_event_attr cmt_attr; + char *buf = NULL; + char *tmp = NULL; + unsigned int event_type, scale; + + if (virFileReadAll("/sys/devices/intel_cqm/type", + 10, &buf) < 0) + goto cleanup; + + if ((tmp = strchr(buf, '\n'))) + *tmp = '\0'; + + if (virStrToLong_ui(buf, NULL, 10, &event_type) < 0) { + virReportSystemError(errno, "%s", + _("failed to get cmt event type")); + goto cleanup; + } + VIR_FREE(buf); + + if (virFileReadAll("/sys/devices/intel_cqm/events/llc_occupancy.scale", + 10, &buf) < 0) + goto cleanup; + + if (virStrToLong_ui(buf, NULL, 10, &scale) < 0) { + virReportSystemError(errno, "%s", + _("failed to get cmt scaling factor")); + goto cleanup; + } + + event->efields.cmt.scale = scale; + + memset(&cmt_attr, 0, sizeof(cmt_attr)); + cmt_attr.size = sizeof(cmt_attr); + cmt_attr.type = event_type; + cmt_attr.config = 1; + cmt_attr.inherit = 1; + cmt_attr.disabled = 1; + cmt_attr.enable_on_exec = 0; + + event->fd = syscall(__NR_perf_event_open, &cmt_attr, pid, -1, -1, 0); + if (event->fd < 0) { + virReportSystemError(errno, + _("Unable to open perf type=%d for pid=%d"), + event_type, pid); + goto cleanup; + } + + if (ioctl(event->fd, PERF_EVENT_IOC_ENABLE) < 0) { + virReportSystemError(errno, "%s", + _("Unable to enable perf event for CMT")); + goto cleanup; + } + + event->enabled = true; + return 0; + + cleanup: + VIR_FORCE_CLOSE(event->fd); + VIR_FREE(buf); + return -1; +} + +int +virPerfEventEnable(virPerfPtr perf, + virPerfEventType type, + pid_t pid) +{ + virPerfEventPtr event = virPerfGetEvent(perf, type); + if (event == NULL) + return -1; + + switch (type) { + case VIR_PERF_EVENT_CMT: + if (virPerfCmtEnable(event, pid)) + return -1; + break; + case VIR_PERF_EVENT_LAST: + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unexpected perf event type=%d"), type); + return -1; + } + + return 0; +} + +int +virPerfEventDisable(virPerfPtr perf, + virPerfEventType type) +{ + virPerfEventPtr event = virPerfGetEvent(perf, type); + if (event == NULL) + return -1; + + if (ioctl(event->fd, PERF_EVENT_IOC_DISABLE) < 0) { + virReportSystemError(errno, + _("Unable to disable perf event type=%d"), + event->type); + return -1; + } + + event->enabled = false; + VIR_FORCE_CLOSE(event->fd); + return 0; +} + +bool virPerfEventIsEnabled(virPerfPtr perf, + virPerfEventType type) +{ + virPerfEventPtr event = virPerfGetEvent(perf, type); + if (event == NULL) + return false; + + return event->enabled; +} + +int virPerfGetEventFd(virPerfPtr perf, + virPerfEventType type) +{ + virPerfEventPtr event = virPerfGetEvent(perf, type); + if (event == NULL) + return false; + + return event->fd; +} + +int +virPerfReadEvent(virPerfPtr perf, + virPerfEventType type, + uint64_t *value) +{ + virPerfEventPtr event = virPerfGetEvent(perf, type); + if (event == NULL || !event->enabled) + return -1; + + if (read(event->fd, value, sizeof(uint64_t)) < 0) { + virReportSystemError(errno, "%s", + _("Unable to read cache data")); + return -1; + } + + if (type == VIR_PERF_EVENT_CMT) + *value *= event->efields.cmt.scale; + + return 0; +} + +#else +int +virPerfEventEnable(virPerfPtr perf ATTRIBUTE_UNUSED, + pid_t pid ATTRIBUTE_UNUSED) +{ + virReportSystemError(ENXIO, "%s", + _("Perf not supported on this platform")); + return -1; +} + +int +virPerfEventDisable(virPerfPtr perf ATTRIBUTE_UNUSED, + int event ATTRIBUTE_UNUSED) +{ + virReportSystemError(ENXIO, "%s", + _("Perf not supported on this platform")); + return -1; +} + +bool +virPerfEventIsEnabled(virPerfPtr perf, + virPerfEventType type) +{ + return false; +} + +int +virPerfGetEventFd(virPerfPtr perf, + virPerfEventType type) +{ + virReportSystemError(ENXIO, "%s", + _("Perf not supported on this platform")); + return -1; +} + +int +virPerfReadEvent(virPerfPtr perf, + virPerfEventType type + uint64_t *value) +{ + virReportSystemError(ENXIO, "%s", + _("Perf not supported on this platform")); + return -1; +} + +#endif diff --git a/src/util/virperf.h b/src/util/virperf.h new file mode 100644 index 0000000..4c36b78 --- /dev/null +++ b/src/util/virperf.h @@ -0,0 +1,63 @@ +/* + * virperf.h: methods for managing perf events + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + * + * Authors: + * Ren Qiaowei <qiaowei.ren@intel.com> + */ + +#ifndef __VIR_PERF_H__ +# define __VIR_PERF_H__ + +# include "virutil.h" + +typedef enum { + VIR_PERF_EVENT_CMT, + + VIR_PERF_EVENT_LAST +} virPerfEventType; + +VIR_ENUM_DECL(virPerfEvent); + +# define VIR_PERF_PARAMETERS \ + VIR_PERF_PARAM_CMT, VIR_TYPED_PARAM_BOOLEAN, \ + NULL + +struct virPerf; +typedef struct virPerf *virPerfPtr; + +virPerfPtr virPerfNew(void); + +void virPerfFree(virPerfPtr perf); + +int virPerfEventEnable(virPerfPtr perf, + virPerfEventType type, + pid_t pid); + +int virPerfEventDisable(virPerfPtr perf, + virPerfEventType type); + +bool virPerfEventIsEnabled(virPerfPtr perf, + virPerfEventType type); + +int virPerfGetEventFd(virPerfPtr perf, + virPerfEventType type); + +int virPerfReadEvent(virPerfPtr perf, + virPerfEventType type, + uint64_t *value); + +#endif /* __VIR_PERF_H__ */ -- 1.9.1
On Mon, Mar 28, 2016 at 09:30:28PM +0800, Qiaowei Ren wrote:
This patch implement a set of interfaces for perf event. Based on these interfaces, we can implement internal driver API for perf, and get the results of perf conuter you care about.
Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- include/libvirt/virterror.h | 2 + src/Makefile.am | 1 + src/libvirt_private.syms | 12 ++ src/util/virerror.c | 2 + src/util/virperf.c | 307 ++++++++++++++++++++++++++++++++++++++++++++ src/util/virperf.h | 63 +++++++++
src/util/virperf.c needs to be added to po/POTFILES.in to get 'make syntax-check' to pass Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
This patch implement the internal driver API for perf event into qemu driver. Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- include/libvirt/libvirt-domain.h | 1 + src/qemu/qemu_domain.h | 3 + src/qemu/qemu_driver.c | 133 +++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_process.c | 6 ++ 4 files changed, 143 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index b171cdf..552a40b 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1804,6 +1804,7 @@ typedef enum { VIR_DOMAIN_STATS_VCPU = (1 << 3), /* return domain virtual CPU info */ VIR_DOMAIN_STATS_INTERFACE = (1 << 4), /* return domain interfaces info */ VIR_DOMAIN_STATS_BLOCK = (1 << 5), /* return domain block info */ + VIR_DOMAIN_STATS_PERF = (1 << 6), /* return domain perf event info */ } virDomainStatsTypes; typedef enum { diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 573968c..4d581d7 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -26,6 +26,7 @@ # include "virthread.h" # include "vircgroup.h" +# include "virperf.h" # include "domain_addr.h" # include "domain_conf.h" # include "snapshot_conf.h" @@ -195,6 +196,8 @@ struct _qemuDomainObjPrivate { virCgroupPtr cgroup; + virPerfPtr perf; + virCond unplugFinished; /* signals that unpluggingDevice was unplugged */ const char *unpluggingDevice; /* alias of the device that is being unplugged */ char **qemuDevices; /* NULL-terminated list of devices aliases known to QEMU */ diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index da039c4..599d5ed 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -99,6 +99,7 @@ #include "virhostdev.h" #include "domain_capabilities.h" #include "vircgroup.h" +#include "virperf.h" #include "virnuma.h" #include "dirname.h" #include "network/bridge_driver.h" @@ -10037,6 +10038,86 @@ qemuSetGlobalBWLive(virCgroupPtr cgroup, unsigned long long period, } static int +qemuDomainSetPerfEvents(virDomainPtr dom, + virTypedParameterPtr params, + int nparams) +{ + size_t i; + virDomainObjPtr vm = NULL; + qemuDomainObjPrivatePtr priv; + int ret = -1; + virPerfEventType type; + bool enabled; + + if (virTypedParamsValidate(params, nparams, VIR_PERF_PARAMETERS) < 0) + return -1; + + if (!(vm = qemuDomObjFromDomain(dom))) + return -1; + + priv = vm->privateData; + + if (virDomainSetPerfEventsEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + for (i = 0; i < nparams; i++) { + virTypedParameterPtr param = ¶ms[i]; + enabled = params->value.b; + type = virPerfEventTypeFromString(param->field); + + if (!enabled && virPerfEventDisable(priv->perf, type)) + goto cleanup; + if (enabled && virPerfEventEnable(priv->perf, type, vm->pid)) + goto cleanup; + } + + ret = 0; + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + +static int +qemuDomainGetPerfEvents(virDomainPtr dom, + virTypedParameterPtr *params, + int *nparams) +{ + size_t i; + virDomainObjPtr vm = NULL; + qemuDomainObjPrivatePtr priv; + int ret = -1; + virTypedParameterPtr par = NULL; + int maxpar = 0; + int npar = 0; + + if (!(vm = qemuDomObjFromDomain(dom))) + goto cleanup; + + priv = vm->privateData; + + if (virDomainGetPerfEventsEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + for (i = 0; i < VIR_PERF_EVENT_LAST; i++) { + if (virTypedParamsAddBoolean(&par, &npar, &maxpar, + virPerfEventTypeToString(i), + virPerfEventIsEnabled(priv->perf, i)) < 0) { + virTypedParamsFree(par, npar); + goto cleanup; + } + } + + *params = par; + *nparams = npar; + ret = 0; + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + +static int qemuSetVcpusBWLive(virDomainObjPtr vm, virCgroupPtr cgroup, unsigned long long period, long long quota) { @@ -19401,6 +19482,55 @@ qemuDomainGetStatsBlock(virQEMUDriverPtr driver, #undef QEMU_ADD_COUNT_PARAM +static int +qemuDomainGetStatsPerfCmt(virPerfPtr perf, + virDomainStatsRecordPtr record, + int *maxparams) +{ + uint64_t cache = 0; + + if (virPerfReadEvent(perf, VIR_PERF_EVENT_CMT, &cache) < 0) + return -1; + + if (virTypedParamsAddULLong(&record->params, + &record->nparams, + maxparams, + "perf.cache", + cache) < 0) + return -1; + + return 0; +} + +static int +qemuDomainGetStatsPerf(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, + virDomainObjPtr dom, + virDomainStatsRecordPtr record, + int *maxparams, + unsigned int privflags ATTRIBUTE_UNUSED) +{ + size_t i; + qemuDomainObjPrivatePtr priv = dom->privateData; + int ret = -1; + + for (i = 0; i < VIR_PERF_EVENT_LAST; i++) { + if (!virPerfEventIsEnabled(priv->perf, i)) + continue; + + switch (i) { + case VIR_PERF_EVENT_CMT: + if (qemuDomainGetStatsPerfCmt(priv->perf, record, maxparams) < 0) + goto cleanup; + break; + } + } + + ret = 0; + + cleanup: + return ret; +} + typedef int (*qemuDomainGetStatsFunc)(virQEMUDriverPtr driver, virDomainObjPtr dom, @@ -19421,6 +19551,7 @@ static struct qemuDomainGetStatsWorker qemuDomainGetStatsWorkers[] = { { qemuDomainGetStatsVcpu, VIR_DOMAIN_STATS_VCPU, false }, { qemuDomainGetStatsInterface, VIR_DOMAIN_STATS_INTERFACE, false }, { qemuDomainGetStatsBlock, VIR_DOMAIN_STATS_BLOCK, true }, + { qemuDomainGetStatsPerf, VIR_DOMAIN_STATS_PERF, false }, { NULL, 0, false } }; @@ -20177,6 +20308,8 @@ static virHypervisorDriver qemuHypervisorDriver = { .domainMigrateFinish3 = qemuDomainMigrateFinish3, /* 0.9.2 */ .domainMigrateConfirm3 = qemuDomainMigrateConfirm3, /* 0.9.2 */ .domainSendKey = qemuDomainSendKey, /* 0.9.4 */ + .domainGetPerfEvents = qemuDomainGetPerfEvents, /* 1.3.3 */ + .domainSetPerfEvents = qemuDomainSetPerfEvents, /* 1.3.3 */ .domainBlockJobAbort = qemuDomainBlockJobAbort, /* 0.9.4 */ .domainGetBlockJobInfo = qemuDomainGetBlockJobInfo, /* 0.9.4 */ .domainBlockJobSetSpeed = qemuDomainBlockJobSetSpeed, /* 0.9.4 */ diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 6ec7b76..f7d30e9 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -5253,6 +5253,10 @@ qemuProcessLaunch(virConnectPtr conn, if (qemuSetupCgroup(driver, vm, nnicindexes, nicindexes) < 0) goto cleanup; + priv->perf = virPerfNew(); + if (!priv->perf) + goto cleanup; + /* This must be done after cgroup placement to avoid resetting CPU * affinity */ if (!vm->def->cputune.emulatorpin && @@ -5890,6 +5894,8 @@ void qemuProcessStop(virQEMUDriverPtr driver, } virCgroupFree(&priv->cgroup); + virPerfFree(priv->perf); + qemuProcessRemoveDomainStatus(driver, vm); /* Remove VNC and Spice ports from port reservation bitmap, but only if -- 1.9.1
This patch adds new xml element, and so we can have the option of also having perf events enabled immediately at startup. Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- docs/schemas/domaincommon.rng | 27 +++++++ src/conf/domain_conf.c | 111 ++++++++++++++++++++++++++ src/conf/domain_conf.h | 10 +++ src/qemu/qemu_driver.c | 26 ++++++ src/qemu/qemu_process.c | 8 +- tests/domainschemadata/domain-perf-simple.xml | 20 +++++ 6 files changed, 200 insertions(+), 2 deletions(-) create mode 100644 tests/domainschemadata/domain-perf-simple.xml diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index da6de40..03b4ed2 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -56,6 +56,9 @@ <ref name="pm"/> </optional> <optional> + <ref name="perf"/> + </optional> + <optional> <ref name="idmap"/> </optional> <optional> @@ -393,6 +396,30 @@ </define> <!-- + Enable or disable perf events for the domain. For each + of the events the following rules apply: + on: the event will be forcefully enabled + off: the event will be forcefully disabled + not specified: the event will be disabled by default + --> + <define name="perf"> + <element name="perf"> + <oneOrMore> + <element name="event"> + <attribute name="name"> + <choice> + <value>cmt</value> + </choice> + </attribute> + <attribute name="enabled"> + <ref name="virYesNo"/> + </attribute> + </element> + </oneOrMore> + </element> + </define> + + <!-- The Identifiers can be: - an optional id attribute with a number on the domain element - a mandatory name diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index d5d9ff7..c004ce9 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -2635,6 +2635,8 @@ void virDomainDefFree(virDomainDefPtr def) VIR_FREE(def->keywrap); + VIR_FREE(def->perf); + if (def->namespaceData && def->ns.free) (def->ns.free)(def->namespaceData); @@ -12561,6 +12563,91 @@ virDomainPMStateParseXML(xmlXPathContextPtr ctxt, static int +virDomainPerfEventDefParseXML(virDomainPerfDefPtr perf, + xmlNodePtr node, + xmlXPathContextPtr ctxt) +{ + char *name = NULL; + char *enabled = NULL; + int enabled_type; + int name_type; + int ret = -1; + + xmlNodePtr oldnode = ctxt->node; + + ctxt->node = node; + if (!(name = virXPathString("string(./@name)", ctxt))) { + virReportError(VIR_ERR_CONF_SYNTAX, "%s", + _("missing name for event")); + goto cleanup; + } + + if ((name_type = virPerfEventTypeFromString(name)) < 0) { + virReportError(VIR_ERR_CONF_SYNTAX, + _("%s is not a supported event name"), name); + goto cleanup; + } + + if (!(enabled = virXPathString("string(./@enabled)", ctxt))) { + virReportError(VIR_ERR_CONF_SYNTAX, + _("missing state for cipher named %s"), name); + goto cleanup; + } + + if ((enabled_type = virTristateBoolTypeFromString(enabled)) < 0) { + virReportError(VIR_ERR_CONF_SYNTAX, + _("%s is not a supported enabled state"), enabled); + goto cleanup; + } + + if (perf->events[VIR_PERF_EVENT_CMT] != VIR_TRISTATE_BOOL_ABSENT) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("A domain definition can have no more than " + "one event node with name %s"), + virTristateBoolTypeToString(name_type)); + + goto cleanup; + } + perf->events[VIR_PERF_EVENT_CMT] = enabled_type; + + ret = 0; + cleanup: + VIR_FREE(name); + VIR_FREE(enabled); + ctxt->node = oldnode; + return ret; +} + +static int +virDomainPerfDefParseXML(virDomainDefPtr def, + xmlXPathContextPtr ctxt) +{ + size_t i; + int ret = -1; + xmlNodePtr *nodes = NULL; + int n; + + if ((n = virXPathNodeSet("./perf/event", ctxt, &nodes)) < 0) + return n; + + if (VIR_ALLOC(def->perf) < 0) + goto cleanup; + + for (i = 0; i < n; i++) { + if (virDomainPerfEventDefParseXML(def->perf, nodes[i], ctxt) < 0) + goto cleanup; + } + + ret = 0; + + cleanup: + if (ret < 0) + VIR_FREE(def->perf); + VIR_FREE(nodes); + return ret; +} + +static int virDomainMemorySourceDefParseXML(xmlNodePtr node, xmlXPathContextPtr ctxt, virDomainMemoryDefPtr def) @@ -15769,6 +15856,9 @@ virDomainDefParseXML(xmlDocPtr xml, &def->pm.s4) < 0) goto error; + if (virDomainPerfDefParseXML(def, ctxt) < 0) + goto error; + if ((tmp = virXPathString("string(./clock/@offset)", ctxt)) && (def->clock.offset = virDomainClockOffsetTypeFromString(tmp)) < 0) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, @@ -21653,6 +21743,24 @@ virDomainKeyWrapDefFormat(virBufferPtr buf, virDomainKeyWrapDefPtr keywrap) virBufferAddLit(buf, "</keywrap>\n"); } +static void +virDomainPerfDefFormat(virBufferPtr buf, virDomainPerfDefPtr perf) +{ + size_t i; + virBufferAddLit(buf, "<perf>\n"); + virBufferAdjustIndent(buf, 2); + + for (i = 0; i < VIR_PERF_EVENT_LAST; i++) { + if (perf->events[i]) + virBufferAsprintf(buf, "<event name='%s' enabled='%s'/>\n", + virPerfEventTypeToString(i), + virTristateBoolTypeToString(perf->events[i])); + } + + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "</perf>\n"); +} + static bool virDomainDefHasCapabilitiesFeatures(virDomainDefPtr def) { @@ -22515,6 +22623,9 @@ virDomainDefFormatInternal(virDomainDefPtr def, virBufferAddLit(buf, "</pm>\n"); } + if (def->perf) + virDomainPerfDefFormat(buf, def->perf); + virBufferAddLit(buf, "<devices>\n"); virBufferAdjustIndent(buf, 2); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 83bdd67..fd104fb 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -51,6 +51,7 @@ # include "virseclabel.h" # include "virprocess.h" # include "virgic.h" +# include "virperf.h" /* forward declarations of all device types, required by * virDomainDeviceDef @@ -2179,6 +2180,13 @@ struct _virDomainPowerManagement { int s4; }; +typedef struct _virDomainPerfDef virDomainPerfDef; +typedef virDomainPerfDef *virDomainPerfDefPtr; +struct _virDomainPerfDef { + /* These options are of type enum virTristateBool */ + int events[VIR_PERF_EVENT_LAST]; +}; + typedef struct _virDomainKeyWrapDef virDomainKeyWrapDef; typedef virDomainKeyWrapDef *virDomainKeyWrapDefPtr; struct _virDomainKeyWrapDef { @@ -2229,6 +2237,8 @@ struct _virDomainDef { virDomainPowerManagement pm; + virDomainPerfDefPtr perf; + virDomainOSDef os; char *emulator; /* These three options are of type virTristateSwitch, diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 599d5ed..7e01459 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -10042,9 +10042,14 @@ qemuDomainSetPerfEvents(virDomainPtr dom, virTypedParameterPtr params, int nparams) { + virQEMUDriverPtr driver = dom->conn->privateData; size_t i; virDomainObjPtr vm = NULL; + virQEMUDriverConfigPtr cfg = NULL; qemuDomainObjPrivatePtr priv; + virDomainDefPtr def; + virDomainDefPtr persistentDef; + unsigned int flags = VIR_DOMAIN_AFFECT_CURRENT; int ret = -1; virPerfEventType type; bool enabled; @@ -10055,11 +10060,15 @@ qemuDomainSetPerfEvents(virDomainPtr dom, if (!(vm = qemuDomObjFromDomain(dom))) return -1; + cfg = virQEMUDriverGetConfig(driver); priv = vm->privateData; if (virDomainSetPerfEventsEnsureACL(dom->conn, vm->def) < 0) goto cleanup; + if (virDomainObjGetDefs(vm, flags, &def, &persistentDef) < 0) + goto cleanup; + for (i = 0; i < nparams; i++) { virTypedParameterPtr param = ¶ms[i]; enabled = params->value.b; @@ -10069,12 +10078,29 @@ qemuDomainSetPerfEvents(virDomainPtr dom, goto cleanup; if (enabled && virPerfEventEnable(priv->perf, type, vm->pid)) goto cleanup; + + if (def) { + def->perf->events[type] = enabled ? + VIR_TRISTATE_BOOL_YES : VIR_TRISTATE_BOOL_NO; + + if (virDomainSaveStatus(driver->xmlopt, cfg->stateDir, vm, driver->caps) < 0) + goto cleanup; + } + + if (persistentDef) { + persistentDef->perf->events[type] = enabled ? + VIR_TRISTATE_BOOL_YES : VIR_TRISTATE_BOOL_NO; + + if (virDomainSaveConfig(cfg->configDir, driver->caps, persistentDef) < 0) + goto cleanup; + } } ret = 0; cleanup: virDomainObjEndAPI(&vm); + virObjectUnref(cfg); return ret; } diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index f7d30e9..cf7df1a 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -5254,8 +5254,12 @@ qemuProcessLaunch(virConnectPtr conn, goto cleanup; priv->perf = virPerfNew(); - if (!priv->perf) - goto cleanup; + if (priv->perf) { + for (size_t i = 0; i < VIR_PERF_EVENT_LAST; i++) { + if (vm->def->perf->events[i] == VIR_TRISTATE_BOOL_YES) + virPerfEventEnable(priv->perf, i, vm->pid); + } + } /* This must be done after cgroup placement to avoid resetting CPU * affinity */ diff --git a/tests/domainschemadata/domain-perf-simple.xml b/tests/domainschemadata/domain-perf-simple.xml new file mode 100644 index 0000000..d7be7c9 --- /dev/null +++ b/tests/domainschemadata/domain-perf-simple.xml @@ -0,0 +1,20 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <perf> + <event name='cmt' enabled='yes'/> + </perf> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + </devices> +</domain> -- 1.9.1
On Mon, Mar 28, 2016 at 09:30:30PM +0800, Qiaowei Ren wrote:
This patch adds new xml element, and so we can have the option of also having perf events enabled immediately at startup.
Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- docs/schemas/domaincommon.rng | 27 +++++++ src/conf/domain_conf.c | 111 ++++++++++++++++++++++++++ src/conf/domain_conf.h | 10 +++ src/qemu/qemu_driver.c | 26 ++++++ src/qemu/qemu_process.c | 8 +- tests/domainschemadata/domain-perf-simple.xml | 20 +++++ 6 files changed, 200 insertions(+), 2 deletions(-) create mode 100644 tests/domainschemadata/domain-perf-simple.xml
+static void +virDomainPerfDefFormat(virBufferPtr buf, virDomainPerfDefPtr perf) +{ + size_t i; + virBufferAddLit(buf, "<perf>\n");
We must skip ading of <perf></perf> when no perf events are enabled otherwise we break many existing test cases. eg add in bool wantPerf = false; for (i = 0; i < VIR_PERF_EVENT_LAST; i++) { if (perf->events[i]) wantPerf = true; } if (!wantPerf) return; before this line.
+ virBufferAdjustIndent(buf, 2); + + for (i = 0; i < VIR_PERF_EVENT_LAST; i++) { + if (perf->events[i]) + virBufferAsprintf(buf, "<event name='%s' enabled='%s'/>\n", + virPerfEventTypeToString(i), + virTristateBoolTypeToString(perf->events[i])); + } + + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "</perf>\n"); +}
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Mon, Mar 28, 2016 at 09:30:30PM +0800, Qiaowei Ren wrote:
This patch adds new xml element, and so we can have the option of also having perf events enabled immediately at startup.
Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- docs/schemas/domaincommon.rng | 27 +++++++ src/conf/domain_conf.c | 111 ++++++++++++++++++++++++++ src/conf/domain_conf.h | 10 +++ src/qemu/qemu_driver.c | 26 ++++++ src/qemu/qemu_process.c | 8 +- tests/domainschemadata/domain-perf-simple.xml | 20 +++++ 6 files changed, 200 insertions(+), 2 deletions(-)
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index f7d30e9..cf7df1a 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -5254,8 +5254,12 @@ qemuProcessLaunch(virConnectPtr conn, goto cleanup;
priv->perf = virPerfNew(); - if (!priv->perf) - goto cleanup; + if (priv->perf) { + for (size_t i = 0; i < VIR_PERF_EVENT_LAST; i++) {
We don't declare variables in the for loop
+ if (vm->def->perf->events[i] == VIR_TRISTATE_BOOL_YES) + virPerfEventEnable(priv->perf, i, vm->pid); + } + }
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
When libvirtd daemon restart, this patch will reenable those perf events previously enabled. Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- src/qemu/qemu_process.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index cf7df1a..2f4e22f 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -3419,6 +3419,34 @@ qemuProcessUpdateDevices(virQEMUDriverPtr driver, return ret; } +static int +qemuDomainPerfRestart(virDomainObjPtr vm) +{ + size_t i; + virDomainDefPtr def = vm->def; + qemuDomainObjPrivatePtr priv = vm->privateData; + + virPerfFree(priv->perf); + + priv->perf = virPerfNew(); + if (!priv->perf) + return -1; + + for (i = 0; i < VIR_PERF_EVENT_LAST; i++) { + if (def->perf->events[i] && + def->perf->events[i] == VIR_TRISTATE_BOOL_YES) { + if (virPerfEventEnable(priv->perf, i, vm->pid)) + goto cleanup; + } + } + + return 0; + + cleanup: + virPerfFree(priv->perf); + return -1; +} + struct qemuProcessReconnectData { virConnectPtr conn; virQEMUDriverPtr driver; @@ -3495,6 +3523,9 @@ qemuProcessReconnect(void *opaque) if (qemuConnectCgroup(driver, obj) < 0) goto error; + if (qemuDomainPerfRestart(obj) < 0) + goto error; + /* XXX: Need to change as long as lock is introduced for * qemu_driver->sharedDevices. */ @@ -6089,6 +6120,9 @@ int qemuProcessAttach(virConnectPtr conn ATTRIBUTE_UNUSED, if (virSecurityManagerGenLabel(driver->securityManager, vm->def) < 0) goto error; + if (qemuDomainPerfRestart(vm) < 0) + goto error; + VIR_DEBUG("Creating domain log file"); if (!(logCtxt = qemuDomainLogContextNew(driver, vm, QEMU_DOMAIN_LOG_CONTEXT_MODE_ATTACH))) -- 1.9.1
This patch add new perf command to enable/disable perf event for a guest domain. Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- tools/virsh-domain.c | 128 +++++++++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 20 ++++++++ 2 files changed, 148 insertions(+) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 17be5b4..0d020a7 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -8506,6 +8506,128 @@ cmdMemtune(vshControl *ctl, const vshCmd *cmd) } /* + * "perf" command + */ +static const vshCmdInfo info_perf[] = { + {.name = "help", + .data = N_("Get or set perf event") + }, + {.name = "desc", + .data = N_("Get or set the current perf events for a guest" + " domain.\n" + " To get the perf events list use following command: \n\n" + " virsh # perf <domain>") + }, + {.name = NULL} +}; + +static const vshCmdOptDef opts_perf[] = { + {.name = "domain", + .type = VSH_OT_DATA, + .flags = VSH_OFLAG_REQ, + .help = N_("domain name, id or uuid") + }, + {.name = "enable", + .type = VSH_OT_STRING, + .help = N_("perf events which will be enabled") + }, + {.name = "disable", + .type = VSH_OT_STRING, + .help = N_("perf events which will be disabled") + }, + {.name = NULL} +}; + +static int +virshParseEventStr(vshControl *ctl, + const char *event, + bool state, + virTypedParameterPtr *params, + int *nparams, + int *maxparams) +{ + char **tok = NULL; + size_t i, ntok; + int ret = -1; + + if (!(tok = virStringSplitCount(event, "|", 0, &ntok))) + return -1; + + if (ntok > VIR_PERF_EVENT_LAST) { + vshError(ctl, _("event string '%s' has too many fields"), event); + goto cleanup; + } + + for(i = 0; i < ntok; i++) { + if ((*tok[i] != '\0') && + virTypedParamsAddBoolean(params, nparams, + maxparams, tok[i], state) < 0) + goto cleanup; + } + + ret = 0; + cleanup: + virStringFreeList(tok); + return ret; +} + +static bool +cmdPerf(vshControl *ctl, const vshCmd *cmd) +{ + virDomainPtr dom; + int nparams = 0; + int maxparams = 0; + size_t i; + virTypedParameterPtr params = NULL; + bool ret = false; + const char *enable = NULL, *disable = NULL; + + if (!(dom = virshCommandOptDomain(ctl, cmd, NULL))) + return false; + + if (vshCommandOptStringReq(ctl, cmd, "enable", &enable) < 0 || + vshCommandOptStringReq(ctl, cmd, "disable", &disable) < 0) + return false; + + if (enable && virshParseEventStr(ctl, enable, true, + ¶ms, &nparams, &maxparams) < 0) + goto cleanup; + + if (disable && virshParseEventStr(ctl, disable, false, + ¶ms, &nparams, &maxparams) < 0) + goto cleanup; + + if (nparams == 0) { + if (virDomainGetPerfEvents(dom, ¶ms, &nparams) != 0) { + vshError(ctl, "%s", _("Unable to get perf events")); + goto cleanup; + } + for (i = 0; i < nparams; i++) { + if (params[i].type == VIR_TYPED_PARAM_BOOLEAN && + params[i].value.b) { + vshPrint(ctl, "%-15s: %s\n", params[i].field, _("enabled")); + } else { + vshPrint(ctl, "%-15s: %s\n", params[i].field, _("disabled")); + } + } + } else { + if (virDomainSetPerfEvents(dom, params, nparams) != 0) + goto error; + } + + ret = true; + cleanup: + virTypedParamsFree(params, nparams); + virDomainFree(dom); + return ret; + + error: + vshError(ctl, "%s", _("Unable to enable/disable perf events")); + goto cleanup; +} + + +/* * "numatune" command */ static const vshCmdInfo info_numatune[] = { @@ -13070,6 +13192,12 @@ const vshCmdDef domManagementCmds[] = { .info = info_memtune, .flags = 0 }, + {.name = "perf", + .handler = cmdPerf, + .opts = opts_perf, + .info = info_perf, + .flags = 0 + }, {.name = "metadata", .handler = cmdMetadata, .opts = opts_metadata, diff --git a/tools/virsh.pod b/tools/virsh.pod index 2c13e04..d0bd6a0 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -2136,6 +2136,26 @@ The guaranteed minimum memory allocation for the guest. Specifying -1 as a value for these limits is interpreted as unlimited. +=item B<perf> I<domain> [I<--enable> B<eventName>] +[I<--disable> B<eventName>] + +Get the current perf events setting or enable/disable specific perf +event for a guest domain. + +Perf is a performance analyzing tool in Linux, and it can instrument +CPU performance counters, tracepoints, kprobes, and uprobes (dynamic +tracing). Perf supports a list of measurable events, and can measure +events coming from different sources. For instance, some event are +pure kernel counters, in this case they are called software events, +including context-switches, minor-faults, etc.. Now dozens of events +from different sources can be supported by perf. + +Currently only QEMU/KVM supports I<--enable> and I<--disable>. +B<eventName> is a string listing one or more events, in the format +of name|name|name. Only "cmt" event is supported presently. CMT is +a PQos (Platform Qos) feature to monitor the usage of cache by +applications running on the platform. + =item B<blkiotune> I<domain> [I<--weight> B<weight>] [I<--device-weights> B<device-weights>] [I<--device-read-iops-sec> B<device-read-iops-sec>] -- 1.9.1
On Mon, Mar 28, 2016 at 09:30:32PM +0800, Qiaowei Ren wrote:
This patch add new perf command to enable/disable perf event for a guest domain.
Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- tools/virsh-domain.c | 128 +++++++++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 20 ++++++++ 2 files changed, 148 insertions(+)
diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 17be5b4..0d020a7 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -8506,6 +8506,128 @@ cmdMemtune(vshControl *ctl, const vshCmd *cmd) }
/* + * "perf" command + */ +static const vshCmdInfo info_perf[] = { + {.name = "help", + .data = N_("Get or set perf event") + }, + {.name = "desc", + .data = N_("Get or set the current perf events for a guest" + " domain.\n" + " To get the perf events list use following command: \n\n" + " virsh # perf <domain>") + }, + {.name = NULL} +}; + +static const vshCmdOptDef opts_perf[] = { + {.name = "domain", + .type = VSH_OT_DATA, + .flags = VSH_OFLAG_REQ, + .help = N_("domain name, id or uuid") + }, + {.name = "enable", + .type = VSH_OT_STRING, + .help = N_("perf events which will be enabled") + }, + {.name = "disable", + .type = VSH_OT_STRING, + .help = N_("perf events which will be disabled") + }, + {.name = NULL} +}; + +static int +virshParseEventStr(vshControl *ctl, + const char *event, + bool state, + virTypedParameterPtr *params, + int *nparams, + int *maxparams) +{ + char **tok = NULL; + size_t i, ntok; + int ret = -1; + + if (!(tok = virStringSplitCount(event, "|", 0, &ntok))) + return -1; + + if (ntok > VIR_PERF_EVENT_LAST) { + vshError(ctl, _("event string '%s' has too many fields"), event); + goto cleanup; + } + + for(i = 0; i < ntok; i++) {
Needs space after the 'for'
+ if ((*tok[i] != '\0') && + virTypedParamsAddBoolean(params, nparams, + maxparams, tok[i], state) < 0) + goto cleanup; + } + + ret = 0; + cleanup: + virStringFreeList(tok); + return ret; +} + +static bool +cmdPerf(vshControl *ctl, const vshCmd *cmd) +{ + virDomainPtr dom; + int nparams = 0; + int maxparams = 0; + size_t i; + virTypedParameterPtr params = NULL; + bool ret = false; + const char *enable = NULL, *disable = NULL; + + if (!(dom = virshCommandOptDomain(ctl, cmd, NULL))) + return false; + + if (vshCommandOptStringReq(ctl, cmd, "enable", &enable) < 0 || + vshCommandOptStringReq(ctl, cmd, "disable", &disable) < 0) + return false; + + if (enable && virshParseEventStr(ctl, enable, true, + ¶ms, &nparams, &maxparams) < 0) + goto cleanup; + + if (disable && virshParseEventStr(ctl, disable, false, + ¶ms, &nparams, &maxparams) < 0) + goto cleanup;
Using tab instead of space Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
This patch extend domstats command to match extended virDomainListGetStats API in previous patch. Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com> --- tools/virsh-domain-monitor.c | 7 +++++++ tools/virsh.pod | 7 +++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/tools/virsh-domain-monitor.c b/tools/virsh-domain-monitor.c index 47a9995..0a93949 100644 --- a/tools/virsh-domain-monitor.c +++ b/tools/virsh-domain-monitor.c @@ -1963,6 +1963,10 @@ static const vshCmdOptDef opts_domstats[] = { .type = VSH_OT_BOOL, .help = N_("report domain block device statistics"), }, + {.name = "perf", + .type = VSH_OT_BOOL, + .help = N_("report domain perf event statistics"), + }, {.name = "list-active", .type = VSH_OT_BOOL, .help = N_("list only active domains"), @@ -2074,6 +2078,9 @@ cmdDomstats(vshControl *ctl, const vshCmd *cmd) if (vshCommandOptBool(cmd, "block")) stats |= VIR_DOMAIN_STATS_BLOCK; + if (vshCommandOptBool(cmd, "perf")) + stats |= VIR_DOMAIN_STATS_PERF; + if (vshCommandOptBool(cmd, "list-active")) flags |= VIR_CONNECT_GET_ALL_DOMAINS_STATS_ACTIVE; diff --git a/tools/virsh.pod b/tools/virsh.pod index d0bd6a0..9595199 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -846,7 +846,7 @@ or unique source names printed by this command. =item B<domstats> [I<--raw>] [I<--enforce>] [I<--backing>] [I<--state>] [I<--cpu-total>] [I<--balloon>] [I<--vcpu>] [I<--interface>] [I<--block>] -[[I<--list-active>] [I<--list-inactive>] [I<--list-persistent>] +[I<--perf>] [[I<--list-active>] [I<--list-inactive>] [I<--list-persistent>] [I<--list-transient>] [I<--list-running>] [I<--list-paused>] [I<--list-shutoff>] [I<--list-other>]] | [I<domain> ...] @@ -864,7 +864,7 @@ behavior use the I<--raw> flag. The individual statistics groups are selectable via specific flags. By default all supported statistics groups are returned. Supported statistics groups flags are: I<--state>, I<--cpu-total>, I<--balloon>, -I<--vcpu>, I<--interface>, I<--block>. +I<--vcpu>, I<--interface>, I<--block>, I<--perf>. When selecting the I<--state> group the following fields are returned: "state.state" - state of the VM, returned as number from virDomainState enum, @@ -899,6 +899,9 @@ I<--interface> returns: "net.<num>.tx.errs" - number of transmission errors, "net.<num>.tx.drop" - number of transmit packets dropped +I<--perf> returns the statistics of all enabled perf events: +"perf.cache" - the cache usage in Byte currently used + I<--block> returns information about disks associated with each domain. Using the I<--backing> flag extends this information to cover all resources in the backing chain, rather than the default -- 1.9.1
On Mon, Mar 28, 2016 at 09:30:25PM +0800, Qiaowei Ren wrote:
The series mainly adds Intel CMT feature support into libvirt. CMT is new introduced PQos (Platform Qos) feature to monitor the usage of cache by applications running on the platform.
Currently CMT patches has been merged into Linux kernel mainline. The CMT implementation in Linux kernel is based on perf mechanism and there is no support for perf in libvirt, and so this series firstly add perf support into libvirt, including two public API and a set of util interfaces. And based on these APIs and interfaces, thie series implements CMT perf event support.
Current state:
* 1/8 - perf: add new public APIs for perf event - ACKed by Daniel
* 2/8 - perf: implement the remote protocol for perf event - ACKed by Daniel
* 3/8 - perf: implement a set of util functions for perf event - ACKed by Daniel
* 4/8 - qemu_driver: add support to perf event - restart perf in qemuProcessReconnect and qemuProcessAttach in patch 6/8
* 5/8 - perf: add new xml element - ACKed by Daniel
* 6/8 - perf: reenable perf events when libvirtd restart - add qemuDomainPerfRestart() into qemuProcessReconnect() and qemuProcessAttach()
* 7/8 - virsh: implement new command to support perf - ACKed by Daniel
* 8/8 - virsh: extend domstats command - ACKed by Daniel
ACK to all patches. I had some comments inline - in future just make sure you run 'make check && make syntax-check' against each patch before submitting. Since the fixes were easy I've made them myself and pushed to GIT master. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
-----Original Message----- From: Daniel P. Berrange [mailto:berrange@redhat.com] Sent: Tuesday, March 29, 2016 8:14 PM To: Ren, Qiaowei Cc: libvir-list@redhat.com; Jiri Denemark Subject: Re: [PATCH v4 0/8] Add perf and Intel CMT feature support
On Mon, Mar 28, 2016 at 09:30:25PM +0800, Qiaowei Ren wrote:
The series mainly adds Intel CMT feature support into libvirt. CMT is new introduced PQos (Platform Qos) feature to monitor the usage of cache by applications running on the platform.
Currently CMT patches has been merged into Linux kernel mainline. The CMT implementation in Linux kernel is based on perf mechanism and there is no support for perf in libvirt, and so this series firstly add perf support into libvirt, including two public API and a set of util interfaces. And based on these APIs and interfaces, thie series implements CMT perf event support.
Current state:
* 1/8 - perf: add new public APIs for perf event - ACKed by Daniel
* 2/8 - perf: implement the remote protocol for perf event - ACKed by Daniel
* 3/8 - perf: implement a set of util functions for perf event - ACKed by Daniel
* 4/8 - qemu_driver: add support to perf event - restart perf in qemuProcessReconnect and qemuProcessAttach in patch 6/8
* 5/8 - perf: add new xml element - ACKed by Daniel
* 6/8 - perf: reenable perf events when libvirtd restart - add qemuDomainPerfRestart() into qemuProcessReconnect() and qemuProcessAttach()
* 7/8 - virsh: implement new command to support perf - ACKed by Daniel
* 8/8 - virsh: extend domstats command - ACKed by Daniel
ACK to all patches. I had some comments inline - in future just make sure you run 'make check && make syntax-check' against each patch before submitting. Since the fixes were easy I've made them myself and pushed to GIT master.
Ok! Got it! Sorry, I just rebased it into latest code and compiled it and ignored this step. Thanks very much for your nice help. ^-^ Thanks, Qiaowei
participants (3)
-
Daniel P. Berrange -
Qiaowei Ren -
Ren, Qiaowei