[libvirt] [PATCHv6 0/6] Add virNodeGetCPUStats() API
Hi, This is v6 of virNodeGetCPUTimeParameters() API. (This time, I rename it to virNodeGetCPUStats()) It returns cpu utilization or cumulative cpu time of the node from /proc/stat since node boots up. This patch only supports linux host. Changes v5->v6 - Rename API name to virNodeGetCPUStats() - virsh nodecpustats subcommand returns raw/absolute cputime value by default, and add --percent option for printing utilization. v4->v5 - Rebase latest libvirt GIT tree. v3->v4 - Rebase this patch like virDomainGetMemoryParameters() from v2 patches. (drop v3 patches except virsh subcommand) - Rename API name to virNodeGetCPUTimeParameters() v2->v3 - Change user I/F. It is able to request what the user want by the @flags. - Minor change of virsh nodecputime I/F. v1->v2 - Change user I/F like virDomainGetMemoryStats() - It can return either cpu utilization or cumulative cpu time of the node depends on each driver. Minoru Usui (6): [v6] virNodeGetCPUStats: Expose new API [v6] virNodeGetCPUStats: Define internal driver API [v6] virNodeGetCPUStats: Implement public API [v6] virNodeGetCPUStats: Implement remote protocol [v6] virNodeGetCPUStats: Implement virsh support [v6] virNodeGetCPUStats: Implement linux support daemon/remote.c | 76 +++++++++++++++++++++++++ include/libvirt/libvirt.h.in | 65 +++++++++++++++++++++ src/driver.h | 8 +++ src/libvirt.c | 85 ++++++++++++++++++++++++++++ src/libvirt_private.syms | 1 + src/libvirt_public.syms | 1 + src/lxc/lxc_driver.c | 1 + src/nodeinfo.c | 127 ++++++++++++++++++++++++++++++++++++++++++ src/nodeinfo.h | 5 +- src/qemu/qemu_driver.c | 1 + src/remote/remote_driver.c | 65 +++++++++++++++++++++ src/remote/remote_protocol.x | 21 +++++++- src/uml/uml_driver.c | 1 + tools/virsh.c | 127 ++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 6 ++ 15 files changed, 588 insertions(+), 2 deletions(-) -- Minoru Usui <usui@mxm.nes.nec.co.jp>
virNodeGetCPUStats: Expose new API Signed-off-by: Minoru Usui <usui@mxm.nes.nec.co.jp> --- include/libvirt/libvirt.h.in | 65 ++++++++++++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 1 + 2 files changed, 66 insertions(+), 0 deletions(-) diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index 7cd6e13..431071b 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -194,6 +194,58 @@ typedef struct _virStream virStream; */ typedef virStream *virStreamPtr; +/** + * VIR_CPU_STATS_FIELD_LENGTH: + * + * Macro providing the field length of virNodeCPUStats + */ +#define VIR_CPU_STATS_FIELD_LENGTH 80 + +/** + * VIR_CPU_STATS_KERNEL: + * + * Macro for the cumulative CPU time which spends by kernel, + * when the node booting up.(in nanoseconds). + */ +#define VIR_CPU_STATS_KERNEL "kernel" + +/** + * The cumulative CPU time which spends by user processes, + * when the node booting up.(in nanoseconds). + */ +#define VIR_CPU_STATS_USER "user" + +/** + * The cumulative idle CPU time, + * when the node booting up.(in nanoseconds). + */ +#define VIR_CPU_STATS_IDLE "idle" + +/** + * The cumulative I/O wait CPU time, + * when the node booting up.(in nanoseconds). + */ +#define VIR_CPU_STATS_IOWAIT "iowait" + +/** + * The CPU utilization. + * The usage value is in percent and 100% represents all CPUs on + * the server. + */ +#define VIR_CPU_STATS_UTILIZATION "utilization" + +/** + * virCPUStats: + * + * a virNodeCPUStats is a structure filled by virNodeGetCPUStats() + * and providing the information for the cpu stats of the node. + */ +typedef struct _virCPUStats virCPUStats; + +struct _virCPUStats { + char field[VIR_CPU_STATS_FIELD_LENGTH]; + unsigned long long value; +}; /** * VIR_SECURITY_LABEL_BUFLEN: @@ -565,6 +617,14 @@ int virDomainMigrateSetMaxSpeed(virDomainPtr domain, typedef virNodeInfo *virNodeInfoPtr; /** + * virCPUStatsPtr: + * + * a virCPUStatsPtr is a pointer to a virCPUStats structure. + */ + +typedef virCPUStats *virCPUStatsPtr; + +/** * virConnectFlags * * Flags when opening a connection to a hypervisor @@ -698,6 +758,11 @@ int virNodeGetInfo (virConnectPtr conn, virNodeInfoPtr info); char * virConnectGetCapabilities (virConnectPtr conn); +int virNodeGetCPUStats (virConnectPtr conn, + virCPUStatsPtr params, + int *nparams, + unsigned int flags); + unsigned long long virNodeGetFreeMemory (virConnectPtr conn); int virNodeGetSecurityModel (virConnectPtr conn, diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 0590535..2e34b89 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -442,6 +442,7 @@ LIBVIRT_0.9.2 { virDomainInjectNMI; virDomainScreenshot; virDomainSetSchedulerParametersFlags; + virNodeGetCPUStats; } LIBVIRT_0.9.0; # .... define new API here using predicted next version number .... -- 1.7.1
virNodeGetCPUStats: Define internal driver API Signed-off-by: Minoru Usui <usui@mxm.nes.nec.co.jp> --- src/driver.h | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/src/driver.h b/src/driver.h index 450dd53..e981a9f 100644 --- a/src/driver.h +++ b/src/driver.h @@ -364,6 +364,13 @@ typedef struct _virDriver virDriver; typedef virDriver *virDriverPtr; typedef int + (*virDrvNodeGetCPUStats) + (virConnectPtr conn, + virCPUStatsPtr params, + int *nparams, + unsigned int flags); + +typedef int (*virDrvNodeGetCellsFreeMemory) (virConnectPtr conn, unsigned long long *freeMems, @@ -694,6 +701,7 @@ struct _virDriver { virDrvDomainBlockPeek domainBlockPeek; virDrvDomainMemoryPeek domainMemoryPeek; virDrvDomainGetBlockInfo domainGetBlockInfo; + virDrvNodeGetCPUStats nodeGetCPUStats; virDrvNodeGetCellsFreeMemory nodeGetCellsFreeMemory; virDrvNodeGetFreeMemory nodeGetFreeMemory; virDrvDomainEventRegister domainEventRegister; -- 1.7.1
virNodeGetCPUStats: Implement public API Signed-off-by: Minoru Usui <usui@mxm.nes.nec.co.jp> --- src/libvirt.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 85 insertions(+), 0 deletions(-) diff --git a/src/libvirt.c b/src/libvirt.c index ff16c48..937d8ab 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -4967,6 +4967,91 @@ error: } /** + * virNodeGetCPUStats: + * @conn: pointer to the hypervisor connection. + * @params: pointer to node cpu time parameter objects + * @nparams: number of node cpu time parameter (this value should be same or + * less than the number of parameters supported) + * @flags: currently unused, for future extension. always pass 0. + * + * This function provides cpu time statistics of the node. + * The @params array will be filled with the values equal to the number of + * parameters suggested by @nparams + * + * As the value of @nparams is dynamic, call the API setting @nparams to 0 and + * @params as NULL, the API returns the number of parameters supported by the + * HV by updating @nparams on SUCCESS. The caller should then allocate @params + * array, i.e. (sizeof(@virCPUStats) * @nparams) bytes and call + * the API again. + * + * Here is the sample code snippet: + * + * if ((virNodeGetCPUStats(conn, NULL, &nparams, 0) == 0) && + * (nparams != 0)) { + * params = vshMalloc(ctl, sizeof(virCPUStats) * nparams); + * memset(params, 0, sizeof(virCPUStats) * nparams); + * if (virNodeGetCPUStats(conn, params, &nparams, 0)) { + * vshError(ctl, "%s", _("Unable to get node cpu stats")); + * goto error; + * } + * } + * + * This function doesn't requires privileged access to the hypervisor. + * This function expects the caller to allocate the @params. + * + * CPU time Statistics: + * + * VIR_NODE_CPU_STATS_KERNEL: + * The cumulative CPU time which spends by kernel, + * when the node booting up.(nanoseconds) + * VIR_NODE_CPU_STATS_USER: + * The cumulative CPU time which spends by user processes, + * when the node booting up.(nanoseconds) + * VIR_NODE_CPU_STATS_IDLE: + * The cumulative idle CPU time, when the node booting up.(nanoseconds) + * VIR_NODE_CPU_STATS_IOWAIT: + * The cumulative I/O wait CPU time, when the node booting up.(nanoseconds) + * VIR_NODE_CPU_STATS_UTILIZATION: + * The CPU utilization. The usage value is in percent and 100% + * represents all CPUs on the server. + * + * Returns -1 in case of error, 0 in case of success. + */ +int virNodeGetCPUStats (virConnectPtr conn, + virCPUStatsPtr params, + int *nparams, unsigned int flags) +{ + VIR_DEBUG("conn=%p, params=%p, nparams=%d, flags=%u", + conn, params, nparams ? *nparams : -1, flags); + + virResetLastError(); + + if (!VIR_IS_CONNECT(conn)) { + virLibConnError(VIR_ERR_INVALID_CONN, __FUNCTION__); + virDispatchError(NULL); + return -1; + } + + if ((nparams == NULL) || (*nparams < 0)) { + virLibConnError(VIR_ERR_INVALID_ARG, __FUNCTION__); + goto error; + } + + if (conn->driver->nodeGetCPUStats) { + int ret; + ret = conn->driver->nodeGetCPUStats (conn, params, nparams, flags); + if (ret < 0) + goto error; + return ret; + } + virLibConnError(VIR_ERR_NO_SUPPORT, __FUNCTION__); + +error: + virDispatchError(conn); + return -1; +} + +/** * virNodeGetFreeMemory: * @conn: pointer to the hypervisor connection * -- 1.7.1
virNodeGetCPUStats: Implement remote protocol Signed-off-by: Minoru Usui <usui@mxm.nes.nec.co.jp> --- daemon/remote.c | 76 ++++++++++++++++++++++++++++++++++++++++++ src/remote/remote_driver.c | 65 +++++++++++++++++++++++++++++++++++ src/remote/remote_protocol.x | 21 +++++++++++- 3 files changed, 161 insertions(+), 1 deletions(-) diff --git a/daemon/remote.c b/daemon/remote.c index 80783b3..fb603f4 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -1788,6 +1788,82 @@ cleanup: return rv; } +static int +remoteDispatchNodeGetCPUStats (struct qemud_server *server ATTRIBUTE_UNUSED, + struct qemud_client *client ATTRIBUTE_UNUSED, + virConnectPtr conn, + remote_message_header *hdr ATTRIBUTE_UNUSED, + remote_error *rerr, + remote_node_get_cpu_stats_args *args, + remote_node_get_cpu_stats_ret *ret) +{ + virCPUStatsPtr params = NULL; + int i; + int nparams = args->nparams; + unsigned int flags; + int rv = -1; + + if (!conn) { + virNetError(VIR_ERR_INTERNAL_ERROR, "%s", _("connection not open")); + goto cleanup; + } + + flags = args->flags; + + if (nparams > REMOTE_NODE_CPU_STATS_MAX) { + virNetError(VIR_ERR_INTERNAL_ERROR, "%s", _("nparams too large")); + goto cleanup; + } + if (VIR_ALLOC_N(params, nparams) < 0) { + virReportOOMError(); + goto cleanup; + } + + if (virNodeGetCPUStats(conn, params, &nparams, flags) < 0) + goto cleanup; + + /* In this case, we need to send back the number of stats + * supported + */ + if (args->nparams == 0) { + ret->nparams = nparams; + goto success; + } + + /* Serialise the memory parameters. */ + ret->params.params_len = nparams; + if (VIR_ALLOC_N(ret->params.params_val, nparams) < 0) + goto no_memory; + + for (i = 0; i < nparams; ++i) { + /* remoteDispatchClientRequest will free this: */ + ret->params.params_val[i].field = strdup(params[i].field); + if (ret->params.params_val[i].field == NULL) + goto no_memory; + + ret->params.params_val[i].value = params[i].value; + } + +success: + rv = 0; + +cleanup: + if (rv < 0) { + remoteDispatchError(rerr); + if (ret->params.params_val) { + for (i = 0; i < nparams; i++) + VIR_FREE(ret->params.params_val[i].field); + VIR_FREE(ret->params.params_val); + } + } + VIR_FREE(params); + return rv; + +no_memory: + virReportOOMError(); + goto cleanup; +} + /*-------------------------------------------------------------*/ static int diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 1691dab..924ea97 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -1694,6 +1694,70 @@ done: } static int +remoteNodeGetCPUStats (virConnectPtr conn, + virCPUStatsPtr params, int *nparams, + unsigned int flags) +{ + int rv = -1; + remote_node_get_cpu_stats_args args; + remote_node_get_cpu_stats_ret ret; + int i = -1; + struct private_data *priv = conn->privateData; + + remoteDriverLock(priv); + + args.nparams = *nparams; + args.flags = flags; + + memset (&ret, 0, sizeof ret); + if (call (conn, priv, 0, REMOTE_PROC_NODE_GET_CPU_STATS, + (xdrproc_t) xdr_remote_node_get_cpu_stats_args, + (char *) &args, + (xdrproc_t) xdr_remote_node_get_cpu_stats_ret, + (char *) &ret) == -1) + goto done; + + /* Check the length of the returned list carefully. */ + if (ret.params.params_len > REMOTE_NODE_CPU_STATS_MAX || + ret.params.params_len > *nparams) { + remoteError(VIR_ERR_RPC, "%s", + _("remoteNodeGetCPUStats: " + "returned number of stats exceeds limit")); + goto cleanup; + } + /* Handle the case when the caller does not know the number of stats + * and is asking for the number of stats supported + */ + if (*nparams == 0) { + *nparams = ret.nparams; + rv = 0; + goto cleanup; + } + + *nparams = ret.params.params_len; + + /* Deserialise the result. */ + for (i = 0; i < *nparams; ++i) { + if (virStrcpyStatic(params[i].field, ret.params.params_val[i].field) == NULL) { + remoteError(VIR_ERR_INTERNAL_ERROR, + _("Stats %s too big for destination"), + ret.params.params_val[i].field); + goto cleanup; + } + params[i].value = ret.params.params_val[i].value; + } + + rv = 0; + +cleanup: + xdr_free ((xdrproc_t) xdr_remote_node_get_cpu_stats_ret, + (char *) &ret); +done: + remoteDriverUnlock(priv); + return rv; +} + +static int remoteNodeGetCellsFreeMemory(virConnectPtr conn, unsigned long long *freeMems, int startCell, @@ -6823,6 +6887,7 @@ static virDriver remote_driver = { .domainBlockPeek = remoteDomainBlockPeek, /* 0.4.2 */ .domainMemoryPeek = remoteDomainMemoryPeek, /* 0.4.2 */ .domainGetBlockInfo = remoteDomainGetBlockInfo, /* 0.8.1 */ + .nodeGetCPUStats = remoteNodeGetCPUStats, /* 0.9.2 */ .nodeGetCellsFreeMemory = remoteNodeGetCellsFreeMemory, /* 0.3.3 */ .nodeGetFreeMemory = remoteNodeGetFreeMemory, /* 0.3.3 */ .domainEventRegister = remoteDomainEventRegister, /* 0.5.0 */ diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index f0da95d..30cb934 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -134,6 +134,9 @@ const REMOTE_DOMAIN_BLKIO_PARAMETERS_MAX = 16; /* Upper limit on list of memory parameters. */ const REMOTE_DOMAIN_MEMORY_PARAMETERS_MAX = 16; +/* Upper limit on list of cpu stats. */ +const REMOTE_NODE_CPU_STATS_MAX = 16; + /* Upper limit on number of NUMA cells */ const REMOTE_NODE_MAX_CELLS = 1024; @@ -359,6 +362,11 @@ struct remote_memory_param { remote_memory_param_value value; }; +struct remote_node_get_cpu_stats { + remote_nonnull_string field; + unsigned hyper value; +}; + /*----- Calls. -----*/ /* For each call we may have a 'remote_CALL_args' and 'remote_CALL_ret' @@ -440,6 +448,16 @@ struct remote_get_capabilities_ret { remote_nonnull_string capabilities; }; +struct remote_node_get_cpu_stats_args { + int nparams; + unsigned int flags; +}; + +struct remote_node_get_cpu_stats_ret { + remote_node_get_cpu_stats params<REMOTE_NODE_CPU_STATS_MAX>; + int nparams; +}; + struct remote_node_get_cells_free_memory_args { int startCell; int maxCells; @@ -2291,7 +2309,8 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_MIGRATE_PERFORM3 = 216, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_FINISH3 = 217, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_MIGRATE_CONFIRM3 = 218, /* skipgen skipgen */ - REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS_FLAGS = 219 /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_SET_SCHEDULER_PARAMETERS_FLAGS = 219, /* skipgen skipgen */ + REMOTE_PROC_NODE_GET_CPU_STATS = 220 /* skipgen skipgen */ /* * Notice how the entries are grouped in sets of 10 ? -- 1.7.1
virNodeGetCPUStats: Implement virsh support Add nodecpustats subcommand to virsh. This subcommand prints below output. [Linux] # virsh nodecpustats user : 3820240000000 system: 3029560000000 idle : 186380870000000 iowait: 518300000000 # virsh nodecpustats --percent usage : 0.5% user : 0.0% system: 0.5% idle : 99.5% iowait: 0.0% [can get cpu utilization directly(ESX?)] # virsh nodecpustats # virsh nodecpustats --percent usage : 0.5% idle : 99.5% Signed-off-by: Minoru Usui <usui@mxm.nes.nec.co.jp> --- tools/virsh.c | 127 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 6 +++ 2 files changed, 133 insertions(+), 0 deletions(-) diff --git a/tools/virsh.c b/tools/virsh.c index de49489..6d6a5dd 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -3490,6 +3490,132 @@ cmdNodeinfo(vshControl *ctl, const vshCmd *cmd ATTRIBUTE_UNUSED) } /* + * "nodecpustats" command + */ +static const vshCmdInfo info_nodecpustats[] = { + {"help", N_("Prints cpu stats of the node.")}, + {"desc", N_("Returns cpu stats of the node.(nsec)")}, + {NULL, NULL} +}; + +static const vshCmdOptDef opts_node_cpustats[] = { + {"percent", VSH_OT_BOOL, 0, N_("prints by percentage during 1 second.")}, + {NULL, 0, 0, NULL} +}; + +static bool +cmdNodeCPUStats(vshControl *ctl, const vshCmd *cmd ATTRIBUTE_UNUSED) +{ + int i, j; + bool flag_utilization = false; + bool flag_percent = vshCommandOptBool(cmd, "percent"); + virCPUStatsPtr params; + int nparams = 0; + bool ret = false; + struct cpu_stats { + unsigned long long user; + unsigned long long sys; + unsigned long long idle; + unsigned long long iowait; + unsigned long long util; + } cpu_stats[2]; + double user_time, sys_time, idle_time, iowait_time, total_time; + double usage; + + if (!vshConnectionUsability(ctl, ctl->conn)) + return false; + + if (virNodeGetCPUStats(ctl->conn, NULL, &nparams, 0) != 0) { + vshError(ctl, "%s", + _("Unable to get number of cpu stats")); + return false; + } + if (nparams == 0) { + /* nothing to output */ + return true; + } + + memset(cpu_stats, 0, sizeof(struct cpu_stats) * 2); + params = vshCalloc(ctl, nparams, sizeof(*params)); + + i = 0; + do { + if (virNodeGetCPUStats(ctl->conn, params, &nparams, 0) != 0) { + vshError(ctl, "%s", _("Unable to get node cpu stats")); + goto cleanup; + } + + for (j = 0; j < nparams; j++) { + unsigned long long value = params[j].value; + + if (strcmp(params[j].field, VIR_CPU_STATS_KERNEL) == 0) + cpu_stats[i].sys = value; + + if (strcmp(params[j].field, VIR_CPU_STATS_USER) == 0) + cpu_stats[i].user = value; + + if (strcmp(params[j].field, VIR_CPU_STATS_IDLE) == 0) + cpu_stats[i].idle = value; + + if (strcmp(params[j].field, VIR_CPU_STATS_IOWAIT) == 0) + cpu_stats[i].iowait = value; + + if (strcmp(params[j].field, VIR_CPU_STATS_UTILIZATION) == 0) { + cpu_stats[i].util = value; + flag_utilization = true; + } + } + + if (flag_utilization || (flag_percent == false)) + break; + + i++; + sleep(1); + } while(i < 2); + + if (flag_percent == false) { + if (flag_utilization == false) { + vshPrint(ctl, "%-15s %20llu\n", _("user :"), cpu_stats[0].user); + vshPrint(ctl, "%-15s %20llu\n", _("system:"), cpu_stats[0].sys); + vshPrint(ctl, "%-15s %20llu\n", _("idle :"), cpu_stats[0].idle); + vshPrint(ctl, "%-15s %20llu\n", _("iowait:"), cpu_stats[0].iowait); + } + } else { + if (flag_utilization) { + usage = cpu_stats[0].util; + + vshPrint(ctl, "%-15s %5.1lf%%\n", _("usage:"), usage); + vshPrint(ctl, "%-15s %5.1lf%%\n", _("idle :"), 100 - usage); + } else { + user_time = cpu_stats[1].user - cpu_stats[0].user; + sys_time = cpu_stats[1].sys - cpu_stats[0].sys; + idle_time = cpu_stats[1].idle - cpu_stats[0].idle; + iowait_time = cpu_stats[1].iowait - cpu_stats[0].iowait; + total_time = user_time + sys_time + idle_time + iowait_time; + + usage = (user_time + sys_time) / total_time * 100; + + vshPrint(ctl, "%-15s %5.1lf%%\n", + _("usage:"), usage); + vshPrint(ctl, "%-15s %5.1lf%%\n", + _(" user :"), user_time / total_time * 100); + vshPrint(ctl, "%-15s %5.1lf%%\n", + _(" system:"), sys_time / total_time * 100); + vshPrint(ctl, "%-15s %5.1lf%%\n", + _("idle :"), idle_time / total_time * 100); + vshPrint(ctl, "%-15s %5.1lf%%\n", + _("iowait:"), iowait_time / total_time * 100); + } + } + + ret = true; + + cleanup: + VIR_FREE(params); + return ret; +} + +/* * "capabilities" command */ static const vshCmdInfo info_capabilities[] = { @@ -10998,6 +11124,7 @@ static const vshCmdDef hostAndHypervisorCmds[] = { {"freecell", cmdFreecell, opts_freecell, info_freecell, 0}, {"hostname", cmdHostname, NULL, info_hostname, 0}, {"nodeinfo", cmdNodeinfo, NULL, info_nodeinfo, 0}, + {"nodecpustats", cmdNodeCPUStats, opts_node_cpustats, info_nodecpustats}, {"qemu-monitor-command", cmdQemuMonitorCommand, opts_qemu_monitor_command, info_qemu_monitor_command, 0}, {"sysinfo", cmdSysinfo, NULL, info_sysinfo, 0}, diff --git a/tools/virsh.pod b/tools/virsh.pod index ef01f41..76f6616 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -237,6 +237,12 @@ Print the XML representation of the hypervisor sysinfo, if available. Returns basic information about the node, like number and type of CPU, and size of the physical memory. +=item B<nodecpustats> + +Returns cpu stats of the node, if I<--percent> +is specified this will prints percentage of each kind of cpu time +during 1 second. + =item B<capabilities> Print an XML document describing the capabilities of the hypervisor -- 1.7.1
virNodeGetCPUStats: Implement linux support Signed-off-by: Minoru Usui <usui@mxm.nes.nec.co.jp> --- src/libvirt_private.syms | 1 + src/lxc/lxc_driver.c | 1 + src/nodeinfo.c | 127 ++++++++++++++++++++++++++++++++++++++++++++++ src/nodeinfo.h | 5 ++- src/qemu/qemu_driver.c | 1 + src/uml/uml_driver.c | 1 + 6 files changed, 135 insertions(+), 1 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 7b6151c..d8f26c4 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -687,6 +687,7 @@ virNodeDeviceObjUnlock; # nodeinfo.h nodeCapsInitNUMA; +nodeGetCPUStats; nodeGetCellsFreeMemory; nodeGetFreeMemory; nodeGetInfo; diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 9e09c95..743219c 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -2754,6 +2754,7 @@ static virDriver lxcDriver = { .domainGetSchedulerParameters = lxcGetSchedulerParameters, /* 0.5.0 */ .domainSetSchedulerParameters = lxcSetSchedulerParameters, /* 0.5.0 */ .domainInterfaceStats = lxcDomainInterfaceStats, /* 0.7.3 */ + .nodeGetCPUStats = nodeGetCPUStats, /* 0.9.2 */ .nodeGetCellsFreeMemory = nodeGetCellsFreeMemory, /* 0.6.5 */ .nodeGetFreeMemory = nodeGetFreeMemory, /* 0.6.5 */ .domainEventRegister = lxcDomainEventRegister, /* 0.7.0 */ diff --git a/src/nodeinfo.c b/src/nodeinfo.c index f55c83e..a1e37eb 100644 --- a/src/nodeinfo.c +++ b/src/nodeinfo.c @@ -57,12 +57,19 @@ #ifdef __linux__ # define CPUINFO_PATH "/proc/cpuinfo" # define CPU_SYS_PATH "/sys/devices/system/cpu" +# define PROCSTAT_PATH "/proc/stat" + +#define LINUX_NB_CPU_STATS 4 /* NB, this is not static as we need to call it from the testsuite */ int linuxNodeInfoCPUPopulate(FILE *cpuinfo, virNodeInfoPtr nodeinfo, bool need_hyperthreads); +int linuxNodeGetCPUStats(FILE *procstat, + virCPUStatsPtr params, + int *nparams); + /* Return the positive decimal contents of the given * CPU_SYS_PATH/cpu%u/FILE, or -1 on error. If MISSING_OK and the * file could not be found, return 1 instead of an error; this is @@ -378,6 +385,99 @@ int linuxNodeInfoCPUPopulate(FILE *cpuinfo, return 0; } +#define TICK_TO_NSEC (1000ull * 1000ull * 1000ull / sysconf(_SC_CLK_TCK)) + +int linuxNodeGetCPUStats(FILE *procstat, + virCPUStatsPtr params, + int *nparams) +{ + int ret = -1; + char line[1024]; + unsigned long long usr, ni, sys, idle, iowait; + unsigned long long irq, softirq, steal, guest, guest_nice; + + if ((*nparams) == 0) { + /* Current number of cpu stats supported by linux */ + *nparams = LINUX_NB_CPU_STATS; + ret = 0; + goto cleanup; + } + + if ((*nparams) != LINUX_NB_CPU_STATS) { + nodeReportError(VIR_ERR_INVALID_ARG, + "%s", _("Invalid parameter count")); + goto cleanup; + } + + while (fgets(line, sizeof(line), procstat) != NULL) { + char *buf = line; + + if (STRPREFIX(buf, "cpu ")) { /* aka total logical CPU time */ + int i; + + if (sscanf(buf, + "%*s %llu %llu %llu %llu %llu" // user ~ iowait + "%llu %llu %llu %llu %llu", // irq ~ guest_nice + &usr, &ni, &sys, &idle, &iowait, + &irq, &softirq, &steal, &guest, &guest_nice) < 4) { + continue; + } + + for (i = 0; i < *nparams; i++) { + virCPUStatsPtr param = ¶ms[i]; + + switch (i) { + case 0: /* fill kernel cpu time here */ + if (virStrcpyStatic(param->field, VIR_CPU_STATS_KERNEL)== NULL) { + nodeReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("Field kernel cpu time too long for destination")); + goto cleanup; + } + param->value = (sys + irq + softirq) * TICK_TO_NSEC; + break; + + case 1: /* fill user cpu time here */ + if (virStrcpyStatic(param->field, VIR_CPU_STATS_USER) == NULL) { + nodeReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("Field kernel cpu time too long for destination")); + goto cleanup; + } + param->value = (usr + ni) * TICK_TO_NSEC; + break; + + case 2: /* fill idle cpu time here */ + if (virStrcpyStatic(param->field, VIR_CPU_STATS_IDLE) == NULL) { + nodeReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("Field kernel cpu time too long for destination")); + goto cleanup; + } + param->value = idle * TICK_TO_NSEC; + break; + + case 3: /* fill iowait cpu time here */ + if (virStrcpyStatic(param->field, VIR_CPU_STATS_IOWAIT) == NULL) { + nodeReportError(VIR_ERR_INTERNAL_ERROR, + "%s", _("Field kernel cpu time too long for destination")); + goto cleanup; + } + param->value = iowait * TICK_TO_NSEC; + break; + + default: + break; + /* should not hit here */ + } + } + ret = 0; + goto cleanup; + } + } + + nodeReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("no \'cpu \' line found")); + +cleanup: + return ret; +} #endif int nodeGetInfo(virConnectPtr conn ATTRIBUTE_UNUSED, virNodeInfoPtr nodeinfo) { @@ -416,6 +516,33 @@ int nodeGetInfo(virConnectPtr conn ATTRIBUTE_UNUSED, virNodeInfoPtr nodeinfo) { #endif } +int nodeGetCPUStats(virConnectPtr conn ATTRIBUTE_UNUSED, + virCPUStatsPtr params, + int *nparams, + unsigned int flags ATTRIBUTE_UNUSED) +{ + +#ifdef __linux__ + { + int ret; + FILE *procstat = fopen(PROCSTAT_PATH, "r"); + if (!procstat) { + virReportSystemError(errno, + _("cannot open %s"), PROCSTAT_PATH); + return -1; + } + ret = linuxNodeGetCPUStats(procstat, params, nparams); + VIR_FORCE_FCLOSE(procstat); + + return ret; + } +#else + nodeReportError(VIR_ERR_NO_SUPPORT, "%s", + _("node CPU stats not implemented on this platform")); + return -1; +#endif +} + #if HAVE_NUMACTL # if LIBNUMA_API_VERSION <= 1 # define NUMA_MAX_N_CPUS 4096 diff --git a/src/nodeinfo.h b/src/nodeinfo.h index 88bac6c..241e521 100644 --- a/src/nodeinfo.h +++ b/src/nodeinfo.h @@ -30,7 +30,10 @@ int nodeGetInfo(virConnectPtr conn, virNodeInfoPtr nodeinfo); int nodeCapsInitNUMA(virCapsPtr caps); - +int nodeGetCPUStats(virConnectPtr conn ATTRIBUTE_UNUSED, + virCPUStatsPtr params, + int *nparams, + unsigned int flags ATTRIBUTE_UNUSED); int nodeGetCellsFreeMemory(virConnectPtr conn, unsigned long long *freeMems, int startCell, diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 691965d..3b58b5a 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -7701,6 +7701,7 @@ static virDriver qemuDriver = { .domainBlockPeek = qemudDomainBlockPeek, /* 0.4.4 */ .domainMemoryPeek = qemudDomainMemoryPeek, /* 0.4.4 */ .domainGetBlockInfo = qemuDomainGetBlockInfo, /* 0.8.1 */ + .nodeGetCPUStats = nodeGetCPUStats, /* 0.9.2 */ .nodeGetCellsFreeMemory = nodeGetCellsFreeMemory, /* 0.4.4 */ .nodeGetFreeMemory = nodeGetFreeMemory, /* 0.4.4 */ .domainEventRegister = qemuDomainEventRegister, /* 0.5.0 */ diff --git a/src/uml/uml_driver.c b/src/uml/uml_driver.c index 536cd8c..175d7b2 100644 --- a/src/uml/uml_driver.c +++ b/src/uml/uml_driver.c @@ -2218,6 +2218,7 @@ static virDriver umlDriver = { .domainGetAutostart = umlDomainGetAutostart, /* 0.5.0 */ .domainSetAutostart = umlDomainSetAutostart, /* 0.5.0 */ .domainBlockPeek = umlDomainBlockPeek, /* 0.5.0 */ + .nodeGetCPUStats = nodeGetCPUStats, /* 0.9.2 */ .nodeGetCellsFreeMemory = nodeGetCellsFreeMemory, /* 0.5.0 */ .nodeGetFreeMemory = nodeGetFreeMemory, /* 0.5.0 */ .isEncrypted = umlIsEncrypted, /* 0.7.3 */ -- 1.7.1
On Wed, May 25, 2011 at 03:01:29PM +0900, Minoru Usui wrote:
Hi,
This is v6 of virNodeGetCPUTimeParameters() API. (This time, I rename it to virNodeGetCPUStats()) It returns cpu utilization or cumulative cpu time of the node from /proc/stat since node boots up. This patch only supports linux host.
The code in this patch series looks fine to me now. I would ACK the whole series, but I have one design question I should have asked previously. This API returns CPU information for the host as a whole. Historically, when adding an API like this, someone has always then asked for a per-CPU breakdown of the same data. So I wonder if we should add a parameter for the CPU num to the API: int virNodeGetCPUStats (virConnectPtr conn, int cpuNum, virCPUStatsPtr params, int *nparams, unsigned int flags); And have a constant for getting the host data as a whole: #define VIR_NODE_CPU_STATS_ALL_CPUS -1 so if you want to get individual data for each CPU you can do virNodeGetCPUStats(conn, 0, params, nparams, flags); virNodeGetCPUStats(conn, 1, params, nparams, flags); virNodeGetCPUStats(conn, 2, params, nparams, flags); virNodeGetCPUStats(conn, 3, params, nparams, flags); But if you just want the data for the host as a whole you can do virNodeGetCPUStats(conn, VIR_NODE_CPU_STATS_ALL_CPUS, params, nparams, flags); The /proc/stat file already has the data we need for both these styles $ grep cpu /proc/stat cpu 2632143 17400 1341532 32819705 1177560 674 21606 0 11997 0 cpu0 1263069 9280 596278 16274365 620428 631 17303 0 6131 0 cpu1 1369073 8119 745254 16545339 557131 43 4302 0 5865 0 Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Fri, May 27, 2011 at 02:34:50PM +0100, Daniel P. Berrange wrote:
On Wed, May 25, 2011 at 03:01:29PM +0900, Minoru Usui wrote:
Hi,
This is v6 of virNodeGetCPUTimeParameters() API. (This time, I rename it to virNodeGetCPUStats()) It returns cpu utilization or cumulative cpu time of the node from /proc/stat since node boots up. This patch only supports linux host.
The code in this patch series looks fine to me now. I would ACK the whole series, but I have one design question I should have asked previously.
This API returns CPU information for the host as a whole. Historically, when adding an API like this, someone has always then asked for a per-CPU breakdown of the same data.
So I wonder if we should add a parameter for the CPU num to the API:
int virNodeGetCPUStats (virConnectPtr conn, int cpuNum, virCPUStatsPtr params, int *nparams, unsigned int flags);
And have a constant for getting the host data as a whole:
#define VIR_NODE_CPU_STATS_ALL_CPUS -1
so if you want to get individual data for each CPU you can do
virNodeGetCPUStats(conn, 0, params, nparams, flags); virNodeGetCPUStats(conn, 1, params, nparams, flags); virNodeGetCPUStats(conn, 2, params, nparams, flags); virNodeGetCPUStats(conn, 3, params, nparams, flags);
But if you just want the data for the host as a whole you can do
virNodeGetCPUStats(conn, VIR_NODE_CPU_STATS_ALL_CPUS, params, nparams, flags);
The /proc/stat file already has the data we need for both these styles
$ grep cpu /proc/stat cpu 2632143 17400 1341532 32819705 1177560 674 21606 0 11997 0 cpu0 1263069 9280 596278 16274365 620428 631 17303 0 6131 0 cpu1 1369073 8119 745254 16545339 557131 43 4302 0 5865 0
Agreed, similary as for the virNodeGetMemoryStats() patch set, the patches looks good enough for being applied but doing the suggested changes (i.e. adding a new argument for the cpu number or the cell number). For virsh command I would make querying all the default but just add an optionnal argument --cpu virsh nodecpustats --cpu cpunumber and similary on the memry patch set add an optional --cell argument virsh nodememstats --cell cellnumber That way the API and virsh command are ready for more precise use and we won't need to make a second set of APIs. thanks ! Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Fri, 27 May 2011 22:33:00 +0800 Daniel Veillard <veillard@redhat.com> wrote:
On Fri, May 27, 2011 at 02:34:50PM +0100, Daniel P. Berrange wrote:
On Wed, May 25, 2011 at 03:01:29PM +0900, Minoru Usui wrote:
Hi,
This is v6 of virNodeGetCPUTimeParameters() API. (This time, I rename it to virNodeGetCPUStats()) It returns cpu utilization or cumulative cpu time of the node from /proc/stat since node boots up. This patch only supports linux host.
The code in this patch series looks fine to me now. I would ACK the whole series, but I have one design question I should have asked previously.
This API returns CPU information for the host as a whole. Historically, when adding an API like this, someone has always then asked for a per-CPU breakdown of the same data.
So I wonder if we should add a parameter for the CPU num to the API:
int virNodeGetCPUStats (virConnectPtr conn, int cpuNum, virCPUStatsPtr params, int *nparams, unsigned int flags);
And have a constant for getting the host data as a whole:
#define VIR_NODE_CPU_STATS_ALL_CPUS -1
so if you want to get individual data for each CPU you can do
virNodeGetCPUStats(conn, 0, params, nparams, flags); virNodeGetCPUStats(conn, 1, params, nparams, flags); virNodeGetCPUStats(conn, 2, params, nparams, flags); virNodeGetCPUStats(conn, 3, params, nparams, flags);
But if you just want the data for the host as a whole you can do
virNodeGetCPUStats(conn, VIR_NODE_CPU_STATS_ALL_CPUS, params, nparams, flags);
The /proc/stat file already has the data we need for both these styles
$ grep cpu /proc/stat cpu 2632143 17400 1341532 32819705 1177560 674 21606 0 11997 0 cpu0 1263069 9280 596278 16274365 620428 631 17303 0 6131 0 cpu1 1369073 8119 745254 16545339 557131 43 4302 0 5865 0
Agreed, similary as for the virNodeGetMemoryStats() patch set, the patches looks good enough for being applied but doing the suggested changes (i.e. adding a new argument for the cpu number or the cell number). For virsh command I would make querying all the default but just add an optionnal argument --cpu virsh nodecpustats --cpu cpunumber and similary on the memry patch set add an optional --cell argument virsh nodememstats --cell cellnumber
That way the API and virsh command are ready for more precise use and we won't need to make a second set of APIs.
thanks !
OK. I'll add cpu/cell argments for more generalization.
Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
-- Minoru Usui <usui@mxm.nes.nec.co.jp>
On 05/29/2011 07:54 PM, Minoru Usui wrote:
So I wonder if we should add a parameter for the CPU num to the API:
int virNodeGetCPUStats (virConnectPtr conn, int cpuNum, virCPUStatsPtr params, int *nparams, unsigned int flags);
And have a constant for getting the host data as a whole:
#define VIR_NODE_CPU_STATS_ALL_CPUS -1
That way the API and virsh command are ready for more precise use and we won't need to make a second set of APIs.
thanks !
OK. I'll add cpu/cell argments for more generalization.
At this point, I'd feel more comfortable delaying this patch series (as well the virNodeGetMemoryStats series) until after the 0.9.2 release. Feel free to post v7 for review before them, but I think we've had enough churn on the interface, and we've already hit the freeze with release candidate 1, that I don't feel right pushing this upstream this late in a release cycle, even though the principle idea has been ACK'd. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On Tue, 31 May 2011 14:50:30 -0600 Eric Blake <eblake@redhat.com> wrote:
On 05/29/2011 07:54 PM, Minoru Usui wrote:
So I wonder if we should add a parameter for the CPU num to the API:
int virNodeGetCPUStats (virConnectPtr conn, int cpuNum, virCPUStatsPtr params, int *nparams, unsigned int flags);
And have a constant for getting the host data as a whole:
#define VIR_NODE_CPU_STATS_ALL_CPUS -1
That way the API and virsh command are ready for more precise use and we won't need to make a second set of APIs.
thanks !
OK. I'll add cpu/cell argments for more generalization.
At this point, I'd feel more comfortable delaying this patch series (as well the virNodeGetMemoryStats series) until after the 0.9.2 release. Feel free to post v7 for review before them, but I think we've had enough churn on the interface, and we've already hit the freeze with release candidate 1, that I don't feel right pushing this upstream this late in a release cycle, even though the principle idea has been ACK'd.
I understand. I'll change my target to 0.9.3. -- Minoru Usui <usui@mxm.nes.nec.co.jp>
On Wed, 25 May 2011 15:01:29 +0900 Minoru Usui <usui@mxm.nes.nec.co.jp> wrote:
Hi,
This is v6 of virNodeGetCPUTimeParameters() API. (This time, I rename it to virNodeGetCPUStats()) It returns cpu utilization or cumulative cpu time of the node from /proc/stat since node boots up. This patch only supports linux host.
Can I make a question ? What "CPU' means here ? CPU usage statistics from the Guest OS's point of view ? Do we get the information by asking an daemon or module on the guest to gather statistics on the guest ? Thanks, -Kame
On Mon, 30 May 2011 09:36:50 +0900 KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> wrote:
On Wed, 25 May 2011 15:01:29 +0900 Minoru Usui <usui@mxm.nes.nec.co.jp> wrote:
Hi,
This is v6 of virNodeGetCPUTimeParameters() API. (This time, I rename it to virNodeGetCPUStats()) It returns cpu utilization or cumulative cpu time of the node from /proc/stat since node boots up. This patch only supports linux host.
Can I make a question ?
What "CPU' means here ?
CPU usage statistics from the Guest OS's point of view ?
Do we get the information by asking an daemon or module on the guest to gather statistics on the guest ?
No. This API gets host cpu statistics, not guest. It's just read statistics from /proc/stat of the host.
Thanks, -Kame
-- Minoru Usui <usui@mxm.nes.nec.co.jp>
On Mon, 30 May 2011 11:03:35 +0900 Minoru Usui <usui@mxm.nes.nec.co.jp> wrote:
On Mon, 30 May 2011 09:36:50 +0900 KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> wrote:
On Wed, 25 May 2011 15:01:29 +0900 Minoru Usui <usui@mxm.nes.nec.co.jp> wrote:
Hi,
This is v6 of virNodeGetCPUTimeParameters() API. (This time, I rename it to virNodeGetCPUStats()) It returns cpu utilization or cumulative cpu time of the node from /proc/stat since node boots up. This patch only supports linux host.
Can I make a question ?
What "CPU' means here ?
CPU usage statistics from the Guest OS's point of view ?
Do we get the information by asking an daemon or module on the guest to gather statistics on the guest ?
No. This API gets host cpu statistics, not guest.
It's just read statistics from /proc/stat of the host.
ok. I have a patch to get per-domain/per-cpu cpu usage statistics by using cpuacct cgroup. (for fixining #virt-top -1.) yours are per-conn, mine is per-domain. I'd like to remake my patch to use the same style as yours and to use the same macro/params. As... == int virDomainGetCPUStats (virDomainPtr dom, virCPUStatsPtr params, int *nparams, unsigned int flags); == I think "kernel" and "user", for linux. Thank you for your patch. Thanks, -Kame
participants (5)
-
Daniel P. Berrange -
Daniel Veillard -
Eric Blake -
KAMEZAWA Hiroyuki -
Minoru Usui