[libvirt] [PATCH 0/4 0/1 0/1 V2] Add new public API virDomainGetPcpusUsage() and pcpuinfo command in virsh
"virt-top -1" can call virDomainGetPcpusUsage() periodically and get the CPU activities per CPU. (See the last patch in this series). virsh is also added a pcpuinfo command which calls virDomainGetPcpusUsage(), it gets information about the physic CPUs, such as the usage of CPUs, the current attached vCPUs. # virsh pcpuinfo rhel6 CPU: 0 Curr VCPU: - Usage: 47.3 CPU: 1 Curr VCPU: 1 Usage: 46.8 CPU: 2 Curr VCPU: 0 Usage: 52.7 CPU: 3 Curr VCPU: - Usage: 44.1 Patch for libvirt(4 patches): daemon/remote.c | 68 ++++++++++++++++++++++++++++++ include/libvirt/libvirt.h.in | 5 ++ python/generator.py | 1 + src/driver.h | 7 +++ src/libvirt.c | 51 +++++++++++++++++++++++ src/libvirt_public.syms | 5 ++ src/qemu/qemu.conf | 5 +- src/qemu/qemu_conf.c | 3 +- src/qemu/qemu_driver.c | 74 +++++++++++++++++++++++++++++++++ src/remote/remote_driver.c | 51 +++++++++++++++++++++++ src/remote/remote_protocol.x | 17 +++++++- src/remote_protocol-structs | 13 ++++++ src/util/cgroup.c | 7 +++ src/util/cgroup.h | 1 + tools/virsh.c | 93 ++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 5 ++ 16 files changed, 402 insertions(+), 4 deletions(-) Patch for ocaml-libvirt (1 patch): libvirt/libvirt.ml | 1 + libvirt/libvirt.mli | 4 ++++ libvirt/libvirt_c_oneoffs.c | 25 +++++++++++++++++++++++++ 3 files changed, 30 insertions(+), 0 deletions(-) Patch for virt-top (1 patch): virt-top/virt_top.ml | 75 +++++++++++++++++-------------------------------- 1 files changed, 26 insertions(+), 49 deletions(-) -- 1.7.4.4
Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- include/libvirt/libvirt.h.in | 5 ++++ python/generator.py | 1 + src/driver.h | 7 +++++ src/libvirt.c | 51 ++++++++++++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 5 ++++ 5 files changed, 69 insertions(+), 0 deletions(-) diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index ad6fcce..fc92143 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -3600,6 +3600,11 @@ int virConnectSetKeepAlive(virConnectPtr conn, int interval, unsigned int count); +int virDomainGetPcpusUsage(virDomainPtr dom, + unsigned long long *usages, + int *nr_usages, + unsigned int flags); + #ifdef __cplusplus } #endif diff --git a/python/generator.py b/python/generator.py index 6fee3a4..0311004 100755 --- a/python/generator.py +++ b/python/generator.py @@ -421,6 +421,7 @@ skip_impl = ( 'virDomainGetBlockIoTune', 'virDomainSetInterfaceParameters', 'virDomainGetInterfaceParameters', + 'virDomainGetPcpusUsage', # not implemented yet ) qemu_skip_impl = ( diff --git a/src/driver.h b/src/driver.h index ec4abf3..2cf408a 100644 --- a/src/driver.h +++ b/src/driver.h @@ -793,6 +793,12 @@ typedef int int *nparams, unsigned int flags); +typedef int + (*virDrvDomainGetPcpusUsage)(virDomainPtr dom, + unsigned long long *usages, + int *nr_usages, + unsigned int flags); + /** * _virDriver: * @@ -961,6 +967,7 @@ struct _virDriver { virDrvNodeSuspendForDuration nodeSuspendForDuration; virDrvDomainSetBlockIoTune domainSetBlockIoTune; virDrvDomainGetBlockIoTune domainGetBlockIoTune; + virDrvDomainGetPcpusUsage domainGetPcpusUsage; }; typedef int diff --git a/src/libvirt.c b/src/libvirt.c index feb3ca6..79ff2df 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -17867,3 +17867,54 @@ error: virDispatchError(dom->conn); return -1; } + +/** + * virDomainGetPcpusUsage: + * @dom: pointer to domain object + * @usages: returned physic cpu usages + * @nr_usages: length of @usages + * @flags: flags to control the operation + * + * Get the cpu usages per every physic cpu + * + * Returns 0 if success, -1 on error + */ +int virDomainGetPcpusUsage(virDomainPtr dom, + unsigned long long *usages, + int *nr_usages, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(dom, "usages=%p, nr_usages=%d, flags=%x", + usages, (nr_usages) ? *nr_usages : -1, flags); + + virResetLastError(); + + if (!VIR_IS_CONNECTED_DOMAIN (dom)) { + virLibDomainError(VIR_ERR_INVALID_DOMAIN, __FUNCTION__); + virDispatchError(NULL); + return -1; + } + + if (nr_usages == NULL && *nr_usages != 0) { + virLibDomainError(VIR_ERR_INVALID_ARG, __FUNCTION__); + goto error; + } + + conn = dom->conn; + + if (conn->driver->domainGetPcpusUsage) { + int ret; + ret = conn->driver->domainGetPcpusUsage(dom, usages, nr_usages, flags); + if (ret < 0) + goto error; + return ret; + } + + virLibDomainError(VIR_ERR_NO_SUPPORT, __FUNCTION__); + +error: + virDispatchError(dom->conn); + return -1; +} diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 4ca7216..15d944c 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -516,4 +516,9 @@ LIBVIRT_0.9.9 { virDomainSetNumaParameters; } LIBVIRT_0.9.8; +LIBVIRT_0.9.10 { + global: + virDomainGetPcpusUsage; +} LIBVIRT_0.9.9; + # .... define new API here using predicted next version number .... -- 1.7.4.4
On Wed, Jan 04, 2012 at 12:09:18PM +0800, Lai Jiangshan wrote:
+int virDomainGetPcpusUsage(virDomainPtr dom, + unsigned long long *usages, + int *nr_usages, + unsigned int flags);
It'd be useful to get feedback from libvirt folk about whether this API is reasonable.
+/** + * virDomainGetPcpusUsage: + * @dom: pointer to domain object + * @usages: returned physic cpu usages
'physical'
+ * @nr_usages: length of @usages + * @flags: flags to control the operation + * + * Get the cpu usages per every physic cpu
'physical' I think this needs to describe what exactly is returned. Nanoseconds since the domain started, I assume? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw
On 01/05/2012 10:23 PM, Richard W.M. Jones wrote:
On Wed, Jan 04, 2012 at 12:09:18PM +0800, Lai Jiangshan wrote:
+int virDomainGetPcpusUsage(virDomainPtr dom, + unsigned long long *usages, + int *nr_usages, + unsigned int flags);
It'd be useful to get feedback from libvirt folk about whether this API is reasonable.
+/** + * virDomainGetPcpusUsage: + * @dom: pointer to domain object + * @usages: returned physic cpu usages
'physical'
+ * @nr_usages: length of @usages + * @flags: flags to control the operation + * + * Get the cpu usages per every physic cpu
'physical'
I think this needs to describe what exactly is returned. Nanoseconds since the domain started, I assume?
Yes. I will apply your comments in next version which will be deferred until 0.9.9 released as Eric suggested. Thank you very much. Lai
Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- daemon/remote.c | 68 ++++++++++++++++++++++++++++++++++++++++++ src/remote/remote_driver.c | 51 +++++++++++++++++++++++++++++++ src/remote/remote_protocol.x | 17 ++++++++++- src/remote_protocol-structs | 13 ++++++++ 4 files changed, 148 insertions(+), 1 deletions(-) diff --git a/daemon/remote.c b/daemon/remote.c index a28a754..9b0206f 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -2084,6 +2084,74 @@ cleanup: return rv; } +static int +remoteDispatchDomainGetPcpusUsage(virNetServerPtr server ATTRIBUTE_UNUSED, + virNetServerClientPtr client ATTRIBUTE_UNUSED, + virNetMessagePtr hdr ATTRIBUTE_UNUSED, + virNetMessageErrorPtr rerr, + remote_domain_get_pcpus_usage_args *args, + remote_domain_get_pcpus_usage_ret *ret) +{ + int i; + virDomainPtr dom = NULL; + int rv = -1; + unsigned long long *usages; + int nr_usages = args->nr_usages; + struct daemonClientPrivate *priv = + virNetServerClientGetPrivateData(client); + + if (!priv->conn) { + virNetError(VIR_ERR_INTERNAL_ERROR, "%s", _("connection not open")); + goto cleanup; + } + + if (nr_usages > REMOTE_DOMAIN_PCPUS_USAGE_PARAMETERS_MAX) { + virNetError(VIR_ERR_INTERNAL_ERROR, "%s", _("nr_usages too large")); + goto cleanup; + } + + if (VIR_ALLOC_N(usages, nr_usages) < 0) { + virReportOOMError(); + goto cleanup; + } + + if (!(dom = get_nonnull_domain(priv->conn, args->dom))) + goto cleanup; + + if (virDomainGetPcpusUsage(dom, usages, &nr_usages, args->flags) < 0) + goto cleanup; + + ret->nr_usages = nr_usages; + + /* + * In this case, we need to send back the number of parameters + * supported + */ + if (args->nr_usages == 0) { + goto success; + } + + ret->usages.usages_len = MIN(args->nr_usages, nr_usages); + if (VIR_ALLOC_N(ret->usages.usages_val, ret->usages.usages_len) < 0) { + virReportOOMError(); + goto cleanup; + } + + for (i = 0; i < ret->usages.usages_len; i++) { + ret->usages.usages_val[i] = usages[i]; + } + +success: + rv = 0; + +cleanup: + if (rv < 0) + virNetMessageSaveError(rerr); + VIR_FREE(usages); + if (dom) + virDomainFree(dom); + return rv; +} #ifdef HAVE_SASL /* diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 7580477..7dd3876 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -2305,6 +2305,56 @@ done: return rv; } +static int remoteDomainGetPcpusUsage(virDomainPtr domain, + unsigned long long *usages, + int *nr_usages, + unsigned int flags) +{ + int i; + int rv = -1; + remote_domain_get_pcpus_usage_args args; + remote_domain_get_pcpus_usage_ret ret; + struct private_data *priv = domain->conn->privateData; + + remoteDriverLock(priv); + + make_nonnull_domain(&args.dom, domain); + args.nr_usages = *nr_usages; + args.flags = flags; + + memset(&ret, 0, sizeof(ret)); + + if (call(domain->conn, priv, 0, REMOTE_PROC_DOMAIN_GET_PCPUS_USAGE, + (xdrproc_t) xdr_remote_domain_get_pcpus_usage_args, + (char *) &args, + (xdrproc_t) xdr_remote_domain_get_pcpus_usage_ret, + (char *) &ret) == -1) { + goto done; + } + + /* Handle the case when the caller does not know the number of parameters + * and is asking for the number of parameters supported + */ + if (*nr_usages == 0) { + *nr_usages = ret.nr_usages; + goto cleanup; + } + + for (i = 0; i < MIN(*nr_usages, ret.usages.usages_len); i++) { + usages[i] = ret.usages.usages_val[i]; + } + *nr_usages = ret.nr_usages; + + rv = 0; + +cleanup: + xdr_free ((xdrproc_t) xdr_remote_domain_get_pcpus_usage_ret, + (char *) &ret); +done: + remoteDriverUnlock(priv); + return rv; +} + /*----------------------------------------------------------------------*/ static virDrvOpenStatus ATTRIBUTE_NONNULL (1) @@ -4739,6 +4789,7 @@ static virDriver remote_driver = { .domainGetBlockIoTune = remoteDomainGetBlockIoTune, /* 0.9.8 */ .domainSetNumaParameters = remoteDomainSetNumaParameters, /* 0.9.9 */ .domainGetNumaParameters = remoteDomainGetNumaParameters, /* 0.9.9 */ + .domainGetPcpusUsage = remoteDomainGetPcpusUsage, /* 0.9.10 */ }; static virNetworkDriver network_driver = { diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index ca739ff..d38206d 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -131,6 +131,9 @@ const REMOTE_DOMAIN_BLOCK_IO_TUNE_PARAMETERS_MAX = 16; /* Upper limit on list of numa parameters. */ const REMOTE_DOMAIN_NUMA_PARAMETERS_MAX = 16; +/* Upper limit on list of physic cpu parameters. */ +const REMOTE_DOMAIN_PCPUS_USAGE_PARAMETERS_MAX = 16; + /* Upper limit on list of node cpu stats. */ const REMOTE_NODE_CPU_STATS_MAX = 16; @@ -1148,6 +1151,17 @@ struct remote_domain_get_block_io_tune_ret { int nparams; }; +struct remote_domain_get_pcpus_usage_args { + remote_nonnull_domain dom; + int nr_usages; + unsigned int flags; +}; + +struct remote_domain_get_pcpus_usage_ret { + uint64_t usages<REMOTE_DOMAIN_PCPUS_USAGE_PARAMETERS_MAX>; + int nr_usages; +}; + /* Network calls: */ struct remote_num_of_networks_ret { @@ -2653,7 +2667,8 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_SET_NUMA_PARAMETERS = 254, /* autogen autogen */ REMOTE_PROC_DOMAIN_GET_NUMA_PARAMETERS = 255, /* skipgen skipgen */ REMOTE_PROC_DOMAIN_SET_INTERFACE_PARAMETERS = 256, /* autogen autogen */ - REMOTE_PROC_DOMAIN_GET_INTERFACE_PARAMETERS = 257 /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_GET_INTERFACE_PARAMETERS = 257, /* skipgen skipgen */ + REMOTE_PROC_DOMAIN_GET_PCPUS_USAGE = 258 /* skipgen skipgen */ /* * Notice how the entries are grouped in sets of 10 ? diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index 2758315..4fd7b6d 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -1832,6 +1832,18 @@ struct remote_node_suspend_for_duration_args { uint64_t duration; u_int flags; }; +struct remote_domain_get_pcpus_usage_args { + remote_nonnull_domain dom; + int nr_usages; + u_int flags; +}; +struct remote_domain_get_pcpus_usage_ret { + struct { + u_int usages_len; + unsigned long long *usages_val; + } usages; + int nr_usages; +}; enum remote_procedure { REMOTE_PROC_OPEN = 1, REMOTE_PROC_CLOSE = 2, @@ -2090,4 +2102,5 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_GET_NUMA_PARAMETERS = 255, REMOTE_PROC_DOMAIN_SET_INTERFACE_PARAMETERS = 256, REMOTE_PROC_DOMAIN_GET_INTERFACE_PARAMETERS = 257, + REMOTE_PROC_DOMAIN_GET_PCPU_USAGE = 258, }; -- 1.7.4.4
The title for this should be 'implement', not 'mplement'.
+/* Upper limit on list of physic cpu parameters. */
'physical'
+const REMOTE_DOMAIN_PCPUS_USAGE_PARAMETERS_MAX = 16;
This seems a bit low. Is this limiting us to returning information about only 16 pCPUs? Meta-comment: libvirt needs a generator ... Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/
Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- src/qemu/qemu.conf | 5 ++- src/qemu/qemu_conf.c | 3 +- src/qemu/qemu_driver.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++ src/util/cgroup.c | 7 ++++ src/util/cgroup.h | 1 + 5 files changed, 87 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index 4ec5e6c..5f75b3e 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -158,18 +158,19 @@ # - 'memory' - use for memory tunables # - 'blkio' - use for block devices I/O tunables # - 'cpuset' - use for CPUs and memory nodes +# - 'cpuacct' - use for CPUs' account # # NB, even if configured here, they won't be used unless # the administrator has mounted cgroups, e.g.: # # mkdir /dev/cgroup -# mount -t cgroup -o devices,cpu,memory,blkio,cpuset none /dev/cgroup +# mount -t cgroup -o devices,cpu,memory,blkio,cpuset,cpuacct none /dev/cgroup # # They can be mounted anywhere, and different controllers # can be mounted in different locations. libvirt will detect # where they are located. # -# cgroup_controllers = [ "cpu", "devices", "memory", "blkio", "cpuset" ] +# cgroup_controllers = [ "cpu", "devices", "memory", "blkio", "cpuset", "cpuacct" ] # This is the basic set of devices allowed / required by # all virtual machines. diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index bc0a646..4775638 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -307,7 +307,8 @@ int qemudLoadDriverConfig(struct qemud_driver *driver, (1 << VIR_CGROUP_CONTROLLER_DEVICES) | (1 << VIR_CGROUP_CONTROLLER_MEMORY) | (1 << VIR_CGROUP_CONTROLLER_BLKIO) | - (1 << VIR_CGROUP_CONTROLLER_CPUSET); + (1 << VIR_CGROUP_CONTROLLER_CPUSET) | + (1 << VIR_CGROUP_CONTROLLER_CPUACCT); } for (i = 0 ; i < VIR_CGROUP_CONTROLLER_LAST ; i++) { if (driver->cgroupControllers & (1 << i)) { diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 82bab67..3d023eb 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -11194,6 +11194,79 @@ cleanup: return ret; } +static int +qemuGetPcpusUsage(virDomainPtr dom, + unsigned long long *usages, + int *nr_usages, + unsigned int flags) +{ + struct qemud_driver *driver = dom->conn->privateData; + virCgroupPtr group = NULL; + virDomainObjPtr vm = NULL; + char *pos, *raw; + unsigned long long val; + int nr_cpus = 0; + int ret = -1; + int rc; + bool isActive; + + virCheckFlags(0, -1); + + qemuDriverLock(driver); + + vm = virDomainFindByUUID(&driver->domains, dom->uuid); + + if (vm == NULL) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, + _("No such domain %s"), dom->uuid); + goto cleanup; + } + + isActive = virDomainObjIsActive(vm); + + if (!isActive) { + qemuReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("domain is not running")); + goto cleanup; + } + + if (!qemuCgroupControllerActive(driver, VIR_CGROUP_CONTROLLER_CPUACCT)) { + qemuReportError(VIR_ERR_OPERATION_INVALID, + "%s", _("cgroup CPUACCT controller is not mounted")); + goto cleanup; + } + + if (virCgroupForDomain(driver->cgroup, vm->def->name, &group, 0) != 0) { + qemuReportError(VIR_ERR_INTERNAL_ERROR, + _("cannot find cgroup for domain %s"), vm->def->name); + goto cleanup; + } + + rc = virCgroupGetCpuacctPcpusUsage(group, &raw); + if (rc != 0) { + virReportSystemError(-rc, "%s", _("unable to get cpu account")); + goto cleanup; + } + + pos = raw; + while (virStrToLong_ull(pos, &pos, 10, &val) >= 0) { + if (nr_cpus < *nr_usages) { + usages[nr_cpus] = val; + } + nr_cpus++; + } + + VIR_FREE(raw); + *nr_usages = nr_cpus; + ret = 0; + +cleanup: + virCgroupFree(&group); + if (vm) + virDomainObjUnlock(vm); + qemuDriverUnlock(driver); + return ret; +} static virDomainPtr qemuDomainAttach(virConnectPtr conn, unsigned int pid, @@ -11997,6 +12070,7 @@ static virDriver qemuDriver = { .domainGetNumaParameters = qemuDomainGetNumaParameters, /* 0.9.9 */ .domainGetInterfaceParameters = qemuDomainGetInterfaceParameters, /* 0.9.9 */ .domainSetInterfaceParameters = qemuDomainSetInterfaceParameters, /* 0.9.9 */ + .domainGetPcpusUsage = qemuGetPcpusUsage, /* 0.9.10 */ }; diff --git a/src/util/cgroup.c b/src/util/cgroup.c index 25f2691..114eeb5 100644 --- a/src/util/cgroup.c +++ b/src/util/cgroup.c @@ -1554,6 +1554,13 @@ int virCgroupGetCpuacctUsage(virCgroupPtr group, unsigned long long *usage) "cpuacct.usage", usage); } +int virCgroupGetCpuacctPcpusUsage(virCgroupPtr group, char **usage) +{ + return virCgroupGetValueStr(group, + VIR_CGROUP_CONTROLLER_CPUACCT, + "cpuacct.usage_percpu", usage); +} + int virCgroupSetFreezerState(virCgroupPtr group, const char *state) { return virCgroupSetValueStr(group, diff --git a/src/util/cgroup.h b/src/util/cgroup.h index 8d75735..f3c4b7d 100644 --- a/src/util/cgroup.h +++ b/src/util/cgroup.h @@ -115,6 +115,7 @@ int virCgroupSetCpuCfsQuota(virCgroupPtr group, long long cfs_quota); int virCgroupGetCpuCfsQuota(virCgroupPtr group, long long *cfs_quota); int virCgroupGetCpuacctUsage(virCgroupPtr group, unsigned long long *usage); +int virCgroupGetCpuacctPcpusUsage(virCgroupPtr group, char **usage); int virCgroupSetFreezerState(virCgroupPtr group, const char *state); int virCgroupGetFreezerState(virCgroupPtr group, char **state); -- 1.7.4.4
This command gets information about the physic CPUs. Example: # virsh pcpuinfo rhel6 CPU: 0 Curr VCPU: - Usage: 47.3 CPU: 1 Curr VCPU: 1 Usage: 46.8 CPU: 2 Curr VCPU: 0 Usage: 52.7 CPU: 3 Curr VCPU: - Usage: 44.1 Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- tools/virsh.c | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 5 +++ 2 files changed, 98 insertions(+), 0 deletions(-) diff --git a/tools/virsh.c b/tools/virsh.c index 0bc0519..f6fac24 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -4578,6 +4578,98 @@ cmdVcpuinfo(vshControl *ctl, const vshCmd *cmd) } /* + * "pcpuinfo" command + */ +static const vshCmdInfo info_pcpuinfo[] = { + {"help", N_("detailed domain pcpu information")}, + {"desc", N_("Returns basic information about the domain physic CPUs.")}, + {NULL, NULL} +}; + +static const vshCmdOptDef opts_pcpuinfo[] = { + {"domain", VSH_OT_DATA, VSH_OFLAG_REQ, N_("domain name, id or uuid")}, + {NULL, 0, 0, NULL} +}; + +static bool +cmdPcpuinfo(vshControl *ctl, const vshCmd *cmd) +{ + virDomainInfo info; + virDomainPtr dom; + virNodeInfo nodeinfo; + virVcpuInfoPtr cpuinfo; + unsigned char *cpumaps; + int ncpus, maxcpu; + size_t cpumaplen; + bool ret = true; + int n, m; + + if (!vshConnectionUsability(ctl, ctl->conn)) + return false; + + if (!(dom = vshCommandOptDomain(ctl, cmd, NULL))) + return false; + + if (virNodeGetInfo(ctl->conn, &nodeinfo) != 0) { + virDomainFree(dom); + return false; + } + + if (virDomainGetInfo(dom, &info) != 0) { + virDomainFree(dom); + return false; + } + + cpuinfo = vshMalloc(ctl, sizeof(virVcpuInfo)*info.nrVirtCpu); + maxcpu = VIR_NODEINFO_MAXCPUS(nodeinfo); + cpumaplen = VIR_CPU_MAPLEN(maxcpu); + cpumaps = vshMalloc(ctl, info.nrVirtCpu * cpumaplen); + + if ((ncpus = virDomainGetVcpus(dom, + cpuinfo, info.nrVirtCpu, + cpumaps, cpumaplen)) >= 0) { + unsigned long long *usages; + int nr_usages = maxcpu; + + if (VIR_ALLOC_N(usages, nr_usages) < 0) { + virReportOOMError(); + goto fail; + } + + if (virDomainGetPcpusUsage(dom, usages, &nr_usages, 0) < 0) { + VIR_FREE(usages); + goto fail; + } + + for (n = 0; n < MIN(maxcpu, nr_usages); n++) { + vshPrint(ctl, "%-15s %d\n", _("CPU:"), n); + for (m = 0; m < ncpus; m++) { + if (cpuinfo[m].cpu == n) { + vshPrint(ctl, "%-15s %d\n", _("Curr VCPU:"), m); + break; + } + } + if (m == ncpus) { + vshPrint(ctl, "%-15s %s\n", _("Curr VCPU:"), _("-")); + } + vshPrint(ctl, "%-15s %.1lf\n\n", _("Usage:"), + usages[n] / 1000000000.0); + } + VIR_FREE(usages); + goto cleanup; + } + +fail: + ret = false; + +cleanup: + VIR_FREE(cpumaps); + VIR_FREE(cpuinfo); + virDomainFree(dom); + return ret; +} + +/* * "vcpupin" command */ static const vshCmdInfo info_vcpupin[] = { @@ -15813,6 +15905,7 @@ static const vshCmdDef domManagementCmds[] = { {"migrate-getspeed", cmdMigrateGetMaxSpeed, opts_migrate_getspeed, info_migrate_getspeed, 0}, {"numatune", cmdNumatune, opts_numatune, info_numatune, 0}, + {"pcpuinfo", cmdPcpuinfo, opts_pcpuinfo, info_pcpuinfo, 0}, {"reboot", cmdReboot, opts_reboot, info_reboot, 0}, {"reset", cmdReset, opts_reset, info_reset, 0}, {"restore", cmdRestore, opts_restore, info_restore, 0}, diff --git a/tools/virsh.pod b/tools/virsh.pod index 138f886..5a8b2e6 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -1286,6 +1286,11 @@ Thus, this command always takes exactly zero or two flags. Returns basic information about the domain virtual CPUs, like the number of vCPUs, the running time, the affinity to physical processors. +=item B<pcpuinfo> I<domain-id> + +Returns information about the physic CPUs of the domain, like the usage of +CPUs, the current attached vCPUs. + =item B<vcpupin> I<domain-id> [I<vcpu>] [I<cpulist>] [[I<--live>] [I<--config>] | [I<--current>]] -- 1.7.4.4
A typo, "physic CPU" should be "physical CPU". On 2012-1-4 12:09, Lai Jiangshan wrote:
This command gets information about the physic CPUs. Example:
# virsh pcpuinfo rhel6 CPU: 0 Curr VCPU: - Usage: 47.3
CPU: 1 Curr VCPU: 1 Usage: 46.8
CPU: 2 Curr VCPU: 0 Usage: 52.7
CPU: 3 Curr VCPU: - Usage: 44.1
Signed-off-by: Lai Jiangshan<laijs@cn.fujitsu.com> --- tools/virsh.c | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 5 +++ 2 files changed, 98 insertions(+), 0 deletions(-)
diff --git a/tools/virsh.c b/tools/virsh.c index 0bc0519..f6fac24 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -4578,6 +4578,98 @@ cmdVcpuinfo(vshControl *ctl, const vshCmd *cmd) }
/* + * "pcpuinfo" command + */ +static const vshCmdInfo info_pcpuinfo[] = { + {"help", N_("detailed domain pcpu information")}, + {"desc", N_("Returns basic information about the domain physic CPUs.")}, + {NULL, NULL} +}; + +static const vshCmdOptDef opts_pcpuinfo[] = { + {"domain", VSH_OT_DATA, VSH_OFLAG_REQ, N_("domain name, id or uuid")}, + {NULL, 0, 0, NULL} +}; + +static bool +cmdPcpuinfo(vshControl *ctl, const vshCmd *cmd) +{ + virDomainInfo info; + virDomainPtr dom; + virNodeInfo nodeinfo; + virVcpuInfoPtr cpuinfo; + unsigned char *cpumaps; + int ncpus, maxcpu; + size_t cpumaplen; + bool ret = true; + int n, m; + + if (!vshConnectionUsability(ctl, ctl->conn)) + return false; + + if (!(dom = vshCommandOptDomain(ctl, cmd, NULL))) + return false; + + if (virNodeGetInfo(ctl->conn,&nodeinfo) != 0) { + virDomainFree(dom); + return false; + } + + if (virDomainGetInfo(dom,&info) != 0) { + virDomainFree(dom); + return false; + } + + cpuinfo = vshMalloc(ctl, sizeof(virVcpuInfo)*info.nrVirtCpu); + maxcpu = VIR_NODEINFO_MAXCPUS(nodeinfo); + cpumaplen = VIR_CPU_MAPLEN(maxcpu); + cpumaps = vshMalloc(ctl, info.nrVirtCpu * cpumaplen); + + if ((ncpus = virDomainGetVcpus(dom, + cpuinfo, info.nrVirtCpu, + cpumaps, cpumaplen))>= 0) { + unsigned long long *usages; + int nr_usages = maxcpu; + + if (VIR_ALLOC_N(usages, nr_usages)< 0) { + virReportOOMError(); + goto fail; + } + + if (virDomainGetPcpusUsage(dom, usages,&nr_usages, 0)< 0) { + VIR_FREE(usages); + goto fail; + } + + for (n = 0; n< MIN(maxcpu, nr_usages); n++) { + vshPrint(ctl, "%-15s %d\n", _("CPU:"), n); + for (m = 0; m< ncpus; m++) { + if (cpuinfo[m].cpu == n) { + vshPrint(ctl, "%-15s %d\n", _("Curr VCPU:"), m); + break; + } + } + if (m == ncpus) { + vshPrint(ctl, "%-15s %s\n", _("Curr VCPU:"), _("-")); + } + vshPrint(ctl, "%-15s %.1lf\n\n", _("Usage:"), + usages[n] / 1000000000.0); + } + VIR_FREE(usages); + goto cleanup; + } + +fail: + ret = false; + +cleanup: + VIR_FREE(cpumaps); + VIR_FREE(cpuinfo); + virDomainFree(dom); + return ret; +} + +/* * "vcpupin" command */ static const vshCmdInfo info_vcpupin[] = { @@ -15813,6 +15905,7 @@ static const vshCmdDef domManagementCmds[] = { {"migrate-getspeed", cmdMigrateGetMaxSpeed, opts_migrate_getspeed, info_migrate_getspeed, 0}, {"numatune", cmdNumatune, opts_numatune, info_numatune, 0}, + {"pcpuinfo", cmdPcpuinfo, opts_pcpuinfo, info_pcpuinfo, 0}, {"reboot", cmdReboot, opts_reboot, info_reboot, 0}, {"reset", cmdReset, opts_reset, info_reset, 0}, {"restore", cmdRestore, opts_restore, info_restore, 0}, diff --git a/tools/virsh.pod b/tools/virsh.pod index 138f886..5a8b2e6 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -1286,6 +1286,11 @@ Thus, this command always takes exactly zero or two flags. Returns basic information about the domain virtual CPUs, like the number of vCPUs, the running time, the affinity to physical processors.
+=item B<pcpuinfo> I<domain-id> + +Returns information about the physic CPUs of the domain, like the usage of +CPUs, the current attached vCPUs. + =item B<vcpupin> I<domain-id> [I<vcpu>] [I<cpulist>] [[I<--live>] [I<--config>] | [I<--current>]]
-- Shu Ming<shuming@linux.vnet.ibm.com> IBM China Systems and Technology Laboratory
Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- libvirt/libvirt.ml | 1 + libvirt/libvirt.mli | 4 ++++ libvirt/libvirt_c_oneoffs.c | 25 +++++++++++++++++++++++++ 3 files changed, 30 insertions(+), 0 deletions(-) diff --git a/libvirt/libvirt.ml b/libvirt/libvirt.ml index fc29264..a8720a9 100644 --- a/libvirt/libvirt.ml +++ b/libvirt/libvirt.ml @@ -419,6 +419,7 @@ struct external set_vcpus : [>`W] t -> int -> unit = "ocaml_libvirt_domain_set_vcpus" external pin_vcpu : [>`W] t -> int -> string -> unit = "ocaml_libvirt_domain_pin_vcpu" external get_vcpus : [>`R] t -> int -> int -> int * vcpu_info array * string = "ocaml_libvirt_domain_get_vcpus" + external get_pcpu_usages : [>`R] t -> int -> int64 array = "ocaml_libvirt_domain_get_pcpu_usages" external get_max_vcpus : [>`R] t -> int = "ocaml_libvirt_domain_get_max_vcpus" external attach_device : [>`W] t -> xml -> unit = "ocaml_libvirt_domain_attach_device" external detach_device : [>`W] t -> xml -> unit = "ocaml_libvirt_domain_detach_device" diff --git a/libvirt/libvirt.mli b/libvirt/libvirt.mli index 7bda889..63bf830 100644 --- a/libvirt/libvirt.mli +++ b/libvirt/libvirt.mli @@ -586,6 +586,10 @@ sig for a domain. See the libvirt documentation for details of the array and bitmap returned from this function. *) + val get_pcpu_usages : [>`R] t -> int -> int64 array + (** [get_pcpu_usages dom nr_pcpu] returns the physic CPU usages + for a domain. See the libvirt documentation for details. + *) val get_max_vcpus : [>`R] t -> int (** Returns the maximum number of vCPUs supported for this domain. *) val attach_device : [>`W] t -> xml -> unit diff --git a/libvirt/libvirt_c_oneoffs.c b/libvirt/libvirt_c_oneoffs.c index d87dd21..68d5ecc 100644 --- a/libvirt/libvirt_c_oneoffs.c +++ b/libvirt/libvirt_c_oneoffs.c @@ -604,6 +604,31 @@ ocaml_libvirt_domain_get_vcpus (value domv, value maxinfov, value maplenv) CAMLreturn (rv); } +CAMLprim value +ocaml_libvirt_domain_get_pcpu_usages (value domv, value nr_pcpusv) +{ + CAMLparam2 (domv, nr_pcpusv); + CAMLlocal1 (usagev); + virDomainPtr dom = Domain_val (domv); + virConnectPtr conn = Connect_domv (domv); + int nr_pcpus = Int_val (nr_pcpusv); + unsigned long long pcpu_usages[nr_pcpus]; + int r, i; + + memset (pcpu_usages, 0, sizeof(pcpu_usages[0]) * nr_pcpus); + + NONBLOCKING (r = virDomainGetPcpusUsage (dom, pcpu_usages, &nr_pcpus, 0)); + CHECK_ERROR (r == -1, conn, "virDomainGetPcpusUsage"); + + /* Copy the pcpu_usages. */ + usagev = caml_alloc (nr_pcpus, 0); + for (i = 0; i < nr_pcpus; ++i) { + Store_field (usagev, i, caml_copy_int64 ((int64_t)pcpu_usages[i])); + } + + CAMLreturn (usagev); +} + #ifdef HAVE_WEAK_SYMBOLS #ifdef HAVE_VIRDOMAINMIGRATE extern virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, -- 1.7.4.4
Old "virt-top -1" is not correct, its output is generated by guess: use average usage for pinned physic CPUs. example(old "virt-top -1"): PHYCPU %CPU rhel6 Windows 0 0.6 0.1= 0.5= 1 0.6 0.1= 0.5=# 2 0.6 0.1= 0.5= 3 0.6 0.1=# 0.5= The output almost makes no sense(all the value are just average, not real). This is new implement, it use cpuacct cgroup to gain *real* physic usages via cpuacct cgroup by virDomainGetPcpusUsage() API. new result: PHYCPU %CPU rhel6 Windows 0 3.3 2.9 0.3 1 1.7 1.1 0.6 2 3.5 1.8 1.6 3 3.4 1.6 1.8 PHYCPU %CPU rhel6 Windows 0 1.2 0.8 0.4 1 1.6 1.6 2 2.2 1.7 0.5 3 3.0 2.5 0.5 Note: average flag(=) is dropped, there is not average value in here. Note: running flag(#) is dropped, because if the value is not empty, it means the guest was once running in the physic CPU in this period between updates. Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com> --- virt-top/virt_top.ml | 75 +++++++++++++++++-------------------------------- 1 files changed, 26 insertions(+), 49 deletions(-) diff --git a/virt-top/virt_top.ml b/virt-top/virt_top.ml index ef5ac67..2556b77 100644 --- a/virt-top/virt_top.ml +++ b/virt-top/virt_top.ml @@ -446,14 +446,14 @@ let collect, clear_pcpu_display_data = let last_info = Hashtbl.create 13 in let last_time = ref (Unix.gettimeofday ()) in - (* Save vcpuinfo structures across redraws too (only for pCPU display). *) - let last_vcpu_info = Hashtbl.create 13 in + (* Save pcpu_usages structures across redraws too (only for pCPU display). *) + let last_pcpu_usages = Hashtbl.create 13 in let clear_pcpu_display_data () = - (* Clear out vcpu_info used by PCPUDisplay display_mode + (* Clear out pcpu_info used by PCPUDisplay display_mode * when we switch back to TaskDisplay mode. *) - Hashtbl.clear last_vcpu_info + Hashtbl.clear last_pcpu_usages in let collect (conn, _, _, _, _, node_info, _, _) = @@ -652,22 +652,23 @@ let collect, clear_pcpu_display_data = (try let domid = rd.rd_domid in let maplen = C.cpumaplen nr_pcpus in + let pcpu_usages = D.get_pcpu_usages rd.rd_dom nr_pcpus in let maxinfo = rd.rd_info.D.nr_virt_cpu in let nr_vcpus, vcpu_infos, cpumaps = D.get_vcpus rd.rd_dom maxinfo maplen in - (* Got previous vcpu_infos for this domain? *) - let prev_vcpu_infos = - try Some (Hashtbl.find last_vcpu_info domid) + (* Got previous pcpu_usages for this domain? *) + let prev_pcpu_usages = + try Some (Hashtbl.find last_pcpu_usages domid) with Not_found -> None in - (* Update last_vcpu_info. *) - Hashtbl.replace last_vcpu_info domid vcpu_infos; - - (match prev_vcpu_infos with - | Some prev_vcpu_infos - when Array.length prev_vcpu_infos = Array.length vcpu_infos -> - Some (domid, name, nr_vcpus, vcpu_infos, prev_vcpu_infos, - cpumaps, maplen) + (* Update last_pcpu_usages. *) + Hashtbl.replace last_pcpu_usages domid pcpu_usages; + + (match prev_pcpu_usages with + | Some prev_pcpu_usages + when Array.length prev_pcpu_usages = Array.length pcpu_usages -> + Some (domid, name, nr_vcpus, vcpu_infos, pcpu_usages, + prev_pcpu_usages, cpumaps, maplen) | _ -> None (* ignore missing / unequal length prev_vcpu_infos *) ); with @@ -680,37 +681,15 @@ let collect, clear_pcpu_display_data = (* Rearrange the data into a matrix. Major axis (down) is * pCPUs. Minor axis (right) is domains. At each node we store: * cpu_time (on this pCPU only, nanosecs), - * average? (if set, then cpu_time is an average because the - * vCPU is pinned to more than one pCPU) - * running? (if set, we were instantaneously running on this pCPU) *) - let empty_node = (0L, false, false) in - let pcpus = Array.make_matrix nr_pcpus nr_doms empty_node in + let pcpus = Array.make_matrix nr_pcpus nr_doms 0L in List.iteri ( - fun di (domid, name, nr_vcpus, vcpu_infos, prev_vcpu_infos, - cpumaps, maplen) -> + fun di (domid, name, nr_vcpus, vcpu_infos, pcpu_usages, + prev_pcpu_usages, cpumaps, maplen) -> (* Which pCPUs can this dom run on? *) - for v = 0 to nr_vcpus-1 do - let pcpu = vcpu_infos.(v).D.cpu in (* instantaneous pCPU *) - let nr_poss_pcpus = ref 0 in (* how many pcpus can it run on? *) - for p = 0 to nr_pcpus-1 do - (* vcpu v can reside on pcpu p *) - if C.cpu_usable cpumaps maplen v p then - incr nr_poss_pcpus - done; - let nr_poss_pcpus = Int64.of_int !nr_poss_pcpus in - for p = 0 to nr_pcpus-1 do - (* vcpu v can reside on pcpu p *) - if C.cpu_usable cpumaps maplen v p then - let vcpu_time_on_pcpu = - vcpu_infos.(v).D.vcpu_time - -^ prev_vcpu_infos.(v).D.vcpu_time in - let vcpu_time_on_pcpu = - vcpu_time_on_pcpu /^ nr_poss_pcpus in - pcpus.(p).(di) <- - (vcpu_time_on_pcpu, nr_poss_pcpus > 1L, p = pcpu) - done + for p = 0 to Array.length pcpu_usages - 1 do + pcpus.(p).(di) <- (pcpu_usages.(p) -^ prev_pcpu_usages.(p)) done ) doms; @@ -719,7 +698,7 @@ let collect, clear_pcpu_display_data = fun row -> let cpu_time = ref 0L in for di = 0 to Array.length row-1 do - let t, _, _ = row.(di) in + let t = row.(di) in cpu_time := !cpu_time +^ t done; Int64.to_float !cpu_time @@ -938,7 +917,7 @@ let redraw = let dom_names = String.concat "" ( List.map ( - fun (_, name, _, _, _, _, _) -> + fun (_, name, _, _, _, _, _, _) -> let len = String.length name in let width = max (len+1) 7 in pad width name @@ -957,8 +936,8 @@ let redraw = addch ' '; List.iteri ( - fun di (domid, name, _, _, _, _, _) -> - let t, is_average, is_running = pcpus.(p).(di) in + fun di (domid, name, _, _, _, _, _, _) -> + let t = pcpus.(p).(di) in let len = String.length name in let width = max (len+1) 7 in let str = @@ -966,9 +945,7 @@ let redraw = else ( let t = Int64.to_float t in let percent = 100. *. t /. total_cpu_per_pcpu in - sprintf "%s%c%c " (Show.percent percent) - (if is_average then '=' else ' ') - (if is_running then '#' else ' ') + sprintf "%s " (Show.percent percent) ) in addstr (pad width str); () -- 1.7.4.4
On Wed, Jan 04, 2012 at 12:09:23PM +0800, Lai Jiangshan wrote:
Old "virt-top -1" is not correct, its output is generated by guess: use average usage for pinned physic CPUs.
The virt-top and ocaml-libvirt patches look fine. I'll wait until others review the libvirt patches before I can push it upstream. Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top
On 01/03/2012 09:09 PM, Lai Jiangshan wrote:
"virt-top -1" can call virDomainGetPcpusUsage() periodically and get the CPU activities per CPU. (See the last patch in this series).
virsh is also added a pcpuinfo command which calls virDomainGetPcpusUsage(), it gets information about the physic CPUs, such as the usage of CPUs, the current attached vCPUs.
I think this is invasive enough that we should defer it until after the 0.9.9 release, but it's definitely on my list of things to review and not forgotten. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 01/05/2012 12:55 AM, Eric Blake wrote:
On 01/03/2012 09:09 PM, Lai Jiangshan wrote:
"virt-top -1" can call virDomainGetPcpusUsage() periodically and get the CPU activities per CPU. (See the last patch in this series).
virsh is also added a pcpuinfo command which calls virDomainGetPcpusUsage(), it gets information about the physic CPUs, such as the usage of CPUs, the current attached vCPUs.
I think this is invasive enough that we should defer it until after the 0.9.9 release, but it's definitely on my list of things to review and not forgotten.
Ping. I'm glad to see your comments. Thanks, Lai
participants (4)
-
Eric Blake -
Lai Jiangshan -
Richard W.M. Jones -
Shu Ming