[libvirt] [PATCH v6 0/5] qemu: expand domain memory statistics
From: Derbyshev Dmitry <dderbyshev@virtuozzo.com> QEMU reports timestamp and available along with other memory statistics. This information was not saved into domain statistics. Also, to collect all balloon statistics for all guests it was necessary to make several libvirt requests (one per VE). Last patch allows doing this via qemuConnectGetAllDomainStats in one request. Changes since v1: * Enum numeration fixed * Macro getting "usage" field fixed Changes since v2: * Previous patches were on wrong branch * qemu's stat name was "stat-available-memory" Changes since v3: * domstats patch added Changes since v4: * Formatted and rephrased commit messages * Fixed libvirt crash, caused by simultaneous incorrect QUERY job execution Changes since v5: * Updated virsh.pod * Splitted patch about domstats into 2 patches * Do not report balloon.current as balloon.actual Derbyshev Dmitry (5): virsh: Add balloon stats description to .pod qemu: expand domain memory statistics with 'usable' qemu: expand domain memory statistics with 'last-update' timestamp qemu: split qemuDomainMemoryStats into internal and external functions qemu: return balloon statistics when all domain statistics reported include/libvirt/libvirt-domain.h | 11 ++++- src/libvirt-domain.c | 5 +++ src/qemu/qemu_driver.c | 94 ++++++++++++++++++++++++++++++---------- src/qemu/qemu_monitor_json.c | 24 ++++++---- tools/virsh-domain-monitor.c | 4 ++ tools/virsh.pod | 29 ++++++++++++- 6 files changed, 133 insertions(+), 34 deletions(-) -- 1.9.5.msysgit.0
From: Derbyshev Dmitry <dderbyshev@virtuozzo.com> Description for existing balloon stats was missing for dommemstat. Signed-off-by: Derbyshev Dmitry <dderbyshev@virtuozzo.com> --- tools/virsh.pod | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tools/virsh.pod b/tools/virsh.pod index 601cb44..3288add 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -800,7 +800,19 @@ on hypervisor. Get memory stats for a running domain. -Depending on the hypervisor a variety of statistics can be returned +Availability of these fields depends on hypervisor. Unsupported fields are +missing from the output. Other fields may appear if communicating with a newer +version of libvirtd. + +B<Explanation of fields>: + swap_in - The amount of data read from swap space (in kB) + swap_out - The amount of memory written out to swap space (in kB) + major_fault - The number of page faults then disk IO was required + minor_fault - The number of other page faults + unused - The amount of memory left unused by the system (in kB) + available - The amount of usable memory as seen by the domain (in kB) + actual - Current balloon value (in KB) + rss - Resident Set Size of the running domain's process (in kB) For QEMU/KVM with a memory balloon, setting the optional I<--period> to a value larger than 0 in seconds will allow the balloon driver to return -- 1.9.5.msysgit.0
From: Derbyshev Dmitry <dderbyshev@virtuozzo.com> Currently 'memtotal' in virtio drivers and qemu corresponds to 'available' in libvirt. Because of that we introduce libvirt 'usable' parameter, which maps to 'stat-available-memory' balloon statistics. As balloon statistics isn't reported in hmp, so no modification is made in qemu_monitor_text.c. Signed-off-by: Derbyshev Dmitry <dderbyshev@virtuozzo.com> --- include/libvirt/libvirt-domain.h | 8 +++++++- src/libvirt-domain.c | 3 +++ src/qemu/qemu_monitor_json.c | 4 ++++ tools/virsh-domain-monitor.c | 2 ++ tools/virsh.pod | 2 ++ 5 files changed, 18 insertions(+), 1 deletion(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 7ea93aa..3fb482b 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -604,10 +604,16 @@ typedef enum { VIR_DOMAIN_MEMORY_STAT_RSS = 7, /* + * How big the balloon can be inflated without pushing the guest system + * to swap, corresponds to 'Available' in /proc/meminfo + */ + VIR_DOMAIN_MEMORY_STAT_USABLE = 8, + + /* * The number of statistics supported by this version of the interface. * To add new statistics, add them to the enum and increase this value. */ - VIR_DOMAIN_MEMORY_STAT_NR = 8, + VIR_DOMAIN_MEMORY_STAT_NR = 9, # ifdef VIR_ENUM_SENTINELS VIR_DOMAIN_MEMORY_STAT_LAST = VIR_DOMAIN_MEMORY_STAT_NR diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 4e71a94..1467030 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -5986,6 +5986,9 @@ virDomainGetInterfaceParameters(virDomainPtr domain, * The amount of memory which is not being used for any purpose (in kb). * VIR_DOMAIN_MEMORY_STAT_AVAILABLE: * The total amount of memory available to the domain's OS (in kb). + * VIR_DOMAIN_MEMORY_STAT_USABLE: + * How big the balloon can be inflated without pushing the guest system + * to swap, corresponds to 'Available' in /proc/meminfo * VIR_DOMAIN_MEMORY_STAT_ACTUAL_BALLOON: * Current balloon value (in kb). * diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index bb426dc..c83e0cc 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -1719,7 +1719,11 @@ int qemuMonitorJSONGetMemoryStats(qemuMonitorPtr mon, VIR_DOMAIN_MEMORY_STAT_UNUSED, 1024); GET_BALLOON_STATS("stat-total-memory", VIR_DOMAIN_MEMORY_STAT_AVAILABLE, 1024); + GET_BALLOON_STATS("stat-available-memory", + VIR_DOMAIN_MEMORY_STAT_USABLE, 1024); + ret = got; + cleanup: virJSONValueFree(cmd); virJSONValueFree(reply); diff --git a/tools/virsh-domain-monitor.c b/tools/virsh-domain-monitor.c index 0a93949..1921ff5 100644 --- a/tools/virsh-domain-monitor.c +++ b/tools/virsh-domain-monitor.c @@ -369,6 +369,8 @@ cmdDomMemStat(vshControl *ctl, const vshCmd *cmd) vshPrint(ctl, "unused %llu\n", stats[i].val); if (stats[i].tag == VIR_DOMAIN_MEMORY_STAT_AVAILABLE) vshPrint(ctl, "available %llu\n", stats[i].val); + if (stats[i].tag == VIR_DOMAIN_MEMORY_STAT_USABLE) + vshPrint(ctl, "usable %llu\n", stats[i].val); if (stats[i].tag == VIR_DOMAIN_MEMORY_STAT_ACTUAL_BALLOON) vshPrint(ctl, "actual %llu\n", stats[i].val); if (stats[i].tag == VIR_DOMAIN_MEMORY_STAT_RSS) diff --git a/tools/virsh.pod b/tools/virsh.pod index 3288add..2dee87a 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -813,6 +813,8 @@ B<Explanation of fields>: available - The amount of usable memory as seen by the domain (in kB) actual - Current balloon value (in KB) rss - Resident Set Size of the running domain's process (in kB) + usable - The amount of memory which can be reclaimed by balloon +without causing host swapping (in KB) For QEMU/KVM with a memory balloon, setting the optional I<--period> to a value larger than 0 in seconds will allow the balloon driver to return -- 1.9.5.msysgit.0
From: Derbyshev Dmitry <dderbyshev@virtuozzo.com> QEMU reports timestamp along with other memory statistics, but this information is not reported by libvirt statistics API. It could be useful to determine if the data reported is fresh or not. Signed-off-by: Derbyshev Dmitry <dderbyshev@virtuozzo.com> --- include/libvirt/libvirt-domain.h | 5 ++++- src/libvirt-domain.c | 2 ++ src/qemu/qemu_monitor_json.c | 22 ++++++++++++---------- tools/virsh-domain-monitor.c | 2 ++ tools/virsh.pod | 1 + 5 files changed, 21 insertions(+), 11 deletions(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 3fb482b..01a2649 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -609,11 +609,14 @@ typedef enum { */ VIR_DOMAIN_MEMORY_STAT_USABLE = 8, + /* Timestamp of the last update of statistics, in seconds. */ + VIR_DOMAIN_MEMORY_STAT_LAST_UPDATE = 9, + /* * The number of statistics supported by this version of the interface. * To add new statistics, add them to the enum and increase this value. */ - VIR_DOMAIN_MEMORY_STAT_NR = 9, + VIR_DOMAIN_MEMORY_STAT_NR = 10, # ifdef VIR_ENUM_SENTINELS VIR_DOMAIN_MEMORY_STAT_LAST = VIR_DOMAIN_MEMORY_STAT_NR diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 1467030..413f8af 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -5991,6 +5991,8 @@ virDomainGetInterfaceParameters(virDomainPtr domain, * to swap, corresponds to 'Available' in /proc/meminfo * VIR_DOMAIN_MEMORY_STAT_ACTUAL_BALLOON: * Current balloon value (in kb). + * VIR_DOMAIN_MEMORY_STAT_LAST_UPDATE + * Timestamp of the last statistic * * Returns: The number of stats provided or -1 in case of failure. */ diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index c83e0cc..6b955f8 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -1633,10 +1633,10 @@ qemuMonitorJSONGetBalloonInfo(qemuMonitorPtr mon, * rates and/or whether data has been collected since a previous cycle. * It's currently unused. */ -#define GET_BALLOON_STATS(FIELD, TAG, DIVISOR) \ - if (virJSONValueObjectHasKey(statsdata, FIELD) && \ +#define GET_BALLOON_STATS(OBJECT, FIELD, TAG, DIVISOR) \ + if (virJSONValueObjectHasKey(OBJECT, FIELD) && \ (got < nr_stats)) { \ - if (virJSONValueObjectGetNumberUlong(statsdata, FIELD, &mem) < 0) { \ + if (virJSONValueObjectGetNumberUlong(OBJECT, FIELD, &mem) < 0) { \ VIR_DEBUG("Failed to get '%s' value", FIELD); \ } else { \ /* Not being collected? No point in providing bad data */ \ @@ -1707,20 +1707,22 @@ int qemuMonitorJSONGetMemoryStats(qemuMonitorPtr mon, goto cleanup; } - GET_BALLOON_STATS("stat-swap-in", + GET_BALLOON_STATS(statsdata, "stat-swap-in", VIR_DOMAIN_MEMORY_STAT_SWAP_IN, 1024); - GET_BALLOON_STATS("stat-swap-out", + GET_BALLOON_STATS(statsdata, "stat-swap-out", VIR_DOMAIN_MEMORY_STAT_SWAP_OUT, 1024); - GET_BALLOON_STATS("stat-major-faults", + GET_BALLOON_STATS(statsdata, "stat-major-faults", VIR_DOMAIN_MEMORY_STAT_MAJOR_FAULT, 1); - GET_BALLOON_STATS("stat-minor-faults", + GET_BALLOON_STATS(statsdata, "stat-minor-faults", VIR_DOMAIN_MEMORY_STAT_MINOR_FAULT, 1); - GET_BALLOON_STATS("stat-free-memory", + GET_BALLOON_STATS(statsdata, "stat-free-memory", VIR_DOMAIN_MEMORY_STAT_UNUSED, 1024); - GET_BALLOON_STATS("stat-total-memory", + GET_BALLOON_STATS(statsdata, "stat-total-memory", VIR_DOMAIN_MEMORY_STAT_AVAILABLE, 1024); - GET_BALLOON_STATS("stat-available-memory", + GET_BALLOON_STATS(statsdata, "stat-available-memory", VIR_DOMAIN_MEMORY_STAT_USABLE, 1024); + GET_BALLOON_STATS(data, "last-update", + VIR_DOMAIN_MEMORY_STAT_LAST_UPDATE, 1); ret = got; diff --git a/tools/virsh-domain-monitor.c b/tools/virsh-domain-monitor.c index 1921ff5..b4ccbf6 100644 --- a/tools/virsh-domain-monitor.c +++ b/tools/virsh-domain-monitor.c @@ -375,6 +375,8 @@ cmdDomMemStat(vshControl *ctl, const vshCmd *cmd) vshPrint(ctl, "actual %llu\n", stats[i].val); if (stats[i].tag == VIR_DOMAIN_MEMORY_STAT_RSS) vshPrint(ctl, "rss %llu\n", stats[i].val); + if (stats[i].tag == VIR_DOMAIN_MEMORY_STAT_LAST_UPDATE) + vshPrint(ctl, "last_update %llu\n", stats[i].val); } ret = true; diff --git a/tools/virsh.pod b/tools/virsh.pod index 2dee87a..881ab76 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -815,6 +815,7 @@ B<Explanation of fields>: rss - Resident Set Size of the running domain's process (in kB) usable - The amount of memory which can be reclaimed by balloon without causing host swapping (in KB) + last-update - Timestamp of the last update of statistics (in seconds) For QEMU/KVM with a memory balloon, setting the optional I<--period> to a value larger than 0 in seconds will allow the balloon driver to return -- 1.9.5.msysgit.0
From: Derbyshev Dmitry <dderbyshev@virtuozzo.com> Is necessary to call it from other contexts, such as qemuDomainGetStatsBalloon. Signed-off-by: Derbyshev Dmitry <dderbyshev@virtuozzo.com> --- src/qemu/qemu_driver.c | 55 ++++++++++++++++++++++++++++++++------------------ 1 file changed, 35 insertions(+), 20 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 2089359..0b76177 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -10957,32 +10957,22 @@ qemuDomainGetInterfaceParameters(virDomainPtr dom, return ret; } +/* This functions assumes that job QEMU_JOB_QUERY is started by a caller */ + static int -qemuDomainMemoryStats(virDomainPtr dom, - virDomainMemoryStatPtr stats, - unsigned int nr_stats, - unsigned int flags) +qemuDomainMemoryStatsInternal(virQEMUDriverPtr driver, + virDomainObjPtr vm, + virDomainMemoryStatPtr stats, + unsigned int nr_stats) + { - virQEMUDriverPtr driver = dom->conn->privateData; - virDomainObjPtr vm; int ret = -1; long rss; - virCheckFlags(0, -1); - - if (!(vm = qemuDomObjFromDomain(dom))) - goto cleanup; - - if (virDomainMemoryStatsEnsureACL(dom->conn, vm->def) < 0) - goto cleanup; - - if (qemuDomainObjBeginJob(driver, vm, QEMU_JOB_QUERY) < 0) - goto cleanup; - if (!virDomainObjIsActive(vm)) { virReportError(VIR_ERR_OPERATION_INVALID, "%s", _("domain is not running")); - goto endjob; + return -1; } if (vm->def->memballoon && @@ -10993,7 +10983,7 @@ qemuDomainMemoryStats(virDomainPtr dom, ret = -1; if (ret < 0 || ret >= nr_stats) - goto endjob; + return ret; } else { ret = 0; } @@ -11007,7 +10997,32 @@ qemuDomainMemoryStats(virDomainPtr dom, ret++; } - endjob: + return ret; +} + +static int +qemuDomainMemoryStats(virDomainPtr dom, + virDomainMemoryStatPtr stats, + unsigned int nr_stats, + unsigned int flags) +{ + virQEMUDriverPtr driver = dom->conn->privateData; + virDomainObjPtr vm; + int ret = -1; + + virCheckFlags(0, -1); + + if (!(vm = qemuDomObjFromDomain(dom))) + goto cleanup; + + if (virDomainMemoryStatsEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (qemuDomainObjBeginJob(driver, vm, QEMU_JOB_QUERY) < 0) + goto cleanup; + + ret = qemuDomainMemoryStatsInternal(driver, vm, stats, nr_stats); + qemuDomainObjEndJob(driver, vm); cleanup: -- 1.9.5.msysgit.0
From: Derbyshev Dmitry <dderbyshev@virtuozzo.com> To collect all balloon statistics for all guests it was necessary to make several libvirt requests. Now it's possible to get all balloon statiscs via single connectGetAllDomainStats call. Signed-off-by: Derbyshev Dmitry <dderbyshev@virtuozzo.com> --- src/qemu/qemu_driver.c | 39 +++++++++++++++++++++++++++++++++++++-- tools/virsh.pod | 12 +++++++++++- 2 files changed, 48 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 0b76177..96343d8 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -18515,15 +18515,29 @@ qemuDomainGetStatsCpu(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, return 0; } + +#define STORE_MEM_RECORD(TAG, NAME) { \ + if (stats[i].tag == VIR_DOMAIN_MEMORY_STAT_ ##TAG) \ + if (virTypedParamsAddULLong(&record->params, \ + &record->nparams, \ + maxparams, \ + "balloon." NAME, \ + stats[i].val) < 0) \ + return -1; \ +} + static int -qemuDomainGetStatsBalloon(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, +qemuDomainGetStatsBalloon(virQEMUDriverPtr driver, virDomainObjPtr dom, virDomainStatsRecordPtr record, int *maxparams, - unsigned int privflags ATTRIBUTE_UNUSED) + unsigned int privflags) { qemuDomainObjPrivatePtr priv = dom->privateData; + virDomainMemoryStatStruct stats[VIR_DOMAIN_MEMORY_STAT_NR]; + int nr_stats; unsigned long long cur_balloon = 0; + size_t i; int err = 0; if (!virDomainDefHasMemballoon(dom->def)) { @@ -18548,8 +18562,29 @@ qemuDomainGetStatsBalloon(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, virDomainDefGetMemoryTotal(dom->def)) < 0) return -1; + if (err || !HAVE_JOB(privflags)) + return 0; + + nr_stats = qemuDomainMemoryStatsInternal(driver, dom, stats, + VIR_DOMAIN_MEMORY_STAT_NR); + if (nr_stats < 0) + return 0; + + for (i = 0; i < nr_stats; i++) { + STORE_MEM_RECORD(SWAP_IN, "swap_in") + STORE_MEM_RECORD(SWAP_OUT, "swap_out") + STORE_MEM_RECORD(MAJOR_FAULT, "major_fault") + STORE_MEM_RECORD(MINOR_FAULT, "minor_fault") + STORE_MEM_RECORD(UNUSED, "unused") + STORE_MEM_RECORD(AVAILABLE, "available") + STORE_MEM_RECORD(RSS, "rss") + STORE_MEM_RECORD(LAST_UPDATE, "last-update") + STORE_MEM_RECORD(USABLE, "usable") + } + return 0; } +#undef STORE_MEM_RECORD static int diff --git a/tools/virsh.pod b/tools/virsh.pod index 881ab76..1d33e6f 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -893,7 +893,17 @@ I<--cpu-total> returns: I<--balloon> returns: "balloon.current" - the memory in kiB currently used, -"balloon.maximum" - the maximum memory in kiB allowed +"balloon.maximum" - the maximum memory in kiB allowed, +"balloon.swap_in" - the amount of data read from swap space (in kB), +"balloon.swap_out" - the amount of memory written out to swap space (in kB), +"balloon.major_fault" - the number of page faults then disk IO was required, +"balloon.minor_fault" - the number of other page faults, +"balloon.unused" - the amount of memory left unused by the system (in kB), +"balloon.available" - the amount of usable memory as seen by the domain (in kB), +"balloon.rss" - Resident Set Size of running domain's process (in kB), +"balloon.usable" - the amount of memory which can be reclaimed by balloon +without causing host swapping (in KB), +"balloon.last-update" - timestamp of the last update of statistics (in seconds), I<--vcpu> returns: "vcpu.current" - current number of online virtual CPUs, -- 1.9.5.msysgit.0
participants (1)
-
Derbyshev Dmitriy