[libvirt] [PATCH 0/2] prep for backing chain stat reporting

I'm still working on more patches to expose the allocation watermark of backing images during a block commit, but want to get these preliminary patches posted now. Eric Blake (2): virsh: document block.n.allocation stat getstats: add block.n.source stat src/libvirt-domain.c | 16 +++++++++++++--- src/qemu/qemu_driver.c | 36 ++++++++++++++++++++---------------- tools/virsh.pod | 10 +++++++--- 3 files changed, 40 insertions(+), 22 deletions(-) -- 1.9.3

Commit 7557ddf added some additional block.* stats to virDomainListGetStats, but failed to document them in 'man virsh'. Also, I noticed some inconsistent use of commas. * tools/virsh.pod (domstats): Tweak commas, add missing stats. Signed-off-by: Eric Blake <eblake@redhat.com> --- tools/virsh.pod | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/tools/virsh.pod b/tools/virsh.pod index da9c894..7cde3fd 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -849,7 +849,7 @@ I<--vcpu>, I<--interface>, I<--block>. When selecting the I<--state> group the following fields are returned: "state.state" - state of the VM, returned as number from virDomainState enum, -"state.reason" - reason for entering given state, returned as int from, +"state.reason" - reason for entering given state, returned as int from virDomain*Reason enum corresponding to given state. I<--cpu-total> returns: @@ -878,9 +878,9 @@ I<--interface> returns: "net.<num>.tx.bytes" - number of bytes transmitted, "net.<num>.tx.pkts" - number of packets transmitted, "net.<num>.tx.errs" - number of transmission errors, -"net.<num>.tx.drop" - number of transmit packets dropped, +"net.<num>.tx.drop" - number of transmit packets dropped -I<block> returns: +I<--block> returns: "block.count" - number of block devices on this domain, "block.<num>.name" - name of the target of the block device <num>, "block.<num>.rd.reqs" - number of read requests, @@ -892,6 +892,9 @@ I<block> returns: "block.<num>.fl.reqs" - total flush requests, "block.<num>.fl.times" - total time (ns) spent on cache flushing, "block.<num>.errors" - Xen only: the 'oo_req' value, +"block.<num>.allocation" - offset of highest written sector in bytes, +"block.<num>.capacity" - logical size of source file in bytes, +"block.<num>.physical" - physical size of source file in bytes Selecting a specific statistics groups doesn't guarantee that the daemon supports the selected group of stats. Flag I<--enforce> -- 1.9.3

On 11/25/14 06:21, Eric Blake wrote:
Commit 7557ddf added some additional block.* stats to virDomainListGetStats, but failed to document them in 'man virsh'. Also, I noticed some inconsistent use of commas.
* tools/virsh.pod (domstats): Tweak commas, add missing stats.
Signed-off-by: Eric Blake <eblake@redhat.com> --- tools/virsh.pod | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
ACK, Peter

On 11/25/2014 01:59 AM, Peter Krempa wrote:
On 11/25/14 06:21, Eric Blake wrote:
Commit 7557ddf added some additional block.* stats to virDomainListGetStats, but failed to document them in 'man virsh'. Also, I noticed some inconsistent use of commas.
* tools/virsh.pod (domstats): Tweak commas, add missing stats.
Signed-off-by: Eric Blake <eblake@redhat.com> --- tools/virsh.pod | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
ACK,
This one is pushed. Later, I'll send a v2 with the adjustments based on your suggestions on 2/2, as well as further patches to make more progress towards backing chain stat reporting. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

I noticed that for an offline domain, 'virsh domstats --block $dom' was producing just the domain name, with no stats. But the older 'virsh domblkinfo' works just fine on offline domains. Furthermore, I'm about to make block stats optionally more complex to cover backing chains, where block.count will no longer equal the number of <disks> for a domain. For these reasons, it is nicer if the statistics output always includes minimal information on every resource being described, with enough to correlate back to host resources, and even when some statistics are available only on a running domain. With this patch, I now see the following for an offline domain with one disk and an empty cdrom drive: $ virsh domstats --block foo Domain: 'foo' block.count=2 block.0.name=hda block.0.source=/dev/sda4 block.1.name=hdc * src/libvirt-domain.c (virConnectGetAllDomainStats) (virDomainListGetStats): Document new field; clarify that not all fields must be present for a group to be supported. * tools/virsh.pod (domstats): Document new field. * src/qemu/qemu_driver.c (qemuDomainGetStatsBlock): Return the new stat always, even for offline domains. (QEMU_ADD_NAME_PARAM): Add parameter. (qemuDomainGetStatsInterface): Update caller. Signed-off-by: Eric Blake <eblake@redhat.com> --- src/libvirt-domain.c | 16 +++++++++++++--- src/qemu/qemu_driver.c | 36 ++++++++++++++++++++---------------- tools/virsh.pod | 1 + 3 files changed, 34 insertions(+), 19 deletions(-) diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 2b0defc..8cd21ae 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -10907,6 +10907,8 @@ virConnectGetDomainCapabilities(virConnectPtr conn, * "block.<num>.name" - name of the block device <num> as string. * matches the target name (vda/sda/hda) of the * block device. + * "block.<num>.source" - string describing the source of block device <num>, + * such as the host path of a file or device. * "block.<num>.rd.reqs" - number of read requests as unsigned long long. * "block.<num>.rd.bytes" - number of read bytes as unsigned long long. * "block.<num>.rd.times" - total time (ns) spent on reads as @@ -10937,7 +10939,11 @@ virConnectGetDomainCapabilities(virConnectPtr conn, * * Specifying VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS as @flags makes * the function return error in case some of the stat types in @stats were - * not recognized by the daemon. + * not recognized by the daemon. However, even with this flag, a hypervisor + * may omit individual fields within a group if the information is not + * available; as an extreme example, a supported group may produce zero + * fields for offline domains if the statistics are meaningful only for a + * running domain. * * Similarly to virConnectListAllDomains, @flags can contain various flags to * filter the list of domains to provide stats for. @@ -11017,9 +11023,13 @@ virConnectGetAllDomainStats(virConnectPtr conn, * * Specifying VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS as @flags makes * the function return error in case some of the stat types in @stats were - * not recognized by the daemon. + * not recognized by the daemon. However, even with this flag, a hypervisor + * may omit individual fields within a group if the information is not + * available; as an extreme example, a supported group may produce zero + * fields for offline domains if the statistics are meaningful only for a + * running domain. * - * Note that any of the domain list filtering flags in @flags will be rejected + * Note that any of the domain list filtering flags in @flags may be rejected * by this function. * * Returns the count of returned statistics structures on success, -1 on error. diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 07da3e3..9295a05 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -18408,11 +18408,11 @@ do { \ return -1; \ } while (0) -#define QEMU_ADD_NAME_PARAM(record, maxparams, type, num, name) \ +#define QEMU_ADD_NAME_PARAM(record, maxparams, type, subtype, num, name) \ do { \ char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; \ snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, \ - "%s.%zu.name", type, num); \ + "%s.%zu.%s", type, num, subtype); \ if (virTypedParamsAddString(&(record)->params, \ &(record)->nparams, \ maxparams, \ @@ -18457,7 +18457,7 @@ qemuDomainGetStatsInterface(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, memset(&tmp, 0, sizeof(tmp)); QEMU_ADD_NAME_PARAM(record, maxparams, - "net", i, dom->def->nets[i]->ifname); + "net", "name", i, dom->def->nets[i]->ifname); if (virNetInterfaceStats(dom->def->nets[i]->ifname, &tmp) < 0) { virResetLastError(); @@ -18526,19 +18526,20 @@ qemuDomainGetStatsBlock(virQEMUDriverPtr driver, int rc; virHashTablePtr stats = NULL; qemuDomainObjPrivatePtr priv = dom->privateData; + bool abbreviated = false; - if (!HAVE_JOB(privflags) || !virDomainObjIsActive(dom)) - return 0; /* it's ok, just go ahead silently */ + if (!HAVE_JOB(privflags) || !virDomainObjIsActive(dom)) { + abbreviated = true; /* it's ok, just go ahead silently */ + } else { + qemuDomainObjEnterMonitor(driver, dom); + rc = qemuMonitorGetAllBlockStatsInfo(priv->mon, &stats); + ignore_value(qemuMonitorBlockStatsUpdateCapacity(priv->mon, stats)); + qemuDomainObjExitMonitor(driver, dom); - qemuDomainObjEnterMonitor(driver, dom); - rc = qemuMonitorGetAllBlockStatsInfo(priv->mon, &stats); - ignore_value(qemuMonitorBlockStatsUpdateCapacity(priv->mon, stats)); - qemuDomainObjExitMonitor(driver, dom); - - if (rc < 0) { - virResetLastError(); - ret = 0; /* still ok, again go ahead silently */ - goto cleanup; + if (rc < 0) { + virResetLastError(); + abbreviated = true; /* still ok, again go ahead silently */ + } } QEMU_ADD_COUNT_PARAM(record, maxparams, "block", dom->def->ndisks); @@ -18547,9 +18548,12 @@ qemuDomainGetStatsBlock(virQEMUDriverPtr driver, qemuBlockStats *entry; virDomainDiskDefPtr disk = dom->def->disks[i]; - QEMU_ADD_NAME_PARAM(record, maxparams, "block", i, disk->dst); + QEMU_ADD_NAME_PARAM(record, maxparams, "block", "name", i, disk->dst); + if (disk->src && disk->src->path) + QEMU_ADD_NAME_PARAM(record, maxparams, "block", "source", + i, disk->src->path); - if (!disk->info.alias || + if (abbreviated || !disk->info.alias || !(entry = virHashLookup(stats, disk->info.alias))) continue; diff --git a/tools/virsh.pod b/tools/virsh.pod index 7cde3fd..e038c7e 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -883,6 +883,7 @@ I<--interface> returns: I<--block> returns: "block.count" - number of block devices on this domain, "block.<num>.name" - name of the target of the block device <num>, +"block.<num>.source" - name of the source of block device <num>, "block.<num>.rd.reqs" - number of read requests, "block.<num>.rd.bytes" - number of read bytes, "block.<num>.rd.times" - total time (ns) spent on reads, -- 1.9.3

On 11/25/14 06:21, Eric Blake wrote:
I noticed that for an offline domain, 'virsh domstats --block $dom' was producing just the domain name, with no stats. But the older 'virsh domblkinfo' works just fine on offline domains. Furthermore, I'm about to make block stats optionally more complex to cover backing chains, where block.count will no longer equal the number of <disks> for a domain. For these reasons, it is nicer if the statistics output always includes minimal information on every resource being described, with enough to correlate back to host resources, and even when some statistics are available only on a running domain.
With this patch, I now see the following for an offline domain with one disk and an empty cdrom drive: $ virsh domstats --block foo Domain: 'foo' block.count=2 block.0.name=hda block.0.source=/dev/sda4 block.1.name=hdc
* src/libvirt-domain.c (virConnectGetAllDomainStats) (virDomainListGetStats): Document new field; clarify that not all fields must be present for a group to be supported. * tools/virsh.pod (domstats): Document new field. * src/qemu/qemu_driver.c (qemuDomainGetStatsBlock): Return the new stat always, even for offline domains. (QEMU_ADD_NAME_PARAM): Add parameter. (qemuDomainGetStatsInterface): Update caller.
Signed-off-by: Eric Blake <eblake@redhat.com> --- src/libvirt-domain.c | 16 +++++++++++++--- src/qemu/qemu_driver.c | 36 ++++++++++++++++++++---------------- tools/virsh.pod | 1 + 3 files changed, 34 insertions(+), 19 deletions(-)
diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 2b0defc..8cd21ae 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -10907,6 +10907,8 @@ virConnectGetDomainCapabilities(virConnectPtr conn, * "block.<num>.name" - name of the block device <num> as string. * matches the target name (vda/sda/hda) of the * block device. + * "block.<num>.source" - string describing the source of block device <num>, + * such as the host path of a file or device.
Fair enough as long as we document that we only return it for non-network sources. We had quite a few problems with network sources so I'd rather not expose it for those due to possible ambiguity.
* "block.<num>.rd.reqs" - number of read requests as unsigned long long. * "block.<num>.rd.bytes" - number of read bytes as unsigned long long. * "block.<num>.rd.times" - total time (ns) spent on reads as @@ -10937,7 +10939,11 @@ virConnectGetDomainCapabilities(virConnectPtr conn, * * Specifying VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS as @flags makes * the function return error in case some of the stat types in @stats were - * not recognized by the daemon. + * not recognized by the daemon. However, even with this flag, a hypervisor + * may omit individual fields within a group if the information is not + * available; as an extreme example, a supported group may produce zero + * fields for offline domains if the statistics are meaningful only for a + * running domain. * * Similarly to virConnectListAllDomains, @flags can contain various flags to * filter the list of domains to provide stats for. @@ -11017,9 +11023,13 @@ virConnectGetAllDomainStats(virConnectPtr conn, * * Specifying VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS as @flags makes * the function return error in case some of the stat types in @stats were - * not recognized by the daemon. + * not recognized by the daemon. However, even with this flag, a hypervisor + * may omit individual fields within a group if the information is not + * available; as an extreme example, a supported group may produce zero + * fields for offline domains if the statistics are meaningful only for a + * running domain. *
The code changes are not entirely related to this doc change.
- * Note that any of the domain list filtering flags in @flags will be rejected + * Note that any of the domain list filtering flags in @flags may be rejected * by this function. * * Returns the count of returned statistics structures on success, -1 on error. diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 07da3e3..9295a05 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -18408,11 +18408,11 @@ do { \ return -1; \ } while (0)
-#define QEMU_ADD_NAME_PARAM(record, maxparams, type, num, name) \ +#define QEMU_ADD_NAME_PARAM(record, maxparams, type, subtype, num, name) \ do { \ char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; \ snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, \ - "%s.%zu.name", type, num); \ + "%s.%zu.%s", type, num, subtype); \ if (virTypedParamsAddString(&(record)->params, \ &(record)->nparams, \ maxparams, \ @@ -18457,7 +18457,7 @@ qemuDomainGetStatsInterface(virQEMUDriverPtr driver ATTRIBUTE_UNUSED, memset(&tmp, 0, sizeof(tmp));
QEMU_ADD_NAME_PARAM(record, maxparams, - "net", i, dom->def->nets[i]->ifname); + "net", "name", i, dom->def->nets[i]->ifname);
if (virNetInterfaceStats(dom->def->nets[i]->ifname, &tmp) < 0) { virResetLastError(); @@ -18526,19 +18526,20 @@ qemuDomainGetStatsBlock(virQEMUDriverPtr driver, int rc; virHashTablePtr stats = NULL; qemuDomainObjPrivatePtr priv = dom->privateData; + bool abbreviated = false;
- if (!HAVE_JOB(privflags) || !virDomainObjIsActive(dom)) - return 0; /* it's ok, just go ahead silently */ + if (!HAVE_JOB(privflags) || !virDomainObjIsActive(dom)) { + abbreviated = true; /* it's ok, just go ahead silently */ + } else { + qemuDomainObjEnterMonitor(driver, dom); + rc = qemuMonitorGetAllBlockStatsInfo(priv->mon, &stats); + ignore_value(qemuMonitorBlockStatsUpdateCapacity(priv->mon, stats)); + qemuDomainObjExitMonitor(driver, dom);
- qemuDomainObjEnterMonitor(driver, dom); - rc = qemuMonitorGetAllBlockStatsInfo(priv->mon, &stats); - ignore_value(qemuMonitorBlockStatsUpdateCapacity(priv->mon, stats)); - qemuDomainObjExitMonitor(driver, dom); - - if (rc < 0) { - virResetLastError(); - ret = 0; /* still ok, again go ahead silently */ - goto cleanup; + if (rc < 0) { + virResetLastError(); + abbreviated = true; /* still ok, again go ahead silently */ + } }
QEMU_ADD_COUNT_PARAM(record, maxparams, "block", dom->def->ndisks); @@ -18547,9 +18548,12 @@ qemuDomainGetStatsBlock(virQEMUDriverPtr driver, qemuBlockStats *entry; virDomainDiskDefPtr disk = dom->def->disks[i];
- QEMU_ADD_NAME_PARAM(record, maxparams, "block", i, disk->dst); + QEMU_ADD_NAME_PARAM(record, maxparams, "block", "name", i, disk->dst); + if (disk->src && disk->src->path)
&& virStorageSourceIsLocalStorage(disk->src)
+ QEMU_ADD_NAME_PARAM(record, maxparams, "block", "source", + i, disk->src->path);
- if (!disk->info.alias || + if (abbreviated || !disk->info.alias || !(entry = virHashLookup(stats, disk->info.alias))) continue;
diff --git a/tools/virsh.pod b/tools/virsh.pod index 7cde3fd..e038c7e 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -883,6 +883,7 @@ I<--interface> returns: I<--block> returns: "block.count" - number of block devices on this domain, "block.<num>.name" - name of the target of the block device <num>, +"block.<num>.source" - name of the source of block device <num>, "block.<num>.rd.reqs" - number of read requests, "block.<num>.rd.bytes" - number of read bytes, "block.<num>.rd.times" - total time (ns) spent on reads,
Peter

On 11/25/2014 02:10 AM, Peter Krempa wrote:
On 11/25/14 06:21, Eric Blake wrote:
I noticed that for an offline domain, 'virsh domstats --block $dom' was producing just the domain name, with no stats. But the older 'virsh domblkinfo' works just fine on offline domains. Furthermore, I'm about to make block stats optionally more complex to cover backing chains, where block.count will no longer equal the number of <disks> for a domain. For these reasons, it is nicer if the statistics output always includes minimal information on every resource being described, with enough to correlate back to host resources, and even when some statistics are available only on a running domain.
+ * "block.<num>.source" - string describing the source of block device <num>, + * such as the host path of a file or device.
Fair enough as long as we document that we only return it for non-network sources. We had quite a few problems with network sources so I'd rather not expose it for those due to possible ambiguity.
Sure, I can do that. Maybe I should then name it block.<num>.path or block.<num>.file, to make it obvious that it is only a file name? And I suppose we could always add other fields later if it turns out to be useful on network devices, but I won't worry about it for now.
* * Specifying VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS as @flags makes * the function return error in case some of the stat types in @stats were - * not recognized by the daemon. + * not recognized by the daemon. However, even with this flag, a hypervisor + * may omit individual fields within a group if the information is not + * available; as an extreme example, a supported group may produce zero + * fields for offline domains if the statistics are meaningful only for a + * running domain. *
The code changes are not entirely related to this doc change.
Should I split it into a separate patch, then?
- QEMU_ADD_NAME_PARAM(record, maxparams, "block", i, disk->dst); + QEMU_ADD_NAME_PARAM(record, maxparams, "block", "name", i, disk->dst); + if (disk->src && disk->src->path)
&& virStorageSourceIsLocalStorage(disk->src)
Indeed, that fits with your request to limit to files. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

On 11/25/14 16:43, Eric Blake wrote:
On 11/25/2014 02:10 AM, Peter Krempa wrote:
On 11/25/14 06:21, Eric Blake wrote:
I noticed that for an offline domain, 'virsh domstats --block $dom' was producing just the domain name, with no stats. But the older 'virsh domblkinfo' works just fine on offline domains. Furthermore, I'm about to make block stats optionally more complex to cover backing chains, where block.count will no longer equal the number of <disks> for a domain. For these reasons, it is nicer if the statistics output always includes minimal information on every resource being described, with enough to correlate back to host resources, and even when some statistics are available only on a running domain.
+ * "block.<num>.source" - string describing the source of block device <num>, + * such as the host path of a file or device.
Fair enough as long as we document that we only return it for non-network sources. We had quite a few problems with network sources so I'd rather not expose it for those due to possible ambiguity.
Sure, I can do that. Maybe I should then name it block.<num>.path or block.<num>.file, to make it obvious that it is only a file name? And I suppose we could always add other fields later if it turns out to be useful on network devices, but I won't worry about it for now.
I'd go with path. File is not quite right with physical devices/lvs used as source.
* * Specifying VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS as @flags makes * the function return error in case some of the stat types in @stats were - * not recognized by the daemon. + * not recognized by the daemon. However, even with this flag, a hypervisor + * may omit individual fields within a group if the information is not + * available; as an extreme example, a supported group may produce zero + * fields for offline domains if the statistics are meaningful only for a + * running domain. *
The code changes are not entirely related to this doc change.
Should I split it into a separate patch, then?
I think it would make more sense.
- QEMU_ADD_NAME_PARAM(record, maxparams, "block", i, disk->dst); + QEMU_ADD_NAME_PARAM(record, maxparams, "block", "name", i, disk->dst); + if (disk->src && disk->src->path)
&& virStorageSourceIsLocalStorage(disk->src)
Indeed, that fits with your request to limit to files.
Peter
participants (2)
-
Eric Blake
-
Peter Krempa