[libvirt] PCP libvirt Plugin
Hi, FYI, I've written a PCP plugin (PMDA in PCP parlance) to support most hypervisor / domain information and metrics available over the libvirt Python API, it's up to date as of libvirt 2.3 (so it already supports the recently added perf event metrics). In case you haven't heard about PCP, here's a brief intro: The Performance Co-Pilot (PCP, http://www.pcp.io/) system is a toolkit for collecting, archiving, and processing performance metrics. It supports system and application metrics from local and remote hosts, archives, and containers. A typical Linux PCP installation offers over 1,000 metrics by default and is easily extensible with its own plugins, or PMDAs ("Performance Metrics Domain Agents"). In addition to very complete /proc based statistics, readily available PCP PMDAs provide support components like 389 Directory Server, Apache, Ceph, GFS2, Gluster, MySQL, NFS, Oracle, Postfix, PostgreSQL, Samba, and Sendmail, among others. PCP also runs on many platforms, including Linux, Mac OS X, FreeBSD, NetBSD, Solaris, and Windows. And here's Quick Guide to get started with PCP: http://www.pcp.io/docs/guide.html (So PCP comes with handy clients like pmrep(1) for custom metrics reporting, sophisticated logging and inferencing tools, and support for Grafana based Web UI.) The libvirt metrics available with PCP libvirt PMDA and their descriptions are listed below, they include all the recently added perf event metrics as well as combined and per-device metrics for each VM. The plugin is available for Fedora in updates-testing and for Ubuntu from https://bintray.com/pcp/ . As this is a recent addition, the plugin or some of the later additions like per-device metrics have not propagated to all distributions yet. For those using PCP already, they may also manually grab the needed files from the below page without the need to upgrade any PCP packages: https://myllynen.fedorapeople.org/pmdalibvirt/ Upstream code lives here: http://oss.sgi.com/cgi-bin/gitweb.cgi?p=pcp/pcp.git;a=tree;f=src/pmdas/libvi... I wonder could this be mentioned at https://libvirt.org/apps.html ? Here's the list of the PCP metrics available with the current git code: libvirt.dominfo.container [VM container] libvirt.dominfo.description [VM description] libvirt.dominfo.memory.boot [VM memory, at boot] libvirt.dominfo.memory.current [VM memory, current] libvirt.dominfo.memory.max [VM memory, maximum] libvirt.dominfo.name [VM name] libvirt.dominfo.os.type [VM OS type] libvirt.dominfo.title [VM title] libvirt.dominfo.type [VM type] libvirt.dominfo.uuid [VM UUID] libvirt.dominfo.vcpu.current [VM vCPUs, current] libvirt.dominfo.vcpu.max [VM vCPUs, maximum] libvirt.domstats.balloon.current [VM balloon size, current] libvirt.domstats.balloon.maximum [VM balloon size, maximum] libvirt.domstats.block.all.allocation [VM backing imgs, total allocation] libvirt.domstats.block.all.backingIndex [VM block devs, backing chain imgs] libvirt.domstats.block.all.capacity [VM backing imgs, total capacity] libvirt.domstats.block.all.fl.reqs [VM block devs, total fl reqs] libvirt.domstats.block.all.fl.times [VM block devs, total fl times] libvirt.domstats.block.all.name [VM block devs, all names] libvirt.domstats.block.all.path [VM block devs, all paths] libvirt.domstats.block.all.physical [VM backing imgs, total physical] libvirt.domstats.block.all.rd.bytes [VM block devs, total rd bytes] libvirt.domstats.block.all.rd.reqs [VM block devs, total rd reqs] libvirt.domstats.block.all.rd.times [VM block devs, total rd times] libvirt.domstats.block.all.wr.bytes [VM block devs, total wr bytes] libvirt.domstats.block.all.wr.reqs [VM block devs, total wr reqs] libvirt.domstats.block.all.wr.times [VM block devs, total wr times] libvirt.domstats.block.allocation [VM backing img, allocation] libvirt.domstats.block.backingIndex [VM block dev, backing chain img] libvirt.domstats.block.capacity [VM backing img, capacity] libvirt.domstats.block.count [VM block devs, count] libvirt.domstats.block.fl.reqs [VM block dev, fl reqs] libvirt.domstats.block.fl.times [VM block dev, fl times] libvirt.domstats.block.name [VM block dev, name] libvirt.domstats.block.path [VM block dev, path] libvirt.domstats.block.physical [VM backing img, physical] libvirt.domstats.block.rd.bytes [VM block dev, rd bytes] libvirt.domstats.block.rd.reqs [VM block dev, rd reqs] libvirt.domstats.block.rd.times [VM block dev, rd times] libvirt.domstats.block.wr.bytes [VM block dev, wr bytes] libvirt.domstats.block.wr.reqs [VM block dev, wr reqs] libvirt.domstats.block.wr.times [VM block dev, wr times] libvirt.domstats.cpu.system [VM CPU time, system] libvirt.domstats.cpu.time [VM CPU time, total] libvirt.domstats.cpu.user [VM CPU time, user] libvirt.domstats.mem.actual [VM memory, balloon size] libvirt.domstats.mem.available [VM memory, available] libvirt.domstats.mem.major_fault [VM memory, major faults] libvirt.domstats.mem.minor_fault [VM memory, minor faults] libvirt.domstats.mem.rss [VM memory, proc RSS] libvirt.domstats.mem.swap_in [VM memory, swapped in total] libvirt.domstats.mem.swap_out [VM memory, swapped out total] libvirt.domstats.mem.unused [VM memory, unused] libvirt.domstats.net.all.name [VM NICs, all names] libvirt.domstats.net.all.rx.bytes [VM NICs, total rx bytes] libvirt.domstats.net.all.rx.drop [VM NICs, total rx drop] libvirt.domstats.net.all.rx.errs [VM NICs, total rx errs] libvirt.domstats.net.all.rx.pkts [VM NICs, total rx pkts] libvirt.domstats.net.all.tx.bytes [VM NICs, total tx bytes] libvirt.domstats.net.all.tx.drop [VM NICs, total tx drop] libvirt.domstats.net.all.tx.errs [VM NICs, total tx errs] libvirt.domstats.net.all.tx.pkts [VM NICs, total tx pkts] libvirt.domstats.net.count [VM NICs, count] libvirt.domstats.net.name [VM NIC, name] libvirt.domstats.net.rx.bytes [VM NIC, rx bytes] libvirt.domstats.net.rx.drop [VM NIC, rx drop] libvirt.domstats.net.rx.errs [VM NIC, rx errs] libvirt.domstats.net.rx.pkts [VM NIC, rx pkts] libvirt.domstats.net.tx.bytes [VM NIC, tx bytes] libvirt.domstats.net.tx.drop [VM NIC, tx drop] libvirt.domstats.net.tx.errs [VM NIC, tx errs] libvirt.domstats.net.tx.pkts [VM NIC, tx pkts] libvirt.domstats.perf.cache_misses [VM perf stats, cache misses] libvirt.domstats.perf.cache_references [VM perf stats, cache references] libvirt.domstats.perf.cmt [VM perf stats, cmt] libvirt.domstats.perf.cpu_cycles [VM perf stats, CPU cycles/instr] libvirt.domstats.perf.instructions [VM perf stats, instructions count] libvirt.domstats.perf.mbml [VM perf stats, mbml] libvirt.domstats.perf.mbmt [VM perf stats, mbmt] libvirt.domstats.vcpu.all.state [VM vCPUs, total state] libvirt.domstats.vcpu.all.time [VM vCPUs, total time] libvirt.domstats.vcpu.all.wait [VM vCPUs, total wait] libvirt.domstats.vcpu.current [VM vCPUs, current] libvirt.domstats.vcpu.maximum [VM vCPUs, maximum] libvirt.domstats.vcpu.state [VM vCPU, state] libvirt.domstats.vcpu.time [VM vCPU, time] libvirt.domstats.vcpu.wait [VM vCPU, wait] libvirt.hv.devices [Libvirt devices] libvirt.hv.domains.active [Libvirt domains, active] libvirt.hv.domains.inactive [Libvirt domains, inactive] libvirt.hv.driver [Libvirt driver] libvirt.hv.interfaces.active [Libvirt interfaces, active] libvirt.hv.interfaces.inactive [Libvirt interfaces, inactive] libvirt.hv.networkfilters [Libvirt network filters] libvirt.hv.networks.active [Libvirt networks, active] libvirt.hv.networks.inactive [Libvirt networks, inactive] libvirt.hv.secrets [Libvirt secrets] libvirt.hv.storagepools.active [Libvirt storage pools, active] libvirt.hv.storagepools.inactive [Libvirt storage pools, inactive] libvirt.hv.uri [Libvirt URI] libvirt.hv.version [Libvirt version] Thanks, -- Marko Myllynen
On Wed, Oct 05, 2016 at 07:28:00PM +0300, Marko Myllynen wrote:
Hi,
FYI, I've written a PCP plugin (PMDA in PCP parlance) to support most hypervisor / domain information and metrics available over the libvirt Python API, it's up to date as of libvirt 2.3 (so it already supports the recently added perf event metrics).
In case you haven't heard about PCP, here's a brief intro:
The Performance Co-Pilot (PCP, http://www.pcp.io/) system is a toolkit for collecting, archiving, and processing performance metrics. It supports system and application metrics from local and remote hosts, archives, and containers. A typical Linux PCP installation offers over 1,000 metrics by default and is easily extensible with its own plugins, or PMDAs ("Performance Metrics Domain Agents"). In addition to very complete /proc based statistics, readily available PCP PMDAs provide support components like 389 Directory Server, Apache, Ceph, GFS2, Gluster, MySQL, NFS, Oracle, Postfix, PostgreSQL, Samba, and Sendmail, among others. PCP also runs on many platforms, including Linux, Mac OS X, FreeBSD, NetBSD, Solaris, and Windows.
And here's Quick Guide to get started with PCP:
http://www.pcp.io/docs/guide.html
(So PCP comes with handy clients like pmrep(1) for custom metrics reporting, sophisticated logging and inferencing tools, and support for Grafana based Web UI.)
The libvirt metrics available with PCP libvirt PMDA and their descriptions are listed below, they include all the recently added perf event metrics as well as combined and per-device metrics for each VM.
The plugin is available for Fedora in updates-testing and for Ubuntu from https://bintray.com/pcp/ . As this is a recent addition, the plugin or some of the later additions like per-device metrics have not propagated to all distributions yet. For those using PCP already, they may also manually grab the needed files from the below page without the need to upgrade any PCP packages:
https://myllynen.fedorapeople.org/pmdalibvirt/
Upstream code lives here:
http://oss.sgi.com/cgi-bin/gitweb.cgi?p=pcp/pcp.git;a=tree;f=src/pmdas/libvi...
I wonder could this be mentioned at https://libvirt.org/apps.html ?
Great to hear! Sure! Would you mind sending a patch against libvirt's docs/apps.html.in with what you'd like to have mentioned there? If you don't like doing that I can do that for you, just let me know. Martin
Hi, On 2016-10-06 15:01, Martin Kletzander wrote:
On Wed, Oct 05, 2016 at 07:28:00PM +0300, Marko Myllynen wrote:
FYI, I've written a PCP plugin (PMDA in PCP parlance) to support most hypervisor / domain information and metrics available over the libvirt Python API, it's up to date as of libvirt 2.3 (so it already supports the recently added perf event metrics).
The libvirt metrics available with PCP libvirt PMDA and their descriptions are listed below, they include all the recently added perf event metrics as well as combined and per-device metrics for each VM.
I wonder could this be mentioned at https://libvirt.org/apps.html ?
Great to hear! Sure! Would you mind sending a patch against libvirt's docs/apps.html.in with what you'd like to have mentioned there? If you don't like doing that I can do that for you, just let me know.
Sure, how about this? --- a/apps.html +++ b/apps.html @@ -372,6 +372,12 @@ You can use this tool to either set up a new Nagios installation for your Xen or QEMU/KVM guests, or to integrate with your existing Nagios installation. + </dd><dt><a href="http://www.pcp.io/man/man1/pmdalibvirt.1.html" shape="rect">PCP</a></dt><dd> + The PCP libvirt PMDA (plugin) is part of the + <a href="http://pcp.io/" shape="rect">PCP<a> toolkit and provides + hypervisor and guest information and complete set of guest performance + metrics. It supports pCPU, vCPU, memory, block device, network interface, + and performance event metrics for each virtual guest. </dd><dt><a href="http://community.zenoss.org/docs/DOC-4687" shape="rect">Zenoss</a></dt><dd> The Zenoss libvirt Zenpack adds support for monitoring virtualization servers. It has been tested with KVM, QEMU, VMware ESX, and VMware Thanks, -- Marko Myllynen
On 07.10.2016 09:22, Marko Myllynen wrote:
Hi,
On 2016-10-06 15:01, Martin Kletzander wrote:
On Wed, Oct 05, 2016 at 07:28:00PM +0300, Marko Myllynen wrote:
FYI, I've written a PCP plugin (PMDA in PCP parlance) to support most hypervisor / domain information and metrics available over the libvirt Python API, it's up to date as of libvirt 2.3 (so it already supports the recently added perf event metrics).
The libvirt metrics available with PCP libvirt PMDA and their descriptions are listed below, they include all the recently added perf event metrics as well as combined and per-device metrics for each VM.
I wonder could this be mentioned at https://libvirt.org/apps.html ?
Great to hear! Sure! Would you mind sending a patch against libvirt's docs/apps.html.in with what you'd like to have mentioned there? If you don't like doing that I can do that for you, just let me know.
Sure, how about this?
Almost :-)
--- a/apps.html +++ b/apps.html
this file is generated from the apps.html.in.
@@ -372,6 +372,12 @@ You can use this tool to either set up a new Nagios installation for your Xen or QEMU/KVM guests, or to integrate with your existing Nagios installation. + </dd><dt><a href="http://www.pcp.io/man/man1/pmdalibvirt.1.html" shape="rect">PCP</a></dt><dd> + The PCP libvirt PMDA (plugin) is part of the + <a href="http://pcp.io/" shape="rect">PCP<a> toolkit and provides
s,<a>,</a>,
+ hypervisor and guest information and complete set of guest performance + metrics. It supports pCPU, vCPU, memory, block device, network interface, + and performance event metrics for each virtual guest. </dd><dt><a href="http://community.zenoss.org/docs/DOC-4687" shape="rect">Zenoss</a></dt><dd> The Zenoss libvirt Zenpack adds support for monitoring virtualization servers. It has been tested with KVM, QEMU, VMware ESX, and VMware
Thanks,
I've fixed all of that, came up with a commit message and pushed. Congratulations on your first libvirt contribution! Michal
Hi, On 2016-10-07 12:55, Michal Privoznik wrote:
On 07.10.2016 09:22, Marko Myllynen wrote:
On 2016-10-06 15:01, Martin Kletzander wrote:
On Wed, Oct 05, 2016 at 07:28:00PM +0300, Marko Myllynen wrote:
FYI, I've written a PCP plugin (PMDA in PCP parlance) to support most hypervisor / domain information and metrics available over the libvirt Python API, it's up to date as of libvirt 2.3 (so it already supports the recently added perf event metrics).
The libvirt metrics available with PCP libvirt PMDA and their descriptions are listed below, they include all the recently added perf event metrics as well as combined and per-device metrics for each VM.
I wonder could this be mentioned at https://libvirt.org/apps.html ?
Great to hear! Sure! Would you mind sending a patch against libvirt's docs/apps.html.in with what you'd like to have mentioned there? If you don't like doing that I can do that for you, just let me know.
Sure, how about this?
Almost :-)
s,<a>,</a>,
Ha, my HTML coding skillz are a bit rusty, it seems! :-)
I've fixed all of that, came up with a commit message and pushed.
Congratulations on your first libvirt contribution!
Thanks! -- Marko Myllynen
participants (3)
-
Marko Myllynen -
Martin Kletzander -
Michal Privoznik