On Mon, Jun 29, 2015 at 04:53:38PM +0200, Michal Privoznik wrote:

So, this is an exercise to show libvirt capabilities. Firstly, for each host NUMA nodes some statistics are printed out, i.e. total

s/nodes/node/

memory and free memory. Then, for each running domain, that has memory strictly bound to certain host nodes, a small statistics of how much memory it takes is printed out too. For instance:

# ./nodestats.py NUMA stats NUMA nodes: 0 1 2 3 MemTotal: 3950 3967 3937 3943 MemFree: 434 674 149 216 Dom 'gentoo': 1048576 1048576 1048576 1048576

We can see 4 host NUMA nodes, all of them having roughly 4GB of RAM. Yeah, some of them has nearly all the memory consumed. Then, there's only one running domain, called 'gentoo', and it has 1GB per each NUMA node configured.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- examples/nodestats.py | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100755 examples/nodestats.py

diff --git a/examples/nodestats.py b/examples/nodestats.py new file mode 100755 index 0000000..dbf5593 --- /dev/null +++ b/examples/nodestats.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python +# Print some host NUMA node statistics +# +# Authors: +# Michal Privoznik <mprivozn@redhat.com

s/$/>/ if you want the authorship here ;)

+ +import libvirt +import sys +from xml.dom import minidom +import libxml2 + +class virBitmap: + def __init__(self): + self.bitmap = 0 + + def setBit(self, offset): + mask = 1 << offset + self.bitmap = self.bitmap | mask + + def clearBit(self, offset): + mask = ~(1 << offset) + self.bitmap = self.bitmap & mask + + def isSet(self, offset): + mask = 1 << offset + return(self.bitmap & mask) + + def setRange(self, start, end): + while (start <= end): + self.setBit(start) + start = start + 1 + + def parse(self, string): + for s in string.split(','): + list = s.split('-', 2) + start = int(list[0]) + if len(list) == 2: + end = int(list[1]) + else: + end = start + self.setRange(start, end) +

This last function is the only useful one. You are storing it all in one integer, so it's not very scalable (well, for python2 it isn't, python3 stores that in a long that has virtually unlimited size). Anyway, you are only storing nodeset here, for everything else you are using arrays. And the nodeset won't be very big. I'd rather you changed this to a normal array.

+def xpath_eval(ctxt, path): + res = ctxt.xpathEval(path) + if res is None or len(res) == 0: + value = None + else: + value = res[0].content + return value + +try: + conn = libvirt.openReadOnly(None) +except libvirt.libvirtError: + print('Failed to connect to the hypervisor') + sys.exit(1) + +try: + capsXML = conn.getCapabilities() +except libvirt.libvirtError: + print('Failed to request capabilities') + sys.exit(1) + +caps = minidom.parseString(capsXML) +cells = caps.getElementsByTagName('cells')[0] + +nodesIDs = [ int(proc.getAttribute('id')) + for proc in cells.getElementsByTagName('cell') ] + +nodesMem = [ conn.getMemoryStats(int(proc)) + for proc in nodesIDs] +

You could do { proc.getAttribute('id') : conn.getMemoryStats(int(proc.getAttribute('id))) } and have it in a dictionary, so it's easier to list through, but I'd prefer more readable: nodes = {} for p in cells.getElementsByTagName('cell'): i = p.getAttribute('id') nodes[i] = conn.getMemoryStats(i);

+doms = conn.listAllDomains(libvirt.VIR_CONNECT_LIST_DOMAINS_ACTIVE) +domsStrict = [ proc + for proc in doms + if proc.numaParameters()['numa_mode'] == libvirt.VIR_DOMAIN_NUMATUNE_MEM_STRICT ] +

This will list most of the domains even though there will be no info for them. You need to check whether there is any memnode with strict memory assigned. Although skipping everything else still feels kinda fishy. Maybe there should be another field for unspecified memnodes. Also if you run this with a domain with: <memnode cellid='0' mode='strict' nodeset='0-3'/> You report the allocated memory 4 times.

+domsStrictCfg = {} + +for dom in domsStrict: + xmlStr = dom.XMLDesc() + doc = libxml2.parseDoc(xmlStr) + ctxt = doc.xpathNewContext() + + domsStrictCfg[dom] = [ 0 for node in nodesIDs ] + + for memnode in ctxt.xpathEval("/domain/numatune/memnode"): + ctxt.setContextNode(memnode) + cellid = xpath_eval(ctxt, "@cellid") + mode = xpath_eval(ctxt, "@mode") + nodeset = xpath_eval(ctxt, "@nodeset") + + bitmap = virBitmap() + bitmap.parse(nodeset) + for node in nodesIDs: + if bitmap.isSet(int(node)): + mem = xpath_eval(ctxt, "/domain/cpu/numa/cell[@id='%s']/@memory" % cellid) + domsStrictCfg[dom][int(node)] += int(mem) + +print("NUMA stats") +print("NUMA nodes:\t\t" + "\t".join(str(node) for node in nodesIDs)) +print("MemTotal:\t\t" + "\t".join(str(i.get('total') / 1024) for i in nodesMem)) +print("MemFree:\t\t" + "\t".join(str(i.get('free') / 1024) for i in nodesMem)) +

Use one TAB so it's intuitive to parse without empty fields (be it with 'cut' or importing to 'libreoffice', doesn't matter). Also this produces ugly output on python3, you might want to round the number or divide it using '//' instead of '/'.

+for dom in domsStrictCfg: + sys.stdout.write("Dom '%s':\t\t" % dom.name()) + print("\t".join(map(str, domsStrictCfg[dom][:])))

No need to copy the list, plus generators are preferred to mapping. And why do you do sys.stdout.write() once, but print() after that? You can do 'print "asdf", ' in python2 or 'print("asdf", end="")' in python3 if you don't want newline at the end, but even better not use it at all. for k, v in domsStrictCfg.items(): print("Dom '%s':\t%s" % (k.name(), "\t".join([str(x) for x in v])))