On 10/30/2012 05:07 AM, Peter Krempa wrote:
Forwarding the response from George-Cristian Bîrzan who initialy reported that:
George-Cristian has a bunch of identical machines where some report having 4 NUMA cells and some just 1:
[...]
I did it on two hosts, one with 1 NUMA cell, one with 4 (as I said before, they both only report 12 cores though):
Both bitmaps show all 24 cores, so hwloc is able to read sysfs and determine the existence of 2 sockets with 12 nodes each, and where the 12 nodes are numbered 0-5 twice according to which bank of cache they are tied to. Which version of libvirt is this tested on where libvirt was only reporting 12 cores, because I thought we already patched that with commit 80533ca in 0.10.0. That is, I think proc1.png should result in: $ virsh nodeinfo CPU model: x86_64 CPU(s): 24 CPU frequency: 2200 MHz CPU socket(s): 2 Core(s) per socket: 12 Thread(s) per core: 1 NUMA cell(s): 1 Memory size: 8047272 KiB and proc4.png would _ideally_ result in: $ virsh nodeinfo CPU model: x86_64 CPU(s): 24 CPU frequency: 2200 MHz CPU socket(s): 2 Core(s) per socket: 12 Thread(s) per core: 1 NUMA cell(s): 4 Memory size: 8047272 KiB except that virNodeGetInfo() is constrained by backwards compatibility to report 'nodes' == 1 on situations where sockets per node is not integral (and here, half a socket per node is not integral), so it _actually_ would give the same data as proc1.png.
------
I think we should take this patch as it resolves this case. The data reported by kernel looks OK and the kernel probably trusts that everything is OK.
At any rate, I'm looking again at the patch, and the proposed linux-test7/node data indeed shows a single NUMA cell with 24 cores (matching up to the proc1.png image). I think the CPU _is_ reporting the complete NUMA topology through sysfs, but that we are probably consolidating information from the wrong files and therefore getting confused. I guess I need to install the linux-test7 files, then step through the code to see what is actually happening. Also, what does the 'virsh capabilities' report for the <topology> section? Whereas 'virsh nodeinfo' is constrained by back-compat to give a lame answer for number of NUMA cells, at least 'virsh capabilities' should be showing a reasonable representation of the machine's topology. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org