On 10/30/2012 05:07 AM, Peter Krempa wrote:
Forwarding the response from George-Cristian Bîrzan who initialy
reported that:
George-Cristian has a bunch of identical machines where some report
having 4 NUMA cells and some just 1:
[...]
I did it on two hosts, one with 1 NUMA cell, one with 4 (as I said before,
they both only report 12 cores though):
http://birzan.org/proc1.png
http://birzan.org/proc4.png
Both bitmaps show all 24 cores, so hwloc is able to read sysfs and
determine the existence of 2 sockets with 12 nodes each, and where the
12 nodes are numbered 0-5 twice according to which bank of cache they
are tied to. Which version of libvirt is this tested on where libvirt
was only reporting 12 cores, because I thought we already patched that
with commit 80533ca in 0.10.0. That is, I think proc1.png should result in:
$ virsh nodeinfo
CPU model: x86_64
CPU(s): 24
CPU frequency: 2200 MHz
CPU socket(s): 2
Core(s) per socket: 12
Thread(s) per core: 1
NUMA cell(s): 1
Memory size: 8047272 KiB
and proc4.png would _ideally_ result in:
$ virsh nodeinfo
CPU model: x86_64
CPU(s): 24
CPU frequency: 2200 MHz
CPU socket(s): 2
Core(s) per socket: 12
Thread(s) per core: 1
NUMA cell(s): 4
Memory size: 8047272 KiB
except that virNodeGetInfo() is constrained by backwards compatibility
to report 'nodes' == 1 on situations where sockets per node is not
integral (and here, half a socket per node is not integral), so it
_actually_ would give the same data as proc1.png.
------
I think we should take this patch as it resolves this case. The data
reported by kernel looks OK and the kernel probably trusts that
everything is OK.
At any rate, I'm looking again at the patch, and the proposed
linux-test7/node data indeed shows a single NUMA cell with 24 cores
(matching up to the proc1.png image).
I think the CPU _is_ reporting the complete NUMA topology through sysfs,
but that we are probably consolidating information from the wrong files
and therefore getting confused.
I guess I need to install the linux-test7 files, then step through the
code to see what is actually happening.
Also, what does the 'virsh capabilities' report for the <topology>
section? Whereas 'virsh nodeinfo' is constrained by back-compat to give
a lame answer for number of NUMA cells, at least 'virsh capabilities'
should be showing a reasonable representation of the machine's topology.
--
Eric Blake eblake(a)redhat.com +1-919-301-3266
Libvirt virtualization library
http://libvirt.org