Re: [libvirt] [PATCH] nodeinfo: Add workaround if kernel reports bogous numa topology.
On Tue, Oct 30, 2012 at 10:08 PM, Eric Blake <eblake(a)redhat.com> wrote:
Both bitmaps show all 24 cores, so hwloc is able to read sysfs and
determine the existence of 2 sockets with 12 nodes each, and where the
12 nodes are numbered 0-5 twice according to which bank of cache they
are tied to. Which version of libvirt is this tested on where libvirt
was only reporting 12 cores, because I thought we already patched that
with commit 80533ca in 0.10.0. That is, I think proc1.png should result
in:
$ virsh nodeinfo
CPU model: x86_64
CPU(s): 24
CPU frequency: 2200 MHz
CPU socket(s): 2
Core(s) per socket: 12
Thread(s) per core: 1
NUMA cell(s): 1
Memory size: 8047272 KiB
(Just for clarity, I am the original reporter, sorry I couldn't answer
earlier, I subscribed to the list too late and didn't want to break the
thread)
On the host with 1 NUMA cell, with libvrit 0.10.2 I get:
[root@host29 ~]# virsh nodeinfo
CPU model: x86_64
CPU(s): 24
CPU frequency: 2200 MHz
CPU socket(s): 2
Core(s) per socket: 6
Thread(s) per core: 1
NUMA cell(s): 1
Memory size: 131971548 KiB
Furthermore, to answer the question at the bottom of your email,
http://birzan.org/capabilities1.txt has this info.
and proc4.png would _ideally_ result in:
$ virsh nodeinfo
CPU model: x86_64
CPU(s): 24
CPU frequency: 2200 MHz
CPU socket(s): 2
Core(s) per socket: 12
Thread(s) per core: 1
NUMA cell(s): 4
Memory size: 8047272 KiB
[root@host34 ~]# virsh nodeinfo
CPU model: x86_64
CPU(s): 24
CPU frequency: 2200 MHz
CPU socket(s): 2
Core(s) per socket: 6
Thread(s) per core: 1
NUMA cell(s): 1
Memory size: 131971020 kB
And http://birzan.org/capabilities4.txt
This is on 0.9.11.5, on 0.10.2:
[root@host34 libvirt]# virsh nodeinfo
CPU model: x86_64
CPU(s): 24
CPU frequency: 2200 MHz
CPU socket(s): 1
Core(s) per socket: 6
Thread(s) per core: 1
NUMA cell(s): 4
Memory size: 131971020 KiB
and http://birzan.org/capabilities4-newlibvirt.txt
This has made the problem even worse, as we now can only use 6 cores out of
24 by default (libvirt pins qemus to the CPUs is sees available, so we have
to manually taskset them after starting).
I think the CPU _is_ reporting the complete NUMA topology through
sysfs,
but that we are probably consolidating information from the wrong files
and therefore getting confused.
The sysfs is tarred up in http://birzan.org/sysdevicessystem.tar.gz for
host29 (the
one with 1 numa cell in capabilities) and
http://birzan.org/sysdevicessystem-34.tar.gz
Also, what does the 'virsh capabilities' report for the
<topology>
section? Whereas 'virsh nodeinfo' is constrained by back-compat to give
a lame answer for number of NUMA cells, at least 'virsh capabilities'
should be showing a reasonable representation of the machine's topology.
I answered this above. If you need any more info, feel free to ask.
--
George-Cristian Bîrzan
Attachments:
- attachment.html (text/html — 4.9 KB)