4:06 p.m.
Hi,
Would it be possible to augment the output produced in virsh
capabilities to also provide the hyperthreaded sibling of a processor?
For example the information would look something like:
<topology>
<cells num='2'>
<cell id='0'>
<cpus num='8'>
<cpu id='1' sibling='9'/>
<cpu id='3' sibling='11'/>
<cpu id='5' sibling='13'/>
<cpu id='7' sibling='15'/>
<cpu id='9' sibling='`1'/>
<cpu id='11' sibling='3'/>
<cpu id='13' sibling='5'/>
<cpu id='15' sibling='7'/>
</cpus>
</cell>
<cell id='1'>
<cpus num='8'>
<cpu id='0' sibling='8'/>
<cpu id='2' sibling='10'/>
<cpu id='4' sibling='12'/>
<cpu id='6' sibling='14'/>
<cpu id='8' sibling='0'/>
<cpu id='10' sibling='2'/>
<cpu id='12' sibling='4'/>
<cpu id='14' sibling='6'/>
</cpus>
</cell>
</cells>
</topology>
I notice that you guys are actually using some of the numactl stuff
(numa.h) to query for the cpus on a given node and I have combed
through that code to see if it provided any info about hyperthreaded
siblings but from what I can tell it does not. I have tinkered around
with the hwloc libraries (http://www.open-mpi.org/projects/hwloc/) and
have written a short program to print out each logical cpu and its
hyperthreaded sibling (see attached)
The output from it looks like so:
[root@hostname ~]# ./a.out
*** PU: 0 Hyperthreaded Sibling: 8
*** PU: 8 Hyperthreaded Sibling: 0
*** PU: 2 Hyperthreaded Sibling: 10
*** PU: 10 Hyperthreaded Sibling: 2
*** PU: 4 Hyperthreaded Sibling: 12
*** PU: 12 Hyperthreaded Sibling: 4
*** PU: 6 Hyperthreaded Sibling: 14
*** PU: 14 Hyperthreaded Sibling: 6
*** PU: 1 Hyperthreaded Sibling: 9
*** PU: 9 Hyperthreaded Sibling: 1
*** PU: 3 Hyperthreaded Sibling: 11
*** PU: 11 Hyperthreaded Sibling: 3
*** PU: 5 Hyperthreaded Sibling: 13
*** PU: 13 Hyperthreaded Sibling: 5
*** PU: 7 Hyperthreaded Sibling: 15
*** PU: 15 Hyperthreaded Sibling: 7
Would you guys ever consider adding something like this? I would be
willing to help create a patch if I had some guidance.
Thanks for your time,
Dusty Mabe
4:34 p.m.
On 10/17/2012 03:06 PM, Dusty Mabe wrote:
Hi,
Would it be possible to augment the output produced in virsh
capabilities to also provide the hyperthreaded sibling of a processor?
For example the information would look something like:
<topology>
<cells num='2'>
<cell id='0'>
<cpus num='8'>
<cpu id='1' sibling='9'/>
Indeed, that sounds useful, and it is strictly additive so there is no
technical reason preventing that addition.
I notice that you guys are actually using some of the numactl stuff
(numa.h) to query for the cpus on a given node and I have combed
through that code to see if it provided any info about hyperthreaded
siblings but from what I can tell it does not. I have tinkered around
with the hwloc libraries (http://www.open-mpi.org/projects/hwloc/) and
have written a short program to print out each logical cpu and its
hyperthreaded sibling (see attached)
We haven't used hwloc before, but if that is the only way to get at the
sibling information, it sounds like it might be a reasonable additional
requirement for libvirt.
On Fedora 17, I see:
hwloc x86_64 1.4.1-2.fc17 updates
1.1 M
hwloc-devel x86_64 1.4.1-2.fc17 updates
150 k
so it looks like -lhwloc is currently bundled alongside the hwloc
command line executable; the library itself is decently small:
-rwxr-xr-x. 1 root root 163168 Apr 18 16:27 /usr/lib64/libhwloc.so.5.0.1
so I wonder if Fedora packaging should split that up so we only drag in
the library instead of the whole command line utility. But that's a
concern for Fedora, not this list.
Would you guys ever consider adding something like this? I would be
willing to help create a patch if I had some guidance.
Absolutely! It sounds like there's several major places to touch to add
this in:
configure.ac - check for presence of hwloc
libvirt.spec.in - require hwloc on new-enough Fedora
docs - document the new XML
docs/schemas - accept the new XML during RNG validation
src/nodeinfo.c - use the new library, when availalble, to get at the new
information
tests - add a new xml file that demonstrates the RNG changes accept the
new XML
You may find it helpful to read http://libvirt.org/hacking.html
--
Eric Blake eblake(a)redhat.com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
12:57 p.m.
On Wed, Oct 17, 2012 at 5:34 PM, Eric Blake <eblake(a)redhat.com> wrote:
On 10/17/2012 03:06 PM, Dusty Mabe wrote:
> Hi,
>
> Would it be possible to augment the output produced in virsh
> capabilities to also provide the hyperthreaded sibling of a processor?
> For example the information would look something like:
>
>
> <topology>
>
> <cells num='2'>
> <cell id='0'>
> <cpus num='8'>
> <cpu id='1' sibling='9'/>
Indeed, that sounds useful, and it is strictly additive so there is no
technical reason preventing that addition.
Along the same lines, I am considering adding numacell memory
information to this output. Some options for formatted output would
be:
<topology>
<cells num='2'>
<cell id='0' memoryKB='8386504'>
<cpus num='8'>
<cpu id='1' sibling='9'/>
.
.
</cpus>
</cell>
</cells>
</topology>
or, more similar to the guest xml
<topology>
<cells num='2'>
<cell id='0'>
<memory unit='KiB'>8386504</memory>
<cpus num='8'>
<cpu id='1' sibling='9'/>
.
.
</cpus>
</cell>
</cells>
</topology>
There are a lot of options here for how to format the output. I don't
anticipate the unit of KiB to change since that is what is in the
/sys/devices/system/node/node0/meminfo files.
Thoughts?
Dusty
6:32 p.m.
On 10/29/2012 11:57 AM, Dusty Mabe wrote:
Along the same lines, I am considering adding numacell memory
information to this output. Some options for formatted output would
be:
<topology>
<cells num='2'>
<cell id='0' memoryKB='8386504'>
memoryKB is an awkward attribute name.
or, more similar to the guest xml
<topology>
<cells num='2'>
<cell id='0'>
<memory unit='KiB'>8386504</memory>
<cpus num='8'>
Much nicer - and because it resembles guest xml it can reuse code.
There are a lot of options here for how to format the output. I don't
anticipate the unit of KiB to change since that is what is in the
/sys/devices/system/node/node0/meminfo files.
We've already used KiB for memory everywhere else in our XML, so it
actually works out nicely.
Thoughts?
Sounds like a worthwhile idea; I'm looking forward to seeing what
patches you come up with.
--
Eric Blake eblake(a)redhat.com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
12:24 p.m.
On Tue, Oct 30, 2012 at 7:32 PM, Eric Blake <eblake(a)redhat.com> wrote:
On 10/29/2012 11:57 AM, Dusty Mabe wrote:
>
>
> Thoughts?
Sounds like a worthwhile idea; I'm looking forward to seeing what
patches you come up with.
I have added thread_siblings, core_siblings, and total memory
information to the virsh capabilities output for a NUMA Cell. In my
opinion the most important of these are the thread_siblings and memory
information, but since it was easy to add, I also included the
core_sibling information.
The problem with this is that some topologies will most likely have
redundant information (see core_siblings below) :
<cell id='0'>
<memory unit='KiB'>12572412</memory>
<cpus num='12'>
<cpu id='0' thread_siblings='0,12'
core_siblings='0,2,4,6,8,10,12,14,16,18,20,22'/>
<cpu id='2' thread_siblings='2,14'
core_siblings='0,2,4,6,8,10,12,14,16,18,20,22'/>
<cpu id='4' thread_siblings='4,16'
core_siblings='0,2,4,6,8,10,12,14,16,18,20,22'/>
<cpu id='6' thread_siblings='6,18'
core_siblings='0,2,4,6,8,10,12,14,16,18,20,22'/>
.
.
</cpus>
</cell>
As I don't exactly have a need for the core_siblings information right
now I will probably just revert it to just have the memory and
thread_sibling information unless others think i should leave it in.
Dusty
4399
days inactive
4420
days old
4 comments
2 participants
participants (2)
-
Dusty Mabe -
Eric Blake