[libvirt] Add hyperthreaded sibling info to virsh capabilities
Hi, Would it be possible to augment the output produced in virsh capabilities to also provide the hyperthreaded sibling of a processor? For example the information would look something like: <topology> <cells num='2'> <cell id='0'> <cpus num='8'> <cpu id='1' sibling='9'/> <cpu id='3' sibling='11'/> <cpu id='5' sibling='13'/> <cpu id='7' sibling='15'/> <cpu id='9' sibling='`1'/> <cpu id='11' sibling='3'/> <cpu id='13' sibling='5'/> <cpu id='15' sibling='7'/> </cpus> </cell> <cell id='1'> <cpus num='8'> <cpu id='0' sibling='8'/> <cpu id='2' sibling='10'/> <cpu id='4' sibling='12'/> <cpu id='6' sibling='14'/> <cpu id='8' sibling='0'/> <cpu id='10' sibling='2'/> <cpu id='12' sibling='4'/> <cpu id='14' sibling='6'/> </cpus> </cell> </cells> </topology> I notice that you guys are actually using some of the numactl stuff (numa.h) to query for the cpus on a given node and I have combed through that code to see if it provided any info about hyperthreaded siblings but from what I can tell it does not. I have tinkered around with the hwloc libraries (http://www.open-mpi.org/projects/hwloc/) and have written a short program to print out each logical cpu and its hyperthreaded sibling (see attached) The output from it looks like so: [root@hostname ~]# ./a.out *** PU: 0 Hyperthreaded Sibling: 8 *** PU: 8 Hyperthreaded Sibling: 0 *** PU: 2 Hyperthreaded Sibling: 10 *** PU: 10 Hyperthreaded Sibling: 2 *** PU: 4 Hyperthreaded Sibling: 12 *** PU: 12 Hyperthreaded Sibling: 4 *** PU: 6 Hyperthreaded Sibling: 14 *** PU: 14 Hyperthreaded Sibling: 6 *** PU: 1 Hyperthreaded Sibling: 9 *** PU: 9 Hyperthreaded Sibling: 1 *** PU: 3 Hyperthreaded Sibling: 11 *** PU: 11 Hyperthreaded Sibling: 3 *** PU: 5 Hyperthreaded Sibling: 13 *** PU: 13 Hyperthreaded Sibling: 5 *** PU: 7 Hyperthreaded Sibling: 15 *** PU: 15 Hyperthreaded Sibling: 7 Would you guys ever consider adding something like this? I would be willing to help create a patch if I had some guidance. Thanks for your time, Dusty Mabe
On 10/17/2012 03:06 PM, Dusty Mabe wrote:
Hi,
Would it be possible to augment the output produced in virsh capabilities to also provide the hyperthreaded sibling of a processor? For example the information would look something like:
<topology>
<cells num='2'> <cell id='0'> <cpus num='8'> <cpu id='1' sibling='9'/>
Indeed, that sounds useful, and it is strictly additive so there is no technical reason preventing that addition.
I notice that you guys are actually using some of the numactl stuff (numa.h) to query for the cpus on a given node and I have combed through that code to see if it provided any info about hyperthreaded siblings but from what I can tell it does not. I have tinkered around with the hwloc libraries (http://www.open-mpi.org/projects/hwloc/) and have written a short program to print out each logical cpu and its hyperthreaded sibling (see attached)
We haven't used hwloc before, but if that is the only way to get at the sibling information, it sounds like it might be a reasonable additional requirement for libvirt. On Fedora 17, I see: hwloc x86_64 1.4.1-2.fc17 updates 1.1 M hwloc-devel x86_64 1.4.1-2.fc17 updates 150 k so it looks like -lhwloc is currently bundled alongside the hwloc command line executable; the library itself is decently small: -rwxr-xr-x. 1 root root 163168 Apr 18 16:27 /usr/lib64/libhwloc.so.5.0.1 so I wonder if Fedora packaging should split that up so we only drag in the library instead of the whole command line utility. But that's a concern for Fedora, not this list.
Would you guys ever consider adding something like this? I would be willing to help create a patch if I had some guidance.
Absolutely! It sounds like there's several major places to touch to add this in: configure.ac - check for presence of hwloc libvirt.spec.in - require hwloc on new-enough Fedora docs - document the new XML docs/schemas - accept the new XML during RNG validation src/nodeinfo.c - use the new library, when availalble, to get at the new information tests - add a new xml file that demonstrates the RNG changes accept the new XML You may find it helpful to read http://libvirt.org/hacking.html -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Wed, Oct 17, 2012 at 5:34 PM, Eric Blake <eblake@redhat.com> wrote:
On 10/17/2012 03:06 PM, Dusty Mabe wrote:
Hi,
Would it be possible to augment the output produced in virsh capabilities to also provide the hyperthreaded sibling of a processor? For example the information would look something like:
<topology>
<cells num='2'> <cell id='0'> <cpus num='8'> <cpu id='1' sibling='9'/>
Indeed, that sounds useful, and it is strictly additive so there is no technical reason preventing that addition.
Along the same lines, I am considering adding numacell memory information to this output. Some options for formatted output would be: <topology> <cells num='2'> <cell id='0' memoryKB='8386504'> <cpus num='8'> <cpu id='1' sibling='9'/> . . </cpus> </cell> </cells> </topology> or, more similar to the guest xml <topology> <cells num='2'> <cell id='0'> <memory unit='KiB'>8386504</memory> <cpus num='8'> <cpu id='1' sibling='9'/> . . </cpus> </cell> </cells> </topology> There are a lot of options here for how to format the output. I don't anticipate the unit of KiB to change since that is what is in the /sys/devices/system/node/node0/meminfo files. Thoughts? Dusty
On 10/29/2012 11:57 AM, Dusty Mabe wrote:
Along the same lines, I am considering adding numacell memory information to this output. Some options for formatted output would be:
<topology> <cells num='2'> <cell id='0' memoryKB='8386504'>
memoryKB is an awkward attribute name.
or, more similar to the guest xml
<topology> <cells num='2'> <cell id='0'> <memory unit='KiB'>8386504</memory> <cpus num='8'>
Much nicer - and because it resembles guest xml it can reuse code.
There are a lot of options here for how to format the output. I don't anticipate the unit of KiB to change since that is what is in the /sys/devices/system/node/node0/meminfo files.
We've already used KiB for memory everywhere else in our XML, so it actually works out nicely.
Thoughts?
Sounds like a worthwhile idea; I'm looking forward to seeing what patches you come up with. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Tue, Oct 30, 2012 at 7:32 PM, Eric Blake <eblake@redhat.com> wrote:
On 10/29/2012 11:57 AM, Dusty Mabe wrote:
Thoughts?
Sounds like a worthwhile idea; I'm looking forward to seeing what patches you come up with.
I have added thread_siblings, core_siblings, and total memory information to the virsh capabilities output for a NUMA Cell. In my opinion the most important of these are the thread_siblings and memory information, but since it was easy to add, I also included the core_sibling information. The problem with this is that some topologies will most likely have redundant information (see core_siblings below) : <cell id='0'> <memory unit='KiB'>12572412</memory> <cpus num='12'> <cpu id='0' thread_siblings='0,12' core_siblings='0,2,4,6,8,10,12,14,16,18,20,22'/> <cpu id='2' thread_siblings='2,14' core_siblings='0,2,4,6,8,10,12,14,16,18,20,22'/> <cpu id='4' thread_siblings='4,16' core_siblings='0,2,4,6,8,10,12,14,16,18,20,22'/> <cpu id='6' thread_siblings='6,18' core_siblings='0,2,4,6,8,10,12,14,16,18,20,22'/> . . </cpus> </cell> As I don't exactly have a need for the core_siblings information right now I will probably just revert it to just have the memory and thread_sibling information unless others think i should leave it in. Dusty
participants (2)
-
Dusty Mabe -
Eric Blake