On 09/08/2017 08:47 AM, Wim Ten Have wrote:
From: Wim ten Have <wim.ten.have@oracle.com>
Add libvirtd NUMA cell domain administration functionality to describe underlying cell id sibling distances in full fashion when configuring HVM guests.
May I suggest wording this paragraph as: Add support for describing sibling vCPU distances within a domain's vNUMA cell configuration.
Schema updates are made to docs/schemas/cputypes.rng enforcing domain administration to follow the syntax below the numa cell id and docs/schemas/basictypes.rng to add "numaDistanceValue".
I'm not sure this paragraph is needed in the commit message.
A minimum value of 10 representing the LOCAL_DISTANCE as 0-9 are reserved values and can not be used as System Locality Distance Information. A value of 20 represents the default setting of REMOTE_DISTANCE where a maximum value of 255 represents UNREACHABLE.
Effectively any cell sibling can be assigned a distance value where practically 'LOCAL_DISTANCE <= value <= UNREACHABLE'.
[below is an example of a 4 node setup]
<cpu> <numa> <cell id='0' cpus='0' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='10'/> <sibling id='1' value='21'/> <sibling id='2' value='31'/> <sibling id='3' value='41'/> </distances> </cell> <cell id='1' cpus='1' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='21'/> <sibling id='1' value='10'/> <sibling id='2' value='31'/> <sibling id='3' value='41'/> </distances> </cell> <cell id='2' cpus='2' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='31'/> <sibling id='1' value='21'/> <sibling id='2' value='10'/> <sibling id='3' value='21'/> </distances> <cell id='3' cpus='3' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='41'/> <sibling id='1' value='31'/> <sibling id='2' value='21'/> <sibling id='3' value='10'/> </distances> </cell> </numa> </cpu>
How would this look when having more than one cpu in a cell? I suppose something like <cpu> <numa> <cell id='0' cpus='0-3' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='10'/> <sibling id='1' value='10'/> <sibling id='2' value='10'/> <sibling id='3' value='10'/> <sibling id='4' value='21'/> <sibling id='5' value='21'/> <sibling id='6' value='21'/> <sibling id='7' value='21'/> </distances> </cell> <cell id='1' cpus='4-7' memory='2097152' unit='KiB'> <distances> <sibling id='0' value='21'/> <sibling id='1' value='21'/> <sibling id='2' value='21'/> <sibling id='3' value='21'/> <sibling id='4' value='10'/> <sibling id='5' value='10'/> <sibling id='6' value='10'/> <sibling id='7' value='10'/> </distances> </cell> </numa> </cpu> In the V3 thread you mentioned "And to reduce even more we could also remove LOCAL_DISTANCES as they make a constant factor where; (cell_id == sibling_id)". In the above example cell_id 1 == sibling_id 1, but it is not LOCAL_DISTANCE.
Whenever a sibling id the cell LOCAL_DISTANCE does apply and for any sibling id not being covered a default of REMOTE_DISTANCE is used for internal computations.
I'm having a hard time understanding this sentence... I didn't look closely at the patch since I'd like to understand how multi-cpu cells are handled before doing so. Regards, Jim