hi all
Thanks all for the comment for previous RFC version, I summary all input here also with Opens which I am not sure, can you please review again?
#Propose Changes
## Expose cache information into capabilities
Extend capabilities to expose all level cache resource information and should tell the topology of caches also tell which kinds of resources can be tuned/allocation
There information comes from /sys/devices/system/cpu/
virsh capabilities
<cache>
<bank id='0, 'type="l3" size="56320" units="KiB" cpus="0,1,2,6,7,8"/> <--------------------- level 3 cache is per socket, so group them by socket id
<control unit="KiB" min="2816"/>
<bank id='1', type="l3" size="56320" units="KiB" cpus="3,4,5,9,10,11"/>
<bank id='2' type="l2" size="256" units="KiB" cpus="0"/>
<bank id='3' type="l2" size="256" units="KiB" cpus="1"/>
<bank id='4' type="l2" size="256" units="KiB" cpus="2"/>
<bank id='5' type="l2" size="256" units="KiB" cpus="3"/>
<bank id='6' type="l2" size="256" units="KiB" cpus="4"/>
...
<cache>
Opens
1. how about add socket id to bank for bank type = l3 ?
2. do we really want to expose l2/l3 cache for now , they are per core resource and linux kernel don't support l2 yet (depend no hardware)?
3. if enable CDP in resctrl, for bank type=l3 , it will be split to l3data l3code, should expose this ability.
<bank type="l3" size="56320" units="KiB" cpus="0,1,2,6,7,8"/> <--------------------- level 3 cache is per socket, so group them by socket id
<control unit="KiB" min="2816" cdp="enabled"/>
## Provide a new API to get the avail cache on each bank, such as the output are:
id=0
type=l3
avail=56320
total = ?? <--------- do we need this?
id=1
type=l3
avail=56320
id=3
type=l2
avail=256
Opens:
· Don't expose the avail cache information if the host can not do the allocation of that type cache(eg, for l2 currently) ?
· We can not make all of the cache , the reservation amount is the min_cbm_len (=1) * min_unit .
· do we need to expose total?
## enable CAT for a domain
1 Domain XML changes
<cputune>
<cache id="1" host_id="0" type="l3" size="5632" unit="KiB"/>
<cache id="2" host_id="1" type="l3" size="5632" unit="KiB"/>
<cpu_cache vcpus="0-3" id="1"/>
<cpu_cache vcpus="4-7" id="2"/>
<iothread_cache iothreads="0-1" id="1"/>
<emulator_cache id="2"/>
</cputune>
2. Extend cputune command ?
Opens:
1. Do we accept to extend existed API ? or using new API/virsh?
2. How to calculate cache size -> CBM bit?
eg:
5632/ 2816 = 2 bits
5733/ 2816 = 2 bits or 3 bits?
## Restriction for using cache tune on multiple sockets' host.
The l3 cache is per socket resource, kernel need to know about what's affinity looks like, so for a VM which running on a multiple socket's host, it should have NUMA setting or vcpuset pin setting. Or cache tune will fail.
[1] kernel support https://git.kernel.
[2] libvirt PoC(not finished yet) https://github.com/taget/