Re: [libvirt] [RFC PATCH 0/3] RFC for X86 RDT Cache Monitoring Technology (CMT) support

Please see my inline reply. easy to parse and understand. Your arrangement by separating group vcpus and group resource value is much better than my version, thanks for suggestion. By the way, I may omit the output of 'rdt.cmt.total', reason is if not all domain's vcpus are covered in the resctrl monitoring groups, the 'rdt.cmt.total' may be confusing, either mean providing whole domain's resource utilization information or a sum of created groups resource utilization. If user want a sum of resource for currently enabled CMT monitoring groups, user can add them by themselves. If user wants whole domain's number, create groups covering all vcpus. OK. I'll update this part of discussion in my next version RFC as well as the cover letter of the POC code. I am bad at naming too :) I agree that 'RDT' and 'resctrl' are pretty confusing for system administrator from the name. But 'Resource Control' or 'Monitoring' is not good choice either, in my opinion. These two phrases have too big scope varying from network resource to memory (DRAM) resource as well as some other resources. Here only focus on CPU resources, currently, the last level cache and memory bandwidth, I would like to use 'cpu-resouce' or 'cpures' as the name for general RDT feature enabling. how about the interfaces shown below: 1. A virsh command 'cpu-resource' for checking domain associated resctrl resource group status and creating/setting resctrl monitoring group in granularity of vcpu. there command may show like this virsh cpu-resource --create <resource type> --destroy <resource type> --vcpulist <vcpulist> --group-name <resctrl group name> *. Using '--create' and '--destroy' to substitute '--enable' and '--disable' that I proposed in my last update. create and destroy are more accurate as the operation action is accurately set up and delete resource groups. *. for <resource type>, here will specify 'monitoring' for feature both CMT and MBM. CAT and MBA could be supported here if it is planned to created function which is similar to command 'cachetune' or 'membwtune' here, his parameter is also extensible for future CPU resources. *. For <vcpulist> it specify the associated vcpu list for a cpu resource group. *. For <group-name>, specifies resource group name, if creating monitoring group for specific vcpu list, a null string for this is expected to match the virResctrlAllocPtr->id string. This argument is also extensible to support some other features, e.g. create an monitoring group for emulator threads with a specific group name, such as 'emulator'. Resource monitoring group for iothread could be done by leveraging 'group-name' argument in a similar way. 2. An update for virsh command 'domstats'. Followed your suggestion you provided in upper discussion, the output related to rdt is like these: cpu-resource.cache-occupancy.group0.value=104331 cpu-resource.cache-occupancy.group0.vcpus=0 cpu-resource.cache-occupancy.group0.value=203200 cpu-resource.cache-occupancy.group0.vcpus=1-2 cpu-resource.cache-occupancy.group0.value=340129 cpu-resource.cache-occupancy.group0.vcpus=3 later for mbm, these outputs would be cpu-resource.memory-bandwidth.group0.value=10331 cpu-resource.memory-bandwidth.group0.vcpus=0 cpu-resource.memory-bandwidthy.group0.value=2000 cpu-resource.memory-bandwidth.group0.vcpus=1-2 To not make user confusing, here changing 'rdt-monitoring' to 'monitoring'. Since 'monitoring' is a sub-node of 'cputune', it obviously means CPU(tune) related 'monitoring'. Also removing attribute 'enable'. The XML configuration would be: <cputune> <cachetune vcpus='1-2'> <cache id='0' level='3' type='both' size='2816' unit='KiB'/> <cache id='1' level='3' type='both' size='2816' unit='KiB'/> </cachetune> <monitoring vcpu='0'> <monitoring vcpu='1-2'> <monitoring vcpu='3'> </cputune> Similar to def->ncachetunes and def->cachetunes, def->nmongroups and def->mongroups are created to preserve monitoring group settings. For the XML changes part, not understand. If we want the feature to save and create some resource groups at domain startup, the XML is the place for keeping the configuration. Do you still want to remove my XML changes after I removed 'enable' attribute? I haven't taken vcpu hotplug in consideration. It may cause some trouble to libvirt RDT function, both resource monitoring and allocation part, because the vcpu thread may be destroyed after command of setvcpu, if rescource control interface does not aware that, it will cause some miss-match, e.g. resource group sub-directory exists but it doesn't work well for missing disappeared vcpu thread. If resource group live change interface (CMT and CAT) exists we could ask user to destroy resource groups first. Or we define the rule that if you want to change vcpu count lively, your resource groups, both allocation and monitoring groups, will disappear. I have no special consideration for emulation threads and io threads. I don't find there is any special action for emulator and io threads by reading CAT source codes, do I miss any part of it? Anyway, do we need this feature right now? You proposed a command, 'cachetune', for a live change of cache allocations, that should be fine to implement, I will involve some code to implement the live change of monitoring groups in my next POC code. Maybe I could submit a patch for 'cachetune' after this RDT monitoring feature.