Re: [libvirt] [PATCHv5 10/19] util: Introduce default monitor

Answers refined. On 10/11/2018 3:14 AM, John Ferlan wrote: Got. Got Rewording like these: * There are two type of monitors, the default monitor and the non-default * monitor. Within one allocation, up to one default monitor and more than * one non-default monitors could be created. * * The flag 'default_monitor' defined in structure virResctrlMonitor denotes * if a monitor is the default one or not. Make you confused, my bad. I was trying to introducing the situation of '/sys/fs/resctrl' filesystem and what I was done here at the same time. Regarding the XML layout for default monitor, following XML lines represents a default monitor, the key rule is the default monitor's <monitor> entry has the same 'vcpus' setting as <cachetune>. <cachetune vcpus='0-1'> <monitor level='3' vcpus='0-1'/> </cachetune> and following example illustrates a <cachetune> with two monitors, one of them is a default monitor <cachetune vcpus='0-1'> <monitor level='3' vcpus='0-1'/> <monitor level='3' vcpus='0'/> </cachetune> Particularly, following XML layout doesn't define any monitor or allocation. Following XML lines does not have any effect, and does not indicate an error. <cachetune vcpus='0-1'/> or <cachetune vcpus='0-1'> </cachetune> Thanks for catching my bug. I did not run the code on my machine with default monitor, because I have trouble in run libvirt with CAT, creating non-default resctrl allocation. I am working on it, and will more tests for monitor. Using following code to fix it: (also changed next lines....) if (monitor->alloc) alloc_path = monitor->alloc->path; else alloc_path = SYSFS_RESCTRL_PATH; if (monitor->default_monitor) { if (VIR_STRDUP(monitor->path, alloc_path) < 0) return -1; return 0; } Yes. Two defaults. But I think the <monitor .. > entry should be placed under <cachetune> entry. I'd like to rewrite your configuration in following way: <cachetune vcpus='0-1'> <monitor levels='3' vcpus='0-1'/> <cachetune/> <cachetune vcpus='2-3'> <monitor levels='3' vcpus='2-3'/> <cachetune/> upper configuration creates two allocations and one default monitor for each allocation. By the way <cachetune vcpus='0-1' /> has no effect for resctrl but not be considered as an error. Unlike default allocation, default monitor could be created for multiple times in scope of system, but with a limitation that you can only create one default monitor for one allocation at most. There is only one default allocation in whole system. Every monitor is linked with some specific allocation, either default allocation or non-default allocation (or just 'allocation'). The upper manner of monitor is determined by kernel's 'resctrl' file system. If I created an allocation by creating a directory 'p0' under '/sys/fs/resctrl/', and after adding two vcpus' PID to file '/sys/fs/resctrl/p0/tasks' and making some changes to 'sys/fs/resctrl/p0/schemata' file, then the allocation is functional for resource limitation. The libvirt CAT code is doing similar operations for VM in an automatic way. Let me show the files under '/sys/fs/resctrl/p0': . ├── cpus ├── cpus_list ├── mon_data │ ├── mon_L3_00 │ │ ├── llc_occupancy │ │ ├── mbm_local_bytes │ │ └── mbm_total_bytes │ └── mon_L3_01 │ ├── llc_occupancy │ ├── mbm_local_bytes │ └── mbm_total_bytes ├── mon_groups ├── schemata └── tasks We can find some files for the monitoring role, the 'llc_occupancy' file, the 'mbm_local_bytes' file and the 'mbm_total_bytes' file. The truth is 'resctrl' fs create a monitoring group for each allocation group along with its creation, and this monitoring group is what I introduced default monitor. The default monitor shares the same 'tasks' file with allocation, so it monitors the resource utilization for all pids existing in 'tasks' file. Since this monitoring group is created whenever creating a libvirt allocation, in the design of libvirt resctrl monitor, I choose to make it shown not shown according to the XML configuration. To create other monitoring groups just making sub-directories under the 'mon_group' directory, and adding corresponding vcpu PID to that sub-directory's 'tasks' file. The virResctrlMonitorCreate function creates this kind of sub-directory under this 'mon_group' directory for non-default monitor. For non-default monitor, you can specify a subset of pids of that in allocation 'tasks' file, and no pid overlap allowed between non-default monitors. For an allocation with one default monitor and a non-default monitor (or just 'monitor' in wording), the files layout are like these: . ├── cpus ├── cpus_list ├── mon_data <--- default monitor interface │ ├── mon_L3_00 │ │ ├── llc_occupancy │ │ ├── mbm_local_bytes │ │ └── mbm_total_bytes │ └── mon_L3_01 │ ├── llc_occupancy │ ├── mbm_local_bytes │ └── mbm_total_bytes ├── mon_groups │ └── mon1 │ ├── cpus │ ├── cpus_list │ ├── mon_data <--- non-default monitor interface │ │ ├── mon_L3_00 │ │ │ ├── llc_occupancy │ │ │ ├── mbm_local_bytes │ │ │ └── mbm_total_bytes │ │ └── mon_L3_01 │ │ ├── llc_occupancy │ │ ├── mbm_local_bytes │ │ └── mbm_total_bytes │ └── tasks ├── schemata └── tasks This patch is trying to let monitor has the capability to mark a monitor as a default monitor, and the default monitor is 'physically' existed in kernel 'resctrl' file system, and which has a different manner with other monitors. default monitor has a different role with other monitors, hope I have documented it clearly. Without identifying the default monitor, all monitors will be create under allocation's 'mon_group' directory, the following configuration will not be supported due to overlap between monitors. <cachetune vcpus='0-1'> <monitor level='3' vcpus='0-1'/> <monitor level='3' vcpus='0'/> </cachetune> So default monitor is valuable if you get to know the backend mechanisim of kernel resctrl file system. Thanks for review. Huaqiang