Hi All,

Some Intel processor families (e.g. the Intel Xeon processor E5 v3 family) introduced some PQos (Platform Qos) features to monitor or control shared resource.

  - CMT (Cache Monitoring Technology): measure the usage of cache by applications running on the platform.

  - CAT (Cache Allocation Technology): enable an OS or Hypervisor/VMM to specify the amount of cache space into which an application can fill.

  - MBM (Memory Bandwidth Monitoring): build on the CMT infrastructure to allow monitoring of bandwidth from one level of the cache hierarchy to the next – in this case focusing on the L3 cache, which is typically backed directly by system memory. As a result of this implementation, memory bandwidth can be monitored.

For more information, see “Intel 64 and IA-32 Architectures Software Developer’s Manual”.

Among these PQos features, currently CMT patches has been merged into Linux kernel mainline. So this patch series proposed in this mail will focus on CMT feature support in libvirt.

At the library API layer, I plan on adding cache related field into virDomainInfo and virNodeInfo:

Add “cache” member into virNodeInfo to get total size of cache in one node.

struct virNodeInfo {

    …

    unsigned int cores;   /* number of cores per socket, total number of

                             processors in case of unusual NUMA topology*/

    unsigned int threads; /* number of threads per core, 1 in case of

                             unusual numa topology */

+  unsigned int cache;   /* cache size in bytes */

};

Add “cacheOcc” member into virDomainInfo to get cache occupancy for one VM.

struct virDomainInfo {

    …

    unsigned short nrVirtCpu;   /* the number of virtual CPUs for the domain */

    unsigned long long cpuTime; /* the CPU time used in nanoseconds */

+  unsigned long long cacheOcc; /* the cache occupancy in Bytes */

};

With this change, those applications like OpenStack based on libvirt can collect cache usage for metering and monitoring (e.g. any VM using more than X% of cache).

Because CMT implementation in Linux kernel is based on perf mechanism and there is no nice and public API library for perf, I have to wrapper system call for perf in libvirt to use CMT feature, and enable/disable perf event for CMT when VM is created/destroyed.

Any thoughts on this plan before I start submitting code patches? Do you think whether I need add several new APIs and XML element for CMT feature (or more PQos features)?

Thanks,

Qiaowei