[libvirt] [PATCH 00/21] Initial support for resctrl (CAT)

This is an initial support for resctrl, so that domains can make use of Cache Allocation Technology (🐈). After bunch of initial discussions I followed most of those things except one detail. In order to distinguish different allocations while minimizing the number of them at the same time, I changed the XML design, but only slightly. See docs in patch 17 for detail. There is still much missing, but all is designed in a way that should not collied with the future additions. Missing features include: - After allocation is created the file should be re-read and parsed into the al the allocation so that the masks reflect what kernel actually uses (just in case there was a change) - When formatting live XML, used masks for allocations should be formatted as well - Each <cache/> should support shared allocations, for now every single allocation is exclusive only - Suuport for different backing, particularly if allocation has type='both' (code as well as data) and the host supports CDP (Code Data Partitioning), meaning it can allocate code and data portions separately the code currently fails even though it is possible to convert such allocation. - APIs for manipulation with various resctrl groups. What's needed is reading and setting values of the default group as well as any cachetune of a domain. Martin Kletzander (21): Only output initialized capabilities with VIR_TEST_DEBUG > 1 util: Introduce virPrettySize util: Make prefix optional in virBitampString util: Rename virBitmapString to virBitmapToString util: Rename virBitmapDataToString to virBitmapDataFormat util: Don't output too many zeros from virBitmapToString util: Introduce virBitmapNewString util: Reintroduce virBitmapSubtract util: Introduce virBitmapShrink conf: Sort cache banks in capabilities XML conf: Format cache banks in capabilities with virPrettySize resctrl: Instantiate all resctrl information at once tests: Remove executable bits on plain data files tests: Change some schemata for the default group caps2xml resctrl-skx-twocaches resctrl: Add functions to work with resctrl allocations conf: cachetune tests: Minor adjustments for test data tests: Add virresctrltest qemu: Add support for resctrl docs: Add CAT (resctrl) support into news.xml docs/formatdomain.html.in | 24 + docs/news.xml | 11 + docs/schemas/domaincommon.rng | 32 + po/POTFILES.in | 1 + src/Makefile.am | 2 +- src/conf/capabilities.c | 113 +- src/conf/capabilities.h | 4 +- src/conf/domain_conf.c | 249 ++++ src/conf/domain_conf.h | 21 + src/libvirt_private.syms | 24 +- src/qemu/qemu_capabilities.c | 4 +- src/qemu/qemu_process.c | 61 +- src/util/virbitmap.c | 118 +- src/util/virbitmap.h | 15 +- src/util/virresctrl.c | 1317 ++++++++++++++++++-- src/util/virresctrl.h | 81 +- src/util/virresctrlpriv.h | 32 + src/util/virutil.c | 50 + src/util/virutil.h | 3 + tests/Makefile.am | 8 +- .../genericxml2xmlindata/generic-cachetune-cdp.xml | 36 + .../generic-cachetune-colliding-allocs.xml | 30 + .../generic-cachetune-colliding-tunes.xml | 32 + .../generic-cachetune-colliding-types.xml | 30 + .../generic-cachetune-small.xml | 29 + tests/genericxml2xmlindata/generic-cachetune.xml | 33 + tests/genericxml2xmltest.c | 10 + tests/testutils.c | 2 +- tests/virbitmaptest.c | 84 +- tests/vircaps2xmldata/linux-caches/system | 1 + .../linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask | 0 .../resctrl/info/L3CODE/min_cbm_bits | 0 .../resctrl/info/L3CODE/num_closids | 0 .../linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask | 0 .../resctrl/info/L3DATA/min_cbm_bits | 0 .../resctrl/info/L3DATA/num_closids | 0 .../linux-resctrl-cdp/resctrl/schemata | 4 +- .../resctrl/info/L3/cbm_mask | 1 + .../resctrl/info/L3}/min_cbm_bits | 0 .../resctrl/info/L3/num_closids | 1 + .../linux-resctrl-skx-twocaches/resctrl/schemata | 1 + .../resctrl/some_reservation/schemata | 1 + .../system/cpu/cpu0/cache/index0/id | 1 + .../system/cpu/cpu0/cache/index0/level | 1 + .../system/cpu/cpu0/cache/index0/shared_cpu_list | 1 + .../system/cpu/cpu0/cache/index0/shared_cpu_map | 1 + .../system/cpu/cpu0/cache/index0/size | 1 + .../system/cpu/cpu0/cache/index0/type | 1 + .../system/cpu/cpu0/cache/index1/id} | 0 .../system/cpu/cpu0/cache/index1/level | 1 + .../system/cpu/cpu0/cache/index1/shared_cpu_list | 1 + .../system/cpu/cpu0/cache/index1/shared_cpu_map | 1 + .../system/cpu/cpu0/cache/index1/size | 1 + .../system/cpu/cpu0/cache/index1/type | 1 + .../system/cpu/cpu0/online} | 0 .../system/cpu/cpu0/topology/core_id | 1 + .../system/cpu/cpu0/topology/core_siblings | 1 + .../system/cpu/cpu0/topology/core_siblings_list | 1 + .../system/cpu/cpu0/topology/physical_package_id | 1 + .../system/cpu/cpu0/topology/thread_siblings | 1 + .../system/cpu/cpu0/topology/thread_siblings_list | 1 + .../linux-resctrl-skx-twocaches/system/cpu/online | 1 + .../linux-resctrl-skx-twocaches/system/cpu/present | 1 + .../system/node/node0/cpu0 | 1 + .../system/node/node0/cpulist | 1 + .../system/node/node0/cpumap | 1 + .../system/node/node0/distance | 1 + .../linux-resctrl-skx-twocaches/system/node/online | 1 + .../linux-resctrl-skx/resctrl/empty/schemata | 0 .../linux-resctrl-skx/resctrl/schemata | 2 +- .../vircaps2xmldata/linux-resctrl/resctrl/schemata | 2 +- tests/vircaps2xmldata/vircaps-x86_64-caches.xml | 2 +- .../vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml | 4 +- ...ml => vircaps-x86_64-resctrl-skx-twocaches.xml} | 7 +- .../vircaps2xmldata/vircaps-x86_64-resctrl-skx.xml | 4 +- tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml | 4 +- tests/vircaps2xmltest.c | 1 + .../resctrl--cachetune/vcpus-0-1.alloc | 1 + .../resctrl--cachetune/vcpus-3.alloc | 1 + .../resctrl-cdp--cachetune-cdp/vcpus-0-1.alloc | 2 + .../resctrl-cdp--cachetune-cdp/vcpus-2.alloc | 1 + .../resctrl-cdp--cachetune-cdp/vcpus-3.alloc | 1 + tests/virresctrldata/resctrl-cdp.schemata | 2 + .../virresctrldata/resctrl-skx-twocaches.schemata | 1 + tests/virresctrldata/resctrl-skx.schemata | 1 + tests/virresctrldata/resctrl.schemata | 1 + tests/virresctrltest.c | 277 ++++ tools/virsh-domain.c | 4 +- tools/virsh-host.c | 2 +- 89 files changed, 2626 insertions(+), 183 deletions(-) create mode 100644 src/util/virresctrlpriv.h create mode 100644 tests/genericxml2xmlindata/generic-cachetune-cdp.xml create mode 100644 tests/genericxml2xmlindata/generic-cachetune-colliding-allocs.xml create mode 100644 tests/genericxml2xmlindata/generic-cachetune-colliding-tunes.xml create mode 100644 tests/genericxml2xmlindata/generic-cachetune-colliding-types.xml create mode 100644 tests/genericxml2xmlindata/generic-cachetune-small.xml create mode 100644 tests/genericxml2xmlindata/generic-cachetune.xml create mode 120000 tests/vircaps2xmldata/linux-caches/system mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/num_closids mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/min_cbm_bits mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/num_closids create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/resctrl/info/L3/cbm_mask copy tests/vircaps2xmldata/{linux-resctrl-cdp/resctrl/info/L3CODE => linux-resctrl-skx-twocaches/resctrl/info/L3}/min_cbm_bits (100%) mode change 100755 => 100644 create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/resctrl/info/L3/num_closids create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/resctrl/schemata create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/resctrl/some_reservation/schemata create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/id create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/level create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/shared_cpu_list create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/shared_cpu_map create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/size create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/type copy tests/vircaps2xmldata/{linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits => linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/id} (100%) mode change 100755 => 100644 create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/level create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/shared_cpu_list create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/shared_cpu_map create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/size create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/type copy tests/vircaps2xmldata/{linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits => linux-resctrl-skx-twocaches/system/cpu/cpu0/online} (100%) mode change 100755 => 100644 create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/core_id create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/core_siblings create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/core_siblings_list create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/physical_package_id create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/thread_siblings create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/thread_siblings_list create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/online create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/present create mode 120000 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/node/node0/cpu0 create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/node/node0/cpulist create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/node/node0/cpumap create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/node/node0/distance create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/node/online create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx/resctrl/empty/schemata copy tests/vircaps2xmldata/{vircaps-x86_64-resctrl-skx.xml => vircaps-x86_64-resctrl-skx-twocaches.xml} (68%) create mode 100644 tests/virresctrldata/resctrl--cachetune/vcpus-0-1.alloc create mode 100644 tests/virresctrldata/resctrl--cachetune/vcpus-3.alloc create mode 100644 tests/virresctrldata/resctrl-cdp--cachetune-cdp/vcpus-0-1.alloc create mode 100644 tests/virresctrldata/resctrl-cdp--cachetune-cdp/vcpus-2.alloc create mode 100644 tests/virresctrldata/resctrl-cdp--cachetune-cdp/vcpus-3.alloc create mode 100644 tests/virresctrldata/resctrl-cdp.schemata create mode 100644 tests/virresctrldata/resctrl-skx-twocaches.schemata create mode 100644 tests/virresctrldata/resctrl-skx.schemata create mode 100644 tests/virresctrldata/resctrl.schemata create mode 100644 tests/virresctrltest.c -- 2.15.0