[PATCH 0/6] resctrl: support whole-process allocations and monitoring
Resctrl support in libvirt currently requires allocations and monitors to be associated with a set of vCPUs. This does not accurately model QEMU resource consumption, as IOThreads, worker threads and emulator threads also consume cache, memory bandwidth and energy. This series introduces a whole-process resctrl scope, allowing ``cachetune``, ``memorytune``, ``energytune`` and ``monitor`` elements to omit the ``vcpus`` attribute and apply to the entire QEMU emulator process. By placing the emulator process into the resctrl group before thread creation, all subsequently created threads inherit the same resctrl configuration. Major design choices: * A resctrl allocation or monitor without a ``vcpus`` attribute is treated as a whole-process group. * Whole-process allocations and per-vCPU allocations are mutually exclusive within a domain to avoid ambiguous resource assignment. * Whole-process monitors require a whole-process allocation of the same resource type and are unique per resource. * Monitors without ``vcpus`` inherit the scope of their parent allocation. Under a whole-process allocation they monitor the whole process; under a per-vCPU allocation they monitor the allocation's vCPU set. This reflects the dependency between resctrl monitoring and allocation groups. * Whole-process resctrl groups are created by placing the QEMU process into the target group before thread creation, allowing all subsequently created threads to inherit the same resctrl configuration. The series also updates XML validation, fixes resctrl monitor statistics reporting, and cleans up monitor domstats output. Jedrzej Wasiukiewicz (6): conf: allow omitting vcpus in cachetune/memorytune/energytune conf: implement whole-process resctrl scope qemu: assign whole-process resctrl groups at domain start qemu: fix resctrl monitor stats reporting the wrong group name qemu: clean up resctrl monitor domstats NEWS: Document whole-process resctrl support NEWS.rst | 11 + docs/formatdomain.rst | 57 +++-- src/conf/domain_conf.c | 231 ++++++++++++------ src/conf/domain_conf.h | 2 + src/conf/schemas/domaincommon.rng | 48 ++-- src/libvirt_private.syms | 1 + src/qemu/qemu_driver.c | 29 ++- src/qemu/qemu_process.c | 40 ++- src/util/virresctrl.c | 20 +- src/util/virresctrl.h | 3 + .../cachetune-monitor-empty-vcpus.xml | 30 +++ .../cachetune-monitor-inherit-alloc.xml | 30 +++ .../cachetune-wholeprocess-duplicate.xml | 32 +++ ...chetune-wholeprocess-monitor-duplicate.xml | 31 +++ .../cachetune-wholeprocess-monitors.xml | 31 +++ .../energytune-colliding-monitor.xml | 30 +++ .../energytune-wholeprocess.xml | 29 +++ .../memorytune-wholeprocess.xml | 29 +++ .../resctrl-wholeprocess-alloc-monitor.xml | 32 +++ .../resctrl-wholeprocess-layering.xml | 32 +++ .../resctrl-wholeprocess-monitors.xml | 33 +++ .../cachetune-monitor-inherit-alloc.xml | 30 +++ tests/genericxml2xmltest.c | 11 + 23 files changed, 694 insertions(+), 128 deletions(-) create mode 100644 tests/genericxml2xmlindata/cachetune-monitor-empty-vcpus.xml create mode 100644 tests/genericxml2xmlindata/cachetune-monitor-inherit-alloc.xml create mode 100644 tests/genericxml2xmlindata/cachetune-wholeprocess-duplicate.xml create mode 100644 tests/genericxml2xmlindata/cachetune-wholeprocess-monitor-duplicate.xml create mode 100644 tests/genericxml2xmlindata/cachetune-wholeprocess-monitors.xml create mode 100644 tests/genericxml2xmlindata/energytune-colliding-monitor.xml create mode 100644 tests/genericxml2xmlindata/energytune-wholeprocess.xml create mode 100644 tests/genericxml2xmlindata/memorytune-wholeprocess.xml create mode 100644 tests/genericxml2xmlindata/resctrl-wholeprocess-alloc-monitor.xml create mode 100644 tests/genericxml2xmlindata/resctrl-wholeprocess-layering.xml create mode 100644 tests/genericxml2xmlindata/resctrl-wholeprocess-monitors.xml create mode 100644 tests/genericxml2xmloutdata/cachetune-monitor-inherit-alloc.xml -- 2.43.0 --------------------------------------------------------------------- Intel Technology Poland sp. z o.o. ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych. Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.
Make the vcpus attribute optional on cachetune, memorytune and energytune. Not specifying vcpus in the XML makes the allocation/monitoring group affect the whole domain process, relying on the resctrl inheritance mechanism so that all of the process' threads and children end up in the same group. Signed-off-by: Jedrzej Wasiukiewicz <jedrzej.wasiukiewicz@intel.com> --- docs/formatdomain.rst | 57 ++++++++++++++++++------------- src/conf/schemas/domaincommon.rng | 48 ++++++++++++++++---------- 2 files changed, 63 insertions(+), 42 deletions(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 28c9170a37..1f8bf378eb 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -1023,10 +1023,13 @@ CPU Tuning Optional ``cachetune`` element can control allocations for CPU caches using the resctrl on the host. Whether or not is this supported can be gathered from capabilities where some limitations like minimum size and required - granularity are reported as well. The required attribute ``vcpus`` specifies - to which vCPUs this allocation applies. A vCPU can only be member of one - ``cachetune`` element allocation. The vCPUs specified by cachetune can be - identical with those in memorytune, however they are not allowed to overlap. + granularity are reported as well. The optional attribute ``vcpus`` specifies + to which vCPUs this allocation applies. If ``vcpus`` is omitted the allocation + applies to the whole emulator process; the resctrl group is then inherited by + all its threads and child processes. A vCPU can only be member of one + ``cachetune`` element allocation. The scope specified by cachetune + can be identical with those in memorytune, however they are not + allowed to overlap. The optional, output only ``id`` attribute identifies cache uniquely. Supported subelements are: @@ -1053,23 +1056,26 @@ CPU Tuning specified, defaults to bytes. ``monitor`` :since:`Since 4.10.0` - The optional element ``monitor`` creates the cache monitor(s) for current - cache allocation and has the following required attributes: + The optional element ``monitor`` creates the cache monitor(s) for + the enclosing ``cachetune`` allocation. It has the following attributes: ``level`` - Host cache level the monitor belongs to. + Required. Host cache level the monitor belongs to. ``vcpus`` - vCPU list the monitor applies to. A monitor's vCPU list can only be the - member(s) of the vCPU list of the associated allocation. The default - monitor has the same vCPU list as the associated allocation. For - non-default monitors, overlapping vCPUs are not permitted. + Optional. The vCPUs to monitor. Must be a subset of the enclosing + allocation's vCPUs and must not overlap another cache monitor of the + same allocation. A monitor covering the allocation's full vCPU list + reports the allocation as a whole. Omit ``vcpus`` to inherit the + enclosing allocation's scope. ``memorytune`` :since:`Since 4.7.0` Optional ``memorytune`` element can control allocations for memory bandwidth using the resctrl on the host. Whether or not is this supported can be gathered from capabilities where some limitations like minimum bandwidth and - required granularity are reported as well. The required attribute ``vcpus`` - specifies to which vCPUs this allocation applies. A vCPU can only be member + required granularity are reported as well. The optional attribute ``vcpus`` + specifies to which vCPUs this allocation applies. If ``vcpus`` is omitted the + allocation applies to the whole emulator process; the resctrl group is then + inherited by all its threads and child processes. A vCPU can only be member of one ``memorytune`` element allocation. The ``vcpus`` specified by ``memorytune`` can be identical to those specified by ``cachetune``. However they are not allowed to overlap each other. Supported subelements are: @@ -1088,21 +1094,24 @@ CPU Tuning configuration. ``energytune`` :since:`Since 12.4.0` - Optional ``energytune`` element allows to monitor energy consumption using the - resctrl filesystem on the host. Whether or not is this supported can be - gathered from capabilities where number of monitors and available features are - reported. The required attribute ``vcpus`` specifies to which allocation group - this monitor belongs. A vCPU can only be member of one allocation group and monitor - group. The ``vcpus`` specified by ``energytune`` can be identical to those - specified by ``cachetune`` or ``memorytune``. However they are not allowed to - overlap each other. Supported subelements are: + Optional ``energytune`` element defines a group for energy consumption + monitoring using the resctrl filesystem on the host. Whether or not is this + supported can be gathered from capabilities where number of monitors and + available features are reported. The optional attribute ``vcpus`` specifies + which vCPUs form this group. If ``vcpus`` is omitted the group covers the whole + emulator process; the resctrl group is then inherited by all its threads and + child processes. A vCPU can only be member of one ``energytune`` group. The + ``vcpus`` specified by ``energytune`` can be identical to those specified by + ``cachetune`` or ``memorytune``. However they are not allowed to overlap each + other. Supported subelements are: ``monitor`` - The optional element ``monitor`` creates the energy monitor for - this allocation group and has the following required attribute: + The optional element creates the energy monitor for the + enclosing ``energytune`` group. It has the following attribute: ``vcpus`` - vCPU list the monitor applies to. + Optional. The vCPUs to monitor. Omit ``vcpus`` to inherit the enclosing + group's scope. Memory Allocation diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index 81ffbfc2fa..aa51429964 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -1224,9 +1224,11 @@ </optional> <zeroOrMore> <element name="cachetune"> - <attribute name="vcpus"> - <ref name="cpuset"/> - </attribute> + <optional> + <attribute name="vcpus"> + <ref name="cpuset"/> + </attribute> + </optional> <optional> <attribute name="id"> <data type="string"/> @@ -1261,9 +1263,11 @@ <attribute name="level"> <ref name="unsignedInt"/> </attribute> - <attribute name="vcpus"> - <ref name="cpuset"/> - </attribute> + <optional> + <attribute name="vcpus"> + <ref name="cpuset"/> + </attribute> + </optional> </element> </choice> </oneOrMore> @@ -1271,9 +1275,11 @@ </zeroOrMore> <zeroOrMore> <element name="memorytune"> - <attribute name="vcpus"> - <ref name="cpuset"/> - </attribute> + <optional> + <attribute name="vcpus"> + <ref name="cpuset"/> + </attribute> + </optional> <oneOrMore> <choice> <element name="node"> @@ -1285,9 +1291,11 @@ </attribute> </element> <element name="monitor"> - <attribute name="vcpus"> - <ref name="cpuset"/> - </attribute> + <optional> + <attribute name="vcpus"> + <ref name="cpuset"/> + </attribute> + </optional> </element> </choice> </oneOrMore> @@ -1295,9 +1303,11 @@ </zeroOrMore> <zeroOrMore> <element name="energytune"> - <attribute name="vcpus"> - <ref name="cpuset"/> - </attribute> + <optional> + <attribute name="vcpus"> + <ref name="cpuset"/> + </attribute> + </optional> <optional> <attribute name="id"> <data type="string"/> @@ -1305,9 +1315,11 @@ </optional> <oneOrMore> <element name="monitor"> - <attribute name="vcpus"> - <ref name="cpuset"/> - </attribute> + <optional> + <attribute name="vcpus"> + <ref name="cpuset"/> + </attribute> + </optional> </element> </oneOrMore> </element> -- 2.43.0 --------------------------------------------------------------------- Intel Technology Poland sp. z o.o. ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych. Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.
Treat vCPU attribute absence as a request to place the whole emulator process in one resctrl group for both allocation and monitoring. Parsing such XML failed until now. A whole-process group has no vcpu list and no id. It formats back as a bare cachetune, memorytune, energytune or monitor element. Track the scope with a whole_process flag and enforce the rules beside the existing monitor checks. A domain's allocations are either all whole-process or all per-vCPU. A whole-process monitor needs a whole-process allocation, does not mix with explicit monitors and is unique per resource type. A whole-process allocation may still carry explicit per-vCPU monitors, as it already spans every thread. Monitors that omit vcpus inside an explicit allocation inherit the allocation's vcpu scope instead of covering the whole process. This behavior represents the resctrl dependency between allocation and monitoring. Out of range and empty vcpus attributes still remain silently dropped. Signed-off-by: Jedrzej Wasiukiewicz <jedrzej.wasiukiewicz@intel.com> --- src/conf/domain_conf.c | 231 ++++++++++++------ src/conf/domain_conf.h | 2 + .../cachetune-monitor-empty-vcpus.xml | 30 +++ .../cachetune-monitor-inherit-alloc.xml | 30 +++ .../cachetune-wholeprocess-duplicate.xml | 32 +++ ...chetune-wholeprocess-monitor-duplicate.xml | 31 +++ .../cachetune-wholeprocess-monitors.xml | 31 +++ .../energytune-colliding-monitor.xml | 30 +++ .../energytune-wholeprocess.xml | 29 +++ .../memorytune-wholeprocess.xml | 29 +++ .../resctrl-wholeprocess-alloc-monitor.xml | 32 +++ .../resctrl-wholeprocess-layering.xml | 32 +++ .../resctrl-wholeprocess-monitors.xml | 33 +++ .../cachetune-monitor-inherit-alloc.xml | 30 +++ tests/genericxml2xmltest.c | 11 + 15 files changed, 545 insertions(+), 68 deletions(-) create mode 100644 tests/genericxml2xmlindata/cachetune-monitor-empty-vcpus.xml create mode 100644 tests/genericxml2xmlindata/cachetune-monitor-inherit-alloc.xml create mode 100644 tests/genericxml2xmlindata/cachetune-wholeprocess-duplicate.xml create mode 100644 tests/genericxml2xmlindata/cachetune-wholeprocess-monitor-duplicate.xml create mode 100644 tests/genericxml2xmlindata/cachetune-wholeprocess-monitors.xml create mode 100644 tests/genericxml2xmlindata/energytune-colliding-monitor.xml create mode 100644 tests/genericxml2xmlindata/energytune-wholeprocess.xml create mode 100644 tests/genericxml2xmlindata/memorytune-wholeprocess.xml create mode 100644 tests/genericxml2xmlindata/resctrl-wholeprocess-alloc-monitor.xml create mode 100644 tests/genericxml2xmlindata/resctrl-wholeprocess-layering.xml create mode 100644 tests/genericxml2xmlindata/resctrl-wholeprocess-monitors.xml create mode 100644 tests/genericxml2xmloutdata/cachetune-monitor-inherit-alloc.xml diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 45235d74bc..5e8f7ea429 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -18771,15 +18771,17 @@ virDomainDefParseBootOptions(virDomainDef *def, static int virDomainResctrlParseVcpus(virDomainDef *def, xmlNodePtr node, - virBitmap **vcpus) + virBitmap **vcpus, + bool *whole_process) { g_autofree char *vcpus_str = NULL; + *vcpus = NULL; vcpus_str = virXMLPropString(node, "vcpus"); - if (!vcpus_str) { - virReportError(VIR_ERR_XML_ERROR, _("Missing %1$s attribute 'vcpus'"), - node->name); - return -1; + *whole_process = !vcpus_str; + if (*whole_process) { + *vcpus = virBitmapNew(0); + return 0; } if (virBitmapParse(vcpus_str, vcpus, VIR_DOMAIN_CPUMASK_LEN) < 0) { virReportError(VIR_ERR_XML_ERROR, @@ -18859,6 +18861,9 @@ virDomainCachetuneDefParseCache(xmlXPathContextPtr ctxt, /* Checking if the monitor's vcpus and tag is conflicted with existing * allocation and monitors. * + * A whole-process monitor must not be mixed with explicit monitors and may + * cover each resource type only once. + * * Returns 1 if @monitor->vcpus equals to @resctrl->vcpus, then the monitor * will share the underlying resctrl group with @resctrl->alloc. Returns -1 * if any conflict found. Returns 0 if no conflict and @monitor->vcpus is @@ -18875,17 +18880,40 @@ virDomainResctrlValidateMonitor(virDomainResctrlDef *resctrl, bool vcpus_overlap_no_resctrl = false; bool default_alloc_monitor = virResctrlAllocIsEmpty(resctrl->alloc); + if (resctrl->nmonitors > 0 && + resctrl->monitors[0]->whole_process != monitor->whole_process) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("Whole-process and explicit monitors cannot be mixed")); + return -1; + } + + if (monitor->whole_process) { + for (i = 0; i < resctrl->nmonitors; i++) { + if (resctrl->monitors[i]->tag == monitor->tag) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("Duplicate whole-process monitor of the same resource type")); + return -1; + } + } + + return 0; + } + if (virBitmapIsAllClear(monitor->vcpus)) { virReportError(VIR_ERR_INVALID_ARG, "%s", _("vcpus is empty")); return -1; } - while ((vcpu = virBitmapNextSetBit(monitor->vcpus, vcpu)) >= 0) { - if (!virBitmapIsBitSet(resctrl->vcpus, vcpu)) { - virReportError(VIR_ERR_INVALID_ARG, "%s", - _("Monitor vcpus conflicts with allocation")); - return -1; + /* A whole-process allocation covers every thread, so it does not constrain + * an explicit monitor's vcpus. */ + if (!resctrl->whole_process) { + while ((vcpu = virBitmapNextSetBit(monitor->vcpus, vcpu)) >= 0) { + if (!virBitmapIsBitSet(resctrl->vcpus, vcpu)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("Monitor vcpus conflicts with allocation")); + return -1; + } } } @@ -18952,6 +18980,7 @@ virDomainResctrlMonDefParse(virDomainDef *def, for (i = 0; i < n; i++) { g_autofree char *id = NULL; + bool whole_process = false; domresmon = g_new0(virDomainResctrlMonDef, 1); @@ -18977,27 +19006,41 @@ virDomainResctrlMonDefParse(virDomainDef *def, } } - if (virDomainResctrlParseVcpus(def, nodes[i], &domresmon->vcpus) < 0) + if (virDomainResctrlParseVcpus(def, nodes[i], &domresmon->vcpus, + &whole_process) < 0) goto cleanup; + /* A monitor that omits vcpus inside an explicit allocation inherits + * the allocation's vcpu scope instead of covering the whole process. */ + if (whole_process && !resctrl->whole_process) { + virBitmapFree(domresmon->vcpus); + domresmon->vcpus = virBitmapNewCopy(resctrl->vcpus); + whole_process = false; + } + + domresmon->whole_process = whole_process; + rv = virDomainResctrlValidateMonitor(resctrl, domresmon); if (rv < 0) goto cleanup; - /* If monitor's vcpu list is identical to the vcpu list of the - * associated allocation, set monitor's id to the same value - * as the allocation. */ - if (rv == 1) { - id = g_strdup(virResctrlAllocGetID(resctrl->alloc)); - } else { - g_autofree char *tmp = virBitmapFormat(domresmon->vcpus); + /* A whole-process monitor keeps its id unset, which selects the bare + * machine name. Otherwise, if the monitor's vcpu list is identical to + * the vcpu list of the associated allocation, share the allocation's + * id. */ + if (!whole_process) { + if (rv == 1) { + id = g_strdup(virResctrlAllocGetID(resctrl->alloc)); + } else { + g_autofree char *tmp = virBitmapFormat(domresmon->vcpus); - id = g_strdup_printf("vcpus_%s", tmp); + id = g_strdup_printf("vcpus_%s", tmp); + } } virResctrlMonitorSetAlloc(domresmon->instance, resctrl->alloc); - if (virResctrlMonitorSetID(domresmon->instance, id) < 0) + if (id && virResctrlMonitorSetID(domresmon->instance, id) < 0) goto cleanup; VIR_APPEND_ELEMENT(resctrl->monitors, resctrl->nmonitors, domresmon); @@ -19014,39 +19057,64 @@ static virDomainResctrlDef * virDomainResctrlNew(xmlNodePtr node, virResctrlAlloc *alloc, virBitmap *vcpus, + bool whole_process, unsigned int flags) { virDomainResctrlDef *resctrl = NULL; g_autofree char *vcpus_str = NULL; g_autofree char *alloc_id = NULL; - /* We need to format it back because we need to be consistent in the naming - * even when users specify some "sub-optimal" string there. */ - vcpus_str = virBitmapFormat(vcpus); + /* A whole-process group omits the "vcpus" suffix. */ + if (!whole_process) { + /* We need to format it back because we need to be consistent in the naming + * even when users specify some "sub-optimal" string there. */ + vcpus_str = virBitmapFormat(vcpus); - if (!(flags & VIR_DOMAIN_DEF_PARSE_INACTIVE)) - alloc_id = virXMLPropString(node, "id"); + if (!(flags & VIR_DOMAIN_DEF_PARSE_INACTIVE)) + alloc_id = virXMLPropString(node, "id"); - if (!alloc_id) { - /* The number of allocations is limited and the directory structure is flat, - * not hierarchical, so we need to have all same allocations in one - * directory, so it's nice to have it named appropriately. For now it's - * 'vcpus_...' but it's designed in order for it to be changeable in the - * future (it's part of the status XML). */ - alloc_id = g_strdup_printf("vcpus_%s", vcpus_str); - } + if (!alloc_id) { + /* The number of allocations is limited and the directory structure is flat, + * not hierarchical, so we need to have all same allocations in one + * directory, so it's nice to have it named appropriately. For now it's + * 'vcpus_...' but it's designed in order for it to be changeable in the + * future (it's part of the status XML). */ + alloc_id = g_strdup_printf("vcpus_%s", vcpus_str); + } - if (virResctrlAllocSetID(alloc, alloc_id) < 0) - return NULL; + if (virResctrlAllocSetID(alloc, alloc_id) < 0) + return NULL; + } resctrl = g_new0(virDomainResctrlDef, 1); resctrl->vcpus = virBitmapNewCopy(vcpus); + resctrl->whole_process = whole_process; resctrl->alloc = virObjectRef(alloc); return resctrl; } +/* Whole-process and per-vcpu allocations cannot be mixed: assigning explicit + * vCPUs to their own group would pull them out of the whole-process group. */ +static int +virDomainResctrlValidateScope(virDomainDef *def, + bool whole_process) +{ + size_t i; + + for (i = 0; i < def->nresctrls; i++) { + if (def->resctrls[i]->whole_process != whole_process) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("Whole-process and per-vcpu resctrl allocations cannot be mixed")); + return -1; + } + } + + return 0; +} + + static int virDomainCachetuneDefParse(virDomainDef *def, xmlXPathContextPtr ctxt, @@ -19058,16 +19126,17 @@ virDomainCachetuneDefParse(virDomainDef *def, ssize_t i = 0; int n; int ret = -1; + bool whole_process = false; g_autoptr(virBitmap) vcpus = NULL; g_autofree xmlNodePtr *nodes = NULL; g_autoptr(virResctrlAlloc) alloc = NULL; ctxt->node = node; - if (virDomainResctrlParseVcpus(def, node, &vcpus) < 0) + if (virDomainResctrlParseVcpus(def, node, &vcpus, &whole_process) < 0) return -1; - if (virBitmapIsAllClear(vcpus)) + if (!whole_process && virBitmapIsAllClear(vcpus)) return 0; if ((n = virXPathNodeSet("./cache", ctxt, &nodes)) < 0) @@ -19082,6 +19151,9 @@ virDomainCachetuneDefParse(virDomainDef *def, return -1; } + if (virDomainResctrlValidateScope(def, whole_process) < 0) + return -1; + if (!(alloc = virResctrlAllocNew())) return -1; @@ -19090,7 +19162,7 @@ virDomainCachetuneDefParse(virDomainDef *def, return -1; } - if (!(resctrl = virDomainResctrlNew(node, alloc, vcpus, flags))) + if (!(resctrl = virDomainResctrlNew(node, alloc, vcpus, whole_process, flags))) return -1; if (virDomainResctrlMonDefParse(def, ctxt, node, @@ -19426,15 +19498,16 @@ virDomainMemorytuneDefParse(virDomainDef *def, ssize_t i = 0; size_t nmons = 0; size_t ret = -1; + bool whole_process = false; int n; ctxt->node = node; - if (virDomainResctrlParseVcpus(def, node, &vcpus) < 0) + if (virDomainResctrlParseVcpus(def, node, &vcpus, &whole_process) < 0) return -1; - if (virBitmapIsAllClear(vcpus)) + if (!whole_process && virBitmapIsAllClear(vcpus)) return 0; if ((n = virXPathNodeSet("./node", ctxt, &nodes)) < 0) @@ -19446,6 +19519,8 @@ virDomainMemorytuneDefParse(virDomainDef *def, if (resctrl) { alloc = virObjectRef(resctrl->alloc); } else { + if (virDomainResctrlValidateScope(def, whole_process) < 0) + return -1; if (!(alloc = virResctrlAllocNew())) return -1; } @@ -19461,7 +19536,8 @@ virDomainMemorytuneDefParse(virDomainDef *def, * just update the existing alloc information, which is done in above * virDomainMemorytuneDefParseMemory */ if (!resctrl) { - if (!(newresctrl = virDomainResctrlNew(node, alloc, vcpus, flags))) + if (!(newresctrl = virDomainResctrlNew(node, alloc, vcpus, + whole_process, flags))) return -1; resctrl = newresctrl; @@ -19501,15 +19577,16 @@ virDomainEnergytuneDefParse(virDomainDef *def, virDomainResctrlDef *newresctrl = NULL; g_autoptr(virBitmap) vcpus = NULL; g_autoptr(virResctrlAlloc) alloc = NULL; + bool whole_process = false; size_t nmons; int ret = -1; ctxt->node = node; - if (virDomainResctrlParseVcpus(def, node, &vcpus) < 0) + if (virDomainResctrlParseVcpus(def, node, &vcpus, &whole_process) < 0) return -1; - if (virBitmapIsAllClear(vcpus)) + if (!whole_process && virBitmapIsAllClear(vcpus)) return 0; if (virDomainResctrlVcpuMatch(def, vcpus, &resctrl) < 0) @@ -19518,9 +19595,12 @@ virDomainEnergytuneDefParse(virDomainDef *def, if (resctrl) { alloc = virObjectRef(resctrl->alloc); } else { + if (virDomainResctrlValidateScope(def, whole_process) < 0) + return -1; if (!(alloc = virResctrlAllocNew())) return -1; - if (!(newresctrl = virDomainResctrlNew(node, alloc, vcpus, flags))) + if (!(newresctrl = virDomainResctrlNew(node, alloc, vcpus, + whole_process, flags))) return -1; resctrl = newresctrl; } @@ -28701,16 +28781,22 @@ virDomainResctrlMonDefFormatHelper(virDomainResctrlMonDef *domresmon, if (domresmon->tag != tag) return 0; - virBufferAddLit(buf, "<monitor "); + virBufferAddLit(buf, "<monitor"); if (tag == VIR_RESCTRL_MONITOR_TYPE_CACHE) { - virBufferAsprintf(buf, "level='%u' ", + virBufferAsprintf(buf, " level='%u'", VIR_DOMAIN_RESCTRL_MONITOR_CACHELEVEL); } + /* A whole-process monitor has no vcpus attribute. */ + if (domresmon->whole_process) { + virBufferAddLit(buf, "/>\n"); + return 0; + } + vcpus = virBitmapFormat(domresmon->vcpus); - virBufferAsprintf(buf, "vcpus='%s'/>\n", vcpus); + virBufferAsprintf(buf, " vcpus='%s'/>\n", vcpus); return 0; } @@ -28741,16 +28827,19 @@ virDomainCachetuneDefFormat(virBuffer *buf, if (!virBufferUse(&childrenBuf)) return 0; - vcpus = virBitmapFormat(resctrl->vcpus); + /* A whole-process group has no vcpus and no id to format. */ + if (!resctrl->whole_process) { + vcpus = virBitmapFormat(resctrl->vcpus); - virBufferAsprintf(&attrBuf, " vcpus='%s'", vcpus); + virBufferAsprintf(&attrBuf, " vcpus='%s'", vcpus); - if (!(flags & VIR_DOMAIN_DEF_FORMAT_INACTIVE)) { - const char *alloc_id = virResctrlAllocGetID(resctrl->alloc); - if (!alloc_id) - return -1; + if (!(flags & VIR_DOMAIN_DEF_FORMAT_INACTIVE)) { + const char *alloc_id = virResctrlAllocGetID(resctrl->alloc); + if (!alloc_id) + return -1; - virBufferAsprintf(&attrBuf, " id='%s'", alloc_id); + virBufferAsprintf(&attrBuf, " id='%s'", alloc_id); + } } virXMLFormatElement(buf, "cachetune", &attrBuf, &childrenBuf); @@ -28798,16 +28887,19 @@ virDomainMemorytuneDefFormat(virBuffer *buf, if (!virBufferUse(&childrenBuf)) return 0; - vcpus = virBitmapFormat(resctrl->vcpus); + /* A whole-process group has no vcpus and no id to format. */ + if (!resctrl->whole_process) { + vcpus = virBitmapFormat(resctrl->vcpus); - virBufferAsprintf(&attrBuf, " vcpus='%s'", vcpus); + virBufferAsprintf(&attrBuf, " vcpus='%s'", vcpus); - if (!(flags & VIR_DOMAIN_DEF_FORMAT_INACTIVE)) { - const char *alloc_id = virResctrlAllocGetID(resctrl->alloc); - if (!alloc_id) - return -1; + if (!(flags & VIR_DOMAIN_DEF_FORMAT_INACTIVE)) { + const char *alloc_id = virResctrlAllocGetID(resctrl->alloc); + if (!alloc_id) + return -1; - virBufferAsprintf(&attrBuf, " id='%s'", alloc_id); + virBufferAsprintf(&attrBuf, " id='%s'", alloc_id); + } } virXMLFormatElement(buf, "memorytune", &attrBuf, &childrenBuf); @@ -28836,15 +28928,18 @@ virDomainEnergytuneDefFormat(virBuffer *buf, if (!virBufferUse(&childrenBuf)) return 0; - vcpus = virBitmapFormat(resctrl->vcpus); - virBufferAsprintf(&attrBuf, " vcpus='%s'", vcpus); + /* A whole-process group has no vcpus and no id to format. */ + if (!resctrl->whole_process) { + vcpus = virBitmapFormat(resctrl->vcpus); + virBufferAsprintf(&attrBuf, " vcpus='%s'", vcpus); - if (!(flags & VIR_DOMAIN_DEF_FORMAT_INACTIVE)) { - const char *alloc_id = virResctrlAllocGetID(resctrl->alloc); - if (!alloc_id) - return -1; + if (!(flags & VIR_DOMAIN_DEF_FORMAT_INACTIVE)) { + const char *alloc_id = virResctrlAllocGetID(resctrl->alloc); + if (!alloc_id) + return -1; - virBufferAsprintf(&attrBuf, " id='%s'", alloc_id); + virBufferAsprintf(&attrBuf, " id='%s'", alloc_id); + } } virXMLFormatElement(buf, "energytune", &attrBuf, &childrenBuf); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 91f57de0f1..ed1c2ae8b0 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -2935,12 +2935,14 @@ struct _virDomainCputune { struct _virDomainResctrlMonDef { virBitmap *vcpus; + bool whole_process; virResctrlMonitorType tag; virResctrlMonitor *instance; }; struct _virDomainResctrlDef { virBitmap *vcpus; + bool whole_process; virResctrlAlloc *alloc; virDomainResctrlMonDef **monitors; diff --git a/tests/genericxml2xmlindata/cachetune-monitor-empty-vcpus.xml b/tests/genericxml2xmlindata/cachetune-monitor-empty-vcpus.xml new file mode 100644 index 0000000000..a79ad71635 --- /dev/null +++ b/tests/genericxml2xmlindata/cachetune-monitor-empty-vcpus.xml @@ -0,0 +1,30 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <cachetune vcpus='0-1'> + <cache id='0' level='3' type='both' size='3' unit='MiB'/> + <monitor level='3' vcpus='8-9'/> + </cachetune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/cachetune-monitor-inherit-alloc.xml b/tests/genericxml2xmlindata/cachetune-monitor-inherit-alloc.xml new file mode 100644 index 0000000000..b8b0460d1f --- /dev/null +++ b/tests/genericxml2xmlindata/cachetune-monitor-inherit-alloc.xml @@ -0,0 +1,30 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <cachetune vcpus='0-1'> + <cache id='0' level='3' type='both' size='3' unit='MiB'/> + <monitor level='3'/> + </cachetune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/cachetune-wholeprocess-duplicate.xml b/tests/genericxml2xmlindata/cachetune-wholeprocess-duplicate.xml new file mode 100644 index 0000000000..c828b659f7 --- /dev/null +++ b/tests/genericxml2xmlindata/cachetune-wholeprocess-duplicate.xml @@ -0,0 +1,32 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <cachetune> + <cache id='0' level='3' type='both' size='3' unit='MiB'/> + </cachetune> + <cachetune> + <cache id='1' level='3' type='both' size='3' unit='MiB'/> + </cachetune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/cachetune-wholeprocess-monitor-duplicate.xml b/tests/genericxml2xmlindata/cachetune-wholeprocess-monitor-duplicate.xml new file mode 100644 index 0000000000..898a51de87 --- /dev/null +++ b/tests/genericxml2xmlindata/cachetune-wholeprocess-monitor-duplicate.xml @@ -0,0 +1,31 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <cachetune> + <cache id='0' level='3' type='both' size='3' unit='MiB'/> + <monitor level='3'/> + <monitor level='3'/> + </cachetune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/cachetune-wholeprocess-monitors.xml b/tests/genericxml2xmlindata/cachetune-wholeprocess-monitors.xml new file mode 100644 index 0000000000..550ae2df31 --- /dev/null +++ b/tests/genericxml2xmlindata/cachetune-wholeprocess-monitors.xml @@ -0,0 +1,31 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <cachetune> + <cache id='0' level='3' type='both' size='3' unit='MiB'/> + <monitor level='3' vcpus='0-1'/> + <monitor level='3' vcpus='2-3'/> + </cachetune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/energytune-colliding-monitor.xml b/tests/genericxml2xmlindata/energytune-colliding-monitor.xml new file mode 100644 index 0000000000..f07da2620f --- /dev/null +++ b/tests/genericxml2xmlindata/energytune-colliding-monitor.xml @@ -0,0 +1,30 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <energytune> + <monitor/> + <monitor vcpus='0-1'/> + </energytune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/energytune-wholeprocess.xml b/tests/genericxml2xmlindata/energytune-wholeprocess.xml new file mode 100644 index 0000000000..ebac5e9bc9 --- /dev/null +++ b/tests/genericxml2xmlindata/energytune-wholeprocess.xml @@ -0,0 +1,29 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <energytune> + <monitor/> + </energytune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/memorytune-wholeprocess.xml b/tests/genericxml2xmlindata/memorytune-wholeprocess.xml new file mode 100644 index 0000000000..496fccc7e6 --- /dev/null +++ b/tests/genericxml2xmlindata/memorytune-wholeprocess.xml @@ -0,0 +1,29 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <memorytune> + <node id='0' bandwidth='60'/> + </memorytune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/resctrl-wholeprocess-alloc-monitor.xml b/tests/genericxml2xmlindata/resctrl-wholeprocess-alloc-monitor.xml new file mode 100644 index 0000000000..ad0a904dc7 --- /dev/null +++ b/tests/genericxml2xmlindata/resctrl-wholeprocess-alloc-monitor.xml @@ -0,0 +1,32 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <cachetune> + <cache id='0' level='3' type='both' size='3' unit='MiB'/> + </cachetune> + <energytune> + <monitor/> + </energytune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/resctrl-wholeprocess-layering.xml b/tests/genericxml2xmlindata/resctrl-wholeprocess-layering.xml new file mode 100644 index 0000000000..aebd613d78 --- /dev/null +++ b/tests/genericxml2xmlindata/resctrl-wholeprocess-layering.xml @@ -0,0 +1,32 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <cachetune> + <cache id='0' level='3' type='both' size='3' unit='MiB'/> + </cachetune> + <memorytune vcpus='0-1'> + <node id='0' bandwidth='60'/> + </memorytune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmlindata/resctrl-wholeprocess-monitors.xml b/tests/genericxml2xmlindata/resctrl-wholeprocess-monitors.xml new file mode 100644 index 0000000000..35f20c6c11 --- /dev/null +++ b/tests/genericxml2xmlindata/resctrl-wholeprocess-monitors.xml @@ -0,0 +1,33 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <cachetune> + <cache id='0' level='3' type='both' size='3' unit='MiB'/> + <monitor level='3'/> + </cachetune> + <energytune> + <monitor/> + </energytune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmloutdata/cachetune-monitor-inherit-alloc.xml b/tests/genericxml2xmloutdata/cachetune-monitor-inherit-alloc.xml new file mode 100644 index 0000000000..70dcf39285 --- /dev/null +++ b/tests/genericxml2xmloutdata/cachetune-monitor-inherit-alloc.xml @@ -0,0 +1,30 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>4</vcpu> + <cputune> + <cachetune vcpus='0-1'> + <cache id='0' level='3' type='both' size='3' unit='MiB'/> + <monitor level='3' vcpus='0-1'/> + </cachetune> + </cputune> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/genericxml2xmltest.c b/tests/genericxml2xmltest.c index 169c71efa3..8492a6266e 100644 --- a/tests/genericxml2xmltest.c +++ b/tests/genericxml2xmltest.c @@ -211,6 +211,17 @@ mymain(void) DO_TEST("cachetune-cdp"); DO_TEST("cachetune"); DO_TEST("energytune"); + DO_TEST("energytune-wholeprocess"); + DO_TEST("memorytune-wholeprocess"); + DO_TEST("cachetune-wholeprocess-monitors"); + DO_TEST("resctrl-wholeprocess-monitors"); + DO_TEST("resctrl-wholeprocess-alloc-monitor"); + DO_TEST_FAIL_INACTIVE("resctrl-wholeprocess-layering"); + DO_TEST_FAIL_INACTIVE("cachetune-wholeprocess-duplicate"); + DO_TEST_FAIL_INACTIVE("cachetune-wholeprocess-monitor-duplicate"); + DO_TEST_DIFFERENT("cachetune-monitor-inherit-alloc"); + DO_TEST_FAIL_INACTIVE("energytune-colliding-monitor"); + DO_TEST_FAIL_INACTIVE("cachetune-monitor-empty-vcpus"); DO_TEST_DIFFERENT("cachetune-extra-tunes"); DO_TEST_FAIL_INACTIVE("cachetune-colliding-allocs"); DO_TEST_FAIL_INACTIVE("cachetune-colliding-tunes"); -- 2.43.0 --------------------------------------------------------------------- Intel Technology Poland sp. z o.o. ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych. Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.
A whole-process resctrl group (allocation or monitor declared without vcpus) covers the entire emulator process rather than a set of vCPU threads. Assign the emulator PID to such groups during the pre-exec handshake in qemuProcessResctrlCreate, so every thread the QEMU process later spawns - vCPUs, iothreads, workers... - inherits the group. virResctrlDeterminePath now treats a NULL id as a whole-process group, resolving to the bare machine name with no id suffix. Signed-off-by: Jedrzej Wasiukiewicz <jedrzej.wasiukiewicz@intel.com> --- src/qemu/qemu_process.c | 40 +++++++++++++++++++++++++++++++++++++--- src/util/virresctrl.c | 10 ++++------ 2 files changed, 41 insertions(+), 9 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index b2506edce0..8907f4f0ec 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -2909,18 +2909,33 @@ qemuProcessResctrlCreate(virQEMUDriver *driver, for (i = 0; i < vm->def->nresctrls; i++) { size_t j = 0; + virDomainResctrlDef *resctrl = vm->def->resctrls[i]; + if (virResctrlAllocCreate(caps->host.resctrl, - vm->def->resctrls[i]->alloc, + resctrl->alloc, priv->machineName) < 0) return -1; - for (j = 0; j < vm->def->resctrls[i]->nmonitors; j++) { + /* A whole-process group covers every emulator thread. + * Assign the emulator PID now, while still in the pre-exec handshake + * window, so the resctrl group is inherited by every thread the QEMU + * process subsequently spawns (vCPUs, iothreads, workers etc.). */ + if (resctrl->whole_process && + !virResctrlAllocIsEmpty(resctrl->alloc) && + virResctrlAllocAddPID(resctrl->alloc, vm->pid) < 0) + return -1; + + for (j = 0; j < resctrl->nmonitors; j++) { virDomainResctrlMonDef *mon = NULL; - mon = vm->def->resctrls[i]->monitors[j]; + mon = resctrl->monitors[j]; if (virResctrlMonitorCreate(mon->instance, priv->machineName) < 0) return -1; + + if (mon->whole_process && + virResctrlMonitorAddPID(mon->instance, vm->pid) < 0) + return -1; } } @@ -6265,6 +6280,25 @@ qemuProcessSetupVcpu(virDomainObj *vm, size_t j = 0; virDomainResctrlDef *ct = vm->def->resctrls[i]; + /* A whole-process allocation covers every thread: its control group is + * assigned the emulator PID once at startup and inherited by every + * thread, so per-vCPU threads need no allocation assignment here. + * Per-vCPU monitors underneath it, however, still need each vCPU's PID + * to carve out their own mon_group. */ + if (ct->whole_process) { + for (j = 0; j < ct->nmonitors; j++) { + mon = ct->monitors[j]; + + if (virBitmapIsBitSet(mon->vcpus, vcpuid)) { + if (virResctrlMonitorAddPID(mon->instance, vcpupid) < 0) + return -1; + break; + } + } + + continue; + } + if (virBitmapIsBitSet(ct->vcpus, vcpuid)) { if (virResctrlAllocAddPID(ct->alloc, vcpupid) < 0) return -1; diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index 66df44fb58..d4728d2e7c 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -2287,12 +2287,10 @@ virResctrlDeterminePath(const char *parentpath, const char *prefix, const char *id) { - if (!id) { - virReportError(VIR_ERR_INTERNAL_ERROR, - _("Resctrl ID must be set before determining resctrl parentpath='%1$s' prefix='%2$s'"), - parentpath, prefix); - return NULL; - } + /* A NULL id denotes a whole-process group, which uses the bare machine + * name with no id suffix. */ + if (!id) + return g_strdup_printf("%s/%s", parentpath, prefix); return g_strdup_printf("%s/%s-%s", parentpath, prefix, id); } -- 2.43.0 --------------------------------------------------------------------- Intel Technology Poland sp. z o.o. ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych. Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.
qemuDomainGetResctrlMonData filled the monitor's reported name from virResctrlMonitorGetID(), the bare id parsed from the domain XML (e.g. "vcpus_0-1"). That is a bug and a duplicate: the id is only a fragment of the real resctrl group name and vcpus are already reported with the vcpus attribute. The group actually lives at mon_groups/<machine>-<id>. Add virResctrlMonitorGetName(), which returns the basename of the monitor's resctrl path. The stats name now maps 1:1 to the mon_group directory backing the monitor. Signed-off-by: Jedrzej Wasiukiewicz <jedrzej.wasiukiewicz@intel.com> --- src/libvirt_private.syms | 1 + src/qemu/qemu_driver.c | 2 +- src/util/virresctrl.c | 10 ++++++++++ src/util/virresctrl.h | 3 +++ 4 files changed, 15 insertions(+), 1 deletion(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index c76e5cb08a..5f0f0951b5 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -3390,6 +3390,7 @@ virResctrlMonitorAddPID; virResctrlMonitorCreate; virResctrlMonitorDeterminePath; virResctrlMonitorGetID; +virResctrlMonitorGetName; virResctrlMonitorGetStats; virResctrlMonitorNew; virResctrlMonitorRemove; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 44b41726fb..689456fd8b 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -17214,7 +17214,7 @@ qemuDomainGetResctrlMonData(virQEMUDriver *driver, * let this newly allocated memory buffer to be freed along with * the free of 'res' */ res->vcpus = virBitmapFormat(domresmon->vcpus); - res->name = g_strdup(virResctrlMonitorGetID(monitor)); + res->name = virResctrlMonitorGetName(monitor); if (virResctrlMonitorGetStats(monitor, (const char **)features, &res->stats, &res->nstats) < 0) diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c index d4728d2e7c..80de5997fa 100644 --- a/src/util/virresctrl.c +++ b/src/util/virresctrl.c @@ -2585,6 +2585,16 @@ virResctrlMonitorGetID(virResctrlMonitor *monitor) } +char * +virResctrlMonitorGetName(virResctrlMonitor *monitor) +{ + if (!monitor->path) + return NULL; + + return g_path_get_basename(monitor->path); +} + + void virResctrlMonitorSetAlloc(virResctrlMonitor *monitor, virResctrlAlloc *alloc) diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h index 2818b6e8cc..c12b521621 100644 --- a/src/util/virresctrl.h +++ b/src/util/virresctrl.h @@ -233,6 +233,9 @@ virResctrlMonitorSetID(virResctrlMonitor *monitor, const char * virResctrlMonitorGetID(virResctrlMonitor *monitor); +char * +virResctrlMonitorGetName(virResctrlMonitor *monitor); + void virResctrlMonitorSetAlloc(virResctrlMonitor *monitor, virResctrlAlloc *alloc); -- 2.43.0 --------------------------------------------------------------------- Intel Technology Poland sp. z o.o. ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych. Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.
A whole-process monitor has no vcpu list, so omit the 'vcpus' field entirely for cache, memory bandwidth and energy monitors rather than reporting it as an empty string. Return early from the cache collector when there are no monitors, so that a monitor count is not reported when there is no data, matching the memory bandwidth and energy collectors. Signed-off-by: Jedrzej Wasiukiewicz <jedrzej.wasiukiewicz@intel.com> --- src/qemu/qemu_driver.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 689456fd8b..3d1d7b8cb9 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -17212,8 +17212,10 @@ qemuDomainGetResctrlMonData(virQEMUDriver *driver, /* If virBitmapFormat successfully returns an vcpu string, then * res.vcpus is assigned with an memory space holding it, * let this newly allocated memory buffer to be freed along with - * the free of 'res' */ - res->vcpus = virBitmapFormat(domresmon->vcpus); + * the free of 'res'. Leave vcpus as NULL to signal a whole-process + * monitor. */ + if (!domresmon->whole_process) + res->vcpus = virBitmapFormat(domresmon->vcpus); res->name = virResctrlMonitorGetName(monitor); if (virResctrlMonitorGetStats(monitor, (const char **)features, @@ -17263,8 +17265,10 @@ qemuDomainGetStatsMemoryBandwidth(virQEMUDriver *driver, for (i = 0; i < nresdata; i++) { virTypedParamListAddString(params, resdata[i]->name, VIR_DOMAIN_STATS_MEMORY_BANDWIDTH_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_MEMORY_BANDWIDTH_MONITOR_SUFFIX_NAME, i); - virTypedParamListAddString(params, resdata[i]->vcpus, - VIR_DOMAIN_STATS_MEMORY_BANDWIDTH_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_MEMORY_BANDWIDTH_MONITOR_SUFFIX_VCPUS, i); + /* A whole-process monitor has no vcpu list; omit the field entirely. */ + if (resdata[i]->vcpus) + virTypedParamListAddString(params, resdata[i]->vcpus, + VIR_DOMAIN_STATS_MEMORY_BANDWIDTH_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_MEMORY_BANDWIDTH_MONITOR_SUFFIX_VCPUS, i); virTypedParamListAddUInt(params, resdata[i]->nstats, VIR_DOMAIN_STATS_MEMORY_BANDWIDTH_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_MEMORY_BANDWIDTH_MONITOR_SUFFIX_NODE_COUNT, i); @@ -17328,8 +17332,10 @@ qemuDomainGetStatsEnergy(virQEMUDriver *driver, virTypedParamListAddString(params, resdata[i]->name, VIR_DOMAIN_STATS_CPU_ENERGY_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_CPU_ENERGY_MONITOR_SUFFIX_NAME, i); - virTypedParamListAddString(params, resdata[i]->vcpus, - VIR_DOMAIN_STATS_CPU_ENERGY_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_CPU_ENERGY_MONITOR_SUFFIX_VCPUS, i); + /* A whole-process monitor has no vcpu list; omit the field entirely. */ + if (resdata[i]->vcpus) + virTypedParamListAddString(params, resdata[i]->vcpus, + VIR_DOMAIN_STATS_CPU_ENERGY_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_CPU_ENERGY_MONITOR_SUFFIX_VCPUS, i); virTypedParamListAddUInt(params, resdata[i]->nstats, VIR_DOMAIN_STATS_CPU_ENERGY_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_CPU_ENERGY_MONITOR_SUFFIX_PKG_COUNT, i); @@ -17376,14 +17382,19 @@ qemuDomainGetStatsCpuCache(virQEMUDriver *driver, return; } + if (nresdata == 0) + return; + virTypedParamListAddUInt(params, nresdata, VIR_DOMAIN_STATS_CPU_CACHE_MONITOR_COUNT); for (i = 0; i < nresdata; i++) { virTypedParamListAddString(params, resdata[i]->name, VIR_DOMAIN_STATS_CPU_CACHE_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_CPU_CACHE_MONITOR_SUFFIX_NAME, i); - virTypedParamListAddString(params, resdata[i]->vcpus, - VIR_DOMAIN_STATS_CPU_CACHE_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_CPU_CACHE_MONITOR_SUFFIX_VCPUS, i); + /* A whole-process monitor has no vcpu list; omit the field entirely. */ + if (resdata[i]->vcpus) + virTypedParamListAddString(params, resdata[i]->vcpus, + VIR_DOMAIN_STATS_CPU_CACHE_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_CPU_CACHE_MONITOR_SUFFIX_VCPUS, i); virTypedParamListAddUInt(params, resdata[i]->nstats, VIR_DOMAIN_STATS_CPU_CACHE_MONITOR_PREFIX "%zu" VIR_DOMAIN_STATS_CPU_CACHE_MONITOR_SUFFIX_BANK_COUNT, i); -- 2.43.0 --------------------------------------------------------------------- Intel Technology Poland sp. z o.o. ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych. Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.
Signed-off-by: Jedrzej Wasiukiewicz <jedrzej.wasiukiewicz@intel.com> --- NEWS.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index ae4d8c18a5..f3f2869b92 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -17,10 +17,21 @@ v12.7.0 (unreleased) * **New features** + * resctrl: Support whole-process allocations and monitoring + + The ``vcpus`` attribute of ``cachetune``, ``memorytune``, + ``energytune`` and ``monitor`` elements is now optional. When + omitted, the allocation or monitor applies to the entire QEMU + emulator process through resctrl inheritance mechanism. + * **Improvements** * **Bug fixes** + * qemu: Fix resctrl monitor reporting + + The resctrl monitor stats no longer reports cache monitor data when no + cache monitors are configured and uses correct monitor group names. v12.6.0 (2026-08-03) ==================== -- 2.43.0 --------------------------------------------------------------------- Intel Technology Poland sp. z o.o. ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych. Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.
participants (1)
-
Jedrzej Wasiukiewicz