[PATCH RFC 00/12] Support throttle block filters

From: Chun Feng Wu <wucf@linux.ibm.com> Hi, I am thinking to leverage "throttle block filter" in QEMU to support more flexible I/O limits(e.g. tiered I/O groups), one sample provided by QEMU doc is: https://github.com/qemu/qemu/blob/master/docs/throttle.txt "For example, let's say that we have three different drives and we want to set I/O limits for each one of them and an additional set of limits for the combined I/O of all three drives." The implementation idea is to - Define throttle groups(limit) in domain - Define throttle filter to reference throttle group within disk - Within domain disk, throttle filters references multiple throttle groups to form filter chain to apply multiple limits in QEMU like above sample - Add new virsh cmds for throttle group management: throttlegroupset Add or update a throttling group. throttlegroupdel Delete a throttling group. throttlegroupinfo Get a throttling group. throttlegrouplist list all domain throttlegroups - Update "attach-disk" to add one more option "--throttle-groups" to apply throttle filters e.g. "virsh attach-disk $VM_ID ${DISK_PATH}/vm1_disk_2.qcow2 vdd --driver qemu --subdriver qcow2 --targetbus virtio --throttle-groups limit2,limit012" - I chose above semantics as I felt they're appropriate, if there are better ones please kindly suggest. This patchset includes: - Throttle group and throttle filter definition in patch 1 - New QMP processing to update and get throttle group in patch 2 - New API definition and implementation in patch 3 - Hotplug and qemuProcessLaunch flow implemenation in patch 4, 5 - Domain XML schema and doc(formatdomain.rst) change in patch 6 - Tests in patch 7, 8 - Virsh cmd implementation in patch 9 - Other enhencement/verification implementation in patch 10, 11, 12 From QMP perspective, the sample flow works this way: - Throttle group creation: virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit0","limits":{"iops-total":200,"iops-read":0,"iops-total-max":200,"iops-total-max-length":1}}}' virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit1","limits":{"iops-total":250,"iops-read":0,"iops-total-max":250,"iops-total-max-length":1}}}' virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit2","limits":{"iops-total":300,"iops-read":0,"iops-total-max":300,"iops-total-max-length":1}}}' virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit012","limits":{"iops-total":400,"iops-read":0,"iops-total-max":400,"iops-total-max-length":1}}}' - Chain up filters during attaching disk to apply two filters(limit0 and limit012): virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments": {"driver":"file","filename":"/virt/disks/vm1_disk_1.qcow2","node-name":"test-3-storage","auto-read-only":true,"discard":"unmap"}}' virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments":{"node-name":"test-3-format","read-only":false,"driver":"qcow2","file":"test-3-storage","backing":null}}' virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments":{"driver":"throttle","node-name":"libvirt-5-filter","throttle-group": "limit0","file":"test-3-format"}}' virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments": {"driver":"throttle","node-name":"libvirt-6-filter","throttle-group":"limit012","file":"libvirt-5-filter"}}' virsh qemu-monitor-command 1 '{"execute": "device_add", "arguments": {"driver":"virtio-blk-pci","scsi":false,"bus":"pci.0","addr":"0x5","drive":"libvirt-6-filter","id":"virtio-disk1"}}' BTW, I also got support from these guys(guyujie@linux.ibm.com, wuyx@linux.ibm.com, xinhaong@linux.ibm.com, commits under their names are included) to help do some enhancement/verification implementation, thanks a lot! Any comments/suggestions will be appriciated! Chun Feng Wu (9): config: Introduce ThrottleGroup and ThrottleFilter qemu: monitor: Add support for ThrottleGroup operations remote: New APIs for ThrottleGroup lifecycle management qemu: hotplug: Support hot attach block disk along with throttle filters qemu: command: Support throttle groups and filters during qemuProcessLaunch schema: Add new domain elements to support multiple throttle filters test: Test throttle group lifecycle APIs tests: Test qemuMonitorJSONGetThrottleGroup and qemuMonitorJSONUpdateThrottleGroup virsh: Add support for throttle group operations Hao Ning Xin (1): config: validate: Verify throttle group fields Yan Xiu Wu (1): config: validate: Use "iotune" and "throttlefilters" exclusively for specific disk Yu Jie Gu (1): remote: Define remote_domain_set_throttle_group_args and adjust throttle structs sequence docs/formatdomain.rst | 48 ++ include/libvirt/libvirt-domain.h | 29 + src/conf/domain_conf.c | 364 ++++++++++++ src/conf/domain_conf.h | 43 ++ src/conf/domain_validate.c | 115 ++-- src/conf/schemas/domaincommon.rng | 164 +++++- src/conf/virconftypes.h | 4 + src/driver-hypervisor.h | 22 + src/libvirt-domain.c | 188 +++++++ src/libvirt_private.syms | 9 + src/libvirt_public.syms | 7 + src/qemu/qemu_block.c | 121 ++++ src/qemu/qemu_block.h | 53 ++ src/qemu/qemu_command.c | 148 +++++ src/qemu/qemu_command.h | 9 + src/qemu/qemu_domain.c | 79 +++ src/qemu/qemu_domain.h | 10 + src/qemu/qemu_driver.c | 520 +++++++++++++++++ src/qemu/qemu_hotplug.c | 23 + src/qemu/qemu_monitor.c | 29 + src/qemu/qemu_monitor.h | 10 + src/qemu/qemu_monitor_json.c | 135 +++++ src/qemu/qemu_monitor_json.h | 13 + src/remote/remote_daemon_dispatch.c | 60 ++ src/remote/remote_driver.c | 46 ++ src/remote/remote_protocol.x | 50 +- src/remote_protocol-structs | 30 + src/test/test_driver.c | 380 +++++++++++++ tests/qemumonitorjsontest.c | 89 +++ .../qemustatusxml2xmldata/backup-pull-in.xml | 1 + .../blockjob-blockdev-in.xml | 1 + .../blockjob-mirror-in.xml | 1 + .../migration-in-params-in.xml | 1 + .../migration-out-nbd-bitmaps-in.xml | 1 + .../migration-out-nbd-out.xml | 1 + .../migration-out-nbd-tls-out.xml | 1 + .../migration-out-params-in.xml | 1 + tests/qemustatusxml2xmldata/modern-in.xml | 1 + tests/qemustatusxml2xmldata/upgrade-out.xml | 1 + .../qemustatusxml2xmldata/vcpus-multi-in.xml | 1 + tools/virsh-completer-domain.c | 62 ++ tools/virsh-completer-domain.h | 11 + tools/virsh-domain.c | 530 ++++++++++++++++++ 43 files changed, 3376 insertions(+), 36 deletions(-) -- 2.34.1

From: Chun Feng Wu <wucf@linux.ibm.com> * Define new structs 'virDomainThrottleGroupDef' and 'virDomainThrottleFilterDef' * Update _virDomainDef to include virDomainThrottleGroupDef * Update _virDomainDiskDef to include virDomainThrottleFilterDef * Support new resource operations for DOM XML and structs, corresponding "Parse" and "Format" methods are provided Signed-off-by: Chun Feng Wu <wucf@linux.ibm.com> --- src/conf/domain_conf.c | 364 ++++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 43 +++++ src/conf/virconftypes.h | 4 + 3 files changed, 411 insertions(+) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 48c5d546da..e21c8076fd 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -3746,6 +3746,50 @@ virDomainIOThreadIDDefArrayInit(virDomainDef *def, return 0; } +static virDomainThrottleFilterDef * +virDomainThrottleFilterDefNew(void) +{ + virDomainThrottleFilterDef *def = g_new0(virDomainThrottleFilterDef, 1); + + def->group_name = NULL; + + return def; +} + +void +virDomainThrottleFilterDefFree(virDomainThrottleFilterDef *def) +{ + if (!def) + return; + VIR_FREE(def->group_name); + VIR_FREE(def->nodename); + virObjectUnref(def); +} + +void +virDomainThrottleGroupDefFree(virDomainThrottleGroupDef *def) +{ + if (!def) + return; + g_free(def->group_name); + g_free(def); +} + +static void +virDomainThrottleGroupDefArrayFree(virDomainThrottleGroupDef **def, + int nthrottlegroups) +{ + size_t i; + + if (!def) + return; + + for (i = 0; i < nthrottlegroups; i++) + virDomainThrottleGroupDefFree(def[i]); + + g_free(def); +} + void virDomainResourceDefFree(virDomainResourceDef *resource) @@ -4026,6 +4070,8 @@ void virDomainDefFree(virDomainDef *def) virDomainIOThreadIDDefArrayFree(def->iothreadids, def->niothreadids); + virDomainThrottleGroupDefArrayFree(def->throttlegroups, def->nthrottlegroups); + g_free(def->defaultIOThread); virBitmapFree(def->cputune.emulatorpin); @@ -7693,6 +7739,157 @@ virDomainDiskDefIotuneParse(virDomainDiskDef *def, } #undef PARSE_IOTUNE +static virDomainThrottleFilterDef * +virDomainDiskThrottleFilterDefParse(xmlNodePtr node, + xmlXPathContextPtr ctxt) +{ + g_autoptr(virDomainThrottleFilterDef) filter = virDomainThrottleFilterDefNew(); + + VIR_XPATH_NODE_AUTORESTORE(ctxt) + ctxt->node = node; + + filter->group_name = virXMLPropString(ctxt->node, "group"); + + return g_steal_pointer(&filter); +} + +static int +virDomainDiskDefThrottleFilterChainParse(virDomainDiskDef *def, + xmlXPathContextPtr ctxt) +{ + size_t i; + int n = 0; + g_autofree xmlNodePtr *nodes = NULL; + + if ((n = virXPathNodeSet("./throttlefilters/throttlefilter", ctxt, &nodes)) < 0) + return -1; + + if (n) + def->throttlefilters = g_new0(virDomainThrottleFilterDef *, n); + + for (i = 0; i < n; i++) { + g_autoptr(virDomainThrottleFilterDef) filter = NULL; + + if (!(filter = virDomainDiskThrottleFilterDefParse(nodes[i], ctxt))) { + return -1; + } + + if (virDomainThrottleFilterFind(def, filter->group_name)) { + virReportError(VIR_ERR_XML_ERROR, + _("duplicate filter name '%1$s' found"), + filter->group_name); + return -1; + } + def->throttlefilters[def->nthrottlefilters++] = g_steal_pointer(&filter); + } + return 0; +} + +#define PARSE_THROTTLEGROUP(val) \ + if (virXPathULongLong("string(./" #val ")", \ + ctxt, &group->val) == -2) { \ + virReportError(VIR_ERR_XML_ERROR, \ + _("throttle group field '%1$s' must be an integer"), #val); \ + return NULL; \ + } + +static virDomainThrottleGroupDef * +virDomainThrottleGroupDefParseXML(xmlNodePtr node, + xmlXPathContextPtr ctxt) +{ + g_autoptr(virDomainThrottleGroupDef) group = g_new0(virDomainThrottleGroupDef, 1); + + VIR_XPATH_NODE_AUTORESTORE(ctxt) + ctxt->node = node; + + PARSE_THROTTLEGROUP(total_bytes_sec); + PARSE_THROTTLEGROUP(read_bytes_sec); + PARSE_THROTTLEGROUP(write_bytes_sec); + PARSE_THROTTLEGROUP(total_iops_sec); + PARSE_THROTTLEGROUP(read_iops_sec); + PARSE_THROTTLEGROUP(write_iops_sec); + + PARSE_THROTTLEGROUP(total_bytes_sec_max); + PARSE_THROTTLEGROUP(read_bytes_sec_max); + PARSE_THROTTLEGROUP(write_bytes_sec_max); + PARSE_THROTTLEGROUP(total_iops_sec_max); + PARSE_THROTTLEGROUP(read_iops_sec_max); + PARSE_THROTTLEGROUP(write_iops_sec_max); + + PARSE_THROTTLEGROUP(size_iops_sec); + + PARSE_THROTTLEGROUP(total_bytes_sec_max_length); + PARSE_THROTTLEGROUP(read_bytes_sec_max_length); + PARSE_THROTTLEGROUP(write_bytes_sec_max_length); + PARSE_THROTTLEGROUP(total_iops_sec_max_length); + PARSE_THROTTLEGROUP(read_iops_sec_max_length); + PARSE_THROTTLEGROUP(write_iops_sec_max_length); + + group->group_name = virXPathString("string(./group_name)", ctxt); + + return g_steal_pointer(&group); +} +#undef PARSE_THROTTLEGROUP + +int +virDomainThrottleGroupIndexByName(virDomainDef *def, const char *name) +{ + virDomainThrottleGroupDef *tgroup; + size_t i; + int candidate = -1; + + for (i = 0; i < def->nthrottlegroups; i++) { + tgroup = def->throttlegroups[i]; + if (STREQ(tgroup->group_name, name)) + return i; + } + return candidate; +} + +virDomainThrottleGroupDef * +virDomainThrottleGroupByName(virDomainDef *def, + const char *name) +{ + int idx = virDomainThrottleGroupIndexByName(def, name); + + if (idx < 0) + return NULL; + + return def->throttlegroups[idx]; +} + +static int +virDomainDefThrottleGroupsParse(virDomainDef *def, + xmlXPathContextPtr ctxt) +{ + size_t i; + int n = 0; + g_autofree xmlNodePtr *nodes = NULL; + + if ((n = virXPathNodeSet("./throttlegroups/throttlegroup", ctxt, &nodes)) < 0) + return -1; + + if (n) + def->throttlegroups = g_new0(virDomainThrottleGroupDef *, n); + + for (i = 0; i < n; i++) { + g_autoptr(virDomainThrottleGroupDef) group = NULL; + + if (!(group = virDomainThrottleGroupDefParseXML(nodes[i], ctxt))) { + return -1; + } + + if (virDomainThrottleGroupFind(def, group->group_name)) { + virReportError(VIR_ERR_XML_ERROR, + _("duplicate group name '%1$s' found"), + group->group_name); + return -1; + } + def->throttlegroups[def->nthrottlegroups++] = g_steal_pointer(&group); + } + return 0; +} + static int virDomainDiskDefMirrorParse(virDomainDiskDef *def, @@ -8184,6 +8381,9 @@ virDomainDiskDefParseXML(virDomainXMLOption *xmlopt, if (virDomainDiskDefIotuneParse(def, ctxt) < 0) return NULL; + if (virDomainDiskDefThrottleFilterChainParse(def, ctxt) < 0) + return NULL; + def->domain_name = virXPathString("string(./backenddomain/@name)", ctxt); def->serial = virXPathString("string(./serial)", ctxt); def->wwn = virXPathString("string(./wwn)", ctxt); @@ -18857,6 +19057,9 @@ virDomainDefParseXML(xmlXPathContextPtr ctxt, if (virDomainDefParseBootOptions(def, ctxt, xmlopt, flags) < 0) return NULL; + if (virDomainDefThrottleGroupsParse(def, ctxt) < 0) + return NULL; + /* analysis of the disk devices */ if ((n = virXPathNodeSet("./devices/disk", ctxt, &nodes)) < 0) return NULL; @@ -22065,6 +22268,83 @@ virDomainIOThreadIDDel(virDomainDef *def, } } +virDomainThrottleGroupDef * +virDomainThrottleGroupFind(const virDomainDef *def, + const char *name) +{ + size_t i; + + if (!def->throttlegroups || !def->nthrottlegroups) + return NULL; + + for (i = 0; i < def->nthrottlegroups; i++) { + if (STREQ(name, def->throttlegroups[i]->group_name)) + return def->throttlegroups[i]; + } + + return NULL; +} + +virDomainThrottleFilterDef * +virDomainThrottleFilterFind(const virDomainDiskDef *def, + const char *name) +{ + size_t i; + + if (!def->throttlefilters || !def->nthrottlefilters) + return NULL; + + for (i = 0; i < def->nthrottlefilters; i++) { + if (STREQ(name, def->throttlefilters[i]->group_name)) + return def->throttlefilters[i]; + } + + return NULL; +} + + +virDomainThrottleGroupDef * +virDomainThrottleGroupAdd(virDomainDef *def, + virDomainThrottleGroupDef *throttle_group) +{ + virDomainThrottleGroupDef * new_group = g_new0(virDomainThrottleGroupDef, 1); + virDomainThrottleGroupDefCopy(throttle_group, new_group); + VIR_APPEND_ELEMENT_COPY(def->throttlegroups, def->nthrottlegroups, new_group); + return new_group; +} + +void +virDomainThrottleGroupUpdate(virDomainDef *def, + virDomainThrottleGroupDef *info) +{ + size_t i; + + if (!info->group_name) + return; + + for (i = 0; i < def->nthrottlegroups; i++) { + virDomainThrottleGroupDef *t = def->throttlegroups[i]; + + if (STREQ_NULLABLE(t->group_name, info->group_name)) { + VIR_FREE(t->group_name); + virDomainThrottleGroupDefCopy(info, t); + } + } +} + +void +virDomainThrottleGroupDel(virDomainDef *def, + const char *name) +{ + size_t i; + for (i = 0; i < def->nthrottlegroups; i++) { + if (STREQ_NULLABLE(def->throttlegroups[i]->group_name, name)) { + virDomainThrottleGroupDefFree(def->throttlegroups[i]); + VIR_DELETE_ELEMENT(def->throttlegroups, i, def->nthrottlegroups); + return; + } + } +} static int virDomainEventActionDefFormat(virBuffer *buf, @@ -22681,6 +22961,20 @@ virDomainDiskDefFormatIotune(virBuffer *buf, #undef FORMAT_IOTUNE +static void +virDomainDiskDefFormatThrottleFilterChain(virBuffer *buf, + virDomainDiskDef *disk) +{ + size_t i; + g_auto(virBuffer) throttleChildBuf = VIR_BUFFER_INIT_CHILD(buf); + for (i = 0; i < disk->nthrottlefilters; i++) { + g_auto(virBuffer) throttleAttrBuf = VIR_BUFFER_INITIALIZER; + virBufferAsprintf(&throttleAttrBuf, " group='%s'", disk->throttlefilters[i]->group_name); + virXMLFormatElement(&throttleChildBuf, "throttlefilter", &throttleAttrBuf, NULL); + } + virXMLFormatElement(buf, "throttlefilters", NULL, &throttleChildBuf); +} + static void virDomainDiskDefFormatDriver(virBuffer *buf, @@ -22968,6 +23262,8 @@ virDomainDiskDefFormat(virBuffer *buf, virDomainDiskDefFormatIotune(&childBuf, def); + virDomainDiskDefFormatThrottleFilterChain(&childBuf, def); + if (def->src->readonly) virBufferAddLit(&childBuf, "<readonly/>\n"); if (def->src->shared) @@ -27136,6 +27432,65 @@ virDomainDefIOThreadsFormat(virBuffer *buf, virDomainDefaultIOThreadDefFormat(buf, def); } +#define FORMAT_THROTTLE_GROUP(val) \ + if (group->val) { \ + virBufferAsprintf(&childBuf, "<" #val ">%llu</" #val ">\n", \ + group->val); \ + } + +static void +virDomainThrottleGroupFormat(virBuffer *buf, + virDomainThrottleGroupDef *group) +{ + g_auto(virBuffer) childBuf = VIR_BUFFER_INIT_CHILD(buf); + + FORMAT_THROTTLE_GROUP(total_bytes_sec); + FORMAT_THROTTLE_GROUP(read_bytes_sec); + FORMAT_THROTTLE_GROUP(write_bytes_sec); + FORMAT_THROTTLE_GROUP(total_iops_sec); + FORMAT_THROTTLE_GROUP(read_iops_sec); + FORMAT_THROTTLE_GROUP(write_iops_sec); + + FORMAT_THROTTLE_GROUP(total_bytes_sec_max); + FORMAT_THROTTLE_GROUP(read_bytes_sec_max); + FORMAT_THROTTLE_GROUP(write_bytes_sec_max); + FORMAT_THROTTLE_GROUP(total_iops_sec_max); + FORMAT_THROTTLE_GROUP(read_iops_sec_max); + FORMAT_THROTTLE_GROUP(write_iops_sec_max); + FORMAT_THROTTLE_GROUP(size_iops_sec); + + if (group->group_name) { + virBufferEscapeString(&childBuf, "<group_name>%s</group_name>\n", + group->group_name); + } + + FORMAT_THROTTLE_GROUP(total_bytes_sec_max_length); + FORMAT_THROTTLE_GROUP(read_bytes_sec_max_length); + FORMAT_THROTTLE_GROUP(write_bytes_sec_max_length); + FORMAT_THROTTLE_GROUP(total_iops_sec_max_length); + FORMAT_THROTTLE_GROUP(read_iops_sec_max_length); + FORMAT_THROTTLE_GROUP(write_iops_sec_max_length); + + virXMLFormatElement(buf, "throttlegroup", NULL, &childBuf); +} + +#undef FORMAT_THROTTLE_GROUP + +static void +virDomainDefThrottleGroupsFormat(virBuffer *buf, + const virDomainDef *def) +{ + g_auto(virBuffer) childrenBuf = VIR_BUFFER_INIT_CHILD(buf); + ssize_t n; + + for (n = 0; n < def->nthrottlegroups; n++) { + virDomainThrottleGroupFormat(&childrenBuf, def->throttlegroups[n]); + } + + virXMLFormatElement(buf, "throttlegroups", NULL, &childrenBuf); +} + + static void virDomainIOMMUDefFormat(virBuffer *buf, @@ -27801,6 +28156,8 @@ virDomainDefFormatInternalSetRootName(virDomainDef *def, virDomainDefIOThreadsFormat(buf, def); + virDomainDefThrottleGroupsFormat(buf, def); + if (virDomainCputuneDefFormat(buf, def, flags) < 0) return -1; @@ -31033,6 +31390,13 @@ virDomainBlockIoTuneInfoCopy(const virDomainBlockIoTuneInfo *src, dst->group_name = g_strdup(src->group_name); } +void +virDomainThrottleGroupDefCopy(const virDomainThrottleGroupDef *src, + virDomainThrottleGroupDef *dst) +{ + *dst = *src; + dst->group_name = g_strdup(src->group_name); +} bool virDomainBlockIoTuneInfoEqual(const virDomainBlockIoTuneInfo *a, diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 5925faaf1a..e22cebd03f 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -464,6 +464,14 @@ struct _virDomainBlockIoTuneInfo { * virDomainBlockIoTuneInfoEqual. */ }; +/* Stores information related to a ThrottleFilter resource. */ +struct _virDomainThrottleFilterDef { + virObject parent; + + unsigned int id; /* throttle filter identifier, 0 is unset */ + char *group_name; + char *nodename; /* node name of throttle filter object */ +}; typedef enum { VIR_DOMAIN_DISK_MIRROR_STATE_NONE = 0, /* No job, or job still not synced */ @@ -550,6 +558,9 @@ struct _virDomainDiskDef { virDomainBlockIoTuneInfo blkdeviotune; + size_t nthrottlefilters; + virDomainThrottleFilterDef **throttlefilters; + char *driverName; char *serial; @@ -2992,6 +3003,9 @@ struct _virDomainDef { virDomainDefaultIOThreadDef *defaultIOThread; + size_t nthrottlegroups; + virDomainThrottleGroupDef **throttlegroups; + virDomainCputune cputune; virDomainResctrlDef **resctrls; @@ -4504,3 +4518,32 @@ virDomainObjGetMessages(virDomainObj *vm, bool virDomainDefHasSpiceGraphics(const virDomainDef *def); + +void virDomainThrottleGroupDefFree(virDomainThrottleGroupDef *def); +G_DEFINE_AUTOPTR_CLEANUP_FUNC(virDomainThrottleGroupDef, virDomainThrottleGroupDefFree); + +void virDomainThrottleFilterDefFree(virDomainThrottleFilterDef *def); +G_DEFINE_AUTOPTR_CLEANUP_FUNC(virDomainThrottleFilterDef, virDomainThrottleFilterDefFree); + +virDomainThrottleGroupDef *virDomainThrottleGroupAdd(virDomainDef *def, + virDomainThrottleGroupDef *throttle_group); + +void virDomainThrottleGroupUpdate(virDomainDef *def, + virDomainThrottleGroupDef *info); + +virDomainThrottleGroupDef *virDomainThrottleGroupFind(const virDomainDef *def, + const char *name); + +void virDomainThrottleGroupDel(virDomainDef *def, + const char *name); + +virDomainThrottleFilterDef *virDomainThrottleFilterFind(const virDomainDiskDef *def, + const char *name); + +int virDomainThrottleGroupIndexByName(virDomainDef *def, const char *name); + +virDomainThrottleGroupDef *virDomainThrottleGroupByName(virDomainDef *def, const char *name); + +void +virDomainThrottleGroupDefCopy(const virDomainThrottleGroupDef *src, + virDomainThrottleGroupDef *dst); diff --git a/src/conf/virconftypes.h b/src/conf/virconftypes.h index 0779bc224b..1cf68b2814 100644 --- a/src/conf/virconftypes.h +++ b/src/conf/virconftypes.h @@ -80,6 +80,10 @@ typedef struct _virDomainBlkiotune virDomainBlkiotune; typedef struct _virDomainBlockIoTuneInfo virDomainBlockIoTuneInfo; +typedef struct _virDomainBlockIoTuneInfo virDomainThrottleGroupDef; + +typedef struct _virDomainThrottleFilterDef virDomainThrottleFilterDef; + typedef struct _virDomainCheckpointDef virDomainCheckpointDef; typedef struct _virDomainCheckpointObj virDomainCheckpointObj; -- 2.34.1

From: Chun Feng Wu <wucf@linux.ibm.com> * ThrottleGroup is updated through "qemuMonitorJSONUpdateThrottleGroup" * ThrottleGroup is retrieved through "qemuMonitorJSONGetThrottleGroup" * ThrottleGroup is deleted by reusing "qemuMonitorDelObject" * ThrottleGroup is added by reusing "qemuMonitorAddObject" * "qemuMonitorMakeThrottleGroupLimits" will be used by building qemu cmd as well Signed-off-by: Chun Feng Wu <wucf@linux.ibm.com> --- src/qemu/qemu_monitor.c | 29 ++++++++ src/qemu/qemu_monitor.h | 10 +++ src/qemu/qemu_monitor_json.c | 135 +++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 13 ++++ 4 files changed, 187 insertions(+) diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 34e2ccab97..0c0bfe3449 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -2995,6 +2995,35 @@ qemuMonitorGetBlockIoThrottle(qemuMonitor *mon, return qemuMonitorJSONGetBlockIoThrottle(mon, qdevid, reply); } +virJSONValue *qemuMonitorThrottleGroupLimits(const virDomainThrottleGroupDef *group) +{ + return qemuMonitorMakeThrottleGroupLimits(group); +} + +int +qemuMonitorUpdateThrottleGroup(qemuMonitor *mon, + const char *qomid, + virDomainBlockIoTuneInfo *info) +{ + VIR_DEBUG("qomid=%s, info=%p", NULLSTR(qomid), info); + + QEMU_CHECK_MONITOR(mon); + + return qemuMonitorJSONUpdateThrottleGroup(mon, qomid, info); +} + + +int +qemuMonitorGetThrottleGroup(qemuMonitor *mon, + const char *groupname, + virDomainBlockIoTuneInfo *reply) +{ + VIR_DEBUG("throttle-group=%s, reply=%p", NULLSTR(groupname), reply); + + QEMU_CHECK_MONITOR(mon); + + return qemuMonitorJSONGetThrottleGroup(mon, groupname, reply); +} int qemuMonitorVMStatusToPausedReason(const char *status) diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 6e81945201..37a0307d40 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -1061,6 +1061,16 @@ int qemuMonitorGetBlockIoThrottle(qemuMonitor *mon, const char *qdevid, virDomainBlockIoTuneInfo *reply); +virJSONValue *qemuMonitorThrottleGroupLimits(const virDomainThrottleGroupDef *group); + +int qemuMonitorUpdateThrottleGroup(qemuMonitor *mon, + const char *qomid, + virDomainBlockIoTuneInfo *info); + +int qemuMonitorGetThrottleGroup(qemuMonitor *mon, + const char *groupname, + virDomainBlockIoTuneInfo *reply); + int qemuMonitorSystemWakeup(qemuMonitor *mon); int qemuMonitorGetVersion(qemuMonitor *mon, diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index eb84a3d938..ab7d2f1298 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -4633,6 +4633,141 @@ int qemuMonitorJSONGetBlockIoThrottle(qemuMonitor *mon, return qemuMonitorJSONBlockIoThrottleInfo(devices, qdevid, reply); } +static void virJSONValueLimitsAppendPositiveNumberLong(virJSONValue *limits, + const char *name, + unsigned long long value) +{ + /* avoid error from QEMU: "the burst length cannot be 0" for throttlelimits + * when seeting max-length + */ + if (value > 0) { + virJSONValueObjectAppendNumberLong(limits, name, value); + } +} + +virJSONValue * +qemuMonitorMakeThrottleGroupLimits(const virDomainThrottleGroupDef *group) +{ + g_autoptr(virJSONValue) limits = virJSONValueNewObject(); + + virJSONValueObjectAppendNumberLong(limits, "bps-total", group->total_bytes_sec); + virJSONValueObjectAppendNumberLong(limits, "bps-read", group->read_bytes_sec); + virJSONValueObjectAppendNumberLong(limits, "bps-write", group->write_bytes_sec); + virJSONValueObjectAppendNumberLong(limits, "iops-total", group->total_iops_sec); + virJSONValueObjectAppendNumberLong(limits, "iops-read", group->read_iops_sec); + virJSONValueObjectAppendNumberLong(limits, "iops-write", group->write_iops_sec); + virJSONValueObjectAppendNumberLong(limits, "bps-total-max", group->total_bytes_sec_max); + virJSONValueObjectAppendNumberLong(limits, "bps-read-max", group->read_bytes_sec_max); + virJSONValueObjectAppendNumberLong(limits, "bps-write-max", group->write_bytes_sec_max); + virJSONValueObjectAppendNumberLong(limits, "iops-total-max", group->total_iops_sec_max); + virJSONValueObjectAppendNumberLong(limits, "iops-read-max", group->read_iops_sec_max); + virJSONValueObjectAppendNumberLong(limits, "iops-write-max", group->write_iops_sec_max); + virJSONValueObjectAppendNumberLong(limits, "iops-size", group->size_iops_sec); + virJSONValueLimitsAppendPositiveNumberLong(limits, "bps-total-max-length", group->total_bytes_sec_max_length); + virJSONValueLimitsAppendPositiveNumberLong(limits, "bps-read-max-length", group->read_bytes_sec_max_length); + virJSONValueLimitsAppendPositiveNumberLong(limits, "bps-write-max-length", group->write_bytes_sec_max_length); + virJSONValueLimitsAppendPositiveNumberLong(limits, "iops-total-max-length", group->total_iops_sec_max_length); + virJSONValueLimitsAppendPositiveNumberLong(limits, "iops-read-max-length", group->read_iops_sec_max_length); + virJSONValueLimitsAppendPositiveNumberLong(limits, "iops-write-max-length", group->write_iops_sec_max_length); + + return g_steal_pointer(&limits); +} + +int qemuMonitorJSONUpdateThrottleGroup(qemuMonitor *mon, + const char *qomid, + virDomainBlockIoTuneInfo *info) +{ + g_autoptr(virJSONValue) cmd = NULL; + g_autoptr(virJSONValue) result = NULL; + g_autoptr(virJSONValue) limits = NULL; + + limits = qemuMonitorMakeThrottleGroupLimits(info); + + if (!(cmd = qemuMonitorJSONMakeCommand("qom-set", + "s:property", "limits", + "s:path", qomid, + "a:value", &limits, + NULL))) + return -1; + + if (qemuMonitorJSONCommand(mon, cmd, &result) < 0) + return -1; + + if (qemuMonitorJSONCheckError(cmd, result) < 0) + return -1; + + return 0; +} + +#define GET_THROTTLE_GROUP_VALUE(FIELD, STORE) \ + if (virJSONValueObjectHasKey(ret, FIELD)) { \ + if (virJSONValueObjectGetNumberUlong(ret, FIELD, &reply->STORE) < 0) { \ + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, \ + _("throttle group field '%1$s' missing in qemu's output"), \ + #STORE); \ + return -1; \ + } \ + } + +int qemuMonitorJSONGetThrottleGroup(qemuMonitor *mon, + const char *gname, + virDomainBlockIoTuneInfo *reply) +{ + char fullpath[100]; + g_autoptr(virJSONValue) cmd = NULL; + g_autoptr(virJSONValue) result = NULL; + g_autofree char *groupCopy = NULL; + virJSONValue *ret; + + + g_snprintf(fullpath, sizeof(fullpath), "%s%s", "/objects/", gname); + if (!(cmd = qemuMonitorJSONMakeCommand("qom-get", + "s:path", fullpath, + "s:property", "limits", + NULL))) + return -1; + + if (qemuMonitorJSONCommand(mon, cmd, &result) < 0) + return -1; + + if (qemuMonitorJSONCheckError(cmd, result) < 0) + return -1; + + if (!(ret = qemuMonitorJSONGetReply(cmd, result, VIR_JSON_TYPE_OBJECT))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("throttle group entry was not in expected format")); + return -1; + } + + GET_THROTTLE_GROUP_VALUE("bps-total", total_bytes_sec); + GET_THROTTLE_GROUP_VALUE("bps-read", read_bytes_sec); + GET_THROTTLE_GROUP_VALUE("bps-write", write_bytes_sec); + GET_THROTTLE_GROUP_VALUE("iops-total", total_iops_sec); + GET_THROTTLE_GROUP_VALUE("iops-read", read_iops_sec); + GET_THROTTLE_GROUP_VALUE("iops-write", write_iops_sec); + + GET_THROTTLE_GROUP_VALUE("bps-total-max", total_bytes_sec_max); + GET_THROTTLE_GROUP_VALUE("bps-read-max", read_bytes_sec_max); + GET_THROTTLE_GROUP_VALUE("bps-write-max", write_bytes_sec_max); + GET_THROTTLE_GROUP_VALUE("iops-total-max", total_iops_sec_max); + GET_THROTTLE_GROUP_VALUE("iops-read-max", read_iops_sec_max); + GET_THROTTLE_GROUP_VALUE("iops-write-max", write_iops_sec_max); + GET_THROTTLE_GROUP_VALUE("iops-size", size_iops_sec); + + GET_THROTTLE_GROUP_VALUE("bps-total-max-length", total_bytes_sec_max_length); + GET_THROTTLE_GROUP_VALUE("bps-read-max-length", read_bytes_sec_max_length); + GET_THROTTLE_GROUP_VALUE("bps-write-max-length", write_bytes_sec_max_length); + GET_THROTTLE_GROUP_VALUE("iops-total-max-length", total_iops_sec_max_length); + GET_THROTTLE_GROUP_VALUE("iops-read-max-length", read_iops_sec_max_length); + GET_THROTTLE_GROUP_VALUE("iops-write-max-length", write_iops_sec_max_length); + + groupCopy = g_strdup(gname); + reply->group_name = g_steal_pointer(&groupCopy); + + return 0; +} +#undef GET_THROTTLE_GROUP_VALUE + int qemuMonitorJSONSystemWakeup(qemuMonitor *mon) { g_autoptr(virJSONValue) cmd = NULL; diff --git a/src/qemu/qemu_monitor_json.h b/src/qemu/qemu_monitor_json.h index 9684660d86..b8ebb77b72 100644 --- a/src/qemu/qemu_monitor_json.h +++ b/src/qemu/qemu_monitor_json.h @@ -404,6 +404,19 @@ qemuMonitorJSONSetBlockIoThrottle(qemuMonitor *mon, const char *qomid, virDomainBlockIoTuneInfo *info); +virJSONValue * +qemuMonitorMakeThrottleGroupLimits(const virDomainThrottleGroupDef *group); + +int +qemuMonitorJSONUpdateThrottleGroup(qemuMonitor *mon, + const char *qomid, + virDomainBlockIoTuneInfo *info); + +int +qemuMonitorJSONGetThrottleGroup(qemuMonitor *mon, + const char *gname, + virDomainBlockIoTuneInfo *reply); + int qemuMonitorJSONGetBlockIoThrottle(qemuMonitor *mon, const char *qdevid, -- 2.34.1

From: Chun Feng Wu <wucf@linux.ibm.com> Support throttlegroup lifecycle management by the following implementation: * New methods defined in "include/libvirt/libvirt-domain.h" * And they're exported in "src/libvirt_public.syms" * Corresponding internal API is defined in "src/driver-hypervisor.h" * Public API calls are implemented in "src/libvirt-domain.c" * Wire protocol is defined in "src/remote/remote_protocol.x" * RPC client implementation is in "src/remote/remote_driver.c" * Server side dispatch is implemented in "src/remote/remote_daemon_dispatch.c" * Also "src/remote_protocol-structs" is updated Signed-off-by: Chun Feng Wu <wucf@linux.ibm.com> --- include/libvirt/libvirt-domain.h | 29 ++ src/driver-hypervisor.h | 22 ++ src/libvirt-domain.c | 188 ++++++++++ src/libvirt_private.syms | 9 + src/libvirt_public.syms | 7 + src/qemu/qemu_domain.c | 12 + src/qemu/qemu_domain.h | 2 + src/qemu/qemu_driver.c | 515 ++++++++++++++++++++++++++++ src/remote/remote_daemon_dispatch.c | 60 ++++ src/remote/remote_driver.c | 46 +++ src/remote/remote_protocol.x | 50 ++- src/remote_protocol-structs | 21 ++ 12 files changed, 960 insertions(+), 1 deletion(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 2f5b01bbfe..5435ab7fcb 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -5593,6 +5593,16 @@ typedef void (*virConnectDomainEventJobCompletedCallback)(virConnectPtr conn, */ # define VIR_DOMAIN_TUNABLE_BLKDEV_WRITE_IOPS_SEC_MAX_LENGTH "blkdeviotune.write_iops_sec_max_length" +/** + * VIR_DOMAIN_THROTTLE_GROUP: + * + * Macro represents the name of throttle group for which the values are updated, + * as VIR_TYPED_PARAM_STRING. + * + * Since: 10.3.0 + */ +# define VIR_DOMAIN_THROTTLE_GROUP "throttlegroup.name" + /** * virConnectDomainEventTunableCallback: * @conn: connection object @@ -6525,4 +6535,23 @@ virDomainGraphicsReload(virDomainPtr domain, unsigned int type, unsigned int flags); +int +virDomainSetThrottleGroup(virDomainPtr dom, + const char *group, + virTypedParameterPtr params, + int nparams, + unsigned int flags); +int +virDomainGetThrottleGroup(virDomainPtr dom, + const char *group, + virTypedParameterPtr params, + int *nparams, + unsigned int flags); + +int +virDomainDelThrottleGroup(virDomainPtr dom, + const char *group, + unsigned int flags); + + #endif /* LIBVIRT_DOMAIN_H */ diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index 4ce8da078d..2c81fc8ad6 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -1453,6 +1453,25 @@ typedef int unsigned int type, unsigned int flags); +typedef int +(*virDrvDomainSetThrottleGroup)(virDomainPtr dom, + const char *groupname, + virTypedParameterPtr params, + int nparams, + unsigned int flags); + +typedef int +(*virDrvDomainGetThrottleGroup)(virDomainPtr dom, + const char *groupname, + virTypedParameterPtr params, + int *nparams, + unsigned int flags); + +typedef int +(*virDrvDomainDelThrottleGroup)(virDomainPtr dom, + const char *groupname, + unsigned int flags); + typedef struct _virHypervisorDriver virHypervisorDriver; /** @@ -1726,4 +1745,7 @@ struct _virHypervisorDriver { virDrvDomainStartDirtyRateCalc domainStartDirtyRateCalc; virDrvDomainFDAssociate domainFDAssociate; virDrvDomainGraphicsReload domainGraphicsReload; + virDrvDomainSetThrottleGroup domainSetThrottleGroup; + virDrvDomainGetThrottleGroup domainGetThrottleGroup; + virDrvDomainDelThrottleGroup domainDelThrottleGroup; }; diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 7c6b93963c..61c67e2392 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -14162,3 +14162,191 @@ virDomainGraphicsReload(virDomainPtr domain, virDispatchError(domain->conn); return -1; } + +/** + * virDomainSetThrottleGroup: + * @dom: pointer to domain object + * @group: throttle group name + * @params: Pointer to blkio parameter objects + * @nparams: Number of blkio parameters (this value can be the same or + * less than the number of parameters supported) + * @flags: bitwise-OR of virDomainModificationImpact + * + * add or change throttle group. + * + * + * Returns -1 in case of error, 0 in case of success. + * + * Since: 10.3.0 + */ +int +virDomainSetThrottleGroup(virDomainPtr dom, + const char *group, + virTypedParameterPtr params, + int nparams, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(dom, "params=%p, nparams=%d, flags=0x%x", + params, nparams, flags); + VIR_TYPED_PARAMS_DEBUG(params, nparams); + + virResetLastError(); + + virCheckDomainReturn(dom, -1); + conn = dom->conn; + + virCheckReadOnlyGoto(conn->flags, error); + virCheckPositiveArgGoto(nparams, error); + virCheckNonNullArgGoto(params, error); + + if (virTypedParameterValidateSet(dom->conn, params, nparams) < 0) + goto error; + + if (conn->driver->domainSetThrottleGroup) { + int ret; + ret = conn->driver->domainSetThrottleGroup(dom, group, params, nparams, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(dom->conn); + return -1; +} + + +/** + * virDomainGetThrottleGroup: + * @dom: pointer to domain object + * @group: throttle group name + * @params: Pointer to blkio parameter object + * (return value, allocated by the caller) + * @nparams: Pointer to number of blkio parameters + * @flags: bitwise-OR of virDomainModificationImpact and virTypedParameterFlags + * + * Get all parameters for specific throttle group. On input, + * @nparams gives the size of the @params array; on output, @nparams + * gives how many slots were filled with parameter information, which + * might be less but will not exceed the input value. + * + * As a special case, calling with @params as NULL and @nparams as 0 + * on input will cause @nparams on output to contain the number of + * parameters supported by the hypervisor, either for the given @group + * or if @group is NULL, for all possible groups. The + * caller should then allocate @params array, + * i.e. (sizeof(@virTypedParameter) * @nparams) bytes and call the API + * again. See virDomainGetMemoryParameters() for more details. + * + * + * Returns -1 in case of error, 0 in case of success. + * + * Since: 10.3.0 + */ +int +virDomainGetThrottleGroup(virDomainPtr dom, + const char *group, + virTypedParameterPtr params, + int *nparams, + unsigned int flags) +{ + virConnectPtr conn; + int rc; + + VIR_DOMAIN_DEBUG(dom, "params=%p, nparams=%d, flags=0x%x", + params, (nparams) ? *nparams : -1, flags); + + virResetLastError(); + + virCheckDomainReturn(dom, -1); + + virCheckNonNullArgGoto(nparams, error); + virCheckNonNegativeArgGoto(*nparams, error); + if (*nparams != 0) { + virCheckNonNullArgGoto(params, error); + virCheckNonNullArgGoto(group, error); + } + + rc = VIR_DRV_SUPPORTS_FEATURE(dom->conn->driver, dom->conn, + VIR_DRV_FEATURE_TYPED_PARAM_STRING); + if (rc < 0) + goto error; + if (rc) + flags |= VIR_TYPED_PARAM_STRING_OKAY; + + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_AFFECT_LIVE, + VIR_DOMAIN_AFFECT_CONFIG, + error); + + conn = dom->conn; + + if (conn->driver->domainGetThrottleGroup) { + int ret; + ret = conn->driver->domainGetThrottleGroup(dom, group, params, nparams, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(dom->conn); + return -1; +} + +/** + * virDomainDelThrottleGroup: + * @dom: pointer to domain object + * @group: throttle group name + * @flags: bitwise-OR of virDomainModificationImpact and virTypedParameterFlags + * + * Delete specific throttle group + * + * Returns -1 in case of error, 0 in case of success. + * + * Since: 10.3.0 + */ +int +virDomainDelThrottleGroup(virDomainPtr dom, + const char *group, + unsigned int flags) +{ + virConnectPtr conn; + int rc; + + virResetLastError(); + + virCheckDomainReturn(dom, -1); + + rc = VIR_DRV_SUPPORTS_FEATURE(dom->conn->driver, dom->conn, + VIR_DRV_FEATURE_TYPED_PARAM_STRING); + if (rc < 0) + goto error; + if (rc) + flags |= VIR_TYPED_PARAM_STRING_OKAY; + + VIR_EXCLUSIVE_FLAGS_GOTO(VIR_DOMAIN_AFFECT_LIVE, + VIR_DOMAIN_AFFECT_CONFIG, + error); + + conn = dom->conn; + + if (conn->driver->domainDelThrottleGroup) { + int ret; + ret = conn->driver->domainDelThrottleGroup(dom, group, flags); + if (ret < 0) + goto error; + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(dom->conn); + return -1; +} diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 84e30b711c..291844d933 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -668,6 +668,15 @@ virDomainTaintMessageTypeFromString; virDomainTaintMessageTypeToString; virDomainTaintTypeFromString; virDomainTaintTypeToString; +virDomainThrottleFilterDefFree; +virDomainThrottleFilterFind; +virDomainThrottleGroupAdd; +virDomainThrottleGroupByName; +virDomainThrottleGroupDefCopy; +virDomainThrottleGroupDefFree; +virDomainThrottleGroupDel; +virDomainThrottleGroupFind; +virDomainThrottleGroupUpdate; virDomainTimerModeTypeFromString; virDomainTimerModeTypeToString; virDomainTimerNameTypeFromString; diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 7a3492d9d7..25560bb501 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -948,4 +948,11 @@ LIBVIRT_10.2.0 { virDomainGraphicsReload; } LIBVIRT_10.1.0; +LIBVIRT_10.3.0 { + global: + virDomainSetThrottleGroup; + virDomainGetThrottleGroup; + virDomainDelThrottleGroup; +} LIBVIRT_10.2.0; + # .... define new API here using predicted next version number .... diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 6b869797a8..7d1c1430ab 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -10245,6 +10245,18 @@ qemuDomainDiskByName(virDomainDef *def, return ret; } +virDomainThrottleGroupDef * +qemuDomainThrottleGroupByName(virDomainDef *def, + const char *name) +{ + virDomainThrottleGroupDef *ret; + + if (!(ret = virDomainThrottleGroupByName(def, name))) { + return NULL; + } + + return ret; +} int qemuDomainPrepareChannel(virDomainChrDef *channel, diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index bd37cb245a..6343383bf7 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -880,6 +880,8 @@ int qemuDomainSetPrivatePaths(virQEMUDriver *driver, virDomainDiskDef *qemuDomainDiskByName(virDomainDef *def, const char *name); +virDomainThrottleGroupDef *qemuDomainThrottleGroupByName(virDomainDef *def, const char *name); + char *qemuDomainGetMasterKeyFilePath(const char *libDir); int qemuDomainMasterKeyReadFile(qemuDomainObjPrivate *priv); diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index d01f788aea..097366cefb 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -19995,6 +19995,518 @@ qemuDomainGraphicsReload(virDomainPtr domain, return ret; } +/* wrapper of qemuDomainSetBlockIoTuneDefaults for throttle group since they use the same data structure */ +static int +qemuDomainSetThrottleGroupDefaults(virDomainBlockIoTuneInfo *newinfo, + virDomainBlockIoTuneInfo *oldinfo, + qemuBlockIoTuneSetFlags set_fields) +{ + return qemuDomainSetBlockIoTuneDefaults(newinfo, oldinfo, set_fields); +} + +static int +qemuDomainCheckThrottleGroupReset(const char *groupname, + virDomainBlockIoTuneInfo *newiotune) +{ + if (virDomainBlockIoTuneInfoHasAny(newiotune)) + return 0; + + if (newiotune->group_name && + STRNEQ_NULLABLE(newiotune->group_name, groupname)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("creating a new group/updating existing with all parameters zero is not supported")); + return -1; + } + + /* all zero means remove any throttling and remove from group for qemu */ + VIR_FREE(newiotune->group_name); + + return 0; +} + +static int +qemuDomainSetThrottleGroup(virDomainPtr dom, + const char *groupname, + virTypedParameterPtr params, + int nparams, + unsigned int flags) +{ + virQEMUDriver *driver = dom->conn->privateData; + virDomainObj *vm = NULL; + virDomainDef *def = NULL; + virDomainDef *persistentDef = NULL; + virDomainThrottleGroupDef info = { 0 }; + virDomainThrottleGroupDef conf_info = { 0 }; + int ret = -1; + size_t i; + qemuBlockIoTuneSetFlags set_fields = 0; + g_autoptr(virQEMUDriverConfig) cfg = NULL; + virObjectEvent *event = NULL; + virTypedParameterPtr eventParams = NULL; + int eventNparams = 0; + int eventMaxparams = 0; + virDomainThrottleGroupDef *cur_info; + virDomainThrottleGroupDef *conf_cur_info; + int rc = 0; + g_autoptr(virJSONValue) props = NULL; + g_autoptr(virJSONValue) limits = NULL; + + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG, -1); + if (virTypedParamsValidate(params, nparams, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_SIZE_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_GROUP_NAME, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + NULL) < 0) + return -1; + + if (!(vm = qemuDomainObjFromDomain(dom))) + return -1; + + if (virDomainSetThrottleGroupEnsureACL(dom->conn, vm->def, flags) < 0) + goto cleanup; + + cfg = virQEMUDriverGetConfig(driver); + + if (virDomainObjBeginJob(vm, VIR_JOB_MODIFY) < 0) + goto cleanup; + + if (virDomainObjGetDefs(vm, flags, &def, &persistentDef) < 0) + goto endjob; + + if (virTypedParamsAddString(&eventParams, &eventNparams, &eventMaxparams, + VIR_DOMAIN_THROTTLE_GROUP, groupname) < 0) + goto endjob; + +#define SET_THROTTLE_FIELD(FIELD, BOOL, CONST) \ + if (STREQ(param->field, VIR_DOMAIN_BLOCK_IOTUNE_##CONST)) { \ + info.FIELD = param->value.ul; \ + set_fields |= QEMU_BLOCK_IOTUNE_SET_##BOOL; \ + if (virTypedParamsAddULLong(&eventParams, &eventNparams, \ + &eventMaxparams, \ + VIR_DOMAIN_TUNABLE_BLKDEV_##CONST, \ + param->value.ul) < 0) \ + goto endjob; \ + continue; \ + } + + for (i = 0; i < nparams; i++) { + virTypedParameterPtr param = ¶ms[i]; + + if (param->value.ul > QEMU_BLOCK_IOTUNE_MAX) { + virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, + _("throttle group value must be no more than %1$llu"), + QEMU_BLOCK_IOTUNE_MAX); + goto endjob; + } + + SET_THROTTLE_FIELD(total_bytes_sec, BYTES, TOTAL_BYTES_SEC); + SET_THROTTLE_FIELD(read_bytes_sec, BYTES, READ_BYTES_SEC); + SET_THROTTLE_FIELD(write_bytes_sec, BYTES, WRITE_BYTES_SEC); + SET_THROTTLE_FIELD(total_iops_sec, IOPS, TOTAL_IOPS_SEC); + SET_THROTTLE_FIELD(read_iops_sec, IOPS, READ_IOPS_SEC); + SET_THROTTLE_FIELD(write_iops_sec, IOPS, WRITE_IOPS_SEC); + + SET_THROTTLE_FIELD(total_bytes_sec_max, BYTES_MAX, + TOTAL_BYTES_SEC_MAX); + SET_THROTTLE_FIELD(read_bytes_sec_max, BYTES_MAX, + READ_BYTES_SEC_MAX); + SET_THROTTLE_FIELD(write_bytes_sec_max, BYTES_MAX, + WRITE_BYTES_SEC_MAX); + SET_THROTTLE_FIELD(total_iops_sec_max, IOPS_MAX, + TOTAL_IOPS_SEC_MAX); + SET_THROTTLE_FIELD(read_iops_sec_max, IOPS_MAX, + READ_IOPS_SEC_MAX); + SET_THROTTLE_FIELD(write_iops_sec_max, IOPS_MAX, + WRITE_IOPS_SEC_MAX); + SET_THROTTLE_FIELD(size_iops_sec, SIZE_IOPS, SIZE_IOPS_SEC); + + if (STREQ(param->field, VIR_DOMAIN_BLOCK_IOTUNE_GROUP_NAME)) { + info.group_name = g_strdup(param->value.s); + set_fields |= QEMU_BLOCK_IOTUNE_SET_GROUP_NAME; + if (virTypedParamsAddString(&eventParams, &eventNparams, + &eventMaxparams, + VIR_DOMAIN_TUNABLE_BLKDEV_GROUP_NAME, + param->value.s) < 0) + goto endjob; + continue; + } + + SET_THROTTLE_FIELD(total_bytes_sec_max_length, BYTES_MAX_LENGTH, + TOTAL_BYTES_SEC_MAX_LENGTH); + SET_THROTTLE_FIELD(read_bytes_sec_max_length, BYTES_MAX_LENGTH, + READ_BYTES_SEC_MAX_LENGTH); + SET_THROTTLE_FIELD(write_bytes_sec_max_length, BYTES_MAX_LENGTH, + WRITE_BYTES_SEC_MAX_LENGTH); + SET_THROTTLE_FIELD(total_iops_sec_max_length, IOPS_MAX_LENGTH, + TOTAL_IOPS_SEC_MAX_LENGTH); + SET_THROTTLE_FIELD(read_iops_sec_max_length, IOPS_MAX_LENGTH, + READ_IOPS_SEC_MAX_LENGTH); + SET_THROTTLE_FIELD(write_iops_sec_max_length, IOPS_MAX_LENGTH, + WRITE_IOPS_SEC_MAX_LENGTH); + } + +#undef SET_THROTTLE_FIELD + + if ((info.total_bytes_sec && info.read_bytes_sec) || + (info.total_bytes_sec && info.write_bytes_sec)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("total and read/write of bytes_sec cannot be set at the same time")); + goto endjob; + } + + if ((info.total_iops_sec && info.read_iops_sec) || + (info.total_iops_sec && info.write_iops_sec)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("total and read/write of iops_sec cannot be set at the same time")); + goto endjob; + } + + if ((info.total_bytes_sec_max && info.read_bytes_sec_max) || + (info.total_bytes_sec_max && info.write_bytes_sec_max)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("total and read/write of bytes_sec_max cannot be set at the same time")); + goto endjob; + } + + if ((info.total_iops_sec_max && info.read_iops_sec_max) || + (info.total_iops_sec_max && info.write_iops_sec_max)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("total and read/write of iops_sec_max cannot be set at the same time")); + goto endjob; + } + + virDomainThrottleGroupDefCopy(&info, &conf_info); + + if (def) { + if (qemuDomainCheckThrottleGroupReset(groupname, &info) < 0) + goto endjob; + +#define CHECK_MAX(val, _bool) \ + do { \ + if (info.val##_max) { \ + if (!info.val) { \ + if (QEMU_BLOCK_IOTUNE_SET_##_bool) { \ + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, \ + _("cannot reset '%1$s' when '%2$s' is set"), \ + #val, #val "_max"); \ + } else { \ + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, \ + _("value '%1$s' cannot be set if '%2$s' is not set"), \ + #val "_max", #val); \ + } \ + goto endjob; \ + } \ + if (info.val##_max < info.val) { \ + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, \ + _("value '%1$s' cannot be smaller than '%2$s'"), \ + #val "_max", #val); \ + goto endjob; \ + } \ + } \ + } while (false) + + CHECK_MAX(total_bytes_sec, BYTES); + CHECK_MAX(read_bytes_sec, BYTES); + CHECK_MAX(write_bytes_sec, BYTES); + CHECK_MAX(total_iops_sec, IOPS); + CHECK_MAX(read_iops_sec, IOPS); + CHECK_MAX(write_iops_sec, IOPS); + +#undef CHECK_MAX + + cur_info = qemuDomainThrottleGroupByName(def, groupname); + if (cur_info != NULL) { // update existing group + if (qemuDomainSetThrottleGroupDefaults(&info, cur_info, + set_fields) < 0) + goto endjob; + qemuDomainObjEnterMonitor(vm); + rc = qemuMonitorUpdateThrottleGroup(qemuDomainGetMonitor(vm), + groupname, + &info); + qemuDomainObjExitMonitor(vm); + if (rc < 0) + goto endjob; + virDomainThrottleGroupUpdate(def, &info); + }else{ + limits = qemuMonitorThrottleGroupLimits(&info); + if (qemuMonitorCreateObjectProps(&props, + "throttle-group", groupname, + "a:limits", &limits, + NULL) < 0) + goto endjob; + qemuDomainObjEnterMonitor(vm); + rc = qemuMonitorAddObject(qemuDomainGetMonitor(vm), &props, NULL); + qemuDomainObjExitMonitor(vm); + if (rc < 0) + goto endjob; + virDomainThrottleGroupAdd(def, &info); + } + + qemuDomainSaveStatus(vm); + + if (eventNparams) { + event = virDomainEventTunableNewFromDom(dom, &eventParams, eventNparams); + virObjectEventStateQueue(driver->domainEventState, event); + } + } + + if (persistentDef) { + conf_cur_info = qemuDomainThrottleGroupByName(persistentDef, groupname); + + if (qemuDomainCheckThrottleGroupReset(groupname, &conf_info) < 0) + goto endjob; + + if (conf_cur_info != NULL) { + if (qemuDomainSetThrottleGroupDefaults(&conf_info, conf_cur_info, + set_fields) < 0) + goto endjob; + virDomainThrottleGroupUpdate(persistentDef, &conf_info); + }else{ + virDomainThrottleGroupAdd(persistentDef, &conf_info); + } + + + if (virDomainDefSave(persistentDef, driver->xmlopt, + cfg->configDir) < 0) + goto endjob; + } + + ret = 0; + endjob: + virDomainObjEndJob(vm); + + cleanup: + virDomainObjEndAPI(&vm); + virTypedParamsFree(eventParams, eventNparams); + return ret; +} + +static int +qemuDomainGetThrottleGroup(virDomainPtr dom, + const char *groupname, + virTypedParameterPtr params, + int *nparams, + unsigned int flags) +{ + virDomainObj *vm = NULL; + virDomainDef *def = NULL; + virDomainDef *persistentDef = NULL; + virDomainThrottleGroupDef groupDef = { 0 }; + virDomainThrottleGroupDef *reply = &groupDef; + int ret = -1; + int maxparams; + int rc = 0; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG | + VIR_TYPED_PARAM_STRING_OKAY, -1); + + /* We don't return strings, and thus trivially support this flag. */ + flags &= ~VIR_TYPED_PARAM_STRING_OKAY; + + if (!(vm = qemuDomainObjFromDomain(dom))) + return -1; + + if (virDomainGetThrottleGroupEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (virDomainObjBeginJob(vm, VIR_JOB_QUERY) < 0) + goto cleanup; + + /* the API check guarantees that only one of the definitions will be set */ + if (virDomainObjGetDefs(vm, flags, &def, &persistentDef) < 0) + goto endjob; + + maxparams = QEMU_NB_BLOCK_IO_TUNE_ALL_PARAMS; + + if (*nparams == 0) { + *nparams = maxparams; + ret = 0; + goto endjob; + } + if (*nparams < maxparams) + maxparams = *nparams; + + if (def) { + qemuDomainObjEnterMonitor(vm); + rc = qemuMonitorGetThrottleGroup(qemuDomainGetMonitor(vm), groupname, reply); + qemuDomainObjExitMonitor(vm); + + if (rc < 0) + goto endjob; + } + + if (persistentDef) { + reply = qemuDomainThrottleGroupByName(persistentDef, groupname); + if (reply == NULL) { + virReportError(VIR_ERR_INVALID_ARG, + _("throttle group '%1$s' was not found in the domain config"), + groupname); + goto endjob; + } + reply->group_name = g_strdup(groupname); + } + + *nparams = 0; + +#define THROTTLE_GROUP_ASSIGN(name, var) \ + if (*nparams < maxparams && \ + virTypedParameterAssign(¶ms[(*nparams)++], \ + VIR_DOMAIN_BLOCK_IOTUNE_ ## name, \ + VIR_TYPED_PARAM_ULLONG, \ + reply->var) < 0) \ + goto endjob; + + if (*nparams < maxparams) { + if (virTypedParameterAssign(¶ms[(*nparams)++], + VIR_DOMAIN_BLOCK_IOTUNE_GROUP_NAME, + VIR_TYPED_PARAM_STRING, + reply->group_name) < 0) + goto endjob; + } + + THROTTLE_GROUP_ASSIGN(TOTAL_BYTES_SEC, total_bytes_sec); + THROTTLE_GROUP_ASSIGN(READ_BYTES_SEC, read_bytes_sec); + THROTTLE_GROUP_ASSIGN(WRITE_BYTES_SEC, write_bytes_sec); + + THROTTLE_GROUP_ASSIGN(TOTAL_IOPS_SEC, total_iops_sec); + THROTTLE_GROUP_ASSIGN(READ_IOPS_SEC, read_iops_sec); + THROTTLE_GROUP_ASSIGN(WRITE_IOPS_SEC, write_iops_sec); + + THROTTLE_GROUP_ASSIGN(TOTAL_BYTES_SEC_MAX, total_bytes_sec_max); + THROTTLE_GROUP_ASSIGN(READ_BYTES_SEC_MAX, read_bytes_sec_max); + THROTTLE_GROUP_ASSIGN(WRITE_BYTES_SEC_MAX, write_bytes_sec_max); + + THROTTLE_GROUP_ASSIGN(TOTAL_IOPS_SEC_MAX, total_iops_sec_max); + THROTTLE_GROUP_ASSIGN(READ_IOPS_SEC_MAX, read_iops_sec_max); + THROTTLE_GROUP_ASSIGN(WRITE_IOPS_SEC_MAX, write_iops_sec_max); + + THROTTLE_GROUP_ASSIGN(SIZE_IOPS_SEC, size_iops_sec); + + THROTTLE_GROUP_ASSIGN(TOTAL_BYTES_SEC_MAX_LENGTH, total_bytes_sec_max_length); + THROTTLE_GROUP_ASSIGN(READ_BYTES_SEC_MAX_LENGTH, read_bytes_sec_max_length); + THROTTLE_GROUP_ASSIGN(WRITE_BYTES_SEC_MAX_LENGTH, write_bytes_sec_max_length); + + THROTTLE_GROUP_ASSIGN(TOTAL_IOPS_SEC_MAX_LENGTH, total_iops_sec_max_length); + THROTTLE_GROUP_ASSIGN(READ_IOPS_SEC_MAX_LENGTH, read_iops_sec_max_length); + THROTTLE_GROUP_ASSIGN(WRITE_IOPS_SEC_MAX_LENGTH, write_iops_sec_max_length); +#undef THROTTLE_GROUP_ASSIGN + + ret = 0; + + endjob: + virDomainObjEndJob(vm); + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + +static int +qemuDomainDelThrottleGroup(virDomainPtr dom, + const char *groupname, + unsigned int flags) +{ + virQEMUDriver *driver = dom->conn->privateData; + virDomainObj *vm = NULL; + virDomainDef *def = NULL; + virDomainDef *persistentDef = NULL; + g_autoptr(virQEMUDriverConfig) cfg = NULL; + int ret = -1; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG | + VIR_TYPED_PARAM_STRING_OKAY, -1); + + /* We don't return strings, and thus trivially support this flag. */ + flags &= ~VIR_TYPED_PARAM_STRING_OKAY; + + if (!(vm = qemuDomainObjFromDomain(dom))) + return -1; + + cfg = virQEMUDriverGetConfig(driver); + + if (virDomainDelThrottleGroupEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (virDomainObjBeginJob(vm, VIR_JOB_QUERY) < 0) + goto cleanup; + + /* the API check guarantees that only one of the definitions will be set */ + if (virDomainObjGetDefs(vm, flags, &def, &persistentDef) < 0) + goto endjob; + + + if (def) { + int rc = 0; + + qemuDomainObjEnterMonitor(vm); + rc = qemuMonitorDelObject(qemuDomainGetMonitor(vm), groupname, true); + qemuDomainObjExitMonitor(vm); + + if (rc < 0) + goto endjob; + + virDomainThrottleGroupDel(def, groupname); + qemuDomainSaveStatus(vm); + } + + if (persistentDef) { + virDomainThrottleGroupDel(persistentDef, groupname); + if (virDomainDefSave(persistentDef, driver->xmlopt, + cfg->configDir) < 0) + goto endjob; + } + + ret = 0; + + endjob: + virDomainObjEndJob(vm); + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} + static virHypervisorDriver qemuHypervisorDriver = { .name = QEMU_DRIVER_NAME, .connectURIProbe = qemuConnectURIProbe, @@ -20245,6 +20757,9 @@ static virHypervisorDriver qemuHypervisorDriver = { .domainSetLaunchSecurityState = qemuDomainSetLaunchSecurityState, /* 8.0.0 */ .domainFDAssociate = qemuDomainFDAssociate, /* 9.0.0 */ .domainGraphicsReload = qemuDomainGraphicsReload, /* 10.2.0 */ + .domainSetThrottleGroup = qemuDomainSetThrottleGroup, /* 10.3.0 */ + .domainGetThrottleGroup = qemuDomainGetThrottleGroup, /* 10.3.0 */ + .domainDelThrottleGroup = qemuDomainDelThrottleGroup, /* 10.3.0 */ }; diff --git a/src/remote/remote_daemon_dispatch.c b/src/remote/remote_daemon_dispatch.c index cfc1067e40..7d28d30e22 100644 --- a/src/remote/remote_daemon_dispatch.c +++ b/src/remote/remote_daemon_dispatch.c @@ -3614,6 +3614,66 @@ remoteDispatchDomainGetBlockIoTune(virNetServer *server G_GNUC_UNUSED, return rv; } +static int +remoteDispatchDomainGetThrottleGroup(virNetServer *server G_GNUC_UNUSED, + virNetServerClient *client, + virNetMessage *hdr G_GNUC_UNUSED, + struct virNetMessageError *rerr, + remote_domain_get_throttle_group_args *args, + remote_domain_get_throttle_group_ret *ret) +{ + virDomainPtr dom = NULL; + int rv = -1; + virTypedParameterPtr params = NULL; + int nparams = 0; + virConnectPtr conn = remoteGetHypervisorConn(client); + + if (!conn) + goto cleanup; + + if (args->nparams > REMOTE_DOMAIN_THROTTLE_GROUP_PARAMETERS_MAX) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("nparams too large")); + goto cleanup; + } + + if (args->nparams) + params = g_new0(virTypedParameter, args->nparams); + nparams = args->nparams; + + if (!(dom = get_nonnull_domain(conn, args->dom))) + goto cleanup; + + if (virDomainGetThrottleGroup(dom, args->group ? *args->group : NULL, + params, &nparams, args->flags) < 0) + goto cleanup; + + /* In this case, we need to send back the number of parameters + * supported + */ + if (args->nparams == 0) { + ret->nparams = nparams; + goto success; + } + + /* Serialize the block I/O tuning parameters. */ + if (virTypedParamsSerialize(params, nparams, + REMOTE_DOMAIN_THROTTLE_GROUP_PARAMETERS_MAX, + (struct _virTypedParameterRemote **) &ret->params.params_val, + &ret->params.params_len, + args->flags) < 0) + goto cleanup; + + success: + rv = 0; + + cleanup: + if (rv < 0) + virNetMessageSaveError(rerr); + virTypedParamsFree(params, nparams); + virObjectUnref(dom); + return rv; +} + /*-------------------------------------------------------------*/ static int diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 7b73d97b7a..4250729e27 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -2583,6 +2583,49 @@ static int remoteDomainGetBlockIoTune(virDomainPtr domain, return 0; } +static int remoteDomainGetThrottleGroup(virDomainPtr domain, + const char *group, + virTypedParameterPtr params, + int *nparams, + unsigned int flags) +{ + remote_domain_get_throttle_group_args args = {0}; + g_auto(remote_domain_get_throttle_group_ret) ret = {0}; + struct private_data *priv = domain->conn->privateData; + VIR_LOCK_GUARD lock = remoteDriverLock(priv); + + make_nonnull_domain(&args.dom, domain); + args.group = group ? (char **)&group : NULL; + args.nparams = *nparams; + args.flags = flags; + + if (call(domain->conn, priv, 0, REMOTE_PROC_DOMAIN_GET_THROTTLE_GROUP, + (xdrproc_t) xdr_remote_domain_get_throttle_group_args, + (char *) &args, + (xdrproc_t) xdr_remote_domain_get_throttle_group_ret, + (char *) &ret) == -1) { + return -1; + } + + /* Handle the case when the caller does not know the number of parameters + * and is asking for the number of parameters supported + */ + if (*nparams == 0) { + *nparams = ret.nparams; + return 0; + } + + if (virTypedParamsDeserialize((struct _virTypedParameterRemote *) ret.params.params_val, + ret.params.params_len, + REMOTE_DOMAIN_THROTTLE_GROUP_PARAMETERS_MAX, + ¶ms, + nparams) < 0) + return -1; + + return 0; + +} + static int remoteDomainGetCPUStats(virDomainPtr domain, virTypedParameterPtr params, unsigned int nparams, @@ -7842,6 +7885,9 @@ static virHypervisorDriver hypervisor_driver = { .domainSetLaunchSecurityState = remoteDomainSetLaunchSecurityState, /* 8.0.0 */ .domainFDAssociate = remoteDomainFDAssociate, /* 9.0.0 */ .domainGraphicsReload = remoteDomainGraphicsReload, /* 10.2.0 */ + .domainSetThrottleGroup = remoteDomainSetThrottleGroup, /* 10.3.0 */ + .domainGetThrottleGroup = remoteDomainGetThrottleGroup, /* 10.3.0 */ + .domainDelThrottleGroup = remoteDomainDelThrottleGroup, /* 10.3.0 */ }; static virNetworkDriver network_driver = { diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 41c045ff78..a7aa3d2d74 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -113,6 +113,9 @@ const REMOTE_DOMAIN_MEMORY_PARAMETERS_MAX = 16; /* Upper limit on list of blockio tuning parameters. */ const REMOTE_DOMAIN_BLOCK_IO_TUNE_PARAMETERS_MAX = 32; +/* Upper limit on list of throttle group parameters. */ +const REMOTE_DOMAIN_THROTTLE_GROUP_PARAMETERS_MAX = 32; + /* Upper limit on list of numa parameters. */ const REMOTE_DOMAIN_NUMA_PARAMETERS_MAX = 16; @@ -1475,6 +1478,31 @@ struct remote_domain_get_block_io_tune_ret { int nparams; }; +struct remote_domain_set_throttle_group_args { + remote_nonnull_domain dom; + remote_nonnull_string group; + remote_typed_param params<REMOTE_DOMAIN_THROTTLE_GROUP_PARAMETERS_MAX>; + unsigned int flags; +}; + +struct remote_domain_get_throttle_group_args { + remote_nonnull_domain dom; + remote_string group; + int nparams; + unsigned int flags; +}; + +struct remote_domain_get_throttle_group_ret { + remote_typed_param params<REMOTE_DOMAIN_THROTTLE_GROUP_PARAMETERS_MAX>; + int nparams; +}; + +struct remote_domain_del_throttle_group_args { + remote_nonnull_domain dom; + remote_string group; + unsigned int flags; +}; + struct remote_domain_get_cpu_stats_args { remote_nonnull_domain dom; unsigned int nparams; @@ -7048,5 +7076,25 @@ enum remote_procedure { * @generate: both * @acl: domain:write */ - REMOTE_PROC_DOMAIN_GRAPHICS_RELOAD = 448 + REMOTE_PROC_DOMAIN_GRAPHICS_RELOAD = 448, + + /** + * @generate: both + * @acl: domain:write + * @acl: domain:save:!VIR_DOMAIN_AFFECT_CONFIG|VIR_DOMAIN_AFFECT_LIVE + * @acl: domain:save:VIR_DOMAIN_AFFECT_CONFIG + */ + REMOTE_PROC_DOMAIN_SET_THROTTLE_GROUP = 449, + + /** + * @generate: none + * @acl: domain:read + */ + REMOTE_PROC_DOMAIN_GET_THROTTLE_GROUP = 450, + + /** + * @generate: both + * @acl: domain:read + */ + REMOTE_PROC_DOMAIN_DEL_THROTTLE_GROUP = 451 }; diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index 4d3dc2d249..0319a5690d 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -1050,6 +1050,24 @@ struct remote_domain_get_block_io_tune_ret { } params; int nparams; }; +struct remote_domain_get_throttle_group_ret { + struct { + u_int params_len; + remote_typed_param * params_val; + } params; + int nparams; +}; +struct remote_domain_get_throttle_group_args { + remote_nonnull_domain dom; + remote_string group; + int nparams; + u_int flags; +}; +struct remote_domain_del_throttle_group_args { + remote_nonnull_domain dom; + remote_string group; + u_int flags; +}; struct remote_domain_get_cpu_stats_args { remote_nonnull_domain dom; u_int nparams; @@ -3755,4 +3773,7 @@ enum remote_procedure { REMOTE_PROC_NETWORK_EVENT_CALLBACK_METADATA_CHANGE = 446, REMOTE_PROC_NODE_DEVICE_UPDATE = 447, REMOTE_PROC_DOMAIN_GRAPHICS_RELOAD = 448, + REMOTE_PROC_DOMAIN_SET_THROTTLE_GROUP = 449, + REMOTE_PROC_DOMAIN_GET_THROTTLE_GROUP = 450, + REMOTE_PROC_DOMAIN_DEL_THROTTLE_GROUP = 451, }; -- 2.34.1

From: Chun Feng Wu <wucf@linux.ibm.com> When attaching disk along with specified throttle groups, those groups will be chained up by parent node name, this change includes service side codes: * Each filter references one throttle group by group name * Each filter has a nodename, and those filters are chained up in sequence * Filter nodename index is persistented in virDomainObj->privateData(qemuDomainObjPrivate) * During hotplug, filter is created through QMP request("blockdev-add" with "driver":"throttle") to QEMU * Finally, "device_add"(attach) QMP request is adjusted to set "drive" to be top filter nodename in chain. Signed-off-by: Chun Feng Wu <wucf@linux.ibm.com> --- src/qemu/qemu_block.c | 121 ++++++++++++++++++ src/qemu/qemu_block.h | 53 ++++++++ src/qemu/qemu_command.c | 61 +++++++++ src/qemu/qemu_command.h | 6 + src/qemu/qemu_domain.c | 67 ++++++++++ src/qemu/qemu_domain.h | 12 +- src/qemu/qemu_hotplug.c | 23 ++++ .../qemustatusxml2xmldata/backup-pull-in.xml | 1 + .../blockjob-blockdev-in.xml | 1 + .../blockjob-mirror-in.xml | 1 + .../migration-in-params-in.xml | 1 + .../migration-out-nbd-bitmaps-in.xml | 1 + .../migration-out-nbd-out.xml | 1 + .../migration-out-nbd-tls-out.xml | 1 + .../migration-out-params-in.xml | 1 + tests/qemustatusxml2xmldata/modern-in.xml | 1 + tests/qemustatusxml2xmldata/upgrade-out.xml | 1 + .../qemustatusxml2xmldata/vcpus-multi-in.xml | 1 + 18 files changed, 352 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index 738b72d7ea..3c784d36ea 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -2738,6 +2738,127 @@ qemuBlockStorageSourceCreateDetectSize(GHashTable *blockNamedNodeData, return 0; } +void +qemuBlockThrottleFilterSetNodename(virDomainThrottleFilterDef *filter, + char *nodename) +{ + g_free(filter->nodename); + filter->nodename = nodename; +} + +const char * +qemuBlockThrottleFilterGetNodename(virDomainThrottleFilterDef *filter) +{ + return filter->nodename; +} + +/** + * qemuBlockThrottleFilterGetProps: + * @filter: throttle filter + * @parentNodeName: parent nodename of @filter + * + * Build "arguments" part of "blockdev-add" QMP cmd. + * e.g. {"execute":"blockdev-add", "arguments":{"driver":"throttle", + * "node-name":"libvirt-2-filter", "throttle-group":"limits0", + * "file": "libvirt-1-format"}} + */ +virJSONValue * +qemuBlockThrottleFilterGetProps(virDomainThrottleFilterDef *filter, + const char *parentNodeName) +{ + g_autoptr(virJSONValue) props = NULL; + + if (virJSONValueObjectAdd(&props, + "s:driver", "throttle", + "s:node-name", qemuBlockThrottleFilterGetNodename(filter), + "s:throttle-group", filter->group_name, + "s:file", parentNodeName, + NULL) < 0) + return 0; + + return g_steal_pointer(&props); +} + +void +qemuBlockThrottleFilterAttachDataFree(qemuBlockThrottleFilterAttachData *data) +{ + if (!data) + return; + + virJSONValueFree(data->filterProps); + g_free(data); +} + +qemuBlockThrottleFilterAttachData * +qemuBlockThrottleFilterAttachPrepareBlockdev(virDomainThrottleFilterDef *filter, + const char *parentNodeName) +{ + g_autoptr(qemuBlockThrottleFilterAttachData) data = NULL; + + data = g_new0(qemuBlockThrottleFilterAttachData, 1); + + if (!(data->filterProps = qemuBlockThrottleFilterGetProps(filter, parentNodeName))) + return NULL; + + data->filterNodeName = qemuBlockThrottleFilterGetNodename(filter); + data->filterAttached = true; + + return g_steal_pointer(&data); +} + +void +qemuBlockThrottleFilterAttachRollback(qemuMonitor *mon, + qemuBlockThrottleFilterAttachData *data) +{ + virErrorPtr orig_err; + + virErrorPreserveLast(&orig_err); + + if (data->filterAttached) + ignore_value(qemuMonitorBlockdevDel(mon, data->filterNodeName)); + + virErrorRestore(&orig_err); +} + +void +qemuBlockThrottleFilterChainDataFree(qemuBlockThrottleFilterChainData *data) +{ + size_t i; + + if (!data) + return; + + for (i = 0; i < data->nfilterdata; i++) + qemuBlockThrottleFilterAttachDataFree(data->filterdata[i]); + + g_free(data->filterdata); + g_free(data); +} + +int +qemuBlockThrottleFilterChainAttach(qemuMonitor *mon, + qemuBlockThrottleFilterChainData *data) +{ + size_t i; + + for (i = 0; i < data->nfilterdata; i++) { + if (qemuMonitorBlockdevAdd(mon, &data->filterdata[i]->filterProps) < 0) + return -1; + data->filterdata[i]->filterAttached = true; + } + + return 0; +} + +void +qemuBlockThrottleFilterChainDetach(qemuMonitor *mon, + qemuBlockThrottleFilterChainData *data) +{ + size_t i; + + for (i = data->nfilterdata; i > 0; i--) + qemuBlockThrottleFilterAttachRollback(mon, data->filterdata[i-1]); +} int qemuBlockRemoveImageMetadata(virQEMUDriver *driver, diff --git a/src/qemu/qemu_block.h b/src/qemu/qemu_block.h index f9e961d85d..66bf7a97c3 100644 --- a/src/qemu/qemu_block.h +++ b/src/qemu/qemu_block.h @@ -214,6 +214,59 @@ qemuBlockStorageSourceCreateDetectSize(GHashTable *blockNamedNodeData, virStorageSource *src, virStorageSource *templ); +void +qemuBlockThrottleFilterSetNodename(virDomainThrottleFilterDef *filter, + char *nodename); + +const char * +qemuBlockThrottleFilterGetNodename(virDomainThrottleFilterDef *filter); + +virJSONValue * +qemuBlockThrottleFilterGetProps(virDomainThrottleFilterDef *filter, + const char *parentNodeName); + +typedef struct qemuBlockThrottleFilterAttachData qemuBlockThrottleFilterAttachData; +struct qemuBlockThrottleFilterAttachData { + virJSONValue *filterProps; + const char *filterNodeName; + bool filterAttached; +}; + +qemuBlockThrottleFilterAttachData * +qemuBlockThrottleFilterAttachPrepareBlockdev(virDomainThrottleFilterDef *throttlefilter, + const char *parentNodeName); + +void +qemuBlockThrottleFilterAttachDataFree(qemuBlockThrottleFilterAttachData *data); + +G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuBlockThrottleFilterAttachData, + qemuBlockThrottleFilterAttachDataFree); + +void +qemuBlockThrottleFilterAttachRollback(qemuMonitor *mon, + qemuBlockThrottleFilterAttachData *data); + +struct _qemuBlockThrottleFilterChainData { + qemuBlockThrottleFilterAttachData **filterdata; + size_t nfilterdata; +}; + +typedef struct _qemuBlockThrottleFilterChainData qemuBlockThrottleFilterChainData; + +void +qemuBlockThrottleFilterChainDataFree(qemuBlockThrottleFilterChainData *data); + +G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuBlockThrottleFilterChainData, + qemuBlockThrottleFilterChainDataFree); + +int +qemuBlockThrottleFilterChainAttach(qemuMonitor *mon, + qemuBlockThrottleFilterChainData *data); + +void +qemuBlockThrottleFilterChainDetach(qemuMonitor *mon, + qemuBlockThrottleFilterChainData *data); + int qemuBlockRemoveImageMetadata(virQEMUDriver *driver, virDomainObj *vm, diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 9d4563861f..278b706616 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1576,6 +1576,12 @@ qemuDiskConfigBlkdeviotuneEnabled(const virDomainDiskDef *disk) virDomainBlockIoTuneInfoHasAny(&disk->blkdeviotune); } +bool +qemuDiskConfigThrottleFilterChainEnabled(const virDomainDiskDef *disk) +{ + return disk->nthrottlefilters > 0; +} + /** * qemuDiskBusIsSD: @@ -1936,6 +1942,10 @@ qemuBuildDiskDeviceProps(const virDomainDef *def, } else { if (qemuDomainDiskGetBackendAlias(disk, &drive) < 0) return NULL; + /* make sure device drive points to top throttle filter */ + if (qemuDiskConfigThrottleFilterChainEnabled(disk)) { + drive = g_strdup(disk->throttlefilters[disk->nthrottlefilters-1]->nodename); + } } /* bootindex for floppies is configured via the fdc controller */ @@ -11025,6 +11035,57 @@ qemuBuildStorageSourceChainAttachPrepareBlockdevOne(qemuBlockStorageSourceChainD return 0; } +/** + * qemuBuildThrottleFilterChainAttachPrepareBlockdevOne: + * @data: filter chain data, which consists of array of filters and size of such array + * @throttlefilter: new filter to be added into filter array + * @parentNodeName: parent nodename for this new throttlefilter, which is used to build "blockdev-add" QMP request + * + * Build filter node chain to provide more flexibility for block disk I/O limits + */ +static int +qemuBuildThrottleFilterChainAttachPrepareBlockdevOne(qemuBlockThrottleFilterChainData *data, + virDomainThrottleFilterDef *throttlefilter, + const char *parentNodeName) +{ + g_autoptr(qemuBlockThrottleFilterAttachData) elem = NULL; + + if (!(elem = qemuBlockThrottleFilterAttachPrepareBlockdev(throttlefilter, parentNodeName))) + return -1; + + VIR_APPEND_ELEMENT(data->filterdata, data->nfilterdata, elem); + return 0; +} + +/** + * qemuBuildThrottleFilterChainAttachPrepareBlockdev: + * @disk: domain disk + * + * Build filter node chain to provide more flexibility for block disk I/O limits + */ +qemuBlockThrottleFilterChainData * +qemuBuildThrottleFilterChainAttachPrepareBlockdev(virDomainDiskDef *disk) +{ + g_autoptr(qemuBlockThrottleFilterChainData) data = NULL; + size_t i; + const char * parentNodeName = NULL; + g_autofree char *tmp_nodename = NULL; + + data = g_new0(qemuBlockThrottleFilterChainData, 1); + /* get starting parentNodename, e.g. libvirt-1-format */ + parentNodeName = qemuBlockStorageSourceGetEffectiveNodename(disk->src); + //build filterdata, which contains all filters info and sequence info through parentNodeName + for (i = 0; i < disk->nthrottlefilters; i++) { + tmp_nodename = g_strdup(parentNodeName); + if (qemuBuildThrottleFilterChainAttachPrepareBlockdevOne(data, disk->throttlefilters[i], tmp_nodename) < 0) + return NULL; + parentNodeName = disk->throttlefilters[i]->nodename; + } + + return g_steal_pointer(&data); +} + + /** * qemuBuildStorageSourceChainAttachPrepareBlockdev: diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index 341ec43f9a..e2dee47906 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -116,6 +116,12 @@ qemuBlockStorageSourceChainData * qemuBuildStorageSourceChainAttachPrepareBlockdevTop(virStorageSource *top, virStorageSource *backingStore); +qemuBlockThrottleFilterChainData * +qemuBuildThrottleFilterChainAttachPrepareBlockdev(virDomainDiskDef *disk); + +bool +qemuDiskConfigThrottleFilterChainEnabled(const virDomainDiskDef *disk); + virJSONValue * qemuBuildDiskDeviceProps(const virDomainDef *def, virDomainDiskDef *disk, diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 7d1c1430ab..c2f5d17c15 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -683,6 +683,17 @@ qemuDomainStorageIDNew(qemuDomainObjPrivate *priv) return ++priv->nodenameindex; } +/** + * qemuDomainThrottleFilterIDNew: + * @priv: qemu VM private data object. + * + * Generate a new unique id for throttle filter object. Useful for node name generation. + */ +static unsigned int +qemuDomainThrottleFilterIDNew(qemuDomainObjPrivate *priv) +{ + return ++priv->filternodenameindex; +} /** * qemuDomainStorageIDReset: @@ -697,6 +708,20 @@ qemuDomainStorageIDReset(qemuDomainObjPrivate *priv) priv->nodenameindex = 0; } +/** + * qemuDomainThrottleFilterIDReset: + * @priv: qemu VM private data object. + * + * Resets the data for the node name generator. The node names need to be unique + * for a single instance, so can be reset on VM shutdown. + */ +static void +qemuDomainThrottleFilterIDReset(qemuDomainObjPrivate *priv) +{ + priv->filternodenameindex = 0; +} + + /** * qemuDomainFDSetIDNew: @@ -1853,6 +1878,8 @@ qemuDomainObjPrivateDataClear(qemuDomainObjPrivate *priv) /* reset node name allocator */ qemuDomainStorageIDReset(priv); + qemuDomainThrottleFilterIDReset(priv); + qemuDomainFDSetIDReset(priv); priv->dbusDaemonRunning = false; @@ -2649,6 +2676,8 @@ qemuDomainObjPrivateXMLFormat(virBuffer *buf, virBufferAsprintf(buf, "<nodename index='%llu'/>\n", priv->nodenameindex); + virBufferAsprintf(buf, "<filternodename index='%llu'/>\n", priv->filternodenameindex); + virBufferAsprintf(buf, "<fdset index='%u'/>\n", priv->fdsetindex); if (priv->memPrealloc) @@ -3376,6 +3405,7 @@ qemuDomainObjPrivateXMLParse(xmlXPathContextPtr ctxt, return -1; qemuDomainStorageIDReset(priv); + qemuDomainThrottleFilterIDReset(priv); if (virXPathULongLong("string(./nodename/@index)", ctxt, &priv->nodenameindex) == -2) { virReportError(VIR_ERR_XML_ERROR, "%s", @@ -3383,6 +3413,13 @@ qemuDomainObjPrivateXMLParse(xmlXPathContextPtr ctxt, return -1; } + if (virXPathULongLong("string(./filternodename/@index)", ctxt, + &priv->filternodenameindex) == -2) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("failed to parse filter node name index")); + return -1; + } + if (virXPathUInt("string(./fdset/@index)", ctxt, &priv->fdsetindex) == 0) priv->fdsetindexParsed = true; @@ -11345,6 +11382,30 @@ qemuDomainPrepareStorageSourceBlockdevNodename(virDomainDiskDef *disk, return 0; } +int +qemuDomainPrepareThrottleFilterBlockdevNodename(virDomainThrottleFilterDef *filter, + const char *nodenameprefix) +{ + char *nodename = g_strdup_printf("%s-filter", nodenameprefix); + + qemuBlockThrottleFilterSetNodename(filter, nodename); + + return 0; +} + +int +qemuDomainPrepareThrottleFilterBlockdev(virDomainThrottleFilterDef *filter, + qemuDomainObjPrivate *priv) +{ + g_autofree char *nodenameprefix = NULL; + + filter->id = qemuDomainThrottleFilterIDNew(priv); + + nodenameprefix = g_strdup_printf("libvirt-%u", filter->id); + + return qemuDomainPrepareThrottleFilterBlockdevNodename(filter, nodenameprefix); +} + int qemuDomainPrepareStorageSourceBlockdev(virDomainDiskDef *disk, @@ -11369,6 +11430,7 @@ qemuDomainPrepareDiskSourceBlockdev(virDomainDiskDef *disk, { qemuDomainDiskPrivate *diskPriv = QEMU_DOMAIN_DISK_PRIVATE(disk); virStorageSource *n; + size_t i; if (disk->copy_on_read == VIR_TRISTATE_SWITCH_ON && !diskPriv->nodeCopyOnRead) @@ -11379,6 +11441,11 @@ qemuDomainPrepareDiskSourceBlockdev(virDomainDiskDef *disk, return -1; } + for (i = 0; i < disk->nthrottlefilters; i++) { + if (qemuDomainPrepareThrottleFilterBlockdev(disk->throttlefilters[i], priv) < 0) + return -1; + } + return 0; } diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 6343383bf7..ce5f5dfb6e 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -204,6 +204,9 @@ struct _qemuDomainObjPrivate { /* counter for generating node names for qemu disks */ unsigned long long nodenameindex; + /* counter for generating node names for throttle filters */ + unsigned long long filternodenameindex; + /* counter for generating IDs of fdsets */ unsigned int fdsetindex; bool fdsetindexParsed; @@ -772,6 +775,13 @@ int qemuDomainPrepareStorageSourceBlockdev(virDomainDiskDef *disk, qemuDomainObjPrivate *priv, virQEMUDriverConfig *cfg); +int qemuDomainPrepareThrottleFilterBlockdev(virDomainThrottleFilterDef *filter, + qemuDomainObjPrivate *priv); +int +qemuDomainPrepareThrottleFilterBlockdevNodename(virDomainThrottleFilterDef *filter, + const char *nodenameprefix); +virDomainThrottleGroupDef *qemuDomainThrottleGroupByName(virDomainDef *def, const char *name); + void qemuDomainCleanupAdd(virDomainObj *vm, qemuDomainCleanupCallback cb); void qemuDomainCleanupRemove(virDomainObj *vm, @@ -880,8 +890,6 @@ int qemuDomainSetPrivatePaths(virQEMUDriver *driver, virDomainDiskDef *qemuDomainDiskByName(virDomainDef *def, const char *name); -virDomainThrottleGroupDef *qemuDomainThrottleGroupByName(virDomainDef *def, const char *name); - char *qemuDomainGetMasterKeyFilePath(const char *libDir); int qemuDomainMasterKeyReadFile(qemuDomainObjPrivate *priv); diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 62dc879ed4..491d8b03ea 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -656,6 +656,7 @@ qemuDomainAttachDiskGeneric(virDomainObj *vm, virDomainAsyncJob asyncJob) { g_autoptr(qemuBlockStorageSourceChainData) data = NULL; + g_autoptr(qemuBlockThrottleFilterChainData) filterData = NULL; qemuDomainObjPrivate *priv = vm->privateData; g_autoptr(virJSONValue) devprops = NULL; bool extensionDeviceAttached = false; @@ -693,6 +694,18 @@ qemuDomainAttachDiskGeneric(virDomainObj *vm, if (rc < 0) goto rollback; + /* Setup throttling filter chain + * add additional "blockdev-add"(throttle filter) between "blockdev-add" (qemuBlockStorageSourceChainAttach) and "device_add" (qemuDomainAttachExtensionDevice) + */ + if ((filterData = qemuBuildThrottleFilterChainAttachPrepareBlockdev(disk))) { + if (qemuDomainObjEnterMonitorAsync(vm, asyncJob) < 0) + return -1; + /* QMP requests("blockdev-add" with "driver":"throttle") to QEMU */ + rc = qemuBlockThrottleFilterChainAttach(priv->mon, filterData); + qemuDomainObjExitMonitor(vm); + if (rc < 0) + goto rollback; + } if (disk->transient) { g_autoptr(qemuBlockStorageSourceAttachData) backend = NULL; @@ -765,6 +778,8 @@ qemuDomainAttachDiskGeneric(virDomainObj *vm, if (extensionDeviceAttached) ignore_value(qemuDomainDetachExtensionDevice(priv->mon, &disk->info)); + qemuBlockThrottleFilterChainDetach(priv->mon, filterData); + qemuBlockStorageSourceChainDetach(priv->mon, data); qemuDomainObjExitMonitor(vm); @@ -4504,6 +4519,7 @@ qemuDomainRemoveDiskDevice(virQEMUDriver *driver, { qemuDomainDiskPrivate *diskPriv = QEMU_DOMAIN_DISK_PRIVATE(disk); g_autoptr(qemuBlockStorageSourceChainData) diskBackend = NULL; + g_autoptr(qemuBlockThrottleFilterChainData) filterData = NULL; size_t i; qemuDomainObjPrivate *priv = vm->privateData; int ret = -1; @@ -4542,6 +4558,13 @@ qemuDomainRemoveDiskDevice(virQEMUDriver *driver, } } + qemuDomainObjEnterMonitor(vm); + /* QMP request("blockdev-del") to QEMU to delete throttle filter*/ + if ((filterData = qemuBuildThrottleFilterChainAttachPrepareBlockdev(disk))) { + qemuBlockThrottleFilterChainDetach(priv->mon, filterData); + } + qemuDomainObjExitMonitor(vm); + qemuDomainObjEnterMonitor(vm); if (diskBackend) diff --git a/tests/qemustatusxml2xmldata/backup-pull-in.xml b/tests/qemustatusxml2xmldata/backup-pull-in.xml index e7fdc6c478..0a7b8bc7ee 100644 --- a/tests/qemustatusxml2xmldata/backup-pull-in.xml +++ b/tests/qemustatusxml2xmldata/backup-pull-in.xml @@ -234,6 +234,7 @@ <chardevStdioLogd/> <allowReboot value='yes'/> <nodename index='0'/> + <filternodename index='0'/> <fdset index='0'/> <blockjobs active='yes'> <blockjob name='backup-vda-libvirt-3-format' type='backup' state='running' jobflags='0x0'> diff --git a/tests/qemustatusxml2xmldata/blockjob-blockdev-in.xml b/tests/qemustatusxml2xmldata/blockjob-blockdev-in.xml index 380ef053d2..c29bbaca81 100644 --- a/tests/qemustatusxml2xmldata/blockjob-blockdev-in.xml +++ b/tests/qemustatusxml2xmldata/blockjob-blockdev-in.xml @@ -233,6 +233,7 @@ <chardevStdioLogd/> <allowReboot value='yes'/> <nodename index='0'/> + <filternodename index='0'/> <fdset index='0'/> <blockjobs active='yes'> <blockjob name='broken-test' type='broken' state='ready' brokentype='commit'/> diff --git a/tests/qemustatusxml2xmldata/blockjob-mirror-in.xml b/tests/qemustatusxml2xmldata/blockjob-mirror-in.xml index 1bcdeffcb8..0e0f52fecf 100644 --- a/tests/qemustatusxml2xmldata/blockjob-mirror-in.xml +++ b/tests/qemustatusxml2xmldata/blockjob-mirror-in.xml @@ -23,6 +23,7 @@ <channelTargetDir path='/var/lib/libvirt/qemu/channel/target'/> <allowReboot value='yes'/> <nodename index='0'/> + <filternodename index='0'/> <fdset index='0'/> <blockjobs active='yes'/> <agentTimeout>-2</agentTimeout> diff --git a/tests/qemustatusxml2xmldata/migration-in-params-in.xml b/tests/qemustatusxml2xmldata/migration-in-params-in.xml index 03773a089b..5510c139f8 100644 --- a/tests/qemustatusxml2xmldata/migration-in-params-in.xml +++ b/tests/qemustatusxml2xmldata/migration-in-params-in.xml @@ -257,6 +257,7 @@ <chardevStdioLogd/> <allowReboot value='yes'/> <nodename index='0'/> + <filternodename index='0'/> <fdset index='0'/> <blockjobs active='no'/> <agentTimeout>-2</agentTimeout> diff --git a/tests/qemustatusxml2xmldata/migration-out-nbd-bitmaps-in.xml b/tests/qemustatusxml2xmldata/migration-out-nbd-bitmaps-in.xml index 4ee44ffbd4..cd38bf3e30 100644 --- a/tests/qemustatusxml2xmldata/migration-out-nbd-bitmaps-in.xml +++ b/tests/qemustatusxml2xmldata/migration-out-nbd-bitmaps-in.xml @@ -343,6 +343,7 @@ <rememberOwner/> <allowReboot value='yes'/> <nodename index='3'/> + <filternodename index='3'/> <fdset index='0'/> <blockjobs active='yes'> <blockjob name='drive-virtio-disk0' type='copy' state='ready' jobflags='0x0'> diff --git a/tests/qemustatusxml2xmldata/migration-out-nbd-out.xml b/tests/qemustatusxml2xmldata/migration-out-nbd-out.xml index de92146eaa..0d8cb8be6a 100644 --- a/tests/qemustatusxml2xmldata/migration-out-nbd-out.xml +++ b/tests/qemustatusxml2xmldata/migration-out-nbd-out.xml @@ -260,6 +260,7 @@ <chardevStdioLogd/> <allowReboot value='yes'/> <nodename index='0'/> + <filternodename index='0'/> <fdset index='0'/> <blockjobs active='no'/> <agentTimeout>-2</agentTimeout> diff --git a/tests/qemustatusxml2xmldata/migration-out-nbd-tls-out.xml b/tests/qemustatusxml2xmldata/migration-out-nbd-tls-out.xml index 6bdd128259..75b019cb23 100644 --- a/tests/qemustatusxml2xmldata/migration-out-nbd-tls-out.xml +++ b/tests/qemustatusxml2xmldata/migration-out-nbd-tls-out.xml @@ -289,6 +289,7 @@ <chardevStdioLogd/> <allowReboot value='yes'/> <nodename index='0'/> + <filternodename index='0'/> <fdset index='0'/> <blockjobs active='no'/> <agentTimeout>-2</agentTimeout> diff --git a/tests/qemustatusxml2xmldata/migration-out-params-in.xml b/tests/qemustatusxml2xmldata/migration-out-params-in.xml index 24ee86e4c0..1e786953e7 100644 --- a/tests/qemustatusxml2xmldata/migration-out-params-in.xml +++ b/tests/qemustatusxml2xmldata/migration-out-params-in.xml @@ -271,6 +271,7 @@ <chardevStdioLogd/> <allowReboot value='yes'/> <nodename index='0'/> + <filternodename index='0'/> <fdset index='0'/> <blockjobs active='no'/> <agentTimeout>-2</agentTimeout> diff --git a/tests/qemustatusxml2xmldata/modern-in.xml b/tests/qemustatusxml2xmldata/modern-in.xml index f0f5df84ab..84240595ff 100644 --- a/tests/qemustatusxml2xmldata/modern-in.xml +++ b/tests/qemustatusxml2xmldata/modern-in.xml @@ -261,6 +261,7 @@ <chardevStdioLogd/> <allowReboot value='yes'/> <nodename index='123'/> + <filternodename index='123'/> <fdset index='321'/> <blockjobs active='no'/> <agentTimeout>-2</agentTimeout> diff --git a/tests/qemustatusxml2xmldata/upgrade-out.xml b/tests/qemustatusxml2xmldata/upgrade-out.xml index e663b3dbb5..2f33166579 100644 --- a/tests/qemustatusxml2xmldata/upgrade-out.xml +++ b/tests/qemustatusxml2xmldata/upgrade-out.xml @@ -259,6 +259,7 @@ <chardevStdioLogd/> <allowReboot value='yes'/> <nodename index='0'/> + <filternodename index='0'/> <fdset index='0'/> <blockjobs active='no'/> <agentTimeout>-2</agentTimeout> diff --git a/tests/qemustatusxml2xmldata/vcpus-multi-in.xml b/tests/qemustatusxml2xmldata/vcpus-multi-in.xml index fa6a6a99f4..b07a04a840 100644 --- a/tests/qemustatusxml2xmldata/vcpus-multi-in.xml +++ b/tests/qemustatusxml2xmldata/vcpus-multi-in.xml @@ -309,6 +309,7 @@ <channelTargetDir path='/var/lib/libvirt/qemu/channel/target'/> <allowReboot value='yes'/> <nodename index='0'/> + <filternodename index='0'/> <fdset index='0'/> <blockjobs active='no'/> <agentTimeout>-2</agentTimeout> -- 2.34.1

From: Chun Feng Wu <wucf@linux.ibm.com> * Add qemuBuildThrottleGroupCommandLine in qemuBuildCommandLine * Add qemuBuildThrottleFiltersCommandLine in qemuBuildDiskCommandLine * Make sure referenced throttle group exists Signed-off-by: Chun Feng Wu <wucf@linux.ibm.com> --- src/conf/domain_validate.c | 14 ++++++ src/qemu/qemu_command.c | 87 ++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_command.h | 3 ++ 3 files changed, 104 insertions(+) diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index 395e036e8f..fffe274afc 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -663,6 +663,7 @@ virDomainDiskDefValidate(const virDomainDef *def, const virDomainDiskDef *disk) { virStorageSource *next; + size_t i; /* disk target is used widely in other code so it must be validated first */ if (!disk->dst) { @@ -942,6 +943,19 @@ virDomainDiskDefValidate(const virDomainDef *def, return -1; } + /* referenced group should be defined */ + for (i = 0; i < disk->nthrottlefilters; i++) { + virDomainThrottleFilterDef *filter = disk->throttlefilters[i]; + if (filter) { + if (!virDomainThrottleGroupFind(def, filter->group_name)) { + virReportError(VIR_ERR_XML_ERROR, + _("throttle group '%1$s' not found"), + filter->group_name); + return -1; + } + } + } + return 0; } diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 278b706616..e7cb20fd71 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1582,6 +1582,13 @@ qemuDiskConfigThrottleFilterChainEnabled(const virDomainDiskDef *disk) return disk->nthrottlefilters > 0; } +bool +qemuDiskConfigThrottleFilterEnabled(const virDomainThrottleGroupDef *group) +{ + return !!group->group_name && + virDomainBlockIoTuneInfoHasAny(group); +} + /** * qemuDiskBusIsSD: @@ -2218,6 +2225,42 @@ qemuBuildBlockStorageSourceAttachDataCommandline(virCommand *cmd, return 0; } +static int +qemuBuildBlockThrottleFilterCommandline(virCommand *cmd, + qemuBlockThrottleFilterAttachData *data) +{ + char *tmp; + + if (data->filterProps) { + if (!(tmp = virJSONValueToString(data->filterProps, false))) + return -1; + + virCommandAddArgList(cmd, "-blockdev", tmp, NULL); + VIR_FREE(tmp); + } + + return 0; +} + +static int +qemuBuildThrottleFiltersCommandLine(virCommand *cmd, + virDomainDiskDef *disk) +{ + g_autoptr(qemuBlockThrottleFilterChainData) data = NULL; + size_t i; + + if (!(data = qemuBuildThrottleFilterChainAttachPrepareBlockdev(disk))) { + return -1; + } else { + for (i = 0; i < data->nfilterdata; i++) { + if (qemuBuildBlockThrottleFilterCommandline(cmd, + data->filterdata[i]) < 0) + return -1; + } + } + + return 0; +} static int qemuBuildDiskSourceCommandLine(virCommand *cmd, @@ -2276,6 +2319,9 @@ qemuBuildDiskCommandLine(virCommand *cmd, if (qemuBuildDiskSourceCommandLine(cmd, disk, qemuCaps) < 0) return -1; + if (qemuBuildThrottleFiltersCommandLine(cmd, disk) < 0) + return -1; + /* SD cards are currently instantiated via -drive if=sd, so the -device * part must be skipped */ if (qemuDiskBusIsSD(disk->bus)) @@ -7448,6 +7494,44 @@ qemuBuildIOThreadCommandLine(virCommand *cmd, return 0; } +/** + * qemuBuildThrottleGroupCommandLine: + * @cmd: the command to modify + * @def: domain definition + * @qemuCaps: qemu capabilities object + * + * build throttle group object in json format + * e.g. -object '{"qom-type":"throttle-group","id":"limit0","limits":{"iops-total":200}}' + */ +static int +qemuBuildThrottleGroupCommandLine(virCommand *cmd, + const virDomainDef *def, + virQEMUCaps *qemuCaps) +{ + size_t i; + + for (i = 0; i < def->nthrottlegroups; i++) { + g_autoptr(virJSONValue) props = NULL; + g_autoptr(virJSONValue) limits = NULL; + virDomainThrottleGroupDef *group = def->throttlegroups[i]; + + if (!qemuDiskConfigThrottleFilterEnabled(group)) { + continue; + } + + limits = qemuMonitorThrottleGroupLimits(group); + + if (qemuMonitorCreateObjectProps(&props, "throttle-group", group->group_name, + "a:limits", &limits, + NULL) < 0) + return -1; + + if (qemuBuildObjectCommandlineFromJSON(cmd, props, qemuCaps) < 0) + return -1; + } + + return 0; +} static int qemuBuildNumaCellCache(virCommand *cmd, @@ -10483,6 +10567,9 @@ qemuBuildCommandLine(virDomainObj *vm, if (qemuBuildIOThreadCommandLine(cmd, def, qemuCaps) < 0) return NULL; + if (qemuBuildThrottleGroupCommandLine(cmd, def, qemuCaps) < 0) + return NULL; + if (virDomainNumaGetNodeCount(def->numa) && qemuBuildNumaCommandLine(cfg, def, cmd, priv) < 0) return NULL; diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index e2dee47906..9b8951d95f 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -122,6 +122,9 @@ qemuBuildThrottleFilterChainAttachPrepareBlockdev(virDomainDiskDef *disk); bool qemuDiskConfigThrottleFilterChainEnabled(const virDomainDiskDef *disk); +bool +qemuDiskConfigThrottleFilterEnabled(const virDomainThrottleGroupDef *group); + virJSONValue * qemuBuildDiskDeviceProps(const virDomainDef *def, virDomainDiskDef *disk, -- 2.34.1

From: Chun Feng Wu <wucf@linux.ibm.com> * Add new elements '<throttlegroups>' and '<throttlefilters>' * <ThrottleGroups> contains <ThrottleGroup> defintions * <ThrottleFilters> can include multiple throttlegroup references to form filter chain in qemu * Chained throttle filters feature in qemu is described at https://github.com/qemu/qemu/blob/master/docs/throttle.txt Signed-off-by: Chun Feng Wu <wucf@linux.ibm.com> --- docs/formatdomain.rst | 48 +++++++++ src/conf/schemas/domaincommon.rng | 164 +++++++++++++++++++++++++++++- 2 files changed, 211 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index e2f66b982c..ee9ee8b10c 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -1957,6 +1957,34 @@ advertisements to the guest OS. (NB: Only qemu driver support) the guest OS itself can choose to circumvent the unavailability of the sleep states (e.g. S4 by turning off completely). +Throttle Group Management +------------------------- + +:since:`Since 10.3.0` it is possible to create multiple named throttle groups +and then reference them within ``throttlefilters`` to form filter chain in QEMU for +specific disk. The limits(throttlegroups) are shared within domain, hence the same group +can be referenced by different filters. + +:: + + <domain> + ... + <throttlegroups> + <throttlegroup> + <group_name>limit0</group_name> + <total_bytes_sec>10000000</total_bytes_sec> + <read_iops_sec>400000</read_iops_sec> + <write_iops_sec>100000</write_iops_sec> + </throttlegroup> + </throttlegroups> + ... + </domain> + +All throttlegroups are listed within the ``throttlegroups`` element + +``throttlegroup`` + It has the same sub-elements as ``iotune`` (See `Hard drives, floppy disks, CDROMs`_), + The difference is that <group_name> is required. Hypervisor features ------------------- @@ -2704,6 +2732,15 @@ paravirtualized driver is specified via the ``disk`` element. <source dev='/dev/vhost-vdpa-0' /> <target dev='vdg' bus='virtio'/> </disk> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2' /> + <source file='/var/lib/libvirt/images/disk.qcow2'/> + <target dev='vdh' bus='virtio'/> + <throttlefilters> + <throttlefilter group='limit2'/> + <throttlefilter group='limit012'/> + </throttlefilters> + </disk> </devices> ... @@ -3185,6 +3222,17 @@ paravirtualized driver is specified via the ``disk`` element. :since:`since after 0.4.4`; "sata" attribute value :since:`since 0.9.7`; "removable" attribute value :since:`since 1.1.3`; "rotation_rate" attribute value :since:`since 7.3.0` +``throttlefilters`` + The optional ``throttlefilters`` element provides the ability to provide additional + per-device throttle chain :since:`Since 10.3.0` + For example, if we have four different disks and we want to limit I/O for each one + and we also want to limit combined I/O of all four disks, we can leverage + ``throttlefilters`` to achieve this goal by setting two ``throttlefilter`` for + each disk: disk's own filter and combined filter. ``throttlefilters`` and ``iotune`` + should be used exclusively. + + ``throttlefilter`` + The optional ``throttlefilter`` element is to reference defined throttle group. ``iotune`` The optional ``iotune`` element provides the ability to provide additional per-device I/O tuning, with values that can vary for each device (contrast diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index f386e46fae..e77149c6ca 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -51,6 +51,7 @@ </zeroOrMore> <ref name="os"/> <ref name="clock"/> + <ref name="throttlegroups"/> <ref name="resources"/> <ref name="features"/> <ref name="events"/> @@ -1577,7 +1578,10 @@ <ref name="encryption"/> </optional> <optional> - <ref name="diskIoTune"/> + <choice> + <ref name="throttlefilters"/> + <ref name="diskIoTune"/> + </choice> </optional> <optional> <ref name="alias"/> @@ -6637,6 +6641,164 @@ </interleave> </element> </define> + <!-- + throttlegroup is similar to <iotune>, however, group_name is required, which is different from <iotune> + --> + <define name="throttlegroup"> + <element name="throttlegroup"> + <interleave> + <!-- required --> + <element name="group_name"> + <text/> + </element> + <choice> + <element name="total_bytes_sec"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_bytes_sec"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_bytes_sec"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + <choice> + <element name="total_iops_sec"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_iops_sec"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_iops_sec"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + <choice> + <element name="total_bytes_sec_max"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_bytes_sec_max"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_bytes_sec_max"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + <choice> + <element name="total_iops_sec_max"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_iops_sec_max"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_iops_sec_max"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + <optional> + <element name="size_iops_sec"> + <data type="unsignedLong"/> + </element> + </optional> + <choice> + <element name="total_bytes_sec_max_length"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_bytes_sec_max_length"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_bytes_sec_max_length"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + <choice> + <element name="total_iops_sec_max_length"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_iops_sec_max_length"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_iops_sec_max_length"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + </interleave> + </element> + </define> + <!-- + A set of optional throttlegroups + --> + <define name="throttlegroups"> + <optional> + <element name="throttlegroups"> + <zeroOrMore> + <ref name="throttlegroup"/> + </zeroOrMore> + </element> + </optional> + </define> + <!-- + A set of throttlefilters to reference throttlegroups + --> + <define name="throttlefilters"> + <element name="throttlefilters"> + <zeroOrMore> + <element name="throttlefilter"> + <attribute name="group"> + <data type="string"/> + </attribute> + </element> + </zeroOrMore> + </element> + </define> <!-- A set of optional features: PAE, APIC, ACPI, GIC, TCG, HyperV Enlightenment, KVM features, paravirtual spinlocks and HAP support -- 2.34.1

From: Chun Feng Wu <wucf@linux.ibm.com> * Test "Set": testDomainSetThrottleGroup * Test "Get": testDomainGetThrottleGroup * Test "Del": testDomainDelThrottleGroup Signed-off-by: Chun Feng Wu <wucf@linux.ibm.com> --- src/test/test_driver.c | 380 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 380 insertions(+) diff --git a/src/test/test_driver.c b/src/test/test_driver.c index ed0cdc0dab..d4c199fd24 100644 --- a/src/test/test_driver.c +++ b/src/test/test_driver.c @@ -4115,6 +4115,383 @@ testDomainGetBlockIoTune(virDomainPtr dom, virDomainObjEndAPI(&vm); return ret; } + +static int +testDomainSetThrottleGroup(virDomainPtr dom, + const char *group, + virTypedParameterPtr params, + int nparams, + unsigned int flags) +{ + virDomainObj *vm = NULL; + virDomainDef *def = NULL; + virDomainThrottleGroupDef info = { 0 }; + virDomainThrottleGroupDef *cur_info = NULL; + virTypedParameterPtr eventParams = NULL; + int eventNparams = 0; + int eventMaxparams = 0; + int ret = -1; + size_t i; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG, -1); + if (virTypedParamsValidate(params, nparams, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_SIZE_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_GROUP_NAME, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, + NULL) < 0) + return -1; + + if (!(vm = testDomObjFromDomain(dom))) + return -1; + + if (!(def = virDomainObjGetOneDef(vm, flags))) + goto cleanup; + + if (virTypedParamsAddString(&eventParams, &eventNparams, &eventMaxparams, + VIR_DOMAIN_THROTTLE_GROUP, group) < 0) + goto cleanup; + +#define SET_THROTTLE_FIELD(FIELD, STR, TUNABLE_STR) \ + if (STREQ(param->field, STR)) { \ + info.FIELD = param->value.ul; \ + if (virTypedParamsAddULLong(&eventParams, &eventNparams, \ + &eventMaxparams, \ + TUNABLE_STR, \ + param->value.ul) < 0) \ + goto cleanup; \ + continue; \ + } + + for (i = 0; i < nparams; i++) { + virTypedParameterPtr param = ¶ms[i]; + + if (param->value.ul > TEST_BLOCK_IOTUNE_MAX) { + virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, + _("throttle group value must be no more than %1$llu"), + TEST_BLOCK_IOTUNE_MAX); + goto cleanup; + } + + SET_THROTTLE_FIELD(total_bytes_sec, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC, + VIR_DOMAIN_TUNABLE_BLKDEV_TOTAL_BYTES_SEC); + SET_THROTTLE_FIELD(read_bytes_sec, + VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC, + VIR_DOMAIN_TUNABLE_BLKDEV_READ_BYTES_SEC); + SET_THROTTLE_FIELD(write_bytes_sec, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC, + VIR_DOMAIN_TUNABLE_BLKDEV_WRITE_BYTES_SEC); + SET_THROTTLE_FIELD(total_iops_sec, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC, + VIR_DOMAIN_TUNABLE_BLKDEV_TOTAL_IOPS_SEC); + SET_THROTTLE_FIELD(read_iops_sec, + VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC, + VIR_DOMAIN_TUNABLE_BLKDEV_READ_IOPS_SEC); + SET_THROTTLE_FIELD(write_iops_sec, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC, + VIR_DOMAIN_TUNABLE_BLKDEV_WRITE_IOPS_SEC); + + SET_THROTTLE_FIELD(total_bytes_sec_max, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX, + VIR_DOMAIN_TUNABLE_BLKDEV_TOTAL_BYTES_SEC_MAX); + SET_THROTTLE_FIELD(read_bytes_sec_max, + VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC_MAX, + VIR_DOMAIN_TUNABLE_BLKDEV_READ_BYTES_SEC_MAX); + SET_THROTTLE_FIELD(write_bytes_sec_max, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC_MAX, + VIR_DOMAIN_TUNABLE_BLKDEV_WRITE_BYTES_SEC_MAX); + SET_THROTTLE_FIELD(total_iops_sec_max, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC_MAX, + VIR_DOMAIN_TUNABLE_BLKDEV_TOTAL_IOPS_SEC_MAX); + SET_THROTTLE_FIELD(read_iops_sec_max, + VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC_MAX, + VIR_DOMAIN_TUNABLE_BLKDEV_READ_IOPS_SEC_MAX); + SET_THROTTLE_FIELD(write_iops_sec_max, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC_MAX, + VIR_DOMAIN_TUNABLE_BLKDEV_WRITE_IOPS_SEC_MAX); + SET_THROTTLE_FIELD(size_iops_sec, + VIR_DOMAIN_BLOCK_IOTUNE_SIZE_IOPS_SEC, + VIR_DOMAIN_TUNABLE_BLKDEV_SIZE_IOPS_SEC); + + /* NB: Cannot use macro since this is a value.s not a value.ul */ + if (STREQ(param->field, VIR_DOMAIN_BLOCK_IOTUNE_GROUP_NAME)) { + VIR_FREE(info.group_name); + info.group_name = g_strdup(param->value.s); + if (virTypedParamsAddString(&eventParams, &eventNparams, + &eventMaxparams, + VIR_DOMAIN_TUNABLE_BLKDEV_GROUP_NAME, + param->value.s) < 0) + goto cleanup; + continue; + } + + SET_THROTTLE_FIELD(total_bytes_sec_max_length, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX_LENGTH, + VIR_DOMAIN_TUNABLE_BLKDEV_TOTAL_BYTES_SEC_MAX_LENGTH); + SET_THROTTLE_FIELD(read_bytes_sec_max_length, + VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC_MAX_LENGTH, + VIR_DOMAIN_TUNABLE_BLKDEV_READ_BYTES_SEC_MAX_LENGTH); + SET_THROTTLE_FIELD(write_bytes_sec_max_length, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC_MAX_LENGTH, + VIR_DOMAIN_TUNABLE_BLKDEV_WRITE_BYTES_SEC_MAX_LENGTH); + SET_THROTTLE_FIELD(total_iops_sec_max_length, + VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC_MAX_LENGTH, + VIR_DOMAIN_TUNABLE_BLKDEV_TOTAL_IOPS_SEC_MAX_LENGTH); + SET_THROTTLE_FIELD(read_iops_sec_max_length, + VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC_MAX_LENGTH, + VIR_DOMAIN_TUNABLE_BLKDEV_READ_IOPS_SEC_MAX_LENGTH); + SET_THROTTLE_FIELD(write_iops_sec_max_length, + VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC_MAX_LENGTH, + VIR_DOMAIN_TUNABLE_BLKDEV_WRITE_IOPS_SEC_MAX_LENGTH); + } + +#undef SET_THROTTLE_FIELD + + if ((info.total_bytes_sec && info.read_bytes_sec) || + (info.total_bytes_sec && info.write_bytes_sec)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("total and read/write of bytes_sec cannot be set at the same time")); + goto cleanup; + } + + if ((info.total_iops_sec && info.read_iops_sec) || + (info.total_iops_sec && info.write_iops_sec)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("total and read/write of iops_sec cannot be set at the same time")); + goto cleanup; + } + + if ((info.total_bytes_sec_max && info.read_bytes_sec_max) || + (info.total_bytes_sec_max && info.write_bytes_sec_max)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("total and read/write of bytes_sec_max cannot be set at the same time")); + goto cleanup; + } + + if ((info.total_iops_sec_max && info.read_iops_sec_max) || + (info.total_iops_sec_max && info.write_iops_sec_max)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("total and read/write of iops_sec_max cannot be set at the same time")); + goto cleanup; + } + +#define TEST_BLOCK_IOTUNE_MAX_CHECK(FIELD, FIELD_MAX) \ + do { \ + if (info.FIELD > info.FIELD_MAX) { \ + virReportError(VIR_ERR_INVALID_ARG, \ + _("%1$s cannot be set higher than %2$s"), \ + #FIELD, #FIELD_MAX); \ + goto cleanup; \ + } \ + } while (0); + + TEST_BLOCK_IOTUNE_MAX_CHECK(total_bytes_sec, total_bytes_sec_max); + TEST_BLOCK_IOTUNE_MAX_CHECK(read_bytes_sec, read_bytes_sec_max); + TEST_BLOCK_IOTUNE_MAX_CHECK(write_bytes_sec, write_bytes_sec_max); + TEST_BLOCK_IOTUNE_MAX_CHECK(total_iops_sec, total_iops_sec_max); + TEST_BLOCK_IOTUNE_MAX_CHECK(read_iops_sec, read_iops_sec_max); + TEST_BLOCK_IOTUNE_MAX_CHECK(write_iops_sec, write_iops_sec_max); + +#undef TEST_BLOCK_IOTUNE_MAX_CHECK + + cur_info = virDomainThrottleGroupByName(def, group); + if (cur_info != NULL) { + virDomainThrottleGroupUpdate(def, &info); + // goto cleanup; + }else{ + virDomainThrottleGroupAdd(def, &info); + } + ret = 0; + + cleanup: + VIR_FREE(info.group_name); + virDomainObjEndAPI(&vm); + if (eventNparams) + virTypedParamsFree(eventParams, eventNparams); + return ret; +} + +static int +testDomainGetThrottleGroup(virDomainPtr dom, + const char *groupname, + virTypedParameterPtr params, + int *nparams, + unsigned int flags) +{ + virDomainObj *vm = NULL; + virDomainDef *def = NULL; + virDomainThrottleGroupDef groupDef = {0}; + virDomainThrottleGroupDef *reply = &groupDef; + int ret = -1; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG | + VIR_TYPED_PARAM_STRING_OKAY, -1); + + /* We don't return strings, and thus trivially support this flag. */ + flags &= ~VIR_TYPED_PARAM_STRING_OKAY; + + if (*nparams == 0) { + *nparams = 20; + return 0; + } + + if (!(vm = testDomObjFromDomain(dom))) + return -1; + + if (!(def = virDomainObjGetOneDef(vm, flags))) + goto cleanup; + + + if (!(reply = virDomainThrottleGroupByName(def, groupname))) { + virReportError(VIR_ERR_INVALID_ARG, + _("throttle group '%1$s' was not found in the domain config"), + groupname); + goto cleanup; + } + reply->group_name = g_strdup(groupname); + TEST_SET_PARAM(0, VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC, + VIR_TYPED_PARAM_ULLONG, reply->total_bytes_sec); + TEST_SET_PARAM(1, VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC, + VIR_TYPED_PARAM_ULLONG, reply->read_bytes_sec); + TEST_SET_PARAM(2, VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC, + VIR_TYPED_PARAM_ULLONG, reply->write_bytes_sec); + + TEST_SET_PARAM(3, VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, reply->total_iops_sec); + TEST_SET_PARAM(4, VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, reply->read_iops_sec); + TEST_SET_PARAM(5, VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, reply->write_iops_sec); + + TEST_SET_PARAM(6, VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, reply->total_bytes_sec_max); + TEST_SET_PARAM(7, VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, reply->read_bytes_sec_max); + TEST_SET_PARAM(8, VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, reply->write_bytes_sec_max); + + TEST_SET_PARAM(9, VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, reply->total_iops_sec_max); + TEST_SET_PARAM(10, VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, reply->read_iops_sec_max); + TEST_SET_PARAM(11, VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC_MAX, + VIR_TYPED_PARAM_ULLONG, reply->write_iops_sec_max); + + TEST_SET_PARAM(12, VIR_DOMAIN_BLOCK_IOTUNE_SIZE_IOPS_SEC, + VIR_TYPED_PARAM_ULLONG, reply->size_iops_sec); + + TEST_SET_PARAM(13, VIR_DOMAIN_BLOCK_IOTUNE_GROUP_NAME, + VIR_TYPED_PARAM_STRING, reply->group_name); + reply->group_name = NULL; + + TEST_SET_PARAM(14, VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, reply->total_bytes_sec_max_length); + TEST_SET_PARAM(15, VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, reply->read_bytes_sec_max_length); + TEST_SET_PARAM(16, VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, reply->write_bytes_sec_max_length); + + TEST_SET_PARAM(17, VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_IOPS_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, reply->total_iops_sec_max_length); + TEST_SET_PARAM(18, VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, reply->read_iops_sec_max_length); + TEST_SET_PARAM(19, VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC_MAX_LENGTH, + VIR_TYPED_PARAM_ULLONG, reply->write_iops_sec_max_length); + + if (*nparams > 20) + *nparams = 20; + + ret = 0; + + cleanup: + if (reply != NULL && reply->group_name != NULL) { + g_free(reply->group_name); + } + virDomainObjEndAPI(&vm); + return ret; +} + +static int +testDomainDelThrottleGroup(virDomainPtr dom, + const char *groupname, + unsigned int flags) +{ + virDomainObj *vm = NULL; + virDomainDef *def = NULL; + virDomainThrottleGroupDef groupDef = {0}; + virDomainThrottleGroupDef *reply = &groupDef; + int ret = -1; + + virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | + VIR_DOMAIN_AFFECT_CONFIG | + VIR_TYPED_PARAM_STRING_OKAY, -1); + + /* We don't return strings, and thus trivially support this flag. */ + flags &= ~VIR_TYPED_PARAM_STRING_OKAY; + + if (!(vm = testDomObjFromDomain(dom))) + return -1; + + /* the API check guarantees that only one of the definitions will be set */ + if (!(def = virDomainObjGetOneDef(vm, flags))) + goto cleanup; + + if (!(reply = virDomainThrottleGroupByName(def, groupname))) { + virReportError(VIR_ERR_INVALID_ARG, + _("throttle group '%1$s' was not found in the domain config"), + groupname); + goto cleanup; + } + + ret = 0; + + cleanup: + if (reply) { + VIR_FREE(reply->group_name); + } + virDomainObjEndAPI(&vm); + return ret; +} + #undef TEST_SET_PARAM @@ -10468,6 +10845,9 @@ static virHypervisorDriver testHypervisorDriver = { .domainGetInterfaceParameters = testDomainGetInterfaceParameters, /* 5.6.0 */ .domainSetBlockIoTune = testDomainSetBlockIoTune, /* 5.7.0 */ .domainGetBlockIoTune = testDomainGetBlockIoTune, /* 5.7.0 */ + .domainSetThrottleGroup = testDomainSetThrottleGroup, /* 10.3.0 */ + .domainGetThrottleGroup = testDomainGetThrottleGroup, /* 10.3.0 */ + .domainDelThrottleGroup = testDomainDelThrottleGroup, /* 10.3.0 */ .domainSetBlkioParameters = testDomainSetBlkioParameters, /* 7.7.0 */ .domainGetBlkioParameters = testDomainGetBlkioParameters, /* 7.7.0 */ .connectListDefinedDomains = testConnectListDefinedDomains, /* 0.1.11 */ -- 2.34.1

From: Chun Feng Wu <wucf@linux.ibm.com> Within "testQemuMonitorJSONqemuMonitorJSONUpdateThrottleGroup" * Test qemuMonitorJSONGetThrottleGroup * Test qemuMonitorJSONUpdateThrottleGroup, which updates limits through "qom-set" Signed-off-by: Chun Feng Wu <wucf@linux.ibm.com> --- tests/qemumonitorjsontest.c | 89 +++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/tests/qemumonitorjsontest.c b/tests/qemumonitorjsontest.c index 45cee23798..dec26d9e61 100644 --- a/tests/qemumonitorjsontest.c +++ b/tests/qemumonitorjsontest.c @@ -147,6 +147,32 @@ const char *queryBlockReply = " \"id\": \"libvirt-10\"" "}"; +const char *qomGetReply = +"{" +" \"return\": {" +" \"bps-total\": 1," +" \"bps-read\": 2," +" \"bps-write\": 3," +" \"iops-total\": 4," +" \"iops-read\": 5," +" \"iops-write\": 6," +" \"bps-total-max\": 7," +" \"bps-read-max\": 8," +" \"bps-write-max\": 9," +" \"iops-total-max\": 10," +" \"iops-read-max\": 11," +" \"iops-write-max\": 12," +" \"iops-size\": 13," +" \"bps-total-max-length\": 15," +" \"bps-read-max-length\": 16," +" \"bps-write-max-length\": 17," +" \"iops-total-max-length\": 18," +" \"iops-read-max-length\": 19," +" \"iops-write-max-length\": 20" +" }," +" \"id\": \"libvirt-12\"" +"}"; + static int testQemuMonitorJSONGetStatus(const void *opaque) { @@ -1853,6 +1879,68 @@ testQemuMonitorJSONqemuMonitorJSONSetBlockIoThrottle(const void *opaque) return ret; } +static int +testQemuMonitorJSONqemuMonitorJSONUpdateThrottleGroup(const void *opaque) +{ + const testGenericData *data = opaque; + virDomainXMLOption *xmlopt = data->xmlopt; + int ret = -1; + virDomainBlockIoTuneInfo info, expectedInfo; + g_autoptr(qemuMonitorTest) test = NULL; + + if (!(test = qemuMonitorTestNewSchema(xmlopt, data->schema))) + return -1; + + expectedInfo = (virDomainBlockIoTuneInfo) {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, NULL, 15, 16, 17, 18, 19, 20}; + expectedInfo.group_name = g_strdup("limit0"); + + if (qemuMonitorTestAddItem(test, "qom-get", qomGetReply) < 0) + return -1; + + if (qemuMonitorTestAddItemVerbatim(test, + "{\"execute\":\"qom-set\"," + " \"arguments\":{\"property\": \"limits\"," + " \"path\": \"limit1\"," + " \"value\":{\"bps-total\": 1," + " \"bps-read\": 2," + " \"bps-write\": 3," + " \"iops-total\": 4," + " \"iops-read\": 5," + " \"iops-write\": 6," + " \"bps-total-max\": 7," + " \"bps-read-max\": 8," + " \"bps-write-max\": 9," + " \"iops-total-max\": 10," + " \"iops-read-max\": 11," + " \"iops-write-max\": 12," + " \"iops-size\": 13," + " \"bps-total-max-length\": 15," + " \"bps-read-max-length\": 16," + " \"bps-write-max-length\": 17," + " \"iops-total-max-length\": 18," + " \"iops-read-max-length\": 19," + " \"iops-write-max-length\": 20}}," + " \"id\":\"libvirt-2\"}", + NULL, + "{ \"return\" : {}}") < 0) + return -1; + + if (qemuMonitorJSONGetThrottleGroup(qemuMonitorTestGetMonitor(test), + "limit0", &info) < 0) + goto cleanup; + + if (testValidateGetBlockIoThrottle(&info, &expectedInfo) < 0) + goto cleanup; + + if (qemuMonitorJSONUpdateThrottleGroup(qemuMonitorTestGetMonitor(test), + "limit1", &info) < 0) + goto cleanup; + + ret = 0; + cleanup: + return ret; +} + static int testQemuMonitorJSONqemuMonitorJSONGetTargetArch(const void *opaque) { @@ -2899,6 +2987,7 @@ mymain(void) DO_TEST(qemuMonitorJSONGetMigrationStats); DO_TEST(qemuMonitorJSONGetChardevInfo); DO_TEST(qemuMonitorJSONSetBlockIoThrottle); + DO_TEST(qemuMonitorJSONUpdateThrottleGroup); DO_TEST(qemuMonitorJSONGetTargetArch); DO_TEST(qemuMonitorJSONGetMigrationCapabilities); DO_TEST(qemuMonitorJSONQueryCPUsFast); -- 2.34.1

From: Chun Feng Wu <wucf@linux.ibm.com> * Add new cmds: throttlegroupset, throttlegrouplist, throttlegroupinfo, throttlegroupdel * Update "attach_disk" to support new option: throttle-groups to form filter chain in QEMU for specific disk Signed-off-by: Chun Feng Wu <wucf@linux.ibm.com> --- tools/virsh-completer-domain.c | 62 ++++ tools/virsh-completer-domain.h | 11 + tools/virsh-domain.c | 530 +++++++++++++++++++++++++++++++++ 3 files changed, 603 insertions(+) diff --git a/tools/virsh-completer-domain.c b/tools/virsh-completer-domain.c index 2891d1399c..f99b5a4a2f 100644 --- a/tools/virsh-completer-domain.c +++ b/tools/virsh-completer-domain.c @@ -247,6 +247,68 @@ virshDomainMigrateDisksCompleter(vshControl *ctl, return virshDomainDiskTargetListCompleter(ctl, cmd, "migrate-disks"); } +char ** +virshDomainThrottleGroupCompleter(vshControl *ctl, + const vshCmd *cmd, + unsigned int flags) +{ + virshControl *priv = ctl->privData; + g_autoptr(xmlDoc) xmldoc = NULL; + g_autoptr(xmlXPathContext) ctxt = NULL; + g_autofree xmlNodePtr *groups = NULL; + int ngroups; + size_t i; + g_auto(GStrv) tmp = NULL; + + virCheckFlags(0, NULL); + + if (!priv->conn || virConnectIsAlive(priv->conn) <= 0) + return NULL; + + if (virshDomainGetXML(ctl, cmd, 0, &xmldoc, &ctxt) < 0) + return NULL; + + ngroups = virXPathNodeSet("./throttlegroups/throttlegroup", ctxt, &groups); + if (ngroups < 0) + return NULL; + + tmp = g_new0(char *, ngroups + 1); + + for (i = 0; i < ngroups; i++) { + ctxt->node = groups[i]; + if (!(tmp[i] = virXPathString("string(./group_name)", ctxt))) + return NULL; + } + + return g_steal_pointer(&tmp); +} + +static char ** +virshDomainThrottleGroupListCompleter(vshControl *ctl, + const vshCmd *cmd, + const char *argname) +{ + const char *curval = NULL; + g_auto(GStrv) groups = virshDomainThrottleGroupCompleter(ctl, cmd, 0); + + if (vshCommandOptStringQuiet(ctl, cmd, argname, &curval) < 0) + return NULL; + + if (!groups) + return NULL; + + return virshCommaStringListComplete(curval, (const char **) groups); +} + +char ** +virshDomainThrottleGroupsCompleter(vshControl *ctl, + const vshCmd *cmd, + unsigned int completeflags G_GNUC_UNUSED) +{ + return virshDomainThrottleGroupListCompleter(ctl, cmd, "throttle-groups"); +} + + char ** virshDomainUndefineStorageDisksCompleter(vshControl *ctl, diff --git a/tools/virsh-completer-domain.h b/tools/virsh-completer-domain.h index 27cf963912..46a652994a 100644 --- a/tools/virsh-completer-domain.h +++ b/tools/virsh-completer-domain.h @@ -41,6 +41,17 @@ virshDomainDiskTargetCompleter(vshControl *ctl, const vshCmd *cmd, unsigned int flags); +char ** +virshDomainThrottleGroupCompleter(vshControl *ctl, + const vshCmd *cmd, + unsigned int flags); + +char ** +virshDomainThrottleGroupsCompleter(vshControl *ctl, + const vshCmd *cmd, + unsigned int completeflags); + + char ** virshDomainInterfaceStateCompleter(vshControl *ctl, const vshCmd *cmd, diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 3d9c48629a..9bee73faee 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -522,6 +522,11 @@ static const vshCmdOptDef opts_attach_disk[] = { .type = VSH_OT_STRING, .help = N_("host socket for source of disk device") }, + {.name = "throttle-groups", + .type = VSH_OT_STRING, + .completer = virshDomainThrottleGroupsCompleter, + .help = N_("comma separated list of throttle groups to be applied") + }, VIRSH_COMMON_OPT_DOMAIN_PERSISTENT, VIRSH_COMMON_OPT_DOMAIN_CONFIG, VIRSH_COMMON_OPT_DOMAIN_LIVE, @@ -611,7 +616,10 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd) const char *host_name = NULL; const char *host_transport = NULL; const char *host_socket = NULL; + const char *throttle_groups = NULL; + g_autofree char **groups = NULL; int ret; + size_t i; unsigned int flags = VIR_DOMAIN_AFFECT_CURRENT; const char *stype = NULL; int type = VIR_STORAGE_TYPE_NONE; @@ -622,6 +630,7 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd) g_auto(virBuffer) sourceAttrBuf = VIR_BUFFER_INITIALIZER; g_auto(virBuffer) sourceChildBuf = VIR_BUFFER_INIT_CHILD(&diskChildBuf); g_auto(virBuffer) hostAttrBuf = VIR_BUFFER_INITIALIZER; + g_auto(virBuffer) throttleChildBuf = VIR_BUFFER_INITIALIZER; g_autofree char *xml = NULL; struct stat st; bool current = vshCommandOptBool(cmd, "current"); @@ -665,9 +674,14 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd) vshCommandOptStringReq(ctl, cmd, "source-protocol", &source_protocol) < 0 || vshCommandOptStringReq(ctl, cmd, "source-host-name", &host_name) < 0 || vshCommandOptStringReq(ctl, cmd, "source-host-transport", &host_transport) < 0 || + vshCommandOptStringReq(ctl, cmd, "throttle-groups", &throttle_groups) < 0 || vshCommandOptStringReq(ctl, cmd, "source-host-socket", &host_socket) < 0) return false; + if (throttle_groups) { + groups = g_strsplit(throttle_groups, ",", 0); + } + if (stype && (type = virshAttachDiskSourceTypeFromString(stype)) < 0) { vshError(ctl, _("Unknown source type: '%1$s'"), stype); @@ -714,6 +728,15 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd) virXMLFormatElement(&diskChildBuf, "driver", &driverAttrBuf, NULL); + if (groups) { + for (i = 0; i < (g_strv_length(groups)); i++) { + g_auto(virBuffer) throttleAttrBuf = VIR_BUFFER_INITIALIZER; + virBufferAsprintf(&throttleAttrBuf, " group='%s'", groups[i]); + virXMLFormatElement(&throttleChildBuf, "throttlefilter", &throttleAttrBuf, NULL); + } + virXMLFormatElement(&diskChildBuf, "throttlefilters", NULL, &throttleChildBuf); + } + switch ((enum virshAttachDiskSourceType) type) { case VIRSH_ATTACH_DISK_SOURCE_TYPE_FILE: virBufferEscapeString(&sourceAttrBuf, " file='%s'", source); @@ -1509,6 +1532,489 @@ cmdBlkdeviotune(vshControl *ctl, const vshCmd *cmd) goto cleanup; } +/* + * "throttlegrouplist" command + */ +static const vshCmdInfo info_throttlegrouplist = { + .help = N_("list all domain throttlegroups."), + .desc = N_("Get the summary of throttle groups for a domain."), +}; + +static const vshCmdOptDef opts_throttlegrouplist[] = { + VIRSH_COMMON_OPT_DOMAIN_FULL(0), + {.name = "inactive", + .type = VSH_OT_BOOL, + .help = N_("get inactive rather than running configuration") + }, + {.name = NULL} +}; + +static bool +cmdThrottleGroupList(vshControl *ctl, const vshCmd *cmd) +{ + unsigned int flags = 0; + size_t i; + g_autoptr(xmlDoc) xml = NULL; + g_autoptr(xmlXPathContext) ctxt = NULL; + g_autofree xmlNodePtr *groups = NULL; + ssize_t ngroups; + g_autoptr(vshTable) table = NULL; + + if (vshCommandOptBool(cmd, "inactive")) + flags |= VIR_DOMAIN_XML_INACTIVE; + + if (virshDomainGetXML(ctl, cmd, flags, &xml, &ctxt) < 0) + return false; + + ngroups = virXPathNodeSet("./throttlegroups/throttlegroup", ctxt, &groups); + if (ngroups < 0) + return false; + + table = vshTableNew(_("Name"), NULL); + + if (!table) + return false; + + for (i = 0; i < ngroups; i++) { + g_autofree char *name = NULL; + ctxt->node = groups[i]; + name = virXPathString("string(./group_name)", ctxt); + if (vshTableRowAppend(table, name, NULL) < 0) + return false; + } + + vshTablePrintToStdout(table, ctl); + + return true; +} + +/* + * "throttlegroupset" command + */ +static const vshCmdInfo info_throttlegroupset = { + .help = N_("Add or update a throttling group."), + .desc = N_("Add or updte a throttling group."), +}; + +static const vshCmdOptDef opts_throttlegroupset[] = { + VIRSH_COMMON_OPT_DOMAIN_FULL(0), + {.name = "group-name", + .type = VSH_OT_STRING, + .positional = true, + .required = true, + .completer = virshCompleteEmpty, + .help = N_("throttle group name") + }, + {.name = "total_bytes_sec", + .type = VSH_OT_ALIAS, + .help = "total-bytes-sec" + }, + {.name = "total-bytes-sec", + .type = VSH_OT_INT, + .help = N_("total throughput limit, as scaled integer (default bytes)") + }, + {.name = "read_bytes_sec", + .type = VSH_OT_ALIAS, + .help = "read-bytes-sec" + }, + {.name = "read-bytes-sec", + .type = VSH_OT_INT, + .help = N_("read throughput limit, as scaled integer (default bytes)") + }, + {.name = "write_bytes_sec", + .type = VSH_OT_ALIAS, + .help = "write-bytes-sec" + }, + {.name = "write-bytes-sec", + .type = VSH_OT_INT, + .help = N_("write throughput limit, as scaled integer (default bytes)") + }, + {.name = "total_iops_sec", + .type = VSH_OT_ALIAS, + .help = "total-iops-sec" + }, + {.name = "total-iops-sec", + .type = VSH_OT_INT, + .help = N_("total I/O operations limit per second") + }, + {.name = "read_iops_sec", + .type = VSH_OT_ALIAS, + .help = "read-iops-sec" + }, + {.name = "read-iops-sec", + .type = VSH_OT_INT, + .help = N_("read I/O operations limit per second") + }, + {.name = "write_iops_sec", + .type = VSH_OT_ALIAS, + .help = "write-iops-sec" + }, + {.name = "write-iops-sec", + .type = VSH_OT_INT, + .help = N_("write I/O operations limit per second") + }, + {.name = "total_bytes_sec_max", + .type = VSH_OT_ALIAS, + .help = "total-bytes-sec-max" + }, + {.name = "total-bytes-sec-max", + .type = VSH_OT_INT, + .help = N_("total max, as scaled integer (default bytes)") + }, + {.name = "read_bytes_sec_max", + .type = VSH_OT_ALIAS, + .help = "read-bytes-sec-max" + }, + {.name = "read-bytes-sec-max", + .type = VSH_OT_INT, + .help = N_("read max, as scaled integer (default bytes)") + }, + {.name = "write_bytes_sec_max", + .type = VSH_OT_ALIAS, + .help = "write-bytes-sec-max" + }, + {.name = "write-bytes-sec-max", + .type = VSH_OT_INT, + .help = N_("write max, as scaled integer (default bytes)") + }, + {.name = "total_iops_sec_max", + .type = VSH_OT_ALIAS, + .help = "total-iops-sec-max" + }, + {.name = "total-iops-sec-max", + .type = VSH_OT_INT, + .help = N_("total I/O operations max") + }, + {.name = "read_iops_sec_max", + .type = VSH_OT_ALIAS, + .help = "read-iops-sec-max" + }, + {.name = "read-iops-sec-max", + .type = VSH_OT_INT, + .help = N_("read I/O operations max") + }, + {.name = "write_iops_sec_max", + .type = VSH_OT_ALIAS, + .help = "write-iops-sec-max" + }, + {.name = "write-iops-sec-max", + .type = VSH_OT_INT, + .help = N_("write I/O operations max") + }, + {.name = "size_iops_sec", + .type = VSH_OT_ALIAS, + .help = "size-iops-sec" + }, + {.name = "size-iops-sec", + .type = VSH_OT_INT, + .help = N_("I/O size in bytes") + }, + {.name = "total_bytes_sec_max_length", + .type = VSH_OT_ALIAS, + .help = "total-bytes-sec-max-length" + }, + {.name = "total-bytes-sec-max-length", + .type = VSH_OT_INT, + .help = N_("duration in seconds to allow total max bytes") + }, + {.name = "read_bytes_sec_max_length", + .type = VSH_OT_ALIAS, + .help = "read-bytes-sec-max-length" + }, + {.name = "read-bytes-sec-max-length", + .type = VSH_OT_INT, + .help = N_("duration in seconds to allow read max bytes") + }, + {.name = "write_bytes_sec_max_length", + .type = VSH_OT_ALIAS, + .help = "write-bytes-sec-max-length" + }, + {.name = "write-bytes-sec-max-length", + .type = VSH_OT_INT, + .help = N_("duration in seconds to allow write max bytes") + }, + {.name = "total_iops_sec_max_length", + .type = VSH_OT_ALIAS, + .help = "total-iops-sec-max-length" + }, + {.name = "total-iops-sec-max-length", + .type = VSH_OT_INT, + .help = N_("duration in seconds to allow total I/O operations max") + }, + {.name = "read_iops_sec_max_length", + .type = VSH_OT_ALIAS, + .help = "read-iops-sec-max-length" + }, + {.name = "read-iops-sec-max-length", + .type = VSH_OT_INT, + .help = N_("duration in seconds to allow read I/O operations max") + }, + {.name = "write_iops_sec_max_length", + .type = VSH_OT_ALIAS, + .help = "write-iops-sec-max-length" + }, + {.name = "write-iops-sec-max-length", + .type = VSH_OT_INT, + .help = N_("duration in seconds to allow write I/O operations max") + }, + VIRSH_COMMON_OPT_DOMAIN_CONFIG, + VIRSH_COMMON_OPT_DOMAIN_LIVE, + VIRSH_COMMON_OPT_DOMAIN_CURRENT, + {.name = NULL} +}; + +static bool +cmdThrottleGroupSet(vshControl *ctl, const vshCmd *cmd) +{ + g_autoptr(virshDomain) dom = NULL; + const char *name; + const char *group_name = NULL; + unsigned long long value; + int nparams = 0; + int maxparams = 0; + virTypedParameterPtr params = NULL; + unsigned int flags = VIR_DOMAIN_AFFECT_CURRENT; + int rv = 0; + bool current = vshCommandOptBool(cmd, "current"); + bool config = vshCommandOptBool(cmd, "config"); + bool live = vshCommandOptBool(cmd, "live"); + bool ret = false; + + VSH_EXCLUSIVE_OPTIONS_VAR(current, live); + VSH_EXCLUSIVE_OPTIONS_VAR(current, config); + + if (config) + flags |= VIR_DOMAIN_AFFECT_CONFIG; + if (live) + flags |= VIR_DOMAIN_AFFECT_LIVE; + + if (!(dom = virshCommandOptDomain(ctl, cmd, &name))) + goto cleanup; + + +#define VSH_SET_THROTTLE_GROUP_SCALED(PARAM, CONST) \ + if ((rv = vshCommandOptScaledInt(ctl, cmd, #PARAM, &value, \ + 1, ULLONG_MAX)) < 0) { \ + goto interror; \ + } else if (rv > 0) { \ + if (virTypedParamsAddULLong(¶ms, &nparams, &maxparams, \ + VIR_DOMAIN_BLOCK_IOTUNE_##CONST, \ + value) < 0) \ + goto save_error; \ + } + + VSH_SET_THROTTLE_GROUP_SCALED(total-bytes-sec, TOTAL_BYTES_SEC); + VSH_SET_THROTTLE_GROUP_SCALED(read-bytes-sec, READ_BYTES_SEC); + VSH_SET_THROTTLE_GROUP_SCALED(write-bytes-sec, WRITE_BYTES_SEC); + VSH_SET_THROTTLE_GROUP_SCALED(total-bytes-sec-max, TOTAL_BYTES_SEC_MAX); + VSH_SET_THROTTLE_GROUP_SCALED(read-bytes-sec-max, READ_BYTES_SEC_MAX); + VSH_SET_THROTTLE_GROUP_SCALED(write-bytes-sec-max, WRITE_BYTES_SEC_MAX); +#undef VSH_SET_THROTTLE_GROUP_SCALED + +#define VSH_SET_THROTTLE_GROUP(PARAM, CONST) \ + if ((rv = vshCommandOptULongLong(ctl, cmd, #PARAM, &value)) < 0) { \ + goto interror; \ + } else if (rv > 0) { \ + if (virTypedParamsAddULLong(¶ms, &nparams, &maxparams, \ + VIR_DOMAIN_BLOCK_IOTUNE_##CONST, \ + value) < 0) \ + goto save_error; \ + } + + VSH_SET_THROTTLE_GROUP(total-iops-sec, TOTAL_IOPS_SEC); + VSH_SET_THROTTLE_GROUP(read-iops-sec, READ_IOPS_SEC); + VSH_SET_THROTTLE_GROUP(write-iops-sec, WRITE_IOPS_SEC); + VSH_SET_THROTTLE_GROUP(total-iops-sec-max, TOTAL_IOPS_SEC_MAX); + VSH_SET_THROTTLE_GROUP(read-iops-sec-max, READ_IOPS_SEC_MAX); + VSH_SET_THROTTLE_GROUP(write-iops-sec-max, WRITE_IOPS_SEC_MAX); + VSH_SET_THROTTLE_GROUP(size-iops-sec, SIZE_IOPS_SEC); + + VSH_SET_THROTTLE_GROUP(total-bytes-sec-max-length, TOTAL_BYTES_SEC_MAX_LENGTH); + VSH_SET_THROTTLE_GROUP(read-bytes-sec-max-length, READ_BYTES_SEC_MAX_LENGTH); + VSH_SET_THROTTLE_GROUP(write-bytes-sec-max-length, WRITE_BYTES_SEC_MAX_LENGTH); + VSH_SET_THROTTLE_GROUP(total-iops-sec-max-length, TOTAL_IOPS_SEC_MAX_LENGTH); + VSH_SET_THROTTLE_GROUP(read-iops-sec-max-length, READ_IOPS_SEC_MAX_LENGTH); + VSH_SET_THROTTLE_GROUP(write-iops-sec-max-length, WRITE_IOPS_SEC_MAX_LENGTH); +#undef VSH_SET_THROTTLE_GROUP + + if (vshCommandOptStringReq(ctl, cmd, "group-name", &group_name) < 0) { + vshError(ctl, "%s", _("Unable to parse group-name parameter")); + goto cleanup; + } + + if (group_name) { + if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_DOMAIN_BLOCK_IOTUNE_GROUP_NAME, + group_name) < 0) + goto save_error; + } + + if (virDomainSetThrottleGroup(dom, group_name, params, nparams, flags) < 0) + goto error; + vshPrintExtra(ctl, "%s", _("Throttle group set successfully\n")); + + ret = true; + + cleanup: + virTypedParamsFree(params, nparams); + return ret; + + save_error: + vshSaveLibvirtError(); + error: + vshError(ctl, "%s", _("Unable to change throttle group")); + goto cleanup; + + interror: + vshError(ctl, "%s", _("Unable to parse integer parameter")); + goto cleanup; +} + +/* + * "throttlegroupdel" command + */ +static const vshCmdInfo info_throttlegroupdel = { + .help = N_("Delete a throttling group."), + .desc = N_("Delete a throttling group."), +}; + +static const vshCmdOptDef opts_throttlegroupdel[] = { + VIRSH_COMMON_OPT_DOMAIN_FULL(0), + {.name = "group-name", + .type = VSH_OT_STRING, + .positional = true, + .required = true, + .completer = virshCompleteEmpty, + .help = N_("throttle group name") + }, + VIRSH_COMMON_OPT_DOMAIN_CONFIG, + VIRSH_COMMON_OPT_DOMAIN_LIVE, + VIRSH_COMMON_OPT_DOMAIN_CURRENT, + {.name = NULL} +}; + +static bool +cmdThrottleGroupDel(vshControl *ctl, const vshCmd *cmd) +{ + g_autoptr(virshDomain) dom = NULL; + const char *group_name = NULL; + bool config = vshCommandOptBool(cmd, "config"); + bool live = vshCommandOptBool(cmd, "live"); + bool current = vshCommandOptBool(cmd, "current"); + unsigned int flags = VIR_DOMAIN_AFFECT_CURRENT; + + VSH_EXCLUSIVE_OPTIONS_VAR(current, live); + VSH_EXCLUSIVE_OPTIONS_VAR(current, config); + + if (config) + flags |= VIR_DOMAIN_AFFECT_CONFIG; + if (live) + flags |= VIR_DOMAIN_AFFECT_LIVE; + + if (!(dom = virshCommandOptDomain(ctl, cmd, NULL))) + return false; + + if (vshCommandOptStringReq(ctl, cmd, "group-name", &group_name) < 0) { + vshError(ctl, "%s", _("Unable to parse group-name parameter")); + return false; + } + + if (virDomainDelThrottleGroup(dom, group_name, flags) < 0) + return false; + vshPrintExtra(ctl, "%s", _("Throttle group deleted successfully\n")); + + return true; +} + +/* + * "throttlegroupinfo" command + */ +static const vshCmdInfo info_throttlegroupinfo = { + .help = N_("Get a throttling group."), + .desc = N_("Get a throttling group."), +}; + +static const vshCmdOptDef opts_throttlegroupinfo[] = { + VIRSH_COMMON_OPT_DOMAIN_FULL(0), + {.name = "group-name", + .type = VSH_OT_STRING, + .positional = true, + .required = true, + .completer = virshCompleteEmpty, + .help = N_("throttle group name") + }, + VIRSH_COMMON_OPT_DOMAIN_CONFIG, + VIRSH_COMMON_OPT_DOMAIN_LIVE, + VIRSH_COMMON_OPT_DOMAIN_CURRENT, + {.name = NULL} +}; + +static bool +cmdThrottleGroupInfo(vshControl *ctl, const vshCmd *cmd) +{ + g_autoptr(virshDomain) dom = NULL; + const char *name; + const char *group_name = NULL; + int nparams = 0; + virTypedParameterPtr params = NULL; + unsigned int flags = VIR_DOMAIN_AFFECT_CURRENT; + size_t i; + bool current = vshCommandOptBool(cmd, "current"); + bool config = vshCommandOptBool(cmd, "config"); + bool live = vshCommandOptBool(cmd, "live"); + bool ret = false; + + VSH_EXCLUSIVE_OPTIONS_VAR(current, live); + VSH_EXCLUSIVE_OPTIONS_VAR(current, config); + + if (config) + flags |= VIR_DOMAIN_AFFECT_CONFIG; + if (live) + flags |= VIR_DOMAIN_AFFECT_LIVE; + + if (!(dom = virshCommandOptDomain(ctl, cmd, &name))) + goto cleanup; + + if (vshCommandOptStringReq(ctl, cmd, "group-name", &group_name) < 0) { + vshError(ctl, "%s", _("Unable to parse group-name parameter")); + goto cleanup; + } + + // get number of params + if (nparams == 0) { + if (virDomainGetThrottleGroup(dom, NULL, NULL, &nparams, flags) != 0) { + vshError(ctl, "%s", + _("Unable to get number of throttle group parameters")); + goto cleanup; + } + + if (nparams == 0) { + ret = true; + goto cleanup; + } + + params = g_new0(virTypedParameter, nparams); + } + + // request with nparams + if (virDomainGetThrottleGroup(dom, group_name, params, &nparams, flags) != 0) { + vshError(ctl, "%s", + _("Unable to get throttle group parameters")); + goto cleanup; + } + + for (i = 0; i < nparams; i++) { + g_autofree char *str = vshGetTypedParamValue(ctl, ¶ms[i]); + vshPrint(ctl, "%-15s: %s\n", params[i].field, str); + } + + ret = true; + + cleanup: + virTypedParamsFree(params, nparams); + return ret; +} + /* * "blkiotune" command */ @@ -13422,6 +13928,30 @@ const vshCmdDef domManagementCmds[] = { .opts = opts_blkdeviotune, .info = &info_blkdeviotune, .flags = 0 + }, + {.name = "throttlegroupset", + .handler = cmdThrottleGroupSet, + .opts = opts_throttlegroupset, + .info = &info_throttlegroupset, + .flags = 0 + }, + {.name = "throttlegroupdel", + .handler = cmdThrottleGroupDel, + .opts = opts_throttlegroupdel, + .info = &info_throttlegroupdel, + .flags = 0 + }, + {.name = "throttlegroupinfo", + .handler = cmdThrottleGroupInfo, + .opts = opts_throttlegroupinfo, + .info = &info_throttlegroupinfo, + .flags = 0 + }, + {.name = "throttlegrouplist", + .handler = cmdThrottleGroupList, + .opts = opts_throttlegrouplist, + .info = &info_throttlegrouplist, + .flags = 0 }, {.name = "blkiotune", .handler = cmdBlkiotune, -- 2.34.1

From: Yu Jie Gu <guyujie@linux.ibm.com> * Add remote_domain_set_throttle_group_args * Adjust sequence to be consistent with remote_procedure section Signed-off-by: Yu Jie Gu <guyujie@linux.ibm.com> --- src/remote_protocol-structs | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index 0319a5690d..5adbea0336 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -1050,12 +1050,14 @@ struct remote_domain_get_block_io_tune_ret { } params; int nparams; }; -struct remote_domain_get_throttle_group_ret { - struct { - u_int params_len; - remote_typed_param * params_val; - } params; - int nparams; +struct remote_domain_set_throttle_group_args { + remote_nonnull_domain dom; + remote_nonnull_string group; + struct { + u_int params_len; + remote_typed_param * params_val; + } params; + u_int flags; }; struct remote_domain_get_throttle_group_args { remote_nonnull_domain dom; @@ -1063,6 +1065,13 @@ struct remote_domain_get_throttle_group_args { int nparams; u_int flags; }; +struct remote_domain_get_throttle_group_ret { + struct { + u_int params_len; + remote_typed_param * params_val; + } params; + int nparams; +}; struct remote_domain_del_throttle_group_args { remote_nonnull_domain dom; remote_string group; -- 2.34.1

From: Yan Xiu Wu <wuyx@linux.ibm.com> No need to use both "iotune" and "throttlefilters" for specific disk Signed-off-by: Yan Xiu Wu <wuyx@linux.ibm.com> --- src/conf/domain_validate.c | 7 +++++++ src/qemu/qemu_driver.c | 5 +++++ 2 files changed, 12 insertions(+) diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index fffe274afc..940c3a92d6 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -955,6 +955,13 @@ virDomainDiskDefValidate(const virDomainDef *def, } } } + if (disk->throttlefilters && (disk->blkdeviotune.group_name || + virDomainBlockIoTuneInfoHasAny(&disk->blkdeviotune))) { + virReportError(VIR_ERR_XML_ERROR, + _("block 'throttlefilters' can't be used together with 'iotune' for disk '%1$s'"), + disk->dst); + return -1; + } return 0; } diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 097366cefb..661dee06cd 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -14827,6 +14827,11 @@ qemuDomainDiskBlockIoTuneIsSupported(virDomainDiskDef *disk) _("block I/O throttling is not supported for disk '%1$s'"), disk->dst); return false; } + if (disk->throttlefilters) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("block 'iotune' can't be used together with 'throttlefilters' for disk '%1$s'"), disk->dst); + return false; + } return true; } -- 2.34.1

From: Hao Ning Xin <xinhaong@linux.ibm.com> Both throttlegroup and iotune share the same fields, so they share the same verification logic Signed-off-by: Hao Ning Xin <xinhaong@linux.ibm.com> --- src/conf/domain_validate.c | 94 ++++++++++++++++++++++++-------------- 1 file changed, 60 insertions(+), 34 deletions(-) diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index 940c3a92d6..a2b615bf82 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -657,6 +657,47 @@ virDomainDiskDefValidateStartupPolicy(const virDomainDiskDef *disk) return 0; } +static int +virDomainDiskIoTuneValidate(const virDomainBlockIoTuneInfo blkdeviotune) +{ + if ((blkdeviotune.total_bytes_sec && + blkdeviotune.read_bytes_sec) || + (blkdeviotune.total_bytes_sec && + blkdeviotune.write_bytes_sec)) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("total and read/write bytes_sec cannot be set at the same time")); + return -1; + } + + if ((blkdeviotune.total_iops_sec && + blkdeviotune.read_iops_sec) || + (blkdeviotune.total_iops_sec && + blkdeviotune.write_iops_sec)) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("total and read/write iops_sec cannot be set at the same time")); + return -1; + } + + if ((blkdeviotune.total_bytes_sec_max && + blkdeviotune.read_bytes_sec_max) || + (blkdeviotune.total_bytes_sec_max && + blkdeviotune.write_bytes_sec_max)) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("total and read/write bytes_sec_max cannot be set at the same time")); + return -1; + } + + if ((blkdeviotune.total_iops_sec_max && + blkdeviotune.read_iops_sec_max) || + (blkdeviotune.total_iops_sec_max && + blkdeviotune.write_iops_sec_max)) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("total and read/write iops_sec_max cannot be set at the same time")); + return -1; + } + + return 0; +} static int virDomainDiskDefValidate(const virDomainDef *def, @@ -713,41 +754,8 @@ virDomainDiskDefValidate(const virDomainDef *def, } /* Validate IotuneParse */ - if ((disk->blkdeviotune.total_bytes_sec && - disk->blkdeviotune.read_bytes_sec) || - (disk->blkdeviotune.total_bytes_sec && - disk->blkdeviotune.write_bytes_sec)) { - virReportError(VIR_ERR_XML_ERROR, "%s", - _("total and read/write bytes_sec cannot be set at the same time")); - return -1; - } - - if ((disk->blkdeviotune.total_iops_sec && - disk->blkdeviotune.read_iops_sec) || - (disk->blkdeviotune.total_iops_sec && - disk->blkdeviotune.write_iops_sec)) { - virReportError(VIR_ERR_XML_ERROR, "%s", - _("total and read/write iops_sec cannot be set at the same time")); - return -1; - } - - if ((disk->blkdeviotune.total_bytes_sec_max && - disk->blkdeviotune.read_bytes_sec_max) || - (disk->blkdeviotune.total_bytes_sec_max && - disk->blkdeviotune.write_bytes_sec_max)) { - virReportError(VIR_ERR_XML_ERROR, "%s", - _("total and read/write bytes_sec_max cannot be set at the same time")); + if (virDomainDiskIoTuneValidate(disk->blkdeviotune) < 0) return -1; - } - - if ((disk->blkdeviotune.total_iops_sec_max && - disk->blkdeviotune.read_iops_sec_max) || - (disk->blkdeviotune.total_iops_sec_max && - disk->blkdeviotune.write_iops_sec_max)) { - virReportError(VIR_ERR_XML_ERROR, "%s", - _("total and read/write iops_sec_max cannot be set at the same time")); - return -1; - } /* Reject disks with a bus type that is not compatible with the * given address type. The function considers only buses that are @@ -1820,6 +1828,21 @@ virDomainDefValidateIOThreads(const virDomainDef *def) return 0; } +static int +virDomainDefValidateThrottleGroups(const virDomainDef *def) +{ + size_t i; + + for (i = 0; i < def->nthrottlegroups; i++) { + virDomainThrottleGroupDef *throttleGroup = def->throttlegroups[i]; + + /* Validate Throttle Group */ + if (virDomainDiskIoTuneValidate(*throttleGroup) < 0) + return -1; + } + + return 0; +} static int virDomainDefValidateInternal(const virDomainDef *def, @@ -1876,6 +1899,9 @@ virDomainDefValidateInternal(const virDomainDef *def, if (virDomainDefValidateIOThreads(def) < 0) return -1; + if (virDomainDefValidateThrottleGroups(def) < 0) + return -1; + return 0; } -- 2.34.1

On Tue, Apr 09, 2024 at 20:13:08 -0700, wucf@linux.ibm.com wrote:
From: Chun Feng Wu <wucf@linux.ibm.com>
Hi,
I am thinking to leverage "throttle block filter" in QEMU to support more flexible I/O limits(e.g. tiered I/O groups), one sample provided by QEMU doc is: https://github.com/qemu/qemu/blob/master/docs/throttle.txt "For example, let's say that we have three different drives and we want to set I/O limits for each one of them and an additional set of limits for the combined I/O of all three drives."
The implementation idea is to - Define throttle groups(limit) in domain - Define throttle filter to reference throttle group within disk - Within domain disk, throttle filters references multiple throttle groups to form filter chain to apply multiple limits in QEMU like above sample - Add new virsh cmds for throttle group management: throttlegroupset Add or update a throttling group. throttlegroupdel Delete a throttling group. throttlegroupinfo Get a throttling group. throttlegrouplist list all domain throttlegroups - Update "attach-disk" to add one more option "--throttle-groups" to apply throttle filters e.g. "virsh attach-disk $VM_ID ${DISK_PATH}/vm1_disk_2.qcow2 vdd --driver qemu --subdriver qcow2 --targetbus virtio --throttle-groups limit2,limit012" - I chose above semantics as I felt they're appropriate, if there are better ones please kindly suggest.
This patchset includes: - Throttle group and throttle filter definition in patch 1 - New QMP processing to update and get throttle group in patch 2 - New API definition and implementation in patch 3 - Hotplug and qemuProcessLaunch flow implemenation in patch 4, 5 - Domain XML schema and doc(formatdomain.rst) change in patch 6 - Tests in patch 7, 8 - Virsh cmd implementation in patch 9 - Other enhencement/verification implementation in patch 10, 11, 12
[...]
Any comments/suggestions will be appriciated!
Hi, I'll be doing a proper review later on, but a couple of observations for now: - The series fails 'check-remote_protocol' test after patch 3 but it's fixed later on. Note that per contribution guidelines the series must compile cleanly after every single patch. - The coding style is inconsistent even inside one patch in terms of function declaration. Please use the: returntype functionname(type1 arg1, type2 arg2) style and use two newlines between functions. - patch 3 implements both the remote driver stuff and directly qemu impl, we try to keep those separated - don't use line comments ( // comment ) - use g_autofree and g_autoptr instead of adding 'cleanup:' sections. We are trying to refactor the code to get away from the design pattern - avoid empty 'cleanup:' labels, return value directly (e.g. patch 8) - try to use virJSONValueAdd instead of virJSONValueLimitsAppendPositiveNumberLong, it can do the same without the extra helper by using the P modifier. Also make sure to check the return value of the JSON functions (patch 2). - the virsh implementation adds a lot of VSH_OT_ALIAS arguments with the spelling we tried to fix in the command you've copied it from. Do not add the old stuff again. - preserve spacing between blocks of code which are not related (e.g. patch 11) Rest of the review after I'll have a deeper look.

Thanks Peter for your quick response! I will fix the issues you listed On 2024/4/10 16:08, Peter Krempa wrote:
On Tue, Apr 09, 2024 at 20:13:08 -0700, wucf@linux.ibm.com wrote:
From: Chun Feng Wu <wucf@linux.ibm.com>
Hi,
I am thinking to leverage "throttle block filter" in QEMU to support more flexible I/O limits(e.g. tiered I/O groups), one sample provided by QEMU doc is: https://github.com/qemu/qemu/blob/master/docs/throttle.txt "For example, let's say that we have three different drives and we want to set I/O limits for each one of them and an additional set of limits for the combined I/O of all three drives."
The implementation idea is to - Define throttle groups(limit) in domain - Define throttle filter to reference throttle group within disk - Within domain disk, throttle filters references multiple throttle groups to form filter chain to apply multiple limits in QEMU like above sample - Add new virsh cmds for throttle group management: throttlegroupset Add or update a throttling group. throttlegroupdel Delete a throttling group. throttlegroupinfo Get a throttling group. throttlegrouplist list all domain throttlegroups - Update "attach-disk" to add one more option "--throttle-groups" to apply throttle filters e.g. "virsh attach-disk $VM_ID ${DISK_PATH}/vm1_disk_2.qcow2 vdd --driver qemu --subdriver qcow2 --targetbus virtio --throttle-groups limit2,limit012" - I chose above semantics as I felt they're appropriate, if there are better ones please kindly suggest.
This patchset includes: - Throttle group and throttle filter definition in patch 1 - New QMP processing to update and get throttle group in patch 2 - New API definition and implementation in patch 3 - Hotplug and qemuProcessLaunch flow implemenation in patch 4, 5 - Domain XML schema and doc(formatdomain.rst) change in patch 6 - Tests in patch 7, 8 - Virsh cmd implementation in patch 9 - Other enhencement/verification implementation in patch 10, 11, 12 [...]
Any comments/suggestions will be appriciated! Hi, I'll be doing a proper review later on, but a couple of observations for now:
- The series fails 'check-remote_protocol' test after patch 3 but it's fixed later on. Note that per contribution guidelines the series must compile cleanly after every single patch.
- The coding style is inconsistent even inside one patch in terms of function declaration. Please use the:
returntype functionname(type1 arg1, type2 arg2)
style and use two newlines between functions.
- patch 3 implements both the remote driver stuff and directly qemu impl, we try to keep those separated
- don't use line comments ( // comment )
- use g_autofree and g_autoptr instead of adding 'cleanup:' sections. We are trying to refactor the code to get away from the design pattern
- avoid empty 'cleanup:' labels, return value directly (e.g. patch 8)
- try to use virJSONValueAdd instead of virJSONValueLimitsAppendPositiveNumberLong, it can do the same without the extra helper by using the P modifier. Also make sure to check the return value of the JSON functions (patch 2).
- the virsh implementation adds a lot of VSH_OT_ALIAS arguments with the spelling we tried to fix in the command you've copied it from. Do not add the old stuff again.
- preserve spacing between blocks of code which are not related (e.g. patch 11)
Rest of the review after I'll have a deeper look.
-- Thanks and Regards, Wu

I addressed those comments in v2 https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/5Z3J4... On 2024/4/10 16:08, Peter Krempa wrote:
On Tue, Apr 09, 2024 at 20:13:08 -0700, wucf@linux.ibm.com wrote:
From: Chun Feng Wu <wucf@linux.ibm.com>
Hi,
I am thinking to leverage "throttle block filter" in QEMU to support more flexible I/O limits(e.g. tiered I/O groups), one sample provided by QEMU doc is: https://github.com/qemu/qemu/blob/master/docs/throttle.txt "For example, let's say that we have three different drives and we want to set I/O limits for each one of them and an additional set of limits for the combined I/O of all three drives."
The implementation idea is to - Define throttle groups(limit) in domain - Define throttle filter to reference throttle group within disk - Within domain disk, throttle filters references multiple throttle groups to form filter chain to apply multiple limits in QEMU like above sample - Add new virsh cmds for throttle group management: throttlegroupset Add or update a throttling group. throttlegroupdel Delete a throttling group. throttlegroupinfo Get a throttling group. throttlegrouplist list all domain throttlegroups - Update "attach-disk" to add one more option "--throttle-groups" to apply throttle filters e.g. "virsh attach-disk $VM_ID ${DISK_PATH}/vm1_disk_2.qcow2 vdd --driver qemu --subdriver qcow2 --targetbus virtio --throttle-groups limit2,limit012" - I chose above semantics as I felt they're appropriate, if there are better ones please kindly suggest.
This patchset includes: - Throttle group and throttle filter definition in patch 1 - New QMP processing to update and get throttle group in patch 2 - New API definition and implementation in patch 3 - Hotplug and qemuProcessLaunch flow implemenation in patch 4, 5 - Domain XML schema and doc(formatdomain.rst) change in patch 6 - Tests in patch 7, 8 - Virsh cmd implementation in patch 9 - Other enhencement/verification implementation in patch 10, 11, 12 [...]
Any comments/suggestions will be appriciated! Hi, I'll be doing a proper review later on, but a couple of observations for now:
- The series fails 'check-remote_protocol' test after patch 3 but it's fixed later on. Note that per contribution guidelines the series must compile cleanly after every single patch.
- The coding style is inconsistent even inside one patch in terms of function declaration. Please use the:
returntype functionname(type1 arg1, type2 arg2)
style and use two newlines between functions.
- patch 3 implements both the remote driver stuff and directly qemu impl, we try to keep those separated
- don't use line comments ( // comment )
- use g_autofree and g_autoptr instead of adding 'cleanup:' sections. We are trying to refactor the code to get away from the design pattern
- avoid empty 'cleanup:' labels, return value directly (e.g. patch 8)
- try to use virJSONValueAdd instead of virJSONValueLimitsAppendPositiveNumberLong, it can do the same without the extra helper by using the P modifier. Also make sure to check the return value of the JSON functions (patch 2).
- the virsh implementation adds a lot of VSH_OT_ALIAS arguments with the spelling we tried to fix in the command you've copied it from. Do not add the old stuff again.
- preserve spacing between blocks of code which are not related (e.g. patch 11)
Rest of the review after I'll have a deeper look.
-- Thanks and Regards, Wu
participants (3)
-
Chun Feng Wu
-
Peter Krempa
-
wucf@linux.ibm.com