[libvirt] [PATCH v2 0/8] Add throttle blkio cgroup support for libvirt
Right now, libvirt only supports the cfq based blkio cgorup, this means if the block devices doesn't use cfq scheduler, the blkio cgroup will loss effect. This patchset adds the throttle blkio cgroup support for libvirt, introduces four elements for domain configuration and extend the virsh command blkiotune. This patchset is a new version of Guan Qiang's patchset ://www.redhat.com/archives/libvir-list/2013-October/msg01066.html Change from v1: 1, rearrange the order of patches 2, change the options/elements of throttle blkio cgroup to consist with disk iotune. 3, fix complie error when cgroup is unavailable. 4, remove virCgroupSetBlkioDevice, split virCgroupSetBlkioDeviceBps and virCgroupSetBlkioDeviceIops Change from Guan Qiang's patchset: 1, split to 8 patches, make logic more clear 2, change the type of read/write iops form unsigned long long to unsigned int, trying to set read/write iops to the value which bigger than max number of unsigned int will fail. 3, fix some logic shortage. Gao feng (8): rename virDomainBlkioDeviceWeightParseXML to virDomainBlkioDeviceParseXML rename virBlkioDeviceWeightArrayClear to virBlkioDeviceArrayClear rename virBlkioDeviceWeightPtr to virBlkioDevicePtr domain: introduce xml elements for throttle blkio cgroup blkio: Setting throttle blkio cgroup for domain virsh: add setting throttle blkio cgroup option to blkiotune qemu: allow to setup throttle blkio cgroup through virsh lxc: allow to setup throttle blkio cgroup through virsh docs/schemas/domaincommon.rng | 28 +- include/libvirt/libvirt.h.in | 45 ++ src/conf/domain_conf.c | 113 +++- src/conf/domain_conf.h | 16 +- src/libvirt_private.syms | 6 +- src/lxc/lxc_cgroup.c | 29 +- src/lxc/lxc_driver.c | 649 ++++++++++++++++++++- src/qemu/qemu_cgroup.c | 29 +- src/qemu/qemu_driver.c | 443 ++++++++++++-- src/util/vircgroup.c | 224 ++++++- src/util/vircgroup.h | 16 + .../qemuxml2argv-blkiotune-device.xml | 8 + tools/virsh-domain.c | 64 ++ tools/virsh.pod | 36 +- 14 files changed, 1583 insertions(+), 123 deletions(-) -- 1.8.3.1
virDomainBlkioDeviceWeightParseXML will be used to parse the xml element read_bps, write_bps, read_iops, write_iops. Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/conf/domain_conf.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index e0ab4b1..e2219f2 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -892,7 +892,7 @@ virBlkioDeviceWeightArrayClear(virBlkioDeviceWeightPtr deviceWeights, } /** - * virDomainBlkioDeviceWeightParseXML + * virDomainBlkioDeviceParseXML * * this function parses a XML node: * @@ -904,8 +904,8 @@ virBlkioDeviceWeightArrayClear(virBlkioDeviceWeightPtr deviceWeights, * and fills a virBlkioDeviceWeight struct. */ static int -virDomainBlkioDeviceWeightParseXML(xmlNodePtr root, - virBlkioDeviceWeightPtr dw) +virDomainBlkioDeviceParseXML(xmlNodePtr root, + virBlkioDeviceWeightPtr dw) { char *c; xmlNodePtr node; @@ -11097,8 +11097,8 @@ virDomainDefParseXML(xmlDocPtr xml, for (i = 0; i < n; i++) { size_t j; - if (virDomainBlkioDeviceWeightParseXML(nodes[i], - &def->blkio.devices[i]) < 0) + if (virDomainBlkioDeviceParseXML(nodes[i], + &def->blkio.devices[i]) < 0) goto error; def->blkio.ndevices++; for (j = 0; j < i; j++) { -- 1.8.3.1
On 12/11/2013 01:29 AM, Gao feng wrote:
virDomainBlkioDeviceWeightParseXML will be used to parse the xml element read_bps, write_bps, read_iops, write_iops.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/conf/domain_conf.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
ACK. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 01/20/2014 09:44 AM, Eric Blake wrote:
On 12/11/2013 01:29 AM, Gao feng wrote:
virDomainBlkioDeviceWeightParseXML will be used to parse the xml element read_bps, write_bps, read_iops, write_iops.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/conf/domain_conf.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
ACK.
Oh, I never saw this get reviewed, but I see it has already been in libvirt.git for more than a month. Sorry for the noise. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 01/21/2014 12:50 AM, Eric Blake wrote:
On 01/20/2014 09:44 AM, Eric Blake wrote:
On 12/11/2013 01:29 AM, Gao feng wrote:
virDomainBlkioDeviceWeightParseXML will be used to parse the xml element read_bps, write_bps, read_iops, write_iops.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/conf/domain_conf.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
ACK.
Oh, I never saw this get reviewed, but I see it has already been in libvirt.git for more than a month. Sorry for the noise.
:) thanks for your ack I just pushed this patchset.
Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/conf/domain_conf.c | 8 ++++---- src/conf/domain_conf.h | 4 ++-- src/libvirt_private.syms | 2 +- src/qemu/qemu_driver.c | 6 +++--- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index e2219f2..397671a 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -882,8 +882,8 @@ virDomainXMLOptionGetNamespace(virDomainXMLOptionPtr xmlopt) void -virBlkioDeviceWeightArrayClear(virBlkioDeviceWeightPtr deviceWeights, - int ndevices) +virBlkioDeviceArrayClear(virBlkioDeviceWeightPtr deviceWeights, + int ndevices) { size_t i; @@ -2050,8 +2050,8 @@ void virDomainDefFree(virDomainDefPtr def) VIR_FREE(def->description); VIR_FREE(def->title); - virBlkioDeviceWeightArrayClear(def->blkio.devices, - def->blkio.ndevices); + virBlkioDeviceArrayClear(def->blkio.devices, + def->blkio.ndevices); VIR_FREE(def->blkio.devices); virDomainWatchdogDefFree(def->watchdog); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 4934911..96598b5 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1911,8 +1911,8 @@ struct _virDomainIdMapDef { }; -void virBlkioDeviceWeightArrayClear(virBlkioDeviceWeightPtr deviceWeights, - int ndevices); +void virBlkioDeviceArrayClear(virBlkioDeviceWeightPtr deviceWeights, + int ndevices); typedef struct _virDomainResourceDef virDomainResourceDef; typedef virDomainResourceDef *virDomainResourceDefPtr; diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 3ca20e4..d8f8f7e 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -104,7 +104,7 @@ virDomainAuditVcpu; # conf/domain_conf.h -virBlkioDeviceWeightArrayClear; +virBlkioDeviceArrayClear; virDiskNameToBusDeviceIndex; virDiskNameToIndex; virDomainActualNetDefFree; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 1c90798..a7d96a9 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -7505,7 +7505,7 @@ error: virReportError(VIR_ERR_INVALID_ARG, _("unable to parse device weight '%s'"), deviceWeightStr); cleanup: - virBlkioDeviceWeightArrayClear(result, ndevices); + virBlkioDeviceArrayClear(result, ndevices); VIR_FREE(result); return -1; } @@ -7636,7 +7636,7 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, &vm->def->blkio.ndevices, devices, ndevices) < 0) ret = -1; - virBlkioDeviceWeightArrayClear(devices, ndevices); + virBlkioDeviceArrayClear(devices, ndevices); VIR_FREE(devices); } } @@ -7673,7 +7673,7 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, &persistentDef->blkio.ndevices, devices, ndevices) < 0) ret = -1; - virBlkioDeviceWeightArrayClear(devices, ndevices); + virBlkioDeviceArrayClear(devices, ndevices); VIR_FREE(devices); } } -- 1.8.3.1
The throttle blkio cgroup will reuse this struct. Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/conf/domain_conf.c | 18 +++++++++--------- src/conf/domain_conf.h | 10 +++++----- src/lxc/lxc_cgroup.c | 6 +++--- src/qemu/qemu_cgroup.c | 8 ++++---- src/qemu/qemu_driver.c | 18 +++++++++--------- 5 files changed, 30 insertions(+), 30 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 397671a..26242b6 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -882,13 +882,13 @@ virDomainXMLOptionGetNamespace(virDomainXMLOptionPtr xmlopt) void -virBlkioDeviceArrayClear(virBlkioDeviceWeightPtr deviceWeights, +virBlkioDeviceArrayClear(virBlkioDevicePtr devices, int ndevices) { size_t i; for (i = 0; i < ndevices; i++) - VIR_FREE(deviceWeights[i].path); + VIR_FREE(devices[i].path); } /** @@ -901,11 +901,11 @@ virBlkioDeviceArrayClear(virBlkioDeviceWeightPtr deviceWeights, * <weight>weight</weight> * </device> * - * and fills a virBlkioDeviceWeight struct. + * and fills a virBlkioDeviceTune struct. */ static int virDomainBlkioDeviceParseXML(xmlNodePtr root, - virBlkioDeviceWeightPtr dw) + virBlkioDevicePtr dev) { char *c; xmlNodePtr node; @@ -913,16 +913,16 @@ virDomainBlkioDeviceParseXML(xmlNodePtr root, node = root->children; while (node) { if (node->type == XML_ELEMENT_NODE) { - if (xmlStrEqual(node->name, BAD_CAST "path") && !dw->path) { - dw->path = (char *)xmlNodeGetContent(node); + if (xmlStrEqual(node->name, BAD_CAST "path") && !dev->path) { + dev->path = (char *)xmlNodeGetContent(node); } else if (xmlStrEqual(node->name, BAD_CAST "weight")) { c = (char *)xmlNodeGetContent(node); - if (virStrToLong_ui(c, NULL, 10, &dw->weight) < 0) { + if (virStrToLong_ui(c, NULL, 10, &dev->weight) < 0) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("could not parse weight %s"), c); VIR_FREE(c); - VIR_FREE(dw->path); + VIR_FREE(dev->path); return -1; } VIR_FREE(c); @@ -930,7 +930,7 @@ virDomainBlkioDeviceParseXML(xmlNodePtr root, } node = node->next; } - if (!dw->path) { + if (!dev->path) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("missing per-device path")); return -1; diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 96598b5..b410fd0 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1860,9 +1860,9 @@ virDomainVcpuPinDefPtr virDomainVcpuPinFindByVcpu(virDomainVcpuPinDefPtr *def, int nvcpupin, int vcpu); -typedef struct _virBlkioDeviceWeight virBlkioDeviceWeight; -typedef virBlkioDeviceWeight *virBlkioDeviceWeightPtr; -struct _virBlkioDeviceWeight { +typedef struct _virBlkioDevice virBlkioDevice; +typedef virBlkioDevice *virBlkioDevicePtr; +struct _virBlkioDevice { char *path; unsigned int weight; }; @@ -1911,7 +1911,7 @@ struct _virDomainIdMapDef { }; -void virBlkioDeviceArrayClear(virBlkioDeviceWeightPtr deviceWeights, +void virBlkioDeviceArrayClear(virBlkioDevicePtr deviceWeights, int ndevices); typedef struct _virDomainResourceDef virDomainResourceDef; @@ -1940,7 +1940,7 @@ struct _virDomainDef { unsigned int weight; size_t ndevices; - virBlkioDeviceWeightPtr devices; + virBlkioDevicePtr devices; } blkio; struct { diff --git a/src/lxc/lxc_cgroup.c b/src/lxc/lxc_cgroup.c index 275e250..310a476 100644 --- a/src/lxc/lxc_cgroup.c +++ b/src/lxc/lxc_cgroup.c @@ -112,10 +112,10 @@ static int virLXCCgroupSetupBlkioTune(virDomainDefPtr def, if (def->blkio.ndevices) { for (i = 0; i < def->blkio.ndevices; i++) { - virBlkioDeviceWeightPtr dw = &def->blkio.devices[i]; - if (!dw->weight) + virBlkioDevicePtr dev = &def->blkio.devices[i]; + if (!dev->weight) continue; - if (virCgroupSetBlkioDeviceWeight(cgroup, dw->path, dw->weight) < 0) + if (virCgroupSetBlkioDeviceWeight(cgroup, dev->path, dev->weight) < 0) return -1; } } diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index f0cacd0..a18955e 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -399,11 +399,11 @@ qemuSetupBlkioCgroup(virDomainObjPtr vm) if (vm->def->blkio.ndevices) { for (i = 0; i < vm->def->blkio.ndevices; i++) { - virBlkioDeviceWeightPtr dw = &vm->def->blkio.devices[i]; - if (!dw->weight) + virBlkioDevicePtr dev = &vm->def->blkio.devices[i]; + if (!dev->weight) continue; - if (virCgroupSetBlkioDeviceWeight(priv->cgroup, dw->path, - dw->weight) < 0) + if (virCgroupSetBlkioDeviceWeight(priv->cgroup, dev->path, + dev->weight) < 0) return -1; } } diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index a7d96a9..45d11cd 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -7432,15 +7432,15 @@ cleanup: */ static int qemuDomainParseDeviceWeightStr(char *deviceWeightStr, - virBlkioDeviceWeightPtr *dw, size_t *size) + virBlkioDevicePtr *dev, size_t *size) { char *temp; int ndevices = 0; int nsep = 0; size_t i; - virBlkioDeviceWeightPtr result = NULL; + virBlkioDevicePtr result = NULL; - *dw = NULL; + *dev = NULL; *size = 0; if (STREQ(deviceWeightStr, "")) @@ -7496,7 +7496,7 @@ qemuDomainParseDeviceWeightStr(char *deviceWeightStr, if (!i) VIR_FREE(result); - *dw = result; + *dev = result; *size = i; return 0; @@ -7513,13 +7513,13 @@ cleanup: /* Modify dest_array to reflect all device weight changes described in * src_array. */ static int -qemuDomainMergeDeviceWeights(virBlkioDeviceWeightPtr *dest_array, +qemuDomainMergeDeviceWeights(virBlkioDevicePtr *dest_array, size_t *dest_size, - virBlkioDeviceWeightPtr src_array, + virBlkioDevicePtr src_array, size_t src_size) { size_t i, j; - virBlkioDeviceWeightPtr dest, src; + virBlkioDevicePtr dest, src; for (i = 0; i < src_size; i++) { bool found = false; @@ -7614,7 +7614,7 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, ret = -1; } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { size_t ndevices; - virBlkioDeviceWeightPtr devices = NULL; + virBlkioDevicePtr devices = NULL; size_t j; if (qemuDomainParseDeviceWeightStr(params[i].value.s, @@ -7660,7 +7660,7 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, persistentDef->blkio.weight = params[i].value.ui; } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { - virBlkioDeviceWeightPtr devices = NULL; + virBlkioDevicePtr devices = NULL; size_t ndevices; if (qemuDomainParseDeviceWeightStr(params[i].value.s, -- 1.8.3.1
This patch introduces new xml elements under <blkiotune>, we use these new elements to setup the throttle blkio cgroup for domain. The new blkiotune node looks like this: <blkiotune> <device> <path>/path/to/block</path> <weight>1000</weight> <read_iops_sec>10000</read_iops_sec> <write_iops_sec>10000</write_iops_sec> <read_bytes_sec>1000000</read_bytes_sec> <write_bytes_sec>1000000</write_bytes_sec> </device> </blkiotune> Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- docs/schemas/domaincommon.rng | 28 ++++++++++++-- src/conf/domain_conf.c | 85 +++++++++++++++++++++++++++++++++++++------ src/conf/domain_conf.h | 4 ++ 3 files changed, 103 insertions(+), 14 deletions(-) diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 38c6801..bc8ed5d 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -621,9 +621,31 @@ <element name="path"> <ref name="absFilePath"/> </element> - <element name="weight"> - <ref name="weight"/> - </element> + <optional> + <element name="weight"> + <ref name="weight"/> + </element> + </optional> + <optional> + <element name="read_iops_sec"> + <data type='unsignedInt'/> + </element> + </optional> + <optional> + <element name="write_iops_sec"> + <data type='unsignedInt'/> + </element> + </optional> + <optional> + <element name="read_bytes_sec"> + <data type='unsignedLong'/> + </element> + </optional> + <optional> + <element name="write_bytes_sec"> + <data type='unsignedLong'/> + </element> + </optional> </interleave> </element> </zeroOrMore> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 26242b6..c4d51b4 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -899,15 +899,19 @@ virBlkioDeviceArrayClear(virBlkioDevicePtr devices, * <device> * <path>/fully/qualified/device/path</path> * <weight>weight</weight> + * <read_bytes_sec>bps</read_bytes_sec> + * <write_bytes_sec>bps</write_bytes_sec> + * <read_iops_sec>iops</read_iops_sec> + * <write_iops_sec>iops</write_iops_sec> * </device> * - * and fills a virBlkioDeviceTune struct. + * and fills a virBlkioDevicePtr struct. */ static int virDomainBlkioDeviceParseXML(xmlNodePtr root, virBlkioDevicePtr dev) { - char *c; + char *c = NULL; xmlNodePtr node; node = root->children; @@ -921,9 +925,43 @@ virDomainBlkioDeviceParseXML(xmlNodePtr root, virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("could not parse weight %s"), c); - VIR_FREE(c); - VIR_FREE(dev->path); - return -1; + goto error; + } + VIR_FREE(c); + } else if (xmlStrEqual(node->name, BAD_CAST "read_bytes_sec")) { + c = (char *)xmlNodeGetContent(node); + if (virStrToLong_ull(c, NULL, 10, &dev->rbps) < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("could not parse read bytes sec %s"), + c); + goto error; + } + VIR_FREE(c); + } else if (xmlStrEqual(node->name, BAD_CAST "write_bytes_sec")) { + c = (char *)xmlNodeGetContent(node); + if (virStrToLong_ull(c, NULL, 10, &dev->wbps) < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("could not parse write bytes sec %s"), + c); + goto error; + } + VIR_FREE(c); + } else if (xmlStrEqual(node->name, BAD_CAST "read_iops_sec")) { + c = (char *)xmlNodeGetContent(node); + if (virStrToLong_ui(c, NULL, 10, &dev->riops) < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("could not parse read iops sec %s"), + c); + goto error; + } + VIR_FREE(c); + } else if (xmlStrEqual(node->name, BAD_CAST "write_iops_sec")) { + c = (char *)xmlNodeGetContent(node); + if (virStrToLong_ui(c, NULL, 10, &dev->wiops) < 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("could not parse write iops sec %s"), + c); + goto error; } VIR_FREE(c); } @@ -937,6 +975,11 @@ virDomainBlkioDeviceParseXML(xmlNodePtr root, } return 0; + +error: + VIR_FREE(c); + VIR_FREE(dev->path); + return -1; } @@ -11105,7 +11148,7 @@ virDomainDefParseXML(xmlDocPtr xml, if (STREQ(def->blkio.devices[j].path, def->blkio.devices[i].path)) { virReportError(VIR_ERR_XML_ERROR, - _("duplicate device weight path '%s'"), + _("duplicate blkio device path '%s'"), def->blkio.devices[i].path); goto error; } @@ -16614,7 +16657,11 @@ virDomainDefFormatInternal(virDomainDefPtr def, blkio = true; } else { for (n = 0; n < def->blkio.ndevices; n++) { - if (def->blkio.devices[n].weight) { + if (def->blkio.devices[n].weight || + def->blkio.devices[n].riops || + def->blkio.devices[n].wiops || + def->blkio.devices[n].rbps || + def->blkio.devices[n].wbps) { blkio = true; break; } @@ -16629,13 +16676,29 @@ virDomainDefFormatInternal(virDomainDefPtr def, def->blkio.weight); for (n = 0; n < def->blkio.ndevices; n++) { - if (def->blkio.devices[n].weight == 0) + virBlkioDevicePtr dev = &def->blkio.devices[n]; + + if (!dev->weight && !dev->riops && !dev->wiops && + !dev->rbps && !dev->wbps) continue; virBufferAddLit(buf, " <device>\n"); virBufferEscapeString(buf, " <path>%s</path>\n", - def->blkio.devices[n].path); - virBufferAsprintf(buf, " <weight>%u</weight>\n", - def->blkio.devices[n].weight); + dev->path); + if (dev->weight) + virBufferAsprintf(buf, " <weight>%u</weight>\n", + dev->weight); + if (dev->riops) + virBufferAsprintf(buf, " <read_iops_sec>%u</read_iops_sec>\n", + dev->riops); + if (dev->wiops) + virBufferAsprintf(buf, " <write_iops_sec>%u</write_iops_sec>\n", + dev->wiops); + if (dev->rbps) + virBufferAsprintf(buf, " <read_bytes_sec>%llu</read_bytes_sec>\n", + dev->rbps); + if (dev->wbps) + virBufferAsprintf(buf, " <write_bytes_sec>%llu</write_bytes_sec>\n", + dev->wbps); virBufferAddLit(buf, " </device>\n"); } diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index b410fd0..5631ccd 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1865,6 +1865,10 @@ typedef virBlkioDevice *virBlkioDevicePtr; struct _virBlkioDevice { char *path; unsigned int weight; + unsigned int riops; + unsigned int wiops; + unsigned long long rbps; + unsigned long long wbps; }; enum virDomainRNGModel { -- 1.8.3.1
On Wed, Dec 11, 2013 at 04:29:49PM +0800, Gao feng wrote:
This patch introduces new xml elements under <blkiotune>, we use these new elements to setup the throttle blkio cgroup for domain. The new blkiotune node looks like this:
<blkiotune> <device> <path>/path/to/block</path> <weight>1000</weight> <read_iops_sec>10000</read_iops_sec> <write_iops_sec>10000</write_iops_sec> <read_bytes_sec>1000000</read_bytes_sec> <write_bytes_sec>1000000</write_bytes_sec> </device> </blkiotune>
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- docs/schemas/domaincommon.rng | 28 ++++++++++++-- src/conf/domain_conf.c | 85 +++++++++++++++++++++++++++++++++++++------ src/conf/domain_conf.h | 4 ++ 3 files changed, 103 insertions(+), 14 deletions(-)
ACK Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 12/12/2013 08:21 PM, Daniel P. Berrange wrote:
On Wed, Dec 11, 2013 at 04:29:49PM +0800, Gao feng wrote:
This patch introduces new xml elements under <blkiotune>, we use these new elements to setup the throttle blkio cgroup for domain. The new blkiotune node looks like this:
<blkiotune> <device> <path>/path/to/block</path> <weight>1000</weight> <read_iops_sec>10000</read_iops_sec> <write_iops_sec>10000</write_iops_sec> <read_bytes_sec>1000000</read_bytes_sec> <write_bytes_sec>1000000</write_bytes_sec> </device> </blkiotune>
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- docs/schemas/domaincommon.rng | 28 ++++++++++++-- src/conf/domain_conf.c | 85 +++++++++++++++++++++++++++++++++++++------ src/conf/domain_conf.h | 4 ++ 3 files changed, 103 insertions(+), 14 deletions(-)
ACK
Pushed, thanks!
This patch introduces virCgroupSetBlkioDeviceReadIops, virCgroupSetBlkioDeviceWriteIops, virCgroupSetBlkioDeviceReadBps and virCgroupSetBlkioDeviceWriteBps, we can use these interfaces to set up throttle blkio cgroup for domain. This patch also adds the new throttle blkio cgroup elements to the test xml. Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/libvirt_private.syms | 4 + src/lxc/lxc_cgroup.c | 27 ++- src/qemu/qemu_cgroup.c | 27 ++- src/util/vircgroup.c | 224 ++++++++++++++++++++- src/util/vircgroup.h | 16 ++ .../qemuxml2argv-blkiotune-device.xml | 8 + 6 files changed, 295 insertions(+), 11 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index d8f8f7e..a535bf3 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1031,7 +1031,11 @@ virCgroupNewVcpu; virCgroupPathOfController; virCgroupRemove; virCgroupRemoveRecursively; +virCgroupSetBlkioDeviceReadBps; +virCgroupSetBlkioDeviceReadIops; virCgroupSetBlkioDeviceWeight; +virCgroupSetBlkioDeviceWriteBps; +virCgroupSetBlkioDeviceWriteIops; virCgroupSetBlkioWeight; virCgroupSetCpuCfsPeriod; virCgroupSetCpuCfsQuota; diff --git a/src/lxc/lxc_cgroup.c b/src/lxc/lxc_cgroup.c index 310a476..cc0d5e8 100644 --- a/src/lxc/lxc_cgroup.c +++ b/src/lxc/lxc_cgroup.c @@ -113,9 +113,30 @@ static int virLXCCgroupSetupBlkioTune(virDomainDefPtr def, if (def->blkio.ndevices) { for (i = 0; i < def->blkio.ndevices; i++) { virBlkioDevicePtr dev = &def->blkio.devices[i]; - if (!dev->weight) - continue; - if (virCgroupSetBlkioDeviceWeight(cgroup, dev->path, dev->weight) < 0) + + if (dev->weight && + (virCgroupSetBlkioDeviceWeight(cgroup, dev->path, + dev->weight) < 0)) + return -1; + + if (dev->riops && + (virCgroupSetBlkioDeviceReadIops(cgroup, dev->path, + dev->riops) < 0)) + return -1; + + if (dev->wiops && + (virCgroupSetBlkioDeviceWriteIops(cgroup, dev->path, + dev->wiops) < 0)) + return -1; + + if (dev->rbps && + (virCgroupSetBlkioDeviceReadBps(cgroup, dev->path, + dev->rbps) < 0)) + return -1; + + if (dev->wbps && + (virCgroupSetBlkioDeviceWriteBps(cgroup, dev->path, + dev->wbps) < 0)) return -1; } } diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index a18955e..1cc929c 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -400,10 +400,29 @@ qemuSetupBlkioCgroup(virDomainObjPtr vm) if (vm->def->blkio.ndevices) { for (i = 0; i < vm->def->blkio.ndevices; i++) { virBlkioDevicePtr dev = &vm->def->blkio.devices[i]; - if (!dev->weight) - continue; - if (virCgroupSetBlkioDeviceWeight(priv->cgroup, dev->path, - dev->weight) < 0) + if (dev->weight && + (virCgroupSetBlkioDeviceWeight(priv->cgroup, dev->path, + dev->weight) < 0)) + return -1; + + if (dev->riops && + (virCgroupSetBlkioDeviceReadIops(priv->cgroup, dev->path, + dev->riops) < 0)) + return -1; + + if (dev->wiops && + (virCgroupSetBlkioDeviceWriteIops(priv->cgroup, dev->path, + dev->wiops) < 0)) + return -1; + + if (dev->rbps && + (virCgroupSetBlkioDeviceReadBps(priv->cgroup, dev->path, + dev->rbps) < 0)) + return -1; + + if (dev->wbps && + (virCgroupSetBlkioDeviceWriteBps(priv->cgroup, dev->path, + dev->wbps) < 0)) return -1; } } diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index 43eb649..a6d60c5 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -1824,12 +1824,189 @@ virCgroupGetBlkioWeight(virCgroupPtr group, unsigned int *weight) return ret; } +/** + * virCgroupSetBlkioDeviceReadIops: + * @group: The cgroup to change block io setting for + * @path: The path of device + * @riops: The new device read iops throttle, or 0 to clear + * + * Returns: 0 on success, -1 on error + */ +int +virCgroupSetBlkioDeviceReadIops(virCgroupPtr group, + const char *path, + unsigned int riops) +{ + char *str; + struct stat sb; + int ret; + + if (stat(path, &sb) < 0) { + virReportSystemError(errno, + _("Path '%s' is not accessible"), + path); + return -1; + } + + if (!S_ISBLK(sb.st_mode)) { + virReportSystemError(EINVAL, + _("Path '%s' must be a block device"), + path); + return -1; + } + + if (virAsprintf(&str, "%d:%d %u", major(sb.st_rdev), + minor(sb.st_rdev), riops) < 0) + return -1; + + ret = virCgroupSetValueStr(group, + VIR_CGROUP_CONTROLLER_BLKIO, + "blkio.throttle.read_iops_device", + str); + + VIR_FREE(str); + return ret; +} + /** - * virCgroupSetBlkioDeviceWeight: + * virCgroupSetBlkioDeviceWriteIops: + * @group: The cgroup to change block io setting for + * @path: The path of device + * @wiops: The new device write iops throttle, or 0 to clear + * + * Returns: 0 on success, -1 on error + */ +int +virCgroupSetBlkioDeviceWriteIops(virCgroupPtr group, + const char *path, + unsigned int wiops) +{ + char *str; + struct stat sb; + int ret; + + if (stat(path, &sb) < 0) { + virReportSystemError(errno, + _("Path '%s' is not accessible"), + path); + return -1; + } + + if (!S_ISBLK(sb.st_mode)) { + virReportSystemError(EINVAL, + _("Path '%s' must be a block device"), + path); + return -1; + } + + if (virAsprintf(&str, "%d:%d %u", major(sb.st_rdev), + minor(sb.st_rdev), wiops) < 0) + return -1; + + ret = virCgroupSetValueStr(group, + VIR_CGROUP_CONTROLLER_BLKIO, + "blkio.throttle.write_iops_device", + str); + + VIR_FREE(str); + return ret; +} + + +/** + * virCgroupSetBlkioDeviceReadBps: + * @group: The cgroup to change block io setting for + * @path: The path of device + * @rbps: The new device read bps throttle, or 0 to clear + * + * Returns: 0 on success, -1 on error + */ +int +virCgroupSetBlkioDeviceReadBps(virCgroupPtr group, + const char *path, + unsigned long long rbps) +{ + char *str; + struct stat sb; + int ret; + + if (stat(path, &sb) < 0) { + virReportSystemError(errno, + _("Path '%s' is not accessible"), + path); + return -1; + } + + if (!S_ISBLK(sb.st_mode)) { + virReportSystemError(EINVAL, + _("Path '%s' must be a block device"), + path); + return -1; + } + + if (virAsprintf(&str, "%d:%d %llu", major(sb.st_rdev), + minor(sb.st_rdev), rbps) < 0) + return -1; + + ret = virCgroupSetValueStr(group, + VIR_CGROUP_CONTROLLER_BLKIO, + "blkio.throttle.read_bps_device", + str); + + VIR_FREE(str); + return ret; +} + +/** + * virCgroupSetBlkioDeviceWriteBps: + * @group: The cgroup to change block io setting for + * @path: The path of device + * @wbps: The new device write bps throttle, or 0 to clear * - * @group: The cgroup to change io device weight device for - * @path: The device with a weight to alter + * Returns: 0 on success, -1 on error + */ +int +virCgroupSetBlkioDeviceWriteBps(virCgroupPtr group, + const char *path, + unsigned long long wbps) +{ + char *str; + struct stat sb; + int ret; + + if (stat(path, &sb) < 0) { + virReportSystemError(errno, + _("Path '%s' is not accessible"), + path); + return -1; + } + + if (!S_ISBLK(sb.st_mode)) { + virReportSystemError(EINVAL, + _("Path '%s' must be a block device"), + path); + return -1; + } + + if (virAsprintf(&str, "%d:%d %llu", major(sb.st_rdev), + minor(sb.st_rdev), wbps) < 0) + return -1; + + ret = virCgroupSetValueStr(group, + VIR_CGROUP_CONTROLLER_BLKIO, + "blkio.throttle.write_bps_device", + str); + + VIR_FREE(str); + return ret; +} + + +/** + * virCgroupSetBlkioDeviceWeight: + * @group: The cgroup to change block io setting for + * @path: The path of device * @weight: The new device weight (100-1000), * (10-1000) after kernel 2.6.39, or 0 to clear * @@ -1874,7 +2051,6 @@ virCgroupSetBlkioDeviceWeight(virCgroupPtr group, } - /** * virCgroupSetMemory: * @@ -3312,6 +3488,46 @@ virCgroupSetBlkioDeviceWeight(virCgroupPtr group ATTRIBUTE_UNUSED, return -1; } +int +virCgroupSetBlkioDeviceReadIops(virCgroupPtr group ATTRIBUTE_UNUSED, + const char *path ATTRIBUTE_UNUSED, + unsigned int riops ATTRIBUTE_UNUSED) +{ + virReportSystemError(ENOSYS, "%s", + _("Control groups not supported on this platform")); + return -1; +} + +int +virCgroupSetBlkioDeviceWriteIops(virCgroupPtr group ATTRIBUTE_UNUSED, + const char *path ATTRIBUTE_UNUSED, + unsigned int wiops ATTRIBUTE_UNUSED) +{ + virReportSystemError(ENOSYS, "%s", + _("Control groups not supported on this platform")); + return -1; +} + +int +virCgroupSetBlkioDeviceReadBps(virCgroupPtr group ATTRIBUTE_UNUSED, + const char *path ATTRIBUTE_UNUSED, + unsigned long long rbps ATTRIBUTE_UNUSED) +{ + virReportSystemError(ENOSYS, "%s", + _("Control groups not supported on this platform")); + return -1; +} + +int +virCgroupSetBlkioDeviceWriteBps(virCgroupPtr group ATTRIBUTE_UNUSED, + const char *path ATTRIBUTE_UNUSED, + unsigned long long wbps ATTRIBUTE_UNUSED) +{ + virReportSystemError(ENOSYS, "%s", + _("Control groups not supported on this platform")); + return -1; +} + int virCgroupSetMemory(virCgroupPtr group ATTRIBUTE_UNUSED, diff --git a/src/util/vircgroup.h b/src/util/vircgroup.h index 835eb30..a70eb18 100644 --- a/src/util/vircgroup.h +++ b/src/util/vircgroup.h @@ -126,6 +126,22 @@ int virCgroupSetBlkioDeviceWeight(virCgroupPtr group, const char *path, unsigned int weight); +int virCgroupSetBlkioDeviceReadIops(virCgroupPtr group, + const char *path, + unsigned int riops); + +int virCgroupSetBlkioDeviceWriteIops(virCgroupPtr group, + const char *path, + unsigned int wiops); + +int virCgroupSetBlkioDeviceReadBps(virCgroupPtr group, + const char *path, + unsigned long long rbps); + +int virCgroupSetBlkioDeviceWriteBps(virCgroupPtr group, + const char *path, + unsigned long long wbps); + int virCgroupSetMemory(virCgroupPtr group, unsigned long long kb); int virCgroupGetMemoryUsage(virCgroupPtr group, unsigned long *kb); diff --git a/tests/qemuxml2argvdata/qemuxml2argv-blkiotune-device.xml b/tests/qemuxml2argvdata/qemuxml2argv-blkiotune-device.xml index 743cf29..bfb5b03 100644 --- a/tests/qemuxml2argvdata/qemuxml2argv-blkiotune-device.xml +++ b/tests/qemuxml2argvdata/qemuxml2argv-blkiotune-device.xml @@ -8,10 +8,18 @@ <device> <path>/dev/sda</path> <weight>400</weight> + <read_iops_sec>10000</read_iops_sec> + <write_iops_sec>10000</write_iops_sec> + <read_bytes_sec>10000</read_bytes_sec> + <write_bytes_sec>10000</write_bytes_sec> </device> <device> <path>/dev/sdb</path> <weight>900</weight> + <read_iops_sec>20000</read_iops_sec> + <write_iops_sec>20000</write_iops_sec> + <read_bytes_sec>20000</read_bytes_sec> + <write_bytes_sec>20000</write_bytes_sec> </device> </blkiotune> <vcpu placement='static'>1</vcpu> -- 1.8.3.1
On Wed, Dec 11, 2013 at 04:29:50PM +0800, Gao feng wrote:
This patch introduces virCgroupSetBlkioDeviceReadIops, virCgroupSetBlkioDeviceWriteIops, virCgroupSetBlkioDeviceReadBps and virCgroupSetBlkioDeviceWriteBps,
we can use these interfaces to set up throttle blkio cgroup for domain.
This patch also adds the new throttle blkio cgroup elements to the test xml.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/libvirt_private.syms | 4 + src/lxc/lxc_cgroup.c | 27 ++- src/qemu/qemu_cgroup.c | 27 ++- src/util/vircgroup.c | 224 ++++++++++++++++++++- src/util/vircgroup.h | 16 ++ .../qemuxml2argv-blkiotune-device.xml | 8 + 6 files changed, 295 insertions(+), 11 deletions(-)
ACK Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 12/12/2013 08:24 PM, Daniel P. Berrange wrote:
On Wed, Dec 11, 2013 at 04:29:50PM +0800, Gao feng wrote:
This patch introduces virCgroupSetBlkioDeviceReadIops, virCgroupSetBlkioDeviceWriteIops, virCgroupSetBlkioDeviceReadBps and virCgroupSetBlkioDeviceWriteBps,
we can use these interfaces to set up throttle blkio cgroup for domain.
This patch also adds the new throttle blkio cgroup elements to the test xml.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/libvirt_private.syms | 4 + src/lxc/lxc_cgroup.c | 27 ++- src/qemu/qemu_cgroup.c | 27 ++- src/util/vircgroup.c | 224 ++++++++++++++++++++- src/util/vircgroup.h | 16 ++ .../qemuxml2argv-blkiotune-device.xml | 8 + 6 files changed, 295 insertions(+), 11 deletions(-)
ACK
Pushed, thanks!
With this patch, user can setup the throttle blkio cgorup for domain through the virsh cmd, such as: virsh blkiotune domain1 --device-read-bytes-sec /dev/sda1,1000000,/dev/sda2,2000000 --device-write-bytes-sec /dev/sda1,1000000 --device-read-iops-sec /dev/sda1,10000 --device-write-iops-sec /dev/sda1,10000,/dev/sda2,0 This patch also add manpage for these new options. Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- include/libvirt/libvirt.h.in | 45 +++++++++++++++++++++++++++++++ tools/virsh-domain.c | 64 ++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 36 +++++++++++++++++++++++-- 3 files changed, 143 insertions(+), 2 deletions(-) diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index 29d4dce..9ebf3c4 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -1806,6 +1806,51 @@ char * virDomainGetSchedulerType(virDomainPtr domain, #define VIR_DOMAIN_BLKIO_DEVICE_WEIGHT "device_weight" +/** + * VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS: + * + * Macro for the blkio tunable throttle.read_iops_device: it represents + * the number of reading the block device per second, as a string. The + * string is parsed as a series of /path/to/device, read_iops elements, + * separated by ','. + */ + +#define VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS "device_read_iops_sec" + + +/** + * VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS: + * + * Macro for the blkio tunable throttle.write_iops_device: it represents + * the number of writing the block device per second, as a string. The + * string is parsed as a series of /path/to/device, write_iops elements, + * separated by ','. + */ +#define VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS "device_write_iops_sec" + + +/** + * VIR_DOMAIN_BLKIO_DEVICE_READ_BPS: + * + * Macro for the blkio tunable throttle.read_iops_device: it represents + * the bytes of reading the block device per second, as a string. The + * string is parsed as a series of /path/to/device, read_bps elements, + * separated by ','. + */ +#define VIR_DOMAIN_BLKIO_DEVICE_READ_BPS "device_read_bytes_sec" + + +/** + * VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS: + * + * Macro for the blkio tunable throttle.read_iops_device: it represents + * the number of reading the block device per second, as a string. The + * string is parsed as a series of /path/to/device, write_bps elements, + * separated by ','. + */ +#define VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS "device_write_bytes_sec" + + /* Set Blkio tunables for the domain*/ int virDomainSetBlkioParameters(virDomainPtr domain, virTypedParameterPtr params, diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 8b80e1e..f7e7959 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -1250,6 +1250,22 @@ static const vshCmdOptDef opts_blkiotune[] = { .type = VSH_OT_STRING, .help = N_("per-device IO Weights, in the form of /path/to/device,weight,...") }, + {.name = "device-read-iops-sec", + .type = VSH_OT_STRING, + .help = N_("per-device read I/O limit per second, in the form of /path/to/device,read_iops_sec,...") + }, + {.name = "device-write-iops-sec", + .type = VSH_OT_STRING, + .help = N_("per-device write I/O limit per second, in the form of /path/to/device,write_iops_sec,...") + }, + {.name = "device-read-bytes-sec", + .type = VSH_OT_STRING, + .help = N_("per-device bytes read per second, in the form of /path/to/device,read_bytes_sec,...") + }, + {.name = "device-write-bytes-sec", + .type = VSH_OT_STRING, + .help = N_("per-device bytes wrote per second, in the form of /path/to/device,write_bytes_sec,...") + }, {.name = "config", .type = VSH_OT_BOOL, .help = N_("affect next boot") @@ -1270,6 +1286,10 @@ cmdBlkiotune(vshControl * ctl, const vshCmd * cmd) { virDomainPtr dom; const char *device_weight = NULL; + const char *device_riops = NULL; + const char *device_wiops = NULL; + const char *device_rbps = NULL; + const char *device_wbps = NULL; int weight = 0; int nparams = 0; int maxparams = 0; @@ -1317,6 +1337,50 @@ cmdBlkiotune(vshControl * ctl, const vshCmd * cmd) goto save_error; } + rv = vshCommandOptString(cmd, "device-read-iops-sec", &device_riops); + if (rv < 0) { + vshError(ctl, "%s", _("Unable to parse string parameter")); + goto cleanup; + } else if (rv > 0) { + if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS, + device_riops) < 0) + goto save_error; + } + + rv = vshCommandOptString(cmd, "device-write-iops-sec", &device_wiops); + if (rv < 0) { + vshError(ctl, "%s", _("Unable to parse string parameter")); + goto cleanup; + } else if (rv > 0) { + if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS, + device_wiops) < 0) + goto save_error; + } + + rv = vshCommandOptString(cmd, "device-read-bytes-sec", &device_rbps); + if (rv < 0) { + vshError(ctl, "%s", _("Unable to parse string parameter")); + goto cleanup; + } else if (rv > 0) { + if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS, + device_rbps) < 0) + goto save_error; + } + + rv = vshCommandOptString(cmd, "device-write-bytes-sec", &device_wbps); + if (rv < 0) { + vshError(ctl, "%s", _("Unable to parse string parameter")); + goto cleanup; + } else if (rv > 0) { + if (virTypedParamsAddString(¶ms, &nparams, &maxparams, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS, + device_wbps) < 0) + goto save_error; + } + if (nparams == 0) { /* get the number of blkio parameters */ if (virDomainGetBlkioParameters(dom, NULL, &nparams, flags) != 0) { diff --git a/tools/virsh.pod b/tools/virsh.pod index c6a8be3..b3c825b 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -1625,8 +1625,12 @@ The guaranteed minimum memory allocation for the guest. Specifying -1 as a value for these limits is interpreted as unlimited. =item B<blkiotune> I<domain> [I<--weight> B<weight>] -[I<--device-weights> B<device-weights>] [[I<--config>] -[I<--live>] | [I<--current>]] +[I<--device-weights> B<device-weights>] +[I<--device-read-iops-sec> B<device-read-iops-sec>] +[I<--device-write-iops-sec> B<device-write-iops-sec>] +[I<--device-read-bytes-sec> B<device-read-bytes-sec>] +[I<--device-write-bytes-sec> B<device-write-bytes-sec>] +[[I<--config>] [I<--live>] | [I<--current>]] Display or set the blkio parameters. QEMU/KVM supports I<--weight>. I<--weight> is in range [100, 1000]. After kernel 2.6.39, the value @@ -1639,6 +1643,34 @@ or the value 0 to remove that device from per-device listings. Only the devices listed in the string are modified; any existing per-device weights for other devices remain unchanged. +B<device-read-iops-sec> is a single string listing one or more device/read_iops_sec +pairs, int the format of /path/to/device,read_iops_sec,/path/to/device,read_iops_sec. +Each read_iops_sec is a number which type is unsigned int, value 0 to remove that +device from per-decice listing. +Only the devices listed in the string are modified; +any existing per-device read_iops_sec for other devices remain unchanged. + +B<device-write-iops-sec> is a single string listing one or more device/write_iops_sec +pairs, int the format of /path/to/device,write_iops_sec,/path/to/device,write_iops_sec. +Each write_iops_sec is a number which type is unsigned int, value 0 to remove that +device from per-decice listing. +Only the devices listed in the string are modified; +any existing per-device write_iops_sec for other devices remain unchanged. + +B<device-read-bytes-sec> is a single string listing one or more device/read_bytes_sec +pairs, int the format of /path/to/device,read_bytes_sec,/path/to/device,read_bytes_sec. +Each read_bytes_sec is a number which type is unsigned long long, value 0 to remove +that device from per-decice listing. +Only the devices listed in the string are modified; +any existing per-device read_bytes_sec for other devices remain unchanged. + +B<device-write-bytes-sec> is a single string listing one or more device/write_bytes_sec +pairs, int the format of /path/to/device,write_bytes_sec,/path/to/device,write_bytes_sec. +Each write_bytes_sec is a number which type is unsigned long long, value 0 to remove +that device from per-decice listing. +Only the devices listed in the string are modified; +any existing per-device write_bytes_sec for other devices remain unchanged. + If I<--live> is specified, affect a running guest. If I<--config> is specified, affect the next boot of a persistent guest. If I<--current> is specified, affect the current guest state. -- 1.8.3.1
On Wed, Dec 11, 2013 at 04:29:51PM +0800, Gao feng wrote:
With this patch, user can setup the throttle blkio cgorup for domain through the virsh cmd, such as:
virsh blkiotune domain1 --device-read-bytes-sec /dev/sda1,1000000,/dev/sda2,2000000 --device-write-bytes-sec /dev/sda1,1000000 --device-read-iops-sec /dev/sda1,10000 --device-write-iops-sec /dev/sda1,10000,/dev/sda2,0
This patch also add manpage for these new options.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- include/libvirt/libvirt.h.in | 45 +++++++++++++++++++++++++++++++ tools/virsh-domain.c | 64 ++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 36 +++++++++++++++++++++++-- 3 files changed, 143 insertions(+), 2 deletions(-)
ACK Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 12/12/2013 08:26 PM, Daniel P. Berrange wrote:
On Wed, Dec 11, 2013 at 04:29:51PM +0800, Gao feng wrote:
With this patch, user can setup the throttle blkio cgorup for domain through the virsh cmd, such as:
virsh blkiotune domain1 --device-read-bytes-sec /dev/sda1,1000000,/dev/sda2,2000000 --device-write-bytes-sec /dev/sda1,1000000 --device-read-iops-sec /dev/sda1,10000 --device-write-iops-sec /dev/sda1,10000,/dev/sda2,0
This patch also add manpage for these new options.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- include/libvirt/libvirt.h.in | 45 +++++++++++++++++++++++++++++++ tools/virsh-domain.c | 64 ++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 36 +++++++++++++++++++++++-- 3 files changed, 143 insertions(+), 2 deletions(-)
ACK
Pushed, thanks!
With this patch, user can setup throttle blkio cgroup through virsh for qemu domain. Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/qemu/qemu_driver.c | 425 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 386 insertions(+), 39 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 45d11cd..0300db9 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -131,7 +131,7 @@ # define KVM_CAP_NR_VCPUS 9 /* returns max vcpus per vm */ #endif -#define QEMU_NB_BLKIO_PARAM 2 +#define QEMU_NB_BLKIO_PARAM 6 #define QEMU_NB_BANDWIDTH_PARAM 6 @@ -7427,12 +7427,12 @@ cleanup: return ret; } -/* deviceWeightStr in the form of /device/path,weight,/device/path,weight +/* blkioDeviceStr in the form of /device/path,weight,/device/path,weight * for example, /dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0,800 */ static int -qemuDomainParseDeviceWeightStr(char *deviceWeightStr, - virBlkioDevicePtr *dev, size_t *size) +qemuDomainParseBlkioDeviceStr(char *blkioDeviceStr, const char *type, + virBlkioDevicePtr *dev, size_t *size) { char *temp; int ndevices = 0; @@ -7443,10 +7443,10 @@ qemuDomainParseDeviceWeightStr(char *deviceWeightStr, *dev = NULL; *size = 0; - if (STREQ(deviceWeightStr, "")) + if (STREQ(blkioDeviceStr, "")) return 0; - temp = deviceWeightStr; + temp = blkioDeviceStr; while (temp) { temp = strchr(temp, ','); if (temp) { @@ -7466,7 +7466,7 @@ qemuDomainParseDeviceWeightStr(char *deviceWeightStr, return -1; i = 0; - temp = deviceWeightStr; + temp = blkioDeviceStr; while (temp) { char *p = temp; @@ -7478,11 +7478,25 @@ qemuDomainParseDeviceWeightStr(char *deviceWeightStr, if (VIR_STRNDUP(result[i].path, temp, p - temp) < 0) goto cleanup; - /* weight */ + /* value */ temp = p + 1; - if (virStrToLong_ui(temp, &p, 10, &result[i].weight) < 0) - goto error; + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].weight) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].riops) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].wiops) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) { + if (virStrToLong_ull(temp, &p, 10, &result[i].rbps) < 0) + goto error; + } else { + if (virStrToLong_ull(temp, &p, 10, &result[i].wbps) < 0) + goto error; + } i++; @@ -7503,20 +7517,21 @@ qemuDomainParseDeviceWeightStr(char *deviceWeightStr, error: virReportError(VIR_ERR_INVALID_ARG, - _("unable to parse device weight '%s'"), deviceWeightStr); + _("unable to parse blkio device '%s'"), blkioDeviceStr); cleanup: virBlkioDeviceArrayClear(result, ndevices); VIR_FREE(result); return -1; } -/* Modify dest_array to reflect all device weight changes described in +/* Modify dest_array to reflect all blkio device changes described in * src_array. */ static int -qemuDomainMergeDeviceWeights(virBlkioDevicePtr *dest_array, - size_t *dest_size, - virBlkioDevicePtr src_array, - size_t src_size) +qemuDomainMergeBlkioDevice(virBlkioDevicePtr *dest_array, + size_t *dest_size, + virBlkioDevicePtr src_array, + size_t src_size, + const char *type) { size_t i, j; virBlkioDevicePtr dest, src; @@ -7529,18 +7544,40 @@ qemuDomainMergeDeviceWeights(virBlkioDevicePtr *dest_array, dest = &(*dest_array)[j]; if (STREQ(src->path, dest->path)) { found = true; - dest->weight = src->weight; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) + dest->weight = src->weight; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) + dest->riops = src->riops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) + dest->wiops = src->wiops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) + dest->rbps = src->rbps; + else + dest->wbps = src->wbps; + break; } } if (!found) { - if (!src->weight) + if (!src->weight && !src->riops && src->wiops && src->rbps && src->wbps) continue; if (VIR_EXPAND_N(*dest_array, *dest_size, 1) < 0) return -1; dest = &(*dest_array)[*dest_size - 1]; dest->path = src->path; - dest->weight = src->weight; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) + dest->weight = src->weight; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) + dest->riops = src->riops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) + dest->wiops = src->wiops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) + dest->rbps = src->rbps; + else + dest->wbps = src->wbps; + src->path = NULL; } } @@ -7570,6 +7607,14 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, VIR_TYPED_PARAM_UINT, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT, VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS, + VIR_TYPED_PARAM_STRING, NULL) < 0) return -1; @@ -7612,29 +7657,74 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, if (virCgroupSetBlkioWeight(priv->cgroup, params[i].value.ui) < 0) ret = -1; - } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) { size_t ndevices; virBlkioDevicePtr devices = NULL; size_t j; - if (qemuDomainParseDeviceWeightStr(params[i].value.s, - &devices, - &ndevices) < 0) { + if (qemuDomainParseBlkioDeviceStr(params[i].value.s, + param->field, + &devices, + &ndevices) < 0) { ret = -1; continue; } - for (j = 0; j < ndevices; j++) { - if (virCgroupSetBlkioDeviceWeight(priv->cgroup, - devices[j].path, - devices[j].weight) < 0) { - ret = -1; - break; + + if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWeight(priv->cgroup, + devices[j].path, + devices[j].weight) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceReadIops(priv->cgroup, + devices[j].path, + devices[j].riops) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWriteIops(priv->cgroup, + devices[j].path, + devices[j].wiops) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceReadBps(priv->cgroup, + devices[j].path, + devices[j].rbps) < 0) { + ret = -1; + break; + } + } + } else { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWriteBps(priv->cgroup, + devices[j].path, + devices[j].wbps) < 0) { + ret = -1; + break; + } } } + if (j != ndevices || - qemuDomainMergeDeviceWeights(&vm->def->blkio.devices, - &vm->def->blkio.ndevices, - devices, ndevices) < 0) + qemuDomainMergeBlkioDevice(&vm->def->blkio.devices, + &vm->def->blkio.ndevices, + devices, ndevices, param->field) < 0) ret = -1; virBlkioDeviceArrayClear(devices, ndevices); VIR_FREE(devices); @@ -7659,19 +7749,24 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, } persistentDef->blkio.weight = params[i].value.ui; - } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) { virBlkioDevicePtr devices = NULL; size_t ndevices; - if (qemuDomainParseDeviceWeightStr(params[i].value.s, - &devices, - &ndevices) < 0) { + if (qemuDomainParseBlkioDeviceStr(params[i].value.s, + params->field, + &devices, + &ndevices) < 0) { ret = -1; continue; } - if (qemuDomainMergeDeviceWeights(&persistentDef->blkio.devices, - &persistentDef->blkio.ndevices, - devices, ndevices) < 0) + if (qemuDomainMergeBlkioDevice(&persistentDef->blkio.devices, + &persistentDef->blkio.ndevices, + devices, ndevices, param->field) < 0) ret = -1; virBlkioDeviceArrayClear(devices, ndevices); VIR_FREE(devices); @@ -7757,6 +7852,7 @@ qemuDomainGetBlkioParameters(virDomainPtr dom, VIR_TYPED_PARAM_UINT, val) < 0) goto cleanup; break; + case 1: /* blkiotune.device_weight */ if (vm->def->blkio.ndevices > 0) { virBuffer buf = VIR_BUFFER_INITIALIZER; @@ -7786,6 +7882,122 @@ qemuDomainGetBlkioParameters(virDomainPtr dom, goto cleanup; break; + case 2: /* blkiotune.device_read_iops */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].riops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].riops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 3: /* blkiotune.device_write_iops */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].wiops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].wiops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 4: /* blkiotune.device_read_bps */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].rbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].rbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 5: /* blkiotune.device_write_bps */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].wbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].wbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + default: break; /* should not hit here */ @@ -7843,6 +8055,141 @@ qemuDomainGetBlkioParameters(virDomainPtr dom, } break; + case 2: /* blkiotune.device_read_iops */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].riops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].riops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS); + goto cleanup; + } + break; + case 3: /* blkiotune.device_write_iops */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].wiops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].wiops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS); + goto cleanup; + } + break; + case 4: /* blkiotune.device_read_bps */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].rbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].rbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS); + goto cleanup; + } + break; + + case 5: /* blkiotune.device_write_bps */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].wbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].wbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS); + goto cleanup; + } + break; + + default: break; /* should not hit here */ -- 1.8.3.1
On Wed, Dec 11, 2013 at 04:29:52PM +0800, Gao feng wrote:
With this patch, user can setup throttle blkio cgroup through virsh for qemu domain.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/qemu/qemu_driver.c | 425 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 386 insertions(+), 39 deletions(-)
- for (j = 0; j < ndevices; j++) { - if (virCgroupSetBlkioDeviceWeight(priv->cgroup, - devices[j].path, - devices[j].weight) < 0) { - ret = -1; - break; + + if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWeight(priv->cgroup, + devices[j].path, + devices[j].weight) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceReadIops(priv->cgroup, + devices[j].path, + devices[j].riops) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWriteIops(priv->cgroup, + devices[j].path, + devices[j].wiops) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceReadBps(priv->cgroup, + devices[j].path, + devices[j].rbps) < 0) { + ret = -1; + break; + } + } + } else {
You should check for WRITE_BPS here, and have a final 'else' clause reporting an error on unknown strings I think
+ for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWriteBps(priv->cgroup, + devices[j].path, + devices[j].wbps) < 0) { + ret = -1; + break; + } } }
Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
With this patch, user can setup throttle blkio cgroup through virsh for qemu domain. Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/qemu/qemu_driver.c | 442 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 404 insertions(+), 38 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 45d11cd..8aac133 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -131,7 +131,7 @@ # define KVM_CAP_NR_VCPUS 9 /* returns max vcpus per vm */ #endif -#define QEMU_NB_BLKIO_PARAM 2 +#define QEMU_NB_BLKIO_PARAM 6 #define QEMU_NB_BANDWIDTH_PARAM 6 @@ -7427,12 +7427,12 @@ cleanup: return ret; } -/* deviceWeightStr in the form of /device/path,weight,/device/path,weight +/* blkioDeviceStr in the form of /device/path,weight,/device/path,weight * for example, /dev/disk/by-path/pci-0000:00:1f.2-scsi-0:0:0:0,800 */ static int -qemuDomainParseDeviceWeightStr(char *deviceWeightStr, - virBlkioDevicePtr *dev, size_t *size) +qemuDomainParseBlkioDeviceStr(char *blkioDeviceStr, const char *type, + virBlkioDevicePtr *dev, size_t *size) { char *temp; int ndevices = 0; @@ -7443,10 +7443,10 @@ qemuDomainParseDeviceWeightStr(char *deviceWeightStr, *dev = NULL; *size = 0; - if (STREQ(deviceWeightStr, "")) + if (STREQ(blkioDeviceStr, "")) return 0; - temp = deviceWeightStr; + temp = blkioDeviceStr; while (temp) { temp = strchr(temp, ','); if (temp) { @@ -7466,7 +7466,7 @@ qemuDomainParseDeviceWeightStr(char *deviceWeightStr, return -1; i = 0; - temp = deviceWeightStr; + temp = blkioDeviceStr; while (temp) { char *p = temp; @@ -7478,11 +7478,27 @@ qemuDomainParseDeviceWeightStr(char *deviceWeightStr, if (VIR_STRNDUP(result[i].path, temp, p - temp) < 0) goto cleanup; - /* weight */ + /* value */ temp = p + 1; - if (virStrToLong_ui(temp, &p, 10, &result[i].weight) < 0) + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].weight) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].riops) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].wiops) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) { + if (virStrToLong_ull(temp, &p, 10, &result[i].rbps) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) { + if (virStrToLong_ull(temp, &p, 10, &result[i].wbps) < 0) + goto error; + } else { goto error; + } i++; @@ -7503,20 +7519,22 @@ qemuDomainParseDeviceWeightStr(char *deviceWeightStr, error: virReportError(VIR_ERR_INVALID_ARG, - _("unable to parse device weight '%s'"), deviceWeightStr); + _("unable to parse blkio device '%s' '%s'"), + type, blkioDeviceStr); cleanup: virBlkioDeviceArrayClear(result, ndevices); VIR_FREE(result); return -1; } -/* Modify dest_array to reflect all device weight changes described in +/* Modify dest_array to reflect all blkio device changes described in * src_array. */ static int -qemuDomainMergeDeviceWeights(virBlkioDevicePtr *dest_array, - size_t *dest_size, - virBlkioDevicePtr src_array, - size_t src_size) +qemuDomainMergeBlkioDevice(virBlkioDevicePtr *dest_array, + size_t *dest_size, + virBlkioDevicePtr src_array, + size_t src_size, + const char *type) { size_t i, j; virBlkioDevicePtr dest, src; @@ -7529,18 +7547,48 @@ qemuDomainMergeDeviceWeights(virBlkioDevicePtr *dest_array, dest = &(*dest_array)[j]; if (STREQ(src->path, dest->path)) { found = true; - dest->weight = src->weight; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) + dest->weight = src->weight; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) + dest->riops = src->riops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) + dest->wiops = src->wiops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) + dest->rbps = src->rbps; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) + dest->wbps = src->wbps; + else { + virReportError(VIR_ERR_INVALID_ARG, _("Unknown parameter %s"), + type); + return -1; + } break; } } if (!found) { - if (!src->weight) + if (!src->weight && !src->riops && !src->wiops && !src->rbps && !src->wbps) continue; if (VIR_EXPAND_N(*dest_array, *dest_size, 1) < 0) return -1; dest = &(*dest_array)[*dest_size - 1]; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) + dest->weight = src->weight; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) + dest->riops = src->riops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) + dest->wiops = src->wiops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) + dest->rbps = src->rbps; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) + dest->wbps = src->wbps; + else { + *dest_size = *dest_size - 1; + return -1; + } + dest->path = src->path; - dest->weight = src->weight; src->path = NULL; } } @@ -7570,6 +7618,14 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, VIR_TYPED_PARAM_UINT, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT, VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS, + VIR_TYPED_PARAM_STRING, NULL) < 0) return -1; @@ -7612,29 +7668,82 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, if (virCgroupSetBlkioWeight(priv->cgroup, params[i].value.ui) < 0) ret = -1; - } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) { size_t ndevices; virBlkioDevicePtr devices = NULL; size_t j; - if (qemuDomainParseDeviceWeightStr(params[i].value.s, - &devices, - &ndevices) < 0) { + if (qemuDomainParseBlkioDeviceStr(params[i].value.s, + param->field, + &devices, + &ndevices) < 0) { ret = -1; continue; } - for (j = 0; j < ndevices; j++) { - if (virCgroupSetBlkioDeviceWeight(priv->cgroup, - devices[j].path, - devices[j].weight) < 0) { - ret = -1; - break; + + if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWeight(priv->cgroup, + devices[j].path, + devices[j].weight) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceReadIops(priv->cgroup, + devices[j].path, + devices[j].riops) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWriteIops(priv->cgroup, + devices[j].path, + devices[j].wiops) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceReadBps(priv->cgroup, + devices[j].path, + devices[j].rbps) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)){ + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWriteBps(priv->cgroup, + devices[j].path, + devices[j].wbps) < 0) { + ret = -1; + break; + } } + } else { + virReportError(VIR_ERR_INVALID_ARG, _("Unknown blkio parameter %s"), + param->field); + ret = -1; + virBlkioDeviceArrayClear(devices, ndevices); + VIR_FREE(devices); + + continue; } + if (j != ndevices || - qemuDomainMergeDeviceWeights(&vm->def->blkio.devices, - &vm->def->blkio.ndevices, - devices, ndevices) < 0) + qemuDomainMergeBlkioDevice(&vm->def->blkio.devices, + &vm->def->blkio.ndevices, + devices, ndevices, param->field) < 0) ret = -1; virBlkioDeviceArrayClear(devices, ndevices); VIR_FREE(devices); @@ -7659,19 +7768,24 @@ qemuDomainSetBlkioParameters(virDomainPtr dom, } persistentDef->blkio.weight = params[i].value.ui; - } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) { virBlkioDevicePtr devices = NULL; size_t ndevices; - if (qemuDomainParseDeviceWeightStr(params[i].value.s, - &devices, - &ndevices) < 0) { + if (qemuDomainParseBlkioDeviceStr(params[i].value.s, + params->field, + &devices, + &ndevices) < 0) { ret = -1; continue; } - if (qemuDomainMergeDeviceWeights(&persistentDef->blkio.devices, - &persistentDef->blkio.ndevices, - devices, ndevices) < 0) + if (qemuDomainMergeBlkioDevice(&persistentDef->blkio.devices, + &persistentDef->blkio.ndevices, + devices, ndevices, param->field) < 0) ret = -1; virBlkioDeviceArrayClear(devices, ndevices); VIR_FREE(devices); @@ -7757,6 +7871,7 @@ qemuDomainGetBlkioParameters(virDomainPtr dom, VIR_TYPED_PARAM_UINT, val) < 0) goto cleanup; break; + case 1: /* blkiotune.device_weight */ if (vm->def->blkio.ndevices > 0) { virBuffer buf = VIR_BUFFER_INITIALIZER; @@ -7786,6 +7901,122 @@ qemuDomainGetBlkioParameters(virDomainPtr dom, goto cleanup; break; + case 2: /* blkiotune.device_read_iops */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].riops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].riops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 3: /* blkiotune.device_write_iops */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].wiops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].wiops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 4: /* blkiotune.device_read_bps */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].rbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].rbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 5: /* blkiotune.device_write_bps */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].wbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].wbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + default: break; /* should not hit here */ @@ -7843,6 +8074,141 @@ qemuDomainGetBlkioParameters(virDomainPtr dom, } break; + case 2: /* blkiotune.device_read_iops */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].riops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].riops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS); + goto cleanup; + } + break; + case 3: /* blkiotune.device_write_iops */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].wiops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].wiops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS); + goto cleanup; + } + break; + case 4: /* blkiotune.device_read_bps */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].rbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].rbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS); + goto cleanup; + } + break; + + case 5: /* blkiotune.device_write_bps */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].wbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].wbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS); + goto cleanup; + } + break; + + default: break; /* should not hit here */ -- 1.8.3.1
On Fri, Dec 13, 2013 at 11:08:10AM +0800, Gao feng wrote:
With this patch, user can setup throttle blkio cgroup through virsh for qemu domain.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/qemu/qemu_driver.c | 442 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 404 insertions(+), 38 deletions(-)
ACK Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
With this patch,user can set throttle blkio cgroup for lxc domain through virsh tool. The functions are copied from qemu_driver. Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_driver.c | 649 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 624 insertions(+), 25 deletions(-) diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index c499182..1f73b52 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -1545,6 +1545,20 @@ static int lxcStateCleanup(void) return 0; } +static int +lxcConnectSupportsFeature(virConnectPtr conn, int feature) +{ + if (virConnectSupportsFeatureEnsureACL(conn) < 0) + return -1; + + switch (feature) { + case VIR_DRV_FEATURE_TYPED_PARAM_STRING: + return 1; + default: + return 0; + } +} + static int lxcConnectGetVersion(virConnectPtr conn, unsigned long *version) { @@ -1910,6 +1924,159 @@ lxcDomainGetSchedulerParameters(virDomainPtr domain, return lxcDomainGetSchedulerParametersFlags(domain, params, nparams, 0); } +static int +lxcDomainParseBlkioDeviceStr(char *blkioDeviceStr, const char *type, + virBlkioDevicePtr *dev, size_t *size) +{ + char *temp; + int ndevices = 0; + int nsep = 0; + size_t i; + virBlkioDevicePtr result = NULL; + + *dev = NULL; + *size = 0; + + if (STREQ(blkioDeviceStr, "")) + return 0; + + temp = blkioDeviceStr; + while (temp) { + temp = strchr(temp, ','); + if (temp) { + temp++; + nsep++; + } + } + + /* A valid string must have even number of fields, hence an odd + * number of commas. */ + if (!(nsep & 1)) + goto error; + + ndevices = (nsep + 1) / 2; + + if (VIR_ALLOC_N(result, ndevices) < 0) + return -1; + + i = 0; + temp = blkioDeviceStr; + while (temp) { + char *p = temp; + + /* device path */ + p = strchr(p, ','); + if (!p) + goto error; + + if (VIR_STRNDUP(result[i].path, temp, p - temp) < 0) + goto cleanup; + + /* value */ + temp = p + 1; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].weight) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].riops) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].wiops) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) { + if (virStrToLong_ull(temp, &p, 10, &result[i].rbps) < 0) + goto error; + } else { + if (virStrToLong_ull(temp, &p, 10, &result[i].wbps) < 0) + goto error; + } + + i++; + + if (*p == '\0') + break; + else if (*p != ',') + goto error; + temp = p + 1; + } + + if (!i) + VIR_FREE(result); + + *dev = result; + *size = i; + + return 0; + +error: + virReportError(VIR_ERR_INVALID_ARG, + _("unable to parse device weight '%s'"), blkioDeviceStr); +cleanup: + virBlkioDeviceArrayClear(result, ndevices); + VIR_FREE(result); + return -1; +} + +static int +lxcDomainMergeBlkioDevice(virBlkioDevicePtr *dest_array, + size_t *dest_size, + virBlkioDevicePtr src_array, + size_t src_size, + const char *type) +{ + size_t i, j; + virBlkioDevicePtr dest, src; + + for (i = 0; i < src_size; i++) { + bool found = false; + + src = &src_array[i]; + for (j = 0; j < *dest_size; j++) { + dest = &(*dest_array)[j]; + if (STREQ(src->path, dest->path)) { + found = true; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) + dest->weight = src->weight; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) + dest->riops = src->riops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) + dest->wiops = src->wiops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) + dest->rbps = src->rbps; + else + dest->wbps = src->wbps; + + break; + } + } + if (!found) { + if (!src->weight && !src->riops && !src->wiops && !src->rbps && !src->wbps) + continue; + if (VIR_EXPAND_N(*dest_array, *dest_size, 1) < 0) + return -1; + dest = &(*dest_array)[*dest_size - 1]; + dest->path = src->path; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) + dest->weight = src->weight; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) + dest->riops = src->riops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) + dest->wiops = src->wiops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) + dest->rbps = src->rbps; + else + dest->wbps = src->wbps; + + src->path = NULL; + } + } + + return 0; +} + static int lxcDomainSetBlkioParameters(virDomainPtr dom, @@ -1918,26 +2085,37 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, unsigned int flags) { virLXCDriverPtr driver = dom->conn->privateData; - virCapsPtr caps = NULL; size_t i; virDomainObjPtr vm = NULL; virDomainDefPtr persistentDef = NULL; int ret = -1; + virLXCDriverConfigPtr cfg = NULL; + virCapsPtr caps = NULL; virLXCDomainObjPrivatePtr priv; - virLXCDriverConfigPtr cfg = virLXCDriverGetConfig(driver); virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | VIR_DOMAIN_AFFECT_CONFIG, -1); if (virTypedParamsValidate(params, nparams, VIR_DOMAIN_BLKIO_WEIGHT, VIR_TYPED_PARAM_UINT, + VIR_DOMAIN_BLKIO_DEVICE_WEIGHT, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS, + VIR_TYPED_PARAM_STRING, NULL) < 0) return -1; if (!(vm = lxcDomObjFromDomain(dom))) - goto cleanup; + return -1; priv = vm->privateData; + cfg = virLXCDriverGetConfig(driver); if (virDomainSetBlkioParametersEnsureACL(dom->conn, vm->def, flags) < 0) goto cleanup; @@ -1945,8 +2123,8 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, if (!(caps = virLXCDriverGetCapabilities(driver, false))) goto cleanup; - if (virDomainLiveConfigHelperMethod(caps, driver->xmlopt, - vm, &flags, &persistentDef) < 0) + if (virDomainLiveConfigHelperMethod(caps, driver->xmlopt, vm, &flags, + &persistentDef) < 0) goto cleanup; if (flags & VIR_DOMAIN_AFFECT_LIVE) { @@ -1955,7 +2133,10 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, _("blkio cgroup isn't mounted")); goto cleanup; } + } + ret = 0; + if (flags & VIR_DOMAIN_AFFECT_LIVE) { for (i = 0; i < nparams; i++) { virTypedParameterPtr param = ¶ms[i]; @@ -1963,14 +2144,86 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, if (params[i].value.ui > 1000 || params[i].value.ui < 100) { virReportError(VIR_ERR_INVALID_ARG, "%s", _("out of blkio weight range.")); - goto cleanup; + ret = -1; + continue; } if (virCgroupSetBlkioWeight(priv->cgroup, params[i].value.ui) < 0) - goto cleanup; + ret = -1; + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) { + size_t ndevices; + virBlkioDevicePtr devices = NULL; + size_t j; + + if (lxcDomainParseBlkioDeviceStr(params[i].value.s, + param->field, + &devices, + &ndevices) < 0) { + ret = -1; + continue; + } + if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWeight(priv->cgroup, + devices[j].path, + devices[j].weight) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceReadIops(priv->cgroup, + devices[j].path, + devices[j].riops) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWriteIops(priv->cgroup, + devices[j].path, + devices[j].wiops) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceReadBps(priv->cgroup, + devices[j].path, + devices[j].rbps) < 0) { + ret = -1; + break; + } + } + } else { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWriteBps(priv->cgroup, + devices[j].path, + devices[j].wbps) < 0) { + ret = -1; + break; + } + } + } + if (j != ndevices || + lxcDomainMergeBlkioDevice(&vm->def->blkio.devices, + &vm->def->blkio.ndevices, + devices, ndevices, param->field) < 0) + ret = -1; + virBlkioDeviceArrayClear(devices, ndevices); + VIR_FREE(devices); } } } + if (ret < 0) + goto cleanup; if (flags & VIR_DOMAIN_AFFECT_CONFIG) { /* Clang can't see that if we get here, persistentDef was set. */ sa_assert(persistentDef); @@ -1982,18 +2235,39 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, if (params[i].value.ui > 1000 || params[i].value.ui < 100) { virReportError(VIR_ERR_INVALID_ARG, "%s", _("out of blkio weight range.")); - goto cleanup; + ret = -1; + continue; } persistentDef->blkio.weight = params[i].value.ui; + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) { + virBlkioDevicePtr devices = NULL; + size_t ndevices; + + if (lxcDomainParseBlkioDeviceStr(params[i].value.s, + params[i].field, + &devices, + &ndevices) < 0) { + ret = -1; + continue; + } + if (lxcDomainMergeBlkioDevice(&persistentDef->blkio.devices, + &persistentDef->blkio.ndevices, + devices, ndevices, param->field) < 0) + ret = -1; + virBlkioDeviceArrayClear(devices, ndevices); + VIR_FREE(devices); } } if (virDomainSaveConfig(cfg->configDir, persistentDef) < 0) - goto cleanup; + ret = -1; } - ret = 0; cleanup: if (vm) virObjectUnlock(vm); @@ -2003,7 +2277,8 @@ cleanup: } -#define LXC_NB_BLKIO_PARAM 1 +#define LXC_NB_BLKIO_PARAM 6 + static int lxcDomainGetBlkioParameters(virDomainPtr dom, virTypedParameterPtr params, @@ -2011,25 +2286,34 @@ lxcDomainGetBlkioParameters(virDomainPtr dom, unsigned int flags) { virLXCDriverPtr driver = dom->conn->privateData; - virCapsPtr caps = NULL; - size_t i; + size_t i, j; virDomainObjPtr vm = NULL; virDomainDefPtr persistentDef = NULL; unsigned int val; int ret = -1; + virCapsPtr caps = NULL; virLXCDomainObjPrivatePtr priv; virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | - VIR_DOMAIN_AFFECT_CONFIG, -1); + VIR_DOMAIN_AFFECT_CONFIG | + VIR_TYPED_PARAM_STRING_OKAY, -1); + + /* We blindly return a string, and let libvirt.c and + * remote_driver.c do the filtering on behalf of older clients + * that can't parse it. */ + flags &= ~VIR_TYPED_PARAM_STRING_OKAY; if (!(vm = lxcDomObjFromDomain(dom))) - goto cleanup; + return -1; priv = vm->privateData; if (virDomainGetBlkioParametersEnsureACL(dom->conn, vm->def) < 0) goto cleanup; + if (!(caps = virLXCDriverGetCapabilities(driver, false))) + goto cleanup; + if ((*nparams) == 0) { /* Current number of blkio parameters supported by cgroups */ *nparams = LXC_NB_BLKIO_PARAM; @@ -2037,11 +2321,8 @@ lxcDomainGetBlkioParameters(virDomainPtr dom, goto cleanup; } - if (!(caps = virLXCDriverGetCapabilities(driver, false))) - goto cleanup; - - if (virDomainLiveConfigHelperMethod(caps, driver->xmlopt, - vm, &flags, &persistentDef) < 0) + if (virDomainLiveConfigHelperMethod(caps, driver->xmlopt, vm, &flags, + &persistentDef) < 0) goto cleanup; if (flags & VIR_DOMAIN_AFFECT_LIVE) { @@ -2064,7 +2345,151 @@ lxcDomainGetBlkioParameters(virDomainPtr dom, goto cleanup; break; - /* coverity[dead_error_begin] */ + case 1: /* blkiotune.device_weight */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].weight) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].weight); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_WEIGHT, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 2: /* blkiotune.device_read_iops */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].riops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].riops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 3: /* blkiotune.device_write_iops */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].wiops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].wiops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 4: /* blkiotune.device_read_bps */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].rbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].rbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 5: /* blkiotune.device_write_bps */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].wbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].wbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + default: break; /* should not hit here */ @@ -2073,16 +2498,189 @@ lxcDomainGetBlkioParameters(virDomainPtr dom, } else if (flags & VIR_DOMAIN_AFFECT_CONFIG) { for (i = 0; i < *nparams && i < LXC_NB_BLKIO_PARAM; i++) { virTypedParameterPtr param = ¶ms[i]; + val = 0; + param->value.ui = 0; + param->type = VIR_TYPED_PARAM_UINT; switch (i) { case 0: /* fill blkio weight here */ - if (virTypedParameterAssign(param, VIR_DOMAIN_BLKIO_WEIGHT, - VIR_TYPED_PARAM_UINT, - persistentDef->blkio.weight) < 0) + if (virStrcpyStatic(param->field, VIR_DOMAIN_BLKIO_WEIGHT) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_WEIGHT); + goto cleanup; + } + param->value.ui = persistentDef->blkio.weight; + break; + + case 1: /* blkiotune.device_weight */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].weight) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].weight); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_WEIGHT) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_WEIGHT); + goto cleanup; + } + break; + + case 2: /* blkiotune.device_read_iops */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].riops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].riops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS); + goto cleanup; + } + break; + case 3: /* blkiotune.device_write_iops */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].wiops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].wiops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS); + goto cleanup; + } + break; + case 4: /* blkiotune.device_read_bps */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].rbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].rbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS); + goto cleanup; + } + break; + + case 5: /* blkiotune.device_write_bps */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].wbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].wbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS); + goto cleanup; + } break; - /* coverity[dead_error_begin] */ default: break; /* should not hit here */ @@ -4623,6 +5221,7 @@ static virDriver lxcDriver = { .name = LXC_DRIVER_NAME, .connectOpen = lxcConnectOpen, /* 0.4.2 */ .connectClose = lxcConnectClose, /* 0.4.2 */ + .connectSupportsFeature = lxcConnectSupportsFeature, /* 1.1.4 */ .connectGetVersion = lxcConnectGetVersion, /* 0.4.6 */ .connectGetHostname = lxcConnectGetHostname, /* 0.6.3 */ .connectGetSysinfo = lxcConnectGetSysinfo, /* 1.0.5 */ -- 1.8.3.1
On Wed, Dec 11, 2013 at 04:29:53PM +0800, Gao feng wrote:
With this patch,user can set throttle blkio cgroup for lxc domain through virsh tool.
The functions are copied from qemu_driver.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_driver.c | 649 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 624 insertions(+), 25 deletions(-)
ACK
+ if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].weight) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].riops) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].wiops) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) { + if (virStrToLong_ull(temp, &p, 10, &result[i].rbps) < 0) + goto error; + } else { + if (virStrToLong_ull(temp, &p, 10, &result[i].wbps) < 0) + goto error; + }
Check the WRITE_BPS string too and report an error in the else clause
+ + i++; + + if (*p == '\0') + break; + else if (*p != ',') + goto error; + temp = p + 1; + } + + if (!i) + VIR_FREE(result); + + *dev = result; + *size = i; + + return 0; + +error: + virReportError(VIR_ERR_INVALID_ARG, + _("unable to parse device weight '%s'"), blkioDeviceStr); +cleanup: + virBlkioDeviceArrayClear(result, ndevices); + VIR_FREE(result); + return -1; +} + +static int +lxcDomainMergeBlkioDevice(virBlkioDevicePtr *dest_array, + size_t *dest_size, + virBlkioDevicePtr src_array, + size_t src_size, + const char *type) +{ + size_t i, j; + virBlkioDevicePtr dest, src; + + for (i = 0; i < src_size; i++) { + bool found = false; + + src = &src_array[i]; + for (j = 0; j < *dest_size; j++) { + dest = &(*dest_array)[j]; + if (STREQ(src->path, dest->path)) { + found = true; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) + dest->weight = src->weight; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) + dest->riops = src->riops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) + dest->wiops = src->wiops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) + dest->rbps = src->rbps; + else + dest->wbps = src->wbps;
Check the WRITE_BPS string too and report an error in the else clause
+ + break; + } + } + if (!found) { + if (!src->weight && !src->riops && !src->wiops && !src->rbps && !src->wbps) + continue; + if (VIR_EXPAND_N(*dest_array, *dest_size, 1) < 0) + return -1; + dest = &(*dest_array)[*dest_size - 1]; + dest->path = src->path; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) + dest->weight = src->weight; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) + dest->riops = src->riops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) + dest->wiops = src->wiops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) + dest->rbps = src->rbps; + else + dest->wbps = src->wbps;
Check the WRITE_BPS string too and report an error in the else clause Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
With this patch,user can set throttle blkio cgroup for lxc domain through virsh tool. Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_driver.c | 671 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 646 insertions(+), 25 deletions(-) diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index c499182..0e9c011 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -1545,6 +1545,20 @@ static int lxcStateCleanup(void) return 0; } +static int +lxcConnectSupportsFeature(virConnectPtr conn, int feature) +{ + if (virConnectSupportsFeatureEnsureACL(conn) < 0) + return -1; + + switch (feature) { + case VIR_DRV_FEATURE_TYPED_PARAM_STRING: + return 1; + default: + return 0; + } +} + static int lxcConnectGetVersion(virConnectPtr conn, unsigned long *version) { @@ -1910,6 +1924,171 @@ lxcDomainGetSchedulerParameters(virDomainPtr domain, return lxcDomainGetSchedulerParametersFlags(domain, params, nparams, 0); } +static int +lxcDomainParseBlkioDeviceStr(char *blkioDeviceStr, const char *type, + virBlkioDevicePtr *dev, size_t *size) +{ + char *temp; + int ndevices = 0; + int nsep = 0; + size_t i; + virBlkioDevicePtr result = NULL; + + *dev = NULL; + *size = 0; + + if (STREQ(blkioDeviceStr, "")) + return 0; + + temp = blkioDeviceStr; + while (temp) { + temp = strchr(temp, ','); + if (temp) { + temp++; + nsep++; + } + } + + /* A valid string must have even number of fields, hence an odd + * number of commas. */ + if (!(nsep & 1)) + goto error; + + ndevices = (nsep + 1) / 2; + + if (VIR_ALLOC_N(result, ndevices) < 0) + return -1; + + i = 0; + temp = blkioDeviceStr; + while (temp) { + char *p = temp; + + /* device path */ + p = strchr(p, ','); + if (!p) + goto error; + + if (VIR_STRNDUP(result[i].path, temp, p - temp) < 0) + goto cleanup; + + /* value */ + temp = p + 1; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].weight) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].riops) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) { + if (virStrToLong_ui(temp, &p, 10, &result[i].wiops) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) { + if (virStrToLong_ull(temp, &p, 10, &result[i].rbps) < 0) + goto error; + } else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)){ + if (virStrToLong_ull(temp, &p, 10, &result[i].wbps) < 0) + goto error; + } else { + goto error; + } + + i++; + + if (*p == '\0') + break; + else if (*p != ',') + goto error; + temp = p + 1; + } + + if (!i) + VIR_FREE(result); + + *dev = result; + *size = i; + + return 0; + +error: + virReportError(VIR_ERR_INVALID_ARG, + _("unable to parse blkio device '%s' '%s'"), + type, blkioDeviceStr); +cleanup: + virBlkioDeviceArrayClear(result, ndevices); + VIR_FREE(result); + return -1; +} + +static int +lxcDomainMergeBlkioDevice(virBlkioDevicePtr *dest_array, + size_t *dest_size, + virBlkioDevicePtr src_array, + size_t src_size, + const char *type) +{ + size_t i, j; + virBlkioDevicePtr dest, src; + + for (i = 0; i < src_size; i++) { + bool found = false; + + src = &src_array[i]; + for (j = 0; j < *dest_size; j++) { + dest = &(*dest_array)[j]; + if (STREQ(src->path, dest->path)) { + found = true; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) + dest->weight = src->weight; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) + dest->riops = src->riops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) + dest->wiops = src->wiops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) + dest->rbps = src->rbps; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) + dest->wbps = src->wbps; + else { + virReportError(VIR_ERR_INVALID_ARG, _("Unknown parameter %s"), + type); + return -1; + } + + break; + } + } + if (!found) { + if (!src->weight && !src->riops && !src->wiops && !src->rbps && !src->wbps) + continue; + if (VIR_EXPAND_N(*dest_array, *dest_size, 1) < 0) + return -1; + dest = &(*dest_array)[*dest_size - 1]; + + if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) + dest->weight = src->weight; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) + dest->riops = src->riops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) + dest->wiops = src->wiops; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) + dest->rbps = src->rbps; + else if (STREQ(type, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) + dest->wbps = src->wbps; + else { + *dest_size = *dest_size - 1; + return -1; + } + + dest->path = src->path; + src->path = NULL; + } + } + + return 0; +} + static int lxcDomainSetBlkioParameters(virDomainPtr dom, @@ -1918,26 +2097,37 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, unsigned int flags) { virLXCDriverPtr driver = dom->conn->privateData; - virCapsPtr caps = NULL; size_t i; virDomainObjPtr vm = NULL; virDomainDefPtr persistentDef = NULL; int ret = -1; + virLXCDriverConfigPtr cfg = NULL; + virCapsPtr caps = NULL; virLXCDomainObjPrivatePtr priv; - virLXCDriverConfigPtr cfg = virLXCDriverGetConfig(driver); virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | VIR_DOMAIN_AFFECT_CONFIG, -1); if (virTypedParamsValidate(params, nparams, VIR_DOMAIN_BLKIO_WEIGHT, VIR_TYPED_PARAM_UINT, + VIR_DOMAIN_BLKIO_DEVICE_WEIGHT, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS, + VIR_TYPED_PARAM_STRING, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS, + VIR_TYPED_PARAM_STRING, NULL) < 0) return -1; if (!(vm = lxcDomObjFromDomain(dom))) - goto cleanup; + return -1; priv = vm->privateData; + cfg = virLXCDriverGetConfig(driver); if (virDomainSetBlkioParametersEnsureACL(dom->conn, vm->def, flags) < 0) goto cleanup; @@ -1945,8 +2135,8 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, if (!(caps = virLXCDriverGetCapabilities(driver, false))) goto cleanup; - if (virDomainLiveConfigHelperMethod(caps, driver->xmlopt, - vm, &flags, &persistentDef) < 0) + if (virDomainLiveConfigHelperMethod(caps, driver->xmlopt, vm, &flags, + &persistentDef) < 0) goto cleanup; if (flags & VIR_DOMAIN_AFFECT_LIVE) { @@ -1955,7 +2145,10 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, _("blkio cgroup isn't mounted")); goto cleanup; } + } + ret = 0; + if (flags & VIR_DOMAIN_AFFECT_LIVE) { for (i = 0; i < nparams; i++) { virTypedParameterPtr param = ¶ms[i]; @@ -1963,14 +2156,96 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, if (params[i].value.ui > 1000 || params[i].value.ui < 100) { virReportError(VIR_ERR_INVALID_ARG, "%s", _("out of blkio weight range.")); - goto cleanup; + ret = -1; + continue; } if (virCgroupSetBlkioWeight(priv->cgroup, params[i].value.ui) < 0) - goto cleanup; + ret = -1; + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) { + size_t ndevices; + virBlkioDevicePtr devices = NULL; + size_t j; + + if (lxcDomainParseBlkioDeviceStr(params[i].value.s, + param->field, + &devices, + &ndevices) < 0) { + ret = -1; + continue; + } + + if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWeight(priv->cgroup, + devices[j].path, + devices[j].weight) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceReadIops(priv->cgroup, + devices[j].path, + devices[j].riops) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWriteIops(priv->cgroup, + devices[j].path, + devices[j].wiops) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS)) { + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceReadBps(priv->cgroup, + devices[j].path, + devices[j].rbps) < 0) { + ret = -1; + break; + } + } + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)){ + for (j = 0; j < ndevices; j++) { + if (virCgroupSetBlkioDeviceWriteBps(priv->cgroup, + devices[j].path, + devices[j].wbps) < 0) { + ret = -1; + break; + } + } + } else { + virReportError(VIR_ERR_INVALID_ARG, _("Unknown blkio parameter %s"), + param->field); + ret = -1; + virBlkioDeviceArrayClear(devices, ndevices); + VIR_FREE(devices); + + continue; + } + + if (j != ndevices || + lxcDomainMergeBlkioDevice(&vm->def->blkio.devices, + &vm->def->blkio.ndevices, + devices, ndevices, param->field) < 0) + ret = -1; + virBlkioDeviceArrayClear(devices, ndevices); + VIR_FREE(devices); } } } + if (ret < 0) + goto cleanup; if (flags & VIR_DOMAIN_AFFECT_CONFIG) { /* Clang can't see that if we get here, persistentDef was set. */ sa_assert(persistentDef); @@ -1982,18 +2257,39 @@ lxcDomainSetBlkioParameters(virDomainPtr dom, if (params[i].value.ui > 1000 || params[i].value.ui < 100) { virReportError(VIR_ERR_INVALID_ARG, "%s", _("out of blkio weight range.")); - goto cleanup; + ret = -1; + continue; } persistentDef->blkio.weight = params[i].value.ui; + } else if (STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WEIGHT) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) || + STREQ(param->field, VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS)) { + virBlkioDevicePtr devices = NULL; + size_t ndevices; + + if (lxcDomainParseBlkioDeviceStr(params[i].value.s, + param->field, + &devices, + &ndevices) < 0) { + ret = -1; + continue; + } + if (lxcDomainMergeBlkioDevice(&persistentDef->blkio.devices, + &persistentDef->blkio.ndevices, + devices, ndevices, param->field) < 0) + ret = -1; + virBlkioDeviceArrayClear(devices, ndevices); + VIR_FREE(devices); } } if (virDomainSaveConfig(cfg->configDir, persistentDef) < 0) - goto cleanup; + ret = -1; } - ret = 0; cleanup: if (vm) virObjectUnlock(vm); @@ -2003,7 +2299,8 @@ cleanup: } -#define LXC_NB_BLKIO_PARAM 1 +#define LXC_NB_BLKIO_PARAM 6 + static int lxcDomainGetBlkioParameters(virDomainPtr dom, virTypedParameterPtr params, @@ -2011,25 +2308,34 @@ lxcDomainGetBlkioParameters(virDomainPtr dom, unsigned int flags) { virLXCDriverPtr driver = dom->conn->privateData; - virCapsPtr caps = NULL; - size_t i; + size_t i, j; virDomainObjPtr vm = NULL; virDomainDefPtr persistentDef = NULL; unsigned int val; int ret = -1; + virCapsPtr caps = NULL; virLXCDomainObjPrivatePtr priv; virCheckFlags(VIR_DOMAIN_AFFECT_LIVE | - VIR_DOMAIN_AFFECT_CONFIG, -1); + VIR_DOMAIN_AFFECT_CONFIG | + VIR_TYPED_PARAM_STRING_OKAY, -1); + + /* We blindly return a string, and let libvirt.c and + * remote_driver.c do the filtering on behalf of older clients + * that can't parse it. */ + flags &= ~VIR_TYPED_PARAM_STRING_OKAY; if (!(vm = lxcDomObjFromDomain(dom))) - goto cleanup; + return -1; priv = vm->privateData; if (virDomainGetBlkioParametersEnsureACL(dom->conn, vm->def) < 0) goto cleanup; + if (!(caps = virLXCDriverGetCapabilities(driver, false))) + goto cleanup; + if ((*nparams) == 0) { /* Current number of blkio parameters supported by cgroups */ *nparams = LXC_NB_BLKIO_PARAM; @@ -2037,11 +2343,8 @@ lxcDomainGetBlkioParameters(virDomainPtr dom, goto cleanup; } - if (!(caps = virLXCDriverGetCapabilities(driver, false))) - goto cleanup; - - if (virDomainLiveConfigHelperMethod(caps, driver->xmlopt, - vm, &flags, &persistentDef) < 0) + if (virDomainLiveConfigHelperMethod(caps, driver->xmlopt, vm, &flags, + &persistentDef) < 0) goto cleanup; if (flags & VIR_DOMAIN_AFFECT_LIVE) { @@ -2064,7 +2367,151 @@ lxcDomainGetBlkioParameters(virDomainPtr dom, goto cleanup; break; - /* coverity[dead_error_begin] */ + case 1: /* blkiotune.device_weight */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].weight) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].weight); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_WEIGHT, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 2: /* blkiotune.device_read_iops */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].riops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].riops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 3: /* blkiotune.device_write_iops */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].wiops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].wiops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 4: /* blkiotune.device_read_bps */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].rbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].rbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + + case 5: /* blkiotune.device_write_bps */ + if (vm->def->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < vm->def->blkio.ndevices; j++) { + if (!vm->def->blkio.devices[j].wbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + vm->def->blkio.devices[j].path, + vm->def->blkio.devices[j].wbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (virTypedParameterAssign(param, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS, + VIR_TYPED_PARAM_STRING, + param->value.s) < 0) + goto cleanup; + break; + default: break; /* should not hit here */ @@ -2073,16 +2520,189 @@ lxcDomainGetBlkioParameters(virDomainPtr dom, } else if (flags & VIR_DOMAIN_AFFECT_CONFIG) { for (i = 0; i < *nparams && i < LXC_NB_BLKIO_PARAM; i++) { virTypedParameterPtr param = ¶ms[i]; + val = 0; + param->value.ui = 0; + param->type = VIR_TYPED_PARAM_UINT; switch (i) { case 0: /* fill blkio weight here */ - if (virTypedParameterAssign(param, VIR_DOMAIN_BLKIO_WEIGHT, - VIR_TYPED_PARAM_UINT, - persistentDef->blkio.weight) < 0) + if (virStrcpyStatic(param->field, VIR_DOMAIN_BLKIO_WEIGHT) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_WEIGHT); + goto cleanup; + } + param->value.ui = persistentDef->blkio.weight; + break; + + case 1: /* blkiotune.device_weight */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].weight) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].weight); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_WEIGHT) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_WEIGHT); + goto cleanup; + } + break; + + case 2: /* blkiotune.device_read_iops */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].riops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].riops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_READ_IOPS); + goto cleanup; + } + break; + case 3: /* blkiotune.device_write_iops */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].wiops) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%u", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].wiops); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_WRITE_IOPS); goto cleanup; + } + break; + case 4: /* blkiotune.device_read_bps */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].rbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].rbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_READ_BPS); + goto cleanup; + } + break; + + case 5: /* blkiotune.device_write_bps */ + if (persistentDef->blkio.ndevices > 0) { + virBuffer buf = VIR_BUFFER_INITIALIZER; + bool comma = false; + + for (j = 0; j < persistentDef->blkio.ndevices; j++) { + if (!persistentDef->blkio.devices[j].wbps) + continue; + if (comma) + virBufferAddChar(&buf, ','); + else + comma = true; + virBufferAsprintf(&buf, "%s,%llu", + persistentDef->blkio.devices[j].path, + persistentDef->blkio.devices[j].wbps); + } + if (virBufferError(&buf)) { + virReportOOMError(); + goto cleanup; + } + param->value.s = virBufferContentAndReset(&buf); + } + if (!param->value.s && VIR_STRDUP(param->value.s, "") < 0) + goto cleanup; + param->type = VIR_TYPED_PARAM_STRING; + if (virStrcpyStatic(param->field, + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS) == NULL) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Field name '%s' too long"), + VIR_DOMAIN_BLKIO_DEVICE_WRITE_BPS); + goto cleanup; + } break; - /* coverity[dead_error_begin] */ default: break; /* should not hit here */ @@ -4623,6 +5243,7 @@ static virDriver lxcDriver = { .name = LXC_DRIVER_NAME, .connectOpen = lxcConnectOpen, /* 0.4.2 */ .connectClose = lxcConnectClose, /* 0.4.2 */ + .connectSupportsFeature = lxcConnectSupportsFeature, /* 1.1.4 */ .connectGetVersion = lxcConnectGetVersion, /* 0.4.6 */ .connectGetHostname = lxcConnectGetHostname, /* 0.6.3 */ .connectGetSysinfo = lxcConnectGetSysinfo, /* 1.0.5 */ -- 1.8.3.1
On Fri, Dec 13, 2013 at 11:09:01AM +0800, Gao feng wrote:
With this patch,user can set throttle blkio cgroup for lxc domain through virsh tool.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_driver.c | 671 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 646 insertions(+), 25 deletions(-)
ACK
@@ -4623,6 +5243,7 @@ static virDriver lxcDriver = { .name = LXC_DRIVER_NAME, .connectOpen = lxcConnectOpen, /* 0.4.2 */ .connectClose = lxcConnectClose, /* 0.4.2 */ + .connectSupportsFeature = lxcConnectSupportsFeature, /* 1.1.4 */
But change this to 1.2.1 Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 12/20/2013 12:34 AM, Daniel P. Berrange wrote:
On Fri, Dec 13, 2013 at 11:09:01AM +0800, Gao feng wrote:
With this patch,user can set throttle blkio cgroup for lxc domain through virsh tool.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_driver.c | 671 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 646 insertions(+), 25 deletions(-)
ACK
@@ -4623,6 +5243,7 @@ static virDriver lxcDriver = { .name = LXC_DRIVER_NAME, .connectOpen = lxcConnectOpen, /* 0.4.2 */ .connectClose = lxcConnectClose, /* 0.4.2 */ + .connectSupportsFeature = lxcConnectSupportsFeature, /* 1.1.4 */
But change this to 1.2.1
So this should be 1.2.2 now, right? Thanks Gao
On Mon, Jan 20, 2014 at 11:25:56AM +0800, Gao feng wrote:
On 12/20/2013 12:34 AM, Daniel P. Berrange wrote:
On Fri, Dec 13, 2013 at 11:09:01AM +0800, Gao feng wrote:
With this patch,user can set throttle blkio cgroup for lxc domain through virsh tool.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_driver.c | 671 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 646 insertions(+), 25 deletions(-)
ACK
@@ -4623,6 +5243,7 @@ static virDriver lxcDriver = { .name = LXC_DRIVER_NAME, .connectOpen = lxcConnectOpen, /* 0.4.2 */ .connectClose = lxcConnectClose, /* 0.4.2 */ + .connectSupportsFeature = lxcConnectSupportsFeature, /* 1.1.4 */
But change this to 1.2.1
So this should be 1.2.2 now, right?
Yes, correct. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 01/20/2014 07:46 PM, Daniel P. Berrange wrote:
On Mon, Jan 20, 2014 at 11:25:56AM +0800, Gao feng wrote:
On 12/20/2013 12:34 AM, Daniel P. Berrange wrote:
On Fri, Dec 13, 2013 at 11:09:01AM +0800, Gao feng wrote:
With this patch,user can set throttle blkio cgroup for lxc domain through virsh tool.
Signed-off-by: Guan Qiang <hzguanqiang@corp.netease.com> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_driver.c | 671 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 646 insertions(+), 25 deletions(-)
ACK
@@ -4623,6 +5243,7 @@ static virDriver lxcDriver = { .name = LXC_DRIVER_NAME, .connectOpen = lxcConnectOpen, /* 0.4.2 */ .connectClose = lxcConnectClose, /* 0.4.2 */ + .connectSupportsFeature = lxcConnectSupportsFeature, /* 1.1.4 */
But change this to 1.2.1
So this should be 1.2.2 now, right?
Yes, correct.
Thanks! change and pushed.
participants (3)
-
Daniel P. Berrange -
Eric Blake -
Gao feng