[libvirt] [RFC PATCH 0/8 v3] Summary on block IO throttle
Changes since V2 - Implement the Python binding support for setting blkio throttling. - Implement --current --live --config options support to unify the libvirt API. - Add changes in docs and tests. - Some changes suggested by Adam Litke, Eric Blake, Daniel P. Berrange. - Change the XML schema. - API name to virDomain{Set, Get}BlockIoTune. - Parameters changed to make them more self-explanatory. - virsh command name to blkdeviotune. - And other fixups. Changes since V1 - Implement the support to get the block io throttling for a device as read only connection - QMP/HMP. - Split virDomainBlockIoThrottle into two separate functions virDomainSetBlockIoThrottle - Set block I/O limits for a device - Requires a connection in 'write' mode. - Limits (info) structure passed as an input parameter virDomainGetBlockIoThrottle - Get the current block I/O limits for a device - Works on a read-only connection. - Current limits are written to the output parameter (reply). - And Other fixups suggested by Adam Litke, Daniel P. Berrange. - For dynamically allocate the blkiothrottle struct, I will fix it when implement --current --live --config options support. Today libvirt supports the cgroups blkio-controller, which handles proportional shares and throughput/iops limits on host block devices. blkio-controller does not support network file systems (NFS) or other QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are not host block devices. QEMU I/O throttling works with all types of drive and can be applied independently to each drive attached to a guest and supports throughput/iops limits. To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings. Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream just now and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong: http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block 1) Enable the blkio throttling in xml when guest is starting up. Add blkio throttling in xml as follows: <disk type='file' device='disk'> ... <iotune> <total_bytes_sec>nnn</total_bytes_sec> ... </iotune> ... </disk> 2) Enable blkio throttling setting at guest running time. virsh blkdeviotune <domain> <device> [--total_bytes_sec<number>] [--read_bytes_sec<number>] \ [--write_bytes_sec<number>] [--total_iops_sec<number>] [--read_iops_sec<number>] [--write_iops_sec<number>] 3) The support to get the current block i/o throttling for a device - HMP/QMP. virsh blkiothrottle <domain> <device> total_bytes_sec: read_bytes_sec: write_bytes_sec: total_iops_sec: read_iops_sec: write_iops_sec: 4) Python binding support for setting blkio throttling. 5) --current --live --config options support to unify the libvirt API. virsh blkdeviotune <domain> <device> [--total_bytes_sec <number>] [--read_bytes_sec <number>] [--write_bytes_sec <number>] [--total_iops_sec <number>] [--read_iops_sec <number>] [--write_iops_sec <number>] [--config] [--live] [--current] daemon/remote.c | 87 +++++++ docs/formatdomain.html.in | 30 ++ docs/schemas/domaincommon.rng | 24 + include/libvirt/libvirt.h.in | 25 ++ python/generator.py | 2 python/libvirt-override-api.xml | 16 + python/libvirt-override.c | 85 +++++++ src/conf/domain_conf.c | 101 ++++++++ src/conf/domain_conf.h | 12 src/driver.h | 18 + src/libvirt.c | 115 +++++++++ src/libvirt_public.syms | 2 src/qemu/qemu_command.c | 33 ++ src/qemu/qemu_driver.c | 217 ++++++++++++++++++ src/qemu/qemu_monitor.c | 36 ++ src/qemu/qemu_monitor.h | 10 src/qemu/qemu_monitor_json.c | 191 +++++++++++++++ src/qemu/qemu_monitor_json.h | 10 src/qemu/qemu_monitor_text.c | 152 ++++++++++++ src/qemu/qemu_monitor_text.h | 10 src/remote/remote_driver.c | 81 ++++++ src/remote/remote_protocol.x | 39 +++ src/remote_protocol-structs | 34 ++ src/util/xml.h | 3 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.args | 4 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.xml | 36 ++ tests/qemuxml2argvtest.c | 2 tests/qemuxml2xmltest.c | 2 tools/virsh.c | 146 ++++++++++++ tools/virsh.pod | 23 + 30 files changed, 1540 insertions(+), 6 deletions(-) -- Lei
On Thu, Nov 10, 2011 at 4:32 AM, Lei Li <lilei@linux.vnet.ibm.com> wrote:
Changes since V2 - Implement the Python binding support for setting blkio throttling. - Implement --current --live --config options support to unify the libvirt API. - Add changes in docs and tests. - Some changes suggested by Adam Litke, Eric Blake, Daniel P. Berrange. - Change the XML schema. - API name to virDomain{Set, Get}BlockIoTune. - Parameters changed to make them more self-explanatory. - virsh command name to blkdeviotune. - And other fixups.
Changes since V1 - Implement the support to get the block io throttling for a device as read only connection - QMP/HMP. - Split virDomainBlockIoThrottle into two separate functions virDomainSetBlockIoThrottle - Set block I/O limits for a device - Requires a connection in 'write' mode. - Limits (info) structure passed as an input parameter virDomainGetBlockIoThrottle - Get the current block I/O limits for a device - Works on a read-only connection. - Current limits are written to the output parameter (reply). - And Other fixups suggested by Adam Litke, Daniel P. Berrange. - For dynamically allocate the blkiothrottle struct, I will fix it when implement --current --live --config options support.
Today libvirt supports the cgroups blkio-controller, which handles proportional shares and throughput/iops limits on host block devices. blkio-controller does not support network file systems (NFS) or other QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are not host block devices. QEMU I/O throttling works with all types of drive and can be applied independently to each drive attached to a guest and supports throughput/iops limits.
To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings.
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream just now and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
1) Enable the blkio throttling in xml when guest is starting up.
Add blkio throttling in xml as follows:
<disk type='file' device='disk'> ... <iotune> <total_bytes_sec>nnn</total_bytes_sec> ... </iotune> ... </disk>
2) Enable blkio throttling setting at guest running time.
virsh blkdeviotune <domain> <device> [--total_bytes_sec<number>] [--read_bytes_sec<number>] \ [--write_bytes_sec<number>] [--total_iops_sec<number>] [--read_iops_sec<number>] [--write_iops_sec<number>]
3) The support to get the current block i/o throttling for a device - HMP/QMP.
virsh blkiothrottle <domain> <device> total_bytes_sec: read_bytes_sec: write_bytes_sec: total_iops_sec: read_iops_sec: write_iops_sec:
4) Python binding support for setting blkio throttling. 5) --current --live --config options support to unify the libvirt API.
virsh blkdeviotune <domain> <device> [--total_bytes_sec <number>] [--read_bytes_sec <number>] [--write_bytes_sec <number>] [--total_iops_sec <number>] [--read_iops_sec <number>] [--write_iops_sec <number>] [--config] [--live] [--current] Thanks Li Lei for the remaining works. Below is only one reminder. QEMU command line options have some limitations as below:
(1) global bps limit. -drive bps=xxx in bytes/s (2) only read bps limit -drive bps_rd=xxx in bytes/s (3) only write bps limit -drive bps_wr=xxx in bytes/s (4) global iops limit -drive iops=xxx in ios/s (5) only read iops limit -drive iops_rd=xxx in ios/s (6) only write iops limit -drive iops_wr=xxx in ios/s (7) the combination of some limits. -drive bps=xxx,iops=xxx Known Limitations: (1) #1 can not be used with #2, #3 together (2) #4 can not be used with #5, #6 together
daemon/remote.c | 87 +++++++ docs/formatdomain.html.in | 30 ++ docs/schemas/domaincommon.rng | 24 + include/libvirt/libvirt.h.in | 25 ++ python/generator.py | 2 python/libvirt-override-api.xml | 16 + python/libvirt-override.c | 85 +++++++ src/conf/domain_conf.c | 101 ++++++++ src/conf/domain_conf.h | 12 src/driver.h | 18 + src/libvirt.c | 115 +++++++++ src/libvirt_public.syms | 2 src/qemu/qemu_command.c | 33 ++ src/qemu/qemu_driver.c | 217 ++++++++++++++++++ src/qemu/qemu_monitor.c | 36 ++ src/qemu/qemu_monitor.h | 10 src/qemu/qemu_monitor_json.c | 191 +++++++++++++++ src/qemu/qemu_monitor_json.h | 10 src/qemu/qemu_monitor_text.c | 152 ++++++++++++ src/qemu/qemu_monitor_text.h | 10 src/remote/remote_driver.c | 81 ++++++ src/remote/remote_protocol.x | 39 +++ src/remote_protocol-structs | 34 ++ src/util/xml.h | 3 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.args | 4 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.xml | 36 ++ tests/qemuxml2argvtest.c | 2 tests/qemuxml2xmltest.c | 2 tools/virsh.c | 146 ++++++++++++ tools/virsh.pod | 23 + 30 files changed, 1540 insertions(+), 6 deletions(-)
-- Lei
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- Regards, Zhi Yong Wu
On 11/10/2011 12:45 PM, Zhi Yong Wu wrote:
On Thu, Nov 10, 2011 at 4:32 AM, Lei Li<lilei@linux.vnet.ibm.com> wrote:
Changes since V2 - Implement the Python binding support for setting blkio throttling. - Implement --current --live --config options support to unify the libvirt API. - Add changes in docs and tests. - Some changes suggested by Adam Litke, Eric Blake, Daniel P. Berrange. - Change the XML schema. - API name to virDomain{Set, Get}BlockIoTune. - Parameters changed to make them more self-explanatory. - virsh command name to blkdeviotune. - And other fixups.
Changes since V1 - Implement the support to get the block io throttling for a device as read only connection - QMP/HMP. - Split virDomainBlockIoThrottle into two separate functions virDomainSetBlockIoThrottle - Set block I/O limits for a device - Requires a connection in 'write' mode. - Limits (info) structure passed as an input parameter virDomainGetBlockIoThrottle - Get the current block I/O limits for a device - Works on a read-only connection. - Current limits are written to the output parameter (reply). - And Other fixups suggested by Adam Litke, Daniel P. Berrange. - For dynamically allocate the blkiothrottle struct, I will fix it when implement --current --live --config options support.
Today libvirt supports the cgroups blkio-controller, which handles proportional shares and throughput/iops limits on host block devices. blkio-controller does not support network file systems (NFS) or other QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are not host block devices. QEMU I/O throttling works with all types of drive and can be applied independently to each drive attached to a guest and supports throughput/iops limits.
To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings.
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream just now and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
1) Enable the blkio throttling in xml when guest is starting up.
Add blkio throttling in xml as follows:
<disk type='file' device='disk'> ... <iotune> <total_bytes_sec>nnn</total_bytes_sec> ... </iotune> ... </disk>
2) Enable blkio throttling setting at guest running time.
virsh blkdeviotune<domain> <device> [--total_bytes_sec<number>] [--read_bytes_sec<number>] \ [--write_bytes_sec<number>] [--total_iops_sec<number>] [--read_iops_sec<number>] [--write_iops_sec<number>]
3) The support to get the current block i/o throttling for a device - HMP/QMP.
virsh blkiothrottle<domain> <device> total_bytes_sec: read_bytes_sec: write_bytes_sec: total_iops_sec: read_iops_sec: write_iops_sec:
4) Python binding support for setting blkio throttling. 5) --current --live --config options support to unify the libvirt API.
virsh blkdeviotune<domain> <device> [--total_bytes_sec<number>] [--read_bytes_sec<number>] [--write_bytes_sec<number>] [--total_iops_sec<number>] [--read_iops_sec<number>] [--write_iops_sec<number>] [--config] [--live] [--current] Thanks Li Lei for the remaining works. Below is only one reminder. QEMU command line options have some limitations as below:
(1) global bps limit. -drive bps=xxx in bytes/s (2) only read bps limit -drive bps_rd=xxx in bytes/s (3) only write bps limit -drive bps_wr=xxx in bytes/s (4) global iops limit -drive iops=xxx in ios/s (5) only read iops limit -drive iops_rd=xxx in ios/s (6) only write iops limit -drive iops_wr=xxx in ios/s (7) the combination of some limits. -drive bps=xxx,iops=xxx
Known Limitations: (1) #1 can not be used with #2, #3 together (2) #4 can not be used with #5, #6 together
Yes, you might want to look into the code about domain_conf.c and qemu_driver.c part, I have already do the limitations on both XML and the command.
daemon/remote.c | 87 +++++++ docs/formatdomain.html.in | 30 ++ docs/schemas/domaincommon.rng | 24 + include/libvirt/libvirt.h.in | 25 ++ python/generator.py | 2 python/libvirt-override-api.xml | 16 + python/libvirt-override.c | 85 +++++++ src/conf/domain_conf.c | 101 ++++++++ src/conf/domain_conf.h | 12 src/driver.h | 18 + src/libvirt.c | 115 +++++++++ src/libvirt_public.syms | 2 src/qemu/qemu_command.c | 33 ++ src/qemu/qemu_driver.c | 217 ++++++++++++++++++ src/qemu/qemu_monitor.c | 36 ++ src/qemu/qemu_monitor.h | 10 src/qemu/qemu_monitor_json.c | 191 +++++++++++++++ src/qemu/qemu_monitor_json.h | 10 src/qemu/qemu_monitor_text.c | 152 ++++++++++++ src/qemu/qemu_monitor_text.h | 10 src/remote/remote_driver.c | 81 ++++++ src/remote/remote_protocol.x | 39 +++ src/remote_protocol-structs | 34 ++ src/util/xml.h | 3 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.args | 4 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.xml | 36 ++ tests/qemuxml2argvtest.c | 2 tests/qemuxml2xmltest.c | 2 tools/virsh.c | 146 ++++++++++++ tools/virsh.pod | 23 + 30 files changed, 1540 insertions(+), 6 deletions(-)
-- Lei
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- Lei
Hi Lei, I've tested this and found that it works well except for one small thing. When the domain is running and I change one of the throttle settings using the --config option in virsh, the on-disk domain xml file is properly updated, but the command 'virsh dumpxml' returns the old version. Is this how it is supposed to work? Otherwise, it seems to work as advertised (based on my sniff testing). Tested-by: Adam Litke <agl@us.ibm.com> On Thu, Nov 10, 2011 at 04:32:50AM +0800, Lei HH Li wrote:
Changes since V2 - Implement the Python binding support for setting blkio throttling. - Implement --current --live --config options support to unify the libvirt API. - Add changes in docs and tests. - Some changes suggested by Adam Litke, Eric Blake, Daniel P. Berrange. - Change the XML schema. - API name to virDomain{Set, Get}BlockIoTune. - Parameters changed to make them more self-explanatory. - virsh command name to blkdeviotune. - And other fixups.
Changes since V1 - Implement the support to get the block io throttling for a device as read only connection - QMP/HMP. - Split virDomainBlockIoThrottle into two separate functions virDomainSetBlockIoThrottle - Set block I/O limits for a device - Requires a connection in 'write' mode. - Limits (info) structure passed as an input parameter virDomainGetBlockIoThrottle - Get the current block I/O limits for a device - Works on a read-only connection. - Current limits are written to the output parameter (reply). - And Other fixups suggested by Adam Litke, Daniel P. Berrange. - For dynamically allocate the blkiothrottle struct, I will fix it when implement --current --live --config options support.
Today libvirt supports the cgroups blkio-controller, which handles proportional shares and throughput/iops limits on host block devices. blkio-controller does not support network file systems (NFS) or other QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are not host block devices. QEMU I/O throttling works with all types of drive and can be applied independently to each drive attached to a guest and supports throughput/iops limits.
To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings.
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream just now and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
1) Enable the blkio throttling in xml when guest is starting up.
Add blkio throttling in xml as follows:
<disk type='file' device='disk'> ... <iotune> <total_bytes_sec>nnn</total_bytes_sec> ... </iotune> ... </disk>
2) Enable blkio throttling setting at guest running time.
virsh blkdeviotune <domain> <device> [--total_bytes_sec<number>] [--read_bytes_sec<number>] \ [--write_bytes_sec<number>] [--total_iops_sec<number>] [--read_iops_sec<number>] [--write_iops_sec<number>]
3) The support to get the current block i/o throttling for a device - HMP/QMP.
virsh blkiothrottle <domain> <device> total_bytes_sec: read_bytes_sec: write_bytes_sec: total_iops_sec: read_iops_sec: write_iops_sec:
4) Python binding support for setting blkio throttling. 5) --current --live --config options support to unify the libvirt API.
virsh blkdeviotune <domain> <device> [--total_bytes_sec <number>] [--read_bytes_sec <number>] [--write_bytes_sec <number>] [--total_iops_sec <number>] [--read_iops_sec <number>] [--write_iops_sec <number>] [--config] [--live] [--current]
daemon/remote.c | 87 +++++++ docs/formatdomain.html.in | 30 ++ docs/schemas/domaincommon.rng | 24 + include/libvirt/libvirt.h.in | 25 ++ python/generator.py | 2 python/libvirt-override-api.xml | 16 + python/libvirt-override.c | 85 +++++++ src/conf/domain_conf.c | 101 ++++++++ src/conf/domain_conf.h | 12 src/driver.h | 18 + src/libvirt.c | 115 +++++++++ src/libvirt_public.syms | 2 src/qemu/qemu_command.c | 33 ++ src/qemu/qemu_driver.c | 217 ++++++++++++++++++ src/qemu/qemu_monitor.c | 36 ++ src/qemu/qemu_monitor.h | 10 src/qemu/qemu_monitor_json.c | 191 +++++++++++++++ src/qemu/qemu_monitor_json.h | 10 src/qemu/qemu_monitor_text.c | 152 ++++++++++++ src/qemu/qemu_monitor_text.h | 10 src/remote/remote_driver.c | 81 ++++++ src/remote/remote_protocol.x | 39 +++ src/remote_protocol-structs | 34 ++ src/util/xml.h | 3 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.args | 4 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.xml | 36 ++ tests/qemuxml2argvtest.c | 2 tests/qemuxml2xmltest.c | 2 tools/virsh.c | 146 ++++++++++++ tools/virsh.pod | 23 + 30 files changed, 1540 insertions(+), 6 deletions(-)
-- Lei
-- Adam Litke <agl@us.ibm.com> IBM Linux Technology Center
On 11/10/2011 02:50 PM, Adam Litke wrote:
Hi Lei,
I've tested this and found that it works well except for one small thing.
When the domain is running and I change one of the throttle settings using the --config option in virsh, the on-disk domain xml file is properly updated, but the command 'virsh dumpxml' returns the old version. Is this how it is supposed to work?
Yes: virsh blkdeviotune --config => on-disk only (takes effect next boot; fails if transient) virsh blkdeviotune --live => running only (takes effect now, and lost on next boot; fails if offline persistent) virsh blkdeviotune --config --live => on-disk and current (takes effect now and persists to next boot; fails if not running persistent) virsh blkdeviotune --current => either --config or --live Also, in thinking about this, I can't help wonder if for extensibility of ALL the virTypedParameter commands, we might be better off with exposing a long-hand option for raw control of virTypedParameter, so that an older virsh can still set options in a newer server without having to recompile virsh to add a new named option to match every new hypervisor named option. Something like: virsh blkdeviotune --set-long total_bytes_sec=number as long-hand for: virsh blkdeviotune --total_bytes_sec number We still need a v4 respinning this to use virTypedParameter; I can help by posting a patch 1/8 that shows the changes that I think we need in the public interface, but I don't want to spend the time rebasing the entire series. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On 11/10/2011 03:04 PM, Eric Blake wrote:
We still need a v4 respinning this to use virTypedParameter; I can help by posting a patch 1/8 that shows the changes that I think we need in the public interface, but I don't want to spend the time rebasing the entire series.
This is more along the lines of what I'm looking for in patch 1/8 (as a diff against your v3 attempt at 1/8). Obviously, it doesn't compile as is, so much as focus on the public interface aspect: diff --git i/include/libvirt/libvirt.h.in w/include/libvirt/libvirt.h.in index 4b05869..52ae0ab 100644 --- i/include/libvirt/libvirt.h.in +++ w/include/libvirt/libvirt.h.in @@ -1684,26 +1684,27 @@ int virDomainBlockPull(virDomainPtr dom, const char *path, * Block I/O throttling support */ -typedef struct _virDomainBlockIoTuneInfo virDomainBlockIoTuneInfo; -struct _virDomainBlockIoTuneInfo { - unsigned long long total_bytes_sec; - unsigned long long read_bytes_sec; - unsigned long long write_bytes_sec; - unsigned long long total_iops_sec; - unsigned long long read_iops_sec; - unsigned long long write_iops_sec; -}; -typedef virDomainBlockIoTuneInfo *virDomainBlockIoTuneInfoPtr; +/** + * VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC: + * + * Macro for the BlockIoTune tunable weight: it represents the total + * bytes per second permitted through a block device, as a ullong. + */ +#define VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC "total_bytes_sec" + +/* ... and so forth - 6 macros total */ int virDomainSetBlockIoTune(virDomainPtr dom, const char *disk, - virDomainBlockIoTuneInfoPtr info, + virTypedParameterPtr params, + int nparams, unsigned int flags); int virDomainGetBlockIoTune(virDomainPtr dom, const char *disk, - virDomainBlockIoTuneInfoPtr reply, + virTypedParameterPtr params, + int *nparams, unsigned int flags); diff --git i/src/libvirt.c w/src/libvirt.c index cd5931f..287dfc5 100644 --- i/src/libvirt.c +++ w/src/libvirt.c @@ -17184,17 +17184,23 @@ error: * virDomainSetBlockIoTune: * @dom: pointer to domain object * @disk: Fully-qualified disk name - * @info: Specify block I/O limits in bytes + * @params: pointer to blkio parameter objects + * @nparams: number of blkio parameters (this value can be the same or + * less than the number of parameters supported) * @flags: An OR'ed set of virDomainModificationImpact * - * This function is mainly to enable Block I/O throttling function in libvirt. - * It is used to change the block I/O throttling setting for specified domain. + * Change all or a subset of the per-device block IO tunables. * - * Returns 0 if the operation has started, -1 on failure. + * The @disk parameter is the name of the block device. Get this + * by calling virDomainGetXMLDesc and finding the <target dev='...'> + * attribute within //domain/devices/disk. (For example, "xvda"). + * + * Returns -1 in case of error, 0 in case of success. */ int virDomainSetBlockIoTune(virDomainPtr dom, const char *disk, - virDomainBlockIoTuneInfoPtr info, + virTypedParameterPtr params, + int nparams, unsigned int flags) { virConnectPtr conn; @@ -17245,18 +17251,35 @@ error: * virDomainGetBlockIoTune: * @dom: pointer to domain object * @disk: Fully-qualified disk name - * @reply: Specify block I/O info in bytes + * @params: pointer to blkio parameter object + * (return value, allocated by the caller) + * @nparams: pointer to number of blkio parameters; input and output * @flags: An OR'ed set of virDomainModificationImpact * - * This function is mainly to enable Block I/O throttling function in libvirt. - * It is used to get the block I/O throttling setting for specified domain. + * Get all block IO tunable parameters for a given device. On input, + * @nparams gives the size of the @params array; on output, @nparams + * gives how many slots were filled with parameter information, which + * might be less but will not exceed the input value. * - * Returns 0 if the operation has started, -1 on failure. + * As a special case, calling with @params as NULL and @nparams as 0 on + * input will cause @nparams on output to contain the number of parameters + * supported by the hypervisor. The caller should then allocate @params + * array, i.e. (sizeof(@virTypedParameter) * @nparams) bytes and call the API + * again. + * + * See virDomainGetMemoryParameters() for an equivalent usage example. + * + * The @disk parameter is the name of the block device. Get this + * by calling virDomainGetXMLDesc and finding the <target dev='...'> + * attribute within //domain/devices/disk. (For example, "xvda"). + * + * Returns -1 in case of error, 0 in case of success. */ int virDomainGetBlockIoTune(virDomainPtr dom, const char *disk, - virDomainBlockIoTuneInfoPtr reply, + virTypedParameterPtr params, + int *nparams, unsigned int flags) { virConnectPtr conn; -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
On 11/11/2011 05:50 AM, Adam Litke wrote:
Hi Lei,
I've tested this and found that it works well except for one small thing.
When the domain is running and I change one of the throttle settings using the --config option in virsh, the on-disk domain xml file is properly updated, but the command 'virsh dumpxml' returns the old version. Is this how it is supposed to work?
Hi Adam, I tried to test this case. I found that the command 'virsh dumpxml' will return the updated version after restart or just destroy the domain. :)
Otherwise, it seems to work as advertised (based on my sniff testing).
Tested-by: Adam Litke<agl@us.ibm.com>
On Thu, Nov 10, 2011 at 04:32:50AM +0800, Lei HH Li wrote:
Changes since V2 - Implement the Python binding support for setting blkio throttling. - Implement --current --live --config options support to unify the libvirt API. - Add changes in docs and tests. - Some changes suggested by Adam Litke, Eric Blake, Daniel P. Berrange. - Change the XML schema. - API name to virDomain{Set, Get}BlockIoTune. - Parameters changed to make them more self-explanatory. - virsh command name to blkdeviotune. - And other fixups.
Changes since V1 - Implement the support to get the block io throttling for a device as read only connection - QMP/HMP. - Split virDomainBlockIoThrottle into two separate functions virDomainSetBlockIoThrottle - Set block I/O limits for a device - Requires a connection in 'write' mode. - Limits (info) structure passed as an input parameter virDomainGetBlockIoThrottle - Get the current block I/O limits for a device - Works on a read-only connection. - Current limits are written to the output parameter (reply). - And Other fixups suggested by Adam Litke, Daniel P. Berrange. - For dynamically allocate the blkiothrottle struct, I will fix it when implement --current --live --config options support.
Today libvirt supports the cgroups blkio-controller, which handles proportional shares and throughput/iops limits on host block devices. blkio-controller does not support network file systems (NFS) or other QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are not host block devices. QEMU I/O throttling works with all types of drive and can be applied independently to each drive attached to a guest and supports throughput/iops limits.
To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings.
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream just now and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
1) Enable the blkio throttling in xml when guest is starting up.
Add blkio throttling in xml as follows:
<disk type='file' device='disk'> ... <iotune> <total_bytes_sec>nnn</total_bytes_sec> ... </iotune> ... </disk>
2) Enable blkio throttling setting at guest running time.
virsh blkdeviotune<domain> <device> [--total_bytes_sec<number>] [--read_bytes_sec<number>] \ [--write_bytes_sec<number>] [--total_iops_sec<number>] [--read_iops_sec<number>] [--write_iops_sec<number>]
3) The support to get the current block i/o throttling for a device - HMP/QMP.
virsh blkiothrottle<domain> <device> total_bytes_sec: read_bytes_sec: write_bytes_sec: total_iops_sec: read_iops_sec: write_iops_sec:
4) Python binding support for setting blkio throttling. 5) --current --live --config options support to unify the libvirt API.
virsh blkdeviotune<domain> <device> [--total_bytes_sec<number>] [--read_bytes_sec<number>] [--write_bytes_sec<number>] [--total_iops_sec<number>] [--read_iops_sec<number>] [--write_iops_sec<number>] [--config] [--live] [--current]
daemon/remote.c | 87 +++++++ docs/formatdomain.html.in | 30 ++ docs/schemas/domaincommon.rng | 24 + include/libvirt/libvirt.h.in | 25 ++ python/generator.py | 2 python/libvirt-override-api.xml | 16 + python/libvirt-override.c | 85 +++++++ src/conf/domain_conf.c | 101 ++++++++ src/conf/domain_conf.h | 12 src/driver.h | 18 + src/libvirt.c | 115 +++++++++ src/libvirt_public.syms | 2 src/qemu/qemu_command.c | 33 ++ src/qemu/qemu_driver.c | 217 ++++++++++++++++++ src/qemu/qemu_monitor.c | 36 ++ src/qemu/qemu_monitor.h | 10 src/qemu/qemu_monitor_json.c | 191 +++++++++++++++ src/qemu/qemu_monitor_json.h | 10 src/qemu/qemu_monitor_text.c | 152 ++++++++++++ src/qemu/qemu_monitor_text.h | 10 src/remote/remote_driver.c | 81 ++++++ src/remote/remote_protocol.x | 39 +++ src/remote_protocol-structs | 34 ++ src/util/xml.h | 3 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.args | 4 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.xml | 36 ++ tests/qemuxml2argvtest.c | 2 tests/qemuxml2xmltest.c | 2 tools/virsh.c | 146 ++++++++++++ tools/virsh.pod | 23 + 30 files changed, 1540 insertions(+), 6 deletions(-)
-- Lei
-- Lei
participants (4)
-
Adam Litke -
Eric Blake -
Lei Li -
Zhi Yong Wu