[libvirt] [PATCH 0/8 v5] Summary on block IO throttle
Changes since V3 - Use virTypedParameterPtr instead of specific struct in libvirt pulic API. - Relevant changes to remote driver, qemu driver, python support and virsh. Changes since V2 - Implement the Python binding support for setting blkio throttling. - Implement --current --live --config options support to unify the libvirt API. - Add changes in docs and tests. - Some changes suggested by Adam Litke, Eric Blake, Daniel P. Berrange. - Change the XML schema. - API name to virDomain{Set, Get}BlockIoTune. - Parameters changed to make them more self-explanatory. - virsh command name to blkdeviotune. - And other fixups. Changes since V1 - Implement the support to get the block io throttling for a device as read only connection - QMP/HMP. - Split virDomainBlockIoThrottle into two separate functions virDomainSetBlockIoThrottle - Set block I/O limits for a device - Requires a connection in 'write' mode. - Limits (info) structure passed as an input parameter virDomainGetBlockIoThrottle - Get the current block I/O limits for a device - Works on a read-only connection. - Current limits are written to the output parameter (reply). - And Other fixups suggested by Adam Litke, Daniel P. Berrange. - For dynamically allocate the blkiothrottle struct, I will fix it when implement --current --live --config options support. Today libvirt supports the cgroups blkio-controller, which handles proportional shares and throughput/iops limits on host block devices. blkio-controller does not support network file systems (NFS) or other QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are not host block devices. QEMU I/O throttling works with all types of drive and can be applied independently to each drive attached to a guest and supports throughput/iops limits. To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings. Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong: http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block 1) Enable the blkio throttling in xml when guest is starting up. Add blkio throttling in xml as follows: <disk type='file' device='disk'> ... <iotune> <total_bytes_sec>nnn</total_bytes_sec> ... </iotune> ... </disk> 2) Enable blkio throttling setting at guest running time. virsh blkdeviotune <domain> <device> [--total_bytes_sec<number>] [--read_bytes_sec<number>] \ [--write_bytes_sec<number>] [--total_iops_sec<number>] [--read_iops_sec<number>] [--write_iops_sec<number>] 3) The support to get the current block i/o throttling for a device - HMP/QMP. virsh blkiothrottle <domain> <device> total_bytes_sec: read_bytes_sec: write_bytes_sec: total_iops_sec: read_iops_sec: write_iops_sec: 4) Python binding support for setting blkio throttling. 5) --current --live --config options support to unify the libvirt API. virsh blkdeviotune <domain> <device> [--total_bytes_sec <number>] [--read_bytes_sec <number>] [--write_bytes_sec <number>] [--total_iops_sec <number>] [--read_iops_sec <number>] [--write_iops_sec <number>] [--config] [--live] [--current] daemon/remote.c | 108 +++++++ docs/formatdomain.html.in | 31 ++ docs/schemas/domaincommon.rng | 24 ++ include/libvirt/libvirt.h.in | 70 ++++ python/generator.py | 2 + python/libvirt-override-api.xml | 16 + python/libvirt-override.c | 179 +++++++++++ src/conf/domain_conf.c | 101 ++++++- src/conf/domain_conf.h | 12 + src/driver.h | 20 ++ src/libvirt.c | 145 +++++++++ src/libvirt_public.syms | 2 + src/qemu/qemu_command.c | 33 ++ src/qemu/qemu_driver.c | 338 ++++++++++++++++++++ src/qemu/qemu_monitor.c | 36 ++ src/qemu/qemu_monitor.h | 22 ++ src/qemu/qemu_monitor_json.c | 185 +++++++++++ src/qemu/qemu_monitor_json.h | 10 + src/qemu/qemu_monitor_text.c | 164 ++++++++++ src/qemu/qemu_monitor_text.h | 10 + src/remote/remote_driver.c | 96 ++++++ src/remote/remote_protocol.x | 26 ++- src/remote_protocol-structs | 24 ++ src/util/xml.h | 2 + .../qemuxml2argv-blkdeviotune.args | 4 + .../qemuxml2argvdata/qemuxml2argv-blkdeviotune.xml | 37 +++ tests/qemuxml2argvtest.c | 1 + tests/qemuxml2xmltest.c | 1 + tools/virsh.c | 240 ++++++++++++++ tools/virsh.pod | 23 ++ 30 files changed, 1958 insertions(+), 4 deletions(-) -- Lei
On 11/15/2011 02:02 AM, Lei Li wrote:
Changes since V3 - Use virTypedParameterPtr instead of specific struct in libvirt pulic API. - Relevant changes to remote driver, qemu driver, python support and virsh.
Hmm, you didn't summarize changes since v4. Sorry for not noticing the v5 a bit sooner (the thread is getting a bit long, so this got buried in my inbox). Hopefully we are closer to getting this applied; I'd certainly like to see it in 0.9.8. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 11/22/2011 08:14 AM, Eric Blake wrote:
On 11/15/2011 02:02 AM, Lei Li wrote:
Changes since V3 - Use virTypedParameterPtr instead of specific struct in libvirt pulic API. - Relevant changes to remote driver, qemu driver, python support and virsh. Hmm, you didn't summarize changes since v4. Sorry for not noticing the v5 a bit sooner (the thread is getting a bit long, so this got buried in my inbox). Hopefully we are closer to getting this applied; I'd certainly like to see it in 0.9.8.
Hi Eric, Just a little minor changes as you mentioned in V4, so I didn't write it into summary change log. - Add device shorthand as a choice option to Fully-qualified name in the definition in src/libvirt.c. - Move the added name of public API to new section LIBVIRT_0.9.8 in src/libvirt_public.syms. Yeah, I hope so. :)
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- Lei
On 11/15/2011 02:02 AM, Lei Li wrote:
Changes since V3 - Use virTypedParameterPtr instead of specific struct in libvirt pulic API. - Relevant changes to remote driver, qemu driver, python support and virsh.
To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings.
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
I just realized that the block_set_io_throttle monitor command is not yet in released upstream qemu. I will continue reviewing this series, but it's harder to justify including it into libvirt 0.9.8 unless we have a good feel that the design for qemu is stable and unlikely to undergo further modifications. We have included libvirt changes for unreleased qemu features in the past, but only after pointing to qemu list archives documenting that there is large consensus for the proposed design, and that any further changes prior to a qemu release containing the design will be internal only. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 11/15/2011 02:02 AM, Lei Li wrote:
Changes since V3 - Use virTypedParameterPtr instead of specific struct in libvirt pulic API. - Relevant changes to remote driver, qemu driver, python support and virsh.
To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings.
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
I just realized that the block_set_io_throttle monitor command is not yet in released upstream qemu. I will continue reviewing this series, but it's harder to justify including it into libvirt 0.9.8 unless we have a good feel that the design for qemu is stable and unlikely to undergo further modifications. We have included libvirt changes for unreleased qemu features in the past, but only after pointing to qemu list archives documenting that there is large consensus for the proposed design, and that any further changes prior to a qemu release containing the design will be internal only. As you've known, qemu 1.1 development branch will be created after qemu 1.0 is announced, and this date is about on 1st Dec this year. So current accepted block features are all kept in kevin's block git
On Wed, Nov 23, 2011 at 12:27 PM, Eric Blake <eblake@redhat.com> wrote: tree.
-- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- Regards, Zhi Yong Wu
On 11/15/2011 02:02 AM, Lei Li wrote:
Changes since V3 - Use virTypedParameterPtr instead of specific struct in libvirt pulic API. - Relevant changes to remote driver, qemu driver, python support and virsh.
To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings.
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
I just realized that the block_set_io_throttle monitor command is not yet in released upstream qemu. I will continue reviewing this series, but it's harder to justify including it into libvirt 0.9.8 unless we have a good feel that the design for qemu is stable and unlikely to I strongly think that the syntax of this qemu monitor command will be not changed later. :) Moreover, it has not been changed since the patches for qemu block I/O
On Wed, Nov 23, 2011 at 12:27 PM, Eric Blake <eblake@redhat.com> wrote: throttling is published.
undergo further modifications. We have included libvirt changes for unreleased qemu features in the past, but only after pointing to qemu list archives documenting that there is large consensus for the proposed design, and that any further changes prior to a qemu release containing the design will be internal only.
-- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- Regards, Zhi Yong Wu
On Wed, Nov 23, 2011 at 4:59 AM, Zhi Yong Wu <zwu.kernel@gmail.com> wrote:
On 11/15/2011 02:02 AM, Lei Li wrote:
Changes since V3 - Use virTypedParameterPtr instead of specific struct in libvirt pulic API. - Relevant changes to remote driver, qemu driver, python support and virsh.
To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings.
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
I just realized that the block_set_io_throttle monitor command is not yet in released upstream qemu. I will continue reviewing this series, but it's harder to justify including it into libvirt 0.9.8 unless we have a good feel that the design for qemu is stable and unlikely to I strongly think that the syntax of this qemu monitor command will be not changed later. :) Moreover, it has not been changed since the patches for qemu block I/O
On Wed, Nov 23, 2011 at 12:27 PM, Eric Blake <eblake@redhat.com> wrote: throttling is published.
It's good to be careful about adding libvirt features that are not yet in QEMU. I got burnt by this with image streaming and have had to work extra hard to finally get the QEMU side merged with the libvirt side already set in stone. However, Kevin has merged the QEMU I/O throttling patches into the block tree. They are waiting for the 1.1 merge window because qemu.git has only been taking QEMU 1.0-rc patches for the past few weeks. QEMU I/O throttling has been through review and accepted by the community: http://patchwork.ozlabs.org/patch/124266/ Stefan
On 11/23/2011 02:18 AM, Stefan Hajnoczi wrote:
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
I just realized that the block_set_io_throttle monitor command is not yet in released upstream qemu. I will continue reviewing this series, but it's harder to justify including it into libvirt 0.9.8 unless we have a good feel that the design for qemu is stable and unlikely to I strongly think that the syntax of this qemu monitor command will be not changed later. :) Moreover, it has not been changed since the patches for qemu block I/O throttling is published.
It's good to be careful about adding libvirt features that are not yet in QEMU. I got burnt by this with image streaming and have had to work extra hard to finally get the QEMU side merged with the libvirt side already set in stone.
However, Kevin has merged the QEMU I/O throttling patches into the block tree. They are waiting for the 1.1 merge window because qemu.git has only been taking QEMU 1.0-rc patches for the past few weeks. QEMU I/O throttling has been through review and accepted by the community:
How can libvirt reliably tell whether block throttling is supported? Does that patch series alter the help output of 'qemu -h' to list something like '-drive ...[,bps=...]'? Is there a magic 'qemu -driver xxx,?' call where we can see that the new bps attribute is settable? I ask, because we need to alter libvirt to properly detect whether qemu supports this new feature, so as to gracefully reject attempts to use the XML if qemu does not support the feature. See also: https://www.redhat.com/archives/libvir-list/2012-January/msg00727.html -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Wed, Jan 18, 2012 at 09:22:01AM -0700, Eric Blake wrote:
On 11/23/2011 02:18 AM, Stefan Hajnoczi wrote:
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
I just realized that the block_set_io_throttle monitor command is not yet in released upstream qemu. I will continue reviewing this series, but it's harder to justify including it into libvirt 0.9.8 unless we have a good feel that the design for qemu is stable and unlikely to I strongly think that the syntax of this qemu monitor command will be not changed later. :) Moreover, it has not been changed since the patches for qemu block I/O throttling is published.
It's good to be careful about adding libvirt features that are not yet in QEMU. I got burnt by this with image streaming and have had to work extra hard to finally get the QEMU side merged with the libvirt side already set in stone.
However, Kevin has merged the QEMU I/O throttling patches into the block tree. They are waiting for the 1.1 merge window because qemu.git has only been taking QEMU 1.0-rc patches for the past few weeks. QEMU I/O throttling has been through review and accepted by the community:
How can libvirt reliably tell whether block throttling is supported? Does that patch series alter the help output of 'qemu -h' to list something like '-drive ...[,bps=...]'? Is there a magic 'qemu -driver xxx,?' call where we can see that the new bps attribute is settable?
I ask, because we need to alter libvirt to properly detect whether qemu supports this new feature, so as to gracefully reject attempts to use the XML if qemu does not support the feature. See also: https://www.redhat.com/archives/libvir-list/2012-January/msg00727.html
The most robust method is to use QMP query-commands or HMP help to look for the block I/O throttling commands. Scraping qemu -h is possible too but fragile, note the last line of help output: -drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i] [,cyls=c,heads=h,secs=s[,trans=t]][,snapshot=on|off] [,cache=writethrough|writeback|none|directsync|unsafe][,format=f] [,serial=s][,addr=A][,id=name][,aio=threads|native] [,readonly=on|off][,copy-on-read=on|off] [[,bps=b]|[[,bps_rd=r][,bps_wr=w]]][[,iops=i]|[[,iops_rd=r][,iops_wr=w]] Here's a hack that really probes. Block I/O throttling supported: $ qemu-system-x86_64 -drive bps=1,bps_rd=1 qemu-system-x86_64: -drive bps=1,bps_rd=1: bps(iops) and bps_rd/bps_wr(iops_rd/iops_wr) cannot be used at the same time Block I/O throttling *not* supported: $ qemu-system-x86_64 -drive bps=1,bps_rd=1 qemu-system-x86_64: -drive bps=1,bps_rd=1: Invalid parameter 'bps' Stefan
Am 18.01.2012 17:22, schrieb Eric Blake:
On 11/23/2011 02:18 AM, Stefan Hajnoczi wrote:
Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong:
http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block
I just realized that the block_set_io_throttle monitor command is not yet in released upstream qemu. I will continue reviewing this series, but it's harder to justify including it into libvirt 0.9.8 unless we have a good feel that the design for qemu is stable and unlikely to I strongly think that the syntax of this qemu monitor command will be not changed later. :) Moreover, it has not been changed since the patches for qemu block I/O throttling is published.
It's good to be careful about adding libvirt features that are not yet in QEMU. I got burnt by this with image streaming and have had to work extra hard to finally get the QEMU side merged with the libvirt side already set in stone.
However, Kevin has merged the QEMU I/O throttling patches into the block tree. They are waiting for the 1.1 merge window because qemu.git has only been taking QEMU 1.0-rc patches for the past few weeks. QEMU I/O throttling has been through review and accepted by the community:
How can libvirt reliably tell whether block throttling is supported? Does that patch series alter the help output of 'qemu -h' to list something like '-drive ...[,bps=...]'? Is there a magic 'qemu -driver xxx,?' call where we can see that the new bps attribute is settable?
The help output has changed, you can use that. There is no such thing as -drive ? Kevin
participants (6)
-
Eric Blake -
Kevin Wolf -
Lei Li -
Stefan Hajnoczi -
Stefan Hajnoczi -
Zhi Yong Wu