10:13 p.m.
From: Chun Feng Wu <wucf(a)linux.ibm.com>
Hi,
I am thinking to leverage "throttle block filter" in QEMU to support more
flexible I/O limits(e.g. tiered I/O groups), one sample provided by QEMU doc is:
https://github.com/qemu/qemu/blob/master/docs/throttle.txt
"For example, let's say that we have three different drives and we want to set
I/O limits for
each one of them and an additional set of limits for the combined I/O of all three
drives."
The implementation idea is to
- Define throttle groups(limit) in domain
- Define throttle filter to reference throttle group within disk
- Within domain disk, throttle filters references multiple throttle groups to form filter
chain to apply multiple limits in QEMU like above sample
- Add new virsh cmds for throttle group management:
throttlegroupset Add or update a throttling group.
throttlegroupdel Delete a throttling group.
throttlegroupinfo Get a throttling group.
throttlegrouplist list all domain throttlegroups
- Update "attach-disk" to add one more option "--throttle-groups" to
apply throttle filters e.g. "virsh attach-disk $VM_ID ${DISK_PATH}/vm1_disk_2.qcow2
vdd --driver qemu --subdriver qcow2 --targetbus virtio --throttle-groups
limit2,limit012"
- I chose above semantics as I felt they're appropriate, if there are better ones
please kindly suggest.
This patchset includes:
- Throttle group and throttle filter definition in patch 1
- New QMP processing to update and get throttle group in patch 2
- New API definition and implementation in patch 3
- Hotplug and qemuProcessLaunch flow implemenation in patch 4, 5
- Domain XML schema and doc(formatdomain.rst) change in patch 6
- Tests in patch 7, 8
- Virsh cmd implementation in patch 9
- Other enhencement/verification implementation in patch 10, 11, 12
From QMP perspective, the sample flow works this way:
- Throttle group creation:
virsh qemu-monitor-command 1 '{"execute":"object-add",
"arguments":{"qom-type":"throttle-group","id":"limit0","limits":{"iops-total":200,"iops-read":0,"iops-total-max":200,"iops-total-max-length":1}}}'
virsh qemu-monitor-command 1 '{"execute":"object-add",
"arguments":{"qom-type":"throttle-group","id":"limit1","limits":{"iops-total":250,"iops-read":0,"iops-total-max":250,"iops-total-max-length":1}}}'
virsh qemu-monitor-command 1 '{"execute":"object-add",
"arguments":{"qom-type":"throttle-group","id":"limit2","limits":{"iops-total":300,"iops-read":0,"iops-total-max":300,"iops-total-max-length":1}}}'
virsh qemu-monitor-command 1 '{"execute":"object-add",
"arguments":{"qom-type":"throttle-group","id":"limit012","limits":{"iops-total":400,"iops-read":0,"iops-total-max":400,"iops-total-max-length":1}}}'
- Chain up filters during attaching disk to apply two filters(limit0 and limit012):
virsh qemu-monitor-command 1 '{"execute":"blockdev-add",
"arguments":
{"driver":"file","filename":"/virt/disks/vm1_disk_1.qcow2","node-name":"test-3-storage","auto-read-only":true,"discard":"unmap"}}'
virsh qemu-monitor-command 1 '{"execute":"blockdev-add",
"arguments":{"node-name":"test-3-format","read-only":false,"driver":"qcow2","file":"test-3-storage","backing":null}}'
virsh qemu-monitor-command 1 '{"execute":"blockdev-add",
"arguments":{"driver":"throttle","node-name":"libvirt-5-filter","throttle-group":
"limit0","file":"test-3-format"}}'
virsh qemu-monitor-command 1 '{"execute":"blockdev-add",
"arguments":
{"driver":"throttle","node-name":"libvirt-6-filter","throttle-group":"limit012","file":"libvirt-5-filter"}}'
virsh qemu-monitor-command 1 '{"execute": "device_add",
"arguments":
{"driver":"virtio-blk-pci","scsi":false,"bus":"pci.0","addr":"0x5","drive":"libvirt-6-filter","id":"virtio-disk1"}}'
BTW, I also got support from these guys(guyujie(a)linux.ibm.com, wuyx(a)linux.ibm.com,
xinhaong(a)linux.ibm.com, commits under their names are included) to help do some
enhancement/verification implementation, thanks a lot!
Any comments/suggestions will be appriciated!
Chun Feng Wu (9):
config: Introduce ThrottleGroup and ThrottleFilter
qemu: monitor: Add support for ThrottleGroup operations
remote: New APIs for ThrottleGroup lifecycle management
qemu: hotplug: Support hot attach block disk along with throttle
filters
qemu: command: Support throttle groups and filters during
qemuProcessLaunch
schema: Add new domain elements to support multiple throttle filters
test: Test throttle group lifecycle APIs
tests: Test qemuMonitorJSONGetThrottleGroup and
qemuMonitorJSONUpdateThrottleGroup
virsh: Add support for throttle group operations
Hao Ning Xin (1):
config: validate: Verify throttle group fields
Yan Xiu Wu (1):
config: validate: Use "iotune" and "throttlefilters" exclusively
for
specific disk
Yu Jie Gu (1):
remote: Define remote_domain_set_throttle_group_args and adjust
throttle structs sequence
docs/formatdomain.rst | 48 ++
include/libvirt/libvirt-domain.h | 29 +
src/conf/domain_conf.c | 364 ++++++++++++
src/conf/domain_conf.h | 43 ++
src/conf/domain_validate.c | 115 ++--
src/conf/schemas/domaincommon.rng | 164 +++++-
src/conf/virconftypes.h | 4 +
src/driver-hypervisor.h | 22 +
src/libvirt-domain.c | 188 +++++++
src/libvirt_private.syms | 9 +
src/libvirt_public.syms | 7 +
src/qemu/qemu_block.c | 121 ++++
src/qemu/qemu_block.h | 53 ++
src/qemu/qemu_command.c | 148 +++++
src/qemu/qemu_command.h | 9 +
src/qemu/qemu_domain.c | 79 +++
src/qemu/qemu_domain.h | 10 +
src/qemu/qemu_driver.c | 520 +++++++++++++++++
src/qemu/qemu_hotplug.c | 23 +
src/qemu/qemu_monitor.c | 29 +
src/qemu/qemu_monitor.h | 10 +
src/qemu/qemu_monitor_json.c | 135 +++++
src/qemu/qemu_monitor_json.h | 13 +
src/remote/remote_daemon_dispatch.c | 60 ++
src/remote/remote_driver.c | 46 ++
src/remote/remote_protocol.x | 50 +-
src/remote_protocol-structs | 30 +
src/test/test_driver.c | 380 +++++++++++++
tests/qemumonitorjsontest.c | 89 +++
.../qemustatusxml2xmldata/backup-pull-in.xml | 1 +
.../blockjob-blockdev-in.xml | 1 +
.../blockjob-mirror-in.xml | 1 +
.../migration-in-params-in.xml | 1 +
.../migration-out-nbd-bitmaps-in.xml | 1 +
.../migration-out-nbd-out.xml | 1 +
.../migration-out-nbd-tls-out.xml | 1 +
.../migration-out-params-in.xml | 1 +
tests/qemustatusxml2xmldata/modern-in.xml | 1 +
tests/qemustatusxml2xmldata/upgrade-out.xml | 1 +
.../qemustatusxml2xmldata/vcpus-multi-in.xml | 1 +
tools/virsh-completer-domain.c | 62 ++
tools/virsh-completer-domain.h | 11 +
tools/virsh-domain.c | 530 ++++++++++++++++++
43 files changed, 3376 insertions(+), 36 deletions(-)
--
2.34.1