- Devel - libvirt List Archives

[libvirt] 0.9.7 virsh console regression?
by Albert W. Hopkins 11 Nov '11

11 Nov '11

Hi folks, I have a few "headless" qemu domains that use serial consoles. Typically i connect to the consoles with # virsh console <domain> However with 0.9.7 I get Connected to domain gentoo-base Escape character is ^] error: internal error character device (null) is not using a PTY I am still able to connect to the console if i go into the virt-manager gui and use the "gui" console, but I'd rather not do that as sometimes I don't have access to the gui. Is this a regression or am I expected to do some kind of migration on my domains. These are qemu guests running on Gentoo Linux. An xml dump of a typical domain follows: <domain type='kvm'> <name>base</name> <uuid>b4a528dd-4c31-03ca-bdde-795c4c38592d</uuid> <memory>524288</memory> <currentMemory>61440</currentMemory> <vcpu>1</vcpu> <os> <type arch='x86_64' machine='pc-0.12'>hvm</type> <kernel>/var/lib/libvirt/images/vmlinuz</kernel> <cmdline>root=/dev/vda1 rootfstype=ext4 quiet console=ttyS0 TERM=vt100</cmdline> <boot dev='hd'/> <bootmenu enable='no'/> </os> <features> <acpi/> <pae/> </features> <cpu match='exact'> <model>Nehalem</model> <vendor>Intel</vendor> <feature policy='require' name='tm2'/> <feature policy='require' name='est'/> <feature policy='require' name='monitor'/> <feature policy='require' name='ss'/> <feature policy='require' name='vme'/> <feature policy='require' name='rdtscp'/> <feature policy='require' name='ht'/> <feature policy='require' name='ds'/> <feature policy='require' name='pbe'/> <feature policy='require' name='tm'/> <feature policy='require' name='vmx'/> <feature policy='require' name='ds_cpl'/> <feature policy='require' name='xtpr'/> <feature policy='require' name='acpi'/> <feature policy='require' name='x2apic'/> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/lib/libvirt/images/base.qcow'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <interface type='network'> <mac address='52:54:00:06:84:c0'/> <source network='default'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </memballoon> </devices> </domain> Thanks in advance, -a

3 2

[libvirt] Open Source Virtualization and Cloud at FOSDEM
by Mark McLoughlin 11 Nov '11

11 Nov '11

Hey, Here's one for the calendar - a 2 day track at FOSDEM devoted to Open Source Virtualization and Cloud. Hope to see you there! :) Cheers, Mark. -------- Forwarded Message -------- > From: Thierry Carrez <thierry(a)openstack.org> > To: openstack(a)lists.launchpad.net <openstack(a)lists.launchpad.net> > Subject: [Openstack] OpenStack at FOSDEM > Date: Fri, 11 Nov 2011 11:34:22 +0100 > > Hi everyone, > > Our OpenStack devroom was accepted for FOSDEM, February 4-5, 2012 in > Brussels. It was actually grouped with other "Open Source virtualization > and cloud" projects in a giant devroom (550-seat auditorium) that lasts > the full 2 days ! > > Expect a CFP in the coming weeks... > > See other devrooms at: > http://fosdem.org/2012/devrooms_for_2012 > > -- > Thierry Carrez (ttx) > Release Manager, OpenStack > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp

1 0

[libvirt] Long QEMU main loop pauses during migration (to file) under heavy load
by Daniel P. Berrange 11 Nov '11

11 Nov '11

Libvirt recently introduced a change to the way it does 'save to file' with QEMU. Historically QEMU has a 32MB/s I/O limit on migration by default. When saving to file, we didn't want any artificial limit, but rather to max out the underlying storage. So when doing save to file, we set a large bandwidth limit (INT64_MAX / (1024 * 1024)) so it is effectively unlimited. After doing this, we discovered that the QEMU monitor was becoming entirely blocked. It did not even return from the 'migrate' command until migration was complete despite the 'detach' flag being set. This was a bug in libvirt, because we passed a plain file descriptor which does not support EAGAIN. Thank you POSIX. Libvirt has another mode where it uses an I/O helper command so get O_DIRECT, and in this mode we pass a pipe() FD to QEMU. After ensuring that this pipe FD really does have O_NONBLOCK set, we still saw some odd behaviour. I'm not sure whether what I describe can neccessarily be called a QEMU bug, but I wanted to raise it for discussion anyway.... The sequence of steps is - libvirt sets qemu migration bandwidth to "unlimited" - libvirt opens a pipe() and sets O_NONBLOCK on the write end - libvirt spawns libvirt-iohelper giving it the target file on disk, and the read end of the pipe - libvirt does 'getfd migfile' monitor command to give QEMU the write end of the pipe - libvirt does 'migrate fd:migfile -d' to run migration - In parallel - QEMU is writing to the pipe (which is non-blocking) - libvirt_helper reading the pipe & writing to disk with O_DIRECT The initial 'migrate' command detaches into the background OK, and libvirt can enter its loop doing 'query-migrate' frequently to monitor progress. Initially this works fine, but at some points during the migration, QEMU will get "stuck" for a very long time and not respond to the monitor (or indeed the mainloop at all). These blackouts are anywhere from 10 to 20 seconds long. Using a combination of systemtap, gdb and strace I managed to determine out the following - Most of the qemu_savevm_state_iterate() calls complete in 10-20 ms - Reasonably often a qemu_savevm_state_iterate() call takes 300-400 ms - Fairly rarely a qemu_savevm_state_iterate() call takes 10-20 *seconds* - I can see EAGAIN from the FD QEMU is migrating from - hence most of the iterations are quite short. - In the 10-20 second long calls, no EAGAIN is seen for the entire period. - The host OS in general is fairly "laggy", presumably due to the high rate of direct I/O being performed by the I/O helper, and bad schedular tuning IIUC, there are two things which will cause a qemu_savevm_state_iterate() call to return - Getting EAGAIN on the migration FD - Hitting the max bandwidth limit We have set effectively unlimited bandwidth, so everything relies on the EAGAIN behaviour. If the OS is doing a good job at scheduling processes & I/O, then this seems to work reasonably well. If the OS is highly loaded and becoming less responsive to scheduling apps, then QEMU gets itself into a bad way. What I think is happening is that the OS is giving too much time to the I/O helper process that is reading the other end of the pipe given to QEMU, and then doing the O_DIRECT to disk. Thus in the shorter-than-normal windows of time when QEMU itself is scheduled by the OS, the pipe is fairly empty, so QEMU does not see EAGAIN for a very long period of wallclock time. So we get into a case where QEMU sees 10-20 second gaps betweeen iterations of the main loop. Having a non-inifinite max-bandwidth for the migration, would likely mitigate this to some extent, but I still think it'd be possible to get QEMU into these pathelogical conditions under high load for a host. Is this a scenario we need to worry about for QEMU ? On the one hand it seems like it is a rare edge case in OS behaviour overall. On the other hand, times when a host is highly loaded and non-responsive are exactly the times when a mgmt app might want to save a guest to disk, or migrate it elsewhere. Which would mean we need QEMU to behave as well as possible in these adverse conditions. Thus should we consider having an absolute bound on the execution time of qemu_savevm_state_iterate(), independant of EAGAIN & bandwidth limits, to ensure the main loop doesn't get starved ? Or perhaps moving migration to a separate thread, out of the mainloop is what we need to strive for ? Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

1 1

[libvirt] [PATCH 0/3] Guest NUMA topology support - v1
by Bharata B Rao 11 Nov '11

11 Nov '11

Hi, This patch series adds support for specifying NUMA topology for guests. <cpu> ... <topology sockets='2' cores='4' threads='2'/> <numa> <cell cpus='0-7' mems='512000'/> <cell cpus='8-15' mems='512000'/> </numa> ... </cpu> This change allows libvirt to generate -numa options for QEMU/KVM. Changes since v0 ---------------- - Patch re-organization as per danpb's comments. - Use of virBufferAsPrintf instead of snprintf as per danpb's comments. - Use "cell" instead of "node" as per danpb's suggestion. - Add a helper to truncate virBuffer as per Eric's suggestion. - Add documentation. - Add testcases - Add basic validity check for cpus specified in <numa> ... </numa> Testing -------- The patches pass all tests except domainschema test. I think this is not a real failure, but the test is probably failing to pickup the right domaincommon.rng file. Will investigate further. TEST: domainschematest ........................................ 40 ........................................ 80 .........!.............................. 120 ........................................ 160 .................!...................... 200 ........................................ 240 ........................................ 280 ........................................ 320 ... 323 FAILED daemon-conf fails, but it fails even without my patches too. I guess my patches are really affecting it. TEST: daemon-conf .....!!!!!!...!!!!!!!!!!!!!!!!!!!./daemon-conf: line 98: kill: (7811) - No such process ! 34 FAILED v0 -- - https://www.redhat.com/archives/libvir-list/2011-October/msg00025.html RFC --- - http://permalink.gmane.org/gmane.comp.emulators.libvirt/44626 Regards, Bharata.

2 11

[libvirt] [RFC PATCH 0/8 v3] Summary on block IO throttle
by Lei Li 11 Nov '11

11 Nov '11

Changes since V2 - Implement the Python binding support for setting blkio throttling. - Implement --current --live --config options support to unify the libvirt API. - Add changes in docs and tests. - Some changes suggested by Adam Litke, Eric Blake, Daniel P. Berrange. - Change the XML schema. - API name to virDomain{Set, Get}BlockIoTune. - Parameters changed to make them more self-explanatory. - virsh command name to blkdeviotune. - And other fixups. Changes since V1 - Implement the support to get the block io throttling for a device as read only connection - QMP/HMP. - Split virDomainBlockIoThrottle into two separate functions virDomainSetBlockIoThrottle - Set block I/O limits for a device - Requires a connection in 'write' mode. - Limits (info) structure passed as an input parameter virDomainGetBlockIoThrottle - Get the current block I/O limits for a device - Works on a read-only connection. - Current limits are written to the output parameter (reply). - And Other fixups suggested by Adam Litke, Daniel P. Berrange. - For dynamically allocate the blkiothrottle struct, I will fix it when implement --current --live --config options support. Today libvirt supports the cgroups blkio-controller, which handles proportional shares and throughput/iops limits on host block devices. blkio-controller does not support network file systems (NFS) or other QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are not host block devices. QEMU I/O throttling works with all types of drive and can be applied independently to each drive attached to a guest and supports throughput/iops limits. To help add QEMU I/O throttling support to libvirt, we plan to complete it with add new API virDomain{Set, Get}BlockIoThrottle(), new command 'blkdeviotune' and Python bindings. Notes: Now all the planed features were implemented (#1#2 were implemented by Zhi Yong Wu), the previous comments were all fixed up too. And the qemu part patches have been accepted upstream just now and are expected to be part of the QEMU 1.1 release, git tree from Zhi Yong: http://repo.or.cz/w/qemu/kevin.git/shortlog/refs/heads/block 1) Enable the blkio throttling in xml when guest is starting up. Add blkio throttling in xml as follows: <disk type='file' device='disk'> ... <iotune> <total_bytes_sec>nnn</total_bytes_sec> ... </iotune> ... </disk> 2) Enable blkio throttling setting at guest running time. virsh blkdeviotune <domain> <device> [--total_bytes_sec<number>] [--read_bytes_sec<number>] \ [--write_bytes_sec<number>] [--total_iops_sec<number>] [--read_iops_sec<number>] [--write_iops_sec<number>] 3) The support to get the current block i/o throttling for a device - HMP/QMP. virsh blkiothrottle <domain> <device> total_bytes_sec: read_bytes_sec: write_bytes_sec: total_iops_sec: read_iops_sec: write_iops_sec: 4) Python binding support for setting blkio throttling. 5) --current --live --config options support to unify the libvirt API. virsh blkdeviotune <domain> <device> [--total_bytes_sec <number>] [--read_bytes_sec <number>] [--write_bytes_sec <number>] [--total_iops_sec <number>] [--read_iops_sec <number>] [--write_iops_sec <number>] [--config] [--live] [--current] daemon/remote.c | 87 +++++++ docs/formatdomain.html.in | 30 ++ docs/schemas/domaincommon.rng | 24 + include/libvirt/libvirt.h.in | 25 ++ python/generator.py | 2 python/libvirt-override-api.xml | 16 + python/libvirt-override.c | 85 +++++++ src/conf/domain_conf.c | 101 ++++++++ src/conf/domain_conf.h | 12 src/driver.h | 18 + src/libvirt.c | 115 +++++++++ src/libvirt_public.syms | 2 src/qemu/qemu_command.c | 33 ++ src/qemu/qemu_driver.c | 217 ++++++++++++++++++ src/qemu/qemu_monitor.c | 36 ++ src/qemu/qemu_monitor.h | 10 src/qemu/qemu_monitor_json.c | 191 +++++++++++++++ src/qemu/qemu_monitor_json.h | 10 src/qemu/qemu_monitor_text.c | 152 ++++++++++++ src/qemu/qemu_monitor_text.h | 10 src/remote/remote_driver.c | 81 ++++++ src/remote/remote_protocol.x | 39 +++ src/remote_protocol-structs | 34 ++ src/util/xml.h | 3 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.args | 4 tests/qemuxml2argvdata/qemuxml2argv-blkdeviotune.xml | 36 ++ tests/qemuxml2argvtest.c | 2 tests/qemuxml2xmltest.c | 2 tools/virsh.c | 146 ++++++++++++ tools/virsh.pod | 23 + 30 files changed, 1540 insertions(+), 6 deletions(-) -- Lei

4 6

[libvirt] Problem with listDomains() in Xen
by PREETHI RAMESH 11 Nov '11

11 Nov '11

This question concerns the usage of the listDomains(). When I run this piece of code in KVM, it works perfectly fine but throws an array out of bounds exception in Xen. Connect conn=null; conn = new Connect("xen:///", true); int[] id = conn.listDomains(); System.out.println("lD of VM " +id[0]); Answer I'd got: Well, your code assumes that id.length is >= 1, but listDomains() can return an empty array when there are no active domains known to libvirt. So you need to check id.length before accessing any element in the array. Follow up- I tried accessing the length of the array and it's 0 all the time. And there are active domains available. In fact, I'm able to getUUID from the domain name. Other functions work fine too. Am I missing something?Can someone please help me? Thanks!

1 0

[libvirt] [PATCH 7/8] Support virDomain{Set, Get}BlockIoThrottle in the python API
by Lei Li 10 Nov '11

10 Nov '11

Python support for both set and get block I/O throttle. Signed-off-by: Zhi Yong Wu <wuzhy(a)linux.vnet.ibm.com> Signed-off-by: Lei Li <lilei(a)linux.vnet.ibm.com> --- python/generator.py | 2 + python/libvirt-override-api.xml | 16 +++++++ python/libvirt-override.c | 85 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 103 insertions(+), 0 deletions(-) diff --git a/python/generator.py b/python/generator.py index 71afdb7..88c52b9 100755 --- a/python/generator.py +++ b/python/generator.py @@ -414,6 +414,8 @@ skip_impl = ( 'virDomainGetBlockJobInfo', 'virDomainMigrateGetMaxSpeed', 'virDomainBlockStatsFlags', + 'virDomainSetBlockIoTune', + 'virDomainGetBlockIoTune', ) qemu_skip_impl = ( diff --git a/python/libvirt-override-api.xml b/python/libvirt-override-api.xml index ef02f34..e8d9138 100644 --- a/python/libvirt-override-api.xml +++ b/python/libvirt-override-api.xml @@ -375,5 +375,21 @@ <arg name='flags' type='unsigned int' info='flags, currently unused, pass 0.'/> <return type='unsigned long' info='current max migration speed, or None in case of error'/> </function> + <function name='virDomainSetBlockIoTune' file='python'> + <info>Change the I/O throttle for a block device</info> + <arg name='dom' type='virDomainPtr' info='pointer to the domain'/> + <arg name='disk' type='const char *' info='disk name'/> + <arg name='info' type='virDomainBlockIoTuneInfoPtr' info='io throttle limits'/> + <arg name='flags' type='unsigned int' info='an OR'ed set of virDomainModificationImpact'/> + <return type='virDomainSetBlockIoTuneInfo' info='0 in case of operation has started, -1 in case of failure'/> + </function> + <function name='virDomainGetBlockIoTune' file='python'> + <info>Get the I/O throttle a block device</info> + <arg name='dom' type='virDomainPtr' info='pointer to the domain'/> + <arg name='disk' type='const char *' info='disk name'/> + <arg name='reply' type='virDomainBlockIoTuneInfoPtr' info='io throttle info'/> + <arg name='flags' type='unsigned int' info='an OR'ed set of virDomainModificationImpact'/> + <return type='virDomainGetBlockIoTuneInfo' info='A dictionary containing block I/O limits information.'/> + </function> </symbols> </api> diff --git a/python/libvirt-override.c b/python/libvirt-override.c index 1759bae..d4b023a 100644 --- a/python/libvirt-override.c +++ b/python/libvirt-override.c @@ -3195,6 +3195,89 @@ LIBVIRT_END_ALLOW_THREADS; return ret; } +static PyObject * +libvirt_virDomainSetBlockIoTune(PyObject *self ATTRIBUTE_UNUSED, + PyObject *args) +{ + virDomainPtr domain; + PyObject *pyobj_domain, *pyinfo; + const char *disk; + unsigned int flags; + virDomainBlockIoTuneInfo info; + int c_ret; + PyObject *ret; + + if (!PyArg_ParseTuple(args, (char *)"Ozi:virDomainSetBlockIoTune", + &pyobj_domain, &disk, &pyinfo, &flags)) + return(NULL); + domain = (virDomainPtr) PyvirDomain_Get(pyobj_domain); + + info.total_bytes_sec = (unsigned long long)PyLong_AsLongLong(PyDict_GetItem(pyinfo, + libvirt_constcharPtrWrap("bps"))); + info.read_bytes_sec = (unsigned long long)PyLong_AsLongLong(PyDict_GetItem(pyinfo, + libvirt_constcharPtrWrap("bps_rd"))); + info.write_bytes_sec = (unsigned long long)PyLong_AsLongLong(PyDict_GetItem(pyinfo, + libvirt_constcharPtrWrap("bps_wr"))); + info.total_iops_sec = (unsigned long long)PyLong_AsLongLong(PyDict_GetItem(pyinfo, + libvirt_constcharPtrWrap("iops"))); + info.read_iops_sec = (unsigned long long)PyLong_AsLongLong(PyDict_GetItem(pyinfo, + libvirt_constcharPtrWrap("iops_rd"))); + info.write_iops_sec = (unsigned long long)PyLong_AsLongLong(PyDict_GetItem(pyinfo, + libvirt_constcharPtrWrap("iops_wr"))); + + LIBVIRT_BEGIN_ALLOW_THREADS; + c_ret = virDomainSetBlockIoTune(domain, disk, &info, flags); + LIBVIRT_END_ALLOW_THREADS; + + if (c_ret < 0) { + return VIR_PY_NONE; + } + + return VIR_PY_INT_SUCCESS; +} + +static PyObject * +libvirt_virDomainGetBlockIoTune(PyObject *self ATTRIBUTE_UNUSED, + PyObject *args) +{ + virDomainPtr domain; + PyObject *pyobj_domain; + const char *disk; + unsigned int flags; + virDomainBlockIoTuneInfo reply; + int c_ret; + PyObject *ret; + + if (!PyArg_ParseTuple(args, (char *)"Oi:virDomainGetBlockIoTune", + &pyobj_domain, &disk, &flags)) + return(NULL); + domain = (virDomainPtr) PyvirDomain_Get(pyobj_domain); + + LIBVIRT_BEGIN_ALLOW_THREADS; + c_ret = virDomainGetBlockIoTune(domain, disk, &reply, flags); + LIBVIRT_END_ALLOW_THREADS; + + if (c_ret < 0) + return VIR_PY_NONE; + + if ((ret = PyDict_New()) == NULL) + return VIR_PY_NONE; + + PyDict_SetItem(ret, libvirt_constcharPtrWrap("total_bytes_sec"), + libvirt_ulonglongWrap(reply.total_bytes_sec)); + PyDict_SetItem(ret, libvirt_constcharPtrWrap("read_bytes_sec"), + libvirt_ulonglongWrap(reply.read_bytes_sec)); + PyDict_SetItem(ret, libvirt_constcharPtrWrap("write_bytes_sec"), + libvirt_ulonglongWrap(reply.write_bytes_sec)); + PyDict_SetItem(ret, libvirt_constcharPtrWrap("total_iops_sec"), + libvirt_ulonglongWrap(reply.total_iops_sec)); + PyDict_SetItem(ret, libvirt_constcharPtrWrap("read_iops_sec"), + libvirt_ulonglongWrap(reply.read_iops_sec)); + PyDict_SetItem(ret, libvirt_constcharPtrWrap("write_iops_sec"), + libvirt_ulonglongWrap(reply.write_iops_sec)); + return ret; +} + /******************************************* * Helper functions to avoid importing modules * for every callback @@ -4837,6 +4920,8 @@ static PyMethodDef libvirtMethods[] = { {(char *) "virDomainSnapshotListNames", libvirt_virDomainSnapshotListNames, METH_VARARGS, NULL}, {(char *) "virDomainRevertToSnapshot", libvirt_virDomainRevertToSnapshot, METH_VARARGS, NULL}, {(char *) "virDomainGetBlockJobInfo", libvirt_virDomainGetBlockJobInfo, METH_VARARGS, NULL}, + {(char *) "virDomainSetBlockIoTune", libvirt_virDomainSetBlockIoTune, METH_VARARGS, NULL}, + {(char *) "virDomainGetBlockIoTune", libvirt_virDomainGetBlockIoTune, METH_VARARGS, NULL}, {(char *) "virDomainSendKey", libvirt_virDomainSendKey, METH_VARARGS, NULL}, {(char *) "virDomainMigrateGetMaxSpeed", libvirt_virDomainMigrateGetMaxSpeed, METH_VARARGS, NULL}, {NULL, NULL, 0, NULL} -- 1.7.1

2 1

[libvirt] Wiki spam!
by Cole Robinson 10 Nov '11

10 Nov '11

The wiki's been spammed, see November 9 for example: http://wiki.libvirt.org/page/Special:Recentchanges virt-manager.org had a similar problem. Any one know any good proactive ways to prevent this? - Cole

2 3

[libvirt] [PATCH 5/8] Enable the blkdeviotune command in virsh
by Lei Li 10 Nov '11

10 Nov '11

Support virsh command blkdeviotune. Can set or query a block disk I/O throttle setting. Signed-off-by: Zhi Yong Wu <wuzhy(a)linux.vnet.ibm.com> Signed-off-by: Lei Li <lilei(a)linux.vnet.ibm.com> --- tools/virsh.c | 146 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 23 +++++++++ 2 files changed, 169 insertions(+), 0 deletions(-) diff --git a/tools/virsh.c b/tools/virsh.c index 83dc3c7..df2b399 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -6023,6 +6023,151 @@ cmdBlockJob(vshControl *ctl, const vshCmd *cmd) return true; } +/* + * "blkdeviotune" command + */ +static const vshCmdInfo info_blkdeviotune[] = { + {"help", N_("Set or query a block disk I/O throttle setting.")}, + {"desc", N_("Set or query a block disk I/O throttle setting.\n" \ + " To query the block disk I/O throttle setting use the following" \ + " command: \n\n" \ + " virsh # blkdeviotune <domain> <device>")}, + {NULL, NULL} +}; + +static const vshCmdOptDef opts_blkdeviotune[] = { + {"domain", VSH_OT_DATA, VSH_OFLAG_REQ, N_("domain name, id or uuid")}, + {"device", VSH_OT_DATA, VSH_OFLAG_REQ, N_("block device")}, + {"total_bytes_sec", VSH_OT_INT, VSH_OFLAG_NONE, N_("total throughput limit in bytes per second/s")}, + {"read_bytes_sec", VSH_OT_INT, VSH_OFLAG_NONE, N_("read throughput limit in bytes per second/s")}, + {"write_bytes_sec", VSH_OT_INT, VSH_OFLAG_NONE, N_("write throughput limit in bytes per second/s")}, + {"total_iops_sec", VSH_OT_INT, VSH_OFLAG_NONE, N_("total I/O operations limit per second/s")}, + {"read_iops_sec", VSH_OT_INT, VSH_OFLAG_NONE, N_("read I/O operations limit per second/s")}, + {"write_iops_sec", VSH_OT_INT, VSH_OFLAG_NONE, N_("write I/O operations limit per second/s")}, + {"config", VSH_OT_BOOL, 0, N_("affect next boot")}, + {"live", VSH_OT_BOOL, 0, N_("affect running domain")}, + {"current", VSH_OT_BOOL, 0, N_("affect current domain")}, + {NULL, 0, 0, NULL} +}; + +static bool +cmdBlkdeviotune(vshControl *ctl, const vshCmd *cmd) +{ + virDomainPtr dom = NULL; + const char *name, *disk; + virDomainBlockIoTuneInfo info; + virDomainBlockIoTuneInfo reply; + unsigned int flags = 0; + int ret = -1; + int rv = 0; + int set = 0; + int current = vshCommandOptBool(cmd, "current"); + int config = vshCommandOptBool(cmd, "config"); + int live = vshCommandOptBool(cmd, "live"); + + if (current) { + if (live || config) { + vshError(ctl, "%s", _("--current must be specified exclusively")); + return false; + } + flags = VIR_DOMAIN_AFFECT_CURRENT; + } else { + if (config) + flags |= VIR_DOMAIN_AFFECT_CONFIG; + if (live) + flags |= VIR_DOMAIN_AFFECT_LIVE; + } + + memset(&info, 0, sizeof(info)); + + if (!vshConnectionUsability(ctl, ctl->conn)) + goto out; + + if (!(dom = vshCommandOptDomain(ctl, cmd, &name))) + goto out; + + if (vshCommandOptString(cmd, "device", &disk) < 0) + goto out; + + if ((rv = vshCommandOptULongLong(cmd, "total_bytes_sec", &info.total_bytes_sec)) < 0) { + vshError(ctl, "%s", + _("Unable to parse integer parameter")); + goto out; + } else if (rv > 0) { + set++; + } + + if ((rv = vshCommandOptULongLong(cmd, "read_bytes_sec", &info.read_bytes_sec)) < 0) { + vshError(ctl, "%s", + _("Unable to parse integer parameter")); + goto out; + } else if (rv > 0) { + set++; + } + + if ((rv = vshCommandOptULongLong(cmd, "write_bytes_sec", &info.write_bytes_sec)) < 0) { + vshError(ctl, "%s", + _("Unable to parse integer parameter")); + goto out; + } else if (rv > 0) { + set++; + } + + if ((rv = vshCommandOptULongLong(cmd, "total_iops_sec", &info.total_iops_sec)) < 0) { + vshError(ctl, "%s", + _("Unable to parse integer parameter")); + goto out; + } else if (rv > 0) { + set++; + } + + if ((rv = vshCommandOptULongLong(cmd, "read_iops_sec", &info.read_iops_sec)) < 0) { + vshError(ctl, "%s", + _("Unable to parse integer parameter")); + goto out; + } else if (rv > 0) { + set++; + } + + if ((rv = vshCommandOptULongLong(cmd, "write_iops_sec", &info.write_iops_sec)) < 0) { + vshError(ctl, "%s", + _("Unable to parse integer parameter")); + goto out; + } else if (rv > 0) { + set++; + } + + if (!set) { + + ret = virDomainGetBlockIoTune(dom, disk, &reply, flags); + + if (ret != 0) + goto out; + + vshPrint(ctl, "%-15s %llu\n", _("total_bytes_sec:"), reply.total_bytes_sec); + vshPrint(ctl, "%-15s %llu\n", _("read_bytes_sec:"), reply.read_bytes_sec); + vshPrint(ctl, "%-15s %llu\n", _("write_bytes_sec:"), reply.write_bytes_sec); + vshPrint(ctl, "%-15s %llu\n", _("total_iops_sec:"), reply.total_iops_sec); + vshPrint(ctl, "%-15s %llu\n", _("read_iops_sec:"), reply.read_iops_sec); + vshPrint(ctl, "%-15s %llu\n", _("write_iops_sec:"), reply.write_iops_sec); + + virDomainFree(dom); + return true; + } else { + + ret = virDomainSetBlockIoTune(dom, disk, &info, flags); + + if (ret == 0) { + virDomainFree(dom); + return true; + } + } + +out: + virDomainFree(dom); + return false; +} + /* * "net-autostart" command @@ -14023,6 +14168,7 @@ static const vshCmdDef domManagementCmds[] = { {"blkiotune", cmdBlkiotune, opts_blkiotune, info_blkiotune, 0}, {"blockpull", cmdBlockPull, opts_block_pull, info_block_pull, 0}, {"blockjob", cmdBlockJob, opts_block_job, info_block_job, 0}, + {"blkdeviotune", cmdBlkdeviotune, opts_blkdeviotune, info_blkdeviotune, 0}, #ifndef WIN32 {"console", cmdConsole, opts_console, info_console, 0}, #endif diff --git a/tools/virsh.pod b/tools/virsh.pod index 775d302..58fcb51 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -572,6 +572,29 @@ operation can be checked with B<blockjob>. I<path> specifies fully-qualified path of the disk. I<bandwidth> specifies copying bandwidth limit in Mbps. +=item B<blkdeviotune> I<domain> I<device> [[I<--total_bytes_sec> B<total_bytes_sec>] +| [[I<--read_bytes_sec> B<read_bytes_sec>] [I<--write_bytes_sec> B<write_bytes_sec>]] +[[I<--total_iops_sec> B<total_iops_sec>] | [[I<--read_iops_sec> B<read_iops_sec>] +[I<--write_iops_sec> B<write_iops_sec>]] [[I<--config>] [I<--live>] | [I<--current>]] + +Set or query the block disk io limits settting. +I<path> specifies block disk name. +I<--total_bytes_sec> specifies total throughput limit in bytes per second/s. +I<--read_bytes_sec> specifies read throughput limit in bytes per second/s. +I<--write_bytes_sec> specifies write throughput limit in bytes per second/s. +I<--total_iops_sec> specifies total I/O operations limit per second/s. +I<--read_iops_sec> specifies read I/O operations limit per second/s. +I<--write_iops_sec> specifies write I/O operations limit per second/s. + +If I<--live> is specified, affect a running guest. +If I<--config> is specified, affect the next boot of a persistent guest. +If I<--current> is specified, affect the current guest state. +Both I<--live> and I<--current> flags may be given, but I<--current> is +exclusive. If no flag is specified, behavior is different depending +on hypervisor. + +If no limit is specified, it will query current I/O limits setting. + =item B<blockjob> I<domain> I<path> [I<--abort>] [I<--info>] [I<bandwidth>] Manage active block operations. -- 1.7.1

2 1

[libvirt] [PATCH 4/8] Support block I/O throtte in XML
by Lei Li 10 Nov '11

10 Nov '11

Enable block I/O throttle for per-disk in XML. Signed-off-by: Zhi Yong Wu <wuzhy(a)linux.vnet.ibm.com> Signed-off-by: Lei Li <lilei(a)linux.vnet.ibm.com> --- src/conf/domain_conf.c | 101 +++++++++++++++++++++++++++++++++++++++++++++- src/conf/domain_conf.h | 12 ++++++ src/qemu/qemu_command.c | 33 +++++++++++++++ src/util/xml.h | 2 + 4 files changed, 145 insertions(+), 3 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 58f4d0f..564914e 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -2318,7 +2318,8 @@ static virDomainDiskDefPtr virDomainDiskDefParseXML(virCapsPtr caps, xmlNodePtr node, virBitmapPtr bootMap, - unsigned int flags) + unsigned int flags, + xmlXPathContextPtr ctxt) { virDomainDiskDefPtr def; xmlNodePtr cur, child; @@ -2517,6 +2518,62 @@ virDomainDiskDefParseXML(virCapsPtr caps, } child = child->next; } + } else if (xmlStrEqual(cur->name, BAD_CAST "iotune")) { + if (virXPathULongLong("string(./devices/disk/iotune/total_bytes_sec)", + ctxt, &def->blkdeviotune.total_bytes_sec) < 0) { + def->blkdeviotune.total_bytes_sec = 0; + } else { + def->blkdeviotune.mark = 1; + } + + if (virXPathULongLong("string(./devices/disk/iotune/read_bytes_sec)", + ctxt, &def->blkdeviotune.read_bytes_sec) < 0) { + def->blkdeviotune.read_bytes_sec = 0; + } else { + def->blkdeviotune.mark = 1; + } + + if (virXPathULongLong("string(./devices/disk/iotune/write_bytes_sec)", + ctxt, &def->blkdeviotune.write_bytes_sec) < 0) { + def->blkdeviotune.write_bytes_sec = 0; + } else { + def->blkdeviotune.mark = 1; + } + + if (virXPathULongLong("string(./devices/disk/iotune/total_iops_sec)", + ctxt, &def->blkdeviotune.total_iops_sec) < 0) { + def->blkdeviotune.total_iops_sec = 0; + } else { + def->blkdeviotune.mark = 1; + } + + if (virXPathULongLong("string(./devices/disk/iotune/read_iops_sec)", + ctxt, &def->blkdeviotune.read_iops_sec) < 0) { + def->blkdeviotune.read_iops_sec = 0; + } else { + def->blkdeviotune.mark = 1; + } + + if (virXPathULongLong("string(./devices/disk/iotune/write_iops_sec)", + ctxt, &def->blkdeviotune.write_iops_sec) < 0) { + def->blkdeviotune.write_iops_sec = 0; + } else { + def->blkdeviotune.mark = 1; + } + + if ((def->blkdeviotune.total_bytes_sec && def->blkdeviotune.read_bytes_sec) + || (def->blkdeviotune.total_bytes_sec && def->blkdeviotune.write_bytes_sec)) { + virDomainReportError(VIR_ERR_XML_ERROR, + _("bps and bps_rd/bps_wr cannot be set at the same time")); + goto error; + } + + if ((def->blkdeviotune.total_iops_sec && def->blkdeviotune.read_iops_sec) + || (def->blkdeviotune.total_iops_sec && def->blkdeviotune.write_iops_sec)) { + virDomainReportError(VIR_ERR_XML_ERROR, + _("iops and iops_rd/iops_wr cannot be set at the same time")); + goto error; + } } else if (xmlStrEqual(cur->name, BAD_CAST "readonly")) { def->readonly = 1; } else if (xmlStrEqual(cur->name, BAD_CAST "shareable")) { @@ -6003,7 +6060,7 @@ virDomainDeviceDefPtr virDomainDeviceDefParse(virCapsPtr caps, if (xmlStrEqual(node->name, BAD_CAST "disk")) { dev->type = VIR_DOMAIN_DEVICE_DISK; if (!(dev->data.disk = virDomainDiskDefParseXML(caps, node, - NULL, flags))) + NULL, flags, NULL))) goto error; } else if (xmlStrEqual(node->name, BAD_CAST "lease")) { dev->type = VIR_DOMAIN_DEVICE_LEASE; @@ -7076,7 +7133,8 @@ static virDomainDefPtr virDomainDefParseXML(virCapsPtr caps, virDomainDiskDefPtr disk = virDomainDiskDefParseXML(caps, nodes[i], bootMap, - flags); + flags, + ctxt); if (!disk) goto error; @@ -9511,6 +9569,43 @@ virDomainDiskDefFormat(virBufferPtr buf, virBufferAsprintf(buf, " <target dev='%s' bus='%s'/>\n", def->dst, bus); + /*disk I/O throttling*/ + if (def->blkdeviotune.mark) { + virBufferAddLit(buf, " <iotune>\n"); + if (def->blkdeviotune.total_bytes_sec) { + virBufferAsprintf(buf, " <total_bytes_sec>%llu</total_bytes_sec>\n", + def->blkdeviotune.total_bytes_sec); + } + + if (def->blkdeviotune.read_bytes_sec) { + virBufferAsprintf(buf, " <read_bytes_sec>%llu</read_bytes_sec>\n", + def->blkdeviotune.read_bytes_sec); + + } + + if (def->blkdeviotune.write_bytes_sec) { + virBufferAsprintf(buf, " <write_bytes_sec>%llu</write_bytes_sec>\n", + def->blkdeviotune.write_bytes_sec); + } + + if (def->blkdeviotune.total_iops_sec) { + virBufferAsprintf(buf, " <total_iops_sec>%llu</total_iops_sec>\n", + def->blkdeviotune.total_iops_sec); + } + + if (def->blkdeviotune.read_iops_sec) { + virBufferAsprintf(buf, " <read_iops_sec>%llu</read_iops_sec>", + def->blkdeviotune.read_iops_sec); + } + + if (def->blkdeviotune.write_iops_sec) { + virBufferAsprintf(buf, " <write_iops_sec>%llu</write_iops_sec>", + def->blkdeviotune.write_iops_sec); + } + + virBufferAddLit(buf, " </iotune>\n"); + } + if (def->bootIndex) virBufferAsprintf(buf, " <boot order='%d'/>\n", def->bootIndex); if (def->readonly) diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index a3cb834..d95e239 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -333,6 +333,18 @@ struct _virDomainDiskDef { } auth; char *driverName; char *driverType; + + /*disk I/O throttling*/ + struct { + unsigned long long total_bytes_sec; + unsigned long long read_bytes_sec; + unsigned long long write_bytes_sec; + unsigned long long total_iops_sec; + unsigned long long read_iops_sec; + unsigned long long write_iops_sec; + unsigned int mark; + } blkdeviotune; + char *serial; int cachemode; int error_policy; /* enum virDomainDiskErrorPolicy */ diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 11ebb69..3a8575b 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1715,6 +1715,39 @@ qemuBuildDriveStr(virConnectPtr conn ATTRIBUTE_UNUSED, } } + /*block I/O throttling*/ + if (disk->blkdeviotune.mark) { + if (disk->blkdeviotune.total_bytes_sec) { + virBufferAsprintf(&opt, ",bps=%llu", + disk->blkdeviotune.total_bytes_sec); + } + + if (disk->blkdeviotune.read_bytes_sec) { + virBufferAsprintf(&opt, ",bps_rd=%llu", + disk->blkdeviotune.read_bytes_sec); + } + + if (disk->blkdeviotune.write_bytes_sec) { + virBufferAsprintf(&opt, ",bps_wr=%llu", + disk->blkdeviotune.write_bytes_sec); + } + + if (disk->blkdeviotune.total_iops_sec) { + virBufferAsprintf(&opt, ",iops=%llu", + disk->blkdeviotune.total_iops_sec); + } + + if (disk->blkdeviotune.read_iops_sec) { + virBufferAsprintf(&opt, ",iops_rd=%llu", + disk->blkdeviotune.read_iops_sec); + } + + if (disk->blkdeviotune.write_iops_sec) { + virBufferAsprintf(&opt, ",iops_wr=%llu", + disk->blkdeviotune.write_iops_sec); + } + } + if (virBufferError(&opt)) { virReportOOMError(); goto error; diff --git a/src/util/xml.h b/src/util/xml.h index c492063..5742f19 100644 --- a/src/util/xml.h +++ b/src/util/xml.h @@ -50,6 +50,8 @@ xmlNodePtr virXPathNode(const char *xpath, int virXPathNodeSet(const char *xpath, xmlXPathContextPtr ctxt, xmlNodePtr **list); +int virXMLStringToULongLong (const char *stringval, + unsigned long long *value); char * virXMLPropString(xmlNodePtr node, const char *name); -- 1.7.1

2 1