[libvirt] [RFC] block I/O throttling: how to enable in libvirt

Subject: Re: The design choice for how to enable block I/O throttling function in libvirt From: Stefan Hajnoczi <stefanha(a)gmail.com&gt; To: Adam Litke <agl(a)us.ibm.com&gt; Cc: libvir-list(a)redhat.com, "Daniel P. Berrange" <berrange(a)redhat.com&gt;, Zhi Yong Wu <wuzhy(a)linux.vnet.ibm.com&gt;, Zhi Yong Wu <zwu.kernel(a)gmail.com&gt; Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Brightmail-Tracker: AAAAAA== X-Xagent-From: stefanha(a)gmail.com X-Xagent-To: wuzhy(a)linux.vnet.ibm.com X-Xagent-Gateway: bldgate.vnet.ibm.com (XAGENTU7 at BLDGATE) On Tue, Aug 30, 2011 at 2:46 PM, Adam Litke <agl(a)us.ibm.com&gt; wrote: > On Tue, Aug 30, 2011 at 09:53:33AM +0100, Stefan Hajnoczi wrote: >> On Tue, Aug 30, 2011 at 3:55 AM, Zhi Yong Wu <zwu.kernel(a)gmail.com&gt; wrote: >> > I am trying to enable block I/O throttling function in libvirt. But >> > currently i met some design questions, and don't make sure if we >> > should extend blkiotune to support block I/O throttling or introduce >> > one new libvirt command "blkiothrottle" to cover it or not. If you >> > have some better idea, pls don't hesitate to drop your comments. >> >> A little bit of context: this discussion is about adding libvirt >> support for QEMU disk I/O throttling. > > Thanks for the additional context Stefan. > >> Today libvirt supports the cgroups blkio-controller, which handles >> proportional shares and throughput/iops limits on host block devices. >> blkio-controller does not support network file systems (NFS) or other >> QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are >> not host block devices.  QEMU I/O throttling works with all types of >> -drive and therefore complements blkio-controller. > > The first question that pops into my mind is: Should a user need to understand > when to use the cgroups blkio-controller vs. the QEMU I/O throttling method?  In > my opinion, it would be nice if libvirt had a single interface for block I/O > throttling and libvirt would decide which mechanism to use based on the type of > device and the specific limits that need to be set. Yes, I agree it would be simplest to pick the right mechanism, depending on the type of throttling the user wants. More below. >> I/O throttling can be applied independently to each -drive attached to >> a guest and supports throughput/iops limits.  For more information on >> this QEMU feature and a comparison with blkio-controller, see Ryan >> Harper's KVM Forum 2011 presentation: > >> http://www.linux-kvm.org/wiki/images/7/72/2011-forum-keep-a-limit-on-it-i... > > From the presentation, it seems that both the cgroups method the the qemu method > offer comparable control (assuming a block device) so it might possible to apply > either method from the same API in a transparent manner.  Am I correct or are we > suggesting that the Qemu throttling approach should always be used for Qemu > domains? QEMU I/O throttling does not provide a proportional share mechanism. So you cannot assign weights to VMs and let them receive a fraction of the available disk time. That is only supported by cgroups blkio-controller because it requires a global view which QEMU does not have. So I think the two are complementary: If proportional share should be used on a host block device, use cgroups blkio-controller. Otherwise use QEMU I/O throttling.

Stefan