Re: [libvirt] Performance about x-data-plane

On Tue, Jan 03, 2017 at 12:02:14PM -0500, Weiwei Jia wrote: > > The expensive part is the virtqueue kick. Recently we tried polling the > > virtqueue instead of waiting for the ioeventfd file descriptor and got > > double-digit performance improvements: > > https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg00148.html > > > > If you want to understand the performance of your benchmark you'll have > > compare host/guest disk stats (e.g. request lifetime, disk utilization, > > queue depth, average request size) to check that the bare metal and > > guest workloads are really sending comparable I/O patterns to the > > physical disk. > > > > Then you using Linux and/or QEMU tracing to analyze the request latency > > by looking at interesting points in the request lifecycle like virtqueue > > kick, host Linux AIO io_submit(2), etc. > > > > Thank you. I will look into "polling the virtqueue" as you said above. > Currently, I just use blktrace to see disk stats and add logs in the > I/O workload to see the time latency for each request. What kind of > tools are you using to analyze request lifecycle like virtqueue kick, > host Linux AIO iosubmit, etc. > > Do you trace the lifecycle like this > (http://www.linux-kvm.org/page/Virtio/Block/Latency#Performance_data) > but it seems to be out of date. Does it > (http://repo.or.cz/qemu-kvm/stefanha.git/shortlog/refs/heads/tracing-dev-0...) > still work on QEMU 2.4.1? The details are out of date but the general approach to tracing the I/O request lifecycle still apply. There are multiple tracing tools that can do what you need. I've CCed Karl Rister who did the latest virtio-blk dataplane tracing. "perf record -a -e kvm:\*" is a good start. You can use "perf probe" to trace QEMU's trace events (recent versions have sdt support, which means SystemTap tracepoints work) and also trace any function in QEMU: http://blog.vmsplice.net/2011/03/how-to-use-perf-probe.html