Re: [libvirt] PATCH: Disable QEMU drive caching

Daniel P. Berrange wrote: >On Wed, Oct 08, 2008 at 01:15:46PM +0200, Chris Lalancette wrote: >>Daniel P. Berrange wrote: >>>QEMU defaults to allowing the host OS to cache all disk I/O. THis has a >>>couple of problems >>> >>> - It is a waste of memory because the guest already caches I/O ops >>> - It is unsafe on host OS crash - all unflushed guest I/O will be >>> lost, and there's no ordering guarentees, so metadata updates could >>> be flushe to disk, while the journal updates were not. Say goodbye >>> to your filesystem. >>> - It makes benchmarking more or less impossible / worthless because >>> what the benchmark things are disk writes just sit around in memory >>> so guest disk performance appears to exceed host diskperformance. >>> >>>This patch disables caching on all QEMU guests. NB, Xen has long done >>>this >>>for both PV & HVM guests - QEMU only gained this ability when -drive was >>>introduced, and sadly kept the default to unsafe cache=on settings. >>I'm for this in general, but I'm a little worried about the "performance >>regression" aspect of this. People are going to upgrade to 0.4.7 (or >>whatever), >>and suddenly find that their KVM guests perform much more slowly. This is >>better in the end for their data, but we might hear large complaints >>about it. > >Yes & no. They will find their guests perform more consistently. With the >current system their guests will perform very erratically depending on >memory & I/O pressure on the host. If the host I/O cache is empty & has >no I/O load, current guests will be "fast", They will perform marginally better than if cache=off. This is the Linux host knows more about the underlying hardware than the guest and is able to do smarter read-ahead. When using cache=off, the host cannot perform any sort of read-ahead. >but if host I/O cache is full >and they do something which requires more host memory (eg start up another >guest), then all existing guests get their I/O performance trashed as the >I/O cache has to be flushed out, and future I/O is unable to be cached. This is not accurate. Dirty pages in the host page cache are not reclaimable until they're written to disk. If you're in a seriously low memory situation, they the thing allocating memory is going to sleep until the data is written to disk. If an existing guest is trying to do I/O, then what things will degenerate to is basically cache=off since the guest must wait for other pending IO to complete >Xen went through this same change and there were not any serious >complaints, particularly when explained that previous system had >zero data integrity guarentees. The current system merely provides an >illusion of performance - any attempt to show that performance has >decreased is impossible because any attempt to run benchmarks with >existing caching just results in meaningless garbage. > >https://bugzilla.redhat.com/show_bug.cgi?id=444047 I can't see this bug, but a quick grep of ioemu in xen-unstable for O_DIRECT reveals that they are not in fact using O_DIRECT.