Re: [libvirt-users] libvirt possibly ignoring cache=none ?

On 08/08/2013 05:03 PM, Brano Zarnovican wrote: > On Thu, Aug 8, 2013 at 9:39 AM, Martin Kletzander <mkletzan(a)redhat.com&gt; wrote: >> At first let me explain that libvirt is not ignoring the cache=none. >> This is propagated to qemu as a parameter for it's disk. From qemu's >> POV (anyone feel free to correct me if I'm mistaken) this means the file >> is opened with O_DIRECT flag; and from the open(2) manual, the O_DIRECT >> means "Try to minimize cache effects of the I/O to and from this >> file...", that doesn't necessarily mean there is no cache at all. > > Thanks for explanation. > >> But even if it does, this applies to files used as disks, but those >> disks are not the only files the process is using. You can check what >> othe files the process has mapped, opened etc. from the '/proc' >> filesystem or using the 'lsof' utility. All the other files can (and >> probably will) take some cache and there is nothing wrong with that. > > In my case there was 4GB of caches. > > Just now, I have thrashed one instance with many read/writes on > various devices. In total, tens of GB of data. But the cache (on host) > did not grow beyond 3MB. I'm not yet able to reproduce the problem. > >> Are you trying to resolve an issue or asking just out of curiosity? >> Because this is wanted behavior and there should be no need for anyone >> to minimize this. > > Once or twice, one of our VMs was OOM killed because it reached 1.5 * > memory limit for its cgroup. > Oh, please report this to us. This is one of the problems we'll be, unfortunately, dealing with forever, I guess. This limit is just a "guess" how much qemu might take and we're setting it to make sure host is not overwhelmed in case qemu is faulty/hacked. Since this isn't ever possible to set exactly, it already happened that thanks to cgroups, qemu was killed, so we had to increase the limit. I Cc'd Michal who might be the right person to know about any further increase.