Adding qemu-block list. I'll summarize the question for qemu folks at the end. On Fri, Dec 12, 2025 at 10:00:17 -0300, João Jandre wrote:
Hi, all
I've been reading the QEMU documentation, the Libvirt documentation, as well as doing some testing and code investigation regarding disk cache modes.
In the Libvirt documentation, it is stated that the disk driver cache modes accepted are "default", "none", "writethrough", "writeback", "directsync" and "unsafe". Analyzing the code, we can see that this is somewhat reflected on the cache modes that QEMU documents on the -drive option (https://qemu-project.gitlab.io/qemu/system/invocation.html): https://github.com/libvirt/libvirt/blob/04c1f458313e9001f5a804a898408e1f4985...
The intention behind that function was to preserve the behaviour from the time when libvirt was using -drive when I was converting libvirt to use -blockdev instead where the cache mode is configured by the 3 bool options.
However, the default option (when we do not define the "cache" option) is different from other options and does not seem to map to any option on the
While I no longer remeber the specifics (it's beel already almost 8 years since I've wrote the code) the code was designed to avoid setting any of the cache options in qemu (thus honour the 'hypervisor default'-type behaviour). Specifically the 'write-cache' option for the disk frontend device is defined as an "OnOffAuto" option and defaults to 'auto', which means we do set it to "default", whatever that means. Now the comment in the aforementioned function which I've copied from the qemu documentation doesn't have the line: "The default mode is cache=writeback." I unfortunately don't remember why thoug. It was present at the time I was copying it from there based on the commit history.
Qemu side. I see that, from testing and analyzing the code, if the default (not defining the "cache" options) is used, cache.direct and cache.no-flush are not informed; also, the write-cache option is not informed.
Just for comparison, here is a disk defined with cache=writeback:
XML:
|<disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='writeback'/> <source file='/mnt/a66f48f5-13c6-3676-b320-a15d32bbf32f/c0e925a9-6697-4ee9-8abf-d36c7a95cea4' index='2'/> <backingStore type='file' index='3'> <format type='qcow2'/> <source file='/mnt/a66f48f5-13c6-3676-b320-a15d32bbf32f/0db52809-9f33-43ac-8151-5452e9dee330'/> <backingStore/> </backingStore> <target dev='vda' bus='virtio'/> <serial>c0e925a966974ee98abf</serial> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> |
Generated QEMU command:
|-blockdev {"driver":"file","filename":"/mnt/a66f48f5-13c6-3676-b320-a15d32bbf32f/c0e925a9-6697-4ee9-8abf-d36c7a95cea4","node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"qcow2","file":"libvirt-2-storage","backing":"libvirt-3-format"} -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=libvirt-2-format,id=virtio-disk0,bootindex=2,write-cache=on,serial=c0e925a966974ee98abf |
And here is a disk that does not have the cache option:
XML:
|<disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/mnt/b59eff0f-5b97-37ed-a513-9e1983d1d19b/b430296b-0924-412e-a7b1-ddc3d4c90f83' index='2'/> <backingStore/> <target dev='vdc' bus='virtio'/> <serial>b430296b0924412ea7b1</serial> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </disk> |
Generated QEMU command:
|blockdev {"driver":"file","filename":"/mnt/b59eff0f-5b97-37ed-a513-9e1983d1d19b/b430296b-0924-412e-a7b1-ddc3d4c90f83","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":false,"driver":"qcow2","file":"libvirt-2-storage","backing":null} -device virtio-blk-pci,bus=pci.0,addr=0x8,drive=libvirt-2-format,id=virtio-disk2,serial=b430296b0924412ea7b1 |
Indeed the differences in the commandlines what you see here are "itended" (in terms that I've likely wanted qemu to use the 'auto' mode of 'write-cache' whatever it is.).
The QEMU documentation states in the -drive option that the default cache mode is |Writeback| (see https://qemu-project.gitlab.io/qemu/system/invocation.html), so I expected that setting |Writeback| or |Default| as the cache mode on Libvirt would also be the same. I ran some performance tests using both options to try and verify if they were the same, but they are clearly different. For each of the configuration profiles below, I created a VM and ran some fio tests:
Combination Disk cache mode Disk Controller IO thread IO policy hds0i default virtio-scsi 0 iouring hds0t default virtio-scsi 0 threads hds1i default virtio-scsi 1 iouring hds1t default virtio-scsi 1 threads hdv0i default virtio 0 iouring hdv0t default virtio 0 threads hdv1i default virtio 1 iouring hdv1t default virtio 1 threads wbs0i writeback virtio-scsi 0 iouring wbs0t writeback virtio-scsi 0 threads wbs1i writeback virtio-scsi 1 iouring wbs1t writeback virtio-scsi 1 threads wbv0i writeback virtio 0 iouring wbv0t writeback virtio 0 threads wbv1i writeback virtio 1 iouring wbv1t writeback virtio 1 threads
The fio test results are in a html attached to the email.
I would like to understand what the practical difference is between not informing the cache and any other option. I can see that the disk is defined in different way, but just by analyzing the code I was not able to understand what that leads to; also, looking at the performance tests, we clearly see a difference in the results. Would anyone be able to explain what is the expected behavior when the disk is defined in such a way?
By the way, I'm using Libvirt 8.0.0 and Qemu 6.2.0.
Libvirt uses the '-blockdev' mode and thus the above logic to pick the cache mode starting with qemu-4.2. In fact the support for '-drive' mode is no longer present in current upstream as we no-longer support anything older than qemu-6.2 thus always use the -blockdev mode.a While the _DEFAULT mode surely gives us the possibility to pick another combination of parameters, qemu-4.2 and thus our switch to -blockdev was 6 years ago. This gives a very long time when this combination of parameters was used and so I'm wary about changing the mode, unless it's bad for some reason. I'll thus CC the qemu folks with this message. TL;DR for qemu-block: When libvirt switched to '-blockdev' mode I've added a function which maps the cache modes used with '-drive' to the 3 options with blockdev. I for some reason chose that the "VIR_DOMAIN_DISK_CACHE_DEFAULT" mode is that *all* of the cache mode flags are just omitted relying on the default in qemu rather than going with the documented 'writeback' mode setting as default. Is the above combination problematic in any way? Should libvirt change the "default"?