Re: [libvirt RFCv11 00/33] multifd save restore prototype

On Mon, Nov 27, 2023 at 10:43:58AM +0100, Claudio Fontana wrote: > Hi all, > > I understand there has been some movement in this topic as the fixed-offset ram and multifd code evolves. > > I think I understood that now the idea is to pass from libvirt to QEMU two file descriptors, > one for writing metadata, > and a separate one for the actual memory pages, which is the one that can potentially be O_DIRECT. We determined that O_DIRECT changes propagate across dup'd file descriptors, so we have only two choices * 1 FD, and QEMU has to take care to toggle O_DIRECT on/off repeatedly depending on what phase it is in * 2 FDs, one with and one without O_DIRECT Either is viable for libvirt. I have a mild preference for having 1 FD, but not enough to call it a design blocker. So at the discretion of whomever implements the QEMU part. > 1) I would assume that libvirt would then need to check if the user requested --parallel / --parallel-connections to enable QEMU multifd. Yes > 2) I would also assume that libvirt would check the presence of --bypass-cache as the condition to set O_DIRECT on the second (memory pages fd), and to enable QEMU "io-direct" feature. Yes > 3) I would tentatively suggest that when it comes to fixed-ram-offset, the condition to enable that one is a check like the one currently in libvirt: > > src/util/virfile.c::virFileDiskCopy() > > ie checking that we are writing to a seekable file that is not S_ISBLK . > > Does this match your understanding/reasoning? Both the io-direct and fixed-ram-offset features are dependent on new QEMU impls, so there is a mild backwards compatibility concern. ie, lets say if we are running QEMU 9.0.0, but with old machine type pc-i440fx-8.0.0, and we save the state, but want to then restore with QEMU 8.2.0. Essentially we must NOT use io-direct/fixed-ram-offset if we want the ability to migrate to older QEMU. At the same time I would like us to be able to take advantage of this new QEMU support to the greatest extent possible, even if not doing the --parallel stuff which was the original motivator. Thus, we need some way to decide whether to use the new or the old on disk format. I wonder if having a setting in '/etc/libvirt/qemu.conf' is sufficient, or whether we must expose a flag via the API. With regards, Daniel