Re: [PATCH 0/1] Support cloning saved VMs

On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote: > > >> On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >> >> On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote: >>> >>> >>>> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>>> >>>> On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >>>>> Hello! >>>>> >>>>> I have a use case which I'm struggling to support with libvirt: >>>>> saving a VM to a file, cloning it (which renames the VM), and restoring it. >>>>> >>>>> My search revealed a number of tutorials for using virt-clone [1], but that >>>>> doesn't seem to cover VMs which are _saved_ (only running or paused). >>>> >>>> Saved in what way ? Managed save ? >>> >>> Thanks for the prompt reply! >>> >>> I'm saving with virDomainSave(). My understanding is that this is not managed. >> >> Functionally it is the same as managed save, just the that file path >> is specified by the client, rather than by libvirt. > > Got it, thanks. > >>>>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >>>>> >>>>> In a nutshell, I want to power on a VM and do some setup, then save its full >>>>> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >>>>> - rename the VM >>>>> - change which bridge its NICs are on (while maintaining mac addresses) >>>>> - change the disk image to a copy (done while the VM is saved) >>>>> >>>>> But the restore operation fails because of a target domain name check >>>>> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >>>>> address this and I'm looking for your views. >>>> >>>> If you're cloning a VM, it needs both a new UUID and name, so I'm surprised >>>> the ABI stability check hasn't already blocked you on the UUID change before >>>> getting to the name change check. >>> >>> I definitely didn't change the UUID. In fact, I want it to be the same (at >>> least in the SMBIOS tables) because the guest OS is not going to expect that >>> value to change without a power cycle/reset. The ABI Check actually ensures the >>> SMBIOS values do not change during restore. >>> >>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... >>> >>> My understanding is that this passed because the other domain was not running >>> (and the save was unmanaged, so libvirt is unaware of the saved VM). >>> >>> What I don't understand is why the UUID has to be unique (or, in fact, the same >>> as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For >>> the clone use case, I surely don't want this to change. >>> >>> In other words, it's not clear to me why this check is needed: >>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... >> >> Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The >> latter is only for running VMs. >> >> UUID is the primary unique identifier that is used for pretty much every >> lookup inside libvirt. Name is a secondary unique identifier largely just >> for external lookups by humans, since UUIDs are not human friendly. >> >> Essentially every API call starts with virDomainObjListFindByUUID to convert >> the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct >> that holds the config & state. > > Ah-ha. Ok, this is really helpful, thanks again! > > My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? > (I'm pointing again at L#12810 above.) > > That feels incorrect. My (new) understanding is that: > - The SMBIOS Type 1 UUID is guest-visible > - The Libvirt UUID is a host identifier > > What comes to mind is that maybe something like guest tools wants to be able to > report back to a control plane what VM it is on based on this value. If that's > the motivation, then isn't Generation ID a better field to rely on? Strictly speaking we don't have to tie them together, but in practice we do, because it is pretty compelling to be correlate data between the host OS and guest OS for apps.

> My understanding is that SMBIOS identifiers cannot change at runtime. Correct.

>> The ABI stability checks are done in places where we allow the XML of an >> existing VM to be replaced, and that doesn't involve calls to virDomainObjListAdd >> since we're not creating a new VM, just altering its backend config. > > So maybe the problem I have is that I'm not modifying this data when I clone > the saved VM? To recap, I'm calling virDomainSave() and giving it a path. The > path creates a file named after the VM. I'm just renaming that file. Maybe I > should be modifying the XML embedded in that file somehow? > >> >>>> Likewise for UUID checks. >>> >>> I still don't understand how UUID is used, so clarification/pointers welcome! >>> >>>>> I'm open to suggestions, for example by plumbing through a flag which makes the >>>>> check optional. Please let me know how you prefer that I take this forward. >>>> >>>> If you're using managed save, then I would think it is already possible to >>>> achieve. >>>> >>>> First cloning the VM: >>>> >>>> virsh dumpxml myvm > myvm.xml >>>> cp myvm.xml myvm-clone.xml >>>> ..modify name & uuid & bridge & disk of myvm-clone.xml >>>> virsh define myvm-clone.xml >>>> >>>> Now modify the saved state >>>> >>>> virsh managedsave-dumpxml myvm-clone > save.xml >>>> ...modify save.xml to change name & uuid to match myvm-clonme... >>>> virsh managedsave-define save.xml >>>> >>>> The ABI stability check done by managedsave-define, will validate against >>>> the name + uuid of the cloned VM, so in fact it will force you to change >>>> the name + uuid in the save XML before loading it back in, and it should >>>> be not possible to restore from the managed save image until fixing this >>> >>> Thanks for these pointers; I'll look into them. But I think my workflows are >>> not managed as these VMs are not persistent on the host. When they die, they >>> may be restarted elsewhere by a higher-level (multi-host) control plane. >> >> There are equivalent save-image-define / save-image-dumpxml for the >> non-managed save scenario. > > And presumably this is how I get a saved XML which I can then modify? Yes, that's the supported way of modifying the XML inside a save image, other than editting the file directly (please don't :-).