Hello! I have a use case which I'm struggling to support with libvirt: saving a VM to a file, cloning it (which renames the VM), and restoring it. My search revealed a number of tutorials for using virt-clone [1], but that doesn't seem to cover VMs which are _saved_ (only running or paused).

[1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py In a nutshell, I want to power on a VM and do some setup, then save its full state to disk (e.g., with virsh save). Finally I want to modify the XML to: - rename the VM - change which bridge its NICs are on (while maintaining mac addresses) - change the disk image to a copy (done while the VM is saved) But the restore operation fails because of a target domain name check implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best address this and I'm looking for your views.

I suspect this call is there for a reason (which may still be relevant), although the name is clearly not part of the ABI; it's the host identifier for that domain and not guest-visible. My first stab at this is therefore just to drop this check (patch attached).

I'm open to suggestions, for example by plumbing through a flag which makes the check optional. Please let me know how you prefer that I take this forward.

> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: > > On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >> Hello! >> >> I have a use case which I'm struggling to support with libvirt: >> saving a VM to a file, cloning it (which renames the VM), and restoring it. >> >> My search revealed a number of tutorials for using virt-clone [1], but that >> doesn't seem to cover VMs which are _saved_ (only running or paused). > > Saved in what way ? Managed save ? Thanks for the prompt reply! I'm saving with virDomainSave(). My understanding is that this is not managed.

>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >> >> In a nutshell, I want to power on a VM and do some setup, then save its full >> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >> - rename the VM >> - change which bridge its NICs are on (while maintaining mac addresses) >> - change the disk image to a copy (done while the VM is saved) >> >> But the restore operation fails because of a target domain name check >> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >> address this and I'm looking for your views. > > If you're cloning a VM, it needs both a new UUID and name, so I'm surprised > the ABI stability check hasn't already blocked you on the UUID change before > getting to the name change check. I definitely didn't change the UUID. In fact, I want it to be the same (at least in the SMBIOS tables) because the guest OS is not going to expect that value to change without a power cycle/reset. The ABI Check actually ensures the SMBIOS values do not change during restore. https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... My understanding is that this passed because the other domain was not running (and the save was unmanaged, so libvirt is unaware of the saved VM). What I don't understand is why the UUID has to be unique (or, in fact, the same as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For the clone use case, I surely don't want this to change. In other words, it's not clear to me why this check is needed: https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c...

>> I suspect this call is there for a reason (which may still be relevant), >> although the name is clearly not part of the ABI; it's the host identifier for >> that domain and not guest-visible. My first stab at this is therefore just to >> drop this check (patch attached). > > The most important thing is that Libvirt has to ensure uniqueness of the > name, within the host. If the name can be silently changed by passing > in change XML, the unique checks will be missing and you can end up with > many VMs with the same name. Sure, but that's different than checking source is the same as destination. Isn't a check of domain name uniqueness within the host better done elsewhere? Maybe as part of domainRestore() / domainCreate() / domainRename() (or some common higher-level ground?).

> Likewise for UUID checks. I still don't understand how UUID is used, so clarification/pointers welcome! >> I'm open to suggestions, for example by plumbing through a flag which makes the >> check optional. Please let me know how you prefer that I take this forward. > > If you're using managed save, then I would think it is already possible to > achieve. > > First cloning the VM: > > virsh dumpxml myvm > myvm.xml > cp myvm.xml myvm-clone.xml > ..modify name & uuid & bridge & disk of myvm-clone.xml > virsh define myvm-clone.xml > > Now modify the saved state > > virsh managedsave-dumpxml myvm-clone > save.xml > ...modify save.xml to change name & uuid to match myvm-clonme... > virsh managedsave-define save.xml > > The ABI stability check done by managedsave-define, will validate against > the name + uuid of the cloned VM, so in fact it will force you to change > the name + uuid in the save XML before loading it back in, and it should > be not possible to restore from the managed save image until fixing this Thanks for these pointers; I'll look into them. But I think my workflows are not managed as these VMs are not persistent on the host. When they die, they may be restarted elsewhere by a higher-level (multi-host) control plane.

F.

> On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: > > On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote: >> >> >>> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>> >>> On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >>>> Hello! >>>> >>>> I have a use case which I'm struggling to support with libvirt: >>>> saving a VM to a file, cloning it (which renames the VM), and restoring it. >>>> >>>> My search revealed a number of tutorials for using virt-clone [1], but that >>>> doesn't seem to cover VMs which are _saved_ (only running or paused). >>> >>> Saved in what way ? Managed save ? >> >> Thanks for the prompt reply! >> >> I'm saving with virDomainSave(). My understanding is that this is not managed. > > Functionally it is the same as managed save, just the that file path > is specified by the client, rather than by libvirt. Got it, thanks. >>>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >>>> >>>> In a nutshell, I want to power on a VM and do some setup, then save its full >>>> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >>>> - rename the VM >>>> - change which bridge its NICs are on (while maintaining mac addresses) >>>> - change the disk image to a copy (done while the VM is saved) >>>> >>>> But the restore operation fails because of a target domain name check >>>> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >>>> address this and I'm looking for your views. >>> >>> If you're cloning a VM, it needs both a new UUID and name, so I'm surprised >>> the ABI stability check hasn't already blocked you on the UUID change before >>> getting to the name change check. >> >> I definitely didn't change the UUID. In fact, I want it to be the same (at >> least in the SMBIOS tables) because the guest OS is not going to expect that >> value to change without a power cycle/reset. The ABI Check actually ensures the >> SMBIOS values do not change during restore. >> >> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... >> >> My understanding is that this passed because the other domain was not running >> (and the save was unmanaged, so libvirt is unaware of the saved VM). >> >> What I don't understand is why the UUID has to be unique (or, in fact, the same >> as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For >> the clone use case, I surely don't want this to change. >> >> In other words, it's not clear to me why this check is needed: >> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... > > Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The > latter is only for running VMs. > > UUID is the primary unique identifier that is used for pretty much every > lookup inside libvirt. Name is a secondary unique identifier largely just > for external lookups by humans, since UUIDs are not human friendly. > > Essentially every API call starts with virDomainObjListFindByUUID to convert > the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct > that holds the config & state. Ah-ha. Ok, this is really helpful, thanks again! My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? (I'm pointing again at L#12810 above.) That feels incorrect. My (new) understanding is that: - The SMBIOS Type 1 UUID is guest-visible - The Libvirt UUID is a host identifier What comes to mind is that maybe something like guest tools wants to be able to report back to a control plane what VM it is on based on this value. If that's the motivation, then isn't Generation ID a better field to rely on?

My understanding is that SMBIOS identifiers cannot change at runtime.

> The ABI stability checks are done in places where we allow the XML of an > existing VM to be replaced, and that doesn't involve calls to virDomainObjListAdd > since we're not creating a new VM, just altering its backend config. So maybe the problem I have is that I'm not modifying this data when I clone the saved VM? To recap, I'm calling virDomainSave() and giving it a path. The path creates a file named after the VM. I'm just renaming that file. Maybe I should be modifying the XML embedded in that file somehow?

> >>> Likewise for UUID checks. >> >> I still don't understand how UUID is used, so clarification/pointers welcome! >> >>>> I'm open to suggestions, for example by plumbing through a flag which makes the >>>> check optional. Please let me know how you prefer that I take this forward. >>> >>> If you're using managed save, then I would think it is already possible to >>> achieve. >>> >>> First cloning the VM: >>> >>> virsh dumpxml myvm > myvm.xml >>> cp myvm.xml myvm-clone.xml >>> ..modify name & uuid & bridge & disk of myvm-clone.xml >>> virsh define myvm-clone.xml >>> >>> Now modify the saved state >>> >>> virsh managedsave-dumpxml myvm-clone > save.xml >>> ...modify save.xml to change name & uuid to match myvm-clonme... >>> virsh managedsave-define save.xml >>> >>> The ABI stability check done by managedsave-define, will validate against >>> the name + uuid of the cloned VM, so in fact it will force you to change >>> the name + uuid in the save XML before loading it back in, and it should >>> be not possible to restore from the managed save image until fixing this >> >> Thanks for these pointers; I'll look into them. But I think my workflows are >> not managed as these VMs are not persistent on the host. When they die, they >> may be restarted elsewhere by a higher-level (multi-host) control plane. > > There are equivalent save-image-define / save-image-dumpxml for the > non-managed save scenario. And presumably this is how I get a saved XML which I can then modify?

> On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: > > On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote: >> >> >>> On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>> >>> On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote: >>>> >>>> >>>>> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>>>> >>>>> On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >>>>>> Hello! >>>>>> >>>>>> I have a use case which I'm struggling to support with libvirt: >>>>>> saving a VM to a file, cloning it (which renames the VM), and restoring it. >>>>>> >>>>>> My search revealed a number of tutorials for using virt-clone [1], but that >>>>>> doesn't seem to cover VMs which are _saved_ (only running or paused). >>>>> >>>>> Saved in what way ? Managed save ? >>>> >>>> Thanks for the prompt reply! >>>> >>>> I'm saving with virDomainSave(). My understanding is that this is not managed. >>> >>> Functionally it is the same as managed save, just the that file path >>> is specified by the client, rather than by libvirt. >> >> Got it, thanks. >> >>>>>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >>>>>> >>>>>> In a nutshell, I want to power on a VM and do some setup, then save its full >>>>>> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >>>>>> - rename the VM >>>>>> - change which bridge its NICs are on (while maintaining mac addresses) >>>>>> - change the disk image to a copy (done while the VM is saved) >>>>>> >>>>>> But the restore operation fails because of a target domain name check >>>>>> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >>>>>> address this and I'm looking for your views. >>>>> >>>>> If you're cloning a VM, it needs both a new UUID and name, so I'm surprised >>>>> the ABI stability check hasn't already blocked you on the UUID change before >>>>> getting to the name change check. >>>> >>>> I definitely didn't change the UUID. In fact, I want it to be the same (at >>>> least in the SMBIOS tables) because the guest OS is not going to expect that >>>> value to change without a power cycle/reset. The ABI Check actually ensures the >>>> SMBIOS values do not change during restore. >>>> >>>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... >>>> >>>> My understanding is that this passed because the other domain was not running >>>> (and the save was unmanaged, so libvirt is unaware of the saved VM). >>>> >>>> What I don't understand is why the UUID has to be unique (or, in fact, the same >>>> as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For >>>> the clone use case, I surely don't want this to change. >>>> >>>> In other words, it's not clear to me why this check is needed: >>>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... >>> >>> Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The >>> latter is only for running VMs. >>> >>> UUID is the primary unique identifier that is used for pretty much every >>> lookup inside libvirt. Name is a secondary unique identifier largely just >>> for external lookups by humans, since UUIDs are not human friendly. >>> >>> Essentially every API call starts with virDomainObjListFindByUUID to convert >>> the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct >>> that holds the config & state. >> >> Ah-ha. Ok, this is really helpful, thanks again! >> >> My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? >> (I'm pointing again at L#12810 above.) >> >> That feels incorrect. My (new) understanding is that: >> - The SMBIOS Type 1 UUID is guest-visible >> - The Libvirt UUID is a host identifier >> >> What comes to mind is that maybe something like guest tools wants to be able to >> report back to a control plane what VM it is on based on this value. If that's >> the motivation, then isn't Generation ID a better field to rely on? > > Strictly speaking we don't have to tie them together, but in practice we > do, because it is pretty compelling to be correlate data between the host > OS and guest OS for apps. Right, so libvirt uses the XML <UUID> as a host unique identifier and also as QEMU's "-uuid" parameter (which, based on my understanding, is a default for any virtual hardware UUID stuff such as SMBIOS Type 1 UUID/Serial. And the reason for that is to allow guests to infer their hypervisor identifier. > >> My understanding is that SMBIOS identifiers cannot change at runtime. > > Correct. Ok, but then I'm confused: how is clone supposed to work? When I clone a saved VM, Libvirt requires that I change its <UUID> and it also requires that this <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change. What am I missing?

> On 10 Jan 2025, at 12:38, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: > > !-------------------------------------------------------------------| > CAUTION: External Email > > |-------------------------------------------------------------------! > > On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote: >> >> >>> On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>> >>> On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote: >>>> >>>> >>>>> On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>>>> >>>>> On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote: >>>>>> >>>>>> >>>>>>> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>>>>>> >>>>>>> On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >>>>>>>> Hello! >>>>>>>> >>>>>>>> I have a use case which I'm struggling to support with libvirt: >>>>>>>> saving a VM to a file, cloning it (which renames the VM), and restoring it. >>>>>>>> >>>>>>>> My search revealed a number of tutorials for using virt-clone [1], but that >>>>>>>> doesn't seem to cover VMs which are _saved_ (only running or paused). >>>>>>> >>>>>>> Saved in what way ? Managed save ? >>>>>> >>>>>> Thanks for the prompt reply! >>>>>> >>>>>> I'm saving with virDomainSave(). My understanding is that this is not managed. >>>>> >>>>> Functionally it is the same as managed save, just the that file path >>>>> is specified by the client, rather than by libvirt. >>>> >>>> Got it, thanks. >>>> >>>>>>>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >>>>>>>> >>>>>>>> In a nutshell, I want to power on a VM and do some setup, then save its full >>>>>>>> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >>>>>>>> - rename the VM >>>>>>>> - change which bridge its NICs are on (while maintaining mac addresses) >>>>>>>> - change the disk image to a copy (done while the VM is saved) >>>>>>>> >>>>>>>> But the restore operation fails because of a target domain name check >>>>>>>> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >>>>>>>> address this and I'm looking for your views. >>>>>>> >>>>>>> If you're cloning a VM, it needs both a new UUID and name, so I'm surprised >>>>>>> the ABI stability check hasn't already blocked you on the UUID change before >>>>>>> getting to the name change check. >>>>>> >>>>>> I definitely didn't change the UUID. In fact, I want it to be the same (at >>>>>> least in the SMBIOS tables) because the guest OS is not going to expect that >>>>>> value to change without a power cycle/reset. The ABI Check actually ensures the >>>>>> SMBIOS values do not change during restore. >>>>>> >>>>>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... >>>>>> >>>>>> My understanding is that this passed because the other domain was not running >>>>>> (and the save was unmanaged, so libvirt is unaware of the saved VM). >>>>>> >>>>>> What I don't understand is why the UUID has to be unique (or, in fact, the same >>>>>> as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For >>>>>> the clone use case, I surely don't want this to change. >>>>>> >>>>>> In other words, it's not clear to me why this check is needed: >>>>>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... >>>>> >>>>> Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The >>>>> latter is only for running VMs. >>>>> >>>>> UUID is the primary unique identifier that is used for pretty much every >>>>> lookup inside libvirt. Name is a secondary unique identifier largely just >>>>> for external lookups by humans, since UUIDs are not human friendly. >>>>> >>>>> Essentially every API call starts with virDomainObjListFindByUUID to convert >>>>> the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct >>>>> that holds the config & state. >>>> >>>> Ah-ha. Ok, this is really helpful, thanks again! >>>> >>>> My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? >>>> (I'm pointing again at L#12810 above.) >>>> >>>> That feels incorrect. My (new) understanding is that: >>>> - The SMBIOS Type 1 UUID is guest-visible >>>> - The Libvirt UUID is a host identifier >>>> >>>> What comes to mind is that maybe something like guest tools wants to be able to >>>> report back to a control plane what VM it is on based on this value. If that's >>>> the motivation, then isn't Generation ID a better field to rely on? >>> >>> Strictly speaking we don't have to tie them together, but in practice we >>> do, because it is pretty compelling to be correlate data between the host >>> OS and guest OS for apps. >> >> Right, so libvirt uses the XML <UUID> as a host unique identifier and also as >> QEMU's "-uuid" parameter (which, based on my understanding, is a default for >> any virtual hardware UUID stuff such as SMBIOS Type 1 UUID/Serial. And the >> reason for that is to allow guests to infer their hypervisor identifier. >> >>> >>>> My understanding is that SMBIOS identifiers cannot change at runtime. >>> >>> Correct. >> >> Ok, but then I'm confused: how is clone supposed to work? When I clone a saved >> VM, Libvirt requires that I change its <UUID> and it also requires that this >> <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change. >> >> What am I missing? > > Yep, the idea of cloning a running (well saved) VM on the same > host is effectively denied due to this policy. > > We have never claimed that cloning a running VM is supported, > and actively discouraged people from trying to do this. > > The only workaround would be if launched on a different host. Failing > that the only option would be for us to remove the requirement that > VM UUID matches SMBIOS UUID. > > Perhaps we could do the latter, but mark the VM as "tainted" to indicate > this undesirable config scenario. That works for me. Let me submit another patch along those lines for review. Also it sounds like there isn't a strong reason for tying up SMBIOS UUID and VM UUID except the use case of the guest inferring its hypervisor identifier. Would it make sense to propose a new device type that can canonically be used for that purpose? Something like Generation ID, perhaps. I can see if someone from our side can work on that if you think it's a good idea.

On 10 Jan 2025, at 16:03, Felipe Franciosi <felipe(a)nutanix.com&gt; wrote: > On 10 Jan 2025, at 15:02, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: > > On Fri, Jan 10, 2025 at 01:46:53PM +0000, Felipe Franciosi wrote: >> >> >>> On 10 Jan 2025, at 12:38, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>> >>> On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote: >>>> >>>> >>>>> On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>>>> >>>>> On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote: >>>>>> >>>>>> >>>>>>> On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>>>>>> >>>>>>> On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>>>>>>>> >>>>>>>>> On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >>>>>>>>>> Hello! >>>>>>>>>> >>>>>>>>>> I have a use case which I'm struggling to support with libvirt: >>>>>>>>>> saving a VM to a file, cloning it (which renames the VM), and restoring it. >>>>>>>>>> >>>>>>>>>> My search revealed a number of tutorials for using virt-clone [1], but that >>>>>>>>>> doesn't seem to cover VMs which are _saved_ (only running or paused). >>>>>>>>> >>>>>>>>> Saved in what way ? Managed save ? >>>>>>>> >>>>>>>> Thanks for the prompt reply! >>>>>>>> >>>>>>>> I'm saving with virDomainSave(). My understanding is that this is not managed. >>>>>>> >>>>>>> Functionally it is the same as managed save, just the that file path >>>>>>> is specified by the client, rather than by libvirt. >>>>>> >>>>>> Got it, thanks. >>>>>> >>>>>>>>>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >>>>>>>>>> >>>>>>>>>> In a nutshell, I want to power on a VM and do some setup, then save its full >>>>>>>>>> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >>>>>>>>>> - rename the VM >>>>>>>>>> - change which bridge its NICs are on (while maintaining mac addresses) >>>>>>>>>> - change the disk image to a copy (done while the VM is saved) >>>>>>>>>> >>>>>>>>>> But the restore operation fails because of a target domain name check >>>>>>>>>> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >>>>>>>>>> address this and I'm looking for your views. >>>>>>>>> >>>>>>>>> If you're cloning a VM, it needs both a new UUID and name, so I'm surprised >>>>>>>>> the ABI stability check hasn't already blocked you on the UUID change before >>>>>>>>> getting to the name change check. >>>>>>>> >>>>>>>> I definitely didn't change the UUID. In fact, I want it to be the same (at >>>>>>>> least in the SMBIOS tables) because the guest OS is not going to expect that >>>>>>>> value to change without a power cycle/reset. The ABI Check actually ensures the >>>>>>>> SMBIOS values do not change during restore. >>>>>>>> >>>>>>>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... >>>>>>>> >>>>>>>> My understanding is that this passed because the other domain was not running >>>>>>>> (and the save was unmanaged, so libvirt is unaware of the saved VM). >>>>>>>> >>>>>>>> What I don't understand is why the UUID has to be unique (or, in fact, the same >>>>>>>> as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For >>>>>>>> the clone use case, I surely don't want this to change. >>>>>>>> >>>>>>>> In other words, it's not clear to me why this check is needed: >>>>>>>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c... >>>>>>> >>>>>>> Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The >>>>>>> latter is only for running VMs. >>>>>>> >>>>>>> UUID is the primary unique identifier that is used for pretty much every >>>>>>> lookup inside libvirt. Name is a secondary unique identifier largely just >>>>>>> for external lookups by humans, since UUIDs are not human friendly. >>>>>>> >>>>>>> Essentially every API call starts with virDomainObjListFindByUUID to convert >>>>>>> the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct >>>>>>> that holds the config & state. >>>>>> >>>>>> Ah-ha. Ok, this is really helpful, thanks again! >>>>>> >>>>>> My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? >>>>>> (I'm pointing again at L#12810 above.) >>>>>> >>>>>> That feels incorrect. My (new) understanding is that: >>>>>> - The SMBIOS Type 1 UUID is guest-visible >>>>>> - The Libvirt UUID is a host identifier >>>>>> >>>>>> What comes to mind is that maybe something like guest tools wants to be able to >>>>>> report back to a control plane what VM it is on based on this value. If that's >>>>>> the motivation, then isn't Generation ID a better field to rely on? >>>>> >>>>> Strictly speaking we don't have to tie them together, but in practice we >>>>> do, because it is pretty compelling to be correlate data between the host >>>>> OS and guest OS for apps. >>>> >>>> Right, so libvirt uses the XML <UUID> as a host unique identifier and also as >>>> QEMU's "-uuid" parameter (which, based on my understanding, is a default for >>>> any virtual hardware UUID stuff such as SMBIOS Type 1 UUID/Serial. And the >>>> reason for that is to allow guests to infer their hypervisor identifier. >>>> >>>>> >>>>>> My understanding is that SMBIOS identifiers cannot change at runtime. >>>>> >>>>> Correct. >>>> >>>> Ok, but then I'm confused: how is clone supposed to work? When I clone a saved >>>> VM, Libvirt requires that I change its <UUID> and it also requires that this >>>> <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change. >>>> >>>> What am I missing? >>> >>> Yep, the idea of cloning a running (well saved) VM on the same >>> host is effectively denied due to this policy. >>> >>> We have never claimed that cloning a running VM is supported, >>> and actively discouraged people from trying to do this. >>> >>> The only workaround would be if launched on a different host. Failing >>> that the only option would be for us to remove the requirement that >>> VM UUID matches SMBIOS UUID. >>> >>> Perhaps we could do the latter, but mark the VM as "tainted" to indicate >>> this undesirable config scenario. >> >> That works for me. Let me submit another patch along those lines for review. I'm having a look at this and it isn't entirely clear to me how to do it. I can see that virDomainObjParseXML() creates the virDomainObj *obj. Most of the interesting parsing then happens in virDomainDefParseXML() which (sensibly) takes the xmlXPathContextPtr and not the entire virDomainObj. That means, however, that I can't directly call virDomainObjTaint() from the relevant parsing bit in virSysinfoSystemParseXML() to taint the obj with my newly introduced VIR_DOMAIN_TAINT_UUID_MISMATCH. However, I can see that further down in virDomainObjParseXML() there's a virXPathNodeSet("./taint", ctxt, &taintNodes). I'm therefore guessing I don't have to call virDomainObjTaint() while parsing the XML. Instead, maybe I can just add "./taint" to the XML while parsing? I can't immediately find any precedence to that, though. Directions welcome!

>> Also it sounds like there isn't a strong reason for tying up SMBIOS UUID and VM >> UUID except the use case of the guest inferring its hypervisor identifier. >> Would it make sense to propose a new device type that can canonically be used >> for that purpose? Something like Generation ID, perhaps. I can see if someone >> from our side can work on that if you think it's a good idea. > > I'd suggest SMBIOS can already handle this. eg We could just document > something along the lines if > > If you make "System" (Type 1 table) "UUID" different from machine UUID, > then set "Base Board" (Type 2 table) "Asset Tag" to hold the machine UUID But that would mean the Asset Tag could change at VM runtime (well, at least while saved to disk). Wouldn't that be a problem?

On 17 Jan 2025, at 12:47, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: On Mon, Jan 13, 2025 at 12:13:43PM +0000, Felipe Franciosi wrote: > > >> On 10 Jan 2025, at 16:03, Felipe Franciosi <felipe(a)nutanix.com&gt; wrote: >> >>> On 10 Jan 2025, at 15:02, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>> >>> On Fri, Jan 10, 2025 at 01:46:53PM +0000, Felipe Franciosi wrote: >>>> >>>> >>>>> On 10 Jan 2025, at 12:38, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>>>> >>>>> On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote: >>>>>> >>>>>> >>>>>>> On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: >>>>>>> >>>>>>> On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote: >>>>>>>> >>>>>>>> My understanding is that SMBIOS identifiers cannot change at runtime. >>>>>>> >>>>>>> Correct. >>>>>> >>>>>> Ok, but then I'm confused: how is clone supposed to work? When I clone a saved >>>>>> VM, Libvirt requires that I change its <UUID> and it also requires that this >>>>>> <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change. >>>>>> >>>>>> What am I missing? >>>>> >>>>> Yep, the idea of cloning a running (well saved) VM on the same >>>>> host is effectively denied due to this policy. >>>>> >>>>> We have never claimed that cloning a running VM is supported, >>>>> and actively discouraged people from trying to do this. >>>>> >>>>> The only workaround would be if launched on a different host. Failing >>>>> that the only option would be for us to remove the requirement that >>>>> VM UUID matches SMBIOS UUID. >>>>> >>>>> Perhaps we could do the latter, but mark the VM as "tainted" to indicate >>>>> this undesirable config scenario. >>>> >>>> That works for me. Let me submit another patch along those lines for review. >> >> I'm having a look at this and it isn't entirely clear to me how to do it. >> >> I can see that virDomainObjParseXML() creates the virDomainObj *obj. Most of >> the interesting parsing then happens in virDomainDefParseXML() which (sensibly) >> takes the xmlXPathContextPtr and not the entire virDomainObj. >> >> That means, however, that I can't directly call virDomainObjTaint() from the >> relevant parsing bit in virSysinfoSystemParseXML() to taint the obj with my >> newly introduced VIR_DOMAIN_TAINT_UUID_MISMATCH. >> >> However, I can see that further down in virDomainObjParseXML() there's a >> virXPathNodeSet("./taint", ctxt, &taintNodes). I'm therefore guessing I don't >> have to call virDomainObjTaint() while parsing the XML. Instead, maybe I can >> just add "./taint" to the XML while parsing? >> >> I can't immediately find any precedence to that, though. Directions welcome! > > I did some more digging and found this: > > ------------8<------------ > commit 7998465005e2ebf26f6e65f5bdb886487374bb18 > Author: Daniel P. Berrangé <berrange(a)redhat.com&gt; > Date: Wed May 4 11:40:59 2011 +0100 > > Add field to virDomainObjPtr to track "tainting" > ------------8<------------ > > Seems like the idea of that taint parsing in virDomainObjParseXML() is to do > with <taint> tags added by virDomainObjFormat(). That answers one of my > questions (the one to do with where the tags come from). > > But I'm still unclear on how you'd prefer I raise this patch. Here are the > options I have in mind: > > 1. Add a <taint> tag while parsing in virSysinfoSystemParseXML() > (I don't really like this as no other "parse" method modifies the XML.) > > 2. Add a check somewhere later, e.g., in virDomainDefValidate() > (This feels better, but virDomainDefValidate() doesn't take the > virDomainObj, only the virDomainDef, so that may need to change.) > > 3. Taint the object in virDomainDefCheckABIStabilityFlags() > (This has the same problem as the option above, so I prefer the former.) > > 4. Check it explicitly in virDomainObjParseXML() > (That doesn't require passing the virDomainObj to the post-parsing methods, > but it just feels wrong as there are post-parsing methods.) > > Let me know what you think. Tainting is something we only apply to running VMs, so its hooked into the startup code path rather than XML parsing or validation time. So generally you'd want to add to "qemuDomainObjCheckTaint" or one of the helpers it calls.

>>>> Also it sounds like there isn't a strong reason for tying up SMBIOS UUID and VM >>>> UUID except the use case of the guest inferring its hypervisor identifier. >>>> Would it make sense to propose a new device type that can canonically be used >>>> for that purpose? Something like Generation ID, perhaps. I can see if someone >>>> from our side can work on that if you think it's a good idea. >>> >>> I'd suggest SMBIOS can already handle this. eg We could just document >>> something along the lines if >>> >>> If you make "System" (Type 1 table) "UUID" different from machine UUID, >>> then set "Base Board" (Type 2 table) "Asset Tag" to hold the machine UUID >> >> But that would mean the Asset Tag could change at VM runtime (well, at least >> while saved to disk). Wouldn't that be a problem? > > I still have the question above. Hmm, right, of course, SMBIOS can't change at runtime and even if we did change it, we can't expect the guest OS to update its view. Lets just forget the whole thing about providing the libvirt UUID elsewhere. If and when someone claims later we can decide whether to worry about it, or show them a different way to get to the same end goal.