[PATCH 0/1] Support cloning saved VMs
Hello! I have a use case which I'm struggling to support with libvirt: saving a VM to a file, cloning it (which renames the VM), and restoring it. My search revealed a number of tutorials for using virt-clone [1], but that doesn't seem to cover VMs which are _saved_ (only running or paused). [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py In a nutshell, I want to power on a VM and do some setup, then save its full state to disk (e.g., with virsh save). Finally I want to modify the XML to: - rename the VM - change which bridge its NICs are on (while maintaining mac addresses) - change the disk image to a copy (done while the VM is saved) But the restore operation fails because of a target domain name check implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best address this and I'm looking for your views. I suspect this call is there for a reason (which may still be relevant), although the name is clearly not part of the ABI; it's the host identifier for that domain and not guest-visible. My first stab at this is therefore just to drop this check (patch attached). I'm open to suggestions, for example by plumbing through a flag which makes the check optional. Please let me know how you prefer that I take this forward. Thanks, F. Felipe Franciosi (1): conf: remove domain name change check on CheckABI src/conf/domain_conf.c | 11 ----------- 1 file changed, 11 deletions(-) -- 2.39.5
virDomainDefCheckABIStabilityFlags() is called to validate two domains during migration. In addition to validating ABI, it also ensures the domain names have not changed. Such a check prevents cloning a saved VM because clones require a new name. This removes the check, allowing the use case described above. Signed-off-by: Felipe Franciosi <felipe@nutanix.com> --- src/conf/domain_conf.c | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index e658b68c48..0030e2b4b4 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -21786,17 +21786,6 @@ virDomainDefCheckABIStabilityFlags(virDomainDef *src, goto error; } - /* Not strictly ABI related, but we want to make sure domains - * don't get silently re-named through the backdoor when passing - * custom XML into various APIs, since this would create havoc - */ - if (STRNEQ_NULLABLE(src->name, dst->name)) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, - _("Target domain name '%1$s' does not match source '%2$s'"), - dst->name, src->name); - goto error; - } - if (!virDomainMemtuneCheckABIStability(src, dst, flags)) goto error; -- 2.39.5
On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote:
Hello!
I have a use case which I'm struggling to support with libvirt: saving a VM to a file, cloning it (which renames the VM), and restoring it.
My search revealed a number of tutorials for using virt-clone [1], but that doesn't seem to cover VMs which are _saved_ (only running or paused).
Saved in what way ? Managed save ?
[1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py
In a nutshell, I want to power on a VM and do some setup, then save its full state to disk (e.g., with virsh save). Finally I want to modify the XML to: - rename the VM - change which bridge its NICs are on (while maintaining mac addresses) - change the disk image to a copy (done while the VM is saved)
But the restore operation fails because of a target domain name check implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best address this and I'm looking for your views.
If you're cloning a VM, it needs both a new UUID and name, so I'm surprised the ABI stability check hasn't already blocked you on the UUID change before getting to the name change check.
I suspect this call is there for a reason (which may still be relevant), although the name is clearly not part of the ABI; it's the host identifier for that domain and not guest-visible. My first stab at this is therefore just to drop this check (patch attached).
The most important thing is that Libvirt has to ensure uniqueness of the name, within the host. If the name can be silently changed by passing in change XML, the unique checks will be missing and you can end up with many VMs with the same name. Likewise for UUID checks.
I'm open to suggestions, for example by plumbing through a flag which makes the check optional. Please let me know how you prefer that I take this forward.
If you're using managed save, then I would think it is already possible to achieve. First cloning the VM: virsh dumpxml myvm > myvm.xml cp myvm.xml myvm-clone.xml ..modify name & uuid & bridge & disk of myvm-clone.xml virsh define myvm-clone.xml Now modify the saved state virsh managedsave-dumpxml myvm-clone > save.xml ...modify save.xml to change name & uuid to match myvm-clonme... virsh managedsave-define save.xml The ABI stability check done by managedsave-define, will validate against the name + uuid of the cloned VM, so in fact it will force you to change the name + uuid in the save XML before loading it back in, and it should be not possible to restore from the managed save image until fixing this With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote:
Hello!
I have a use case which I'm struggling to support with libvirt: saving a VM to a file, cloning it (which renames the VM), and restoring it.
My search revealed a number of tutorials for using virt-clone [1], but that doesn't seem to cover VMs which are _saved_ (only running or paused).
Saved in what way ? Managed save ?
Thanks for the prompt reply! I'm saving with virDomainSave(). My understanding is that this is not managed.
[1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py
In a nutshell, I want to power on a VM and do some setup, then save its full state to disk (e.g., with virsh save). Finally I want to modify the XML to: - rename the VM - change which bridge its NICs are on (while maintaining mac addresses) - change the disk image to a copy (done while the VM is saved)
But the restore operation fails because of a target domain name check implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best address this and I'm looking for your views.
If you're cloning a VM, it needs both a new UUID and name, so I'm surprised the ABI stability check hasn't already blocked you on the UUID change before getting to the name change check.
I definitely didn't change the UUID. In fact, I want it to be the same (at least in the SMBIOS tables) because the guest OS is not going to expect that value to change without a power cycle/reset. The ABI Check actually ensures the SMBIOS values do not change during restore. https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0... My understanding is that this passed because the other domain was not running (and the save was unmanaged, so libvirt is unaware of the saved VM). What I don't understand is why the UUID has to be unique (or, in fact, the same as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For the clone use case, I surely don't want this to change. In other words, it's not clear to me why this check is needed: https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
I suspect this call is there for a reason (which may still be relevant), although the name is clearly not part of the ABI; it's the host identifier for that domain and not guest-visible. My first stab at this is therefore just to drop this check (patch attached).
The most important thing is that Libvirt has to ensure uniqueness of the name, within the host. If the name can be silently changed by passing in change XML, the unique checks will be missing and you can end up with many VMs with the same name.
Sure, but that's different than checking source is the same as destination. Isn't a check of domain name uniqueness within the host better done elsewhere? Maybe as part of domainRestore() / domainCreate() / domainRename() (or some common higher-level ground?).
Likewise for UUID checks.
I still don't understand how UUID is used, so clarification/pointers welcome!
I'm open to suggestions, for example by plumbing through a flag which makes the check optional. Please let me know how you prefer that I take this forward.
If you're using managed save, then I would think it is already possible to achieve.
First cloning the VM:
virsh dumpxml myvm > myvm.xml cp myvm.xml myvm-clone.xml ..modify name & uuid & bridge & disk of myvm-clone.xml virsh define myvm-clone.xml
Now modify the saved state
virsh managedsave-dumpxml myvm-clone > save.xml ...modify save.xml to change name & uuid to match myvm-clonme... virsh managedsave-define save.xml
The ABI stability check done by managedsave-define, will validate against the name + uuid of the cloned VM, so in fact it will force you to change the name + uuid in the save XML before loading it back in, and it should be not possible to restore from the managed save image until fixing this
Thanks for these pointers; I'll look into them. But I think my workflows are not managed as these VMs are not persistent on the host. When they die, they may be restarted elsewhere by a higher-level (multi-host) control plane. F.
On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote:
Hello!
I have a use case which I'm struggling to support with libvirt: saving a VM to a file, cloning it (which renames the VM), and restoring it.
My search revealed a number of tutorials for using virt-clone [1], but that doesn't seem to cover VMs which are _saved_ (only running or paused).
Saved in what way ? Managed save ?
Thanks for the prompt reply!
I'm saving with virDomainSave(). My understanding is that this is not managed.
Functionally it is the same as managed save, just the that file path is specified by the client, rather than by libvirt.
[1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py
In a nutshell, I want to power on a VM and do some setup, then save its full state to disk (e.g., with virsh save). Finally I want to modify the XML to: - rename the VM - change which bridge its NICs are on (while maintaining mac addresses) - change the disk image to a copy (done while the VM is saved)
But the restore operation fails because of a target domain name check implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best address this and I'm looking for your views.
If you're cloning a VM, it needs both a new UUID and name, so I'm surprised the ABI stability check hasn't already blocked you on the UUID change before getting to the name change check.
I definitely didn't change the UUID. In fact, I want it to be the same (at least in the SMBIOS tables) because the guest OS is not going to expect that value to change without a power cycle/reset. The ABI Check actually ensures the SMBIOS values do not change during restore.
https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
My understanding is that this passed because the other domain was not running (and the save was unmanaged, so libvirt is unaware of the saved VM).
What I don't understand is why the UUID has to be unique (or, in fact, the same as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For the clone use case, I surely don't want this to change.
In other words, it's not clear to me why this check is needed: https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The latter is only for running VMs. UUID is the primary unique identifier that is used for pretty much every lookup inside libvirt. Name is a secondary unique identifier largely just for external lookups by humans, since UUIDs are not human friendly. Essentially every API call starts with virDomainObjListFindByUUID to convert the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct that holds the config & state.
I suspect this call is there for a reason (which may still be relevant), although the name is clearly not part of the ABI; it's the host identifier for that domain and not guest-visible. My first stab at this is therefore just to drop this check (patch attached).
The most important thing is that Libvirt has to ensure uniqueness of the name, within the host. If the name can be silently changed by passing in change XML, the unique checks will be missing and you can end up with many VMs with the same name.
Sure, but that's different than checking source is the same as destination.
Isn't a check of domain name uniqueness within the host better done elsewhere? Maybe as part of domainRestore() / domainCreate() / domainRename() (or some common higher-level ground?).
Name uniqueness is validated in any code path that adds a new virDomainObjPtr to the virDomainObjList, via validatin in virDomainObjListAdd. The ABI stability checks are done in places where we allow the XML of an existing VM to be replaced, and that doesn't involve calls to virDomainObjListAdd since we're not creating a new VM, just altering its backend config.
Likewise for UUID checks.
I still don't understand how UUID is used, so clarification/pointers welcome!
I'm open to suggestions, for example by plumbing through a flag which makes the check optional. Please let me know how you prefer that I take this forward.
If you're using managed save, then I would think it is already possible to achieve.
First cloning the VM:
virsh dumpxml myvm > myvm.xml cp myvm.xml myvm-clone.xml ..modify name & uuid & bridge & disk of myvm-clone.xml virsh define myvm-clone.xml
Now modify the saved state
virsh managedsave-dumpxml myvm-clone > save.xml ...modify save.xml to change name & uuid to match myvm-clonme... virsh managedsave-define save.xml
The ABI stability check done by managedsave-define, will validate against the name + uuid of the cloned VM, so in fact it will force you to change the name + uuid in the save XML before loading it back in, and it should be not possible to restore from the managed save image until fixing this
Thanks for these pointers; I'll look into them. But I think my workflows are not managed as these VMs are not persistent on the host. When they die, they may be restarted elsewhere by a higher-level (multi-host) control plane.
There are equivalent save-image-define / save-image-dumpxml for the non-managed save scenario.
F.
With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote:
Hello!
I have a use case which I'm struggling to support with libvirt: saving a VM to a file, cloning it (which renames the VM), and restoring it.
My search revealed a number of tutorials for using virt-clone [1], but that doesn't seem to cover VMs which are _saved_ (only running or paused).
Saved in what way ? Managed save ?
Thanks for the prompt reply!
I'm saving with virDomainSave(). My understanding is that this is not managed.
Functionally it is the same as managed save, just the that file path is specified by the client, rather than by libvirt.
Got it, thanks.
[1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py
In a nutshell, I want to power on a VM and do some setup, then save its full state to disk (e.g., with virsh save). Finally I want to modify the XML to: - rename the VM - change which bridge its NICs are on (while maintaining mac addresses) - change the disk image to a copy (done while the VM is saved)
But the restore operation fails because of a target domain name check implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best address this and I'm looking for your views.
If you're cloning a VM, it needs both a new UUID and name, so I'm surprised the ABI stability check hasn't already blocked you on the UUID change before getting to the name change check.
I definitely didn't change the UUID. In fact, I want it to be the same (at least in the SMBIOS tables) because the guest OS is not going to expect that value to change without a power cycle/reset. The ABI Check actually ensures the SMBIOS values do not change during restore.
https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
My understanding is that this passed because the other domain was not running (and the save was unmanaged, so libvirt is unaware of the saved VM).
What I don't understand is why the UUID has to be unique (or, in fact, the same as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For the clone use case, I surely don't want this to change.
In other words, it's not clear to me why this check is needed: https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The latter is only for running VMs.
UUID is the primary unique identifier that is used for pretty much every lookup inside libvirt. Name is a secondary unique identifier largely just for external lookups by humans, since UUIDs are not human friendly.
Essentially every API call starts with virDomainObjListFindByUUID to convert the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct that holds the config & state.
Ah-ha. Ok, this is really helpful, thanks again! My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? (I'm pointing again at L#12810 above.) That feels incorrect. My (new) understanding is that: - The SMBIOS Type 1 UUID is guest-visible - The Libvirt UUID is a host identifier What comes to mind is that maybe something like guest tools wants to be able to report back to a control plane what VM it is on based on this value. If that's the motivation, then isn't Generation ID a better field to rely on? My understanding is that SMBIOS identifiers cannot change at runtime.
I suspect this call is there for a reason (which may still be relevant), although the name is clearly not part of the ABI; it's the host identifier for that domain and not guest-visible. My first stab at this is therefore just to drop this check (patch attached).
The most important thing is that Libvirt has to ensure uniqueness of the name, within the host. If the name can be silently changed by passing in change XML, the unique checks will be missing and you can end up with many VMs with the same name.
Sure, but that's different than checking source is the same as destination.
Isn't a check of domain name uniqueness within the host better done elsewhere? Maybe as part of domainRestore() / domainCreate() / domainRename() (or some common higher-level ground?).
Name uniqueness is validated in any code path that adds a new virDomainObjPtr to the virDomainObjList, via validatin in virDomainObjListAdd.
Ah, great. So this is already there.
The ABI stability checks are done in places where we allow the XML of an existing VM to be replaced, and that doesn't involve calls to virDomainObjListAdd since we're not creating a new VM, just altering its backend config.
So maybe the problem I have is that I'm not modifying this data when I clone the saved VM? To recap, I'm calling virDomainSave() and giving it a path. The path creates a file named after the VM. I'm just renaming that file. Maybe I should be modifying the XML embedded in that file somehow?
Likewise for UUID checks.
I still don't understand how UUID is used, so clarification/pointers welcome!
I'm open to suggestions, for example by plumbing through a flag which makes the check optional. Please let me know how you prefer that I take this forward.
If you're using managed save, then I would think it is already possible to achieve.
First cloning the VM:
virsh dumpxml myvm > myvm.xml cp myvm.xml myvm-clone.xml ..modify name & uuid & bridge & disk of myvm-clone.xml virsh define myvm-clone.xml
Now modify the saved state
virsh managedsave-dumpxml myvm-clone > save.xml ...modify save.xml to change name & uuid to match myvm-clonme... virsh managedsave-define save.xml
The ABI stability check done by managedsave-define, will validate against the name + uuid of the cloned VM, so in fact it will force you to change the name + uuid in the save XML before loading it back in, and it should be not possible to restore from the managed save image until fixing this
Thanks for these pointers; I'll look into them. But I think my workflows are not managed as these VMs are not persistent on the host. When they die, they may be restarted elsewhere by a higher-level (multi-host) control plane.
There are equivalent save-image-define / save-image-dumpxml for the non-managed save scenario.
And presumably this is how I get a saved XML which I can then modify? Thanks again for the prompt help, F.
On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote:
Hello!
I have a use case which I'm struggling to support with libvirt: saving a VM to a file, cloning it (which renames the VM), and restoring it.
My search revealed a number of tutorials for using virt-clone [1], but that doesn't seem to cover VMs which are _saved_ (only running or paused).
Saved in what way ? Managed save ?
Thanks for the prompt reply!
I'm saving with virDomainSave(). My understanding is that this is not managed.
Functionally it is the same as managed save, just the that file path is specified by the client, rather than by libvirt.
Got it, thanks.
[1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py
In a nutshell, I want to power on a VM and do some setup, then save its full state to disk (e.g., with virsh save). Finally I want to modify the XML to: - rename the VM - change which bridge its NICs are on (while maintaining mac addresses) - change the disk image to a copy (done while the VM is saved)
But the restore operation fails because of a target domain name check implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best address this and I'm looking for your views.
If you're cloning a VM, it needs both a new UUID and name, so I'm surprised the ABI stability check hasn't already blocked you on the UUID change before getting to the name change check.
I definitely didn't change the UUID. In fact, I want it to be the same (at least in the SMBIOS tables) because the guest OS is not going to expect that value to change without a power cycle/reset. The ABI Check actually ensures the SMBIOS values do not change during restore.
https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
My understanding is that this passed because the other domain was not running (and the save was unmanaged, so libvirt is unaware of the saved VM).
What I don't understand is why the UUID has to be unique (or, in fact, the same as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For the clone use case, I surely don't want this to change.
In other words, it's not clear to me why this check is needed: https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The latter is only for running VMs.
UUID is the primary unique identifier that is used for pretty much every lookup inside libvirt. Name is a secondary unique identifier largely just for external lookups by humans, since UUIDs are not human friendly.
Essentially every API call starts with virDomainObjListFindByUUID to convert the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct that holds the config & state.
Ah-ha. Ok, this is really helpful, thanks again!
My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? (I'm pointing again at L#12810 above.)
That feels incorrect. My (new) understanding is that: - The SMBIOS Type 1 UUID is guest-visible - The Libvirt UUID is a host identifier
What comes to mind is that maybe something like guest tools wants to be able to report back to a control plane what VM it is on based on this value. If that's the motivation, then isn't Generation ID a better field to rely on?
Strictly speaking we don't have to tie them together, but in practice we do, because it is pretty compelling to be correlate data between the host OS and guest OS for apps.
My understanding is that SMBIOS identifiers cannot change at runtime.
Correct.
The ABI stability checks are done in places where we allow the XML of an existing VM to be replaced, and that doesn't involve calls to virDomainObjListAdd since we're not creating a new VM, just altering its backend config.
So maybe the problem I have is that I'm not modifying this data when I clone the saved VM? To recap, I'm calling virDomainSave() and giving it a path. The path creates a file named after the VM. I'm just renaming that file. Maybe I should be modifying the XML embedded in that file somehow?
Likewise for UUID checks.
I still don't understand how UUID is used, so clarification/pointers welcome!
I'm open to suggestions, for example by plumbing through a flag which makes the check optional. Please let me know how you prefer that I take this forward.
If you're using managed save, then I would think it is already possible to achieve.
First cloning the VM:
virsh dumpxml myvm > myvm.xml cp myvm.xml myvm-clone.xml ..modify name & uuid & bridge & disk of myvm-clone.xml virsh define myvm-clone.xml
Now modify the saved state
virsh managedsave-dumpxml myvm-clone > save.xml ...modify save.xml to change name & uuid to match myvm-clonme... virsh managedsave-define save.xml
The ABI stability check done by managedsave-define, will validate against the name + uuid of the cloned VM, so in fact it will force you to change the name + uuid in the save XML before loading it back in, and it should be not possible to restore from the managed save image until fixing this
Thanks for these pointers; I'll look into them. But I think my workflows are not managed as these VMs are not persistent on the host. When they die, they may be restarted elsewhere by a higher-level (multi-host) control plane.
There are equivalent save-image-define / save-image-dumpxml for the non-managed save scenario.
And presumably this is how I get a saved XML which I can then modify?
Yes, that's the supported way of modifying the XML inside a save image, other than editting the file directly (please don't :-). With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote:
Hello!
I have a use case which I'm struggling to support with libvirt: saving a VM to a file, cloning it (which renames the VM), and restoring it.
My search revealed a number of tutorials for using virt-clone [1], but that doesn't seem to cover VMs which are _saved_ (only running or paused).
Saved in what way ? Managed save ?
Thanks for the prompt reply!
I'm saving with virDomainSave(). My understanding is that this is not managed.
Functionally it is the same as managed save, just the that file path is specified by the client, rather than by libvirt.
Got it, thanks.
[1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py
In a nutshell, I want to power on a VM and do some setup, then save its full state to disk (e.g., with virsh save). Finally I want to modify the XML to: - rename the VM - change which bridge its NICs are on (while maintaining mac addresses) - change the disk image to a copy (done while the VM is saved)
But the restore operation fails because of a target domain name check implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best address this and I'm looking for your views.
If you're cloning a VM, it needs both a new UUID and name, so I'm surprised the ABI stability check hasn't already blocked you on the UUID change before getting to the name change check.
I definitely didn't change the UUID. In fact, I want it to be the same (at least in the SMBIOS tables) because the guest OS is not going to expect that value to change without a power cycle/reset. The ABI Check actually ensures the SMBIOS values do not change during restore.
https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
My understanding is that this passed because the other domain was not running (and the save was unmanaged, so libvirt is unaware of the saved VM).
What I don't understand is why the UUID has to be unique (or, in fact, the same as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For the clone use case, I surely don't want this to change.
In other words, it's not clear to me why this check is needed: https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The latter is only for running VMs.
UUID is the primary unique identifier that is used for pretty much every lookup inside libvirt. Name is a secondary unique identifier largely just for external lookups by humans, since UUIDs are not human friendly.
Essentially every API call starts with virDomainObjListFindByUUID to convert the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct that holds the config & state.
Ah-ha. Ok, this is really helpful, thanks again!
My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? (I'm pointing again at L#12810 above.)
That feels incorrect. My (new) understanding is that: - The SMBIOS Type 1 UUID is guest-visible - The Libvirt UUID is a host identifier
What comes to mind is that maybe something like guest tools wants to be able to report back to a control plane what VM it is on based on this value. If that's the motivation, then isn't Generation ID a better field to rely on?
Strictly speaking we don't have to tie them together, but in practice we do, because it is pretty compelling to be correlate data between the host OS and guest OS for apps.
Right, so libvirt uses the XML <UUID> as a host unique identifier and also as QEMU's "-uuid" parameter (which, based on my understanding, is a default for any virtual hardware UUID stuff such as SMBIOS Type 1 UUID/Serial. And the reason for that is to allow guests to infer their hypervisor identifier.
My understanding is that SMBIOS identifiers cannot change at runtime.
Correct.
Ok, but then I'm confused: how is clone supposed to work? When I clone a saved VM, Libvirt requires that I change its <UUID> and it also requires that this <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change. What am I missing?
The ABI stability checks are done in places where we allow the XML of an existing VM to be replaced, and that doesn't involve calls to virDomainObjListAdd since we're not creating a new VM, just altering its backend config.
So maybe the problem I have is that I'm not modifying this data when I clone the saved VM? To recap, I'm calling virDomainSave() and giving it a path. The path creates a file named after the VM. I'm just renaming that file. Maybe I should be modifying the XML embedded in that file somehow?
Likewise for UUID checks.
I still don't understand how UUID is used, so clarification/pointers welcome!
I'm open to suggestions, for example by plumbing through a flag which makes the check optional. Please let me know how you prefer that I take this forward.
If you're using managed save, then I would think it is already possible to achieve.
First cloning the VM:
virsh dumpxml myvm > myvm.xml cp myvm.xml myvm-clone.xml ..modify name & uuid & bridge & disk of myvm-clone.xml virsh define myvm-clone.xml
Now modify the saved state
virsh managedsave-dumpxml myvm-clone > save.xml ...modify save.xml to change name & uuid to match myvm-clonme... virsh managedsave-define save.xml
The ABI stability check done by managedsave-define, will validate against the name + uuid of the cloned VM, so in fact it will force you to change the name + uuid in the save XML before loading it back in, and it should be not possible to restore from the managed save image until fixing this
Thanks for these pointers; I'll look into them. But I think my workflows are not managed as these VMs are not persistent on the host. When they die, they may be restarted elsewhere by a higher-level (multi-host) control plane.
There are equivalent save-image-define / save-image-dumpxml for the non-managed save scenario.
And presumably this is how I get a saved XML which I can then modify?
Yes, that's the supported way of modifying the XML inside a save image, other than editting the file directly (please don't :-).
lol what could possibly go wrong with opening a binary file, inferring some offsets, and patching it? j/k :) F.
On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: > Hello! > > I have a use case which I'm struggling to support with libvirt: > saving a VM to a file, cloning it (which renames the VM), and restoring it. > > My search revealed a number of tutorials for using virt-clone [1], but that > doesn't seem to cover VMs which are _saved_ (only running or paused).
Saved in what way ? Managed save ?
Thanks for the prompt reply!
I'm saving with virDomainSave(). My understanding is that this is not managed.
Functionally it is the same as managed save, just the that file path is specified by the client, rather than by libvirt.
Got it, thanks.
> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py > > In a nutshell, I want to power on a VM and do some setup, then save its full > state to disk (e.g., with virsh save). Finally I want to modify the XML to: > - rename the VM > - change which bridge its NICs are on (while maintaining mac addresses) > - change the disk image to a copy (done while the VM is saved) > > But the restore operation fails because of a target domain name check > implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best > address this and I'm looking for your views.
If you're cloning a VM, it needs both a new UUID and name, so I'm surprised the ABI stability check hasn't already blocked you on the UUID change before getting to the name change check.
I definitely didn't change the UUID. In fact, I want it to be the same (at least in the SMBIOS tables) because the guest OS is not going to expect that value to change without a power cycle/reset. The ABI Check actually ensures the SMBIOS values do not change during restore.
https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
My understanding is that this passed because the other domain was not running (and the save was unmanaged, so libvirt is unaware of the saved VM).
What I don't understand is why the UUID has to be unique (or, in fact, the same as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For the clone use case, I surely don't want this to change.
In other words, it's not clear to me why this check is needed: https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The latter is only for running VMs.
UUID is the primary unique identifier that is used for pretty much every lookup inside libvirt. Name is a secondary unique identifier largely just for external lookups by humans, since UUIDs are not human friendly.
Essentially every API call starts with virDomainObjListFindByUUID to convert the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct that holds the config & state.
Ah-ha. Ok, this is really helpful, thanks again!
My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? (I'm pointing again at L#12810 above.)
That feels incorrect. My (new) understanding is that: - The SMBIOS Type 1 UUID is guest-visible - The Libvirt UUID is a host identifier
What comes to mind is that maybe something like guest tools wants to be able to report back to a control plane what VM it is on based on this value. If that's the motivation, then isn't Generation ID a better field to rely on?
Strictly speaking we don't have to tie them together, but in practice we do, because it is pretty compelling to be correlate data between the host OS and guest OS for apps.
Right, so libvirt uses the XML <UUID> as a host unique identifier and also as QEMU's "-uuid" parameter (which, based on my understanding, is a default for any virtual hardware UUID stuff such as SMBIOS Type 1 UUID/Serial. And the reason for that is to allow guests to infer their hypervisor identifier.
My understanding is that SMBIOS identifiers cannot change at runtime.
Correct.
Ok, but then I'm confused: how is clone supposed to work? When I clone a saved VM, Libvirt requires that I change its <UUID> and it also requires that this <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change.
What am I missing?
Yep, the idea of cloning a running (well saved) VM on the same host is effectively denied due to this policy. We have never claimed that cloning a running VM is supported, and actively discouraged people from trying to do this. The only workaround would be if launched on a different host. Failing that the only option would be for us to remove the requirement that VM UUID matches SMBIOS UUID. Perhaps we could do the latter, but mark the VM as "tainted" to indicate this undesirable config scenario. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 10 Jan 2025, at 12:38, Daniel P. Berrangé <berrange@redhat.com> wrote:
!-------------------------------------------------------------------| CAUTION: External Email
|-------------------------------------------------------------------!
On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote:
> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >> Hello! >> >> I have a use case which I'm struggling to support with libvirt: >> saving a VM to a file, cloning it (which renames the VM), and restoring it. >> >> My search revealed a number of tutorials for using virt-clone [1], but that >> doesn't seem to cover VMs which are _saved_ (only running or paused). > > Saved in what way ? Managed save ?
Thanks for the prompt reply!
I'm saving with virDomainSave(). My understanding is that this is not managed.
Functionally it is the same as managed save, just the that file path is specified by the client, rather than by libvirt.
Got it, thanks.
>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >> >> In a nutshell, I want to power on a VM and do some setup, then save its full >> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >> - rename the VM >> - change which bridge its NICs are on (while maintaining mac addresses) >> - change the disk image to a copy (done while the VM is saved) >> >> But the restore operation fails because of a target domain name check >> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >> address this and I'm looking for your views. > > If you're cloning a VM, it needs both a new UUID and name, so I'm surprised > the ABI stability check hasn't already blocked you on the UUID change before > getting to the name change check.
I definitely didn't change the UUID. In fact, I want it to be the same (at least in the SMBIOS tables) because the guest OS is not going to expect that value to change without a power cycle/reset. The ABI Check actually ensures the SMBIOS values do not change during restore.
https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
My understanding is that this passed because the other domain was not running (and the save was unmanaged, so libvirt is unaware of the saved VM).
What I don't understand is why the UUID has to be unique (or, in fact, the same as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For the clone use case, I surely don't want this to change.
In other words, it's not clear to me why this check is needed: https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The latter is only for running VMs.
UUID is the primary unique identifier that is used for pretty much every lookup inside libvirt. Name is a secondary unique identifier largely just for external lookups by humans, since UUIDs are not human friendly.
Essentially every API call starts with virDomainObjListFindByUUID to convert the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct that holds the config & state.
Ah-ha. Ok, this is really helpful, thanks again!
My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? (I'm pointing again at L#12810 above.)
That feels incorrect. My (new) understanding is that: - The SMBIOS Type 1 UUID is guest-visible - The Libvirt UUID is a host identifier
What comes to mind is that maybe something like guest tools wants to be able to report back to a control plane what VM it is on based on this value. If that's the motivation, then isn't Generation ID a better field to rely on?
Strictly speaking we don't have to tie them together, but in practice we do, because it is pretty compelling to be correlate data between the host OS and guest OS for apps.
Right, so libvirt uses the XML <UUID> as a host unique identifier and also as QEMU's "-uuid" parameter (which, based on my understanding, is a default for any virtual hardware UUID stuff such as SMBIOS Type 1 UUID/Serial. And the reason for that is to allow guests to infer their hypervisor identifier.
My understanding is that SMBIOS identifiers cannot change at runtime.
Correct.
Ok, but then I'm confused: how is clone supposed to work? When I clone a saved VM, Libvirt requires that I change its <UUID> and it also requires that this <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change.
What am I missing?
Yep, the idea of cloning a running (well saved) VM on the same host is effectively denied due to this policy.
We have never claimed that cloning a running VM is supported, and actively discouraged people from trying to do this.
The only workaround would be if launched on a different host. Failing that the only option would be for us to remove the requirement that VM UUID matches SMBIOS UUID.
Perhaps we could do the latter, but mark the VM as "tainted" to indicate this undesirable config scenario.
That works for me. Let me submit another patch along those lines for review. Also it sounds like there isn't a strong reason for tying up SMBIOS UUID and VM UUID except the use case of the guest inferring its hypervisor identifier. Would it make sense to propose a new device type that can canonically be used for that purpose? Something like Generation ID, perhaps. I can see if someone from our side can work on that if you think it's a good idea. F.
With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Fri, Jan 10, 2025 at 01:46:53PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 12:38, Daniel P. Berrangé <berrange@redhat.com> wrote:
!-------------------------------------------------------------------| CAUTION: External Email
|-------------------------------------------------------------------!
On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote: > > >> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@redhat.com> wrote: >> >> On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >>> Hello! >>> >>> I have a use case which I'm struggling to support with libvirt: >>> saving a VM to a file, cloning it (which renames the VM), and restoring it. >>> >>> My search revealed a number of tutorials for using virt-clone [1], but that >>> doesn't seem to cover VMs which are _saved_ (only running or paused). >> >> Saved in what way ? Managed save ? > > Thanks for the prompt reply! > > I'm saving with virDomainSave(). My understanding is that this is not managed.
Functionally it is the same as managed save, just the that file path is specified by the client, rather than by libvirt.
Got it, thanks.
>>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >>> >>> In a nutshell, I want to power on a VM and do some setup, then save its full >>> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >>> - rename the VM >>> - change which bridge its NICs are on (while maintaining mac addresses) >>> - change the disk image to a copy (done while the VM is saved) >>> >>> But the restore operation fails because of a target domain name check >>> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >>> address this and I'm looking for your views. >> >> If you're cloning a VM, it needs both a new UUID and name, so I'm surprised >> the ABI stability check hasn't already blocked you on the UUID change before >> getting to the name change check. > > I definitely didn't change the UUID. In fact, I want it to be the same (at > least in the SMBIOS tables) because the guest OS is not going to expect that > value to change without a power cycle/reset. The ABI Check actually ensures the > SMBIOS values do not change during restore. > > https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0... > > My understanding is that this passed because the other domain was not running > (and the save was unmanaged, so libvirt is unaware of the saved VM). > > What I don't understand is why the UUID has to be unique (or, in fact, the same > as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For > the clone use case, I surely don't want this to change. > > In other words, it's not clear to me why this check is needed: > https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0...
Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The latter is only for running VMs.
UUID is the primary unique identifier that is used for pretty much every lookup inside libvirt. Name is a secondary unique identifier largely just for external lookups by humans, since UUIDs are not human friendly.
Essentially every API call starts with virDomainObjListFindByUUID to convert the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct that holds the config & state.
Ah-ha. Ok, this is really helpful, thanks again!
My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? (I'm pointing again at L#12810 above.)
That feels incorrect. My (new) understanding is that: - The SMBIOS Type 1 UUID is guest-visible - The Libvirt UUID is a host identifier
What comes to mind is that maybe something like guest tools wants to be able to report back to a control plane what VM it is on based on this value. If that's the motivation, then isn't Generation ID a better field to rely on?
Strictly speaking we don't have to tie them together, but in practice we do, because it is pretty compelling to be correlate data between the host OS and guest OS for apps.
Right, so libvirt uses the XML <UUID> as a host unique identifier and also as QEMU's "-uuid" parameter (which, based on my understanding, is a default for any virtual hardware UUID stuff such as SMBIOS Type 1 UUID/Serial. And the reason for that is to allow guests to infer their hypervisor identifier.
My understanding is that SMBIOS identifiers cannot change at runtime.
Correct.
Ok, but then I'm confused: how is clone supposed to work? When I clone a saved VM, Libvirt requires that I change its <UUID> and it also requires that this <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change.
What am I missing?
Yep, the idea of cloning a running (well saved) VM on the same host is effectively denied due to this policy.
We have never claimed that cloning a running VM is supported, and actively discouraged people from trying to do this.
The only workaround would be if launched on a different host. Failing that the only option would be for us to remove the requirement that VM UUID matches SMBIOS UUID.
Perhaps we could do the latter, but mark the VM as "tainted" to indicate this undesirable config scenario.
That works for me. Let me submit another patch along those lines for review.
Also it sounds like there isn't a strong reason for tying up SMBIOS UUID and VM UUID except the use case of the guest inferring its hypervisor identifier. Would it make sense to propose a new device type that can canonically be used for that purpose? Something like Generation ID, perhaps. I can see if someone from our side can work on that if you think it's a good idea.
I'd suggest SMBIOS can already handle this. eg We could just document something along the lines if If you make "System" (Type 1 table) "UUID" different from machine UUID, then set "Base Board" (Type 2 table) "Asset Tag" to hold the machine UUID With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 10 Jan 2025, at 15:02, Daniel P. Berrangé <berrange@redhat.com> wrote:
!-------------------------------------------------------------------| CAUTION: External Email
|-------------------------------------------------------------------!
On Fri, Jan 10, 2025 at 01:46:53PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 12:38, Daniel P. Berrangé <berrange@redhat.com> wrote:
!-------------------------------------------------------------------| CAUTION: External Email
|-------------------------------------------------------------------!
On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote:
> On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote: >> >> >>> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@redhat.com> wrote: >>> >>> On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >>>> Hello! >>>> >>>> I have a use case which I'm struggling to support with libvirt: >>>> saving a VM to a file, cloning it (which renames the VM), and restoring it. >>>> >>>> My search revealed a number of tutorials for using virt-clone [1], but that >>>> doesn't seem to cover VMs which are _saved_ (only running or paused). >>> >>> Saved in what way ? Managed save ? >> >> Thanks for the prompt reply! >> >> I'm saving with virDomainSave(). My understanding is that this is not managed. > > Functionally it is the same as managed save, just the that file path > is specified by the client, rather than by libvirt.
Got it, thanks.
>>>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >>>> >>>> In a nutshell, I want to power on a VM and do some setup, then save its full >>>> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >>>> - rename the VM >>>> - change which bridge its NICs are on (while maintaining mac addresses) >>>> - change the disk image to a copy (done while the VM is saved) >>>> >>>> But the restore operation fails because of a target domain name check >>>> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >>>> address this and I'm looking for your views. >>> >>> If you're cloning a VM, it needs both a new UUID and name, so I'm surprised >>> the ABI stability check hasn't already blocked you on the UUID change before >>> getting to the name change check. >> >> I definitely didn't change the UUID. In fact, I want it to be the same (at >> least in the SMBIOS tables) because the guest OS is not going to expect that >> value to change without a power cycle/reset. The ABI Check actually ensures the >> SMBIOS values do not change during restore. >> >> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0... >> >> My understanding is that this passed because the other domain was not running >> (and the save was unmanaged, so libvirt is unaware of the saved VM). >> >> What I don't understand is why the UUID has to be unique (or, in fact, the same >> as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For >> the clone use case, I surely don't want this to change. >> >> In other words, it's not clear to me why this check is needed: >> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0... > > Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The > latter is only for running VMs. > > UUID is the primary unique identifier that is used for pretty much every > lookup inside libvirt. Name is a secondary unique identifier largely just > for external lookups by humans, since UUIDs are not human friendly. > > Essentially every API call starts with virDomainObjListFindByUUID to convert > the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct > that holds the config & state.
Ah-ha. Ok, this is really helpful, thanks again!
My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? (I'm pointing again at L#12810 above.)
That feels incorrect. My (new) understanding is that: - The SMBIOS Type 1 UUID is guest-visible - The Libvirt UUID is a host identifier
What comes to mind is that maybe something like guest tools wants to be able to report back to a control plane what VM it is on based on this value. If that's the motivation, then isn't Generation ID a better field to rely on?
Strictly speaking we don't have to tie them together, but in practice we do, because it is pretty compelling to be correlate data between the host OS and guest OS for apps.
Right, so libvirt uses the XML <UUID> as a host unique identifier and also as QEMU's "-uuid" parameter (which, based on my understanding, is a default for any virtual hardware UUID stuff such as SMBIOS Type 1 UUID/Serial. And the reason for that is to allow guests to infer their hypervisor identifier.
My understanding is that SMBIOS identifiers cannot change at runtime.
Correct.
Ok, but then I'm confused: how is clone supposed to work? When I clone a saved VM, Libvirt requires that I change its <UUID> and it also requires that this <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change.
What am I missing?
Yep, the idea of cloning a running (well saved) VM on the same host is effectively denied due to this policy.
We have never claimed that cloning a running VM is supported, and actively discouraged people from trying to do this.
The only workaround would be if launched on a different host. Failing that the only option would be for us to remove the requirement that VM UUID matches SMBIOS UUID.
Perhaps we could do the latter, but mark the VM as "tainted" to indicate this undesirable config scenario.
That works for me. Let me submit another patch along those lines for review.
I'm having a look at this and it isn't entirely clear to me how to do it. I can see that virDomainObjParseXML() creates the virDomainObj *obj. Most of the interesting parsing then happens in virDomainDefParseXML() which (sensibly) takes the xmlXPathContextPtr and not the entire virDomainObj. That means, however, that I can't directly call virDomainObjTaint() from the relevant parsing bit in virSysinfoSystemParseXML() to taint the obj with my newly introduced VIR_DOMAIN_TAINT_UUID_MISMATCH. However, I can see that further down in virDomainObjParseXML() there's a virXPathNodeSet("./taint", ctxt, &taintNodes). I'm therefore guessing I don't have to call virDomainObjTaint() while parsing the XML. Instead, maybe I can just add "./taint" to the XML while parsing? I can't immediately find any precedence to that, though. Directions welcome!
Also it sounds like there isn't a strong reason for tying up SMBIOS UUID and VM UUID except the use case of the guest inferring its hypervisor identifier. Would it make sense to propose a new device type that can canonically be used for that purpose? Something like Generation ID, perhaps. I can see if someone from our side can work on that if you think it's a good idea.
I'd suggest SMBIOS can already handle this. eg We could just document something along the lines if
If you make "System" (Type 1 table) "UUID" different from machine UUID, then set "Base Board" (Type 2 table) "Asset Tag" to hold the machine UUID
But that would mean the Asset Tag could change at VM runtime (well, at least while saved to disk). Wouldn't that be a problem? Thanks, F.
On 10 Jan 2025, at 16:03, Felipe Franciosi <felipe@nutanix.com> wrote:
On 10 Jan 2025, at 15:02, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 01:46:53PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 12:38, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote: > > >> On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange@redhat.com> wrote: >> >> On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote: >>> >>> >>>> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@redhat.com> wrote: >>>> >>>> On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >>>>> Hello! >>>>> >>>>> I have a use case which I'm struggling to support with libvirt: >>>>> saving a VM to a file, cloning it (which renames the VM), and restoring it. >>>>> >>>>> My search revealed a number of tutorials for using virt-clone [1], but that >>>>> doesn't seem to cover VMs which are _saved_ (only running or paused). >>>> >>>> Saved in what way ? Managed save ? >>> >>> Thanks for the prompt reply! >>> >>> I'm saving with virDomainSave(). My understanding is that this is not managed. >> >> Functionally it is the same as managed save, just the that file path >> is specified by the client, rather than by libvirt. > > Got it, thanks. > >>>>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >>>>> >>>>> In a nutshell, I want to power on a VM and do some setup, then save its full >>>>> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >>>>> - rename the VM >>>>> - change which bridge its NICs are on (while maintaining mac addresses) >>>>> - change the disk image to a copy (done while the VM is saved) >>>>> >>>>> But the restore operation fails because of a target domain name check >>>>> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >>>>> address this and I'm looking for your views. >>>> >>>> If you're cloning a VM, it needs both a new UUID and name, so I'm surprised >>>> the ABI stability check hasn't already blocked you on the UUID change before >>>> getting to the name change check. >>> >>> I definitely didn't change the UUID. In fact, I want it to be the same (at >>> least in the SMBIOS tables) because the guest OS is not going to expect that >>> value to change without a power cycle/reset. The ABI Check actually ensures the >>> SMBIOS values do not change during restore. >>> >>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0... >>> >>> My understanding is that this passed because the other domain was not running >>> (and the save was unmanaged, so libvirt is unaware of the saved VM). >>> >>> What I don't understand is why the UUID has to be unique (or, in fact, the same >>> as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For >>> the clone use case, I surely don't want this to change. >>> >>> In other words, it's not clear to me why this check is needed: >>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0... >> >> Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The >> latter is only for running VMs. >> >> UUID is the primary unique identifier that is used for pretty much every >> lookup inside libvirt. Name is a secondary unique identifier largely just >> for external lookups by humans, since UUIDs are not human friendly. >> >> Essentially every API call starts with virDomainObjListFindByUUID to convert >> the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct >> that holds the config & state. > > Ah-ha. Ok, this is really helpful, thanks again! > > My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? > (I'm pointing again at L#12810 above.) > > That feels incorrect. My (new) understanding is that: > - The SMBIOS Type 1 UUID is guest-visible > - The Libvirt UUID is a host identifier > > What comes to mind is that maybe something like guest tools wants to be able to > report back to a control plane what VM it is on based on this value. If that's > the motivation, then isn't Generation ID a better field to rely on?
Strictly speaking we don't have to tie them together, but in practice we do, because it is pretty compelling to be correlate data between the host OS and guest OS for apps.
Right, so libvirt uses the XML <UUID> as a host unique identifier and also as QEMU's "-uuid" parameter (which, based on my understanding, is a default for any virtual hardware UUID stuff such as SMBIOS Type 1 UUID/Serial. And the reason for that is to allow guests to infer their hypervisor identifier.
> My understanding is that SMBIOS identifiers cannot change at runtime.
Correct.
Ok, but then I'm confused: how is clone supposed to work? When I clone a saved VM, Libvirt requires that I change its <UUID> and it also requires that this <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change.
What am I missing?
Yep, the idea of cloning a running (well saved) VM on the same host is effectively denied due to this policy.
We have never claimed that cloning a running VM is supported, and actively discouraged people from trying to do this.
The only workaround would be if launched on a different host. Failing that the only option would be for us to remove the requirement that VM UUID matches SMBIOS UUID.
Perhaps we could do the latter, but mark the VM as "tainted" to indicate this undesirable config scenario.
That works for me. Let me submit another patch along those lines for review.
I'm having a look at this and it isn't entirely clear to me how to do it.
I can see that virDomainObjParseXML() creates the virDomainObj *obj. Most of the interesting parsing then happens in virDomainDefParseXML() which (sensibly) takes the xmlXPathContextPtr and not the entire virDomainObj.
That means, however, that I can't directly call virDomainObjTaint() from the relevant parsing bit in virSysinfoSystemParseXML() to taint the obj with my newly introduced VIR_DOMAIN_TAINT_UUID_MISMATCH.
However, I can see that further down in virDomainObjParseXML() there's a virXPathNodeSet("./taint", ctxt, &taintNodes). I'm therefore guessing I don't have to call virDomainObjTaint() while parsing the XML. Instead, maybe I can just add "./taint" to the XML while parsing?
I can't immediately find any precedence to that, though. Directions welcome!
I did some more digging and found this: ------------8<------------ commit 7998465005e2ebf26f6e65f5bdb886487374bb18 Author: Daniel P. Berrangé <berrange@redhat.com> Date: Wed May 4 11:40:59 2011 +0100 Add field to virDomainObjPtr to track "tainting" ------------8<------------ Seems like the idea of that taint parsing in virDomainObjParseXML() is to do with <taint> tags added by virDomainObjFormat(). That answers one of my questions (the one to do with where the tags come from). But I'm still unclear on how you'd prefer I raise this patch. Here are the options I have in mind: 1. Add a <taint> tag while parsing in virSysinfoSystemParseXML() (I don't really like this as no other "parse" method modifies the XML.) 2. Add a check somewhere later, e.g., in virDomainDefValidate() (This feels better, but virDomainDefValidate() doesn't take the virDomainObj, only the virDomainDef, so that may need to change.) 3. Taint the object in virDomainDefCheckABIStabilityFlags() (This has the same problem as the option above, so I prefer the former.) 4. Check it explicitly in virDomainObjParseXML() (That doesn't require passing the virDomainObj to the post-parsing methods, but it just feels wrong as there are post-parsing methods.) Let me know what you think.
Also it sounds like there isn't a strong reason for tying up SMBIOS UUID and VM UUID except the use case of the guest inferring its hypervisor identifier. Would it make sense to propose a new device type that can canonically be used for that purpose? Something like Generation ID, perhaps. I can see if someone from our side can work on that if you think it's a good idea.
I'd suggest SMBIOS can already handle this. eg We could just document something along the lines if
If you make "System" (Type 1 table) "UUID" different from machine UUID, then set "Base Board" (Type 2 table) "Asset Tag" to hold the machine UUID
But that would mean the Asset Tag could change at VM runtime (well, at least while saved to disk). Wouldn't that be a problem?
I still have the question above. Cheers, F.
On Mon, Jan 13, 2025 at 12:13:43PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 16:03, Felipe Franciosi <felipe@nutanix.com> wrote:
On 10 Jan 2025, at 15:02, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 01:46:53PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 12:38, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote:
> On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote: >> >> My understanding is that SMBIOS identifiers cannot change at runtime. > > Correct.
Ok, but then I'm confused: how is clone supposed to work? When I clone a saved VM, Libvirt requires that I change its <UUID> and it also requires that this <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change.
What am I missing?
Yep, the idea of cloning a running (well saved) VM on the same host is effectively denied due to this policy.
We have never claimed that cloning a running VM is supported, and actively discouraged people from trying to do this.
The only workaround would be if launched on a different host. Failing that the only option would be for us to remove the requirement that VM UUID matches SMBIOS UUID.
Perhaps we could do the latter, but mark the VM as "tainted" to indicate this undesirable config scenario.
That works for me. Let me submit another patch along those lines for review.
I'm having a look at this and it isn't entirely clear to me how to do it.
I can see that virDomainObjParseXML() creates the virDomainObj *obj. Most of the interesting parsing then happens in virDomainDefParseXML() which (sensibly) takes the xmlXPathContextPtr and not the entire virDomainObj.
That means, however, that I can't directly call virDomainObjTaint() from the relevant parsing bit in virSysinfoSystemParseXML() to taint the obj with my newly introduced VIR_DOMAIN_TAINT_UUID_MISMATCH.
However, I can see that further down in virDomainObjParseXML() there's a virXPathNodeSet("./taint", ctxt, &taintNodes). I'm therefore guessing I don't have to call virDomainObjTaint() while parsing the XML. Instead, maybe I can just add "./taint" to the XML while parsing?
I can't immediately find any precedence to that, though. Directions welcome!
I did some more digging and found this:
------------8<------------ commit 7998465005e2ebf26f6e65f5bdb886487374bb18 Author: Daniel P. Berrangé <berrange@redhat.com> Date: Wed May 4 11:40:59 2011 +0100
Add field to virDomainObjPtr to track "tainting" ------------8<------------
Seems like the idea of that taint parsing in virDomainObjParseXML() is to do with <taint> tags added by virDomainObjFormat(). That answers one of my questions (the one to do with where the tags come from).
But I'm still unclear on how you'd prefer I raise this patch. Here are the options I have in mind:
1. Add a <taint> tag while parsing in virSysinfoSystemParseXML() (I don't really like this as no other "parse" method modifies the XML.)
2. Add a check somewhere later, e.g., in virDomainDefValidate() (This feels better, but virDomainDefValidate() doesn't take the virDomainObj, only the virDomainDef, so that may need to change.)
3. Taint the object in virDomainDefCheckABIStabilityFlags() (This has the same problem as the option above, so I prefer the former.)
4. Check it explicitly in virDomainObjParseXML() (That doesn't require passing the virDomainObj to the post-parsing methods, but it just feels wrong as there are post-parsing methods.)
Let me know what you think.
Tainting is something we only apply to running VMs, so its hooked into the startup code path rather than XML parsing or validation time. So generally you'd want to add to "qemuDomainObjCheckTaint" or one of the helpers it calls.
Also it sounds like there isn't a strong reason for tying up SMBIOS UUID and VM UUID except the use case of the guest inferring its hypervisor identifier. Would it make sense to propose a new device type that can canonically be used for that purpose? Something like Generation ID, perhaps. I can see if someone from our side can work on that if you think it's a good idea.
I'd suggest SMBIOS can already handle this. eg We could just document something along the lines if
If you make "System" (Type 1 table) "UUID" different from machine UUID, then set "Base Board" (Type 2 table) "Asset Tag" to hold the machine UUID
But that would mean the Asset Tag could change at VM runtime (well, at least while saved to disk). Wouldn't that be a problem?
I still have the question above.
Hmm, right, of course, SMBIOS can't change at runtime and even if we did change it, we can't expect the guest OS to update its view. Lets just forget the whole thing about providing the libvirt UUID elsewhere. If and when someone claims later we can decide whether to worry about it, or show them a different way to get to the same end goal. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 17 Jan 2025, at 12:47, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Jan 13, 2025 at 12:13:43PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 16:03, Felipe Franciosi <felipe@nutanix.com> wrote:
On 10 Jan 2025, at 15:02, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 01:46:53PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 12:38, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote: > > >> On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange@redhat.com> wrote: >> >> On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote: >>> >>> My understanding is that SMBIOS identifiers cannot change at runtime. >> >> Correct. > > Ok, but then I'm confused: how is clone supposed to work? When I clone a saved > VM, Libvirt requires that I change its <UUID> and it also requires that this > <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change. > > What am I missing?
Yep, the idea of cloning a running (well saved) VM on the same host is effectively denied due to this policy.
We have never claimed that cloning a running VM is supported, and actively discouraged people from trying to do this.
The only workaround would be if launched on a different host. Failing that the only option would be for us to remove the requirement that VM UUID matches SMBIOS UUID.
Perhaps we could do the latter, but mark the VM as "tainted" to indicate this undesirable config scenario.
That works for me. Let me submit another patch along those lines for review.
I'm having a look at this and it isn't entirely clear to me how to do it.
I can see that virDomainObjParseXML() creates the virDomainObj *obj. Most of the interesting parsing then happens in virDomainDefParseXML() which (sensibly) takes the xmlXPathContextPtr and not the entire virDomainObj.
That means, however, that I can't directly call virDomainObjTaint() from the relevant parsing bit in virSysinfoSystemParseXML() to taint the obj with my newly introduced VIR_DOMAIN_TAINT_UUID_MISMATCH.
However, I can see that further down in virDomainObjParseXML() there's a virXPathNodeSet("./taint", ctxt, &taintNodes). I'm therefore guessing I don't have to call virDomainObjTaint() while parsing the XML. Instead, maybe I can just add "./taint" to the XML while parsing?
I can't immediately find any precedence to that, though. Directions welcome!
I did some more digging and found this:
------------8<------------ commit 7998465005e2ebf26f6e65f5bdb886487374bb18 Author: Daniel P. Berrangé <berrange@redhat.com> Date: Wed May 4 11:40:59 2011 +0100
Add field to virDomainObjPtr to track "tainting" ------------8<------------
Seems like the idea of that taint parsing in virDomainObjParseXML() is to do with <taint> tags added by virDomainObjFormat(). That answers one of my questions (the one to do with where the tags come from).
But I'm still unclear on how you'd prefer I raise this patch. Here are the options I have in mind:
1. Add a <taint> tag while parsing in virSysinfoSystemParseXML() (I don't really like this as no other "parse" method modifies the XML.)
2. Add a check somewhere later, e.g., in virDomainDefValidate() (This feels better, but virDomainDefValidate() doesn't take the virDomainObj, only the virDomainDef, so that may need to change.)
3. Taint the object in virDomainDefCheckABIStabilityFlags() (This has the same problem as the option above, so I prefer the former.)
4. Check it explicitly in virDomainObjParseXML() (That doesn't require passing the virDomainObj to the post-parsing methods, but it just feels wrong as there are post-parsing methods.)
Let me know what you think.
Tainting is something we only apply to running VMs, so its hooked into the startup code path rather than XML parsing or validation time.
So generally you'd want to add to "qemuDomainObjCheckTaint" or one of the helpers it calls.
Yes! I had worked that out this week when I fiddle with it some more. I will send a patchset for this soon.
Also it sounds like there isn't a strong reason for tying up SMBIOS UUID and VM UUID except the use case of the guest inferring its hypervisor identifier. Would it make sense to propose a new device type that can canonically be used for that purpose? Something like Generation ID, perhaps. I can see if someone from our side can work on that if you think it's a good idea.
I'd suggest SMBIOS can already handle this. eg We could just document something along the lines if
If you make "System" (Type 1 table) "UUID" different from machine UUID, then set "Base Board" (Type 2 table) "Asset Tag" to hold the machine UUID
But that would mean the Asset Tag could change at VM runtime (well, at least while saved to disk). Wouldn't that be a problem?
I still have the question above.
Hmm, right, of course, SMBIOS can't change at runtime and even if we did change it, we can't expect the guest OS to update its view.
Lets just forget the whole thing about providing the libvirt UUID elsewhere. If and when someone claims later we can decide whether to worry about it, or show them a different way to get to the same end goal.
Upon reviewing some more code, my understanding is that the "-uuid" argument, in QEMU, is used to define parts of the virtual hardware. However, in libvirt, the <uuid> tag is used both to set the internal identifier for the VM and also for defining the "-uuid" argument. I think that can be improved by: 1. Adding more options to QEMU (and corresponding libvirt code) which allow fine-grained control of what UUIDs, in general, are used for different parts of the virtual hardware. The QEMU -uuid therefore becomes merely a default. 2. (probably optional of "1" above is complete) Breaking the relationship of libvirt's <uuid> and what it uses internally as a VM identifier OR the relationship of libvirt's <uuid> (which can remain an identifier) and what is passed to QEMU's -uuid. 3. Proposing an official device type akin to Generation ID which hypervisors can use to tell guests what is their hypervisor unique identifier. Based on your response above, I think we both agree leveraging SMBIOS is not adequate (although everyone does that... including Nutanix's Guest Tools). This can be done separately, though. My immediate concern is with VM fast-clone (the whole save-clone-restore idea) whilst maintaining the same virtual hardware (and those VMs do not have guest tools in my current use case). Thoughts welcome! ps. Congratulations on the big release earlier this week! F.
On Fri, Jan 17, 2025 at 01:11:54PM +0000, Felipe Franciosi wrote:
On 17 Jan 2025, at 12:47, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Jan 13, 2025 at 12:13:43PM +0000, Felipe Franciosi wrote:
On 10 Jan 2025, at 16:03, Felipe Franciosi <felipe@nutanix.com> wrote:
On 10 Jan 2025, at 15:02, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Fri, Jan 10, 2025 at 01:46:53PM +0000, Felipe Franciosi wrote:
> On 10 Jan 2025, at 12:38, Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Fri, Jan 10, 2025 at 12:31:00PM +0000, Felipe Franciosi wrote: >> >> >>> On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange@redhat.com> wrote: >>> >>> On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote: >>>> >>>> My understanding is that SMBIOS identifiers cannot change at runtime. >>> >>> Correct. >> >> Ok, but then I'm confused: how is clone supposed to work? When I clone a saved >> VM, Libvirt requires that I change its <UUID> and it also requires that this >> <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change. >> >> What am I missing? > > Yep, the idea of cloning a running (well saved) VM on the same > host is effectively denied due to this policy. > > We have never claimed that cloning a running VM is supported, > and actively discouraged people from trying to do this. > > The only workaround would be if launched on a different host. Failing > that the only option would be for us to remove the requirement that > VM UUID matches SMBIOS UUID. > > Perhaps we could do the latter, but mark the VM as "tainted" to indicate > this undesirable config scenario.
That works for me. Let me submit another patch along those lines for review.
I'm having a look at this and it isn't entirely clear to me how to do it.
I can see that virDomainObjParseXML() creates the virDomainObj *obj. Most of the interesting parsing then happens in virDomainDefParseXML() which (sensibly) takes the xmlXPathContextPtr and not the entire virDomainObj.
That means, however, that I can't directly call virDomainObjTaint() from the relevant parsing bit in virSysinfoSystemParseXML() to taint the obj with my newly introduced VIR_DOMAIN_TAINT_UUID_MISMATCH.
However, I can see that further down in virDomainObjParseXML() there's a virXPathNodeSet("./taint", ctxt, &taintNodes). I'm therefore guessing I don't have to call virDomainObjTaint() while parsing the XML. Instead, maybe I can just add "./taint" to the XML while parsing?
I can't immediately find any precedence to that, though. Directions welcome!
I did some more digging and found this:
------------8<------------ commit 7998465005e2ebf26f6e65f5bdb886487374bb18 Author: Daniel P. Berrangé <berrange@redhat.com> Date: Wed May 4 11:40:59 2011 +0100
Add field to virDomainObjPtr to track "tainting" ------------8<------------
Seems like the idea of that taint parsing in virDomainObjParseXML() is to do with <taint> tags added by virDomainObjFormat(). That answers one of my questions (the one to do with where the tags come from).
But I'm still unclear on how you'd prefer I raise this patch. Here are the options I have in mind:
1. Add a <taint> tag while parsing in virSysinfoSystemParseXML() (I don't really like this as no other "parse" method modifies the XML.)
2. Add a check somewhere later, e.g., in virDomainDefValidate() (This feels better, but virDomainDefValidate() doesn't take the virDomainObj, only the virDomainDef, so that may need to change.)
3. Taint the object in virDomainDefCheckABIStabilityFlags() (This has the same problem as the option above, so I prefer the former.)
4. Check it explicitly in virDomainObjParseXML() (That doesn't require passing the virDomainObj to the post-parsing methods, but it just feels wrong as there are post-parsing methods.)
Let me know what you think.
Tainting is something we only apply to running VMs, so its hooked into the startup code path rather than XML parsing or validation time.
So generally you'd want to add to "qemuDomainObjCheckTaint" or one of the helpers it calls.
Yes! I had worked that out this week when I fiddle with it some more. I will send a patchset for this soon.
Also it sounds like there isn't a strong reason for tying up SMBIOS UUID and VM UUID except the use case of the guest inferring its hypervisor identifier. Would it make sense to propose a new device type that can canonically be used for that purpose? Something like Generation ID, perhaps. I can see if someone from our side can work on that if you think it's a good idea.
I'd suggest SMBIOS can already handle this. eg We could just document something along the lines if
If you make "System" (Type 1 table) "UUID" different from machine UUID, then set "Base Board" (Type 2 table) "Asset Tag" to hold the machine UUID
But that would mean the Asset Tag could change at VM runtime (well, at least while saved to disk). Wouldn't that be a problem?
I still have the question above.
Hmm, right, of course, SMBIOS can't change at runtime and even if we did change it, we can't expect the guest OS to update its view.
Lets just forget the whole thing about providing the libvirt UUID elsewhere. If and when someone claims later we can decide whether to worry about it, or show them a different way to get to the same end goal.
Upon reviewing some more code, my understanding is that the "-uuid" argument, in QEMU, is used to define parts of the virtual hardware. However, in libvirt, the <uuid> tag is used both to set the internal identifier for the VM and also for defining the "-uuid" argument. I think that can be improved by:
1. Adding more options to QEMU (and corresponding libvirt code) which allow fine-grained control of what UUIDs, in general, are used for different parts of the virtual hardware. The QEMU -uuid therefore becomes merely a default.
Yes, if a device needs to have a UUID, it ought to be controllable, even if it defaults to the overall machine UUID.
2. (probably optional of "1" above is complete) Breaking the relationship of libvirt's <uuid> and what it uses internally as a VM identifier OR the relationship of libvirt's <uuid> (which can remain an identifier) and what is passed to QEMU's -uuid.
Don't think we need this, if we can control UUID on each individual device that wants a UUID.
3. Proposing an official device type akin to Generation ID which hypervisors can use to tell guests what is their hypervisor unique identifier. Based on your response above, I think we both agree leveraging SMBIOS is not adequate (although everyone does that... including Nutanix's Guest Tools).
Yep, but not a priority IMHO.
This can be done separately, though. My immediate concern is with VM fast-clone (the whole save-clone-restore idea) whilst maintaining the same virtual hardware (and those VMs do not have guest tools in my current use case). Thoughts welcome!
ps. Congratulations on the big release earlier this week!
Technically no libvirt releases are "big" releases. Both the major & minor parts of the version number are just boring time counters with no semantics wrt what's in the release :-) We follow a strictly time based release cadence, on the 1st of each month, except in Jan/Feb where we do a Jan 15th release instead of Jan 1st/Feb 1st to avoid new year holidays. We just bump the major version in Jan every year. QEMU does a similar bump major version for first release of each year. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (2)
-
Daniel P. Berrangé -
Felipe Franciosi