Domain XML of a restored VM
Hi, I am looking for clarification on what exactly is happening to the domain XML when resuming a suspended VM. We would like to make sure we don't lose any configuration and we also see some unexpected behavior. The VM with the name/UUID same as resumed VM may or may not already exist. If it does, what role does the original domain XML of a VM play when resuming? Is there any merging taking place between the existing domain XML and the domain XML stored in save image? Is the original XML simply thrown away and replaced? Is there any difference how the stored XML is treated compared to the XML passed in `virDomainRestoreFlags()`? I see some unexpected changes here. When I use `virDomainSaveImageGetXMLDesc()` to retrieve the stored XML and pass it unchanged to the `virDomainRestoreFlags()` the final domain XML is different then when `virDomainRestore()` is used or when `virDomainRestoreFlags()` with no `dxml` argument is used. We have: <graphics ... passwdValidTo='1970-01-01T00:00:01'> in the domain XML and the `passwdValidTo` argument disappears when domain XML is passed in `dxml` argument but it is preserved in other cases. Is this behavior expected? If yes, can we do anything to preserve the configuration? What other changes in domain XML can be expected? Thanks, Tomas -- Tomáš Golembiovský <tgolembi@redhat.com>
On 6/10/22 15:31, Tomáš Golembiovský wrote:
Hi,
I am looking for clarification on what exactly is happening to the domain XML when resuming a suspended VM. We would like to make sure we don't lose any configuration and we also see some unexpected behavior.
The VM with the name/UUID same as resumed VM may or may not already exist. If it does, what role does the original domain XML of a VM play when resuming? Is there any merging taking place between the existing domain XML and the domain XML stored in save image? Is the original XML simply thrown away and replaced?
A guest can have up to two XMLs: 1) defined guests have so called 'inactive' or 'config' XML. This XML does not contain any runtime information, and is used as the initial configuration when starting the guest. Once the guest is running, it'll also have ... 2) live XML. This XML contains runtime information and because of hotplug and plenty of APIs that tweak various XML knobs (virDomainSetMemoryParameters(), virDomainSetInterfaceParameters(), etc.) can be very different to the inactive XML. Changes to the live XML are thrown away as soon as the guest is shut off and we're back to square 1 where only the inactive XML is preserved (and used as the initial config for the next run). However, this live XML is very important because it reflects the guest ABI. For instance, some of our APIs offer a way for users to supply modified domain XML (as you say below, virDomainRestoreFlags() is onw of them, migration APIs are another). After this user supplied XML is parsed, it's thrown at an excessive ABI stability checker, who's only goal is to make sure that the live XML maintained by Libvirt and the user provided XML differ only in those ways that won't break guest ABI (for instance, changing the port where SPICE is allowed, removing a <disk/> isn't). Having said all of this - the XML that's saved among the domain by virDomainSave() API is the live XML. Because that's the one that reflects the current state of the saved image. Inactive XML is disconnected from all of this.
Is there any difference how the stored XML is treated compared to the XML passed in `virDomainRestoreFlags()`? I see some unexpected changes here. When I use `virDomainSaveImageGetXMLDesc()` to retrieve the stored XML and pass it unchanged to the `virDomainRestoreFlags()` the final domain XML is different then when `virDomainRestore()` is used or when `virDomainRestoreFlags()` with no `dxml` argument is used. We have:
<graphics ... passwdValidTo='1970-01-01T00:00:01'>
in the domain XML and the `passwdValidTo` argument disappears when domain XML is passed in `dxml` argument but it is preserved in other cases. Is this behavior expected? If yes, can we do anything to preserve the configuration? What other changes in domain XML can be expected?
No, this behaviour is definitely not expected. As explained above, runtime values, that don't affect guest ABI are allowed to change. The rest is not. Password timeout should be allowed to change, because guest has no idea about the password in the first place. This is all done in the backed (that part of SPICE that faces the host/clients). Michal
On Mon, Jun 13, 2022 at 05:15:13PM +0200, Michal Prívozník wrote:
On 6/10/22 15:31, Tomáš Golembiovský wrote:
Hi,
I am looking for clarification on what exactly is happening to the domain XML when resuming a suspended VM. We would like to make sure we don't lose any configuration and we also see some unexpected behavior.
The VM with the name/UUID same as resumed VM may or may not already exist. If it does, what role does the original domain XML of a VM play when resuming? Is there any merging taking place between the existing domain XML and the domain XML stored in save image? Is the original XML simply thrown away and replaced?
A guest can have up to two XMLs:
1) defined guests have so called 'inactive' or 'config' XML. This XML does not contain any runtime information, and is used as the initial configuration when starting the guest. Once the guest is running, it'll also have ...
I see, so this confirms it. When restoring a running VM this XML is not taken into account at all.
2) live XML. This XML contains runtime information and because of hotplug and plenty of APIs that tweak various XML knobs (virDomainSetMemoryParameters(), virDomainSetInterfaceParameters(), etc.) can be very different to the inactive XML. Changes to the live XML are thrown away as soon as the guest is shut off and we're back to square 1 where only the inactive XML is preserved (and used as the initial config for the next run).
However, this live XML is very important because it reflects the guest ABI. For instance, some of our APIs offer a way for users to supply modified domain XML (as you say below, virDomainRestoreFlags() is onw of them, migration APIs are another). After this user supplied XML is parsed, it's thrown at an excessive ABI stability checker, who's only goal is to make sure that the live XML maintained by Libvirt and the user provided XML differ only in those ways that won't break guest ABI (for instance, changing the port where SPICE is allowed, removing a <disk/> isn't).
Having said all of this - the XML that's saved among the domain by virDomainSave() API is the live XML. Because that's the one that reflects the current state of the saved image. Inactive XML is disconnected from all of this.
Is there any difference how the stored XML is treated compared to the XML passed in `virDomainRestoreFlags()`? I see some unexpected changes here. When I use `virDomainSaveImageGetXMLDesc()` to retrieve the stored XML and pass it unchanged to the `virDomainRestoreFlags()` the final domain XML is different then when `virDomainRestore()` is used or when `virDomainRestoreFlags()` with no `dxml` argument is used. We have:
<graphics ... passwdValidTo='1970-01-01T00:00:01'>
in the domain XML and the `passwdValidTo` argument disappears when domain XML is passed in `dxml` argument but it is preserved in other cases. Is this behavior expected? If yes, can we do anything to preserve the configuration? What other changes in domain XML can be expected?
No, this behaviour is definitely not expected. As explained above, runtime values, that don't affect guest ABI are allowed to change. The rest is not. Password timeout should be allowed to change, because guest has no idea about the password in the first place. This is all done in the backed (that part of SPICE that faces the host/clients).
I have already figured out what is happening here. The problem is missing secrets. Libvirt discards `passwdValidTo` if there is no `passwd` argument too. The solution is to either supply new password or use VIR_DOMAIN_SAVE_IMAGE_XML_SECURE flag when retrieving the saved XML. Tomas -- Tomáš Golembiovský <tgolembi@redhat.com>
participants (2)
-
Michal Prívozník -
Tomáš Golembiovský