RFC: qemu: use uuid instead of name for misc filenames
Hi, everyone. I'm working on supporting domain renaming when it has snapshots which is not supported now. And it strikes me things will be much simplier to manage on renaming if we use uuid in filenames instead of domain names. 1. Renaming will only involve saving updated config. The saving is atomic thanx to tmp file and rename(2) approach. In constast current renaming on error paths can leave config with old or new name. Thus on libvirt restart extra VM will appear. And we don't need to rename autostart links, snapshot directories etc. 2. Renaming will be possible for running domains with no efforts. We only need to pass uuid instead of name in '-name guest=...' command line. 3. Mgmt can stop using autogenerated names for domains. I guess openstack for example uses names like instance-000002ff because we have many limitations on domain renaming. And if these limitations are removed then openstack can just use user supplied names for domains. 4. No issues with long domain names and filename length limit If the above conversion makes sense I guess the good time to apply it is on domain start (and rename to support renaming with snapshots). I guess we can also have tool (some virsh command) for developers to generate symlinks so one can access logs, configs etc by name instead of uuid. Nikolay
On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote:
Hi, everyone.
I'm working on supporting domain renaming when it has snapshots which is not supported now. And it strikes me things will be much simplier to manage on renaming if we use uuid in filenames instead of domain names.
1. Renaming will only involve saving updated config. The saving is atomic thanx to tmp file and rename(2) approach. In constast current renaming on error paths can leave config with old or new name. Thus on libvirt restart extra VM will appear.
And we don't need to rename autostart links, snapshot directories etc.
Yes, renaming is hard due to the non-atomic nature of the problem. It ought to be possible to recover/rollback from all but the most serious failure scenarios though. Problems in the most serious scenarios, are likely to cause failures in other parts of libvirt already, such as existing VM shutdown/startup.
2. Renaming will be possible for running domains with no efforts. We only need to pass uuid instead of name in '-name guest=...' command line.
Passing a uuid for the -name arg would be incorrect. This is a user visible string, for example displayed in the SDL window title or the VNC display name. UUIDs are not desirable for exposure to users, they are for machine usage.
3. Mgmt can stop using autogenerated names for domains. I guess openstack for example uses names like instance-000002ff because we have many limitations on domain renaming. And if these limitations are removed then openstack can just use user supplied names for domains.
In openstack the instance names are unique within the scope of a single project. In Libvirt the VM names are unique within the scope of a single libvirt driver connection (effectively a single host). Even if libvirt didn't use the name for on disk files, it will still have the requirement for unique names per-host in libvirt. This is the key reason why openstack uses "instance-$HEX" as the libvirt guest name. It could perhaps use "projectname-guestname" as the name, but i'm not sure there's much appetite for change in this respect, as it is a long standing convention now.
4. No issues with long domain names and filename length limit
If the above conversion makes sense I guess the good time to apply it is on domain start (and rename to support renaming with snapshots).
The above has not considered the benefit that using the VM name has. Essentially the UUID is good for machines, the VM name is good for humans. Seeing the guest XML files, or VM log files using a filename based on UUID instead of name is a *really* unappealing idea to me. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 27.02.2020 16:48, Daniel P. Berrangé wrote:
On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote:
Hi, everyone.
I'm working on supporting domain renaming when it has snapshots which is not supported now. And it strikes me things will be much simplier to manage on renaming if we use uuid in filenames instead of domain names.
1. Renaming will only involve saving updated config. The saving is atomic thanx to tmp file and rename(2) approach. In constast current renaming on error paths can leave config with old or new name. Thus on libvirt restart extra VM will appear.
And we don't need to rename autostart links, snapshot directories etc.
Yes, renaming is hard due to the non-atomic nature of the problem. It ought to be possible to recover/rollback from all but the most serious failure scenarios though. Problems in the most serious scenarios, are likely to cause failures in other parts of libvirt already, such as existing VM shutdown/startup.
2. Renaming will be possible for running domains with no efforts. We only need to pass uuid instead of name in '-name guest=...' command line.
Passing a uuid for the -name arg would be incorrect. This is a user visible string, for example displayed in the SDL window title or the VNC display name. UUIDs are not desirable for exposure to users, they are for machine usage.
Yes, this is not convinient. The broader idea was to stop passing name on command line as we can't change it after process is started. So instead we can omit name parameter altogether and set/update name thru QMP. Also with domain name in path renaming of an active domain looks really messy to implement.
3. Mgmt can stop using autogenerated names for domains. I guess openstack for example uses names like instance-000002ff because we have many limitations on domain renaming. And if these limitations are removed then openstack can just use user supplied names for domains.
In openstack the instance names are unique within the scope of a single project.
In Libvirt the VM names are unique within the scope of a single libvirt driver connection (effectively a single host).
Even if libvirt didn't use the name for on disk files, it will still have the requirement for unique names per-host in libvirt.
This is the key reason why openstack uses "instance-$HEX" as the libvirt guest name.
It could perhaps use "projectname-guestname" as the name, but i'm not sure there's much appetite for change in this respect, as it is a long standing convention now.
4. No issues with long domain names and filename length limit
If the above conversion makes sense I guess the good time to apply it is on domain start (and rename to support renaming with snapshots).
The above has not considered the benefit that using the VM name has. Essentially the UUID is good for machines, the VM name is good for humans. Seeing the guest XML files, or VM log files using a filename based on UUID instead of name is a *really* unappealing idea to me.
I agree. But we can also keep symlinks with domain names for configs/logs etc This can be done as a separate tool as I suggested in the letter or maintain symlinks always. The idea is failing in this symlinking won't affect daemon functionality as symlinks are for humans) Nikolay
On Fri, Feb 28, 2020 at 10:09:41AM +0300, Nikolay Shirokovskiy wrote:
On 27.02.2020 16:48, Daniel P. Berrangé wrote:
On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote:
Hi, everyone.
I'm working on supporting domain renaming when it has snapshots which is not supported now. And it strikes me things will be much simplier to manage on renaming if we use uuid in filenames instead of domain names.
1. Renaming will only involve saving updated config. The saving is atomic thanx to tmp file and rename(2) approach. In constast current renaming on error paths can leave config with old or new name. Thus on libvirt restart extra VM will appear.
And we don't need to rename autostart links, snapshot directories etc.
Yes, renaming is hard due to the non-atomic nature of the problem. It ought to be possible to recover/rollback from all but the most serious failure scenarios though. Problems in the most serious scenarios, are likely to cause failures in other parts of libvirt already, such as existing VM shutdown/startup.
2. Renaming will be possible for running domains with no efforts. We only need to pass uuid instead of name in '-name guest=...' command line.
Passing a uuid for the -name arg would be incorrect. This is a user visible string, for example displayed in the SDL window title or the VNC display name. UUIDs are not desirable for exposure to users, they are for machine usage.
Yes, this is not convinient. The broader idea was to stop passing name on command line as we can't change it after process is started. So instead we can omit name parameter altogether and set/update name thru QMP.
I see no problem in continuing to use -name at startup, but certainly if we wish to propagate renames, then we'll need a QMP command for that.
Also with domain name in path renaming of an active domain looks really messy to implement.
Yep, this goes to the disk image too, as its normal practice to name the root disk image, based on the VM name. Admins are not going to like looking in /var/lib/libvirt/images/ and seeing 100's of UUIDs. QEMU can be told to re-open disk images, via the blockjob QMP commands, which could allow us to rename disk images on running VMs. I wonder if a simple "rename()" works on UNIX sockets - I can't say I've ever tried it, but I think it probably would work.
3. Mgmt can stop using autogenerated names for domains. I guess openstack for example uses names like instance-000002ff because we have many limitations on domain renaming. And if these limitations are removed then openstack can just use user supplied names for domains.
In openstack the instance names are unique within the scope of a single project.
In Libvirt the VM names are unique within the scope of a single libvirt driver connection (effectively a single host).
Even if libvirt didn't use the name for on disk files, it will still have the requirement for unique names per-host in libvirt.
This is the key reason why openstack uses "instance-$HEX" as the libvirt guest name.
It could perhaps use "projectname-guestname" as the name, but i'm not sure there's much appetite for change in this respect, as it is a long standing convention now.
4. No issues with long domain names and filename length limit
If the above conversion makes sense I guess the good time to apply it is on domain start (and rename to support renaming with snapshots).
The above has not considered the benefit that using the VM name has. Essentially the UUID is good for machines, the VM name is good for humans. Seeing the guest XML files, or VM log files using a filename based on UUID instead of name is a *really* unappealing idea to me.
I agree. But we can also keep symlinks with domain names for configs/logs etc This can be done as a separate tool as I suggested in the letter or maintain symlinks always. The idea is failing in this symlinking won't affect daemon functionality as symlinks are for humans)
Have you faced specific failure scenarios with renaming in the real world production deployments, or is this largely a theoretical problem ? Given that libvirtd has full privileges over the files in question, it seems like the most likely causes of failure to rename some of the files are all scenarios where we'd be doomed no matter what. eg If an I/O error caused storage to be remounted read-only, the guest is doomed. If an NFS mount permissions are changed on the server, such that libvirtd can no longer rename, the guest is probably doomed too. If some bad admin has created clashing filenames behind the back of libvirt this is going to be problematic no matter what. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 06.03.2020 21:20, Daniel P. Berrangé wrote:
On Fri, Feb 28, 2020 at 10:09:41AM +0300, Nikolay Shirokovskiy wrote:
On 27.02.2020 16:48, Daniel P. Berrangé wrote:
On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote:
Hi, everyone.
I'm working on supporting domain renaming when it has snapshots which is not supported now. And it strikes me things will be much simplier to manage on renaming if we use uuid in filenames instead of domain names.
1. Renaming will only involve saving updated config. The saving is atomic thanx to tmp file and rename(2) approach. In constast current renaming on error paths can leave config with old or new name. Thus on libvirt restart extra VM will appear.
And we don't need to rename autostart links, snapshot directories etc.
Yes, renaming is hard due to the non-atomic nature of the problem. It ought to be possible to recover/rollback from all but the most serious failure scenarios though. Problems in the most serious scenarios, are likely to cause failures in other parts of libvirt already, such as existing VM shutdown/startup.
2. Renaming will be possible for running domains with no efforts. We only need to pass uuid instead of name in '-name guest=...' command line.
Passing a uuid for the -name arg would be incorrect. This is a user visible string, for example displayed in the SDL window title or the VNC display name. UUIDs are not desirable for exposure to users, they are for machine usage.
Yes, this is not convinient. The broader idea was to stop passing name on command line as we can't change it after process is started. So instead we can omit name parameter altogether and set/update name thru QMP.
I see no problem in continuing to use -name at startup, but certainly if we wish to propagate renames, then we'll need a QMP command for that.
I use quite regularly 'ps ax | grep $NAME' to check if process is running when debugging so having domain name in command line can be be confusing.
Also with domain name in path renaming of an active domain looks really messy to implement.
Yep, this goes to the disk image too, as its normal practice to name the root disk image, based on the VM name. Admins are not going to like looking in /var/lib/libvirt/images/ and seeing 100's of UUIDs.
QEMU can be told to re-open disk images, via the blockjob QMP commands, which could allow us to rename disk images on running VMs. I wonder if a simple "rename()" works on UNIX sockets - I can't say I've ever tried it, but I think it probably would work.
Yeah, renaming should work I guess. But together with renaming disk images it is lot of operations thus in case of errors we can end up with some paths renamed and some not.
3. Mgmt can stop using autogenerated names for domains. I guess openstack for example uses names like instance-000002ff because we have many limitations on domain renaming. And if these limitations are removed then openstack can just use user supplied names for domains.
In openstack the instance names are unique within the scope of a single project.
In Libvirt the VM names are unique within the scope of a single libvirt driver connection (effectively a single host).
Even if libvirt didn't use the name for on disk files, it will still have the requirement for unique names per-host in libvirt.
This is the key reason why openstack uses "instance-$HEX" as the libvirt guest name.
It could perhaps use "projectname-guestname" as the name, but i'm not sure there's much appetite for change in this respect, as it is a long standing convention now.
4. No issues with long domain names and filename length limit
If the above conversion makes sense I guess the good time to apply it is on domain start (and rename to support renaming with snapshots).
The above has not considered the benefit that using the VM name has. Essentially the UUID is good for machines, the VM name is good for humans. Seeing the guest XML files, or VM log files using a filename based on UUID instead of name is a *really* unappealing idea to me.
I agree. But we can also keep symlinks with domain names for configs/logs etc This can be done as a separate tool as I suggested in the letter or maintain symlinks always. The idea is failing in this symlinking won't affect daemon functionality as symlinks are for humans)
Have you faced specific failure scenarios with renaming in the real world production deployments, or is this largely a theoretical problem ?
Just theoretical.
Given that libvirtd has full privileges over the files in question, it seems like the most likely causes of failure to rename some of the files are all scenarios where we'd be doomed no matter what. eg If an I/O error caused storage to be remounted read-only, the guest is doomed. If an NFS mount permissions are changed on the server, such that libvirtd can no longer rename, the guest is probably doomed too. If some bad admin has created clashing filenames behind the back of libvirt this is going to be problematic no matter what.
Regards, Daniel
On Fri, Feb 28, 2020 at 10:09:41AM +0300, Nikolay Shirokovskiy wrote:
On 27.02.2020 16:48, Daniel P. Berrangé wrote:
On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote:
Hi, everyone.
I'm working on supporting domain renaming when it has snapshots which is not supported now. And it strikes me things will be much simplier to manage on renaming if we use uuid in filenames instead of domain names.
4. No issues with long domain names and filename length limit
If the above conversion makes sense I guess the good time to apply it is on domain start (and rename to support renaming with snapshots).
The above has not considered the benefit that using the VM name has. Essentially the UUID is good for machines, the VM name is good for humans. Seeing the guest XML files, or VM log files using a filename based on UUID instead of name is a *really* unappealing idea to me.
I agree. But we can also keep symlinks with domain names for configs/logs etc This can be done as a separate tool as I suggested in the letter or maintain symlinks always. The idea is failing in this symlinking won't affect daemon functionality as symlinks are for humans)
I've just realized that there is potential overlap between what we're discussing in this thread, and in the thread about localhost migration: https://www.redhat.com/archives/libvir-list/2020-February/msg00061.html In the localhost migration case, we need to be able to startup a new guest with the same name as an existing guest. The way we can achieve that is by thinking of localhost migration as being a pair of domain rename operations. ie, consider guest "foo" we want to localhost-migrate - Start target guest "foo-incoming" - Run live migration from "foo" -> "foo-incoming" - Migration completes, CPUs stop - Rename "foo" to "foo-outgoing" - Rename "foo-incoming" to "foo" - Tidy up migration state - Destroy source guest "foo-outgoing" In both this thread and the localhost migration thread, we seem to have come towards a view that symlinks are the only viable way to deal with the naming problem for resources on disk that are based on VM name. IOW, it would be desirable if whatever solution we take for symlink mgmt will solve the localhost migration and domain rename problems at the same time. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 26.03.2020 20:50, Daniel P. Berrangé wrote:
On Fri, Feb 28, 2020 at 10:09:41AM +0300, Nikolay Shirokovskiy wrote:
On 27.02.2020 16:48, Daniel P. Berrangé wrote:
On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote:
Hi, everyone.
I'm working on supporting domain renaming when it has snapshots which is not supported now. And it strikes me things will be much simplier to manage on renaming if we use uuid in filenames instead of domain names.
4. No issues with long domain names and filename length limit
If the above conversion makes sense I guess the good time to apply it is on domain start (and rename to support renaming with snapshots).
The above has not considered the benefit that using the VM name has. Essentially the UUID is good for machines, the VM name is good for humans. Seeing the guest XML files, or VM log files using a filename based on UUID instead of name is a *really* unappealing idea to me.
I agree. But we can also keep symlinks with domain names for configs/logs etc This can be done as a separate tool as I suggested in the letter or maintain symlinks always. The idea is failing in this symlinking won't affect daemon functionality as symlinks are for humans)
I've just realized that there is potential overlap between what we're discussing in this thread, and in the thread about localhost migration:
https://www.redhat.com/archives/libvir-list/2020-February/msg00061.html
In the localhost migration case, we need to be able to startup a new guest with the same name as an existing guest. The way we can achieve that is by thinking of localhost migration as being a pair of domain rename operations.
ie, consider guest "foo" we want to localhost-migrate
- Start target guest "foo-incoming" - Run live migration from "foo" -> "foo-incoming" - Migration completes, CPUs stop - Rename "foo" to "foo-outgoing" - Rename "foo-incoming" to "foo" - Tidy up migration state - Destroy source guest "foo-outgoing"
I think local migration does not fit really nicely in this scheme: - one can not treat outgoing and incoming VMs as just regular VMs as one can not put them into same list as they have same UUID - it is not just mere rename. In example reflected in [1] the path given by mgmt is not subjected to rename operation. The switch has to be done by local migration specific code. [1] https://www.redhat.com/archives/libvir-list/2020-February/msg01026.html
In both this thread and the localhost migration thread, we seem to have come towards a view that symlinks are the only viable way to deal with the naming problem for resources on disk that are based on VM name.
IOW, it would be desirable if whatever solution we take for symlink mgmt will solve the localhost migration and domain rename problems at the same time.
Agree, symlinks approach itself seems to work well in both cases. We can use naming scheme like UUID-gen for "stable" paths to fit both rename and local migration cases. Gen here is for generation, like 1 for domain after first local migration, 2 after second and so on. I already has a pending patch series [2] to remove some limitation on renaming. Can I treat this letter as some agreement that it is useful to move current paths naming towards "uuid based real path" + "name based symlinks" approach? [2] https://www.redhat.com/archives/libvir-list/2020-March/msg00018.html Nikolay
On Sun, Mar 29, 2020 at 02:33:41PM +0300, nshirokovskiy wrote:
On 26.03.2020 20:50, Daniel P. Berrangé wrote:
On Fri, Feb 28, 2020 at 10:09:41AM +0300, Nikolay Shirokovskiy wrote:
On 27.02.2020 16:48, Daniel P. Berrangé wrote:
On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote:
Hi, everyone.
I'm working on supporting domain renaming when it has snapshots which is not supported now. And it strikes me things will be much simplier to manage on renaming if we use uuid in filenames instead of domain names.
4. No issues with long domain names and filename length limit
If the above conversion makes sense I guess the good time to apply it is on domain start (and rename to support renaming with snapshots).
The above has not considered the benefit that using the VM name has. Essentially the UUID is good for machines, the VM name is good for humans. Seeing the guest XML files, or VM log files using a filename based on UUID instead of name is a *really* unappealing idea to me.
I agree. But we can also keep symlinks with domain names for configs/logs etc This can be done as a separate tool as I suggested in the letter or maintain symlinks always. The idea is failing in this symlinking won't affect daemon functionality as symlinks are for humans)
I've just realized that there is potential overlap between what we're discussing in this thread, and in the thread about localhost migration:
https://www.redhat.com/archives/libvir-list/2020-February/msg00061.html
In the localhost migration case, we need to be able to startup a new guest with the same name as an existing guest. The way we can achieve that is by thinking of localhost migration as being a pair of domain rename operations.
ie, consider guest "foo" we want to localhost-migrate
- Start target guest "foo-incoming" - Run live migration from "foo" -> "foo-incoming" - Migration completes, CPUs stop - Rename "foo" to "foo-outgoing" - Rename "foo-incoming" to "foo" - Tidy up migration state - Destroy source guest "foo-outgoing"
I think local migration does not fit really nicely in this scheme:
- one can not treat outgoing and incoming VMs as just regular VMs as one can not put them into same list as they have same UUID
Yes, that is a tricky issue, but one that we need to solve, as the need to have a completely separate of list VMs is the thing I dislike the most about the local migration patches. One option is to make the target VM have a different UUID by pickling its UUID. eg have a migration UUID generated on daemon startup. 0466e1ae-a71a-4e75-89ca-c3591a4cf220. Then XOR this migration UUID with the source VM's UUID. So during live migration the target VM will appear with this XOR'd UUID, and once completed, it will get the real UUID again. A different option is to not keep the target VM in the domain list at all. Instead virDomainObjPtr, could have a pointer to a second virDomainObjPtr which stores the target VM temporarily.
- it is not just mere rename. In example reflected in [1] the path given by mgmt is not subjected to rename operation. The switch has to be done by local migration specific code.
[1] https://www.redhat.com/archives/libvir-list/2020-February/msg01026.html
That is true right now, but I'm thinking that we need to solve this particular case, even for the domain rename operation. For a graphics backend using UNIX socket, the socket path is almost certainly going to be based off the guest name. So when renaming a guest we really want to be able to rename the UNIX socket. Thus I think that we need to introduce a new way to do UNIX sockets for graphics, using an "autopath=yes|no" attribute, in the same way that we have autoport=yes|no for TCP sockets. With autopath=yes, then the path can be changed during a domain rename operation, and also handled for local migration.
In both this thread and the localhost migration thread, we seem to have come towards a view that symlinks are the only viable way to deal with the naming problem for resources on disk that are based on VM name.
IOW, it would be desirable if whatever solution we take for symlink mgmt will solve the localhost migration and domain rename problems at the same time.
Agree, symlinks approach itself seems to work well in both cases. We can use naming scheme like UUID-gen for "stable" paths to fit both rename and local migration cases. Gen here is for generation, like 1 for domain after first local migration, 2 after second and so on.
We only really need 2 "generations", as we can oscillate back & forth between the two after each migration, hence my thought of using a global migration UUID which we just XOR with the real UUID to get the alternate.
I already has a pending patch series [2] to remove some limitation on renaming. Can I treat this letter as some agreement that it is useful to move current paths naming towards "uuid based real path" + "name based symlinks" approach?
Yes, I think we can move forward. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 30.03.2020 13:41, Daniel P. Berrangé wrote:
On Sun, Mar 29, 2020 at 02:33:41PM +0300, nshirokovskiy wrote:
On 26.03.2020 20:50, Daniel P. Berrangé wrote:
On Fri, Feb 28, 2020 at 10:09:41AM +0300, Nikolay Shirokovskiy wrote:
On 27.02.2020 16:48, Daniel P. Berrangé wrote:
On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote:
Hi, everyone.
I'm working on supporting domain renaming when it has snapshots which is not supported now. And it strikes me things will be much simplier to manage on renaming if we use uuid in filenames instead of domain names.
4. No issues with long domain names and filename length limit
If the above conversion makes sense I guess the good time to apply it is on domain start (and rename to support renaming with snapshots).
The above has not considered the benefit that using the VM name has. Essentially the UUID is good for machines, the VM name is good for humans. Seeing the guest XML files, or VM log files using a filename based on UUID instead of name is a *really* unappealing idea to me.
I agree. But we can also keep symlinks with domain names for configs/logs etc This can be done as a separate tool as I suggested in the letter or maintain symlinks always. The idea is failing in this symlinking won't affect daemon functionality as symlinks are for humans)
I've just realized that there is potential overlap between what we're discussing in this thread, and in the thread about localhost migration:
https://www.redhat.com/archives/libvir-list/2020-February/msg00061.html
In the localhost migration case, we need to be able to startup a new guest with the same name as an existing guest. The way we can achieve that is by thinking of localhost migration as being a pair of domain rename operations.
ie, consider guest "foo" we want to localhost-migrate
- Start target guest "foo-incoming" - Run live migration from "foo" -> "foo-incoming" - Migration completes, CPUs stop - Rename "foo" to "foo-outgoing" - Rename "foo-incoming" to "foo" - Tidy up migration state - Destroy source guest "foo-outgoing"
I think local migration does not fit really nicely in this scheme:
- one can not treat outgoing and incoming VMs as just regular VMs as one can not put them into same list as they have same UUID
Yes, that is a tricky issue, but one that we need to solve, as the need to have a completely separate of list VMs is the thing I dislike the most about the local migration patches.
One option is to make the target VM have a different UUID by pickling its UUID. eg have a migration UUID generated on daemon startup. 0466e1ae-a71a-4e75-89ca-c3591a4cf220. Then XOR this migration UUID with the source VM's UUID. So during live migration the target VM will appear with this XOR'd UUID, and once completed, it will get the real UUID again.
A different option is to not keep the target VM in the domain list at all. Instead virDomainObjPtr, could have a pointer to a second virDomainObjPtr which stores the target VM temporarily.
Both choices have its issues/advantages With the first approach incoming VM is visible as regular one. This can be beneficial that one can inspect the VM in debug purpuose just like regular one. On the other hand the appearance of the VM can be unexpected to mgmt thus some may mgmt even try to destroy it. So the other approach looks more transparent from mgmt POV. I should say in Virtuozzo we have a patch series for local migration. I decided not to send it to the list as previously the decision was that the trade off for complexity/utility is not on the feature side. I decided to use the latter approach to keep the link to the peer. The link is bidirectional, thus is is very simple to get peer object from both sides. I want to mention another decision that turns to be successful - using same mutex for both domain objects. This way we don't need to care of "locking order"/"re-locking to keep the locking order" to avoid deadlocks. Accessing peer object is as simple as vm->peer->... And this plays nicely with current migration code when we drop the lock at the end of migration phase so that next phase can take the same lock. As to handling clashing resources we support next cases: - tap interfaces - tcp/unix chardev backends - vnc For taps we use multiqueue mode so that we can have multiple fds for a tap during local migration. The traffic is splitted somehow between them so effectively traffic to the domain is damaged during the migration but this should not go on for a long time and tcp is supposed to handle it. For chardev we decided to use unplug/plug approach for backends. Incoming domain is started with null backend then later right after CPU's of outgoing domain are stopped backend is unplugged from the outgoing domain and plugged to to incoming domain. I guess this is a bit worse than the approach with symlinks as there is gap in time when there is no backend available to client in case of server mode. At the same time this is more suitable for client mode so there is always just single client connection. For the vnc approach is similar to chardevs - unplug after CPU's are stopped from outgoing domain and plug into incoming domain. This way we can make migration even without autoport. Nikolay
participants (3)
-
Daniel P. Berrangé -
Nikolay Shirokovskiy -
nshirokovskiy