On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote: > Hi, everyone. > > I'm working on supporting domain renaming when it has snapshots which is not > supported now. And it strikes me things will be much simplier to manage on > renaming if we use uuid in filenames instead of domain names. > > 1. Renaming will only involve saving updated config. > The saving is atomic thanx to tmp file and rename(2) approach. In constast > current renaming on error paths can leave config with old or new name. Thus > on libvirt restart extra VM will appear. > > And we don't need to rename autostart links, snapshot directories etc. Yes, renaming is hard due to the non-atomic nature of the problem. It ought to be possible to recover/rollback from all but the most serious failure scenarios though. Problems in the most serious scenarios, are likely to cause failures in other parts of libvirt already, such as existing VM shutdown/startup. > 2. Renaming will be possible for running domains with no efforts. > We only need to pass uuid instead of name in '-name guest=...' command line. Passing a uuid for the -name arg would be incorrect. This is a user visible string, for example displayed in the SDL window title or the VNC display name. UUIDs are not desirable for exposure to users, they are for machine usage.

> > 3. Mgmt can stop using autogenerated names for domains. > I guess openstack for example uses names like instance-000002ff because we > have many limitations on domain renaming. And if these limitations are removed > then openstack can just use user supplied names for domains. In openstack the instance names are unique within the scope of a single project. In Libvirt the VM names are unique within the scope of a single libvirt driver connection (effectively a single host). Even if libvirt didn't use the name for on disk files, it will still have the requirement for unique names per-host in libvirt. This is the key reason why openstack uses "instance-$HEX" as the libvirt guest name. It could perhaps use "projectname-guestname" as the name, but i'm not sure there's much appetite for change in this respect, as it is a long standing convention now. > 4. No issues with long domain names and filename length limit > > If the above conversion makes sense I guess the good time to apply it is > on domain start (and rename to support renaming with snapshots). The above has not considered the benefit that using the VM name has. Essentially the UUID is good for machines, the VM name is good for humans. Seeing the guest XML files, or VM log files using a filename based on UUID instead of name is a *really* unappealing idea to me.

On Fri, Feb 28, 2020 at 10:09:41AM +0300, Nikolay Shirokovskiy wrote: > On 27.02.2020 16:48, Daniel P. Berrangé wrote: >> On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote: >>> Hi, everyone. >>> >>> I'm working on supporting domain renaming when it has snapshots which is not >>> supported now. And it strikes me things will be much simplier to manage on >>> renaming if we use uuid in filenames instead of domain names. >>> >>> 1. Renaming will only involve saving updated config. >>> The saving is atomic thanx to tmp file and rename(2) approach. In constast >>> current renaming on error paths can leave config with old or new name. Thus >>> on libvirt restart extra VM will appear. >>> >>> And we don't need to rename autostart links, snapshot directories etc. >> >> Yes, renaming is hard due to the non-atomic nature of the problem. >> It ought to be possible to recover/rollback from all but the most >> serious failure scenarios though. Problems in the most serious >> scenarios, are likely to cause failures in other parts of libvirt >> already, such as existing VM shutdown/startup. >> >> >>> 2. Renaming will be possible for running domains with no efforts. >>> We only need to pass uuid instead of name in '-name guest=...' command line. >> >> Passing a uuid for the -name arg would be incorrect. This is a user >> visible string, for example displayed in the SDL window title or >> the VNC display name. UUIDs are not desirable for exposure to users, >> they are for machine usage. > > Yes, this is not convinient. The broader idea was to stop passing name on command > line as we can't change it after process is started. So instead we can omit name > parameter altogether and set/update name thru QMP. I see no problem in continuing to use -name at startup, but certainly if we wish to propagate renames, then we'll need a QMP command for that.

> Also with domain name in path renaming of an active domain looks really messy > to implement. Yep, this goes to the disk image too, as its normal practice to name the root disk image, based on the VM name. Admins are not going to like looking in /var/lib/libvirt/images/ and seeing 100's of UUIDs. QEMU can be told to re-open disk images, via the blockjob QMP commands, which could allow us to rename disk images on running VMs. I wonder if a simple "rename()" works on UNIX sockets - I can't say I've ever tried it, but I think it probably would work.

>>> 3. Mgmt can stop using autogenerated names for domains. >>> I guess openstack for example uses names like instance-000002ff because we >>> have many limitations on domain renaming. And if these limitations are removed >>> then openstack can just use user supplied names for domains. >> >> In openstack the instance names are unique within the scope of a single >> project. >> >> In Libvirt the VM names are unique within the scope of a single libvirt >> driver connection (effectively a single host). >> >> Even if libvirt didn't use the name for on disk files, it will still have >> the requirement for unique names per-host in libvirt. >> >> This is the key reason why openstack uses "instance-$HEX" as the libvirt >> guest name. >> >> It could perhaps use "projectname-guestname" as the name, but i'm not >> sure there's much appetite for change in this respect, as it is a long >> standing convention now. >> >>> 4. No issues with long domain names and filename length limit >>> >>> If the above conversion makes sense I guess the good time to apply it is >>> on domain start (and rename to support renaming with snapshots). >> >> The above has not considered the benefit that using the VM name >> has. Essentially the UUID is good for machines, the VM name is >> good for humans. Seeing the guest XML files, or VM log files >> using a filename based on UUID instead of name is a *really* >> unappealing idea to me. > > I agree. But we can also keep symlinks with domain names for configs/logs etc > This can be done as a separate tool as I suggested in the letter or maintain > symlinks always. The idea is failing in this symlinking won't affect daemon > functionality as symlinks are for humans) Have you faced specific failure scenarios with renaming in the real world production deployments, or is this largely a theoretical problem ?

Given that libvirtd has full privileges over the files in question, it seems like the most likely causes of failure to rename some of the files are all scenarios where we'd be doomed no matter what. eg If an I/O error caused storage to be remounted read-only, the guest is doomed. If an NFS mount permissions are changed on the server, such that libvirtd can no longer rename, the guest is probably doomed too. If some bad admin has created clashing filenames behind the back of libvirt this is going to be problematic no matter what. Regards, Daniel

On Fri, Feb 28, 2020 at 10:09:41AM +0300, Nikolay Shirokovskiy wrote: > On 27.02.2020 16:48, Daniel P. Berrangé wrote: >> On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote: >>> Hi, everyone. >>> >>> I'm working on supporting domain renaming when it has snapshots which is not >>> supported now. And it strikes me things will be much simplier to manage on >>> renaming if we use uuid in filenames instead of domain names. >>> >>> 4. No issues with long domain names and filename length limit >>> >>> If the above conversion makes sense I guess the good time to apply it is >>> on domain start (and rename to support renaming with snapshots). >> >> The above has not considered the benefit that using the VM name >> has. Essentially the UUID is good for machines, the VM name is >> good for humans. Seeing the guest XML files, or VM log files >> using a filename based on UUID instead of name is a *really* >> unappealing idea to me. > > I agree. But we can also keep symlinks with domain names for configs/logs etc > This can be done as a separate tool as I suggested in the letter or maintain > symlinks always. The idea is failing in this symlinking won't affect daemon > functionality as symlinks are for humans) I've just realized that there is potential overlap between what we're discussing in this thread, and in the thread about localhost migration: https://www.redhat.com/archives/libvir-list/2020-February/msg00061.html In the localhost migration case, we need to be able to startup a new guest with the same name as an existing guest. The way we can achieve that is by thinking of localhost migration as being a pair of domain rename operations. ie, consider guest "foo" we want to localhost-migrate - Start target guest "foo-incoming" - Run live migration from "foo" -> "foo-incoming" - Migration completes, CPUs stop - Rename "foo" to "foo-outgoing" - Rename "foo-incoming" to "foo" - Tidy up migration state - Destroy source guest "foo-outgoing"

In both this thread and the localhost migration thread, we seem to have come towards a view that symlinks are the only viable way to deal with the naming problem for resources on disk that are based on VM name. IOW, it would be desirable if whatever solution we take for symlink mgmt will solve the localhost migration and domain rename problems at the same time.

On Sun, Mar 29, 2020 at 02:33:41PM +0300, nshirokovskiy wrote: > > > On 26.03.2020 20:50, Daniel P. Berrangé wrote: >> On Fri, Feb 28, 2020 at 10:09:41AM +0300, Nikolay Shirokovskiy wrote: >>> On 27.02.2020 16:48, Daniel P. Berrangé wrote: >>>> On Thu, Feb 27, 2020 at 03:57:04PM +0300, Nikolay Shirokovskiy wrote: >>>>> Hi, everyone. >>>>> >>>>> I'm working on supporting domain renaming when it has snapshots which is not >>>>> supported now. And it strikes me things will be much simplier to manage on >>>>> renaming if we use uuid in filenames instead of domain names. >>>>> >> >> >> >>>>> 4. No issues with long domain names and filename length limit >>>>> >>>>> If the above conversion makes sense I guess the good time to apply it is >>>>> on domain start (and rename to support renaming with snapshots). >>>> >>>> The above has not considered the benefit that using the VM name >>>> has. Essentially the UUID is good for machines, the VM name is >>>> good for humans. Seeing the guest XML files, or VM log files >>>> using a filename based on UUID instead of name is a *really* >>>> unappealing idea to me. >>> >>> I agree. But we can also keep symlinks with domain names for configs/logs etc >>> This can be done as a separate tool as I suggested in the letter or maintain >>> symlinks always. The idea is failing in this symlinking won't affect daemon >>> functionality as symlinks are for humans) >> >> I've just realized that there is potential overlap between what we're >> discussing in this thread, and in the thread about localhost migration: >> >> https://www.redhat.com/archives/libvir-list/2020-February/msg00061.html >> >> In the localhost migration case, we need to be able to startup a new >> guest with the same name as an existing guest. The way we can achieve >> that is by thinking of localhost migration as being a pair of domain >> rename operations. >> >> ie, consider guest "foo" we want to localhost-migrate >> >> - Start target guest "foo-incoming" >> - Run live migration from "foo" -> "foo-incoming" >> - Migration completes, CPUs stop >> - Rename "foo" to "foo-outgoing" >> - Rename "foo-incoming" to "foo" >> - Tidy up migration state >> - Destroy source guest "foo-outgoing" > > I think local migration does not fit really nicely in this scheme: > > - one can not treat outgoing and incoming VMs as just regular VMs as > one can not put them into same list as they have same UUID Yes, that is a tricky issue, but one that we need to solve, as the need to have a completely separate of list VMs is the thing I dislike the most about the local migration patches. One option is to make the target VM have a different UUID by pickling its UUID. eg have a migration UUID generated on daemon startup. 0466e1ae-a71a-4e75-89ca-c3591a4cf220. Then XOR this migration UUID with the source VM's UUID. So during live migration the target VM will appear with this XOR'd UUID, and once completed, it will get the real UUID again. A different option is to not keep the target VM in the domain list at all. Instead virDomainObjPtr, could have a pointer to a second virDomainObjPtr which stores the target VM temporarily.