[libvirt] Local qemu migration
Hi, Attempting to migration from session to system qemu fails because of the following checks in qemuMigrationCookieXMLParse(): if (STREQ(mig->remoteHostname, mig->localHostname)) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Attempt to migrate guest to the same host %s"), mig->remoteHostname); goto error; } .... if (memcmp(mig->remoteHostuuid, mig->localHostuuid, VIR_UUID_BUFLEN) == 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Attempt to migrate guest to the same host %s"), tmp); goto error; Is there a technical limitation for this error? If not, could it be overriden with an additional flag? cheers -- Marc-André Lureau
On 11/04/2014 10:43 AM, Marc-André Lureau wrote:
Hi,
Attempting to migration from session to system qemu fails because of the following checks in qemuMigrationCookieXMLParse():
if (STREQ(mig->remoteHostname, mig->localHostname)) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Attempt to migrate guest to the same host %s"), mig->remoteHostname); goto error; } .... if (memcmp(mig->remoteHostuuid, mig->localHostuuid, VIR_UUID_BUFLEN) == 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Attempt to migrate guest to the same host %s"), tmp); goto error;
Is there a technical limitation for this error? If not, could it be overriden with an additional flag?
Trying to prevent migrating to the same libvirtd instance which would historically deadlock in libvirt's qemu driver. But migrating session -> system shouldn't suffer from that problem, so maybe it should be smarter. Maybe with the more fine grained locking these days we can actually make localhost migration work? Would be useful for testing - Cole
On Tue, Nov 04, 2014 at 10:49:44AM -0500, Cole Robinson wrote:
On 11/04/2014 10:43 AM, Marc-André Lureau wrote:
Hi,
Attempting to migration from session to system qemu fails because of the following checks in qemuMigrationCookieXMLParse():
if (STREQ(mig->remoteHostname, mig->localHostname)) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Attempt to migrate guest to the same host %s"), mig->remoteHostname); goto error; } .... if (memcmp(mig->remoteHostuuid, mig->localHostuuid, VIR_UUID_BUFLEN) == 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Attempt to migrate guest to the same host %s"), tmp); goto error;
Is there a technical limitation for this error? If not, could it be overriden with an additional flag?
Trying to prevent migrating to the same libvirtd instance which would historically deadlock in libvirt's qemu driver. But migrating session -> system shouldn't suffer from that problem, so maybe it should be smarter.
Maybe with the more fine grained locking these days we can actually make localhost migration work? Would be useful for testing
Nope, there's stuff that's on the filesystem that is tied to the QEMU process and that would clash when migrating as you have two copies of the same VM running at the sme time. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Hi On Wed, Nov 5, 2014 at 9:10 AM, Daniel P. Berrange <berrange@redhat.com> wrote:
Nope, there's stuff that's on the filesystem that is tied to the QEMU process and that would clash when migrating as you have two copies of the same VM running at the sme time.
Ok, there is no guarantee that the VMs won't write concurrently to the disk during migration? Also, there are cases where VM disks/images are readonly, or image-less VMs. Except the obvious testing case, there are other use cases that could be interesting: moving a running VM from system to session libvirt, or the other way around, or to a different user. -- Marc-André Lureau
On Wed, Nov 05, 2014 at 01:16:08PM +0100, Marc-André Lureau wrote:
Hi
On Wed, Nov 5, 2014 at 9:10 AM, Daniel P. Berrange <berrange@redhat.com> wrote:
Nope, there's stuff that's on the filesystem that is tied to the QEMU process and that would clash when migrating as you have two copies of the same VM running at the sme time.
Ok, there is no guarantee that the VMs won't write concurrently to the disk during migration?
Also, there are cases where VM disks/images are readonly, or image-less VMs.
No, it isn't about the disk images. The actual guest CPUs are only executing in one QEMU at a time, so there's not a problem wrt disk image usage. The issue is with various other files we use. For example the QEMU monitor is a UNIX domain socket at a specific path - you can't have both QEMUs have the same UNIX domain socket. That's not under user control so we could change the monitor socket path, but the problem extends to stuff that is directly from the guest XML. ie VNC can be told to listen on a UNIX domain socket path. virtio-serial devices are typically bound to UNIX domain sockets. Serial ports, parallel ports, certain network device backends, and so on. Migration fundamentally requires that the two QEMU processes involved have completely separate filesystem namespaces, which effectively means separate hosts (or at least separate containers which is effectively the same thing)
Except the obvious testing case, there are other use cases that could be interesting: moving a running VM from system to session libvirt, or the other way around, or to a different user.
Migrating from session to system libvirt or vica-verca isn't something that is reasonable to support either because of the different privilege levels. The session QEMU will require the disk images to be owned by one user account, the system QEMU will require them to be owned by a different user account. We can't chown the images to satisfy both the system and session QEMU at the same time. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 11/05/2014 01:28 PM, Daniel P. Berrange wrote:
On Wed, Nov 05, 2014 at 01:16:08PM +0100, Marc-André Lureau wrote:
Hi
On Wed, Nov 5, 2014 at 9:10 AM, Daniel P. Berrange <berrange@redhat.com> wrote:
Nope, there's stuff that's on the filesystem that is tied to the QEMU process and that would clash when migrating as you have two copies of the same VM running at the sme time.
Ok, there is no guarantee that the VMs won't write concurrently to the disk during migration?
Also, there are cases where VM disks/images are readonly, or image-less VMs.
No, it isn't about the disk images. The actual guest CPUs are only executing in one QEMU at a time, so there's not a problem wrt disk image usage. The issue is with various other files we use. For example the QEMU monitor is a UNIX domain socket at a specific path - you can't have both QEMUs have the same UNIX domain socket. That's not under user control so we could change the monitor socket path, but the problem extends to stuff that is directly from the guest XML. ie VNC can be told to listen on a UNIX domain socket path. virtio-serial devices are typically bound to UNIX domain sockets. Serial ports, parallel ports, certain network device backends, and so on.
The monitor socket location is different for qemu:///system than for qemu:///session (or when going between two different users' qemu:///session). But things like the VNC port do start to be an issue.
Migration fundamentally requires that the two QEMU processes involved have completely separate filesystem namespaces, which effectively means separate hosts (or at least separate containers which is effectively the same thing)
Except the obvious testing case, there are other use cases that could be interesting: moving a running VM from system to session libvirt, or the other way around, or to a different user.
Migrating from session to system libvirt or vica-verca isn't something that is reasonable to support either because of the different privilege levels. The session QEMU will require the disk images to be owned by one user account, the system QEMU will require them to be owned by a different user account. We can't chown the images to satisfy both the system and session QEMU at the same time.
This, coupled with the fact that we don't allow connection to a remote qemu:///session instance (right now, the RPC code assumes that it will only connect to qemu:///system), means that it is not possible to do live migration in or out of a session instance, regardless of whether it is on the same machine, and regardless of whether it is between the session libvirtd of two different users. It might be possible to migrate to file (such as virsh save), then update permissions on the save file and other resources, then reload from that file; but it is not a live migration. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Wed, Nov 5, 2014 at 2:54 PM, Eric Blake <eblake@redhat.com> wrote:
The monitor socket location is different for qemu:///system than for qemu:///session (or when going between two different users' qemu:///session). But things like the VNC port do start to be an issue.
Not if the port is allocated, or the unix path is namespaced. But I understand better where the problems may come from. Still, I would imagine that libvirt would simply fail to migrate and no bad things would happen in those cases.
Migration fundamentally requires that the two QEMU processes involved have completely separate filesystem namespaces, which effectively means separate hosts (or at least separate containers which is effectively the same thing)
Except the obvious testing case, there are other use cases that could be interesting: moving a running VM from system to session libvirt, or the other way around, or to a different user.
Migrating from session to system libvirt or vica-verca isn't something that is reasonable to support either because of the different privilege levels. The session QEMU will require the disk images to be owned by one user account, the system QEMU will require them to be owned by a different user account. We can't chown the images to satisfy both the system and session QEMU at the same time.
This, coupled with the fact that we don't allow connection to a remote qemu:///session instance (right now, the RPC code assumes that it will only connect to qemu:///system), means that it is not possible to do live migration in or out of a session instance, regardless of whether it is on the same machine, and regardless of whether it is between the session libvirtd of two different users.
Well, I managed to migrate from system to session with a disk less VM, it was useful for testing. All you have to do is to provide the unix socket path, ex: qemu+unix:///session?socket=/run/user/1000/libvirt/libvirt-sock Although it would be nice to make this easier to lookup eventually
It might be possible to migrate to file (such as virsh save), then update permissions on the save file and other resources, then reload from that file; but it is not a live migration.
Yes, that's what Boxes is doing afaik. I should have stated clearly I was talking about live migration cheers -- Marc-André Lureau
participants (4)
-
Cole Robinson -
Daniel P. Berrange -
Eric Blake -
Marc-André Lureau