Re: [libvirt] [PATCH v1 00/11] Rework storage migration
On Tue, Nov 27, 2012 at 07:49:54PM +0100, Michal Privoznik wrote:
This patch set re-implements migration with storage for enough new
qemu.
Currently, you can migrate a domain to a host without need for shared storage.
This is done by setting 'blk' or 'inc' attribute (representing
VIR_MIGRATE_NON_SHARED_DISK and VIR_MIGRATE_NON_SHARED_INC flags respectively)
of 'migrate' monitor command. However, the qemu implementation is
buggy and applications are advised to switch to new impementation
which, moreover, offers some nice features, like migrating only explicitly
specified disks.
The new functionality is controlled via 'nbd-server-*' and
'drive-mirror'
commands. The flow is meant to look like this:
1) User invokes libvirt's migrate functionality.
2) libvirt checks that no block jobs are active on the source.
3) libvirt starts the destination QEMU and sets up the NBD server using the
nbd-server-start and nbd-server-add commands.
4) libvirt starts drive-mirror with a destination pointing to the remote NBD
server, for example nbd:host:port:exportname=diskname (where diskname is the
-drive id specified on the destination).
This is where I have a problem with automatically using the new impl. If
we consider that we need to allow multiple concurrent migrations, We are
introducing a requirement to open arbitrary number of ports in the firewall.
This data stream is also cleartext without any encryption, even using an
encrypted qcow2 disk won't help, since we're transferring the logical guest
side blocks, not the physical host side blocks.
What you have here is fine in the direct migration case, since we already
require arbitrary open ports and non-encrypted data stream.
If the user has requested TUNNELLED migration, we need follow up work done.
We need the NBD server to be able to accept a pre-opened file descriptor
to rather than listening on a TCP host/port, and then for libvirtd to
tunnel the data stream somehow. Alternatively just use the old impl in
the TUNNELLED case.
5) once all mirroring jobs reach steady state, libvirt invokes the
migrate
command.
6) once migration completed, libvirt invokes the nbd-server-stop command on the
destination QEMU.
If we just skip the 2nd step and there is an active block-job, qemu will fail in
step 4. No big deal.
Since we try to NOT break migration and keep things compatible, this feature is
enabled iff both sides support it. Since there's obvious need for some data
transfer between src and dst, I've put it into qemuCookieMigration:
1) src -> dest: (QEMU_MIGRATION_PHASE_BEGIN3 -> QEMU_MIGRATION_PHASE_PREPARE)
<nbd>
<disk src='/var/lib/libvirt/images/f17.img'
size='17179869184'/>
</nbd>
Hey destination, I know how to use this cool new feature. Moreover,
these are the paths I'll send you. Each file is X bytes big.
It's one of the prerequisite - the file on dst exists and has at least the
same size as on dst.
2) dst -> src: (QEMU_MIGRATION_PHASE_PREPARE -> QEMU_MIGRATION_PHASE_PERFORM3)
<nbd port='X'/>
Okay, I (destination) support this feature as well. I've created all
files as you (src) told me to and you can start rolling data. I am listening
on port X.
3) src -> dst: (QEMU_MIGRATION_PHASE_PERFORM3 -> QEMU_MIGRATION_PHASE_FINISH3)
<nbd port='-1'/>
Migration completed, destination, you may shut the NBD server down.
If either src or dst doesn't support NBD, it is not used and whole process fall
backs to old implementation.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|