On Thu, Jan 15, 2015 at 02:45:13PM +0100, Vojtech Cima wrote:
Hello everyone,
we are very interested about the post-copy live migration mechanism in libvirt [1] using QEMU. Mentioning post-copy in this context means the setup with the first full iteration of standard pre-copy and then switching to the post-copy. VIR_MIGRATE_ENABLE_POSTCOPY, VIR_MIGRATE_POSTCOPY_AFTER_PRECOPY together with VIR_MIGRATE_TUNNELLED flag.
Recently we have noticed the issue that post-copy live migration doesn't work over the tunnel since since current tunnel implementation in libvirt supports only uni-directional communication. I spent some time trying to enable bidirectional communication over the tunnel but still facing several issues and I would appreciate any hints how to move forward.
The idea is to add a function (probably a callback function) responsible for handling the traffic in the opposite direction (dest->src) reading from the virStream and writing to the QEMU [2]. To be able to do so I replaced the current 'pipe' calls with the 'socketpair' calls to create bidirectional paths.
Now, trying to use tunnelled post-copy live migration, it gets initiated, there is some traffic going from source to destination (obviously the pre-copy phase) but at some point it gets stuck on the 'saferead' function [3] which is now blocking. Can this issue issue be linked with the default socket blocking policy? Trying to use standard 'read' function instead of 'saferead' results that the lower data transfers and it seems it's missing significant part of information.
Since the code was historically unidirectional, the qemuMigrationIOFunc() method used the virStream object in blocking I/O mode, and used a simple loop to read from the pipe / write to the stream. If you want to do bi-directional transfer that won't be possible - you'll need to put the stream into fully non-blocking mode and use event callbacks to deal with all I/O. This also means you can't use saferead either. The code in tools/virsh-console.c is a reasonably good example of how to use a stream in non-blocking mode doing bi-directional I/O between the stream and regular file descriptors. The virsh-console uses stdin/stdout but you'd just use your socketpair instead.
Another point is that we suppose that the virStream on its own supports bidirectional communication, is that correct?
Yes, absolutely.
[1] https://github.com/orbitfp7/qemu/tree/wp3-postcopy
[2] http://libvirt.org/git/?p=libvirt.git;a=blob;f=src/qemu/qemu_migration.c;h=0...
[3] http://libvirt.org/git/?p=libvirt.git;a=blob;f=src/qemu/qemu_migration.c;h=0...
Thank you very much for any ideas on this topic.
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|