Re: [libvirt] [RFC] Proposed API to support block device streaming

On Wed, Nov 10, 2010 at 08:45:20AM -0600, Adam Litke wrote: I'm not too worried about the code, I just wanted to understand what logical set of QEMU operations it maps to. What if you have a disk image and want to activate streaming without running a VM ? eg, so you can ensure the image is fully downloaded to the host and thus avoid a runtime problem which would result in IO error for the guest NB, libvirt intentionally doesn't expose the device backend aliases in the API. So this should refer to the device alias which is included in the XML. So a migration lifecycle based on this design has a pretty dangerous failure mode. The guest can loose access to the NBD server before the disk copy is complete, and we'd be unable to switch back to the original QEMU instance since the target has already started dirtying memory which has invalidated the source. These last few points are my biggest concern with the API. If we iteratively add a bunch of APIs for each piece of functionality involved here, then we'll end up with a migration lifecycle that requires the app to know about invoking 10's of different API calls in a perfect sequence. This seems like a very complex and fragile design for apps to have to deal with. Direct QEMU<->QEMU migration is already sub-optimal in that it requires opening many ports in the firewall (assuming you want to allow multiple concurrent VMs to migrate). We can address that limitation by having libvirt take ownership of the port on the destination hosts, and then pass the incoming client socket onto QEMU, or manually forward traffic. Adding in multiple NBD network sockets makes the firewall management problem even worse. If we want to be able to use this functionality without requiring apps to have a direct shell into the host, then we need a set of APIs for managing NBD server instances for migration, which is another level of complexity. A simpler architecture would be to have the NBD server embedded inside the source QEMU VM, and tunnel the NBD protocol over the existing migration socket. So QEMU would do a normal migration of RAM, and when that completes and source QEMU CPUs are stopped, but QEMU is left running to continue serving the disk data. This avoids any extra network connections, and avoids having to add any new APIs to manage NBD servers, and avoids all the security driver & lock manger integration problems that the latter will involve. If it is critical to free up RAM on the source host, then the main VM ram area can be munmap()d on the source once main migration completes, since its not required for the ongoing NBD data stream. This kind of architecture means that apps would need near zero knowledge of disk streaming to make use of it. The existing virDomainMigrate() would be sufficient, with an extra flag to request post-migration streaming. There would still be a probable need for your suggested API to force immediate streaming of a disk, instead of relying on NBD, but most apps wouldn't have to care about that if they didn't want to. In summary though, I'm not inclined to proceed with adding ad-hoc APIs for disk streaming to libvirt, without fully considering the design of a full migration+disk streaming architecture. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|