7:56 p.m.
On Wed, Jul 18, 2007 at 08:22:37PM +0100, Richard W.M. Jones wrote:
This is version 4 of the virDomainMigrate patch. It includes remote
support, but no qemu, virsh or python yet. And it needs lots more
testing which I intend to do more of tomorrow.
Interface
---------
The interface is now this:
virDomainPtr
virDomainMigrate (virDomainPtr domain, virConnectPtr dconn,
unsigned long flags, const char *dname,
const char *uri, unsigned long resource);
The caller may set dname, uri and resource to 0/NULL and forget about
them. Or else the caller may set, in particular, uri to allow for more
complicated migration strategies (especially for qemu).
https://www.redhat.com/archives/libvir-list/2007-July/msg00249.html
Driver support
--------------
As outlined in the diagram in this email:
https://www.redhat.com/archives/libvir-list/2007-July/msg00264.html
migration happens in two stages.
Firstly we send a "prepare" message to the destination host. The
destination host may reply with a cookie. It may also suggest a URI (in
the current Xen implementation it just returns gethostname). Secondly
we send a "perform" message to the source host.
Correspondingly, there are two new driver API functions:
typedef int
(*virDrvDomainMigratePrepare)
(virConnectPtr dconn,
char **cookie,
int *cookielen,
const char *uri_in,
char **uri_out,
unsigned long flags,
const char *dname,
unsigned long resource);
typedef int
(*virDrvDomainMigratePerform)
(virDomainPtr domain,
const char *cookie,
int cookielen,
const char *uri,
unsigned long flags,
const char *dname,
unsigned long resource);
To make this work in the remote case I have had to export two private
functions from the API which only remote should call:
__virDomainMigratePrepare
__virDomainMigratePerform
The reason for this is that libvirtd is just linked to the regular
libvirt.so, so can only make calls into libvirt through exported symbols
in the dynamic symbol table.
There are two corresponding wire messages
(REMOTE_PROC_DOMAIN_MIGRATE_PREPARE and
REMOTE_PROC_DOMAIN_MIGRATE_PERFORM) but they just do dumb argument
shuffling, albeit rather complicated because of the number of arguments
passed in and out.
The complete list of messages which go across the wire during a
migration is:
client -- prepare --> destination host
client <-- prepare reply -- destination host
client -- perform --> source host
client <-- perform reply -- source host
client -- lookupbyname --> destination host
client <-- lookupbyname reply -- destination host
I wonder where we should allow for multiple back-forth between prepare and
perform. The 'perform' method could return a tri-state - success, failure
or continue. In the latter case it would call prepare on the destination again
before calling perform on the source a second time. eg you might end up with
an exchange like
1. --> client calls prepare at destination
2. <-- destination returns some metadata
3. --> client calls perform at source with metdata
4. <-- source returns 'continue'
5. --> client calls prepare are destination again
6. <-- destination returns more metdata
7. --> client calls perform at source with more metdata
8. <-- source returns 'success'
In fact thinking about DV's suggestion that we should have a 'finalize'
method at the destination. If we generalized just one step more we could
handle this. Have just 2 methods
virDrvDomainMigrateSource
virDrvDomainMigrateDestination
Both methods return a tri-state, 'complete', 'failed', 'continue'.
The impl
of virDomainMigrate starts with virDrvDomainMigrateDestination, then calls
virDrvDomainMigrateSource, then calls virDrvDomainMigrateDestination again,
and then virDrvDomainMigrateSource again, etc, etc. The sequence stops when
both virDrvDomainMigrateSource & virDrvDomainMigrateDestination have returned
'complete', or one of them has returned 'failure'.
In the case of one returning 'failure' it is possible we might still need
to talk to the other end to initiate a recovery process.
Finally the 'cookie' param is currently an 'out' parameter for prepare,
and
an 'in' parameter for perform. If we allowed for the arbitrary back-and-forth,
then I think the cookie should probably be an 'in' and 'out' parameter
for
both calls - to allow them to pass metadata in both directions.
Maybe this is all overkill, and hopefully, no impl would ever need more than a
single call to either method, but I'm worried that we're designing this wire
level protocol with single exchange+cookie, based on a rough idea for a potential
future impl in Xen which has not actually been implemented yet.
I think it could be a useful future-proofing to allow for the arbitrary back-and-
forth at the wire level, could make use of it without needing to break client
<-> daemon compatability.
Alternatively, we could hold-off on adding a migrate API to libvirt, and get
involved in upstream Xen to sort out secure migration for XenD. Then come back
and finalize the libvirt design based on more knowledge of what we're going to
be talking to.
I have extended capabilities with <migration_features>. For
Xen this is:
<capabilities>
<host>
<migration_features>
<live/>
<uri_transports>
<uri_transport>tcp</uri_transport>
</uri_transports>
</migration_features>
Makes a lot of sense
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|