[Libvir] KVM migration
It turns out that the general migration strategy I defined[1] in reference to Xen, ie. the 3 steps of Prepare/Perform/Finish, isn't sufficient to support KVM migration. The problem is that the destination needs to start qemu-kvm with pretty much the exact same configuration / command line parameters as were used at the source[2]. eg. If the source qemu-kvm has 512 MB of RAM, then the destination had better have exactly 512 MB of RAM also. (This is not the same as Xen where the migration protocol itself carries this information between the two xend daemons). We can modify the Prepare step to pass this information -- I'm suggesting that we just pass the domain XML of the source domain. Thus the Prepare step would change from: static int qemudDomainMigratePrepare (virConnectPtr dconn, char **cookie, int *cookielen, const char *uri_in, char **uri_out, unsigned long flags, const char *dname, unsigned long resource) to: static int qemudDomainMigratePreparev2 (virConnectPtr dconn, char **cookie, int *cookielen, const char *uri_in, char **uri_out, + const char *dom_xml, unsigned long flags, const char *dname, unsigned long resource) As hinted there, we also need a new version of the migration protocol, and a new remote call (Preparev2). So questions: (a) Does anyone have any objections? (b) Does anyone see a simpler way to do this? (c) Since we're adding a second version of the protocol, does anyone want anything else added? (d) Will the domain XML alone be sufficient to recreate the exact qemu command line? (Seems to be the case, and in fact KVM suspend/ restore support seems to implicitly rely on this too). Rich. Notes: [1] https://www.redhat.com/archives/libvir-list/2007-July/msg00357.html and https://www.redhat.com/archives/libvir-list/2007-July/msg00304.html [2] http://kvm.qumranet.com/kvmwiki/Migration#head-71900dbcf45d8ac81649fe4f031c5... -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top
On Mon, Apr 07, 2008 at 04:40:41PM +0100, Richard W.M. Jones wrote:
It turns out that the general migration strategy I defined[1] in reference to Xen, ie. the 3 steps of Prepare/Perform/Finish, isn't sufficient to support KVM migration.
The problem is that the destination needs to start qemu-kvm with pretty much the exact same configuration / command line parameters as were used at the source[2]. eg. If the source qemu-kvm has 512 MB of RAM, then the destination had better have exactly 512 MB of RAM also. (This is not the same as Xen where the migration protocol itself carries this information between the two xend daemons).
We can modify the Prepare step to pass this information -- I'm suggesting that we just pass the domain XML of the source domain. Thus the Prepare step would change from:
static int qemudDomainMigratePrepare (virConnectPtr dconn, char **cookie, int *cookielen, const char *uri_in, char **uri_out, unsigned long flags, const char *dname, unsigned long resource)
to:
static int qemudDomainMigratePreparev2 (virConnectPtr dconn, char **cookie, int *cookielen, const char *uri_in, char **uri_out, + const char *dom_xml, unsigned long flags, const char *dname, unsigned long resource)
As hinted there, we also need a new version of the migration protocol, and a new remote call (Preparev2).
So questions:
(a) Does anyone have any objections?
Not really but i wonder if we can't hack some backward compatible implementation. Basically if you pass the XML you don't need to pass the dname domain name. And on an older implementation if receiving XML on a domain name if properly checked you should error out early on the receiving end allowing to retry the Prepare step with just the name.
(b) Does anyone see a simpler way to do this?
Actually passing the XML is the simplest since you send all the informations What concerns me is to try to keep old and new versions to still communicate okay. maybe the Preparev2/Prepare allows already the do this when falling back if the receiving end has no Preparev2.
(c) Since we're adding a second version of the protocol, does anyone want anything else added?
Never need a v3 ;-)
(d) Will the domain XML alone be sufficient to recreate the exact qemu command line? (Seems to be the case, and in fact KVM suspend/ restore support seems to implicitly rely on this too).
Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Mon, Apr 07, 2008 at 11:58:24AM -0400, Daniel Veillard wrote:
static int qemudDomainMigratePreparev2 (virConnectPtr dconn, char **cookie, int *cookielen, const char *uri_in, char **uri_out, + const char *dom_xml, unsigned long flags, const char *dname, unsigned long resource)
As hinted there, we also need a new version of the migration protocol, and a new remote call (Preparev2).
So questions:
(a) Does anyone have any objections?
Not really but i wonder if we can't hack some backward compatible implementation. Basically if you pass the XML you don't need to pass the dname domain name. And on an older implementation if receiving XML on a domain name if properly checked you should error out early on the receiving end allowing to retry the Prepare step with just the name.
The dname is used when you want to rename the domain during the migration (if indeed this is supported - it isn't in Xen, it will be in KVM). So in most cases dname is actually NULL. However ... the current Xen driver turns out to ignore the dname during Prepare, but later on errors out if dname != NULL in the Perform step. Could that behaviour be used to make a backwards-compatible version I wonder ...? I think _not_ actually, but I'm not 100% sure.
(b) Does anyone see a simpler way to do this?
Actually passing the XML is the simplest since you send all the informations What concerns me is to try to keep old and new versions to still communicate okay. maybe the Preparev2/Prepare allows already the do this when falling back if the receiving end has no Preparev2.
Yes, we will use the SupportsFeature driver call to detect if migration V2 is supported. Xen will be changed to support both (very simple change) and KVM will support only V2. In either case, libvirt will have to continue to support both styles in case it needs to talk to old libvirtd's.
(c) Since we're adding a second version of the protocol, does anyone want anything else added?
Never need a v3 ;-)
(d) Will the domain XML alone be sufficient to recreate the exact qemu command line? (Seems to be the case, and in fact KVM suspend/ restore support seems to implicitly rely on this too).
Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
On Mon, Apr 07, 2008 at 04:40:41PM +0100, Richard W.M. Jones wrote:
It turns out that the general migration strategy I defined[1] in reference to Xen, ie. the 3 steps of Prepare/Perform/Finish, isn't sufficient to support KVM migration. [...]
What ever happened to this? Is it being worked on somewhere by someone? -- Soren Hansen | Virtualisation specialist | Ubuntu Server Team Canonical Ltd. | http://www.ubuntu.com/
On Fri, May 16, 2008 at 12:12:00AM +0200, Soren Hansen wrote:
On Mon, Apr 07, 2008 at 04:40:41PM +0100, Richard W.M. Jones wrote:
It turns out that the general migration strategy I defined[1] in reference to Xen, ie. the 3 steps of Prepare/Perform/Finish, isn't sufficient to support KVM migration. [...]
What ever happened to this? Is it being worked on somewhere by someone?
I've been concentrating on our toolset for the Red Hat Summit next month is what :-) Here's an updated patch. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
On Mon, May 19, 2008 at 11:50:25AM +0100, Richard W.M. Jones wrote:
On Fri, May 16, 2008 at 12:12:00AM +0200, Soren Hansen wrote:
On Mon, Apr 07, 2008 at 04:40:41PM +0100, Richard W.M. Jones wrote:
It turns out that the general migration strategy I defined[1] in reference to Xen, ie. the 3 steps of Prepare/Perform/Finish, isn't sufficient to support KVM migration. [...]
What ever happened to this? Is it being worked on somewhere by someone?
I've been concentrating on our toolset for the Red Hat Summit next month is what :-) Here's an updated patch.
Hum, what is blocking pushing this update ? You're afraid of changing the network interfaces now (IMHO the earlier the better !) ? Lack of review ? Lack of testing ? Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Tue, May 20, 2008 at 05:25:35AM -0400, Daniel Veillard wrote:
On Mon, May 19, 2008 at 11:50:25AM +0100, Richard W.M. Jones wrote:
On Fri, May 16, 2008 at 12:12:00AM +0200, Soren Hansen wrote:
On Mon, Apr 07, 2008 at 04:40:41PM +0100, Richard W.M. Jones wrote:
It turns out that the general migration strategy I defined[1] in reference to Xen, ie. the 3 steps of Prepare/Perform/Finish, isn't sufficient to support KVM migration. [...]
What ever happened to this? Is it being worked on somewhere by someone?
I've been concentrating on our toolset for the Red Hat Summit next month is what :-) Here's an updated patch.
Hum, what is blocking pushing this update ? You're afraid of changing the network interfaces now (IMHO the earlier the better !) ? Lack of review ? Lack of testing ?
Lack of testing. Apparently oVirt people are going to play with it at some point. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v
participants (3)
-
Daniel Veillard -
Richard W.M. Jones -
Soren Hansen