[Libvir] [PATCH][RFC] Adding Cloning Feature
Hi all, I've been planing to adding cloning feature. Then, I tried to make the patch in the following way. 1)Get the XML used virDomainGetXMLDesc 2)Delete <uuid>, <mac> node element This intened to automatically generated. 3)Changing <name>, <source dev> string node element 4)Fork "dd" process 5)Define XML used virDomainDefineXML This feature is just a first revision, because I make this patch within current libvirt API limitation. Also Remote Cloining is Limited. But if we can libvirtd for feature, it will be enhance remote cloning based on this pach (as http://libvirt.org/remote.html#Remote_libvirtd_configuration) Finally, to have to communicate with the demon remotely as libvirt library, actually I want to put down to new libvirt API (e.g. virDomainClone). Could you hear your comments about this ? Attached patch shows below. --------------------------------------------- # ./virsh --help clone NAME clone - clone a domain SYNOPSIS virsh clone <domain> <devices,...> DESCRIPTION Clone a domain. OPTIONS <domain> shut off state domain name <devices> destination devices (comma separated) # ./virsh clone PV_RH5 /dev/sdb2 Cloning from /dev/sda5 to /dev/sdb2 ..... # ./virsh list --all Id Name State ---------------------------------- 0 Domain-0 running - PV_RH5 shut off - PV_RH5_CLONE shut off #./virsh start PV_RH5_CLONE Domain PV_RH5_CLONE started --------------------------------------------- Thanks, Kazuki Mizushima
Kazuki Mizushima wrote:
I've been planing to adding cloning feature.
I think cloning is a great feature. From when I used to sysadmin a Xen server, I used cloning as the main technique to create new servers - ie. take an existing machine which is similar to the new one which I want, and just clone it.
Then, I tried to make the patch in the following way. 1)Get the XML used virDomainGetXMLDesc 2)Delete <uuid>, <mac> node element This intened to automatically generated. 3)Changing <name>, <source dev> string node element 4)Fork "dd" process
You may want to pause the source node first? Otherwise won't you get disk corruption? Also, what about other devices? You "dd" the main disk, but what if the machine has swap devices or other disks?
5)Define XML used virDomainDefineXML
This feature is just a first revision, because I make this patch within current libvirt API limitation. Also Remote Cloining is Limited. But if we can libvirtd for feature, it will be enhance remote cloning based on this pach (as http://libvirt.org/remote.html#Remote_libvirtd_configuration)
There is no problem with interaction with the remote patch. Nor do you have to worry about the remote stuff when implementing this (the "dd" will fork on the remote libvirtd automagically). ... Unless you want to clone across servers :-) ... a.k.a. migration, and of course we haven't really worked out how we should do that (but if you have ideas, please post them).
# ./virsh list --all Id Name State ---------------------------------- 0 Domain-0 running - PV_RH5 shut off - PV_RH5_CLONE shut off
So the idea is that you can only clone an inactive domain? Rich. -- Emerging Technologies, Red Hat http://et.redhat.com/~rjones/ 64 Baker Street, London, W1U 7DF Mobile: +44 7866 314 421 Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 Directors: Michael Cunningham (USA), Charlie Peters (USA) and David Owens (Ireland)
On Tue, Apr 24, 2007 at 10:57:09PM +0900, Kazuki Mizushima wrote:
Hi all,
Hi Kazuki,
I've been planing to adding cloning feature.
Hum, interesting usage, of course it sounds difficult to really fully automate it in a completely generic way.
Then, I tried to make the patch in the following way.
1)Get the XML used virDomainGetXMLDesc 2)Delete <uuid>, <mac> node element This intened to automatically generated. 3)Changing <name>, <source dev> string node element 4)Fork "dd" process 5)Define XML used virDomainDefineXML
This feature is just a first revision,
Okay, so let's look first at how it would integrate in virsh ... So the command take the domain reference and a comma separated list of device. I assume it expect those device to be already available (for example if using LVM they were already created) and already initialized. I'm surprized the name for the new clone is not an argument. I wonder how we could simplify the device creation, and also how we could preinitialize the disks. It sounds to me a bit more on the area of Cobbler than libvirt/virsh though.
because I make this patch within current libvirt API limitation. Also Remote Cloining is Limited. But if we can libvirtd for feature, it will be enhance remote cloning based on this pach (as http://libvirt.org/remote.html#Remote_libvirtd_configuration)
Finally, to have to communicate with the demon remotely as libvirt library, actually I want to put down to new libvirt API (e.g. virDomainClone).
Could you hear your comments about this ?
The problem of adding a Clone operation at the libvirt level is that it would be very hard to make it solid, I mean just modifying an XML file is not hard, it's the checking of the state, the provisionning and making sure that no disaster will happen if we try to start the domain which are the hard part. I'm not sure we can do that in a relatively generic fashion, it probably depends a lot on the scenario. To get down to something concrete, suppose I have a domain with an Apache server running a RHEL4 paravirt. The new API may sound like it could create a new domain with a similar Apache server, but: - how do we provision it ? - how do we avoid name clashes, i.e. IP, machine name, ... in practice it would mean changing settings in the server - are we using resource which may not be shared (or shareable) We would need to be very precise about what the API call does, and warn about what it does not do, before I would start feeling comfortable about adding it in libvirt, but it's definitely something which could have some value, at least in some scenario ! Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
HI Daniel, Thank you for reply and comments.
Okay, so let's look first at how it would integrate in virsh ... So the command take the domain reference and a comma separated list of device. I assume it expect those device to be already available (for example if using LVM they were already created) and already initialized.
Yes. The user is preparing those destination devices beforehand.
I'm surprized the name for the new clone is not an argument.
There is the clone which I think about for the purpose of reprinting an as possible state as is. So this first revision is primmitive. Also It is thought that information that must not overlap as a virtual machine should change automatically. But I am thinking this enhaunce it below. 1)--name new clone vm name 2)--mac new clone mac address 3)--format destination devies initialized befor cloning 4)--wait wait cloning porcess I want to dettach the cloning process and to show the state with the "cloning" by virsh.
The problem of adding a Clone operation at the libvirt level is that it would be very hard to make it solid, I mean just modifying an XML file is not hard, it's the checking of the state, the provisionning and making sure that no disaster will happen if we try to start the domain which are the hard part. I'm not sure we can do that in a relatively generic fashion, it probably depends a lot on the scenario. To get down to something concrete, suppose I have a domain with an Apache server running a RHEL4 paravirt. The new API may sound like it could create a new domain with a similar Apache server, but: - how do we provision it ? - how do we avoid name clashes, i.e. IP, machine name, ... in practice it would mean changing settings in the server - are we using resource which may not be shared (or shareable)
It is surely so. Cloning I defined aims at creating a just duplication as it except informations that must not overlap as a virtual machine. So, by my just now cloning, it is necessary for a user to edit peculiar information of the OS and Middlewares. I thought about service such as load dispersion than by 1 installed the OS and Middlewares in each node when I set each node a clone did the machine at the stage that came by one setup, and to edit peculiarity information.
We would need to be very precise about what the API call does, and warn about what it does not do, before I would start feeling comfortable about adding it in libvirt,
I see. So I put down to virsh used libvirt library instead of libvirt. # but I also unterstand that put ing down to virsh cannot remote support ..... Thanks, Kazuki Mizushima
On Wed, Apr 25, 2007 at 04:32:06PM +0900, Kazuki Mizushima wrote:
HI Daniel,
Hi Kazuki,
I'm surprized the name for the new clone is not an argument.
There is the clone which I think about for the purpose of reprinting an as possible state as is. So this first revision is primmitive. Also It is thought that information that must not overlap as a virtual machine should change automatically. But I am thinking this enhaunce it below. 1)--name new clone vm name 2)--mac new clone mac address 3)--format destination devies initialized befor cloning 4)--wait wait cloning porcess I want to dettach the cloning process and to show the state with the "cloning" by virsh.
Look this is all arguments already presents in some form at the virt-manager level. There is just too many similarities to ignore it.
We would need to be very precise about what the API call does, and warn about what it does not do, before I would start feeling comfortable about adding it in libvirt,
I see. So I put down to virsh used libvirt library instead of libvirt. # but I also unterstand that put ing down to virsh cannot remote support
Remote support will need plugging at the API level. I think prototyping within virt-manager will get you to more feedback and features faster. Then in turn this will make the analysis required to check feasibility and do the API design faster too. IMHO you will get where you want faster if you do the first tool step in virt-manager, really. Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Hi Daniel, all Thank you for various comments.
Remote support will need plugging at the API level. I think prototyping within virt-manager will get you to more feedback and features faster. Then in turn this will make the analysis required to check feasibility and do the API design faster too. IMHO you will get where you want faster if you do the first tool step in virt-manager, really.
Yes. I agree with you and I understand. I try to put down this feature(virt-clone or virt-install) to virt-manager framework used libvirt library. So I will propose for virt-manager ML. Thanks, Kazuki Mizushima
Kazuki Mizushima wrote:
Hi all,
I've been planing to adding cloning feature. Then, I tried to make the patch in the following way. [...]
Right - didn't see the patch the first time around ... I think the clone should be done as a libvirt call, rather than in virsh. Adding stuff to virsh doesn't really help other libvirt users, nor would it work in the remote case. If it goes into libvirt on the other hand, then you get remote "for free" (in this case) and other users of libvirt can implement it, in particular virt-manager. Rich. -- Emerging Technologies, Red Hat http://et.redhat.com/~rjones/ 64 Baker Street, London, W1U 7DF Mobile: +44 7866 314 421 Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 Directors: Michael Cunningham (USA), Charlie Peters (USA) and David Owens (Ireland)
On Tue, Apr 24, 2007 at 03:33:24PM +0100, Richard W.M. Jones wrote:
Kazuki Mizushima wrote:
Hi all,
I've been planing to adding cloning feature. Then, I tried to make the patch in the following way. [...]
Right - didn't see the patch the first time around ...
I think the clone should be done as a libvirt call, rather than in virsh. Adding stuff to virsh doesn't really help other libvirt users, nor would it work in the remote case. If it goes into libvirt on the other hand, then you get remote "for free" (in this case) and other users of libvirt can implement it, in particular virt-manager.
I disagree :-) I think cloning is a pretty high level operation and is better off outside the core libvirt API, building on top of the various APIs libvirt provides (or should provide). The most obvious set of missing APIs which are needed for clone are the storage management APIs we've discussed a few times in the past. Sooner or later we're going to have to tackle those APIs, since even regular guest provisioning will need them once we have remote support. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Tue, Apr 24, 2007 at 10:57:09PM +0900, Kazuki Mizushima wrote:
Hi all,
I've been planing to adding cloning feature. Then, I tried to make the patch in the following way.
1)Get the XML used virDomainGetXMLDesc 2)Delete <uuid>, <mac> node element This intened to automatically generated. 3)Changing <name>, <source dev> string node element 4)Fork "dd" process 5)Define XML used virDomainDefineXML
While this works ok in simple cases, I'm not sure it neccessarily scales upto handle the more complex cases easily enough. With this prototype the user only has the option of specifying new device names. Conceivably the user could want to specify a whole lot more 'new' parameters - eg change the VNC port if it was not previously auto-allocated, specify a new MAC address (for any/all NICs), specify a UUID, etc. If using LVM for the backing store, instead of dd'ing the disk they could simply take a (writable) snapshot of the disk - this could even be done while the source VM was active (assuming use of a journalling FS inside the guest). If doing a flat copy of the disks, it will also be important to provide progress feedback to the user - when initially allocating guests it takes a good 30-60 seconds per GB of disk to be allocated - cloning an existing guest will be longer because instead of just writing zero's you are reading & writing data.
This feature is just a first revision, because I make this patch within current libvirt API limitation. Also Remote Cloining is Limited. But if we can libvirtd for feature, it will be enhance remote cloning based on this pach (as http://libvirt.org/remote.html#Remote_libvirtd_configuration)
Finally, to have to communicate with the demon remotely as libvirt library, actually I want to put down to new libvirt API (e.g. virDomainClone).
For remote support, we'd really need to have some basic storage management APIs to let us allocate new 'chunks' of storage for the virtual disks.
Could you hear your comments about this ?
I think cloning should be kept separate from the main libvirt API because there are simply too many different ways to approach it, depending on your application use-cases. The way I like to look at it, is that we've just got several different ways to 'install' a guest 1. Traditional installation media (eg CDROM, HTTP kickstart) 2. Boot a LiveCD which can install itself to disk 3. Create from a VM 'template', possibly cloning some base disk 4. Cloning an existing provisioned VM Now virt-install currently deals with the first option, and there is some work going on to experiment with making it do options 2 & 3 too. So I think that it'd be worth trying to add some form of cloning capability to virt-install too. Thinking how it might work.. Currently to install from HTTP you might do virt-install --name myvm --ram 500 --mac 54:24:52:23:55:43 --vnc --vncunused --url http://download.fedora.redhat.com/pub/fedora/linux/core/6/x86_64/os/ --filesize 5 --file /var/lib/xen/images/myvm.img So, how might it work with cloning ? Basically install of the --url arg to point to an install source, we'd allow '--clone VMNAME' to point to an existing virtual machine. The main config would be determined by the settings of that VM, with the command line args being used to override specific bits. As a bare minimum example one would do virt-install --name othervm --clone myvm --file /var/lib/xen/images/othervm.img But for a more complex case one could use all the various args virt-install --name othervm --clone myvm --uuid 2452:ef23:ee3e:2232:442424252525ffe --vncport 5902 --mac 42:55:23:66:23:44 --file /var/lib/xen/images/myvm.img --file /var/lib/xen/images/myvmdata.img Now without prototyping it, I'm not sure how much code re-use we'd get within virt-install, so it may turn out that it is simpler to just have a separate tool called virt-clone for this purpose. In either case I reckon we could distribute this as part of the main virtinst codebase, so it is still easy for us to access the functionality from virt-manager. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Hi Dan, Thank you for comment and useful suggestion.
While this works ok in simple cases, I'm not sure it neccessarily scales upto handle the more complex cases easily enough. With this prototype the user only has the option of specifying new device names. Conceivably the user could want to specify a whole lot more 'new' parameters - eg change the VNC port if it was not previously auto-allocated, specify a new MAC address (for any/all NICs), specify a UUID, etc.
Cloning I defined aims at creating a just duplication as it except informations that must not overlap as a virtual machine. But I wanto to specify the more 'new' parameters(name, mac ..) for next. Thank you for below suggestion. Because I want to add 'new' parameters for next, It is goot way for also me. I agree. I try to propose for this feature for virt-install et-mgmt-tools@redhat.com.
I think cloning should be kept separate from the main libvirt API because there are simply too many different ways to approach it, depending on your application use-cases. The way I like to look at it, is that we've just got several different ways to 'install' a guest
1. Traditional installation media (eg CDROM, HTTP kickstart) 2. Boot a LiveCD which can install itself to disk 3. Create from a VM 'template', possibly cloning some base disk 4. Cloning an existing provisioned VM
Now virt-install currently deals with the first option, and there is some work going on to experiment with making it do options 2 & 3 too. So I think that it'd be worth trying to add some form of cloning capability to virt-install too.
Thinking how it might work..
Currently to install from HTTP you might do
virt-install --name myvm --ram 500 --mac 54:24:52:23:55:43 --vnc --vncunused --url http://download.fedora.redhat.com/pub/fedora/linux/core/6/x86_64/os/ --filesize 5 --file /var/lib/xen/images/myvm.img
So, how might it work with cloning ? Basically install of the --url arg to point to an install source, we'd allow '--clone VMNAME' to point to an existing virtual machine. The main config would be determined by the settings of that VM, with the command line args being used to override specific bits. As a bare minimum example one would do
virt-install --name othervm --clone myvm --file /var/lib/xen/images/othervm.img
But for a more complex case one could use all the various args
virt-install --name othervm --clone myvm --uuid 2452:ef23:ee3e:2232:442424252525ffe --vncport 5902 --mac 42:55:23:66:23:44 --file /var/lib/xen/images/myvm.img --file /var/lib/xen/images/myvmdata.img
Now without prototyping it, I'm not sure how much code re-use we'd get within virt-install, so it may turn out that it is simpler to just have a separate tool called virt-clone for this purpose. In either case I reckon we could distribute this as part of the main virtinst codebase, so it is still easy for us to access the functionality from virt-manager.
Thanks, Kazuki Mizushima
participants (4)
-
Daniel P. Berrange -
Daniel Veillard -
Kazuki Mizushima -
Richard W.M. Jones