wrote:

Hi, I would like to save a running domain (ie. disk + memory) and be able to restore it multiple times creating duplicates of the orignal domain all starting from the same state. Use case: I'm building a task-processing system for use in a CI flow. I want to run multiple VMs in parallel using the same image (always starting from the same state). And to avoid needlessly booting between each task, I would like to save (and distribute) the domain state, so that I just restore from memory. However, I can't seems to change the UUID or the name of a domain once it is saved. Nor do I seem able to rename a domain while it is running. I can obviously duplicate both the disks and the file to which I saved the domain state using "virsh save". But I seem unable to rename before I restore.. Any ideas? Could I do this with snapshots? I suspect not since I see virDomainSnapshotRedefinePrep() calling virDomainDefCheckABIStability which raises the error here: https://fossies.org/dox/libvirt-1.3.3/domain__conf_8c_source.html#l17991 Out of curiosity does anyone know what horrors might befall me if I were to remove the lines protecting against name and UUID changes? Then compile my own libvirt... The comment in the code says name can be changed, but I'm guessing I would have to change the UUID too. Does anyone see how that would create issues? I'm not sure how libvirt uses the UUID internally. -- Regards Jonas Finnemann Jensen. _______________________________________________ libvirt-users mailing list libvirt-users(a)redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users

On Mon, Apr 18, 2016 at 10:05:02PM +0300, Michael Ravits wrote: > Hi Jonas, > > I asked a similar question here last week. You could find it by searching > for this topic in the mailing list archives: "Clone VM with saved state". > That being said I'd be very glad if you could update here when you find a > complete solution. > > - Michael > > What I *think* Jones wants is to use virt-builder, it could cause some problems if you started all of them. What Michael wants is: virsh save running_domain saved-image-file.img virsh save-image-edit saved-image-file.img Hope that helped, Martin On Mon, Apr 18, 2016 at 9:49 PM, Jonas Finnemann Jensen < > jonasfj(a)mozilla.com > >> wrote: >> > > Hi, >> >> I would like to save a running domain (ie. disk + memory) and be able to >> restore it multiple times creating duplicates of the orignal domain all >> starting from the same state. >> Use case: >> I'm building a task-processing system for use in a CI flow. >> I want to run multiple VMs in parallel using the same image (always >> starting from the same state). >> And to avoid needlessly booting between each task, I would like to save >> (and distribute) the domain state, so that I just restore from memory. >> >> However, I can't seems to change the UUID or the name of a domain once it >> is saved. >> Nor do I seem able to rename a domain while it is running. >> >> I can obviously duplicate both the disks and the file to which I saved >> the >> domain state using "virsh save". >> But I seem unable to rename before I restore.. Any ideas? >> >> Could I do this with snapshots? I suspect not since I see >> virDomainSnapshotRedefinePrep() calling >> virDomainDefCheckABIStability which raises the error here: >> https://fossies.org/dox/libvirt-1.3.3/domain__conf_8c_source.html#l17991 >> >> Out of curiosity does anyone know what horrors might befall me if I were >> to remove the lines protecting against name and UUID changes? Then >> compile >> my own libvirt... >> The comment in the code says name can be changed, but I'm guessing I >> would >> have to change the UUID too. Does anyone see how that would create >> issues? >> I'm not sure how libvirt uses the UUID internally. >> >> -- >> Regards Jonas Finnemann Jensen. >> >> _______________________________________________ >> libvirt-users mailing list >> libvirt-users(a)redhat.com >> https://www.redhat.com/mailman/listinfo/libvirt-users >> >> _______________________________________________ > libvirt-users mailing list > libvirt-users(a)redhat.com > https://www.redhat.com/mailman/listinfo/libvirt-users >

Well, as long as they're completely isolated from each other, you may have a better chance of success. However there will still be the issue of the IP address of the network interface. You can't have two networks using the same IP range (since libvirt doesn't use network namespaces for its networks), so the guest will need to change its IP, which means it will need to be notified of this need, possibly by having the host toggle the interface offline and back on - you can use virsh domif-setlink to do this.

> > You'll also need to change the name and uuid of the domain at the very > least. Agree, but is that possible with libvirt?

If you do that (restore a previously running image with a different MAC > address) Yeah, probably I wouldn't change the MAC address. As I want to attach the VMs to different networks. I rely on the IP being in a different subnet to identify the VM in my metadata service. Using IP filters to enforce the subnet seems like the most robust way of being sure which VM I'm talking to. possibly by having the host toggle the interface offline and back on Yeah, I think unplugging the virtual network cable before I save the VM memory, and plugging it back in after I load the VM, then DHCP would run immediately. As an added benefit any guest program I have talking to my meta-data service would be able to detect that the VM has been loaded, by looking for network connection. -- Regards Jonas Finnemann Jensen. 2016-04-19 14:26 GMT-07:00 Laine Stump <laine(a)laine.org&gt;: > (please don't top-post. Put your responses inline, in context) > > On 04/19/2016 01:09 PM, Jonas Finnemann Jensen wrote: > >> virt-builder looks like some fancy guest/host interaction related to >> building VM images. >> >> What I'm looking for is more like: >> virsh save running_domain saved-domain-A.img >> cp saved-domain-A.img saved-domain-B.img >> virsh save-image-edit saved-domain-B.img // Change the network, >> possibly MAC, VNC port >> > > You'll also need to change the name and uuid of the domain at the very > least. And I assume these will all be transient domains, not persistent. > > >> Then in parallel I want to do: >> virsh restore saved-domain-A.img >> virsh restore saved-domain-B.img >> > > If you do that (restore a previously running image with a different MAC > address), at the very least the guest OS will be confused about the MAC > address of the network card, and you'll very likely end up with both guests > responding to ARP requests for the original MAC address. There's likely > other problems that I haven't thought of that will happen as well. > > >> So that I have two instances of the same virtual machine starting from >> the same state. >> This way I can reset the VMs without having to reboot them (booting is >> rather slow). >> >> I practice I'll probably have ~16 instances at the same time. Constantly >> being reset to the same state. >> I tried with QEMU, and it's seems totally doable with savevm, copy file, >> then doing loadvm twice in parallel. >> (I'll be using a separate network for each VM, so I can be sure which one >> I'm talking to). >> > > Well, as long as they're completely isolated from each other, you may have > a better chance of success. However there will still be the issue of the IP > address of the network interface. You can't have two networks using the > same IP range (since libvirt doesn't use network namespaces for its > networks), so the guest will need to change its IP, which means it will > need to be notified of this need, possibly by having the host toggle the > interface offline and back on - you can use virsh domif-setlink to do this. > > > >> Is this doable with libvirt, or am I better off using QEMU directly? and >> how? I couldn't do internal snapshots with --live, and snapshot-revert says >> it can't revert to external snapshots yet :) >> (using QEMU directly would certainly leave me with a lot of manual >> network configuration) >> > > Someone else will have to talk about the particulars of snapshots... > >

Fair enough :) I just wanted to make sure it wasn't supported... I'm probably better off using QEMU directly. I totally understand that libvirt makes some sane decisions that makes sense for data center management. I'm not sure why it couldn't be done. But honestly hacking libvirt to violate a core invariant is probably asking for trouble :)

Den 19. apr. 2016 11.17 PM skrev "Martin Kletzander" <mkletzan(a)redhat.com&gt;: > On Tue, Apr 19, 2016 at 03:22:18PM -0700, Jonas Finnemann Jensen wrote: > >> >>> You'll also need to change the name and uuid of the domain at the very >>> least. >>> >> >> Agree, but is that possible with libvirt? >> >> > Not in a supported way. But you can, technically, edit the save file > (not using virsh), change the name and uuid and restore it. But don't > seek help when something doesn't work for you after that =) > > If you do that (restore a previously running image with a different MAC >> >>> address) >>> >> >> Yeah, probably I wouldn't change the MAC address. As I want to attach the >> VMs to different networks. >> I rely on the IP being in a different subnet to identify the VM in my >> metadata service. >> Using IP filters to enforce the subnet seems like the most robust way of >> being sure which VM I'm talking to. >> >> possibly by having the host toggle the interface offline and back on >> >> Yeah, I think unplugging the virtual network cable before I save the VM >> memory, and plugging it back in after I load the VM, then DHCP would run >> immediately. >> As an added benefit any guest program I have talking to my meta-data >> service would be able to detect that the VM has been loaded, by looking >> for >> network connection. >> >> >> -- >> Regards Jonas Finnemann Jensen. >> >> 2016-04-19 14:26 GMT-07:00 Laine Stump <laine(a)laine.org&gt;: >> >> (please don't top-post. Put your responses inline, in context) >>> >>> On 04/19/2016 01:09 PM, Jonas Finnemann Jensen wrote: >>> >>> virt-builder looks like some fancy guest/host interaction related to >>>> building VM images. >>>> >>>> What I'm looking for is more like: >>>> virsh save running_domain saved-domain-A.img >>>> cp saved-domain-A.img saved-domain-B.img >>>> virsh save-image-edit saved-domain-B.img // Change the network, >>>> possibly MAC, VNC port >>>> >>>> >>> You'll also need to change the name and uuid of the domain at the very >>> least. And I assume these will all be transient domains, not persistent. >>> >>> >>> Then in parallel I want to do: >>>> virsh restore saved-domain-A.img >>>> virsh restore saved-domain-B.img >>>> >>>> >>> If you do that (restore a previously running image with a different MAC >>> address), at the very least the guest OS will be confused about the MAC >>> address of the network card, and you'll very likely end up with both >>> guests >>> responding to ARP requests for the original MAC address. There's likely >>> other problems that I haven't thought of that will happen as well. >>> >>> >>> So that I have two instances of the same virtual machine starting from >>>> the same state. >>>> This way I can reset the VMs without having to reboot them (booting is >>>> rather slow). >>>> >>>> I practice I'll probably have ~16 instances at the same time. Constantly >>>> being reset to the same state. >>>> I tried with QEMU, and it's seems totally doable with savevm, copy file, >>>> then doing loadvm twice in parallel. >>>> (I'll be using a separate network for each VM, so I can be sure which >>>> one >>>> I'm talking to). >>>> >>>> >>> Well, as long as they're completely isolated from each other, you may >>> have >>> a better chance of success. However there will still be the issue of the >>> IP >>> address of the network interface. You can't have two networks using the >>> same IP range (since libvirt doesn't use network namespaces for its >>> networks), so the guest will need to change its IP, which means it will >>> need to be notified of this need, possibly by having the host toggle the >>> interface offline and back on - you can use virsh domif-setlink to do >>> this. >>> >>> >>> >>> Is this doable with libvirt, or am I better off using QEMU directly? and >>>> how? I couldn't do internal snapshots with --live, and snapshot-revert >>>> says >>>> it can't revert to external snapshots yet :) >>>> (using QEMU directly would certainly leave me with a lot of manual >>>> network configuration) >>>> >>>> >>> Someone else will have to talk about the particulars of snapshots... >>> >>> >>>