You'll also need to change the name and uuid of the domain at the very least.

Agree, but is that possible with libvirt?

If you do that (restore a previously running image with a different MAC address)

Yeah, probably I wouldn't change the MAC address. As I want to attach the VMs to different networks.

I rely on the IP being in a different subnet to identify the VM in my metadata service.

Using IP filters to enforce the subnet seems like the most robust way of being sure which VM I'm talking to.

possibly by having the host toggle the interface offline and back on

Yeah, I think unplugging the virtual network cable before I save the VM memory, and plugging it back in after I load the VM, then DHCP would run immediately.

As an added benefit any guest program I have talking to my meta-data service would be able to detect that the VM has been loaded, by looking for network connection.

--
Regards Jonas Finnemann Jensen.

2016-04-19 14:26 GMT-07:00 Laine Stump <laine@laine.org>:

(please don't top-post. Put your responses inline, in context)

On 04/19/2016 01:09 PM, Jonas Finnemann Jensen wrote:

virt-builder looks like some fancy guest/host interaction related to building VM images.

What I'm looking for is more like:
virsh save running_domain saved-domain-A.img
cp saved-domain-A.img saved-domain-B.img
virsh save-image-edit saved-domain-B.img // Change the network, possibly MAC, VNC port

You'll also need to change the name and uuid of the domain at the very least. And I assume these will all be transient domains, not persistent.

Then in parallel I want to do:
virsh restore saved-domain-A.img
virsh restore saved-domain-B.img

If you do that (restore a previously running image with a different MAC address), at the very least the guest OS will be confused about the MAC address of the network card, and you'll very likely end up with both guests responding to ARP requests for the original MAC address. There's likely other problems that I haven't thought of that will happen as well.

So that I have two instances of the same virtual machine starting from the same state.
This way I can reset the VMs without having to reboot them (booting is rather slow).

I practice I'll probably have ~16 instances at the same time. Constantly being reset to the same state.
I tried with QEMU, and it's seems totally doable with savevm, copy file, then doing loadvm twice in parallel.
(I'll be using a separate network for each VM, so I can be sure which one I'm talking to).

Well, as long as they're completely isolated from each other, you may have a better chance of success. However there will still be the issue of the IP address of the network interface. You can't have two networks using the same IP range (since libvirt doesn't use network namespaces for its networks), so the guest will need to change its IP, which means it will need to be notified of this need, possibly by having the host toggle the interface offline and back on - you can use virsh domif-setlink to do this.

Is this doable with libvirt, or am I better off using QEMU directly? and how? I couldn't do internal snapshots with --live, and snapshot-revert says it can't revert to external snapshots yet :)
(using QEMU directly would certainly leave me with a lot of manual network configuration)

Someone else will have to talk about the particulars of snapshots...