[libvirt-users] qemu:///session and network-mounted home directories
Dear libvirt-users, Suppose my home directory is network mounted on to several different machines. And suppose I defined some VMs in qemu:///session. The configuration files and disk images for qemu:///session are stored in my home directory. Now suppose I log in to some machine and use virsh. The qemu:///session VMs that are marked autostart are started. Now if I log into another machine and use virsh, it will presumably try to start my autostart VMs again, since the same configuration is present in both homedirs. Is there anything libvirt does that might prevent it from starting the same VM multiple times on different machines? Or might allow machine-specific autostart configuration? Maybe something could be done with /etc/machine-id! In terms of qemu:///system, this is like having /var/lib/libvirt/images/ live on a network-mounted filesystem (fairly normal I assume) and also having /etc/libvirt/qemu live on a network-mounted filesystem (not as normal). Not sure how you would deal with the issue there either but I can see why you might want to have your same VM configuration available on all hosts, so it would be cool to figure out that problem as well. The answer may be the VM configuration is host-specific, so sharing VM configuration between hosts is not a good idea. In that case, any other recommendations on how to deal with the networked homedir case? Thanks!
On 10/10/2015 10:53 AM, Spencer Baugh wrote:
Dear libvirt-users,
Suppose my home directory is network mounted on to several different machines. And suppose I defined some VMs in qemu:///session. The configuration files and disk images for qemu:///session are stored in my home directory.
Now suppose I log in to some machine and use virsh. The qemu:///session VMs that are marked autostart are started. Now if I log into another machine and use virsh, it will presumably try to start my autostart VMs again, since the same configuration is present in both homedirs.
Is there anything libvirt does that might prevent it from starting the same VM multiple times on different machines? Or might allow machine-specific autostart configuration? Maybe something could be done with /etc/machine-id!
In terms of qemu:///system, this is like having /var/lib/libvirt/images/ live on a network-mounted filesystem (fairly normal I assume) and also having /etc/libvirt/qemu live on a network-mounted filesystem (not as normal). Not sure how you would deal with the issue there either but I can see why you might want to have your same VM configuration available on all hosts, so it would be cool to figure out that problem as well.
The answer may be the VM configuration is host-specific, so sharing VM configuration between hosts is not a good idea. In that case, any other recommendations on how to deal with the networked homedir case?
The proper way to make sure shared VMs aren't started across multiple machines is libvirt's locking support: https://libvirt.org/locking.html It requires running a separate daemon though so isn't trivial, and I have no idea if it can be made to work with qemu:///session. It's an interesting problem though, but not one that I can see libvirt adding explicit autostart+machine-id support to handle since it's pretty niche. Maybe you can disable autostart at the libvirt level, and add a custom ~/bin/virsh wrapper to do the manually autostart VMs only for the machine-id you care about, or something similarly crazy - Cole
Cole Robinson <crobinso@redhat.com> writes:
The proper way to make sure shared VMs aren't started across multiple machines is libvirt's locking support: https://libvirt.org/locking.html
It requires running a separate daemon though so isn't trivial, and I have no idea if it can be made to work with qemu:///session.
Hmm, but what do you mean by shared VMs here? Just shared disk images, or also shared configuration, or also marked autostart on multiple hosts? It seems like there are two closely related problems here: 1. For some shared VM and host, should we start that VM on that host? 2. Preventing VMs from running on multiple hosts at the same time Problem 2 is definitely solved pretty well by libvirt's locking support; even if that locking doesn't currently work properly with qemu:///session, I accept the idea that the correct answer is to get that working. And problem 1 is solved as a side effect of solving problem 2, because now one can just mark the VMs autostart on all hosts and let them race to get a lock on the disk image, arbitrarily distributing them over the hosts. But surely there is a better way to solve problem 1 than that! I suppose one could punt that question to a cross-host VM scheduler like OpenStack. In that case I guess what one might do is not mark anything autostart, and have the external scheduler come in on bootup and kick specific things into action. Then you would rely on the cross-host scheduler knowing what should run where. But that doesn't really scale elegantly between the one-host and multi-host cases, since in the one-host case it would be somewhat awkward and unnecessary to use a multi-host scheduler. Hmm, I suppose if the external scheduler instead specified a mapping from /etc/machine-id to a list of autostarted VMs, that mapping could be shared on all hosts, and so would work for both cases rather nicely. Alternatively, even better: the autostart mechanism could be made generic, allowing multiple ways to get the list of VMs that should be autostarted. Then the most simple way (certainly not the best way) to get the desired functionality would be to write a script that returns a list of VMs that should be running on the current host. Does that seem like a plausible approach? (This would definitely work well with my future plan to write a simple, minimal multi-host scheduler that can work with qemu:///session...)
It's an interesting problem though, but not one that I can see libvirt adding explicit autostart+machine-id support to handle since it's pretty niche.
Hmm, well, I was going to say that that was just a throwaway idea, but I've actually convinced myself that it might be a good idea now. Since it has some pretty good applicability to actual qemu:///system stuff.
Maybe you can disable autostart at the libvirt level, and add a custom ~/bin/virsh wrapper to do the manually autostart VMs only for the machine-id you care about, or something similarly crazy
Yeah, I could definitely do some local hack to get things working nicely, but local hacks are what I am trying to avoid...
On 10/10/2015 02:40 PM, Spencer Baugh wrote:
Cole Robinson <crobinso@redhat.com> writes:
The proper way to make sure shared VMs aren't started across multiple machines is libvirt's locking support: https://libvirt.org/locking.html
It requires running a separate daemon though so isn't trivial, and I have no idea if it can be made to work with qemu:///session.
Hmm, but what do you mean by shared VMs here? Just shared disk images, or also shared configuration, or also marked autostart on multiple hosts?
Just shared libvirt XML and disk images, not specifically autostart.
It seems like there are two closely related problems here: 1. For some shared VM and host, should we start that VM on that host? 2. Preventing VMs from running on multiple hosts at the same time
Problem 2 is definitely solved pretty well by libvirt's locking support; even if that locking doesn't currently work properly with qemu:///session, I accept the idea that the correct answer is to get that working.
And problem 1 is solved as a side effect of solving problem 2, because now one can just mark the VMs autostart on all hosts and let them race to get a lock on the disk image, arbitrarily distributing them over the hosts.
But surely there is a better way to solve problem 1 than that!
I suppose one could punt that question to a cross-host VM scheduler like OpenStack. In that case I guess what one might do is not mark anything autostart, and have the external scheduler come in on bootup and kick specific things into action. Then you would rely on the cross-host scheduler knowing what should run where.
But that doesn't really scale elegantly between the one-host and multi-host cases, since in the one-host case it would be somewhat awkward and unnecessary to use a multi-host scheduler.
Hmm, I suppose if the external scheduler instead specified a mapping from /etc/machine-id to a list of autostarted VMs, that mapping could be shared on all hosts, and so would work for both cases rather nicely.
Alternatively, even better: the autostart mechanism could be made generic, allowing multiple ways to get the list of VMs that should be autostarted. Then the most simple way (certainly not the best way) to get the desired functionality would be to write a script that returns a list of VMs that should be running on the current host.
Does that seem like a plausible approach?
Plausible/workable, sure. But is it worth implementing and maintaining in libvirt? IMO no. I see libvirt's built-in autostart feature as just a simple convenience feature for end users, and this strays from that. Also you would already need to configure each host to provide this custom list of VMs to autostart, so you aren't getting away from per-host config anyways. At that point really how much more work is it to implement this external to libvirt? And it doesn't seem useful for existing tools: high level multi host management like openstack, ovirt don't need to use libvirt's autostart feature, since it's trivial to roll their own already have a daemon running on each physical machine, and they need to do a ton of other host tasks. Adding a 'virsh start' equivalent is easy. So really I think if the existing autostart impl isn't sufficient, and the use case is niche (which it is), it's better to roll your own solution. Of course, I'm not the authority here, just a guy that writes libvirt patches on occasion :) - Cole
participants (2)
-
Cole Robinson -
Spencer Baugh