Hello, Is it possible to share a single host's Logical Volume among multiple local KVM guests which mount it read-only? I'm asking this because I have an idea to run multiple idential KVM guests (they all have exactly the same software installed on them), booting them from a shared local Logical Volume read-only root file system, or alternatively let them share the bulk of the software (/usr, /opt, /lib) from a common KVM host Logical volume. Is this possible? All my searches so far failed to turn up anything like this.

From the libvirt POV, there's nothing much todo except add <readonly/>

Would it be possible using qcow2 instead of raw LV? If so - would it be worth the performance hit of switching from LV to qcow2?

On 21 September 2012 21:30, Daniel P. Berrange <berrange(a)redhat.com&gt; wrote: > On Fri, Sep 21, 2012 at 09:22:24PM +1000, Amos Shapira wrote: > > Hello, > > > > Is it possible to share a single host's Logical Volume among multiple > local > > KVM guests which mount it read-only? > > > > I'm asking this because I have an idea to run multiple idential KVM > guests > > (they all have exactly the same software installed on them), booting them > > from a shared local Logical Volume read-only root file system, or > > alternatively let them share the bulk of the software (/usr, /opt, /lib) > > from a common KVM host Logical volume. > > > > Is this possible? All my searches so far failed to turn up anything like > > this. > > From the libvirt POV, there's nothing much todo except add <readonly/> > inside the <disk> element. This will ensure QEMU only gets given read > permission on the disk backend. The important thing is to then make sure > your guests actually mount the filesystem with the readonly flag. > > > Would it be possible using qcow2 instead of raw LV? If so - would it be > > worth the performance hit of switching from LV to qcow2? > > The type of backend storage doesn't really affect things if the > disk is fully readonly. > Thanks very much! That's very helpful to know. Is this something that someone has already done before (booting multiple KVM guests from shared read-only root file system) or am I on my own with this?

On 21 September 2012 21:40, Daniel P. Berrange <berrange(a)redhat.com&gt; wrote: > On Fri, Sep 21, 2012 at 09:34:02PM +1000, Amos Shapira wrote: > > On 21 September 2012 21:30, Daniel P. Berrange <berrange(a)redhat.com&gt; > wrote: > > > > > On Fri, Sep 21, 2012 at 09:22:24PM +1000, Amos Shapira wrote: > > > > Hello, > > > > > > > > Is it possible to share a single host's Logical Volume among multiple > > > local > > > > KVM guests which mount it read-only? > > > > > > > > I'm asking this because I have an idea to run multiple idential KVM > > > guests > > > > (they all have exactly the same software installed on them), booting > them > > > > from a shared local Logical Volume read-only root file system, or > > > > alternatively let them share the bulk of the software (/usr, /opt, > /lib) > > > > from a common KVM host Logical volume. > > > > > > > > Is this possible? All my searches so far failed to turn up anything > like > > > > this. > > > > > > From the libvirt POV, there's nothing much todo except add <readonly/> > > > inside the <disk> element. This will ensure QEMU only gets given read > > > permission on the disk backend. The important thing is to then make > sure > > > your guests actually mount the filesystem with the readonly flag. > > > > > > > Would it be possible using qcow2 instead of raw LV? If so - would it > be > > > > worth the performance hit of switching from LV to qcow2? > > > > > > The type of backend storage doesn't really affect things if the > > > disk is fully readonly. > > > > > > > Thanks very much! That's very helpful to know. > > > > Is this something that someone has already done before (booting multiple > > KVM guests from shared read-only root file system) or am I on my own with > > this? > > I've do it with libvirt-sandbox, but using a different approach. Instead > of a shared readonly disk image, I use the plan9 filesystem to pass a > chroot area from the host to the guest. Then I have to mount certain > areas with tmpfs or other writable filesystems (eg /var, /tmp/, parts > of /etc, etc). > > What you want todo is definitely doable - the hard part is just in making > your OS image in a way to works with readonly root. This is typically not > to well documented, so involves alot of trial and error at first. > Thanks. I'll try to make my workplace interested in this and report back if/when I have interesting results. In general, I'm thinking of replicating the way that diskless NFS clients used to boot from a common root file system on an NFS server, where the only difference among them is their MAC address, which in turn lets them pull a little different information (e.g. host name) from DHCP and use this as a basis to mount the private parts of the disk (e.g. /var, /etc?) and configuration. The other question is whether I am correct in expecting that this could improve my disk IO profile, since all the guests would be accessing the same host's disk blocks.

On Fri, Sep 21, 2012 at 09:22:24PM +1000, Amos Shapira wrote: > 2. Maintenance - if I get to share the common disk image among all of them > then I only need to install/update it once instead of once for each KVM > guest (we already use Puppet but it doesn't mean it comes for free). How do you plan on upgrading the read-only VMs after the initial installation, or are they short-lived and frequently recreated from scratch using a fresh image? Read-only, shared /usr between machines is a nice idea in theory, but at least the modern linux distros' package management tend not to support it that well in practice.

IMHO using a QCOW2 backing files as base templates works pretty well and gives you similar disk space savings, at least initially. And KSM does the same with RAM.