Re: any advantage to using pool type glusterfs if no GFAPI
On Fri, Oct 06, 2023 at 13:23:18 -0700, W Kern wrote:
I have used fuse mounted gluster with 'dir' for years. Works
great, aside
from having to use the --unsafe flag on migrations.
I am building up a new system and wondering if using the glusterfs type
would be a better/safer choice.
I'm not concerned about gfapi (host is U22LTS so I'm not even sure if it has
it).
I'm not concerned about the automount feature as that happens on start up
anyway.
I'm more interested if there is any technical or performance reasons to
prefer 'glusterfs' pool type over 'dir'
of course not having the --unsafe flag would be nice, but thats all scripted
in anyway.
Let's break down multiple things:
Firstly regarding use of '--unsafe' for migration on fuse-mounted
glusterfs. This should work without the need to use --unafe at least
since:
commit 478da65fb46c866973886848ae17f1e16199a77d
Author: Michal Prívozník <mprivozn(a)redhat.com>
Date: Thu Sep 27 16:19:31 2018 +0200
virFileIsSharedFSType: Check for fuse.glusterfs too
$ git desc 478da65fb46c866973886848ae17f1e16199a77d
v4.8.0-21-g478da65fb4
Thus released in libvirt-4.9
Secondly let's discuss the actual real way how qemu interacts with the
blocks. With fuse in use the gluster volume is mounted on the host and
qemu accesses it directly as any other file. Thus for qemu this is
transparent.
With libgfapi, each qemu instance runs their own gluster client
internally. Advantage is that you don't have it mounted in your host
and maybe save few cycles on fuse not being needed. Disadvantage is that
you have multiple copies of the gluster client in memory.
Now the third issue and that is storage pool type in libvirt.
So in the first place having a storage pool in libvirt is not really
needed. This depends on how you configure the disks in the VM:
If you do it via:
<disk type'file' device='disk'>
[...]
<source file='/path/to/some/glusterfs/mount/os.img'/>
</disk>
Then you are not even using a storage pool in the first place. That
happens only if you have a <disk type='volume'...>.
In such case I'd suggest you don't even set up one.
This brings us to pool type gluster. This pool type uses libgfapi to
natively access the gluster volume without mounting it in the host. Thus
if you need it to automount the images, it will not do that.
Additionally support for <disk type='volume'> was never finished for
that one either.
Conclusion: You don't want to use storage pool of type 'gluster', and
very likely you don't need any storage pool.