Re: [EXT] Re: any advantage to using pool type glusterfs if no GFAPI

On Fri, Oct 06, 2023 at 13:23:18 -0700, W Kern wrote: > I have used fuse mounted gluster with 'dir' for years. Works great, aside > from  having to use the --unsafe flag on migrations. > > I am building up a new system  and wondering if using the glusterfs type > would be a better/safer choice. > > I'm not concerned about gfapi (host is U22LTS so I'm not even sure if it has > it). > > I'm not concerned about the automount feature as that happens on start up > anyway. > > I'm more interested if there is any technical or performance reasons to > prefer 'glusterfs' pool type over 'dir' > > of course not having the --unsafe flag would be nice, but thats all scripted > in anyway. Let's break down multiple things: Firstly regarding use of '--unsafe' for migration on fuse-mounted glusterfs. This should work without the need to use --unafe at least since: commit 478da65fb46c866973886848ae17f1e16199a77d Author: Michal Prívozník <mprivozn(a)redhat.com&gt; Date: Thu Sep 27 16:19:31 2018 +0200 virFileIsSharedFSType: Check for fuse.glusterfs too $ git desc 478da65fb46c866973886848ae17f1e16199a77d v4.8.0-21-g478da65fb4 Thus released in libvirt-4.9 Secondly let's discuss the actual real way how qemu interacts with the blocks. With fuse in use the gluster volume is mounted on the host and qemu accesses it directly as any other file. Thus for qemu this is transparent. With libgfapi, each qemu instance runs their own gluster client internally. Advantage is that you don't have it mounted in your host and maybe save few cycles on fuse not being needed. Disadvantage is that you have multiple copies of the gluster client in memory. Now the third issue and that is storage pool type in libvirt. So in the first place having a storage pool in libvirt is not really needed. This depends on how you configure the disks in the VM: If you do it via: <disk type'file' device='disk'> [...] <source file='/path/to/some/glusterfs/mount/os.img'/> </disk> Then you are not even using a storage pool in the first place. That happens only if you have a <disk type='volume'...>. In such case I'd suggest you don't even set up one. This brings us to pool type gluster. This pool type uses libgfapi to natively access the gluster volume without mounting it in the host. Thus if you need it to automount the images, it will not do that. Additionally support for <disk type='volume'> was never finished for that one either. Conclusion: You don't want to use storage pool of type 'gluster', and very likely you don't need any storage pool.