Permission Denied error when I try to open the image file stored on the ZFS disk accessed through sshfs from another machine...

Hello to everyone. I would like to boot the FreeBSD 13.2 image file using Libvirt + virt-manager. I have stored the image on the external hard drive,"formatted" with ZFS while I'm using Devuan 5 installed on my ARM (32 bit) Chromebook,where I have access to the ZFS disk using sshfs using this command : sshfs -o Compression=no -o allow_root -o transform_symlinks -o password_stdin root@192.168.1.2:/mnt/zroot2/zroot2 /mnt/zroot2/zroot2 <<< 'pass' This is the error I get when I try to boot the image file using virt-manager : error : qemuProcessReportLogError:1990 : internal error: process exited while connecting to monitor: 2023-11-28T20:53:46.882586Z qemu-system-arm: -blockdev {"driver":"file","filename":"/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm/FreeBSD-13.2-RELEASE-armv7.img","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm/FreeBSD-13.2-RELEASE-armv7.img': Permission denied According with this post : https://forum.manjaro.org/t/virt-manager-doesnt-work-permission-denied/10814... I tried to do the following : root@devuan-bunsen:/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm# getfacl /mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm/ getfacl: Removing leading '/' from absolute path names # file: mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm/ # owner: root # group: root user::rwx group::rwx other::rwx root@devuan-bunsen:/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm# setfacl -m user:libvirt-qemu:--x /mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm/ setfacl: /mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm/: Operation not supported Can someone help me to understand where the error is and how to fix it ? thanks.

On Tue, Nov 28, 2023 at 21:32:39 -0000, marietto2008@gmail.com wrote:
Hello to everyone.
I would like to boot the FreeBSD 13.2 image file using Libvirt + virt-manager. I have stored the image on the external hard drive,"formatted" with ZFS while I'm using Devuan 5 installed on my ARM (32 bit) Chromebook,where I have access to the ZFS disk using sshfs using this command :
sshfs -o Compression=no -o allow_root -o transform_symlinks -o password_stdin root@192.168.1.2:/mnt/zroot2/zroot2 /mnt/zroot2/zroot2 <<< 'pass'
This is the error I get when I try to boot the image file using virt-manager :
error : qemuProcessReportLogError:1990 : internal error: process exited while connecting to monitor: 2023-11-28T20:53:46.882586Z qemu-system-arm: -blockdev {"driver":"file","filename":"/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm/FreeBSD-13.2-RELEASE-armv7.img","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm/FreeBSD-13.2-RELEASE-armv7.img': Permission denied
Note that you didn't provide the VM xml or details on how the VM is configured so I'll speculate based on what most users would use. Important fact is that 'sshfs' by default doesn't allow other users to access the mounted directory. You partially bypassed that with "-o allow_root" but only for the root user. When a VM is run in the systme context (libvirt uri 'qemu:///system') then the VM process itself runs as the 'qemu' user and not root. So unless you've mounted the 'sshfs' as the qemu user, which would be hard to do it's most likely what's causing your problem. You can use -o allow_others, but beware that it indeed allows any user to access the sshfs mount. Some distros compile qemu with a direct ssh driver for disks, but that requires a very new libvirt and also you must setup SSH key authentication accessible from the user running your vm. Another obvious but very insecure option is to instruct libvirt to run the VM as the root user.

----> Another obvious but very insecure option is to instruct libvirt to run the VM as the root user. Is what I do,currently. Did you see my login prompt ? *root* @devuan-bunsen:/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm# It means that I do launch libvirtd & and virtlogd & as root. And this is the reason why I use "-o allow_root" on the sshfs command. But despite this,I can't access the image file stored on the zfs disk. On Wed, Nov 29, 2023 at 9:28 AM Peter Krempa <pkrempa@redhat.com> wrote:
Hello to everyone.
I would like to boot the FreeBSD 13.2 image file using Libvirt + virt-manager. I have stored the image on the external hard drive,"formatted" with ZFS while I'm using Devuan 5 installed on my ARM (32 bit) Chromebook,where I have access to the ZFS disk using sshfs using this command :
sshfs -o Compression=no -o allow_root -o transform_symlinks -o
On Tue, Nov 28, 2023 at 21:32:39 -0000, marietto2008@gmail.com wrote: password_stdin root@192.168.1.2:/mnt/zroot2/zroot2 /mnt/zroot2/zroot2 <<< 'pass'
This is the error I get when I try to boot the image file using
virt-manager :
error : qemuProcessReportLogError:1990 : internal error: process exited
while connecting to monitor: 2023-11-28T20:53:46.882586Z qemu-system-arm: -blockdev {"driver":"file","filename":"/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm/FreeBSD-13.2-RELEASE-armv7.img","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm/FreeBSD-13.2-RELEASE-armv7.img': Permission denied
Note that you didn't provide the VM xml or details on how the VM is configured so I'll speculate based on what most users would use.
Important fact is that 'sshfs' by default doesn't allow other users to access the mounted directory. You partially bypassed that with "-o allow_root" but only for the root user.
When a VM is run in the systme context (libvirt uri 'qemu:///system') then the VM process itself runs as the 'qemu' user and not root.
So unless you've mounted the 'sshfs' as the qemu user, which would be hard to do it's most likely what's causing your problem.
You can use -o allow_others, but beware that it indeed allows any user to access the sshfs mount.
Some distros compile qemu with a direct ssh driver for disks, but that requires a very new libvirt and also you must setup SSH key authentication accessible from the user running your vm.
Another obvious but very insecure option is to instruct libvirt to run the VM as the root user.
-- Mario.

On Wed, Nov 29, 2023 at 10:07:54 +0100, Mario Marietto wrote:
----> Another obvious but very insecure option is to instruct libvirt to run the VM as the root user.
Is what I do,currently. Did you see my login prompt ?
*root* @devuan-bunsen:/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm#
It means that I do launch libvirtd & and virtlogd & as root. And this is the reason why I use "-o allow_root" on the sshfs command. But despite this,I can't access the image file stored on the zfs disk.
Any default installation of libvirt will run a VM started from a 'root' user as user 'qemu' for security reasons. You'd have to actively change that to run the qemu process as 'root' user. You didn't post any VM xml or libvirt config, so I've assumed the default. Did you explicitly set the user for the VM to run as in the VM xml? Or did you modify /etc/libvirt/qemu.conf?

Fixed,adding to : /usr/local/etc/libvirt/qemu.conf the following parameters : user = "root" group = "root" I'd forgotten this step. Very thanks bro'. On Wed, Nov 29, 2023 at 10:15 AM Peter Krempa <pkrempa@redhat.com> wrote:
On Wed, Nov 29, 2023 at 10:07:54 +0100, Mario Marietto wrote:
----> Another obvious but very insecure option is to instruct libvirt to run the VM as the root user.
Is what I do,currently. Did you see my login prompt ?
*root*
@devuan-bunsen:/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/freebsd-kvm#
It means that I do launch libvirtd & and virtlogd & as root. And this is the reason why I use "-o allow_root" on the sshfs command. But despite this,I can't access the
image
file stored on the zfs disk.
Any default installation of libvirt will run a VM started from a 'root' user as user 'qemu' for security reasons. You'd have to actively change that to run the qemu process as 'root' user.
You didn't post any VM xml or libvirt config, so I've assumed the default. Did you explicitly set the user for the VM to run as in the VM xml? Or did you modify /etc/libvirt/qemu.conf?
-- Mario.
participants (3)
-
marietto2008@gmail.com
-
Mario Marietto
-
Peter Krempa