[libvirt-users] Using qemu active blockcommit results in 'Permission denied' error
Hello all, I tried following this guide: https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit Unfortunately when I try to do the final virsh blockcommit step I always get the following error: error: internal error: unable to execute QEMU command 'block-commit': Could not reopen file: Permission denied I checked directory und image file permissions, app-armor profiles (set to complain mode for now.) and libvirt logs but nothing there gives me any hints what might be going wrong. This is on a debian buster system, using libvirtd (libvirt) 5.0.0 qemu 3.1+dfsg-7 I would appreciate any hints on how to proceed/get more debug info on this. Thanks! Marcus
On Thu, May 30, 2019 at 22:12:14 +0200, Marcus Hoffmann wrote:
Hello all,
Hi,
I tried following this guide: https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit
Unfortunately when I try to do the final virsh blockcommit step I always get the following error:
error: internal error: unable to execute QEMU command 'block-commit': Could not reopen file: Permission denied
I checked directory und image file permissions, app-armor profiles (set to complain mode for now.) and libvirt logs but nothing there gives me any hints what might be going wrong.
This is on a debian buster system, using
I was doing some changes in the blockcommit code recently so I might have messed something up. Could you please collect debug logs: https://wiki.libvirt.org/page/DebugLogs when the problem reproduces and also a directory listing of the path where the image is stored so I can check if the permission code is working properly.
Hi Peter, On 31.05.19 09:57, Peter Krempa wrote:
On Thu, May 30, 2019 at 22:12:14 +0200, Marcus Hoffmann wrote:
Hello all,
Hi,
I tried following this guide: https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit
Unfortunately when I try to do the final virsh blockcommit step I always get the following error:
error: internal error: unable to execute QEMU command 'block-commit': Could not reopen file: Permission denied
I checked directory und image file permissions, app-armor profiles (set to complain mode for now.) and libvirt logs but nothing there gives me any hints what might be going wrong.
This is on a debian buster system, using
I was doing some changes in the blockcommit code recently so I might have messed something up. Could you please collect debug logs:
https://wiki.libvirt.org/page/DebugLogs
when the problem reproduces and also a directory listing of the path where the image is stored so I can check if the permission code is working properly.
Logfile attached. ~ # ls -al /var/lib/libvirt/images/ total 1271917588 drwxr-xr-x 2 root root 4096 Mai 29 14:47 . drwxr-xr-x 7 root root 4096 Mai 27 15:55 .. -rw-r--r-- 1 libvirt-qemu libvirt-qemu 214821961728 Mai 29 14:47 drache3.qcow2 -rw-r--r-- 1 libvirt-qemu libvirt-qemu 395671371776 Mai 29 14:47 drache_addon.qcow2 -rw-r--r-- 1 libvirt-qemu libvirt-qemu 90944765952 Mai 31 14:02 drache_overlaya.qcow2 -rw-r--r-- 1 libvirt-qemu libvirt-qemu 115860766720 Mai 31 14:02 drache_overlayb.qcow2 Marcus
On May 31, 2019 2:03:40 PM GMT+02:00, Marcus Hoffmann <bubu@bubu1.eu> wrote:
Hi Peter,
On 31.05.19 09:57, Peter Krempa wrote:
On Thu, May 30, 2019 at 22:12:14 +0200, Marcus Hoffmann wrote:
Hello all,
Hi,
I tried following this guide:
https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit
Unfortunately when I try to do the final virsh blockcommit step I
always
get the following error:
error: internal error: unable to execute QEMU command 'block-commit': Could not reopen file: Permission denied
I checked directory und image file permissions, app-armor profiles (set to complain mode for now.) and libvirt logs but nothing there gives me any hints what might be going wrong.
This is on a debian buster system, using
I was doing some changes in the blockcommit code recently so I might have messed something up. Could you please collect debug logs:
https://wiki.libvirt.org/page/DebugLogs
when the problem reproduces and also a directory listing of the path where the image is stored so I can check if the permission code is working properly.
Logfile attached.
~ # ls -al /var/lib/libvirt/images/ total 1271917588 drwxr-xr-x 2 root root 4096 Mai 29 14:47 . drwxr-xr-x 7 root root 4096 Mai 27 15:55 .. -rw-r--r-- 1 libvirt-qemu libvirt-qemu 214821961728 Mai 29 14:47 drache3.qcow2 -rw-r--r-- 1 libvirt-qemu libvirt-qemu 395671371776 Mai 29 14:47 drache_addon.qcow2 -rw-r--r-- 1 libvirt-qemu libvirt-qemu 90944765952 Mai 31 14:02 drache_overlaya.qcow2 -rw-r--r-- 1 libvirt-qemu libvirt-qemu 115860766720 Mai 31 14:02 drache_overlayb.qcow2
Marcus
Hi Peter, did you get the logfile I sent? (Just wanting to make sure the email actually got delivered. I get a lot of dmarc failures from the mailing list :-/) Did you have a chance to take a look? Best, Marcus
On Wed, Jun 05, 2019 at 17:02:05 +0200, Marcus Hoffmann wrote:
On May 31, 2019 2:03:40 PM GMT+02:00, Marcus Hoffmann <bubu@bubu1.eu> wrote:
[...]
Hi Peter,
Hi,
did you get the logfile I sent? (Just wanting to make sure the email actually got delivered. I get a lot of dmarc failures from the mailing list :-/)
Did you have a chance to take a look?
I did in fact get it, but I somehow forgot to look at it. I'm marking this and I'll have a look tomorrow. Sorry for that.
On Fri, May 31, 2019 at 14:03:40 +0200, Marcus Hoffmann wrote:
Hi Peter,
On 31.05.19 09:57, Peter Krempa wrote:
On Thu, May 30, 2019 at 22:12:14 +0200, Marcus Hoffmann wrote:
Hello all,
Hi,
I tried following this guide: https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit
Unfortunately when I try to do the final virsh blockcommit step I always get the following error:
error: internal error: unable to execute QEMU command 'block-commit': Could not reopen file: Permission denied
I managed to reproduce this issue but when using selinux. I'll try to fix it with selinux and will try to assess whether it has the possiblity to fix apparmor too. I'll cc you on a patch when I'll be able to fix it.
On Tue, Jun 11, 2019 at 14:35:46 +0200, Peter Krempa wrote:
On Fri, May 31, 2019 at 14:03:40 +0200, Marcus Hoffmann wrote:
Hi Peter,
On 31.05.19 09:57, Peter Krempa wrote:
On Thu, May 30, 2019 at 22:12:14 +0200, Marcus Hoffmann wrote:
Hello all,
Hi,
I tried following this guide: https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit
Unfortunately when I try to do the final virsh blockcommit step I always get the following error:
error: internal error: unable to execute QEMU command 'block-commit': Could not reopen file: Permission denied
I managed to reproduce this issue but when using selinux. I'll try to fix it with selinux and will try to assess whether it has the possiblity to fix apparmor too. I'll cc you on a patch when I'll be able to fix it.
Well, The problem I managed to fix had the same symptoms but probably was not what you see, as you are using libvirt 5.0.0 and I broke the permissions code in libvirt 5.4.0. Unfortunately I can't tell what's wrong from the debug logs you've provided. Is there a possibility to collect anything from apparmor? In selinux world we do collect denials of the security model in a log file which might indicate what's happening. Also I've pushed a patch which adds more logging to the permission-changing code executed while doing blockjobs: commit e6635c626a252669c79a84fe0a2af11a361aa341 (HEAD -> master, origin/master, origin/HEAD) Author: Peter Krempa <pkrempa@redhat.com> Date: Wed Jun 12 13:49:57 2019 +0200 qemu: domain: Log some useful data in qemuDomainStorageSourceAccessModify Log the flags passed to the function in a exploded state so that it's easily visible what's happening to the image. Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Unfortunately that commit can't be applied to libvirt 5.0 because it depends on a refactor which I pushed in 5.4 (which also caused the problem I was fixing recently). If you could test the upstream version it would be great. Thanks for reporting the problem and I'd be grateful if you could collect logs from the apparmor security thing.
Hello Peter, On 13.06.19 10:08, Peter Krempa wrote:
On Tue, Jun 11, 2019 at 14:35:46 +0200, Peter Krempa wrote:
On Fri, May 31, 2019 at 14:03:40 +0200, Marcus Hoffmann wrote: [...]
I managed to reproduce this issue but when using selinux. I'll try to fix it with selinux and will try to assess whether it has the possiblity to fix apparmor too. I'll cc you on a patch when I'll be able to fix it.
Well,
The problem I managed to fix had the same symptoms but probably was not what you see, as you are using libvirt 5.0.0 and I broke the permissions code in libvirt 5.4.0.
Unfortunately I can't tell what's wrong from the debug logs you've provided. Is there a possibility to collect anything from apparmor? In selinux world we do collect denials of the security model in a log file which might indicate what's happening.
As I wrote in my original email I *thought* I had disabled apparmor enforcement for libvirt completely at this point. I'm not an apparmor expert, so I'm really not too sure. I'll see if I can gather more information.
Also I've pushed a patch which adds more logging to the permission-changing code executed while doing blockjobs:
commit e6635c626a252669c79a84fe0a2af11a361aa341 (HEAD -> master, origin/master, origin/HEAD) Author: Peter Krempa <pkrempa@redhat.com> Date: Wed Jun 12 13:49:57 2019 +0200
qemu: domain: Log some useful data in qemuDomainStorageSourceAccessModify
Log the flags passed to the function in a exploded state so that it's easily visible what's happening to the image.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com>
Unfortunately that commit can't be applied to libvirt 5.0 because it depends on a refactor which I pushed in 5.4 (which also caused the problem I was fixing recently). If you could test the upstream version it would be great.
Thanks for reporting the problem and I'd be grateful if you could collect logs from the apparmor security thing.
I'll try to upgrade to upstream libvirt. It will probably take me a bit to get around to this. Thanks looing into this. Marcus
participants (2)
-
Marcus Hoffmann -
Peter Krempa