[libvirt-users] Libvirt: dynamic ownership did not work
Hi, I have a very strange problem with libvirt. I work on some machines with libvirt (Debian/ Arch Linux) and libvirt set the ownership of images file automatically to the qemu user / group for example on Arch Linux to nobody:kvm. So when I copy an image file with root and use I then with qemu, libvirt change the owner/ group to nobody:kvm. But I also compiled libvirt for a machine (gcc 4.9.4 glibc 2.12) and on this machine libvirt did not change the ownership of the image files which results in this error: libvirtError: internal error: process exited while connecting to monitor: able-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0: Could not open '/data/hdd1/libvirt/images/test.img': Permission denied When I set the ownership manually to nobody:kvm everything is fine, but I could not work out why libvirt is unable to set the ownership automatically. Can anybody give me a hint where I could search further to work out the problem? My libvirt version is 1.3.3.2 and the settings dynamic_ownership = 1 in /etc/libvirt/qemu.conf is set. I also created a bug report where I described the problem a little bit more detailed. https://bugzilla.redhat.com/show_bug.cgi?id=1363864 Thanks for every help. Regards Jonatan
On 03.08.2016 21:17, Jonatan Schlag wrote:
Hi, I have a very strange problem with libvirt. I work on some machines with libvirt (Debian/ Arch Linux) and libvirt set the ownership of images file automatically to the qemu user / group for example on Arch Linux to nobody:kvm. So when I copy an image file with root and use I then with qemu, libvirt change the owner/ group to nobody:kvm.
But I also compiled libvirt for a machine (gcc 4.9.4 glibc 2.12) and on this machine libvirt did not change the ownership of the image files which results in this error:
libvirtError: internal error: process exited while connecting to monitor: able-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0: Could not open '/data/hdd1/libvirt/images/test.img': Permission denied
Can you please share the debug logs? http://wiki.libvirt.org/page/DebugLogs Also, my initial suspect, before diving any deeper is that usually, when users compile libvirt on their own, they forget to set the correct prefix, therefore libvirt is looking for its config files NOT under /etc/libvirt but /usr/local/etc/ or whatever. BTW: is the daemon running under root? Michal
Am Do, 4. Aug, 2016 um 11:32 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 03.08.2016 21:17, Jonatan Schlag wrote:
Hi, I have a very strange problem with libvirt. I work on some machines with libvirt (Debian/ Arch Linux) and libvirt set the ownership of images file automatically to the qemu user / group for example on Arch Linux to nobody:kvm. So when I copy an image file with root and use I then with qemu, libvirt change the owner/ group to nobody:kvm.
But I also compiled libvirt for a machine (gcc 4.9.4 glibc 2.12) and on this machine libvirt did not change the ownership of the image files which results in this error:
libvirtError: internal error: process exited while connecting to monitor: able-ticketing,seamless-migration=on -device
qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive
file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0: Could not open '/data/hdd1/libvirt/images/test.img': Permission denied
Can you please share the debug logs?
http://wiki.libvirt.org/page/DebugLogs
Also, my initial suspect, before diving any deeper is that usually, when users compile libvirt on their own, they forget to set the correct prefix, therefore libvirt is looking for its config files NOT under /etc/libvirt but /usr/local/etc/ or whatever.
BTW: is the daemon running under root?
Michal
Hi, The daemon runs under root. I uploaded the debug logs to: http://people.ipfire.org/~jschlag/1363864/1_libvirtd.log The UID of the user nobody is 99, the GID of the group kvm is 1011. I added my configure options to the bug report. Following the log the ownership is changed but why is the file still owned by root:root? Regards Jonatan
On 04.08.2016 12:12, Jonatan Schlag wrote:
Am Do, 4. Aug, 2016 um 11:32 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 03.08.2016 21:17, Jonatan Schlag wrote:
Hi, I have a very strange problem with libvirt. I work on some machines with libvirt (Debian/ Arch Linux) and libvirt set the ownership of images file automatically to the qemu user / group for example on Arch Linux to nobody:kvm. So when I copy an image file with root and use I then with qemu, libvirt change the owner/ group to nobody:kvm.
But I also compiled libvirt for a machine (gcc 4.9.4 glibc 2.12) and on this machine libvirt did not change the ownership of the image files which results in this error:
libvirtError: internal error: process exited while connecting to monitor: able-ticketing,seamless-migration=on -device
qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2
-device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive
file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0:
Could not open '/data/hdd1/libvirt/images/test.img': Permission denied
Can you please share the debug logs?
http://wiki.libvirt.org/page/DebugLogs
Also, my initial suspect, before diving any deeper is that usually, when users compile libvirt on their own, they forget to set the correct prefix, therefore libvirt is looking for its config files NOT under /etc/libvirt but /usr/local/etc/ or whatever.
BTW: is the daemon running under root?
Michal
Hi,
The daemon runs under root.
I uploaded the debug logs to:
http://people.ipfire.org/~jschlag/1363864/1_libvirtd.log
The UID of the user nobody is 99, the GID of the group kvm is 1011.
I added my configure options to the bug report.
Following the log the ownership is changed but why is the file still owned by root:root?
Right. the file is set ownership. One thing though - libvirt does not label the whole path, just the file. So maybe you should check whether nobody:kvm has access into the /data/hdd1/libvirt/images dir (and also each one in the path). What about apparmor? In the bug report you say that selinux is disabled, but what about apparmor? Michal
Am Do, 4. Aug, 2016 um 1:38 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 04.08.2016 12:12, Jonatan Schlag wrote:
Am Do, 4. Aug, 2016 um 11:32 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 03.08.2016 21:17, Jonatan Schlag wrote:
Hi, I have a very strange problem with libvirt. I work on some machines with libvirt (Debian/ Arch Linux) and libvirt set the ownership of images file automatically to the qemu user / group for example on Arch Linux to nobody:kvm. So when I copy an image file with root and use I then with qemu, libvirt change the owner/ group to nobody:kvm.
But I also compiled libvirt for a machine (gcc 4.9.4 glibc 2.12) and on this machine libvirt did not change the ownership of the image files which results in this error:
libvirtError: internal error: process exited while connecting to monitor: able-ticketing,seamless-migration=on -device
qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2
-device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive
file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0:
Could not open '/data/hdd1/libvirt/images/test.img': Permission denied
Can you please share the debug logs?
http://wiki.libvirt.org/page/DebugLogs
Also, my initial suspect, before diving any deeper is that usually, when users compile libvirt on their own, they forget to set the correct prefix, therefore libvirt is looking for its config files NOT under /etc/libvirt but /usr/local/etc/ or whatever.
BTW: is the daemon running under root?
Michal
Hi,
The daemon runs under root.
I uploaded the debug logs to:
http://people.ipfire.org/~jschlag/1363864/1_libvirtd.log
The UID of the user nobody is 99, the GID of the group kvm is 1011.
I added my configure options to the bug report.
Following the log the ownership is changed but why is the file still owned by root:root?
Right. the file is set ownership. But the file ist still owned by root:root and so it is not accessable by qemu as nobody:kvm. In the moment the only possible way is that the change of the ownership fail, but then there should be an error message, but there is no error message in the log.
One thing though - libvirt does not label the whole path, just the file. So maybe you should check whether nobody:kvm has access into the /data/hdd1/libvirt/images dir (and also each one in the path).
When is set the ownership manually to nobody:kcm qemu is able to acces the fail so i think nobody:kvm has access to each folder in the path
What about apparmor? In the bug report you say that selinux is disabled, but what about apparmor?
apparmor is also disabled or better it is like se linux not installed.
Michal
On 08/04/2016 07:59 AM, Jonatan Schlag wrote:
Am Do, 4. Aug, 2016 um 1:38 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 04.08.2016 12:12, Jonatan Schlag wrote:
Am Do, 4. Aug, 2016 um 11:32 schrieb Michal Privoznik <mprivozn@redhat.com <mailto:mprivozn@redhat.com>>:
On 03.08.2016 21:17, Jonatan Schlag wrote:
Hi, I have a very strange problem with libvirt. I work on some machines with libvirt (Debian/ Arch Linux) and libvirt set the ownership of images file automatically to the qemu user / group for example on Arch Linux to nobody:kvm. So when I copy an image file with root and use I then with qemu, libvirt change the owner/ group to nobody:kvm. But I also compiled libvirt for a machine (gcc 4.9.4 glibc 2.12) and on this machine libvirt did not change the ownership of the image files which results in this error: libvirtError: internal error: process exited while connecting to monitor: able-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0: Could not open '/data/hdd1/libvirt/images/test.img': Permission denied
Can you please share the debug logs? http://wiki.libvirt.org/page/DebugLogs Also, my initial suspect, before diving any deeper is that usually, when users compile libvirt on their own, they forget to set the correct prefix, therefore libvirt is looking for its config files NOT under /etc/libvirt but /usr/local/etc/ or whatever. BTW: is the daemon running under root? Michal
Hi, The daemon runs under root. I uploaded the debug logs to: http://people.ipfire.org/~jschlag/1363864/1_libvirtd.log <http://people.ipfire.org/%7Ejschlag/1363864/1_libvirtd.log> The UID of the user nobody is 99, the GID of the group kvm is 1011. I added my configure options to the bug report. Following the log the ownership is changed but why is the file still owned by root:root?
Right. the file is set ownership. But the file ist still owned by root:root and so it is not accessable by qemu as nobody:kvm. In the moment the only possible way is that the change of the ownership fail, but then there should be an error message, but there is no error message in the log.
Are you checking it after the operation has failed and libvirt has continued? If libvirt fails to access the file, it will then set everything back to its original state, so looking at the file ownership after the fact tells you nothing. If you really want to see if the file has the proper ownership, you need to attach gdb to libvirtd and set an appropriate breakpoint.
On 04.08.2016 13:59, Jonatan Schlag wrote:
Am Do, 4. Aug, 2016 um 1:38 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 04.08.2016 12:12, Jonatan Schlag wrote:
Am Do, 4. Aug, 2016 um 11:32 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 03.08.2016 21:17, Jonatan Schlag wrote:
Hi, I have a very strange problem with libvirt. I work on some machines with libvirt (Debian/ Arch Linux) and libvirt set the ownership of images file automatically to the qemu user / group for example on Arch Linux to nobody:kvm. So when I copy an image file with root and use I then with qemu, libvirt change the owner/ group to nobody:kvm.
But I also compiled libvirt for a machine (gcc 4.9.4 glibc 2.12) and on this machine libvirt did not change the ownership of the image files which results in this error:
libvirtError: internal error: process exited while connecting to monitor: able-ticketing,seamless-migration=on -device
qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2
-device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive
file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0:
Could not open '/data/hdd1/libvirt/images/test.img': Permission denied
Can you please share the debug logs?
http://wiki.libvirt.org/page/DebugLogs
Also, my initial suspect, before diving any deeper is that usually, when users compile libvirt on their own, they forget to set the correct prefix, therefore libvirt is looking for its config files NOT under /etc/libvirt but /usr/local/etc/ or whatever.
BTW: is the daemon running under root?
Michal
Hi,
The daemon runs under root.
I uploaded the debug logs to:
http://people.ipfire.org/~jschlag/1363864/1_libvirtd.log
The UID of the user nobody is 99, the GID of the group kvm is 1011.
I added my configure options to the bug report.
Following the log the ownership is changed but why is the file still owned by root:root?
Right. the file is set ownership. But the file ist still owned by root:root and so it is not accessable by qemu as nobody:kvm. In the moment the only possible way is that the change of the ownership fail, but then there should be an error message, but there is no error message in the log.
Then the other option that comes to my mind is a race with somebody else on the system. You can attach gdb to the daemon and set breakpoint to virSecurityDACSetOwnershipInternal(). In the arguments you should see the path eventually among with uid:gid. BTW: what's the domain XML? Michal
Am Do, 4. Aug, 2016 um 5:07 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 04.08.2016 13:59, Jonatan Schlag wrote:
Am Do, 4. Aug, 2016 um 1:38 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 04.08.2016 12:12, Jonatan Schlag wrote:
Am Do, 4. Aug, 2016 um 11:32 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 03.08.2016 21:17, Jonatan Schlag wrote:
Hi, I have a very strange problem with libvirt. I work on some machines with libvirt (Debian/ Arch Linux) and libvirt set the ownership of images file automatically to the qemu user / group for example on Arch Linux to nobody:kvm. So when I copy an image file with root and use I then with qemu, libvirt change the owner/ group to nobody:kvm.
But I also compiled libvirt for a machine (gcc 4.9.4 glibc 2.12) and on this machine libvirt did not change the ownership of the image files which results in this error:
libvirtError: internal error: process exited while connecting to monitor: able-ticketing,seamless-migration=on -device
qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2
-device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive
file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0:
Could not open '/data/hdd1/libvirt/images/test.img': Permission denied
Can you please share the debug logs?
http://wiki.libvirt.org/page/DebugLogs
Also, my initial suspect, before diving any deeper is that usually, when users compile libvirt on their own, they forget to set the correct prefix, therefore libvirt is looking for its config files NOT under /etc/libvirt but /usr/local/etc/ or whatever.
BTW: is the daemon running under root?
Michal
Hi,
The daemon runs under root.
I uploaded the debug logs to:
http://people.ipfire.org/~jschlag/1363864/1_libvirtd.log
The UID of the user nobody is 99, the GID of the group kvm is 1011.
I added my configure options to the bug report.
Following the log the ownership is changed but why is the file still owned by root:root?
Right. the file is set ownership. But the file ist still owned by root:root and so it is not accessable by qemu as nobody:kvm. In the moment the only possible way is that the change of the ownership fail, but then there should be an error message, but there is no error message in the log.
Then the other option that comes to my mind is a race with somebody else on the system. You can attach gdb to the daemon and set breakpoint to virSecurityDACSetOwnershipInternal(). In the arguments you should see the path eventually among with uid:gid.
BTW: what's the domain XML?
What did you need the xml file the domain is called test and the image file is /data/hdd1/libvirt/images/test.img
Michal
I did together with Michael Tremer some debugging and Michael posted our results in the bugtracker. So it seems that the chown function is not executed, because a other function return a wrong value. Maybe the describtion in the bug report hepls to go furhter, when not say what you need (logs) to debug the problem. Regards Jonatan
On 04.08.2016 20:28, Jonatan Schlag wrote:
Then the other option that comes to my mind is a race with somebody else on the system. You can attach gdb to the daemon and set breakpoint to virSecurityDACSetOwnershipInternal(). In the arguments you should see the path eventually among with uid:gid.
BTW: what's the domain XML?
What did you need the xml file the domain is called test and the image file is /data/hdd1/libvirt/images/test.img
Well, in the domain XML. there's <seclabel/> section in the domain XML that can fine tune relabelling for a domain. Moreover, some devices - like disks have the <seclabel/> too. And I was wondering whether you don't have those elements in the XML. The other reason for me asking domain XML is so that I could try to reproduce locally on my system.
I did together with Michael Tremer some debugging and Michael posted our results in the bugtracker. So it seems that the chown function is not executed, because a other function return a wrong value.
Ah, reading the bz transcript, you are not passing the path directly into the XML rather than use a volume from a storage pool. This is supported but the previous case is more tested. Again, this would help me to narrow down the possible causes.
Maybe the describtion in the bug report hepls to go furhter, when not say what you need (logs) to debug the problem.
Maybe I'm misreading this, but I think I've told you what I need to debug the problem. Moreover, it's usually better to provide as many information as possible when debugging a problem. Even a tiny little thing that user think of as trivial may look crucial in eyes of experienced developer with insight in the project. Anyway, I'd like to continue the discussion in the BZ. Michal
Am Fr, 5. Aug, 2016 um 9:45 schrieb Michal Privoznik <mprivozn@redhat.com>:
On 04.08.2016 20:28, Jonatan Schlag wrote:
Then the other option that comes to my mind is a race with somebody else on the system. You can attach gdb to the daemon and set breakpoint to virSecurityDACSetOwnershipInternal(). In the arguments you should see the path eventually among with uid:gid.
BTW: what's the domain XML?
What did you need the xml file the domain is called test and the image file is /data/hdd1/libvirt/images/test.img
Well, in the domain XML. there's <seclabel/> section in the domain XML that can fine tune relabelling for a domain. Moreover, some devices - like disks have the <seclabel/> too. And I was wondering whether you don't have those elements in the XML.
The other reason for me asking domain XML is so that I could try to reproduce locally on my system.
I did together with Michael Tremer some debugging and Michael posted our results in the bugtracker. So it seems that the chown function is not executed, because a other function return a wrong value.
Ah, reading the bz transcript, you are not passing the path directly into the XML rather than use a volume from a storage pool. This is supported but the previous case is more tested. Again, this would help me to narrow down the possible causes.
Maybe the describtion in the bug report hepls to go furhter, when not say what you need (logs) to debug the problem.
Maybe I'm misreading this, but I think I've told you what I need to debug the problem. Moreover, it's usually better to provide as many information as possible when debugging a problem. Even a tiny little thing that user think of as trivial may look crucial in eyes of experienced developer with insight in the project.
Hi, I am sorry this sentence causes more trouble, then he helps. First, I posted the domain XML and the storage XML file in Bugzilla. I could not post this files earlier because I had no time to do this. I will post what I can to help you and I will not hide any information which could help, but at the moment I have really no idea what I can provide what can help. So the intention of this sentence was, to say that I will provide what I can to help and not that I provide only the bit what you want. Regards Jonatan
Anyway, I'd like to continue the discussion in the BZ.
Ok, I will post now everything in the bugtracker.
Michal
participants (3)
-
Jonatan Schlag -
Laine Stump -
Michal Privoznik