[libvirt-users] libvirtd via unix socket using system uri
I want to run libvirtd as a special user, and allowing users that belong to a special group to connect via qemu+unix:///system (eg: unix socket). I did everything necessary to do so: created a libvirt user and group, added the libvirt user to the kvm group, added my normal user to the libvirt group, and made sure the socket is owned by libvirt:libvirt with permissions set to 770. libvirtd starts successfully, but when I try to connect as the normal user I get this error: bash$ virsh --connect qemu+unix://system error: failed to connect to the hypervisor error: invalid argument: using unix socket and remote server 'system' is not supported. A trace shows virsh is not even trying to open the socket. I want to use the socket because I just need local connectivity and don't want to run sasl and set up certificates for this, but at the same time want to run libvirtd as a dedicated user. Is there any reason to prevent libvirt from being used like this?
On 4/29/19 1:06 PM, lameventanas@gmail.com wrote:
I want to run libvirtd as a special user, and allowing users that belong to a special group to connect via qemu+unix:///system (eg: unix socket).
I did everything necessary to do so: created a libvirt user and group, added the libvirt user to the kvm group, added my normal user to the libvirt group, and made sure the socket is owned by libvirt:libvirt with permissions set to 770.
libvirtd starts successfully, but when I try to connect as the normal user I get this error:
bash$ virsh --connect qemu+unix://system error: failed to connect to the hypervisor error: invalid argument: using unix socket and remote server 'system' is not supported.
This is not valid URI. https://libvirt.org/uri.html You may want to use 'qemu+unix:///system' or simply 'qemu:///system' which is the same because libvirt connects via unix socket by default. Anyway, you'll need to run libvirtd privileged, otherwise it'll get EPERM when trying to do almost anything. Michal
On 29/04/2019 22.01, Michal Privoznik wrote:
On 4/29/19 1:06 PM, lameventanas@gmail.com wrote:
I want to run libvirtd as a special user, and allowing users that belong to a special group to connect via qemu+unix:///system (eg: unix socket).
I did everything necessary to do so: created a libvirt user and group, added the libvirt user to the kvm group, added my normal user to the libvirt group, and made sure the socket is owned by libvirt:libvirt with permissions set to 770.
libvirtd starts successfully, but when I try to connect as the normal user I get this error:
bash$ virsh --connect qemu+unix://system error: failed to connect to the hypervisor error: invalid argument: using unix socket and remote server 'system' is not supported.
This is not valid URI.
You may want to use 'qemu+unix:///system' or simply 'qemu:///system' which is the same because libvirt connects via unix socket by default.
Thanks, I missed that. Now I get this: $ virsh --connect qemu:///system error: failed to connect to the hypervisor error: internal error: unexpected QEMU URI path '/system', try qemu:///session But now I see virsh opening the socket, writing, etc. The server log shows this: 2019-04-29 15:32:07.306+0000: 20863: info : libvirt version: 5.2.0 2019-04-29 15:32:07.306+0000: 20863: info : hostname: koji 2019-04-29 15:32:07.306+0000: 20863: error : virDBusGetSessionBus:169 : internal error: Unable to get DBus session bus connection: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11 2019-04-29 15:32:24.534+0000: 20852: error : qemuConnectOpen:1127 : internal error: unexpected QEMU URI path '/system', try qemu:///session 2019-04-29 15:32:24.534+0000: 20851: error : virNetSocketReadWire:1803 : End of file while reading data: Input/output error I also wonder if dbus is important, and if so, how to fix it.
Anyway, you'll need to run libvirtd privileged, otherwise it'll get EPERM when trying to do almost anything.
I'm aware of that, I want to try and see the limitations, and maybe get around them by using capabilities?
On Tue, Apr 30, 2019 at 12:42:11AM +0900, lameventanas@gmail.com wrote:
On 29/04/2019 22.01, Michal Privoznik wrote:
On 4/29/19 1:06 PM, lameventanas@gmail.com wrote:
I want to run libvirtd as a special user, and allowing users that belong to a special group to connect via qemu+unix:///system (eg: unix socket).
I did everything necessary to do so: created a libvirt user and group, added the libvirt user to the kvm group, added my normal user to the libvirt group, and made sure the socket is owned by libvirt:libvirt with permissions set to 770.
libvirtd starts successfully, but when I try to connect as the normal user I get this error:
bash$ virsh --connect qemu+unix://system error: failed to connect to the hypervisor error: invalid argument: using unix socket and remote server 'system' is not supported.
This is not valid URI.
You may want to use 'qemu+unix:///system' or simply 'qemu:///system' which is the same because libvirt connects via unix socket by default.
Thanks, I missed that.
Now I get this:
$ virsh --connect qemu:///system error: failed to connect to the hypervisor error: internal error: unexpected QEMU URI path '/system', try qemu:///session
But now I see virsh opening the socket, writing, etc.
The server log shows this:
2019-04-29 15:32:07.306+0000: 20863: info : libvirt version: 5.2.0 2019-04-29 15:32:07.306+0000: 20863: info : hostname: koji 2019-04-29 15:32:07.306+0000: 20863: error : virDBusGetSessionBus:169 : internal error: Unable to get DBus session bus connection: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11 2019-04-29 15:32:24.534+0000: 20852: error : qemuConnectOpen:1127 : internal error: unexpected QEMU URI path '/system', try qemu:///session 2019-04-29 15:32:24.534+0000: 20851: error : virNetSocketReadWire:1803 : End of file while reading data: Input/output error
I also wonder if dbus is important, and if so, how to fix it.
Anyway, you'll need to run libvirtd privileged, otherwise it'll get EPERM when trying to do almost anything.
I'm aware of that, I want to try and see the limitations, and maybe get around them by using capabilities?
Libvirt has two distinct modes of operation - system mode and session mode. When uid == 0, libvirtd runs in system mode & offers full privileges. When uid != 0 it runs in session mode & disables alot of functionality, or it will simply fail to use it. It is impossible to run system mode as uid != 0, as it will always activate session mode in this case. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 4/29/19 5:42 PM, lameventanas@gmail.com wrote:
On 29/04/2019 22.01, Michal Privoznik wrote:
On 4/29/19 1:06 PM, lameventanas@gmail.com wrote:
I want to run libvirtd as a special user, and allowing users that belong to a special group to connect via qemu+unix:///system (eg: unix socket).
I did everything necessary to do so: created a libvirt user and group, added the libvirt user to the kvm group, added my normal user to the libvirt group, and made sure the socket is owned by libvirt:libvirt with permissions set to 770.
libvirtd starts successfully, but when I try to connect as the normal user I get this error:
bash$ virsh --connect qemu+unix://system error: failed to connect to the hypervisor error: invalid argument: using unix socket and remote server 'system' is not supported.
This is not valid URI.
You may want to use 'qemu+unix:///system' or simply 'qemu:///system' which is the same because libvirt connects via unix socket by default.
Thanks, I missed that.
Now I get this:
$ virsh --connect qemu:///system error: failed to connect to the hypervisor error: internal error: unexpected QEMU URI path '/system', try qemu:///session
But now I see virsh opening the socket, writing, etc.
The server log shows this:
2019-04-29 15:32:07.306+0000: 20863: info : libvirt version: 5.2.0 2019-04-29 15:32:07.306+0000: 20863: info : hostname: koji 2019-04-29 15:32:07.306+0000: 20863: error : virDBusGetSessionBus:169 : internal error: Unable to get DBus session bus connection: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11 2019-04-29 15:32:24.534+0000: 20852: error : qemuConnectOpen:1127 : internal error: unexpected QEMU URI path '/system', try qemu:///session 2019-04-29 15:32:24.534+0000: 20851: error : virNetSocketReadWire:1803 : End of file while reading data: Input/output error
I also wonder if dbus is important, and if so, how to fix it.
This happened because qemu driver for /system doesn't initialize unless libvirtd's EUID == 0.
Anyway, you'll need to run libvirtd privileged, otherwise it'll get EPERM when trying to do almost anything.
I'm aware of that, I want to try and see the limitations, and maybe get around them by using capabilities?
Is there any problem running libvirtd as root? Michal
Is there any problem running libvirtd as root?
Yes, in the regulated environment in which I work! I have to do far more
On Tue, 30 Apr 2019 at 10:40, Michal Privoznik <mprivozn@redhat.com> wrote: thorough threat analysis than I would do if I knew which capabilities it had. So far, we've accepted the extra work; but it would be wonderful to be able to run a locked-down virtualisation environment. Cheers, - Peter
On Tue, Apr 30, 2019 at 10:45:03AM +0100, Peter Crowther wrote:
On Tue, 30 Apr 2019 at 10:40, Michal Privoznik <mprivozn@redhat.com> wrote:
Is there any problem running libvirtd as root?
Yes, in the regulated environment in which I work! I have to do far more thorough threat analysis than I would do if I knew which capabilities it had. So far, we've accepted the extra work; but it would be wonderful to be able to run a locked-down virtualisation environment.
Libvirtd system mode will want cap_net_admin in order to setup TAP devices and cap_sys_admin to manage disk permissions to grant QEMU access, at which point you've lost any security benefit of running it unprivileged with selective capabilities. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Tue, Apr 30, 2019 at 10:48:51AM +0100, Daniel P. Berrangé wrote:
On Tue, Apr 30, 2019 at 10:45:03AM +0100, Peter Crowther wrote:
On Tue, 30 Apr 2019 at 10:40, Michal Privoznik <mprivozn@redhat.com> wrote:
Is there any problem running libvirtd as root?
Yes, in the regulated environment in which I work! I have to do far more thorough threat analysis than I would do if I knew which capabilities it had. So far, we've accepted the extra work; but it would be wonderful to be able to run a locked-down virtualisation environment.
Libvirtd system mode will want cap_net_admin in order to setup TAP devices and cap_sys_admin to manage disk permissions to grant QEMU access, at which point you've lost any security benefit of running it unprivileged with selective capabilities.
Opps, I mean cap_dac_override not cap_sys_admin but the point still stands. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Tue, Apr 30, 2019 at 10:45:03AM +0100, Peter Crowther wrote:
On Tue, 30 Apr 2019 at 10:40, Michal Privoznik <mprivozn@redhat.com> wrote:
Is there any problem running libvirtd as root?
Yes, in the regulated environment in which I work! I have to do far more thorough threat analysis than I would do if I knew which capabilities it had. So far, we've accepted the extra work; but it would be wonderful to be able to run a locked-down virtualisation environment.
Libvirtd system mode will want cap_net_admin in order to setup TAP devices and cap_sys_admin to manage disk permissions to grant QEMU access, at which point you've lost any security benefit of running it unprivileged with selective capabilities.
Would it fail hard without these, even if using (for example) pre-created Ceph block storage, which is our use case? Or would it only fail when it
On Tue, 30 Apr 2019 at 10:48, Daniel P. Berrangé <berrange@redhat.com> wrote: tried to make use of a capability that wasn't present? My reading of capabilities is that behaviour is indistinguishable until you get an EPERM? I agree that CAP_DAC_OVERRIDE (per your later mail) is game over for any system, as that allows write of any config file. It'd be lovely to find a way of not requiring that. Knowing that a piece of software can't maliciously insert kernel modules, can't write or clear audit trails, and can't do raw I/O already considerably reduces the analysis. - Peter
On 4/30/19 3:15 PM, Peter Crowther wrote:
On Tue, 30 Apr 2019 at 10:48, Daniel P. Berrangé <berrange@redhat.com> wrote:
On Tue, Apr 30, 2019 at 10:45:03AM +0100, Peter Crowther wrote:
On Tue, 30 Apr 2019 at 10:40, Michal Privoznik <mprivozn@redhat.com> wrote:
Is there any problem running libvirtd as root?
Yes, in the regulated environment in which I work! I have to do far more thorough threat analysis than I would do if I knew which capabilities it had. So far, we've accepted the extra work; but it would be wonderful to be able to run a locked-down virtualisation environment.
Libvirtd system mode will want cap_net_admin in order to setup TAP devices and cap_sys_admin to manage disk permissions to grant QEMU access, at which point you've lost any security benefit of running it unprivileged with selective capabilities.
Would it fail hard without these, even if using (for example) pre-created Ceph block storage, which is our use case? Or would it only fail when it tried to make use of a capability that wasn't present? My reading of capabilities is that behaviour is indistinguishable until you get an EPERM?
I agree that CAP_DAC_OVERRIDE (per your later mail) is game over for any
CAP_DAC_OVERRIDE won't be required if you don't need libvirt to chown()/setfilecon() disk images (dynamic_ownership in qemu.conf). CAP_SYS_ADMIN is going to be required if you want libvirt to mount some nfs based storage pools/create namespaces (note that libvirt creates a small namespace for qemu to run in - might need CAP_MKNOD then). Long story short, why bother with /system if you can't use it and not use /session instead? Michal
On Tue, 30 Apr 2019 at 16:43, Michal Privoznik <mprivozn@redhat.com> wrote:
Long story short, why bother with /system if you can't use it and not use /session instead?
Because according to the FAQ, /session isn't suitable for my use:
- You will definitely want to use qemu:///system if your VMs are acting as servers. VM autostart on host boot only works for 'system' [Yes, my VMs are acting as servers] - the root libvirtd instance has necessary permissions to use proper networkings via bridges or virtual networks. [Yes, I use OVS, with quite a complex bridge+VLAN system configured at boot] - qemu:///session has a serious drawback: [...] the only out of the box network option is qemu's usermode networking, which has nonobvious limitations, so its usage is discouraged. (Source: https://wiki.libvirt.org/page/FAQ#What_is_the_difference_between_qemu:.2F.2F... ) So I have to use /system, according to the FAQ. But it'd be nice to nail the daemon down to reduce the attack surface. - Peter
participants (4)
-
Daniel P. Berrangé -
lameventanas＠gmail.com -
Michal Privoznik -
Peter Crowther