6:06 a.m.
I want to run libvirtd as a special user, and allowing users that belong
to a special group to connect via qemu+unix:///system (eg: unix socket).
I did everything necessary to do so: created a libvirt user and group,
added the libvirt user to the kvm group, added my normal user to the
libvirt group, and made sure the socket is owned by libvirt:libvirt with
permissions set to 770.
libvirtd starts successfully, but when I try to connect as the normal
user I get this error:
bash$ virsh --connect qemu+unix://system
error: failed to connect to the hypervisor
error: invalid argument: using unix socket and remote server 'system' is
not supported.
A trace shows virsh is not even trying to open the socket.
I want to use the socket because I just need local connectivity and
don't want to run sasl and set up certificates for this, but at the same
time want to run libvirtd as a dedicated user.
Is there any reason to prevent libvirt from being used like this?
8:01 a.m.
On 4/29/19 1:06 PM, lameventanas(a)gmail.com wrote:
I want to run libvirtd as a special user, and allowing users that
belong
to a special group to connect via qemu+unix:///system (eg: unix socket).
I did everything necessary to do so: created a libvirt user and group,
added the libvirt user to the kvm group, added my normal user to the
libvirt group, and made sure the socket is owned by libvirt:libvirt with
permissions set to 770.
libvirtd starts successfully, but when I try to connect as the normal
user I get this error:
bash$ virsh --connect qemu+unix://system
error: failed to connect to the hypervisor
error: invalid argument: using unix socket and remote server 'system' is
not supported.
This is not valid URI.
https://libvirt.org/uri.html
You may want to use 'qemu+unix:///system' or simply 'qemu:///system'
which is the same because libvirt connects via unix socket by default.
Anyway, you'll need to run libvirtd privileged, otherwise it'll get
EPERM when trying to do almost anything.
Michal
10:42 a.m.
On 29/04/2019 22.01, Michal Privoznik wrote:
On 4/29/19 1:06 PM, lameventanas(a)gmail.com wrote:
> I want to run libvirtd as a special user, and allowing users that belong
> to a special group to connect via qemu+unix:///system (eg: unix socket).
>
> I did everything necessary to do so: created a libvirt user and group,
> added the libvirt user to the kvm group, added my normal user to the
> libvirt group, and made sure the socket is owned by libvirt:libvirt with
> permissions set to 770.
>
> libvirtd starts successfully, but when I try to connect as the normal
> user I get this error:
>
> bash$ virsh --connect qemu+unix://system
> error: failed to connect to the hypervisor
> error: invalid argument: using unix socket and remote server 'system' is
> not supported.
This is not valid URI.
https://libvirt.org/uri.html
You may want to use 'qemu+unix:///system' or simply 'qemu:///system'
which is the same because libvirt connects via unix socket by default.
Thanks, I missed that.
Now I get this:
$ virsh --connect qemu:///system
error: failed to connect to the hypervisor
error: internal error: unexpected QEMU URI path '/system', try
qemu:///session
But now I see virsh opening the socket, writing, etc.
The server log shows this:
2019-04-29 15:32:07.306+0000: 20863: info : libvirt version: 5.2.0
2019-04-29 15:32:07.306+0000: 20863: info : hostname: koji
2019-04-29 15:32:07.306+0000: 20863: error : virDBusGetSessionBus:169 :
internal error: Unable to get DBus session bus connection: Unable to
autolaunch a dbus-daemon without a $DISPLAY for X11
2019-04-29 15:32:24.534+0000: 20852: error : qemuConnectOpen:1127 :
internal error: unexpected QEMU URI path '/system', try qemu:///session
2019-04-29 15:32:24.534+0000: 20851: error : virNetSocketReadWire:1803 :
End of file while reading data: Input/output error
I also wonder if dbus is important, and if so, how to fix it.
Anyway, you'll need to run libvirtd privileged, otherwise
it'll get
EPERM when trying to do almost anything.
I'm aware of that, I want to try and see the limitations, and maybe get
around them by using capabilities?
4:38 a.m.
On Tue, Apr 30, 2019 at 12:42:11AM +0900, lameventanas(a)gmail.com wrote:
On 29/04/2019 22.01, Michal Privoznik wrote:
> On 4/29/19 1:06 PM, lameventanas(a)gmail.com wrote:
>> I want to run libvirtd as a special user, and allowing users that belong
>> to a special group to connect via qemu+unix:///system (eg: unix socket).
>>
>> I did everything necessary to do so: created a libvirt user and group,
>> added the libvirt user to the kvm group, added my normal user to the
>> libvirt group, and made sure the socket is owned by libvirt:libvirt with
>> permissions set to 770.
>>
>> libvirtd starts successfully, but when I try to connect as the normal
>> user I get this error:
>>
>> bash$ virsh --connect qemu+unix://system
>> error: failed to connect to the hypervisor
>> error: invalid argument: using unix socket and remote server 'system'
is
>> not supported.
>
> This is not valid URI.
>
> https://libvirt.org/uri.html
>
> You may want to use 'qemu+unix:///system' or simply
'qemu:///system'
> which is the same because libvirt connects via unix socket by default.
Thanks, I missed that.
Now I get this:
$ virsh --connect qemu:///system
error: failed to connect to the hypervisor
error: internal error: unexpected QEMU URI path '/system', try
qemu:///session
But now I see virsh opening the socket, writing, etc.
The server log shows this:
2019-04-29 15:32:07.306+0000: 20863: info : libvirt version: 5.2.0
2019-04-29 15:32:07.306+0000: 20863: info : hostname: koji
2019-04-29 15:32:07.306+0000: 20863: error : virDBusGetSessionBus:169 :
internal error: Unable to get DBus session bus connection: Unable to
autolaunch a dbus-daemon without a $DISPLAY for X11
2019-04-29 15:32:24.534+0000: 20852: error : qemuConnectOpen:1127 :
internal error: unexpected QEMU URI path '/system', try qemu:///session
2019-04-29 15:32:24.534+0000: 20851: error : virNetSocketReadWire:1803 :
End of file while reading data: Input/output error
I also wonder if dbus is important, and if so, how to fix it.
> Anyway, you'll need to run libvirtd privileged, otherwise it'll get
> EPERM when trying to do almost anything.
I'm aware of that, I want to try and see the limitations, and maybe get
around them by using capabilities?
Libvirt has two distinct modes of operation - system mode and session
mode. When uid == 0, libvirtd runs in system mode & offers full
privileges. When uid != 0 it runs in session mode & disables alot of
functionality, or it will simply fail to use it.
It is impossible to run system mode as uid != 0, as it will always
activate session mode in this case.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
4:40 a.m.
On 4/29/19 5:42 PM, lameventanas(a)gmail.com wrote:
On 29/04/2019 22.01, Michal Privoznik wrote:
> On 4/29/19 1:06 PM, lameventanas(a)gmail.com wrote:
>> I want to run libvirtd as a special user, and allowing users that belong
>> to a special group to connect via qemu+unix:///system (eg: unix socket).
>>
>> I did everything necessary to do so: created a libvirt user and group,
>> added the libvirt user to the kvm group, added my normal user to the
>> libvirt group, and made sure the socket is owned by libvirt:libvirt with
>> permissions set to 770.
>>
>> libvirtd starts successfully, but when I try to connect as the normal
>> user I get this error:
>>
>> bash$ virsh --connect qemu+unix://system
>> error: failed to connect to the hypervisor
>> error: invalid argument: using unix socket and remote server 'system' is
>> not supported.
>
> This is not valid URI.
>
> https://libvirt.org/uri.html
>
> You may want to use 'qemu+unix:///system' or simply 'qemu:///system'
> which is the same because libvirt connects via unix socket by default.
Thanks, I missed that.
Now I get this:
$ virsh --connect qemu:///system
error: failed to connect to the hypervisor
error: internal error: unexpected QEMU URI path '/system', try
qemu:///session
But now I see virsh opening the socket, writing, etc.
The server log shows this:
2019-04-29 15:32:07.306+0000: 20863: info : libvirt version: 5.2.0
2019-04-29 15:32:07.306+0000: 20863: info : hostname: koji
2019-04-29 15:32:07.306+0000: 20863: error : virDBusGetSessionBus:169 :
internal error: Unable to get DBus session bus connection: Unable to
autolaunch a dbus-daemon without a $DISPLAY for X11
2019-04-29 15:32:24.534+0000: 20852: error : qemuConnectOpen:1127 :
internal error: unexpected QEMU URI path '/system', try qemu:///session
2019-04-29 15:32:24.534+0000: 20851: error : virNetSocketReadWire:1803 :
End of file while reading data: Input/output error
I also wonder if dbus is important, and if so, how to fix it.
This happened because qemu driver for /system doesn't initialize unless
libvirtd's EUID == 0.
> Anyway, you'll need to run libvirtd privileged, otherwise it'll get
> EPERM when trying to do almost anything.
I'm aware of that, I want to try and see the limitations, and maybe get
around them by using capabilities?
Is there any problem running libvirtd as root?
Michal
4:45 a.m.
On Tue, 30 Apr 2019 at 10:40, Michal Privoznik <mprivozn(a)redhat.com> wrote:
Is there any problem running libvirtd as root?
Yes, in the regulated environment in which I work! I have to do far more
thorough
threat analysis than I would do if I knew which capabilities it
had. So far, we've accepted the extra work; but it would be wonderful to
be able to run a locked-down virtualisation environment.
Cheers,
- Peter
4:48 a.m.
On Tue, Apr 30, 2019 at 10:45:03AM +0100, Peter Crowther wrote:
On Tue, 30 Apr 2019 at 10:40, Michal Privoznik
<mprivozn(a)redhat.com> wrote:
> Is there any problem running libvirtd as root?
>
> Yes, in the regulated environment in which I work! I have to do far more
thorough threat analysis than I would do if I knew which capabilities it
had. So far, we've accepted the extra work; but it would be wonderful to
be able to run a locked-down virtualisation environment.
Libvirtd system mode will want cap_net_admin in order to setup TAP devices
and cap_sys_admin to manage disk permissions to grant QEMU access, at which
point you've lost any security benefit of running it unprivileged with
selective capabilities.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
4:50 a.m.
On Tue, Apr 30, 2019 at 10:48:51AM +0100, Daniel P. Berrangé wrote:
On Tue, Apr 30, 2019 at 10:45:03AM +0100, Peter Crowther wrote:
> On Tue, 30 Apr 2019 at 10:40, Michal Privoznik <mprivozn(a)redhat.com> wrote:
>
> > Is there any problem running libvirtd as root?
> >
> > Yes, in the regulated environment in which I work! I have to do far more
> thorough threat analysis than I would do if I knew which capabilities it
> had. So far, we've accepted the extra work; but it would be wonderful to
> be able to run a locked-down virtualisation environment.
Libvirtd system mode will want cap_net_admin in order to setup TAP devices
and cap_sys_admin to manage disk permissions to grant QEMU access, at which
point you've lost any security benefit of running it unprivileged with
selective capabilities.
Opps, I mean cap_dac_override not cap_sys_admin but the point still stands.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
8:15 a.m.
On Tue, 30 Apr 2019 at 10:48, Daniel P. Berrangé <berrange(a)redhat.com>
wrote:
On Tue, Apr 30, 2019 at 10:45:03AM +0100, Peter Crowther wrote:
> On Tue, 30 Apr 2019 at 10:40, Michal Privoznik <mprivozn(a)redhat.com>
wrote:
>
> > Is there any problem running libvirtd as root?
> >
> > Yes, in the regulated environment in which I work! I have to do far
more
> thorough threat analysis than I would do if I knew which capabilities it
> had. So far, we've accepted the extra work; but it would be wonderful to
> be able to run a locked-down virtualisation environment.
Libvirtd system mode will want cap_net_admin in order to setup TAP devices
and cap_sys_admin to manage disk permissions to grant QEMU access, at which
point you've lost any security benefit of running it unprivileged with
selective capabilities.
Would it fail hard without these, even if using (for example) pre-created
Ceph
block storage, which is our use case? Or would it only fail when it
tried to make use of a capability that wasn't present? My reading of
capabilities is that behaviour is indistinguishable until you get an EPERM?
I agree that CAP_DAC_OVERRIDE (per your later mail) is game over for any
system, as that allows write of any config file. It'd be lovely to find a
way of not requiring that. Knowing that a piece of software can't
maliciously insert kernel modules, can't write or clear audit trails, and
can't do raw I/O already considerably reduces the analysis.
- Peter
10:43 a.m.
On 4/30/19 3:15 PM, Peter Crowther wrote:
On Tue, 30 Apr 2019 at 10:48, Daniel P. Berrangé
<berrange(a)redhat.com>
wrote:
> On Tue, Apr 30, 2019 at 10:45:03AM +0100, Peter Crowther wrote:
>> On Tue, 30 Apr 2019 at 10:40, Michal Privoznik <mprivozn(a)redhat.com>
> wrote:
>>
>>> Is there any problem running libvirtd as root?
>>>
>>> Yes, in the regulated environment in which I work! I have to do far
> more
>> thorough threat analysis than I would do if I knew which capabilities it
>> had. So far, we've accepted the extra work; but it would be wonderful to
>> be able to run a locked-down virtualisation environment.
>
> Libvirtd system mode will want cap_net_admin in order to setup TAP devices
> and cap_sys_admin to manage disk permissions to grant QEMU access, at which
> point you've lost any security benefit of running it unprivileged with
> selective capabilities.
>
> Would it fail hard without these, even if using (for example) pre-created
Ceph block storage, which is our use case? Or would it only fail when it
tried to make use of a capability that wasn't present? My reading of
capabilities is that behaviour is indistinguishable until you get an EPERM?
I agree that CAP_DAC_OVERRIDE (per your later mail) is game over for any
CAP_DAC_OVERRIDE won't be required if you don't need libvirt to
chown()/setfilecon() disk images (dynamic_ownership in qemu.conf).
CAP_SYS_ADMIN is going to be required if you want libvirt to mount some
nfs based storage pools/create namespaces (note that libvirt creates a
small namespace for qemu to run in - might need CAP_MKNOD then).
Long story short, why bother with /system if you can't use it and not
use /session instead?
Michal
10:53 a.m.
On Tue, 30 Apr 2019 at 16:43, Michal Privoznik <mprivozn(a)redhat.com> wrote:
Long story short, why bother with /system if you can't use it and
not
use /session instead?
Because according to the FAQ, /session isn't suitable for my use:
- You will definitely want to use qemu:///system if your VMs are acting
as servers. VM autostart on host boot only works for 'system' [Yes, my VMs
are acting as servers]
- the root libvirtd instance has necessary permissions to use proper
networkings via bridges or virtual networks. [Yes, I use OVS, with quite a
complex bridge+VLAN system configured at boot]
- qemu:///session has a serious drawback: [...] the only out of the box
network option is qemu's usermode networking, which has nonobvious
limitations, so its usage is discouraged.
(Source:
https://wiki.libvirt.org/page/FAQ#What_is_the_difference_between_qemu:.2F...
)
So I have to use /system, according to the FAQ. But it'd be nice to nail
the daemon down to reduce the attack surface.
- Peter
1914
days inactive
1915
days old
10 comments
4 participants
participants (4)
-
Daniel P. Berrangé -
lameventanas＠gmail.com -
Michal Privoznik -
Peter Crowther