Re: [libvirt] Problem configuring selective dropping of root (SOLVED)

On Wed, 10 Jul 2019 15:55:04 +0200 Martin Kletzander <mkletzan(a)redhat.com&gt; wrote: > On Wed, Jul 10, 2019 at 03:44:54PM +0200, Stephan von Krawczynski wrote: > >On Wed, 10 Jul 2019 14:48:14 +0200 > >Martin Kletzander <mkletzan(a)redhat.com&gt; wrote: > > > >> On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski > >> wrote: > >> >On Tue, 9 Jul 2019 14:26:08 +0200 > >> >Pavel Hrdina <phrdina(a)redhat.com&gt; wrote: > >> > > >> >> On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski > >> >> wrote: > >> >> > On Tue, 9 Jul 2019 09:40:23 +0100 > >> >> > Daniel P. Berrangé <berrange(a)redhat.com&gt; wrote: > >> >> > > >> >> > > On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski > >> >> > > wrote: > >> >> > > > Hello list, > >> >> > > > > >> >> > > > I came across a fundamental flaw in the libvirt user > >> >> > > > configuration lately and try to find a solution now. Here is > >> >> > > > the problem: I run several qemu instances on arch linux all > >> >> > > > configured via libvirt. The default config as user nobody:kvm > >> >> > > > was fine up to the day I tried to use a host filesystem via > >> >> > > > 9p. If you want to gain all user rights on the guest inside > >> >> > > > that fs you have to run qemu as root. So far so good. But if > >> >> > > > you have several qemus running and only one needs to be root, > >> >> > > > what to do? You can try to give a -runas by using <qemu:args>. > >> >> > > > But that does not work, qemu instantly crashes. I think this > >> >> > > > is because to have _one_ root qemu, you have to configure > >> >> > > > libvirt to use root user. This means all rights to fs and so > >> >> > > > on are set to root and this is what lets qemu probably go > >> >> > > > crazy if dropping root by -runas. The whole thing would be a > >> >> > > > lot easier and more transparent if the user in libvirt > >> >> > > > wouldn't be a global config, but instead be part of the domain > >> >> > > > xml. This way every qemu started could use a different user > >> >> > > > and have different rights. In my case all but one could be > >> >> > > > nobody:kvm, and one root:root. This should not be to > >> >> > > > complicated based on whats already there, is it? > >> >> > > > >> >> > > Libvirt needs to know about the user/group QEMU is running at in > >> >> > > order to ensure it gets given access to the various files it > >> >> > > needs to use. If you look at the XML of the running guest you > >> >> > > should see a <seclabel> describing the user/group it is running > >> >> > > as currently. > >> >> > > > >> >> > > If no <seclabel> is in the offline config, libvirt adds the > >> >> > > default seclabel, but if you want a different user/group, you > >> >> > > can add the <seclabel> yourself. > >> >> > > > >> >> > > Regards, > >> >> > > Daniel > >> >> > > >> >> > Hello Daniel, > >> >> > > >> >> > well, tried that (as good as the docs are) by adding: > >> >> > > >> >> > <seclabel type='dynamic' model='dac'> > >> >> > <label>nobody:kvm</label> > >> >> > </seclabel> > >> >> > > >> >> > This edit worked in virsh without giving errors. > >> >> > Starting the domain and then looking into the xml showed: > >> >> > > >> >> > <seclabel type='dynamic' model='dac' relabel='yes'/> > >> >> > > >> >> > Consequently qemu runs still as root. My user:group setting simply > >> >> > vanished. > >> >> > > >> >> > I think at least some better docs are needed with a striking > >> >> > example of how to change user and group ... > >> >> > I may be biased, but how to set user and group is probably the > >> >> > most basic example of how to use seclabel - and I cannot find > >> >> > one. > >> >> > >> >> I agree that the documentation is not the best one. > >> >> > >> >> You need to use type='static' relabel='yes': > >> >> > >> >> <seclabel type='static' model='dac' relabel='yes'> > >> >> <label>nobody:kvm</label> > >> >> </seclabel> > >> >> > >> >> To achieve that. > >> >> > >> >> In addition if you would like to have only one VM as root:root you > >> >> should keep the default config as nobody:kvm and use the root:root > >> >> for that specific VM. > >> >> > >> >> Pavel > >> > > >> >Hello Pavel, > >> > > >> >thank you for taking up the thread, but unfortunately your suggestion > >> >does not work: > >> > > >> >virsh # start collabora > >> >Fehler: Domain collabora konnte nicht gestartet werden > >> >Fehler: Interner Fehler: process exited while connecting to monitor: > >> >2019-07-09T12:34:00.735392Z qemu-system-x86_64: -object > >> >secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: > >> >Unable to > >> >read /var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Failed > >> >to open file > >> >“/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes”: Permission > >> >denied > >> > > >> >Obviously this is because "type='static'" means that libvirt does not > >> >care about setting the user rights for qemu, which then leads to > >> >this. > >> > >> No, 'static' means you tell libvirt what the label should be, 'dynamic' > >> means libvirt will generate it automatically. > >> > >> >I did think "relabel='yes'" should do that, but does not - or I have a > >> >deep misunderstanding concerning the seclabel parameters ... > >> >Thanks for any help to solve this, if there is no bug involved. > >> > > >> > >> Relabel definitely does that and unless there is a bug it uses the > >> seclabel for the domain. What could be happening is that one of the > >> parent directories is missing a browse permission for others (the last > >> 'x' in rwxrwxrwx). Could you run the following commands and reply with > >> the output? > >> > >> ls -ld /var/lib/ > >> ls -ld /var/lib/libvirt/ > >> ls -ld /var/lib/libvirt/qemu/ > >> > >> Also what distribution are you using? I remember there were some > >> differences in packaging related to the directory permissions. > >> > >> Martin > >> > >> >Dumpxml shows this btw: > >> > > >> > <seclabel type='static' model='dac' relabel='yes'> > >> > <label>nobody:kvm</label> > >> > </seclabel> > >> > > >> >which at least is what was configured. > >> >-- > >> >Regards, > >> >Stephan > > > >Hello Martin, > > > >thanks for picking up. Here is the output you requested: > > > >[root@vserver-a01 /]# ls -ld /var/lib/ > >drwxr-xr-x 21 root root 4096 10. Jul 00:00 /var/lib/ > >[root@vserver-a01 /]# ls -ld /var/lib/libvirt/ > >drwxr-xr-x 11 root root 4096 4. Jul 10:08 /var/lib/libvirt/ > >[root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ > >drwxrwx--- 11 root root 4096 10. Jul 15:36 /var/lib/libvirt/qemu/ > > > >It seems your guess is right. After > > > >[root@vserver-a01 /]# chmod a+x /var/lib/libvirt/qemu/ > >[root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ > >drwxrwx--x 11 root root 4096 10. Jul 15:41 /var/lib/libvirt/qemu/ > > > >it started: > > > >virsh # start collabora > >Domain collabora gestartet > > > >So that's a bug in the Arch Linux packaging? > >Who to tell? > > > > Our Makefile specifies what to do on installation: > > $(MKDIR_P) -m 0751 "$(DESTDIR)$(localstatedir)/lib/libvirt/qemu" > > so I guess this is a packaging issue. No idea where/how the arch packaging > works, sorry. > > >Thank you Martin, Pavel and Daniel for tracking that down. > > > >-- > >Regards, > >Stephan Hello Martin, I will take care of that for arch. Topic closed. -- Regards, Stephan