[libvirt] Problem configuring selective dropping of root
Hello list, I came across a fundamental flaw in the libvirt user configuration lately and try to find a solution now. Here is the problem: I run several qemu instances on arch linux all configured via libvirt. The default config as user nobody:kvm was fine up to the day I tried to use a host filesystem via 9p. If you want to gain all user rights on the guest inside that fs you have to run qemu as root. So far so good. But if you have several qemus running and only one needs to be root, what to do? You can try to give a -runas by using <qemu:args>. But that does not work, qemu instantly crashes. I think this is because to have _one_ root qemu, you have to configure libvirt to use root user. This means all rights to fs and so on are set to root and this is what lets qemu probably go crazy if dropping root by -runas. The whole thing would be a lot easier and more transparent if the user in libvirt wouldn't be a global config, but instead be part of the domain xml. This way every qemu started could use a different user and have different rights. In my case all but one could be nobody:kvm, and one root:root. This should not be to complicated based on whats already there, is it? -- Regards, Stephan
On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski wrote:
Hello list,
I came across a fundamental flaw in the libvirt user configuration lately and try to find a solution now. Here is the problem: I run several qemu instances on arch linux all configured via libvirt. The default config as user nobody:kvm was fine up to the day I tried to use a host filesystem via 9p. If you want to gain all user rights on the guest inside that fs you have to run qemu as root. So far so good. But if you have several qemus running and only one needs to be root, what to do? You can try to give a -runas by using <qemu:args>. But that does not work, qemu instantly crashes. I think this is because to have _one_ root qemu, you have to configure libvirt to use root user. This means all rights to fs and so on are set to root and this is what lets qemu probably go crazy if dropping root by -runas. The whole thing would be a lot easier and more transparent if the user in libvirt wouldn't be a global config, but instead be part of the domain xml. This way every qemu started could use a different user and have different rights. In my case all but one could be nobody:kvm, and one root:root. This should not be to complicated based on whats already there, is it?
Libvirt needs to know about the user/group QEMU is running at in order to ensure it gets given access to the various files it needs to use. If you look at the XML of the running guest you should see a <seclabel> describing the user/group it is running as currently. If no <seclabel> is in the offline config, libvirt adds the default seclabel, but if you want a different user/group, you can add the <seclabel> yourself. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Tue, 9 Jul 2019 09:40:23 +0100 Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski wrote:
Hello list,
I came across a fundamental flaw in the libvirt user configuration lately and try to find a solution now. Here is the problem: I run several qemu instances on arch linux all configured via libvirt. The default config as user nobody:kvm was fine up to the day I tried to use a host filesystem via 9p. If you want to gain all user rights on the guest inside that fs you have to run qemu as root. So far so good. But if you have several qemus running and only one needs to be root, what to do? You can try to give a -runas by using <qemu:args>. But that does not work, qemu instantly crashes. I think this is because to have _one_ root qemu, you have to configure libvirt to use root user. This means all rights to fs and so on are set to root and this is what lets qemu probably go crazy if dropping root by -runas. The whole thing would be a lot easier and more transparent if the user in libvirt wouldn't be a global config, but instead be part of the domain xml. This way every qemu started could use a different user and have different rights. In my case all but one could be nobody:kvm, and one root:root. This should not be to complicated based on whats already there, is it?
Libvirt needs to know about the user/group QEMU is running at in order to ensure it gets given access to the various files it needs to use. If you look at the XML of the running guest you should see a <seclabel> describing the user/group it is running as currently.
If no <seclabel> is in the offline config, libvirt adds the default seclabel, but if you want a different user/group, you can add the <seclabel> yourself.
Regards, Daniel
Hello Daniel, well, tried that (as good as the docs are) by adding: <seclabel type='dynamic' model='dac'> <label>nobody:kvm</label> </seclabel> This edit worked in virsh without giving errors. Starting the domain and then looking into the xml showed: <seclabel type='dynamic' model='dac' relabel='yes'/> Consequently qemu runs still as root. My user:group setting simply vanished. I think at least some better docs are needed with a striking example of how to change user and group ... I may be biased, but how to set user and group is probably the most basic example of how to use seclabel - and I cannot find one. -- Regards, Stephan
On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 09:40:23 +0100 Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski wrote:
Hello list,
I came across a fundamental flaw in the libvirt user configuration lately and try to find a solution now. Here is the problem: I run several qemu instances on arch linux all configured via libvirt. The default config as user nobody:kvm was fine up to the day I tried to use a host filesystem via 9p. If you want to gain all user rights on the guest inside that fs you have to run qemu as root. So far so good. But if you have several qemus running and only one needs to be root, what to do? You can try to give a -runas by using <qemu:args>. But that does not work, qemu instantly crashes. I think this is because to have _one_ root qemu, you have to configure libvirt to use root user. This means all rights to fs and so on are set to root and this is what lets qemu probably go crazy if dropping root by -runas. The whole thing would be a lot easier and more transparent if the user in libvirt wouldn't be a global config, but instead be part of the domain xml. This way every qemu started could use a different user and have different rights. In my case all but one could be nobody:kvm, and one root:root. This should not be to complicated based on whats already there, is it?
Libvirt needs to know about the user/group QEMU is running at in order to ensure it gets given access to the various files it needs to use. If you look at the XML of the running guest you should see a <seclabel> describing the user/group it is running as currently.
If no <seclabel> is in the offline config, libvirt adds the default seclabel, but if you want a different user/group, you can add the <seclabel> yourself.
Regards, Daniel
Hello Daniel,
well, tried that (as good as the docs are) by adding:
<seclabel type='dynamic' model='dac'> <label>nobody:kvm</label> </seclabel>
This edit worked in virsh without giving errors. Starting the domain and then looking into the xml showed:
<seclabel type='dynamic' model='dac' relabel='yes'/>
Consequently qemu runs still as root. My user:group setting simply vanished.
I think at least some better docs are needed with a striking example of how to change user and group ... I may be biased, but how to set user and group is probably the most basic example of how to use seclabel - and I cannot find one.
I agree that the documentation is not the best one. You need to use type='static' relabel='yes': <seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel> To achieve that. In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM. Pavel
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 09:40:23 +0100 Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski wrote:
Hello list,
I came across a fundamental flaw in the libvirt user configuration lately and try to find a solution now. Here is the problem: I run several qemu instances on arch linux all configured via libvirt. The default config as user nobody:kvm was fine up to the day I tried to use a host filesystem via 9p. If you want to gain all user rights on the guest inside that fs you have to run qemu as root. So far so good. But if you have several qemus running and only one needs to be root, what to do? You can try to give a -runas by using <qemu:args>. But that does not work, qemu instantly crashes. I think this is because to have _one_ root qemu, you have to configure libvirt to use root user. This means all rights to fs and so on are set to root and this is what lets qemu probably go crazy if dropping root by -runas. The whole thing would be a lot easier and more transparent if the user in libvirt wouldn't be a global config, but instead be part of the domain xml. This way every qemu started could use a different user and have different rights. In my case all but one could be nobody:kvm, and one root:root. This should not be to complicated based on whats already there, is it?
Libvirt needs to know about the user/group QEMU is running at in order to ensure it gets given access to the various files it needs to use. If you look at the XML of the running guest you should see a <seclabel> describing the user/group it is running as currently.
If no <seclabel> is in the offline config, libvirt adds the default seclabel, but if you want a different user/group, you can add the <seclabel> yourself.
Regards, Daniel
Hello Daniel,
well, tried that (as good as the docs are) by adding:
<seclabel type='dynamic' model='dac'> <label>nobody:kvm</label> </seclabel>
This edit worked in virsh without giving errors. Starting the domain and then looking into the xml showed:
<seclabel type='dynamic' model='dac' relabel='yes'/>
Consequently qemu runs still as root. My user:group setting simply vanished.
I think at least some better docs are needed with a striking example of how to change user and group ... I may be biased, but how to set user and group is probably the most basic example of how to use seclabel - and I cannot find one.
I agree that the documentation is not the best one.
You need to use type='static' relabel='yes':
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
To achieve that.
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Hello Pavel, thank you for taking up the thread, but unfortunately your suggestion does not work: virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-09T12:34:00.735392Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes”: Permission denied Obviously this is because "type='static'" means that libvirt does not care about setting the user rights for qemu, which then leads to this. I did think "relabel='yes'" should do that, but does not - or I have a deep misunderstanding concerning the seclabel parameters ... Thanks for any help to solve this, if there is no bug involved. Dumpxml shows this btw: <seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel> which at least is what was configured. -- Regards, Stephan
On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 09:40:23 +0100 Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski wrote:
Hello list,
I came across a fundamental flaw in the libvirt user configuration lately and try to find a solution now. Here is the problem: I run several qemu instances on arch linux all configured via libvirt. The default config as user nobody:kvm was fine up to the day I tried to use a host filesystem via 9p. If you want to gain all user rights on the guest inside that fs you have to run qemu as root. So far so good. But if you have several qemus running and only one needs to be root, what to do? You can try to give a -runas by using <qemu:args>. But that does not work, qemu instantly crashes. I think this is because to have _one_ root qemu, you have to configure libvirt to use root user. This means all rights to fs and so on are set to root and this is what lets qemu probably go crazy if dropping root by -runas. The whole thing would be a lot easier and more transparent if the user in libvirt wouldn't be a global config, but instead be part of the domain xml. This way every qemu started could use a different user and have different rights. In my case all but one could be nobody:kvm, and one root:root. This should not be to complicated based on whats already there, is it?
Libvirt needs to know about the user/group QEMU is running at in order to ensure it gets given access to the various files it needs to use. If you look at the XML of the running guest you should see a <seclabel> describing the user/group it is running as currently.
If no <seclabel> is in the offline config, libvirt adds the default seclabel, but if you want a different user/group, you can add the <seclabel> yourself.
Regards, Daniel
Hello Daniel,
well, tried that (as good as the docs are) by adding:
<seclabel type='dynamic' model='dac'> <label>nobody:kvm</label> </seclabel>
This edit worked in virsh without giving errors. Starting the domain and then looking into the xml showed:
<seclabel type='dynamic' model='dac' relabel='yes'/>
Consequently qemu runs still as root. My user:group setting simply vanished.
I think at least some better docs are needed with a striking example of how to change user and group ... I may be biased, but how to set user and group is probably the most basic example of how to use seclabel - and I cannot find one.
I agree that the documentation is not the best one.
You need to use type='static' relabel='yes':
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
To achieve that.
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Hello Pavel,
thank you for taking up the thread, but unfortunately your suggestion does not work:
virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-09T12:34:00.735392Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes”: Permission denied
Obviously this is because "type='static'" means that libvirt does not care about setting the user rights for qemu, which then leads to this.
No, 'static' means you tell libvirt what the label should be, 'dynamic' means libvirt will generate it automatically.
I did think "relabel='yes'" should do that, but does not - or I have a deep misunderstanding concerning the seclabel parameters ... Thanks for any help to solve this, if there is no bug involved.
Relabel definitely does that and unless there is a bug it uses the seclabel for the domain. What could be happening is that one of the parent directories is missing a browse permission for others (the last 'x' in rwxrwxrwx). Could you run the following commands and reply with the output? ls -ld /var/lib/ ls -ld /var/lib/libvirt/ ls -ld /var/lib/libvirt/qemu/ Also what distribution are you using? I remember there were some differences in packaging related to the directory permissions. Martin
Dumpxml shows this btw:
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
which at least is what was configured. -- Regards, Stephan
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, 10 Jul 2019 14:48:14 +0200 Martin Kletzander <mkletzan@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 09:40:23 +0100 Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski wrote:
Hello list,
I came across a fundamental flaw in the libvirt user configuration lately and try to find a solution now. Here is the problem: I run several qemu instances on arch linux all configured via libvirt. The default config as user nobody:kvm was fine up to the day I tried to use a host filesystem via 9p. If you want to gain all user rights on the guest inside that fs you have to run qemu as root. So far so good. But if you have several qemus running and only one needs to be root, what to do? You can try to give a -runas by using <qemu:args>. But that does not work, qemu instantly crashes. I think this is because to have _one_ root qemu, you have to configure libvirt to use root user. This means all rights to fs and so on are set to root and this is what lets qemu probably go crazy if dropping root by -runas. The whole thing would be a lot easier and more transparent if the user in libvirt wouldn't be a global config, but instead be part of the domain xml. This way every qemu started could use a different user and have different rights. In my case all but one could be nobody:kvm, and one root:root. This should not be to complicated based on whats already there, is it?
Libvirt needs to know about the user/group QEMU is running at in order to ensure it gets given access to the various files it needs to use. If you look at the XML of the running guest you should see a <seclabel> describing the user/group it is running as currently.
If no <seclabel> is in the offline config, libvirt adds the default seclabel, but if you want a different user/group, you can add the <seclabel> yourself.
Regards, Daniel
Hello Daniel,
well, tried that (as good as the docs are) by adding:
<seclabel type='dynamic' model='dac'> <label>nobody:kvm</label> </seclabel>
This edit worked in virsh without giving errors. Starting the domain and then looking into the xml showed:
<seclabel type='dynamic' model='dac' relabel='yes'/>
Consequently qemu runs still as root. My user:group setting simply vanished.
I think at least some better docs are needed with a striking example of how to change user and group ... I may be biased, but how to set user and group is probably the most basic example of how to use seclabel - and I cannot find one.
I agree that the documentation is not the best one.
You need to use type='static' relabel='yes':
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
To achieve that.
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Hello Pavel,
thank you for taking up the thread, but unfortunately your suggestion does not work:
virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-09T12:34:00.735392Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes”: Permission denied
Obviously this is because "type='static'" means that libvirt does not care about setting the user rights for qemu, which then leads to this.
No, 'static' means you tell libvirt what the label should be, 'dynamic' means libvirt will generate it automatically.
I did think "relabel='yes'" should do that, but does not - or I have a deep misunderstanding concerning the seclabel parameters ... Thanks for any help to solve this, if there is no bug involved.
Relabel definitely does that and unless there is a bug it uses the seclabel for the domain. What could be happening is that one of the parent directories is missing a browse permission for others (the last 'x' in rwxrwxrwx). Could you run the following commands and reply with the output?
ls -ld /var/lib/ ls -ld /var/lib/libvirt/ ls -ld /var/lib/libvirt/qemu/
Also what distribution are you using? I remember there were some differences in packaging related to the directory permissions.
Martin
Dumpxml shows this btw:
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
which at least is what was configured. -- Regards, Stephan
Hello Martin, thanks for picking up. Here is the output you requested: [root@vserver-a01 /]# ls -ld /var/lib/ drwxr-xr-x 21 root root 4096 10. Jul 00:00 /var/lib/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/ drwxr-xr-x 11 root root 4096 4. Jul 10:08 /var/lib/libvirt/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ drwxrwx--- 11 root root 4096 10. Jul 15:36 /var/lib/libvirt/qemu/ It seems your guess is right. After [root@vserver-a01 /]# chmod a+x /var/lib/libvirt/qemu/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ drwxrwx--x 11 root root 4096 10. Jul 15:41 /var/lib/libvirt/qemu/ it started: virsh # start collabora Domain collabora gestartet So that's a bug in the Arch Linux packaging? Who to tell? Thank you Martin, Pavel and Daniel for tracking that down. -- Regards, Stephan
On Wed, Jul 10, 2019 at 03:44:54PM +0200, Stephan von Krawczynski wrote:
On Wed, 10 Jul 2019 14:48:14 +0200 Martin Kletzander <mkletzan@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 09:40:23 +0100 Daniel P. Berrangé <berrange@redhat.com> wrote:
On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski wrote: > Hello list, > > I came across a fundamental flaw in the libvirt user configuration > lately and try to find a solution now. Here is the problem: > I run several qemu instances on arch linux all configured via libvirt. > The default config as user nobody:kvm was fine up to the day I tried > to use a host filesystem via 9p. If you want to gain all user rights > on the guest inside that fs you have to run qemu as root. So far so > good. But if you have several qemus running and only one needs to be > root, what to do? You can try to give a -runas by using <qemu:args>. > But that does not work, qemu instantly crashes. I think this is > because to have _one_ root qemu, you have to configure libvirt to use > root user. This means all rights to fs and so on are set to root and > this is what lets qemu probably go crazy if dropping root by -runas. > The whole thing would be a lot easier and more transparent if the user > in libvirt wouldn't be a global config, but instead be part of the > domain xml. This way every qemu started could use a different user and > have different rights. In my case all but one could be nobody:kvm, and > one root:root. This should not be to complicated based on whats > already there, is it?
Libvirt needs to know about the user/group QEMU is running at in order to ensure it gets given access to the various files it needs to use. If you look at the XML of the running guest you should see a <seclabel> describing the user/group it is running as currently.
If no <seclabel> is in the offline config, libvirt adds the default seclabel, but if you want a different user/group, you can add the <seclabel> yourself.
Regards, Daniel
Hello Daniel,
well, tried that (as good as the docs are) by adding:
<seclabel type='dynamic' model='dac'> <label>nobody:kvm</label> </seclabel>
This edit worked in virsh without giving errors. Starting the domain and then looking into the xml showed:
<seclabel type='dynamic' model='dac' relabel='yes'/>
Consequently qemu runs still as root. My user:group setting simply vanished.
I think at least some better docs are needed with a striking example of how to change user and group ... I may be biased, but how to set user and group is probably the most basic example of how to use seclabel - and I cannot find one.
I agree that the documentation is not the best one.
You need to use type='static' relabel='yes':
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
To achieve that.
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Hello Pavel,
thank you for taking up the thread, but unfortunately your suggestion does not work:
virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-09T12:34:00.735392Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes”: Permission denied
Obviously this is because "type='static'" means that libvirt does not care about setting the user rights for qemu, which then leads to this.
No, 'static' means you tell libvirt what the label should be, 'dynamic' means libvirt will generate it automatically.
I did think "relabel='yes'" should do that, but does not - or I have a deep misunderstanding concerning the seclabel parameters ... Thanks for any help to solve this, if there is no bug involved.
Relabel definitely does that and unless there is a bug it uses the seclabel for the domain. What could be happening is that one of the parent directories is missing a browse permission for others (the last 'x' in rwxrwxrwx). Could you run the following commands and reply with the output?
ls -ld /var/lib/ ls -ld /var/lib/libvirt/ ls -ld /var/lib/libvirt/qemu/
Also what distribution are you using? I remember there were some differences in packaging related to the directory permissions.
Martin
Dumpxml shows this btw:
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
which at least is what was configured. -- Regards, Stephan
Hello Martin,
thanks for picking up. Here is the output you requested:
[root@vserver-a01 /]# ls -ld /var/lib/ drwxr-xr-x 21 root root 4096 10. Jul 00:00 /var/lib/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/ drwxr-xr-x 11 root root 4096 4. Jul 10:08 /var/lib/libvirt/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ drwxrwx--- 11 root root 4096 10. Jul 15:36 /var/lib/libvirt/qemu/
It seems your guess is right. After
[root@vserver-a01 /]# chmod a+x /var/lib/libvirt/qemu/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ drwxrwx--x 11 root root 4096 10. Jul 15:41 /var/lib/libvirt/qemu/
it started:
virsh # start collabora Domain collabora gestartet
So that's a bug in the Arch Linux packaging? Who to tell?
Our Makefile specifies what to do on installation: $(MKDIR_P) -m 0751 "$(DESTDIR)$(localstatedir)/lib/libvirt/qemu" so I guess this is a packaging issue. No idea where/how the arch packaging works, sorry.
Thank you Martin, Pavel and Daniel for tracking that down.
-- Regards, Stephan
On Wed, 10 Jul 2019 15:55:04 +0200 Martin Kletzander <mkletzan@redhat.com> wrote:
On Wed, Jul 10, 2019 at 03:44:54PM +0200, Stephan von Krawczynski wrote:
On Wed, 10 Jul 2019 14:48:14 +0200 Martin Kletzander <mkletzan@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 09:40:23 +0100 Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski > wrote: > > Hello list, > > > > I came across a fundamental flaw in the libvirt user configuration > > lately and try to find a solution now. Here is the problem: > > I run several qemu instances on arch linux all configured via libvirt. > > The default config as user nobody:kvm was fine up to the day I tried > > to use a host filesystem via 9p. If you want to gain all user rights > > on the guest inside that fs you have to run qemu as root. So far so > > good. But if you have several qemus running and only one needs to be > > root, what to do? You can try to give a -runas by using <qemu:args>. > > But that does not work, qemu instantly crashes. I think this is > > because to have _one_ root qemu, you have to configure libvirt to use > > root user. This means all rights to fs and so on are set to root and > > this is what lets qemu probably go crazy if dropping root by -runas. > > The whole thing would be a lot easier and more transparent if the user > > in libvirt wouldn't be a global config, but instead be part of the > > domain xml. This way every qemu started could use a different user and > > have different rights. In my case all but one could be nobody:kvm, and > > one root:root. This should not be to complicated based on whats > > already there, is it? > > Libvirt needs to know about the user/group QEMU is running at in order to > ensure it gets given access to the various files it needs to use. If you > look at the XML of the running guest you should see a <seclabel> > describing the user/group it is running as currently. > > If no <seclabel> is in the offline config, libvirt adds the default > seclabel, but if you want a different user/group, you can add the > <seclabel> yourself. > > Regards, > Daniel
Hello Daniel,
well, tried that (as good as the docs are) by adding:
<seclabel type='dynamic' model='dac'> <label>nobody:kvm</label> </seclabel>
This edit worked in virsh without giving errors. Starting the domain and then looking into the xml showed:
<seclabel type='dynamic' model='dac' relabel='yes'/>
Consequently qemu runs still as root. My user:group setting simply vanished.
I think at least some better docs are needed with a striking example of how to change user and group ... I may be biased, but how to set user and group is probably the most basic example of how to use seclabel - and I cannot find one.
I agree that the documentation is not the best one.
You need to use type='static' relabel='yes':
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
To achieve that.
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Hello Pavel,
thank you for taking up the thread, but unfortunately your suggestion does not work:
virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-09T12:34:00.735392Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes”: Permission denied
Obviously this is because "type='static'" means that libvirt does not care about setting the user rights for qemu, which then leads to this.
No, 'static' means you tell libvirt what the label should be, 'dynamic' means libvirt will generate it automatically.
I did think "relabel='yes'" should do that, but does not - or I have a deep misunderstanding concerning the seclabel parameters ... Thanks for any help to solve this, if there is no bug involved.
Relabel definitely does that and unless there is a bug it uses the seclabel for the domain. What could be happening is that one of the parent directories is missing a browse permission for others (the last 'x' in rwxrwxrwx). Could you run the following commands and reply with the output?
ls -ld /var/lib/ ls -ld /var/lib/libvirt/ ls -ld /var/lib/libvirt/qemu/
Also what distribution are you using? I remember there were some differences in packaging related to the directory permissions.
Martin
Dumpxml shows this btw:
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
which at least is what was configured. -- Regards, Stephan
Hello Martin,
thanks for picking up. Here is the output you requested:
[root@vserver-a01 /]# ls -ld /var/lib/ drwxr-xr-x 21 root root 4096 10. Jul 00:00 /var/lib/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/ drwxr-xr-x 11 root root 4096 4. Jul 10:08 /var/lib/libvirt/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ drwxrwx--- 11 root root 4096 10. Jul 15:36 /var/lib/libvirt/qemu/
It seems your guess is right. After
[root@vserver-a01 /]# chmod a+x /var/lib/libvirt/qemu/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ drwxrwx--x 11 root root 4096 10. Jul 15:41 /var/lib/libvirt/qemu/
it started:
virsh # start collabora Domain collabora gestartet
So that's a bug in the Arch Linux packaging? Who to tell?
Our Makefile specifies what to do on installation:
$(MKDIR_P) -m 0751 "$(DESTDIR)$(localstatedir)/lib/libvirt/qemu"
so I guess this is a packaging issue. No idea where/how the arch packaging works, sorry.
Thank you Martin, Pavel and Daniel for tracking that down.
-- Regards, Stephan
Hello Martin, I will take care of that for arch. Topic closed. -- Regards, Stephan
On Thu, 11 Jul 2019 08:06:35 +0200 Stephan von Krawczynski <skraw.ml@ithnet.com> wrote:
On Wed, 10 Jul 2019 15:55:04 +0200 Martin Kletzander <mkletzan@redhat.com> wrote:
On Wed, Jul 10, 2019 at 03:44:54PM +0200, Stephan von Krawczynski wrote:
On Wed, 10 Jul 2019 14:48:14 +0200 Martin Kletzander <mkletzan@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski wrote: > On Tue, 9 Jul 2019 09:40:23 +0100 > Daniel P. Berrangé <berrange@redhat.com> wrote: > > > On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski > > wrote: > > > Hello list, > > > > > > I came across a fundamental flaw in the libvirt user > > > configuration lately and try to find a solution now. Here is > > > the problem: I run several qemu instances on arch linux all > > > configured via libvirt. The default config as user nobody:kvm > > > was fine up to the day I tried to use a host filesystem via > > > 9p. If you want to gain all user rights on the guest inside > > > that fs you have to run qemu as root. So far so good. But if > > > you have several qemus running and only one needs to be root, > > > what to do? You can try to give a -runas by using <qemu:args>. > > > But that does not work, qemu instantly crashes. I think this > > > is because to have _one_ root qemu, you have to configure > > > libvirt to use root user. This means all rights to fs and so > > > on are set to root and this is what lets qemu probably go > > > crazy if dropping root by -runas. The whole thing would be a > > > lot easier and more transparent if the user in libvirt > > > wouldn't be a global config, but instead be part of the domain > > > xml. This way every qemu started could use a different user > > > and have different rights. In my case all but one could be > > > nobody:kvm, and one root:root. This should not be to > > > complicated based on whats already there, is it? > > > > Libvirt needs to know about the user/group QEMU is running at in > > order to ensure it gets given access to the various files it > > needs to use. If you look at the XML of the running guest you > > should see a <seclabel> describing the user/group it is running > > as currently. > > > > If no <seclabel> is in the offline config, libvirt adds the > > default seclabel, but if you want a different user/group, you > > can add the <seclabel> yourself. > > > > Regards, > > Daniel > > Hello Daniel, > > well, tried that (as good as the docs are) by adding: > > <seclabel type='dynamic' model='dac'> > <label>nobody:kvm</label> > </seclabel> > > This edit worked in virsh without giving errors. > Starting the domain and then looking into the xml showed: > > <seclabel type='dynamic' model='dac' relabel='yes'/> > > Consequently qemu runs still as root. My user:group setting simply > vanished. > > I think at least some better docs are needed with a striking > example of how to change user and group ... > I may be biased, but how to set user and group is probably the > most basic example of how to use seclabel - and I cannot find > one.
I agree that the documentation is not the best one.
You need to use type='static' relabel='yes':
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
To achieve that.
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Hello Pavel,
thank you for taking up the thread, but unfortunately your suggestion does not work:
virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-09T12:34:00.735392Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes”: Permission denied
Obviously this is because "type='static'" means that libvirt does not care about setting the user rights for qemu, which then leads to this.
No, 'static' means you tell libvirt what the label should be, 'dynamic' means libvirt will generate it automatically.
I did think "relabel='yes'" should do that, but does not - or I have a deep misunderstanding concerning the seclabel parameters ... Thanks for any help to solve this, if there is no bug involved.
Relabel definitely does that and unless there is a bug it uses the seclabel for the domain. What could be happening is that one of the parent directories is missing a browse permission for others (the last 'x' in rwxrwxrwx). Could you run the following commands and reply with the output?
ls -ld /var/lib/ ls -ld /var/lib/libvirt/ ls -ld /var/lib/libvirt/qemu/
Also what distribution are you using? I remember there were some differences in packaging related to the directory permissions.
Martin
Dumpxml shows this btw:
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
which at least is what was configured. -- Regards, Stephan
Hello Martin,
thanks for picking up. Here is the output you requested:
[root@vserver-a01 /]# ls -ld /var/lib/ drwxr-xr-x 21 root root 4096 10. Jul 00:00 /var/lib/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/ drwxr-xr-x 11 root root 4096 4. Jul 10:08 /var/lib/libvirt/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ drwxrwx--- 11 root root 4096 10. Jul 15:36 /var/lib/libvirt/qemu/
It seems your guess is right. After
[root@vserver-a01 /]# chmod a+x /var/lib/libvirt/qemu/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ drwxrwx--x 11 root root 4096 10. Jul 15:41 /var/lib/libvirt/qemu/
it started:
virsh # start collabora Domain collabora gestartet
So that's a bug in the Arch Linux packaging? Who to tell?
Our Makefile specifies what to do on installation:
$(MKDIR_P) -m 0751 "$(DESTDIR)$(localstatedir)/lib/libvirt/qemu"
so I guess this is a packaging issue. No idea where/how the arch packaging works, sorry.
Thank you Martin, Pavel and Daniel for tracking that down.
-- Regards, Stephan
Hello Martin,
I will take care of that for arch. Topic closed.
-- Regards, Stephan
Ok, there is a problem (in libvirt), please take a look at the answer to the bugreport in arch: --- Erm, patch is definitely wrong. Look a few lines above in the PKGBUILD and note: rm -rf \ .. "${pkgdir}"/var/lib/libvirt/qemu Arch is deleting that dir and letting libvirt create it on first run of daemon which appears to work fine. On my system the perms are correct: drwxr-xr-x 8 zzz kvm 4096 Jul 11 12:43 /var/lib/libvirt/qemu (Sidenote: I'm not sure what libvirt does when /etc/libvirt/qemu.conf is subsequently edited with different user/group.) In summary, it looks like upstream bug reporter has somehow acquired wrong perms on /var/lib/libvirt/qemu On the question of whether Arch should be touching that dir or not? Dunno. --- I can assure you I never touched the permissions. So it may well be that some time ago something changed in libvirts handling of these permissions and my installation is older. I would suggest to let libvirt on startup check not just the existence but also the permissions and set them correctly. -- Regards, Stephan
On Thu, 11 Jul 2019 08:06:35 +0200 Stephan von Krawczynski <skraw.ml@ithnet.com> wrote:
On Wed, 10 Jul 2019 15:55:04 +0200 Martin Kletzander <mkletzan@redhat.com> wrote:
On Wed, Jul 10, 2019 at 03:44:54PM +0200, Stephan von Krawczynski wrote:
On Wed, 10 Jul 2019 14:48:14 +0200 Martin Kletzander <mkletzan@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski wrote: > On Tue, 9 Jul 2019 09:40:23 +0100 > Daniel P. Berrangé <berrange@redhat.com> wrote: > > > On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski > > wrote: > > > Hello list, > > > > > > I came across a fundamental flaw in the libvirt user > > > configuration lately and try to find a solution now. Here is > > > the problem: I run several qemu instances on arch linux all > > > configured via libvirt. The default config as user nobody:kvm > > > was fine up to the day I tried to use a host filesystem via > > > 9p. If you want to gain all user rights on the guest inside > > > that fs you have to run qemu as root. So far so good. But if > > > you have several qemus running and only one needs to be root, > > > what to do? You can try to give a -runas by using <qemu:args>. > > > But that does not work, qemu instantly crashes. I think this > > > is because to have _one_ root qemu, you have to configure > > > libvirt to use root user. This means all rights to fs and so > > > on are set to root and this is what lets qemu probably go > > > crazy if dropping root by -runas. The whole thing would be a > > > lot easier and more transparent if the user in libvirt > > > wouldn't be a global config, but instead be part of the domain > > > xml. This way every qemu started could use a different user > > > and have different rights. In my case all but one could be > > > nobody:kvm, and one root:root. This should not be to > > > complicated based on whats already there, is it? > > > > Libvirt needs to know about the user/group QEMU is running at in > > order to ensure it gets given access to the various files it > > needs to use. If you look at the XML of the running guest you > > should see a <seclabel> describing the user/group it is running > > as currently. > > > > If no <seclabel> is in the offline config, libvirt adds the > > default seclabel, but if you want a different user/group, you > > can add the <seclabel> yourself. > > > > Regards, > > Daniel > > Hello Daniel, > > well, tried that (as good as the docs are) by adding: > > <seclabel type='dynamic' model='dac'> > <label>nobody:kvm</label> > </seclabel> > > This edit worked in virsh without giving errors. > Starting the domain and then looking into the xml showed: > > <seclabel type='dynamic' model='dac' relabel='yes'/> > > Consequently qemu runs still as root. My user:group setting simply > vanished. > > I think at least some better docs are needed with a striking > example of how to change user and group ... > I may be biased, but how to set user and group is probably the > most basic example of how to use seclabel - and I cannot find > one.
I agree that the documentation is not the best one.
You need to use type='static' relabel='yes':
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
To achieve that.
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Hello Pavel,
thank you for taking up the thread, but unfortunately your suggestion does not work:
virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-09T12:34:00.735392Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes”: Permission denied
Obviously this is because "type='static'" means that libvirt does not care about setting the user rights for qemu, which then leads to this.
No, 'static' means you tell libvirt what the label should be, 'dynamic' means libvirt will generate it automatically.
I did think "relabel='yes'" should do that, but does not - or I have a deep misunderstanding concerning the seclabel parameters ... Thanks for any help to solve this, if there is no bug involved.
Relabel definitely does that and unless there is a bug it uses the seclabel for the domain. What could be happening is that one of the parent directories is missing a browse permission for others (the last 'x' in rwxrwxrwx). Could you run the following commands and reply with the output?
ls -ld /var/lib/ ls -ld /var/lib/libvirt/ ls -ld /var/lib/libvirt/qemu/
Also what distribution are you using? I remember there were some differences in packaging related to the directory permissions.
Martin
Dumpxml shows this btw:
<seclabel type='static' model='dac' relabel='yes'> <label>nobody:kvm</label> </seclabel>
which at least is what was configured. -- Regards, Stephan
Hello Martin,
thanks for picking up. Here is the output you requested:
[root@vserver-a01 /]# ls -ld /var/lib/ drwxr-xr-x 21 root root 4096 10. Jul 00:00 /var/lib/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/ drwxr-xr-x 11 root root 4096 4. Jul 10:08 /var/lib/libvirt/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ drwxrwx--- 11 root root 4096 10. Jul 15:36 /var/lib/libvirt/qemu/
It seems your guess is right. After
[root@vserver-a01 /]# chmod a+x /var/lib/libvirt/qemu/ [root@vserver-a01 /]# ls -ld /var/lib/libvirt/qemu/ drwxrwx--x 11 root root 4096 10. Jul 15:41 /var/lib/libvirt/qemu/
it started:
virsh # start collabora Domain collabora gestartet
So that's a bug in the Arch Linux packaging? Who to tell?
Our Makefile specifies what to do on installation:
$(MKDIR_P) -m 0751 "$(DESTDIR)$(localstatedir)/lib/libvirt/qemu"
so I guess this is a packaging issue. No idea where/how the arch packaging works, sorry.
Thank you Martin, Pavel and Daniel for tracking that down.
-- Regards, Stephan
Hello Martin,
I will take care of that for arch. Topic closed.
-- Regards, Stephan
Ok, there is a problem (in libvirt), please take a look at the answer to the bugreport in arch: --- Erm, patch is definitely wrong. Look a few lines above in the PKGBUILD and note: rm -rf \ .. "${pkgdir}"/var/lib/libvirt/qemu Arch is deleting that dir and letting libvirt create it on first run of daemon which appears to work fine. On my system the perms are correct: drwxr-xr-x 8 zzz kvm 4096 Jul 11 12:43 /var/lib/libvirt/qemu (Sidenote: I'm not sure what libvirt does when /etc/libvirt/qemu.conf is subsequently edited with different user/group.) In summary, it looks like upstream bug reporter has somehow acquired wrong perms on /var/lib/libvirt/qemu On the question of whether Arch should be touching that dir or not? Dunno. --- I can assure you I never touched the permissions. So it may well be that some time ago something changed in libvirts handling of these permissions and my installation is older. I would suggest to let libvirt on startup check not just the existence but also the permissions and set them correctly. -- MfG, Stephan von Krawczynski ------------------------------------------------------ ith Kommunikationstechnik GmbH Lieferanschrift : Reiterstrasse 24, D-94447 Plattling Telefon : +49 9931 9188 0 Fax : +49 9931 9188 44 Geschaeftsfuehrer: Stephan von Krawczynski Registergericht : Deggendorf HRB 1625 ------------------------------------------------------
Martin Kletzander <mkletzan@redhat.com> [2019-07-10, 03:55PM +0200]:
Our Makefile specifies what to do on installation:
$(MKDIR_P) -m 0751 "$(DESTDIR)$(localstatedir)/lib/libvirt/qemu"
so I guess this is a packaging issue. No idea where/how the arch packaging works, sorry.
But the case that Stephan has where libvirtd does create the directories on startup is broken. I've ran into this problem myself, but forgot about it until now, when running libvirtd from the source tree with prefix /usr/local. It seems that when creating the directories in the driver initialization, at least the QEMU driver does not set the umask accordingly before, manginling the resulting permissions in the process. Bjoern -- IBM Systems Linux on Z & Virtualization Development -------------------------------------------------- IBM Deutschland Research & Development GmbH Schönaicher Str. 220, 71032 Böblingen Phone: +49 7031 16 1819 -------------------------------------------------- Vorsitzende des Aufsichtsrats: Matthias Hartmann Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
On Thu, Jul 11, 2019 at 01:15:26PM +0200, Bjoern Walk wrote:
Martin Kletzander <mkletzan@redhat.com> [2019-07-10, 03:55PM +0200]:
Our Makefile specifies what to do on installation:
$(MKDIR_P) -m 0751 "$(DESTDIR)$(localstatedir)/lib/libvirt/qemu"
so I guess this is a packaging issue. No idea where/how the arch packaging works, sorry.
But the case that Stephan has where libvirtd does create the directories on startup is broken. I've ran into this problem myself, but forgot about it until now, when running libvirtd from the source tree with prefix /usr/local.
It seems that when creating the directories in the driver initialization, at least the QEMU driver does not set the umask accordingly before, manginling the resulting permissions in the process.
If that is the case it should definitely be fixed in the code in order for it to match the permissions in the makefile. If nobody sends a patch for that in few days, remind me and I'll do that.
Bjoern
-- IBM Systems Linux on Z & Virtualization Development -------------------------------------------------- IBM Deutschland Research & Development GmbH Schönaicher Str. 220, 71032 Böblingen Phone: +49 7031 16 1819 -------------------------------------------------- Vorsitzende des Aufsichtsrats: Matthias Hartmann Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
[...]
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Let me answer this part in another post. Generally I agree with you. But there is one question: if I configure libvirt to use nobody:kvm as user, how is it possible to start a qemu with root privileges? I thought it not to be possible that it runs a root process while its config says it should be nobody ...? I thought it can only _drop_ privileges from root to nobody, because its primary user is root. Or is it in fact always running as root, and only "fake-dropping" to the configured user (maybe a spawned child), while still being able to spawn other root processes? -- Regards, Stephan
On Wed, Jul 10, 2019 at 12:01:18AM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
[...]
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Let me answer this part in another post. Generally I agree with you. But there is one question: if I configure libvirt to use nobody:kvm as user, how is it possible to start a qemu with root privileges? I thought it not to be possible that it runs a root process while its config says it should be nobody ...?
That configuration is in /etc/libvirt/qemu.conf which configures things related to QEMU process and the user:group configuration tells how the QEMU process will be started. The system libvirtd daemon runs always as root:root in order to have permissions to execute QEMU process under any user and to configure a lot of other things when starting a VM.
I thought it can only _drop_ privileges from root to nobody, because its primary user is root. Or is it in fact always running as root, and only "fake-dropping" to the configured user (maybe a spawned child), while still being able to spawn other root processes?
I'm not sure what do you mean by "fake-dropping", libvirt forks itself in order to create a new process where the QEMU binary is executed and the permissions are configured for that newly created process. All of this is true only for the system libvirt, that means if you use qemu:///system connection, for the session libvirt everything runs as your user and there is no session libvirt for root user. The XML and configuration that I've suggested should work as I've tried it before sending the mail. Pavel
On Wed, 10 Jul 2019 09:56:35 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Wed, Jul 10, 2019 at 12:01:18AM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
[...]
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Let me answer this part in another post. Generally I agree with you. But there is one question: if I configure libvirt to use nobody:kvm as user, how is it possible to start a qemu with root privileges? I thought it not to be possible that it runs a root process while its config says it should be nobody ...?
That configuration is in /etc/libvirt/qemu.conf which configures things related to QEMU process and the user:group configuration tells how the QEMU process will be started. The system libvirtd daemon runs always as root:root in order to have permissions to execute QEMU process under any user and to configure a lot of other things when starting a VM.
I thought it can only _drop_ privileges from root to nobody, because its primary user is root. Or is it in fact always running as root, and only "fake-dropping" to the configured user (maybe a spawned child), while still being able to spawn other root processes?
I'm not sure what do you mean by "fake-dropping", libvirt forks itself in order to create a new process where the QEMU binary is executed and the permissions are configured for that newly created process.
All of this is true only for the system libvirt, that means if you use qemu:///system connection, for the session libvirt everything runs as your user and there is no session libvirt for root user.
The XML and configuration that I've suggested should work as I've tried it before sending the mail.
Pavel
Thank you for clearing things up a bit for me. I am using arch linux (see OP) and the libvirt version gives: virsh # version Kompiliert gegen die Bibliothek: libvirt 5.4.0 Verwende Bibliothek: libvirt 5.4.0 Verwende API: QEMU 5.4.0 Laufender Hypervisor: QEMU 4.0.0 Using your seclabel with root:root in libvirt/qemu.conf throws: virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-10T09:49:52.519211Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-18-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-18-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-18-collabora/master-key.aes”: Permission denied Unfortunately I cannot verify what file permissions the requested file has, as it is vanished as soon as the crash happens. I bet though that it has root:root and the correctly set qemu user nobody:kvm has no read rights. So obviously relabel does not work. As it works on your side question is what's different? You are sure that you did not try the other way round and seclabel to root:root for a setup with standard user nobody:kvm. This would explain why you do not get this error... I generally try not to patch around in libvirt or qemu or the hosts' arch system. Which makes this probably at least a bug in the distro... -- Regards, Stephan
On Wed, Jul 10, 2019 at 11:56:38AM +0200, Stephan von Krawczynski wrote:
On Wed, 10 Jul 2019 09:56:35 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Wed, Jul 10, 2019 at 12:01:18AM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
[...]
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Let me answer this part in another post. Generally I agree with you. But there is one question: if I configure libvirt to use nobody:kvm as user, how is it possible to start a qemu with root privileges? I thought it not to be possible that it runs a root process while its config says it should be nobody ...?
That configuration is in /etc/libvirt/qemu.conf which configures things related to QEMU process and the user:group configuration tells how the QEMU process will be started. The system libvirtd daemon runs always as root:root in order to have permissions to execute QEMU process under any user and to configure a lot of other things when starting a VM.
I thought it can only _drop_ privileges from root to nobody, because its primary user is root. Or is it in fact always running as root, and only "fake-dropping" to the configured user (maybe a spawned child), while still being able to spawn other root processes?
I'm not sure what do you mean by "fake-dropping", libvirt forks itself in order to create a new process where the QEMU binary is executed and the permissions are configured for that newly created process.
All of this is true only for the system libvirt, that means if you use qemu:///system connection, for the session libvirt everything runs as your user and there is no session libvirt for root user.
The XML and configuration that I've suggested should work as I've tried it before sending the mail.
Pavel
Thank you for clearing things up a bit for me. I am using arch linux (see OP) and the libvirt version gives:
virsh # version Kompiliert gegen die Bibliothek: libvirt 5.4.0 Verwende Bibliothek: libvirt 5.4.0
Verwende API: QEMU 5.4.0 Laufender Hypervisor: QEMU 4.0.0
Using your seclabel with root:root in libvirt/qemu.conf throws:
virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-10T09:49:52.519211Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-18-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-18-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-18-collabora/master-key.aes”: Permission denied
Unfortunately I cannot verify what file permissions the requested file has, as it is vanished as soon as the crash happens. I bet though that it has root:root and the correctly set qemu user nobody:kvm has no read rights. So obviously relabel does not work. As it works on your side question is what's different? You are sure that you did not try the other way round and seclabel to root:root for a setup with standard user nobody:kvm. This would explain why you do not get this error... I generally try not to patch around in libvirt or qemu or the hosts' arch system. Which makes this probably at least a bug in the distro...
Since the conversation of solution for the issue continues in different part of this thread I would like to just point out that you should try doing it the other way around. You've stated in previous mails that you would like to have only one VM as root:root and the rest of the VMs as the default nobody:kvm. In general it's really bad idea to configure root:root as the default, as it creates unnecessary security risk for all of the VMs. Regardless of that, if there is a bug in libvirt when having root:root as default we should fix it. Pavel
On Thu, 11 Jul 2019 10:07:11 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Wed, Jul 10, 2019 at 11:56:38AM +0200, Stephan von Krawczynski wrote:
On Wed, 10 Jul 2019 09:56:35 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
On Wed, Jul 10, 2019 at 12:01:18AM +0200, Stephan von Krawczynski wrote:
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina <phrdina@redhat.com> wrote:
[...]
In addition if you would like to have only one VM as root:root you should keep the default config as nobody:kvm and use the root:root for that specific VM.
Pavel
Let me answer this part in another post. Generally I agree with you. But there is one question: if I configure libvirt to use nobody:kvm as user, how is it possible to start a qemu with root privileges? I thought it not to be possible that it runs a root process while its config says it should be nobody ...?
That configuration is in /etc/libvirt/qemu.conf which configures things related to QEMU process and the user:group configuration tells how the QEMU process will be started. The system libvirtd daemon runs always as root:root in order to have permissions to execute QEMU process under any user and to configure a lot of other things when starting a VM.
I thought it can only _drop_ privileges from root to nobody, because its primary user is root. Or is it in fact always running as root, and only "fake-dropping" to the configured user (maybe a spawned child), while still being able to spawn other root processes?
I'm not sure what do you mean by "fake-dropping", libvirt forks itself in order to create a new process where the QEMU binary is executed and the permissions are configured for that newly created process.
All of this is true only for the system libvirt, that means if you use qemu:///system connection, for the session libvirt everything runs as your user and there is no session libvirt for root user.
The XML and configuration that I've suggested should work as I've tried it before sending the mail.
Pavel
Thank you for clearing things up a bit for me. I am using arch linux (see OP) and the libvirt version gives:
virsh # version Kompiliert gegen die Bibliothek: libvirt 5.4.0 Verwende Bibliothek: libvirt 5.4.0
Verwende API: QEMU 5.4.0 Laufender Hypervisor: QEMU 4.0.0
Using your seclabel with root:root in libvirt/qemu.conf throws:
virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-10T09:49:52.519211Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-18-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-18-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-18-collabora/master-key.aes”: Permission denied
Unfortunately I cannot verify what file permissions the requested file has, as it is vanished as soon as the crash happens. I bet though that it has root:root and the correctly set qemu user nobody:kvm has no read rights. So obviously relabel does not work. As it works on your side question is what's different? You are sure that you did not try the other way round and seclabel to root:root for a setup with standard user nobody:kvm. This would explain why you do not get this error... I generally try not to patch around in libvirt or qemu or the hosts' arch system. Which makes this probably at least a bug in the distro...
Since the conversation of solution for the issue continues in different part of this thread I would like to just point out that you should try doing it the other way around.
You've stated in previous mails that you would like to have only one VM as root:root and the rest of the VMs as the default nobody:kvm. In general it's really bad idea to configure root:root as the default, as it creates unnecessary security risk for all of the VMs.
Regardless of that, if there is a bug in libvirt when having root:root as default we should fix it.
Pavel
Hello Pavel, I totally agree with you that root:root is not the right choice for the global setup. I was thinking falsely that root:root would be required to be able to drop the rights to nobody for some qemus instead of increasing from nobody to root for one qemu. Since I learned libvirt is running as root anyways obviously setting global user to nobody is the right choice (which I did in the meantime). And since we found (elsewhere in the thread) that it is indeed a dir permission problem my trust in the libvirt project is at least 99% restored :-) Could be 100% if libvirt would check and set the permissions correctly ... ;-) -- Regards, Stephan
participants (6)
-
Bjoern Walk -
Daniel P. Berrangé -
Martin Kletzander -
Pavel Hrdina -
Stephan von Krawczynski -
Stephan von Krawczynski