Hi,

I am trying to understand libvirt dynamic ownership behavior. I have a VM that uses a qcow2 image with the following permissions: $ ll t257kvxg-10-20-101-40.qcow2 -rw-r--r-- 1 jmuro libvirt 2279079936 Mar 20 11:10 t257kvxg-10-20-101-40.qcow2 When I start the domain the permissions are changed: $ virsh start t257kvxg-10-20-101-40 Domain t257kvxg-10-20-101-40 started $ ll t257kvxg-10-20-101-40.qcow2 -rw-r--r-- 1 libvirt-qemu libvirt 2279079936 Mar 20 11:18 t257kvxg-10-20-101-40.qcow2 This is expected behavior based on the settings in /etc/libvirt/qemu.conf: user = "libvirt-qemu" group = "libvirt" # Whether libvirt should dynamically change file ownership # to match the configured user/group above. Defaults to 1. # Set to 0 to disable file ownership changes. #dynamic_ownership = 1 However, when I shutdown the domain, the file permissions revert to root. $ ll t257kvxg-10-20-101-40.qcow2 -rw-r--r-- 1 root root 2282749952 Mar 20 11:20 t257kvxg-10-20-101-40.qcow2 I expect libvirt to revert the file permissions back to the original. Otherwise, a regular user would lose ownership of the image file. FWIW: I am starting the domain as a non-root user under qemu:///system

I am running on Ubuntu 20.04 LTS (Focal Fossa) with the following libvirt level: libvirt-clients/focal,now 6.0.0-0ubuntu5 s390x [installed] libvirt-daemon-driver-qemu/focal,now 6.0.0-0ubuntu5 s390x [installed,automatic] libvirt-daemon-driver-storage-rbd/focal,now 6.0.0-0ubuntu5 s390x [installed,automatic] libvirt-daemon-system-systemd/focal,now 6.0.0-0ubuntu5 s390x [installed,automatic] libvirt-daemon-system/focal,now 6.0.0-0ubuntu5 s390x [installed] libvirt-daemon/focal,now 6.0.0-0ubuntu5 s390x [installed] libvirt-glib-1.0-0/focal,now 2.0.0-2 s390x [installed,automatic] libvirt0/focal,now 6.0.0-0ubuntu5 s390x [installed,automatic] python3-libvirt/focal,now 6.0.0-0ubuntu3 s390x [installed] Thanks -Joe

Hi Martin, thanks for the explanation. Now I understand why libvirt doesn't revert the file permissions back to the original. I am running these VMs on an isolated test machine, so I'll disable dynamic file ownership and make sure libvirt has access to image files.

Sorry about the message formatting. I modified settings on my client, hopefully it sends plaintext now. (I'll switch to personal email going forward, as the choice of email clients at work is limited.)

-Joe From: Martin Kletzander <mkletzan(a)redhat.com&gt; To: Joe Muro <joemuro(a)us.ibm.com&gt; Cc: libvirt-users(a)redhat.com Date: 03/20/2020 04:01 PM Subject: [EXTERNAL] Re: libvirt dynamic file ownership On Fri, Mar 20, 2020 at 03:38:36PM +0000, Joe Muro wrote: >Hi, > Hi, could you please configure your client to send plaintext version as well? We mainly prefer plaintext on this list ;-) >I am trying to understand libvirt dynamic ownership behavior. I have a VM that >uses a qcow2 image with the following permissions: > >$ ll t257kvxg-10-20-101-40.qcow2 >-rw-r--r-- 1 jmuro libvirt 2279079936 Mar 20 11:10 t257kvxg-10-20-101-40.qcow2 > >When I start the domain the permissions are changed: > >$ virsh start t257kvxg-10-20-101-40 >Domain t257kvxg-10-20-101-40 started >$ ll t257kvxg-10-20-101-40.qcow2 >-rw-r--r-- 1 libvirt-qemu libvirt 2279079936 Mar 20 11:18 >t257kvxg-10-20-101-40.qcow2 > >This is expected behavior based on the settings in /etc/libvirt/qemu.conf: > >user = "libvirt-qemu" >group = "libvirt" ># Whether libvirt should dynamically change file ownership ># to match the configured user/group above. Defaults to 1. ># Set to 0 to disable file ownership changes. >#dynamic_ownership = 1 > >However, when I shutdown the domain, the file permissions revert to root. > >$ ll t257kvxg-10-20-101-40.qcow2 >-rw-r--r-- 1 root root 2282749952 Mar 20 11:20 t257kvxg-10-20-101-40.qcow2 > >I expect libvirt to revert the file permissions back to the original. >Otherwise, a regular user would lose ownership of the image file. FWIW: I am >starting the domain as a non-root user under qemu:///system > This has always been the case because the original information is lost (which is actually not that easy to store properly, race-free, etc.) and the safest way to make sure nobody accesses the disks (e.g. another domain running under libvirt-qemu:libvirt, that would get exploited) is to just change it to root:root. Michal finally managed to make this work, in limited cases, but I think it landed in 6.1.0, I'm not sure. Anyway, there are some workarounds you can do: a) set relabel=no for the disk in the XML (and make sure the VM will be able to access it), b) set relabel=no for the whole domain (and make sure the VM will be able to access everything), or c) if worse comes to worse, just disable the whole dynamic ownership and handle it yourself If possible, try upgrading libvirt and checking if that helps. >I am running on Ubuntu 20.04 LTS (Focal Fossa) with the following libvirt >level: > >libvirt-clients/focal,now 6.0.0-0ubuntu5 s390x [installed] >libvirt-daemon-driver-qemu/focal,now 6.0.0-0ubuntu5 s390x [installed,automatic] >libvirt-daemon-driver-storage-rbd/focal,now 6.0.0-0ubuntu5 s390x >[installed,automatic] >libvirt-daemon-system-systemd/focal,now 6.0.0-0ubuntu5 s390x >[installed,automatic] >libvirt-daemon-system/focal,now 6.0.0-0ubuntu5 s390x [installed] >libvirt-daemon/focal,now 6.0.0-0ubuntu5 s390x [installed] >libvirt-glib-1.0-0/focal,now 2.0.0-2 s390x [installed,automatic] >libvirt0/focal,now 6.0.0-0ubuntu5 s390x [installed,automatic] >python3-libvirt/focal,now 6.0.0-0ubuntu3 s390x [installed] > >Thanks > >-Joe > [attachment "signature.asc" deleted by Joe Muro/Poughkeepsie/IBM]

From: "Michal Prívozník" <mprivozn(a)redhat.com&gt; To: Martin Kletzander <mkletzan(a)redhat.com&gt;, Joe Muro

Cc: libvirt-users(a)redhat.com Date: 03/23/2020 12:26 PM Subject: [EXTERNAL] Re: libvirt dynamic file ownership On 20. 3. 2020 20:57, Martin Kletzander wrote: > On Fri, Mar 20, 2020 at 03:38:36PM +0000, Joe Muro wrote: >> Hi, >> > > Hi, could you please configure your client to send plaintext version as > well? > We mainly prefer plaintext on this list ;-) > >> I am trying to understand libvirt dynamic ownership behavior. I have a >> VM that >> uses a qcow2 image with the following permissions: >> >> $ ll t257kvxg-10-20-101-40.qcow2 >> -rw-r--r-- 1 jmuro libvirt 2279079936 Mar 20 11:10 >> t257kvxg-10-20-101-40.qcow2 >> >> When I start the domain the permissions are changed: >> >> $ virsh start t257kvxg-10-20-101-40 >> Domain t257kvxg-10-20-101-40 started >> $ ll t257kvxg-10-20-101-40.qcow2 >> -rw-r--r-- 1 libvirt-qemu libvirt 2279079936 Mar 20 11:18 >> t257kvxg-10-20-101-40.qcow2 >> >> This is expected behavior based on the settings in >> /etc/libvirt/qemu.conf: >> >> user = "libvirt-qemu" >> group = "libvirt" >> # Whether libvirt should dynamically change file ownership >> # to match the configured user/group above. Defaults to 1. >> # Set to 0 to disable file ownership changes. >> #dynamic_ownership = 1 >> >> However, when I shutdown the domain, the file permissions revert to

>> >> $ ll t257kvxg-10-20-101-40.qcow2 >> -rw-r--r-- 1 root root 2282749952 Mar 20 11:20 >> t257kvxg-10-20-101-40.qcow2 >> >> I expect libvirt to revert the file permissions back to the original. >> Otherwise, a regular user would lose ownership of the image file. >> FWIW: I am >> starting the domain as a non-root user under qemu:///system >> > > This has always been the case because the original information is lost > (which is > actually not that easy to store properly, race-free, etc.) and the > safest way to > make sure nobody accesses the disks (e.g. another domain running under > libvirt-qemu:libvirt, that would get exploited) is to just change it to > root:root.  Michal finally managed to make this work, in limited cases, > but I > think it landed in 6.1.0, I'm not sure. > > Anyway, there are some workarounds you can do: > >  a) set relabel=no for the disk in the XML (and make sure the VM will

> able to >     access it), > >  b) set relabel=no for the whole domain (and make sure the VM will be > able to >     access everything), or > >  c) if worse comes to worse, just disable the whole dynamic ownership > and handle >     it yourself > > If possible, try upgrading libvirt and checking if that helps. Remembering of the original owner was enabled even in 6.0.0, but there are some prerequisites: 1) the FS that hosts the image must be capable of XATTRs. Note the NFS still isn't.

2) the disk XML. Can you please share the <disk/> snippet for this disk? It needs to be the top most layer of backing chain (if you have some snapshots over it).

Upgrading to 6.1.0 would help though, because I'm constantly fixing some bugs in that area as I go along. BTW: you can check if the original owner remembering is enabled for your domain if you look whether the domain status XML has rememberOwner set: grep rememberOwner /var/run/libvirt/qemu/$domain.xml

Michal