Hey Martin & Michal, thank you both for the replies!

Yes, sorry I messed up that seclabel entry, thanks. I fixed it and there is no more error, but the images are still copied unfortunately.

@Michal yes, I have been reading the Debian Live Manual like a contract, after messing with qemu.conf and permissions for days, I think its something in the live build and not libvirt.

@Martin yeah both virsh start and virt-manager cause the images to be copied to /run..... first.

Yes, when I boot the Debian Live ISO, plug in the USB (mounted rw), copy the images to the live system from the USB and click run in virt-manager, the VM starts instantly and no copies are made in /run....

I use ISO loopback and toram to load the system directly into ram via tmpfs mount. That's my grub.cfg

menuentry "debian-live.iso" { set iso="/ISO/debian-live.iso" set root=(hd0,5) loopback loop $iso linux (loop)/live/vmlinuz-4.19.0-6-amd64 boot=live components toram findiso=$iso initrd (loop)/live/initrd.img-4.19.0-6-amd64 }

So, the squashfs is mounted read-only in /run/live/rootfs/filesystem.squashfs and its /dev/loop1

/proc/mounts says:

tmpfs /run/live/overlay rw
overlay rw  lowerdir=/run/live/rootfs/filesystem.squashfs 
upperdir=/run/live/overlay/rw  (same place where the images are copied)

So you think that because the images are part of the squashfs which is read-only, they are first copied to /run/live/overlay/rw... to be written to?

I always thought that tmpfs is mounted on top of squash, so it's as if the squashfs were writable, without having to literally copy between the two filesystems.

Also I made the same build with Virtualbox and there is no copying of the images to /run/live/overlay.

I will do some serious research into squashfs and tmpfs rn.




On Wed, Nov 24, 2021, 5:10 PM Martin Kletzander <mkletzan@redhat.com> wrote:
On Wed, Nov 24, 2021 at 04:01:34PM +0100, Elias Mobery wrote:
>Hello Michal, thank you for the reply!
>
>I've carefully tested everything you suggested, thanks.
>
>I set dynamic_ownership=0 and use these hooks during the live build for
>permissions. (I googled a lot, and apparently libvirt needs the images to
>be executable too)
>
>chown -R libvirt-qemu:kvm  /var/lib/libvirt/images
>chmod -R g+rwx /var/lib/libvirt/images
>

I do not see why would the images need to be executable, you might need
an executable bit set on the directory so that your and qemu user can
browse it.

>Booting the live debian iso everything works in virt-manager, but again,
>after clicking "run", a copy of the vm image is created in
>/run/live/overlay/rw/var/lib/libvirt/images and only then does the VM start.
>

Does that happen when you try to run it with virsh instead of
virt-manager?  Are you connected to the system daemon?

>Either it's still being chowned or chmodded somehow, or it's something
>else, but I can't stop this copy being made.
>
>Interestingly, when I boot the Live debian iso and then copy the images
>into /var/lib/libvirt/images from my USB stick, the VM starts immediately
>without creating any copies in the /run/live.... directory. So my guess is
>that maybe the squashfs could be the issue?
>

Oh, interesting, is the USB stick filesystem mounted R/W and the live
ISO filesystem mounted read-only?  How is the overlay mounted?

>Editing the XML
>
><source file='/var/lib/libvirt/images/vm1.qcow2'>
>      <seclabel relabel='no'/>
>    </source>
>
>This results in an error:
>Unsupported Configuration:
>Security driver model 'null' not available
>

For issues like this it is most helpful to check the documentation:

   https://libvirt.org/formatdomain.html

where you can see it is just a missing attribute `model`, in this case
model="dac".

>Here I tried setting security_driver=none in qemu.conf but same error after.
>
></devices>
>    <seclabel type='none'/>
>  </domain>
>

Same here.

>This also returns an Error but I'm still googling to understand it properly.
>
>XML document failed to validate against schema
>Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
>Invalid element relabel  for element seclabel
>Extra element seclabel in interleave
>Element domain failed to validate content
>
>Thanks again so much for your helo, I've been messing with this for weeks
>now and it's killing me.
>
>On Tue, Nov 23, 2021, 9:43 PM Michal Prívozník <mprivozn@redhat.com> wrote:
>
>> On 11/23/21 17:25, Elias Mobery wrote:
>> >
>> > Hi everyone!
>> >
>> > I've built a Debian Live ISO with packages qemu and libvirt to run a VM
>> > in the live environment.
>> >
>> > The guest images are placed in  /var/lib/libvirt/images and 2GB each.
>> >
>> > Everything works great, except for one issue.
>> >
>> > When starting a VM, libvirt automatically issues a chown command to the
>> > images, changing ownership.
>> >
>> > This results in a copy of the images being created in
>> > /run/live/overlay/rw/var/lib/libvirt/images
>> >
>> > I don't want these copies to be made but can't stop it.
>> >
>> > I've tried editing qemu.conf user/group, dynamic ownership etc. without
>> > any luck.
>> >
>> > Is there a way to STOP libvirt from changing the ownership of these
>> images?
>> >
>> >
>>
>> Setting dynamic_ownership=0 in qemu.conf is the way to go (don't forget
>> to restart the daemon after you made the change).
>>
>> Alternatively, you can set <seclabel/> either for whole domain or
>> individual disks, e.g. like this:
>>
>>   <disk type='file' device='disk'>
>>     <driver name='qemu' type='qcow2'/>
>>     <source file='/var/lib/libvirt/images/vm1.qcow2'>
>>       <seclabel relabel='no'/>
>>     </source>
>>   </disk>
>>
>> or for whole domain:
>>
>>     ...
>>     </devices>
>>     <seclabel type='none'/>
>>   </domain>
>>
>> I'm not sure what your setup is, but if chown() is a problem then what
>> if guest tries to write onto its disk? Wouldn't that create a copy in
>> overlay?
>>
>> Michal
>>
>>