Hi guys. I've have a guest and that guest differs from all other guest by:   <os>     <type arch='x86_64' machine='pc-q35-rhel9.0.0'>hvm</type>     <loader readonly='yes' secure='yes' type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader> <nvram>/var/lib/libvirt/qemu/nvram/ubusrv1_VARS.fd</nvram>     <boot dev='hd'/>     <bootmenu enable='yes'/>   </os> whereas everything else has:   <os>     <type arch='x86_64' machine='pc-q35-rhel9.0.0'>hvm</type>     <boot dev='hd'/>     <boot dev='cdrom'/>     <bootmenu enable='yes'/>   </os> Now, that different guest fails - as the only one - to start, to boot after its qcow2 image was luks-encrypted. Guest starts but says that: BdsDxe: failed to load Boot0001 "Uefi Misc Device" from PciRoot (0x0)/Pci(0x2,0x3)/Pci(0x0,0x0): Not found revert back to original, non-encrypted qcow2 image and all works a ok.

On 11/04/2023 09:13, Peter Krempa wrote: > On Sat, Apr 08, 2023 at 11:25:18 +0200, lejeczek wrote: > > Hi guys. > > > > I've have a guest and that guest differs from all other guest by: > > > >   <os> > >     <type arch='x86_64' machine='pc-q35-rhel9.0.0'>hvm</type> > >     <loader readonly='yes' secure='yes' > > type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader> > > <nvram>/var/lib/libvirt/qemu/nvram/ubusrv1_VARS.fd</nvram> > >     <boot dev='hd'/> > >     <bootmenu enable='yes'/> > >   </os> > > > > whereas everything else has: > > > >   <os> > >     <type arch='x86_64' machine='pc-q35-rhel9.0.0'>hvm</type> > >     <boot dev='hd'/> > >     <boot dev='cdrom'/> > >     <bootmenu enable='yes'/> > >   </os> > > > > Now, that different guest fails - as the only one - to start, to boot after > > its qcow2 image was luks-encrypted. > > Guest starts but says that: > > > > BdsDxe: failed to load Boot0001 "Uefi Misc Device" from PciRoot > > (0x0)/Pci(0x2,0x3)/Pci(0x0,0x0): Not found > > > > revert back to original, non-encrypted qcow2 image and all works a ok. > Please attach either the full XML or at least the disk part for *both* > the case where it doesn't work and where it does work.

  <devices>     <emulator>/usr/libexec/qemu-kvm</emulator>     <disk type='file' device='disk'>       <driver name='qemu' type='qcow2' cache='none' discard='unmap'/>       <source file='/00-VMs/ubusrv1.qcow2'/>       <target dev='vda' bus='virtio'/>       <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>     </disk> ... When I add encryption to <disk> & use encrypted qcow2 then VM fails as I described.

On Fri, Apr 14, 2023 at 13:39:17 +0200, lejeczek wrote: > > On 11/04/2023 09:13, Peter Krempa wrote: >> On Sat, Apr 08, 2023 at 11:25:18 +0200, lejeczek wrote: >>> Hi guys. >>> >>> I've have a guest and that guest differs from all other guest by: >>> >>>   <os> >>>     <type arch='x86_64' machine='pc-q35-rhel9.0.0'>hvm</type> >>>     <loader readonly='yes' secure='yes' >>> type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader> >>> <nvram>/var/lib/libvirt/qemu/nvram/ubusrv1_VARS.fd</nvram> >>>     <boot dev='hd'/> >>>     <bootmenu enable='yes'/> >>>   </os> >>> >>> whereas everything else has: >>> >>>   <os> >>>     <type arch='x86_64' machine='pc-q35-rhel9.0.0'>hvm</type> >>>     <boot dev='hd'/> >>>     <boot dev='cdrom'/> >>>     <bootmenu enable='yes'/> >>>   </os> >>> >>> Now, that different guest fails - as the only one - to start, to boot after >>> its qcow2 image was luks-encrypted. >>> Guest starts but says that: >>> >>> BdsDxe: failed to load Boot0001 "Uefi Misc Device" from PciRoot >>> (0x0)/Pci(0x2,0x3)/Pci(0x0,0x0): Not found >>> >>> revert back to original, non-encrypted qcow2 image and all works a ok. >> Please attach either the full XML or at least the disk part for *both* >> the case where it doesn't work and where it does work. [...] >   <devices> >     <emulator>/usr/libexec/qemu-kvm</emulator> >     <disk type='file' device='disk'> >       <driver name='qemu' type='qcow2' cache='none' discard='unmap'/> >       <source file='/00-VMs/ubusrv1.qcow2'/> >       <target dev='vda' bus='virtio'/> >       <address type='pci' domain='0x0000' bus='0x04' slot='0x00' > function='0x0'/> >     </disk> > ... > > When I add encryption to <disk> & use encrypted qcow2 then VM fails as I > described. I specifically asked for '*both*' XMLs. The working one. And the non-working one.

On Fri, Apr 14, 2023 at 14:26:53 +0200, lejeczek wrote: > > On 14/04/2023 13:57, Peter Krempa wrote: >> On Fri, Apr 14, 2023 at 13:39:17 +0200, lejeczek wrote: >>> On 11/04/2023 09:13, Peter Krempa wrote: >>>> On Sat, Apr 08, 2023 at 11:25:18 +0200, lejeczek wrote: >>>>> Hi guys. >>>>> >>>>> I've have a guest and that guest differs from all other guest by: >>>>> >>>>>   <os> >>>>>     <type arch='x86_64' machine='pc-q35-rhel9.0.0'>hvm</type> >>>>>     <loader readonly='yes' secure='yes' >>>>> type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd</loader> >>>>> <nvram>/var/lib/libvirt/qemu/nvram/ubusrv1_VARS.fd</nvram> >>>>>     <boot dev='hd'/> >>>>>     <bootmenu enable='yes'/> >>>>>   </os> >>>>> >>>>> whereas everything else has: >>>>> >>>>>   <os> >>>>>     <type arch='x86_64' machine='pc-q35-rhel9.0.0'>hvm</type> >>>>>     <boot dev='hd'/> >>>>>     <boot dev='cdrom'/> >>>>>     <bootmenu enable='yes'/> >>>>>   </os> >>>>> >>>>> Now, that different guest fails - as the only one - to start, to boot after >>>>> its qcow2 image was luks-encrypted. >>>>> Guest starts but says that: >>>>> >>>>> BdsDxe: failed to load Boot0001 "Uefi Misc Device" from PciRoot >>>>> (0x0)/Pci(0x2,0x3)/Pci(0x0,0x0): Not found >>>>> >>>>> revert back to original, non-encrypted qcow2 image and all works a ok. >>>> Please attach either the full XML or at least the disk part for *both* >>>> the case where it doesn't work and where it does work. >> [...] >> >>>   <devices> >>>     <emulator>/usr/libexec/qemu-kvm</emulator> >>>     <disk type='file' device='disk'> >>>       <driver name='qemu' type='qcow2' cache='none' discard='unmap'/> >>>       <source file='/00-VMs/ubusrv1.qcow2'/> >>>       <target dev='vda' bus='virtio'/> >>>       <address type='pci' domain='0x0000' bus='0x04' slot='0x00' >>> function='0x0'/> >>>     </disk> >>> ... >>> >>> When I add encryption to <disk> & use encrypted qcow2 then VM fails as I >>> described. >> I specifically asked for '*both*' XMLs. The working one. And the >> non-working one. >> > In case it might not be clear - which in my mind should not have been as > it's simple, sure only in my mind - it is: > all guests use almost identical "template" with obvious differences, such as > names/paths, hw adresses, now... > > - tree guests have used 'pflash' in <os> from the beginning, always. > > and I point to that as only those three guest fail to boot after their > qcow2s were encrypted, just like all other VM's were, but those other VM's > start & boot okey. > If in those three VMs I use original, non-encrypted qcow2 - without changing > anything else in XML definition but 'encryption' relevant in <disk> - then > those VMs start & boot just fine. Well, your problem is then most likely with the guest image or the way how you've created/converted them. I've ran a few experiments and my machines that I've converted to encrypted qcows run properly with uefi. > BdsDxe: failed to load Boot0001 "Uefi Misc Device" from PciRoot > (0x0)/Pci(0x2,0x3)/Pci(0x0,0x0): Not found The error seems to indicate that the boot entry was not found, but the disk encryption is transparent when the disk is decrypted by qemu, so there should be no difference between an encrypted and unencrypted image. Thus why I suspect that the guest image is somehow not properly converted. Specifically I've managed to get this error when I had an empty encrypted image used with a VM.