On Mon, Nov 18, 2024 at 01:46:57PM +0000, Andrea Bolognani wrote:
On Fri, Nov 01, 2024 at 02:19:46PM +0000, Daniel P. Berrangé wrote:
On Fri, Nov 01, 2024 at 09:54:14AM -0400, Lars Kellogg-Stedman wrote:
Hey folks,
I'm running libvirt 10.1.0/qemu-system-x86-core-9.0.1-1.fc40.x86_64 on Fedora 40. I'm trying to boot an Ubuntu image in UEFI mode, like this:
virt-install -r 2048 -n ubuntu.virt --os-variant ubuntu24.04 \ --disk pool=default,size=10,backing_store=mantic-server-cloudimg-amd64.img,backing_format=qcow2 \ --cloud-init root-ssh-key=$HOME/.ssh/id_ed25519.pub \ --boot uefi
This results in the domain booting up and then immediately resetting:
BdsDxe: loading Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x2,0x3)/Pci(0x0,0x0) BdsDxe: starting Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x2,0x3)/Pci(0x0,0x0) Reset System
Domain creation completed.
At this point, the machine is actually powered down and needs to be restarted manually:
virsh start ubuntu.virt
This works fine, and the domain boots successfully, but now the cloud-init metadata provided by the `--cloud-init` option to `virt-install` is no longer available (because this is no longer the initial, virt-install managed boot of the domain).
What is causing the firmware to reset the system when it first boots?
This will be shim configuring the NVRAM bootloade entries.
You've taken a pre-installed disk image, but have not been provided the associated UEFI NVRAM that was populated when the disk was first installed. This is normal, since the NVRAM created at the time is not guaranteed to be compatible with your EFI binary.
Your VM is booting with a clean NVRAM, and no bootloader entries will be present. Thus EFI launches the default binary $ESP/EFI/BOOT/BOOTX64.EFI, which is a copy of shim.
Shim sees that has been launched with no bootloader entries present, and so will then invoke $ESP/EFI/BOOT/fbx64.efi which will read the info in $ESP/<vendor>/BOOTX64.CSV file and create boot loader entries in the EFI NVRAM. Then it will trigger a system reset, which is the one you see. <vendor> in your case is probably 'ubuntu' I guess.
When you run virsh start, EFI will see bootloader entries and so instead of $ESP/EFI/BOOT/BOOTX64.EFI, it will probably be launching $ESP/EFI/ubuntu/shim.efi, which will then run grub, etc, as normal The latter will be done forever more, unless you purge the NVRAM of all bootloader entries.
Ideally virt-install would be aware of this and automatically restart the VM after shim populates the NVRAM. Depends if we can come up with a relativel foolproof way to identify this situation.
The need for a full reset (as opposed to shim just creating the missing entries and moving on) is apparently triggered by the presence of a TPM, which these days we add by default when UEFI is used.
virt-manager has added some code to deal with this very situation:
https://github.com/virt-manager/virt-manager/commit/ec434948a8384541c56bfa04...
Unfortunately it's not part of any release yet, but once it is the scenario we're discussion should no longer present itself.
The approach in that patch of disabling the TPM is pretty undesirable as a long term solution to this problem, because it assumes there are no ill effects from disabling the TPM, which is dubious Tieing it to use of --cloud-init seems fragile as well, as I'd expect extra reboots could happen in non-cloud-init cases too. That all said, doing anything better than that is pretty complicated With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|