[libvirt-users] uefi built from tiancore via edk2 can't persist boot changes
Apologies if this has been gone over, but I believe I have checked the intertubes more than a bit..... I am using libvirt and have vms booting under an OVMF.fd to use an efi firmware. I can create vms, linux ubuntu, and they will boot up. However, everytime I reboot am I dropped into the default efi shell provide by the tianocore build. Then I must walk the FS to the booting efi app and run, in this case grubx64.efi, to actually finish booting the host. I had tried adding boot entries with efibootmgr within the OS and also bcfg with the efi shell. I get no errors when adding an entry, and the new entry shows up and I can manipulate the entry i.e. set it to next boot and the like. Soon as I reboot I get dumped back to efi shell. Am I missing something? Are the var changes not being stored? Is there someplace to look for an error perhaps? I also can use the same efi bootloader under libvirt to boot a hybrid iso. location of the efi application on the hybrid iso is /EFI/boot/bootx64.efi. Drops me right into my grub menu, after falling thru from EFI FLOPPY1 and EFI FLOPPY2 to EFI DVD So since that was/is working, tried adding that to my efi partition so it has /EFI/boot/bootx64.efi. The thinking being that since I can't add an entry, I will setup for the "known" efi boot path. Of course that didn't work either I am on arch linux: Linux X 4.4.5-1-ARCH #1 SMP PREEMPT Thu Mar 10 07:38:19 CET 2016 x86_64 GNU/Linux local/libvirt 1.3.2-3 API for controlling virtualization engines (openvz,kvm,qemu,virtualbox,xen,etc) local/libvirt-python 1.3.1-1 libvirt python binding UEFI: Shell> ver UEFI Interactive Shell v2.1 EDK II UEFI v2.50 (EDK II, 0x00010000) for this host....my nvram setting look like <os> <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type> <loader readonly='yes' type='pflash'>/home/xyz/OVMF.fd</loader> <nvram template='/usr/share/OVMF/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/X_VARS.fd</nvram> <boot dev='hd'/> </os>
On Mon, 2016-03-21 at 13:53 -0500, jsl6uy js16uy wrote:
for this host....my nvram setting look like <os> <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type> <loader readonly='yes' type='pflash'>/home/xyz/OVMF.fd</loader> <nvram template='/usr/share/OVMF/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/X_VARS.fd</nvram> <boot dev='hd'/> </os>
Have you tried deleting the *VARS.fd in /var/lib/libvirt/qemu/nvram and changing the 'template' attribute so that it points to the *VARS.fd file you should have in your edk2 source tree, eg. the one matching the OVMF.fd you're using to boot? Not sure if that would help, but it's definitely worth a shot :) Cheers. -- Andrea Bolognani Software Engineer - Virtualization Team
Thanks very much for the help! Unfortunately I did try that previously and was still dumped into the efi shell. Looking round again, I didn't build my OVMF with secure boot enabled. Going to see if I get any joy rebuilding with secure boot support. thanks again On Tue, Mar 22, 2016 at 3:35 AM, Andrea Bolognani <abologna@redhat.com> wrote:
On Mon, 2016-03-21 at 13:53 -0500, jsl6uy js16uy wrote:
for this host....my nvram setting look like <os> <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type> <loader readonly='yes' type='pflash'>/home/xyz/OVMF.fd</loader> <nvram template='/usr/share/OVMF/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/X_VARS.fd</nvram> <boot dev='hd'/> </os>
Have you tried deleting the *VARS.fd in /var/lib/libvirt/qemu/nvram and changing the 'template' attribute so that it points to the *VARS.fd file you should have in your edk2 source tree, eg. the one matching the OVMF.fd you're using to boot?
Not sure if that would help, but it's definitely worth a shot :)
Cheers.
-- Andrea Bolognani Software Engineer - Virtualization Team
On 03/21/16 19:53, jsl6uy js16uy wrote:
Apologies if this has been gone over, but I believe I have checked the intertubes more than a bit.....
I am using libvirt and have vms booting under an OVMF.fd to use an efi firmware. I can create vms, linux ubuntu, and they will boot up. However, everytime I reboot am I dropped into the default efi shell provide by the tianocore build. Then I must walk the FS to the booting efi app and run, in this case grubx64.efi, to actually finish booting the host. I had tried adding boot entries with efibootmgr within the OS and also bcfg with the efi shell. I get no errors when adding an entry, and the new entry shows up and I can manipulate the entry i.e. set it to next boot and the like. Soon as I reboot I get dumped back to efi shell.
Am I missing something? Are the var changes not being stored? Is there someplace to look for an error perhaps?
I also can use the same efi bootloader under libvirt to boot a hybrid iso. location of the efi application on the hybrid iso is /EFI/boot/bootx64.efi. Drops me right into my grub menu, after falling thru from EFI FLOPPY1 and EFI FLOPPY2 to EFI DVD So since that was/is working, tried adding that to my efi partition so it has /EFI/boot/bootx64.efi. The thinking being that since I can't add an entry, I will setup for the "known" efi boot path. Of course that didn't work either
I am on arch linux:
Linux X 4.4.5-1-ARCH #1 SMP PREEMPT Thu Mar 10 07:38:19 CET 2016 x86_64 GNU/Linux local/libvirt 1.3.2-3 API for controlling virtualization engines (openvz,kvm,qemu,virtualbox,xen,etc) local/libvirt-python 1.3.1-1 libvirt python binding
UEFI: Shell> ver UEFI Interactive Shell v2.1 EDK II UEFI v2.50 (EDK II, 0x00010000)
for this host....my nvram setting look like <os> <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type> <loader readonly='yes' type='pflash'>/home/xyz/OVMF.fd</loader> <nvram template='/usr/share/OVMF/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/X_VARS.fd</nvram> <boot dev='hd'/> </os>
Your domain XML is wrong. You can use OVMF with libvirt in three setups. (1) worst setup: map the unified OVMF.fd file (which includes variable store and fimrware executable) as ROM. I don't want people to use this, ever, so although libvirt does support it, I'm not going to share the XML snippet that implements this. (2) a bit less wrong setup (but still pretty wrong): map the unified OVMF.fd file as writeable pflash. This is bad because (a) you can't share the unified OVMF.fd file between guests, and (b) whenever you rebuild the OVMF.fd file, the guest that is pointed to it will lose all of its persistent variables. (3) the right setup: you almost got it right, but the <loader> element in this setup must not point to the unified OVMF.fd file. It must point to the executable-only part, the OVMF_CODE.fd file. So here's what you need: <os> <loader readonly='yes' type='pflash' >/your/build/output/OVMF_CODE.fd</loader> <nvram template='/your/build/output/OVMF_VARS.fd' >/var/lib/libvirt/qemu/nvram/guest_name_VARS.fd</nvram> </os> This allows the firmware binary to be upgraded centrally, while every VM can keep its private varstore. The virt-install(1) manual describes this, under the example --boot loader=/.../OVMF_CODE.fd,loader_ro=yes,loader_type=pflash,nvram_template=/.../OVMF_VARS.fd The way you specified it implies that the varstore range will be mapped as read-only pflash directly from the unified OVMF.fd file, which happens to make the varstore non-writeable. In addition, the real varstore file will be mapped below *that*, which is completely bogus. (4) Instead of <boot> elements in the <os> element, <boot order='...'/> elements are preferred under <disk> and friends. (5) You can (and are encouraged to) look at the OVMF log. If you want to capture it, you should implement the following changes: - add the following attribute to the root element of the domain XML: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' <-- this one
- add the following XML fragment near the end of the domain XML, as a direct child of the <domain> element: <qemu:arg value='-global'/> <qemu:arg value='isa-debugcon.iobase=0x402'/> <qemu:arg value='-debugcon'/> <qemu:arg value='file:/tmp/guest_name.log'/> This will send the OVMF log to /tmp/guest_name.log. (6) If you are already missing a boot option to boot your installed OS, you might have to reestablish that boot option manually (after the above changes) -- add the option, and place it first in the boot order. You can use efibootmgr for this, or the BCFG command, or simply the firmware TUI. I could go into why, but this time I won't. If you are interested, you can mostly find the reason in the OVMF whitepaper <http://people.redhat.com/~lersek/ovmf-whitepaper-c770f8c.txt>, section "Platform-specific boot policy", bullet (b), "Auto-generation of boot options". The point is, boot options pointing to arbitrary boot loader UEFI applications on non-removable media cannot be autogenerated, hence OVMF cannot filter them in. If you install the guest OS with the correct domain XML in the first place, then things will work. Some UEFI OSes also install "system rescue" utilities to the target non-removable media, under the default removable media pathname. The idea is that if you lose your "main" boot option (pointing to an arbitrary OS boot loader), then the firmware can still auto-generate a boot option for this fallback utility, and the fallback utility can restore the OS-specific boot loader's boot option. You can read more about this at <https://blog.uncooperative.org/blog/2014/02/06/the-efi-system-partition/>. Summary: - use a domain XML like described in (3) and (4) - when in doubt, capture the OVMF debug log as described in (5) Thanks Laszlo
Thanks VERY MUCH for all the info and help! Apologies for the extreme delay. I got distracted by other threads that forced out this work to later date. Also some frustration as well. ;) I completely missed this update. Apologies and thanks Will be diving back into this shortly! On Thu, Mar 24, 2016 at 3:57 PM, Laszlo Ersek <lersek@redhat.com> wrote:
On 03/21/16 19:53, jsl6uy js16uy wrote:
Apologies if this has been gone over, but I believe I have checked the intertubes more than a bit.....
I am using libvirt and have vms booting under an OVMF.fd to use an efi firmware. I can create vms, linux ubuntu, and they will boot up. However, everytime I reboot am I dropped into the default efi shell provide by the tianocore build. Then I must walk the FS to the booting efi app and run, in this case grubx64.efi, to actually finish booting the host. I had tried adding boot entries with efibootmgr within the OS and also bcfg with the efi shell. I get no errors when adding an entry, and the new entry shows up and I can manipulate the entry i.e. set it to next boot and the like. Soon as I reboot I get dumped back to efi shell.
Am I missing something? Are the var changes not being stored? Is there someplace to look for an error perhaps?
I also can use the same efi bootloader under libvirt to boot a hybrid iso. location of the efi application on the hybrid iso is /EFI/boot/bootx64.efi. Drops me right into my grub menu, after falling thru from EFI FLOPPY1 and EFI FLOPPY2 to EFI DVD So since that was/is working, tried adding that to my efi partition so it has /EFI/boot/bootx64.efi. The thinking being that since I can't add an entry, I will setup for the "known" efi boot path. Of course that didn't work either
I am on arch linux:
Linux X 4.4.5-1-ARCH #1 SMP PREEMPT Thu Mar 10 07:38:19 CET 2016 x86_64 GNU/Linux local/libvirt 1.3.2-3 API for controlling virtualization engines (openvz,kvm,qemu,virtualbox,xen,etc) local/libvirt-python 1.3.1-1 libvirt python binding
UEFI: Shell> ver UEFI Interactive Shell v2.1 EDK II UEFI v2.50 (EDK II, 0x00010000)
for this host....my nvram setting look like <os> <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type> <loader readonly='yes' type='pflash'>/home/xyz/OVMF.fd</loader> <nvram
template='/usr/share/OVMF/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/X_VARS.fd</nvram>
<boot dev='hd'/> </os>
Your domain XML is wrong. You can use OVMF with libvirt in three setups.
(1) worst setup: map the unified OVMF.fd file (which includes variable store and fimrware executable) as ROM. I don't want people to use this, ever, so although libvirt does support it, I'm not going to share the XML snippet that implements this.
(2) a bit less wrong setup (but still pretty wrong): map the unified OVMF.fd file as writeable pflash. This is bad because (a) you can't share the unified OVMF.fd file between guests, and (b) whenever you rebuild the OVMF.fd file, the guest that is pointed to it will lose all of its persistent variables.
(3) the right setup: you almost got it right, but the <loader> element in this setup must not point to the unified OVMF.fd file. It must point to the executable-only part, the OVMF_CODE.fd file. So here's what you need:
<os> <loader readonly='yes' type='pflash' >/your/build/output/OVMF_CODE.fd</loader> <nvram template='/your/build/output/OVMF_VARS.fd' >/var/lib/libvirt/qemu/nvram/guest_name_VARS.fd</nvram> </os>
This allows the firmware binary to be upgraded centrally, while every VM can keep its private varstore.
The virt-install(1) manual describes this, under the example
--boot
loader=/.../OVMF_CODE.fd,loader_ro=yes,loader_type=pflash,nvram_template=/.../OVMF_VARS.fd
The way you specified it implies that the varstore range will be mapped as read-only pflash directly from the unified OVMF.fd file, which happens to make the varstore non-writeable. In addition, the real varstore file will be mapped below *that*, which is completely bogus.
(4) Instead of <boot> elements in the <os> element, <boot order='...'/> elements are preferred under <disk> and friends.
(5) You can (and are encouraged to) look at the OVMF log. If you want to capture it, you should implement the following changes:
- add the following attribute to the root element of the domain XML:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' <-- this one
- add the following XML fragment near the end of the domain XML, as a direct child of the <domain> element:
<qemu:arg value='-global'/> <qemu:arg value='isa-debugcon.iobase=0x402'/> <qemu:arg value='-debugcon'/> <qemu:arg value='file:/tmp/guest_name.log'/>
This will send the OVMF log to /tmp/guest_name.log.
(6) If you are already missing a boot option to boot your installed OS, you might have to reestablish that boot option manually (after the above changes) -- add the option, and place it first in the boot order. You can use efibootmgr for this, or the BCFG command, or simply the firmware TUI.
I could go into why, but this time I won't. If you are interested, you can mostly find the reason in the OVMF whitepaper <http://people.redhat.com/~lersek/ovmf-whitepaper-c770f8c.txt>, section "Platform-specific boot policy", bullet (b), "Auto-generation of boot options". The point is, boot options pointing to arbitrary boot loader UEFI applications on non-removable media cannot be autogenerated, hence OVMF cannot filter them in. If you install the guest OS with the correct domain XML in the first place, then things will work.
Some UEFI OSes also install "system rescue" utilities to the target non-removable media, under the default removable media pathname. The idea is that if you lose your "main" boot option (pointing to an arbitrary OS boot loader), then the firmware can still auto-generate a boot option for this fallback utility, and the fallback utility can restore the OS-specific boot loader's boot option. You can read more about this at <https://blog.uncooperative.org/blog/2014/02/06/the-efi-system-partition/
.
Summary: - use a domain XML like described in (3) and (4) - when in doubt, capture the OVMF debug log as described in (5)
Thanks Laszlo
Thanks again works like a charm! I will also move to use the boot order directives instead of the os section so my issues were addressed specifically by (3) will move to (4) for best practices Best Regards On Wed, Apr 20, 2016 at 1:27 PM, jsl6uy js16uy <js16uy@gmail.com> wrote:
Thanks VERY MUCH for all the info and help! Apologies for the extreme delay. I got distracted by other threads that forced out this work to later date. Also some frustration as well. ;) I completely missed this update. Apologies and thanks Will be diving back into this shortly!
On Thu, Mar 24, 2016 at 3:57 PM, Laszlo Ersek <lersek@redhat.com> wrote:
On 03/21/16 19:53, jsl6uy js16uy wrote:
Apologies if this has been gone over, but I believe I have checked the intertubes more than a bit.....
I am using libvirt and have vms booting under an OVMF.fd to use an efi firmware. I can create vms, linux ubuntu, and they will boot up. However, everytime I reboot am I dropped into the default efi shell provide by the tianocore build. Then I must walk the FS to the booting efi app and run, in this case grubx64.efi, to actually finish booting the host. I had tried adding boot entries with efibootmgr within the OS and also bcfg with the efi shell. I get no errors when adding an entry, and the new entry shows up and I can manipulate the entry i.e. set it to next boot and the like. Soon as I reboot I get dumped back to efi shell.
Am I missing something? Are the var changes not being stored? Is there someplace to look for an error perhaps?
I also can use the same efi bootloader under libvirt to boot a hybrid iso. location of the efi application on the hybrid iso is /EFI/boot/bootx64.efi. Drops me right into my grub menu, after falling thru from EFI FLOPPY1 and EFI FLOPPY2 to EFI DVD So since that was/is working, tried adding that to my efi partition so it has /EFI/boot/bootx64.efi. The thinking being that since I can't add an entry, I will setup for the "known" efi boot path. Of course that didn't work either
I am on arch linux:
Linux X 4.4.5-1-ARCH #1 SMP PREEMPT Thu Mar 10 07:38:19 CET 2016 x86_64 GNU/Linux local/libvirt 1.3.2-3 API for controlling virtualization engines (openvz,kvm,qemu,virtualbox,xen,etc) local/libvirt-python 1.3.1-1 libvirt python binding
UEFI: Shell> ver UEFI Interactive Shell v2.1 EDK II UEFI v2.50 (EDK II, 0x00010000)
for this host....my nvram setting look like <os> <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type> <loader readonly='yes' type='pflash'>/home/xyz/OVMF.fd</loader> <nvram
template='/usr/share/OVMF/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/X_VARS.fd</nvram>
<boot dev='hd'/> </os>
Your domain XML is wrong. You can use OVMF with libvirt in three setups.
(1) worst setup: map the unified OVMF.fd file (which includes variable store and fimrware executable) as ROM. I don't want people to use this, ever, so although libvirt does support it, I'm not going to share the XML snippet that implements this.
(2) a bit less wrong setup (but still pretty wrong): map the unified OVMF.fd file as writeable pflash. This is bad because (a) you can't share the unified OVMF.fd file between guests, and (b) whenever you rebuild the OVMF.fd file, the guest that is pointed to it will lose all of its persistent variables.
(3) the right setup: you almost got it right, but the <loader> element in this setup must not point to the unified OVMF.fd file. It must point to the executable-only part, the OVMF_CODE.fd file. So here's what you need:
<os> <loader readonly='yes' type='pflash' >/your/build/output/OVMF_CODE.fd</loader> <nvram template='/your/build/output/OVMF_VARS.fd' >/var/lib/libvirt/qemu/nvram/guest_name_VARS.fd</nvram> </os>
This allows the firmware binary to be upgraded centrally, while every VM can keep its private varstore.
The virt-install(1) manual describes this, under the example
--boot
loader=/.../OVMF_CODE.fd,loader_ro=yes,loader_type=pflash,nvram_template=/.../OVMF_VARS.fd
The way you specified it implies that the varstore range will be mapped as read-only pflash directly from the unified OVMF.fd file, which happens to make the varstore non-writeable. In addition, the real varstore file will be mapped below *that*, which is completely bogus.
(4) Instead of <boot> elements in the <os> element, <boot order='...'/> elements are preferred under <disk> and friends.
(5) You can (and are encouraged to) look at the OVMF log. If you want to capture it, you should implement the following changes:
- add the following attribute to the root element of the domain XML:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' <-- this one
- add the following XML fragment near the end of the domain XML, as a direct child of the <domain> element:
<qemu:arg value='-global'/> <qemu:arg value='isa-debugcon.iobase=0x402'/> <qemu:arg value='-debugcon'/> <qemu:arg value='file:/tmp/guest_name.log'/>
This will send the OVMF log to /tmp/guest_name.log.
(6) If you are already missing a boot option to boot your installed OS, you might have to reestablish that boot option manually (after the above changes) -- add the option, and place it first in the boot order. You can use efibootmgr for this, or the BCFG command, or simply the firmware TUI.
I could go into why, but this time I won't. If you are interested, you can mostly find the reason in the OVMF whitepaper <http://people.redhat.com/~lersek/ovmf-whitepaper-c770f8c.txt>, section "Platform-specific boot policy", bullet (b), "Auto-generation of boot options". The point is, boot options pointing to arbitrary boot loader UEFI applications on non-removable media cannot be autogenerated, hence OVMF cannot filter them in. If you install the guest OS with the correct domain XML in the first place, then things will work.
Some UEFI OSes also install "system rescue" utilities to the target non-removable media, under the default removable media pathname. The idea is that if you lose your "main" boot option (pointing to an arbitrary OS boot loader), then the firmware can still auto-generate a boot option for this fallback utility, and the fallback utility can restore the OS-specific boot loader's boot option. You can read more about this at <https://blog.uncooperative.org/blog/2014/02/06/the-efi-system-partition/
.
Summary: - use a domain XML like described in (3) and (4) - when in doubt, capture the OVMF debug log as described in (5)
Thanks Laszlo
participants (3)
-
Andrea Bolognani -
jsl6uy js16uy -
Laszlo Ersek