[PATCH 0/4] conf, qemu: add loader type='none'
Hi, Fedora Rawhide for RISC-V requires '-bios none' to properly boot because its kernel is overwriting the default OpenSBI binary QEMU uses, causing the following error: $ sudo ./run tools/virsh start --console riscv-fedora error: Failed to start domain 'riscv-fedora' error: internal error: process exited while connecting to monitor: 2023-03-20T17:31:02.650862Z qemu-system-riscv64: Some ROM regions are overlapping These ROM regions might have been loaded by direct user request or by default. They could be BIOS/firmware images, a guest kernel, initrd or some other file loaded into guest memory. Check whether you intended to load all this guest code, and whether it has been built to load to the correct addresses. Other archs, such as PPC64 pseries, also requires "-bios none" if the user wants QEMU to not load any default firmware. At this moment libvirt doesn't support this option in the official API, meaning we need to go to the <qemu:commandline> route to allow the domain to boot, tainting it. And with a chance of weird interactions with firmware autoselect. These patches add official XML support for '-bios none' for QEMU domains using a XML as follows: <os> <loader type='none'/> (...) </os> The pre-conditions of this format are (1) no loader->path and (2) only manual autoselect. Everything else is already covered by libvirt as corner cases of existing firmware features. Daniel Henrique Barboza (4): conf: add loader type 'none' qemu: handle bios 'none' case in qemuFirmwareFillDomain() qemu, tests: add -bios none command line docs: Document loader 'none' attribute docs/formatdomain.rst | 7 +++++ src/conf/domain_conf.c | 5 +-- src/conf/domain_validate.c | 2 +- src/conf/schemas/domaincommon.rng | 1 + src/qemu/qemu_command.c | 6 ++++ src/qemu/qemu_firmware.c | 10 ++++++ .../firmware-bios-none.riscv64-latest.args | 31 +++++++++++++++++++ tests/qemuxml2argvdata/firmware-bios-none.xml | 18 +++++++++++ tests/qemuxml2argvtest.c | 2 ++ 9 files changed, 79 insertions(+), 3 deletions(-) create mode 100644 tests/qemuxml2argvdata/firmware-bios-none.riscv64-latest.args create mode 100644 tests/qemuxml2argvdata/firmware-bios-none.xml -- 2.39.2
Today, trying to boot a RISC-V Fedora Rawhide image in a RISC-V QEMU domain results in the following error: ==== error: Failed to start domain 'riscv-fedora' error: internal error: process exited while connecting to monitor: 2023-03-20T17:31:02.650862Z qemu-system-riscv64: Some ROM regions are overlapping These ROM regions might have been loaded by direct user request or by default. They could be BIOS/firmware images, a guest kernel, initrd or some other file loaded into guest memory. Check whether you intended to load all this guest code, and whether it has been built to load to the correct addresses. ==== This happens because the default RISC-V QEMU firmware, OpenSBI, is always loaded unless "-bios none" is passed in the command line, and the Fedora Rawhide guest kernel has its own ROM. Other machines such as PPC64 'pseries' shows the same behavior: the default firmware is always loaded unless specified otherwise with the '-bios' option. At this moment we don't have XML support for '-bios none'. Using "<qemu:commandline>" works but it will leave the domain in a tainted state. It'll also have unpredictable consequences with the autoselect firmware feature libvirt has. Add a new loader type 'none' that, if no path is specified and we're not use firmware autoselection, will tell QEMU that no default firmware should be used: <os> <loader type='none'/> (...) </os> Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> --- src/conf/domain_conf.c | 5 +++-- src/conf/domain_validate.c | 2 +- src/conf/schemas/domaincommon.rng | 1 + 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 9ef50c818b..d7a5cd094b 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -16837,7 +16837,7 @@ virDomainLoaderDefParseXMLLoader(virDomainLoaderDef *loader, return -1; if (virXMLPropEnum(loaderNode, "type", virDomainLoaderTypeFromString, - VIR_XML_PROP_NONZERO, &loader->type) < 0) + VIR_XML_PROP_NONE, &loader->type) < 0) return -1; if (!(loader->path = virXMLNodeContentString(loaderNode))) @@ -26259,7 +26259,8 @@ virDomainLoaderDefFormat(virBuffer *buf, virBufferAsprintf(&loaderAttrBuf, " secure='%s'", virTristateBoolTypeToString(loader->secure)); - if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE) + if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE || + (loader->type == VIR_DOMAIN_LOADER_TYPE_NONE && !loader->path)) virBufferAsprintf(&loaderAttrBuf, " type='%s'", virDomainLoaderTypeToString(loader->type)); diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index 5fb2d4971c..1fea9c8e6b 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -1664,7 +1664,7 @@ virDomainDefOSValidate(const virDomainDef *def, if (!loader) return 0; - if (!loader->path) { + if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE && !loader->path) { virReportError(VIR_ERR_XML_DETAIL, "%s", _("no loader path specified and firmware auto selection disabled")); return -1; diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index 6158ed79ac..c96658e3a2 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -341,6 +341,7 @@ <choice> <value>rom</value> <value>pflash</value> + <value>none</value> </choice> </attribute> </optional> -- 2.39.2
On Wed, Mar 22, 2023 at 06:10:18AM -0300, Daniel Henrique Barboza wrote:
Today, trying to boot a RISC-V Fedora Rawhide image in a RISC-V QEMU domain results in the following error:
==== error: Failed to start domain 'riscv-fedora' error: internal error: process exited while connecting to monitor: 2023-03-20T17:31:02.650862Z qemu-system-riscv64: Some ROM regions are overlapping These ROM regions might have been loaded by direct user request or by default. They could be BIOS/firmware images, a guest kernel, initrd or some other file loaded into guest memory. Check whether you intended to load all this guest code, and whether it has been built to load to the correct addresses. ====
This happens because the default RISC-V QEMU firmware, OpenSBI, is always loaded unless "-bios none" is passed in the command line, and the Fedora Rawhide guest kernel has its own ROM. Other machines such as PPC64 'pseries' shows the same behavior: the default firmware is always loaded unless specified otherwise with the '-bios' option.
At this moment we don't have XML support for '-bios none'. Using "<qemu:commandline>" works but it will leave the domain in a tainted state. It'll also have unpredictable consequences with the autoselect firmware feature libvirt has.
Add a new loader type 'none' that, if no path is specified and we're not use firmware autoselection, will tell QEMU that no default firmware should be used:
<os> <loader type='none'/> (...) </os>
Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> --- src/conf/domain_conf.c | 5 +++-- src/conf/domain_validate.c | 2 +- src/conf/schemas/domaincommon.rng | 1 + 3 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 9ef50c818b..d7a5cd094b 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -16837,7 +16837,7 @@ virDomainLoaderDefParseXMLLoader(virDomainLoaderDef *loader, return -1;
if (virXMLPropEnum(loaderNode, "type", virDomainLoaderTypeFromString, - VIR_XML_PROP_NONZERO, &loader->type) < 0) + VIR_XML_PROP_NONE, &loader->type) < 0) return -1;
if (!(loader->path = virXMLNodeContentString(loaderNode))) @@ -26259,7 +26259,8 @@ virDomainLoaderDefFormat(virBuffer *buf, virBufferAsprintf(&loaderAttrBuf, " secure='%s'", virTristateBoolTypeToString(loader->secure));
- if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE) + if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE || + (loader->type == VIR_DOMAIN_LOADER_TYPE_NONE && !loader->path)) virBufferAsprintf(&loaderAttrBuf, " type='%s'", virDomainLoaderTypeToString(loader->type));
VIR_DOMAIN_LOADER_TYPE_NONE is a constant we're already using internally to track when the user has not given any <loader> element at all. It is not a good idea to overload for this for the user explicitly requesting a config. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 3/22/23 06:25, Daniel P. Berrangé wrote:
On Wed, Mar 22, 2023 at 06:10:18AM -0300, Daniel Henrique Barboza wrote:
Today, trying to boot a RISC-V Fedora Rawhide image in a RISC-V QEMU domain results in the following error:
==== error: Failed to start domain 'riscv-fedora' error: internal error: process exited while connecting to monitor: 2023-03-20T17:31:02.650862Z qemu-system-riscv64: Some ROM regions are overlapping These ROM regions might have been loaded by direct user request or by default. They could be BIOS/firmware images, a guest kernel, initrd or some other file loaded into guest memory. Check whether you intended to load all this guest code, and whether it has been built to load to the correct addresses. ====
This happens because the default RISC-V QEMU firmware, OpenSBI, is always loaded unless "-bios none" is passed in the command line, and the Fedora Rawhide guest kernel has its own ROM. Other machines such as PPC64 'pseries' shows the same behavior: the default firmware is always loaded unless specified otherwise with the '-bios' option.
At this moment we don't have XML support for '-bios none'. Using "<qemu:commandline>" works but it will leave the domain in a tainted state. It'll also have unpredictable consequences with the autoselect firmware feature libvirt has.
Add a new loader type 'none' that, if no path is specified and we're not use firmware autoselection, will tell QEMU that no default firmware should be used:
<os> <loader type='none'/> (...) </os>
Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> --- src/conf/domain_conf.c | 5 +++-- src/conf/domain_validate.c | 2 +- src/conf/schemas/domaincommon.rng | 1 + 3 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 9ef50c818b..d7a5cd094b 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -16837,7 +16837,7 @@ virDomainLoaderDefParseXMLLoader(virDomainLoaderDef *loader, return -1;
if (virXMLPropEnum(loaderNode, "type", virDomainLoaderTypeFromString, - VIR_XML_PROP_NONZERO, &loader->type) < 0) + VIR_XML_PROP_NONE, &loader->type) < 0) return -1;
if (!(loader->path = virXMLNodeContentString(loaderNode))) @@ -26259,7 +26259,8 @@ virDomainLoaderDefFormat(virBuffer *buf, virBufferAsprintf(&loaderAttrBuf, " secure='%s'", virTristateBoolTypeToString(loader->secure));
- if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE) + if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE || + (loader->type == VIR_DOMAIN_LOADER_TYPE_NONE && !loader->path)) virBufferAsprintf(&loaderAttrBuf, " type='%s'", virDomainLoaderTypeToString(loader->type));
VIR_DOMAIN_LOADER_TYPE_NONE is a constant we're already using internally to track when the user has not given any <loader> element at all.
It is not a good idea to overload for this for the user explicitly requesting a config.
Makes sense. Any name suggestions for this new constant? VIR_DOMAIN_LOADER_TYPE_SKIP is the first thing that comes to mind. If we want to keep it consistent we should also change this new type to <loader type='skip'/> as well. Thanks, Daniel
With regards, Daniel
On 3/22/23 06:42, Daniel Henrique Barboza wrote:
On 3/22/23 06:25, Daniel P. Berrangé wrote:
On Wed, Mar 22, 2023 at 06:10:18AM -0300, Daniel Henrique Barboza wrote:
Today, trying to boot a RISC-V Fedora Rawhide image in a RISC-V QEMU domain results in the following error:
==== error: Failed to start domain 'riscv-fedora' error: internal error: process exited while connecting to monitor: 2023-03-20T17:31:02.650862Z qemu-system-riscv64: Some ROM regions are overlapping These ROM regions might have been loaded by direct user request or by default. They could be BIOS/firmware images, a guest kernel, initrd or some other file loaded into guest memory. Check whether you intended to load all this guest code, and whether it has been built to load to the correct addresses. ====
This happens because the default RISC-V QEMU firmware, OpenSBI, is always loaded unless "-bios none" is passed in the command line, and the Fedora Rawhide guest kernel has its own ROM. Other machines such as PPC64 'pseries' shows the same behavior: the default firmware is always loaded unless specified otherwise with the '-bios' option.
At this moment we don't have XML support for '-bios none'. Using "<qemu:commandline>" works but it will leave the domain in a tainted state. It'll also have unpredictable consequences with the autoselect firmware feature libvirt has.
Add a new loader type 'none' that, if no path is specified and we're not use firmware autoselection, will tell QEMU that no default firmware should be used:
<os> <loader type='none'/> (...) </os>
Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> --- src/conf/domain_conf.c | 5 +++-- src/conf/domain_validate.c | 2 +- src/conf/schemas/domaincommon.rng | 1 + 3 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 9ef50c818b..d7a5cd094b 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -16837,7 +16837,7 @@ virDomainLoaderDefParseXMLLoader(virDomainLoaderDef *loader, return -1; if (virXMLPropEnum(loaderNode, "type", virDomainLoaderTypeFromString, - VIR_XML_PROP_NONZERO, &loader->type) < 0) + VIR_XML_PROP_NONE, &loader->type) < 0) return -1; if (!(loader->path = virXMLNodeContentString(loaderNode))) @@ -26259,7 +26259,8 @@ virDomainLoaderDefFormat(virBuffer *buf, virBufferAsprintf(&loaderAttrBuf, " secure='%s'", virTristateBoolTypeToString(loader->secure)); - if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE) + if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE || + (loader->type == VIR_DOMAIN_LOADER_TYPE_NONE && !loader->path)) virBufferAsprintf(&loaderAttrBuf, " type='%s'", virDomainLoaderTypeToString(loader->type));
VIR_DOMAIN_LOADER_TYPE_NONE is a constant we're already using internally to track when the user has not given any <loader> element at all.
It is not a good idea to overload for this for the user explicitly requesting a config.
Makes sense. Any name suggestions for this new constant?
VIR_DOMAIN_LOADER_TYPE_SKIP is the first thing that comes to mind. If we want to keep it consistent we should also change this new type to <loader type='skip'/> as well.
VIR_DOMAIN_LOADER_TYPE_OTHER seems more appropriate to indicate that the firmware will be loaded not by QEMU, but other means (e.g. kernel). I'll do that for v2. Daniel
Thanks,
Daniel
With regards, Daniel
On Wed, Mar 22, 2023 at 09:32:21AM -0300, Daniel Henrique Barboza wrote:
On 3/22/23 06:42, Daniel Henrique Barboza wrote:
On 3/22/23 06:25, Daniel P. Berrangé wrote:
On Wed, Mar 22, 2023 at 06:10:18AM -0300, Daniel Henrique Barboza wrote:
+ if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE || + (loader->type == VIR_DOMAIN_LOADER_TYPE_NONE && !loader->path)) virBufferAsprintf(&loaderAttrBuf, " type='%s'", virDomainLoaderTypeToString(loader->type));
VIR_DOMAIN_LOADER_TYPE_NONE is a constant we're already using internally to track when the user has not given any <loader> element at all.
It is not a good idea to overload for this for the user explicitly requesting a config.
Makes sense. Any name suggestions for this new constant?
VIR_DOMAIN_LOADER_TYPE_SKIP is the first thing that comes to mind. If we want to keep it consistent we should also change this new type to <loader type='skip'/> as well.
VIR_DOMAIN_LOADER_TYPE_OTHER seems more appropriate to indicate that the firmware will be loaded not by QEMU, but other means (e.g. kernel).
I'm not sure we really need a new loader type. Can't we just add '-bios none' unconditionally on RISC-V when a loader path or a kernel path have been provided, either manually or via autoselection? -- Andrea Bolognani / Red Hat / Virtualization
On 3/22/23 10:02, Andrea Bolognani wrote:
On Wed, Mar 22, 2023 at 09:32:21AM -0300, Daniel Henrique Barboza wrote:
On 3/22/23 06:42, Daniel Henrique Barboza wrote:
On 3/22/23 06:25, Daniel P. Berrangé wrote:
On Wed, Mar 22, 2023 at 06:10:18AM -0300, Daniel Henrique Barboza wrote:
+ if (loader->type != VIR_DOMAIN_LOADER_TYPE_NONE || + (loader->type == VIR_DOMAIN_LOADER_TYPE_NONE && !loader->path)) virBufferAsprintf(&loaderAttrBuf, " type='%s'", virDomainLoaderTypeToString(loader->type));
VIR_DOMAIN_LOADER_TYPE_NONE is a constant we're already using internally to track when the user has not given any <loader> element at all.
It is not a good idea to overload for this for the user explicitly requesting a config.
Makes sense. Any name suggestions for this new constant?
VIR_DOMAIN_LOADER_TYPE_SKIP is the first thing that comes to mind. If we want to keep it consistent we should also change this new type to <loader type='skip'/> as well.
VIR_DOMAIN_LOADER_TYPE_OTHER seems more appropriate to indicate that the firmware will be loaded not by QEMU, but other means (e.g. kernel).
I'm not sure we really need a new loader type. Can't we just add '-bios none' unconditionally on RISC-V when a loader path or a kernel path have been provided, either manually or via autoselection?
Ubuntu for RISC-V works in libvirt without '-bios none' and with a -kernel path, so it's not like all RISC-V domains requires '-bios none'. Thanks, Daniel
On Wed, Mar 22, 2023 at 10:40:33AM -0300, Daniel Henrique Barboza wrote:
On 3/22/23 10:02, Andrea Bolognani wrote:
On Wed, Mar 22, 2023 at 09:32:21AM -0300, Daniel Henrique Barboza wrote:
VIR_DOMAIN_LOADER_TYPE_OTHER seems more appropriate to indicate that the firmware will be loaded not by QEMU, but other means (e.g. kernel).
I'm not sure we really need a new loader type. Can't we just add '-bios none' unconditionally on RISC-V when a loader path or a kernel path have been provided, either manually or via autoselection?
Ubuntu for RISC-V works in libvirt without '-bios none' and with a -kernel path, so it's not like all RISC-V domains requires '-bios none'.
But does it *not* work with '-bios none'? When -kernel is used QEMU should jump directly into the kernel anyway, so having a BIOS image loaded shouldn't be necessary. I seem to recall that QEMU had specific logic for this. -- Andrea Bolognani / Red Hat / Virtualization
On Wed, Mar 22, 2023 at 06:10:18AM -0300, Daniel Henrique Barboza wrote:
Today, trying to boot a RISC-V Fedora Rawhide image in a RISC-V QEMU domain results in the following error:
==== error: Failed to start domain 'riscv-fedora' error: internal error: process exited while connecting to monitor: 2023-03-20T17:31:02.650862Z qemu-system-riscv64: Some ROM regions are overlapping These ROM regions might have been loaded by direct user request or by default. They could be BIOS/firmware images, a guest kernel, initrd or some other file loaded into guest memory. Check whether you intended to load all this guest code, and whether it has been built to load to the correct addresses. ====
This happens because the default RISC-V QEMU firmware, OpenSBI, is always loaded unless "-bios none" is passed in the command line, and the Fedora Rawhide guest kernel has its own ROM. Other machines such as PPC64 'pseries' shows the same behavior: the default firmware is always loaded unless specified otherwise with the '-bios' option.
What happens when these kernels are used on bare metal ? Presumably bare metal always has firmware, and the kernels successfuly overwrite it ? If so, what makes QEMU special such that it breaks ? With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 3/22/23 10:05, Daniel P. Berrangé wrote:
On Wed, Mar 22, 2023 at 06:10:18AM -0300, Daniel Henrique Barboza wrote:
Today, trying to boot a RISC-V Fedora Rawhide image in a RISC-V QEMU domain results in the following error:
==== error: Failed to start domain 'riscv-fedora' error: internal error: process exited while connecting to monitor: 2023-03-20T17:31:02.650862Z qemu-system-riscv64: Some ROM regions are overlapping These ROM regions might have been loaded by direct user request or by default. They could be BIOS/firmware images, a guest kernel, initrd or some other file loaded into guest memory. Check whether you intended to load all this guest code, and whether it has been built to load to the correct addresses. ====
This happens because the default RISC-V QEMU firmware, OpenSBI, is always loaded unless "-bios none" is passed in the command line, and the Fedora Rawhide guest kernel has its own ROM. Other machines such as PPC64 'pseries' shows the same behavior: the default firmware is always loaded unless specified otherwise with the '-bios' option.
What happens when these kernels are used on bare metal ? Presumably bare metal always has firmware, and the kernels successfuly overwrite it ? If so, what makes QEMU special such that it breaks ?
I'm not sure if the OS overwrites the firmware when running bare metal. Usually they provide different OS images for QEMU/libvirt and bare metal systems, probably to account for these differences. Baking the firmware in the kernel like Fedora Rawhide is doing was important a few years ago when RISC-V QEMU wasn't loading the firmware by default, but now it's getting in the way. All this said, having a look at the recently updated Fedora for RISC-V wiki, the instructions for booting with libvirt and QEMU are different. libvirt [1] is using the '-bios none' attribute with 'virt-install' + a Fedora Rawhide image, but QEMU instructions [2] doesn't use '-bios none' and it's using Fedora 37. At first I don't see any particular reason of why this Fedora 37 image would work on QEMU and not on libvirt. So what t I'll do now is do some testings with libvirt and Fedora 37. If it works then this series becomes way less important. Thanks, Daniel [1] https://fedoraproject.org/wiki/Architectures/RISC-V/Installing#Boot_with_lib... [2] https://fedoraproject.org/wiki/Architectures/RISC-V/Installing#Boot_under_QE...
With regards, Daniel
On Wed, Mar 22, 2023 at 10:36:20AM -0300, Daniel Henrique Barboza wrote:
I'm not sure if the OS overwrites the firmware when running bare metal. Usually they provide different OS images for QEMU/libvirt and bare metal systems, probably to account for these differences. Baking the firmware in the kernel like Fedora Rawhide is doing was important a few years ago when RISC-V QEMU wasn't loading the firmware by default, but now it's getting in the way.
All this said, having a look at the recently updated Fedora for RISC-V wiki, the instructions for booting with libvirt and QEMU are different. libvirt [1] is using the '-bios none' attribute with 'virt-install' + a Fedora Rawhide image, but QEMU instructions [2] doesn't use '-bios none' and it's using Fedora 37.
The libvirt instructions in that page seem to have been updated in the Fedora 30 timeframe, whereas the QEMU instructions appear to be current as of Fedora 37.
At first I don't see any particular reason of why this Fedora 37 image would work on QEMU and not on libvirt. So what t I'll do now is do some testings with libvirt and Fedora 37. If it works then this series becomes way less important.
The libvirt instructions tell you to use --boot kernel=Fedora-Developer-Rawhide-*-fw_payload-uboot-qemu-virt-smode.elf --qemu-commandline='-bios none' (aka -bios none -kernel foo) while the QEMU ones suggest -bios u/usr/share/uboot/qemu-riscv64_spl/u-boot-spl.bin -device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000 Those are completely different approaches to booting the guest OS. The latter is much saner IMO, because it keeps the firmware under control of the host (as is the case for SeaBIOS/edk2) instead of mixing the kernel and the firmware and requiring guest-specific files to be extracted from the disk image before being able to boot. libvirt already provides the ability to pass arbitrary paths to -bios via the <loader type='rom'> element, so making the first part work should just be a matter of pointing it to the correct path. We don't seem to have the -device loader part wired up though, so that would need to be added. Probably as an additional attribute, in the vein of nvram.template? The address would have to be specified as well. Note that the firmware descriptor format already supports u-boot as the firmware type. So in the long run ideally you'd only need to specify <os firmware='uboot'> and, assuming the uboot-images-riscv64 package is installed on the host, everything should just work. -- Andrea Bolognani / Red Hat / Virtualization
On Wed, Mar 22, 2023 at 11:37:10AM -0500, Andrea Bolognani wrote:
On Wed, Mar 22, 2023 at 10:36:20AM -0300, Daniel Henrique Barboza wrote:
I'm not sure if the OS overwrites the firmware when running bare metal. Usually they provide different OS images for QEMU/libvirt and bare metal systems, probably to account for these differences. Baking the firmware in the kernel like Fedora Rawhide is doing was important a few years ago when RISC-V QEMU wasn't loading the firmware by default, but now it's getting in the way.
All this said, having a look at the recently updated Fedora for RISC-V wiki, the instructions for booting with libvirt and QEMU are different. libvirt [1] is using the '-bios none' attribute with 'virt-install' + a Fedora Rawhide image, but QEMU instructions [2] doesn't use '-bios none' and it's using Fedora 37.
The libvirt instructions in that page seem to have been updated in the Fedora 30 timeframe, whereas the QEMU instructions appear to be current as of Fedora 37.
At first I don't see any particular reason of why this Fedora 37 image would work on QEMU and not on libvirt. So what t I'll do now is do some testings with libvirt and Fedora 37. If it works then this series becomes way less important.
The libvirt instructions tell you to use
--boot kernel=Fedora-Developer-Rawhide-*-fw_payload-uboot-qemu-virt-smode.elf --qemu-commandline='-bios none'
(aka -bios none -kernel foo) while the QEMU ones suggest
-bios u/usr/share/uboot/qemu-riscv64_spl/u-boot-spl.bin -device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000
Those are completely different approaches to booting the guest OS. The latter is much saner IMO, because it keeps the firmware under control of the host (as is the case for SeaBIOS/edk2) instead of mixing the kernel and the firmware and requiring guest-specific files to be extracted from the disk image before being able to boot.
libvirt already provides the ability to pass arbitrary paths to -bios via the <loader type='rom'> element, so making the first part work should just be a matter of pointing it to the correct path. We don't seem to have the -device loader part wired up though, so that would need to be added. Probably as an additional attribute, in the vein of nvram.template? The address would have to be specified as well.
Note that the firmware descriptor format already supports u-boot as the firmware type. So in the long run ideally you'd only need to specify
<os firmware='uboot'>
and, assuming the uboot-images-riscv64 package is installed on the host, everything should just work.
And that would bring alignment with other architectures approach, which is preferrable to adding a special hack needed for riscv, because the latter is something that most mgmt apps will forget to use. Of course even better would be for riscv64 to use UEFI :-) With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Wed, Mar 22, 2023 at 04:52:34PM +0000, Daniel P. Berrangé wrote:
On Wed, Mar 22, 2023 at 11:37:10AM -0500, Andrea Bolognani wrote:
Note that the firmware descriptor format already supports u-boot as the firmware type. So in the long run ideally you'd only need to specify
<os firmware='uboot'>
and, assuming the uboot-images-riscv64 package is installed on the host, everything should just work.
And that would bring alignment with other architectures approach, which is preferrable to adding a special hack needed for riscv, because the latter is something that most mgmt apps will forget to use.
Absolutely. We still need support for manual firmware selection and direct kernel boot though.
Of course even better would be for riscv64 to use UEFI :-)
That should already be possible[1], but it's still unclear whether in the long run it will become the de facto standard. Either way, it might take a few more years to get to that point. [1] https://packages.fedoraproject.org/pkgs/edk2/edk2-riscv64/ -- Andrea Bolognani / Red Hat / Virtualization
On 3/22/23 13:37, Andrea Bolognani wrote:
On Wed, Mar 22, 2023 at 10:36:20AM -0300, Daniel Henrique Barboza wrote:
I'm not sure if the OS overwrites the firmware when running bare metal. Usually they provide different OS images for QEMU/libvirt and bare metal systems, probably to account for these differences. Baking the firmware in the kernel like Fedora Rawhide is doing was important a few years ago when RISC-V QEMU wasn't loading the firmware by default, but now it's getting in the way.
All this said, having a look at the recently updated Fedora for RISC-V wiki, the instructions for booting with libvirt and QEMU are different. libvirt [1] is using the '-bios none' attribute with 'virt-install' + a Fedora Rawhide image, but QEMU instructions [2] doesn't use '-bios none' and it's using Fedora 37.
The libvirt instructions in that page seem to have been updated in the Fedora 30 timeframe, whereas the QEMU instructions appear to be current as of Fedora 37.
At first I don't see any particular reason of why this Fedora 37 image would work on QEMU and not on libvirt. So what t I'll do now is do some testings with libvirt and Fedora 37. If it works then this series becomes way less important.
The libvirt instructions tell you to use
--boot kernel=Fedora-Developer-Rawhide-*-fw_payload-uboot-qemu-virt-smode.elf --qemu-commandline='-bios none'
(aka -bios none -kernel foo) while the QEMU ones suggest
-bios u/usr/share/uboot/qemu-riscv64_spl/u-boot-spl.bin -device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000
Those are completely different approaches to booting the guest OS. The latter is much saner IMO, because it keeps the firmware under control of the host (as is the case for SeaBIOS/edk2) instead of mixing the kernel and the firmware and requiring guest-specific files to be extracted from the disk image before being able to boot.
libvirt already provides the ability to pass arbitrary paths to -bios via the <loader type='rom'> element, so making the first part work should just be a matter of pointing it to the correct path. We don't seem to have the -device loader part wired up though, so that would need to be added. Probably as an additional attribute, in the vein of nvram.template? The address would have to be specified as well.
Note that the firmware descriptor format already supports u-boot as the firmware type. So in the long run ideally you'd only need to specify
<os firmware='uboot'>
and, assuming the uboot-images-riscv64 package is installed on the host, everything should just work.
So, I got the chance to try this out. We can remove the '-loader' from the Fedora 37 command line. This: -device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000 Is loading the u-boot image in the address that the 'virt' machine loads the kernel. So in the end it's similar to: -kernel u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb This means that RISC-V Fedora 37 is bootable without any libvirt changes. Similar to what we already do with Ubuntu. Thus, unless there's another reason to add '-bios none' support, we can ditch this series. Thanks, Daniel
On Tue, Mar 28, 2023 at 04:46:45PM -0300, Daniel Henrique Barboza wrote:
On 3/22/23 13:37, Andrea Bolognani wrote:
On Wed, Mar 22, 2023 at 10:36:20AM -0300, Daniel Henrique Barboza wrote:
I'm not sure if the OS overwrites the firmware when running bare metal. Usually they provide different OS images for QEMU/libvirt and bare metal systems, probably to account for these differences. Baking the firmware in the kernel like Fedora Rawhide is doing was important a few years ago when RISC-V QEMU wasn't loading the firmware by default, but now it's getting in the way.
All this said, having a look at the recently updated Fedora for RISC-V wiki, the instructions for booting with libvirt and QEMU are different. libvirt [1] is using the '-bios none' attribute with 'virt-install' + a Fedora Rawhide image, but QEMU instructions [2] doesn't use '-bios none' and it's using Fedora 37.
The libvirt instructions in that page seem to have been updated in the Fedora 30 timeframe, whereas the QEMU instructions appear to be current as of Fedora 37.
At first I don't see any particular reason of why this Fedora 37 image would work on QEMU and not on libvirt. So what t I'll do now is do some testings with libvirt and Fedora 37. If it works then this series becomes way less important.
The libvirt instructions tell you to use
--boot kernel=Fedora-Developer-Rawhide-*-fw_payload-uboot-qemu-virt-smode.elf --qemu-commandline='-bios none'
(aka -bios none -kernel foo) while the QEMU ones suggest
-bios u/usr/share/uboot/qemu-riscv64_spl/u-boot-spl.bin -device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000
Those are completely different approaches to booting the guest OS. The latter is much saner IMO, because it keeps the firmware under control of the host (as is the case for SeaBIOS/edk2) instead of mixing the kernel and the firmware and requiring guest-specific files to be extracted from the disk image before being able to boot.
libvirt already provides the ability to pass arbitrary paths to -bios via the <loader type='rom'> element, so making the first part work should just be a matter of pointing it to the correct path. We don't seem to have the -device loader part wired up though, so that would need to be added. Probably as an additional attribute, in the vein of nvram.template? The address would have to be specified as well.
Note that the firmware descriptor format already supports u-boot as the firmware type. So in the long run ideally you'd only need to specify
<os firmware='uboot'>
and, assuming the uboot-images-riscv64 package is installed on the host, everything should just work.
So, I got the chance to try this out. We can remove the '-loader' from the Fedora 37 command line. This:
-device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000
Is loading the u-boot image in the address that the 'virt' machine loads the kernel. So in the end it's similar to:
-kernel u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb
Excellent insight! I've confirmed that you are correct, and that <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000'/> </qemu:commandline> can indeed be replaced with <kernel>u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb</kernel> without apparently affecting the VM's ability to boot.
This means that RISC-V Fedora 37 is bootable without any libvirt changes. Similar to what we already do with Ubuntu.
I've spent some time on this as well. Here are my findings. First, let's briefly mention how RISC-V VMs had to be booted a few years back. This is described in detail at [1] and preceding sections, but the quick version is that you would have a single file containing (I believe) OpenSBI, u-boot and the kernel. This file would need to be extracted from the disk image, and would then be passed to -kernel. Having the firmware and the kernel in a single file, which lives on the host rather than the guest, was obviously extremely cumbersome and unsuitable for scenarios beyond experimentation. Today, things are looking a lot better: the firmware lives in the host and the kernel lives in the guest, as they should. However, there seem to be two different approaches to loading the firmware floating around. The first one, described in the Fedora wiki[2], involves passing two separate files to QEMU. They both come from the uboot-images-riscv64 package, and their use looks like -bios /usr/share/uboot/qemu-riscv64_spl/u-boot-spl.bin -kernel /usr/share/uboot/qemu-riscv64_spl/u-boot.itb As far as I understand, this is a u-boot binary, just divided in two parts and loaded separately. They might run in different hardware privilege contexts? I think the first file might also embed OpenSBI somehow. The second approach is the one described in the Ubuntu wiki[3], and also requires passing two files to QEMU, except this time they come from the opensbi and u-boot-qemu packages respectively. The usage looks like -bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf -kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf I think in this case the first file is a minimal build of OpenSBI that likely just initializes enough hardware before handing control to an arbitrary payload - in this case, u-boot. There's a twist, however: the Ubuntu wiki claims that the -bios part is not necessary with QEMU >= 7.0, and I've verified that at least with the QEMU 7.2 package in Fedora 37 this is accurate. I believe this is because opensbi-riscv64-generic-fw_dynamic.bin, which is included in the QEMU package, gets loaded automatically, and this build of OpenSBI apparently also knows how to jump ahead to where -kernel loads u-boot. I've tried mixing things, that is, boot the Fedora VM using the firmware taken from Ubuntu and vice versa, and thankfully it looks like this is perfectly fine. In conclusion, the current version of libvirt is perfectly capable of booting modern RISC-V guests through a combination of <loader> and <kernel> elements. That said, even if things nominally work, way too much handholding is necessary. So I'm wondering how we can push things forward to make the process more user-friendly. The obvious first step would be for the u-boot-qemu package in Ubuntu to ship a JSON descriptor file containing something like { "interface-types": [ "u-boot" ], "mapping": { "device": "kernel", "filename": "/usr/lib/u-boot/qemu-riscv64_smode/uboot.elf" }, "targets": [ { "architecture": "riscv64", "machines": [ "virt" ] } ] } With some relatively straightforward plumbing on the libvirt side, we should be able to use the information provided in that file to make the simple VM configuration <os firmware='u-boot'> work out of the box. Getting the Fedora version to work would be trickier. As far as I can tell, there's currently no spec-compliant way to describe a firmware that requires both -bios and -kernel to be used at the same time. Should the spec be extended? Can we get Fedora to standardize on the, at least to an outside observer, simpler approach that Ubuntu has adopted? [1] https://fedoraproject.org/wiki/Architectures/RISC-V/Installing#Boot_with_lib... [2] https://fedoraproject.org/wiki/Architectures/RISC-V/Installing#Boot_under_QE... [3] https://wiki.ubuntu.com/RISC-V#Booting_with_QEMU -- Andrea Bolognani / Red Hat / Virtualization
On Fri, Apr 07, 2023 at 11:12:22AM -0700, Andrea Bolognani wrote:
Getting the Fedora version to work would be trickier. As far as I can tell, there's currently no spec-compliant way to describe a firmware that requires both -bios and -kernel to be used at the same time. Should the spec be extended? Can we get Fedora to standardize on the, at least to an outside observer, simpler approach that Ubuntu has adopted?
It looks like it's not just Ubuntu going for the split OpenSBI/u-boot approach: Debian[1] of course is doing the same, but so is openSUSE[2]. Gentoo[3], very conveniently, goes as far as providing a working libvirt XML :) Outside of the Linux world, FreeBSD[4] and OpenBSD[5] also seem to have adopted this strategy. It looks like Fedora might be the only outlier... [1] https://wiki.debian.org/RISC-V#Setting_up_a_riscv64_virtual_machine [2] https://en.opensuse.org/openSUSE:RISC-V#QEMU_system_emulation [3] https://wiki.gentoo.org/wiki/RISC-V_Qemu_setup [4] https://wiki.freebsd.org/riscv#Quick_Start [5] https://briancallahan.net/blog/20220418.html -- Andrea Bolognani / Red Hat / Virtualization
On 4/7/23 15:12, Andrea Bolognani wrote:
On Tue, Mar 28, 2023 at 04:46:45PM -0300, Daniel Henrique Barboza wrote:
On 3/22/23 13:37, Andrea Bolognani wrote:
On Wed, Mar 22, 2023 at 10:36:20AM -0300, Daniel Henrique Barboza wrote:
I'm not sure if the OS overwrites the firmware when running bare metal. Usually they provide different OS images for QEMU/libvirt and bare metal systems, probably to account for these differences. Baking the firmware in the kernel like Fedora Rawhide is doing was important a few years ago when RISC-V QEMU wasn't loading the firmware by default, but now it's getting in the way.
All this said, having a look at the recently updated Fedora for RISC-V wiki, the instructions for booting with libvirt and QEMU are different. libvirt [1] is using the '-bios none' attribute with 'virt-install' + a Fedora Rawhide image, but QEMU instructions [2] doesn't use '-bios none' and it's using Fedora 37.
The libvirt instructions in that page seem to have been updated in the Fedora 30 timeframe, whereas the QEMU instructions appear to be current as of Fedora 37.
At first I don't see any particular reason of why this Fedora 37 image would work on QEMU and not on libvirt. So what t I'll do now is do some testings with libvirt and Fedora 37. If it works then this series becomes way less important.
The libvirt instructions tell you to use
--boot kernel=Fedora-Developer-Rawhide-*-fw_payload-uboot-qemu-virt-smode.elf --qemu-commandline='-bios none'
(aka -bios none -kernel foo) while the QEMU ones suggest
-bios u/usr/share/uboot/qemu-riscv64_spl/u-boot-spl.bin -device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000
Those are completely different approaches to booting the guest OS. The latter is much saner IMO, because it keeps the firmware under control of the host (as is the case for SeaBIOS/edk2) instead of mixing the kernel and the firmware and requiring guest-specific files to be extracted from the disk image before being able to boot.
libvirt already provides the ability to pass arbitrary paths to -bios via the <loader type='rom'> element, so making the first part work should just be a matter of pointing it to the correct path. We don't seem to have the -device loader part wired up though, so that would need to be added. Probably as an additional attribute, in the vein of nvram.template? The address would have to be specified as well.
Note that the firmware descriptor format already supports u-boot as the firmware type. So in the long run ideally you'd only need to specify
<os firmware='uboot'>
and, assuming the uboot-images-riscv64 package is installed on the host, everything should just work.
So, I got the chance to try this out. We can remove the '-loader' from the Fedora 37 command line. This:
-device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000
Is loading the u-boot image in the address that the 'virt' machine loads the kernel. So in the end it's similar to:
-kernel u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb
Excellent insight! I've confirmed that you are correct, and that
<qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000'/> </qemu:commandline>
can indeed be replaced with
<kernel>u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb</kernel>
without apparently affecting the VM's ability to boot.
This means that RISC-V Fedora 37 is bootable without any libvirt changes. Similar to what we already do with Ubuntu.
I've spent some time on this as well. Here are my findings.
First, let's briefly mention how RISC-V VMs had to be booted a few years back. This is described in detail at [1] and preceding sections, but the quick version is that you would have a single file containing (I believe) OpenSBI, u-boot and the kernel. This file would need to be extracted from the disk image, and would then be passed to -kernel. Having the firmware and the kernel in a single file, which lives on the host rather than the guest, was obviously extremely cumbersome and unsuitable for scenarios beyond experimentation.
Today, things are looking a lot better: the firmware lives in the host and the kernel lives in the guest, as they should. However, there seem to be two different approaches to loading the firmware floating around.
The first one, described in the Fedora wiki[2], involves passing two separate files to QEMU. They both come from the uboot-images-riscv64 package, and their use looks like
-bios /usr/share/uboot/qemu-riscv64_spl/u-boot-spl.bin -kernel /usr/share/uboot/qemu-riscv64_spl/u-boot.itb
As far as I understand, this is a u-boot binary, just divided in two parts and loaded separately. They might run in different hardware privilege contexts? I think the first file might also embed OpenSBI somehow.
The second approach is the one described in the Ubuntu wiki[3], and also requires passing two files to QEMU, except this time they come from the opensbi and u-boot-qemu packages respectively. The usage looks like
-bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf -kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf
I think in this case the first file is a minimal build of OpenSBI that likely just initializes enough hardware before handing control to an arbitrary payload - in this case, u-boot.
There's a twist, however: the Ubuntu wiki claims that the -bios part is not necessary with QEMU >= 7.0, and I've verified that at least with the QEMU 7.2 package in Fedora 37 this is accurate. I believe this is because opensbi-riscv64-generic-fw_dynamic.bin, which is included in the QEMU package, gets loaded automatically, and this build of OpenSBI apparently also knows how to jump ahead to where -kernel loads u-boot.
Yeah, until not so long ago QEMU wasn't loading any default OpenSBI firmware, requiring the distro/user to manually add a -bios entry. Hopefully distros can now use the default QEMU OpenSBI binary.
I've tried mixing things, that is, boot the Fedora VM using the firmware taken from Ubuntu and vice versa, and thankfully it looks like this is perfectly fine.
In conclusion, the current version of libvirt is perfectly capable of booting modern RISC-V guests through a combination of <loader> and <kernel> elements.
That said, even if things nominally work, way too much handholding is necessary. So I'm wondering how we can push things forward to make the process more user-friendly.
The obvious first step would be for the u-boot-qemu package in Ubuntu to ship a JSON descriptor file containing something like
{ "interface-types": [ "u-boot" ], "mapping": { "device": "kernel", "filename": "/usr/lib/u-boot/qemu-riscv64_smode/uboot.elf" }, "targets": [ { "architecture": "riscv64", "machines": [ "virt" ] } ] }
So, I talked with Canonical a few months back about this package. I'm using a F37 dev box and I had to uncompress the u-boot .deb package by hand to extract the uboot.elf file to be able to launch the guest, meaning that users that aren't using an Ubuntu host would need to go through that to boot an Ubuntu RISC-V domain. I got told that any u-boot build would work. Didn't have the opportunity to test it. I also believe that there's nothing particularly different in how RISC-V works to require a -kernel argument every time. Users don't have to do that with x86 and even ppc64 pSeries domains: you give a disk with the guest image and everything just works. Ideally we would want that for RISC-V domains as well. We'll probably need stuff done in QEMU and OpenSBI to support this use case but I believe it's worth it. Depending on how much work/how long it would take we can discuss how libvirt can ease the users pain in the meantime.
With some relatively straightforward plumbing on the libvirt side, we should be able to use the information provided in that file to make the simple VM configuration
<os firmware='u-boot'>
work out of the box.
Getting the Fedora version to work would be trickier. As far as I can tell, there's currently no spec-compliant way to describe a firmware that requires both -bios and -kernel to be used at the same time. Should the spec be extended? Can we get Fedora to standardize on the, at least to an outside observer, simpler approach that Ubuntu has adopted?
I believe this has to be answered by the Fedora folks, but yeah, it would be nice if Fedora could adopt the same approach that Ubuntu, Debian, OpenSuse and others are using. Thanks, Daniel
[1] https://fedoraproject.org/wiki/Architectures/RISC-V/Installing#Boot_with_lib... [2] https://fedoraproject.org/wiki/Architectures/RISC-V/Installing#Boot_under_QE... [3] https://wiki.ubuntu.com/RISC-V#Booting_with_QEMU
On Mon, Apr 10, 2023 at 07:50:36PM -0300, Daniel Henrique Barboza wrote:
On 4/7/23 15:12, Andrea Bolognani wrote:
On Tue, Mar 28, 2023 at 04:46:45PM -0300, Daniel Henrique Barboza wrote: So, I talked with Canonical a few months back about this package. I'm using a F37 dev box and I had to uncompress the u-boot .deb package by hand to extract the uboot.elf file to be able to launch the guest, meaning that users that aren't using an Ubuntu host would need to go through that to boot an Ubuntu RISC-V domain. I got told that any u-boot build would work. Didn't have the opportunity to test it.
I've just tested using the u-boot.bin coming from the Ubuntu package to boot a Fedora, openSUSE and FreeBSD guests. It works fine. I might try the same with the openSUSE and FreeBSD builds of u-boot, but I don't expect the results to be any different.
I also believe that there's nothing particularly different in how RISC-V works to require a -kernel argument every time. Users don't have to do that with x86 and even ppc64 pSeries domains: you give a disk with the guest image and everything just works.
That's only partially accurate. In the case of x86_64, BIOS boot works out of the box without passing any additional parameters to QEMU, but using EFI requires a bunch more work.
Ideally we would want that for RISC-V domains as well. We'll probably need stuff done in QEMU and OpenSBI to support this use case but I believe it's worth it.
In the case of RISC-V we have additional moving parts because OpenSBI is loaded by default, but then you need u-boot on top of that. Or, further down the line, edk2. So I'm not sure we'll ever be able to get to the point where you can just pass a disk image to QEMU and expect it to boot.
Depending on how much work/how long it would take we can discuss how libvirt can ease the users pain in the meantime.
Yeah, if we can get to the point where the user only needs <os firmware='u-boot'> or <os firmware='efi'>, then that's good enough as far as I'm concerned. You already need <os firmware='efi'> for aarch64, for example, so I don't feel that would be an unreasonable burden.
Getting the Fedora version to work would be trickier. As far as I can tell, there's currently no spec-compliant way to describe a firmware that requires both -bios and -kernel to be used at the same time. Should the spec be extended? Can we get Fedora to standardize on the, at least to an outside observer, simpler approach that Ubuntu has adopted?
I believe this has to be answered by the Fedora folks, but yeah, it would be nice if Fedora could adopt the same approach that Ubuntu, Debian, OpenSuse and others are using.
Note that the Fedora RISC-V disk images can be booted just fine using a u-boot.bin file coming from Ubuntu, so this is really about Fedora as a host rather than as a guest. But yeah, we should strive to make things more seamless, which as a first step would require having u-boot.bin available in Fedora. -- Andrea Bolognani / Red Hat / Virtualization
Hi,
The second approach is the one described in the Ubuntu wiki[3], and also requires passing two files to QEMU, except this time they come from the opensbi and u-boot-qemu packages respectively. The usage looks like
-bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf -kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf
I think in this case the first file is a minimal build of OpenSBI that likely just initializes enough hardware before handing control to an arbitrary payload - in this case, u-boot.
Yes. These days opensbi seems to be loaded by default, so the first line is not needed. In fact I'm running a guest here, on fedora 37 + virt-preview with just this ... <os> <type arch='riscv64' machine='virt'>hvm</type> <kernel>/home/kraxel/projects/u-boot/build-qemu-riscv64-smode/u-boot.bin</kernel> <boot dev='hd'/> </os> ... and it works fine. There is also this variant ... <qemu:commandline> <qemu:arg value='-drive'/> <qemu:arg value='if=pflash,index=1,format=raw,file=/vmdisk/hdd/pool-risc-v/RISCV_VIRT.raw'/> </qemu:commandline> ... to boot edk2 firmware. Note this is a single image carrying both code and vars. Also note 'index=1', which I think is needed because the (default) opensbi is loaded to the pflash device with 'index=0'. This doesn't boot the distro due to grub2 not having full riscv64 efi support (yet). take care, Gerd
On Tue, Apr 11, 2023 at 01:14:27PM +0200, Gerd Hoffmann wrote:
The second approach is the one described in the Ubuntu wiki[3], and also requires passing two files to QEMU, except this time they come from the opensbi and u-boot-qemu packages respectively. The usage looks like
-bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf -kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf
I think in this case the first file is a minimal build of OpenSBI that likely just initializes enough hardware before handing control to an arbitrary payload - in this case, u-boot.
Yes. These days opensbi seems to be loaded by default, so the first line is not needed. In fact I'm running a guest here, on fedora 37 + virt-preview with just this ...
<os> <type arch='riscv64' machine='virt'>hvm</type> <kernel>/home/kraxel/projects/u-boot/build-qemu-riscv64-smode/u-boot.bin</kernel> <boot dev='hd'/> </os>
... and it works fine.
This matches my experience. I've tried both u-boot.bin (referenced in openSUSE and FreeBSD documentation) and uboot.elf (Debian and Ubuntu documentation) and it doesn't seem to make a difference which one you choose.
There is also this variant ...
<qemu:commandline> <qemu:arg value='-drive'/> <qemu:arg value='if=pflash,index=1,format=raw,file=/vmdisk/hdd/pool-risc-v/RISCV_VIRT.raw'/> </qemu:commandline>
... to boot edk2 firmware. Note this is a single image carrying both code and vars. Also note 'index=1', which I think is needed because the (default) opensbi is loaded to the pflash device with 'index=0'.
This doesn't boot the distro due to grub2 not having full riscv64 efi support (yet).
The openSUSE images are using grub2, and possibly EFI somehow? Right after selecting the entry I see EFI stub: Booting Linux Kernel... EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path EFI stub: Using DTB from configuration table EFI stub: Exiting boot services... The image never manages to boot all the way for me: it looks like it can't find the root filesystem. But eventually I get dropped into a rescue shell, and /sys/firmware/efi/ contains some data. Note that I'm not using edk2 in this case, just the same u-boot.bin file that I've used to boot other images. -- Andrea Bolognani / Red Hat / Virtualization
On Wed, Mar 22, 2023 at 11:37:10AM -0500, Andrea Bolognani wrote:
On Wed, Mar 22, 2023 at 10:36:20AM -0300, Daniel Henrique Barboza wrote:
I'm not sure if the OS overwrites the firmware when running bare metal. Usually they provide different OS images for QEMU/libvirt and bare metal systems, probably to account for these differences. Baking the firmware in the kernel like Fedora Rawhide is doing was important a few years ago when RISC-V QEMU wasn't loading the firmware by default, but now it's getting in the way.
All this said, having a look at the recently updated Fedora for RISC-V wiki, the instructions for booting with libvirt and QEMU are different. libvirt [1] is using the '-bios none' attribute with 'virt-install' + a Fedora Rawhide image, but QEMU instructions [2] doesn't use '-bios none' and it's using Fedora 37.
The libvirt instructions in that page seem to have been updated in the Fedora 30 timeframe, whereas the QEMU instructions appear to be current as of Fedora 37.
At first I don't see any particular reason of why this Fedora 37 image would work on QEMU and not on libvirt. So what t I'll do now is do some testings with libvirt and Fedora 37. If it works then this series becomes way less important.
The libvirt instructions tell you to use
--boot kernel=Fedora-Developer-Rawhide-*-fw_payload-uboot-qemu-virt-smode.elf --qemu-commandline='-bios none'
(aka -bios none -kernel foo) while the QEMU ones suggest
-bios u/usr/share/uboot/qemu-riscv64_spl/u-boot-spl.bin -device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000
Those are completely different approaches to booting the guest OS. The latter is much saner IMO, because it keeps the firmware under control of the host (as is the case for SeaBIOS/edk2) instead of mixing the kernel and the firmware and requiring guest-specific files to be extracted from the disk image before being able to boot.
libvirt already provides the ability to pass arbitrary paths to -bios via the <loader type='rom'> element, so making the first part work should just be a matter of pointing it to the correct path. We don't seem to have the -device loader part wired up though, so that would need to be added. Probably as an additional attribute, in the vein of nvram.template? The address would have to be specified as well.
Note that the firmware descriptor format already supports u-boot as the firmware type. So in the long run ideally you'd only need to specify
<os firmware='uboot'>
and, assuming the uboot-images-riscv64 package is installed on the host, everything should just work.
I just came across this thread while trying to update the libvirt instructions on that page. Specifically I need to add these to the qemu command line: -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 Was any change made to libvirt / virt-install to make this possible? - - - For the benefit of the search engine gods, this works for now: # virt-install --import --memory 8192 -n fedora-37-riscv \ --arch riscv64 --vcpus 8 \ --disk fedora-37-riscv.qcow2,format=qcow2 \ --osinfo fedora37 \ --qemu-commandline=' -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 ' 'Course you have to disable SELinux ... Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
On Tue, May 23, 2023 at 11:59:41AM +0100, Richard W.M. Jones wrote:
I just came across this thread while trying to update the libvirt instructions on that page. Specifically I need to add these to the qemu command line:
-bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000
I recommend going through the entire thread if you haven't already, it's full of interesting details. And it would be useful to have another pair of eyes go over it, just in case something was missed :) Anyway, we found that the -device loader incantation above is effectively equivalent to -kernel, so you can just use that instead. Both -bios and -kernel are exposed by libvirt through XML elements of the same names. Further, since QEMU loads OpenSBI as -bios by default these days, you don't even need that part and can just use -kernel to point to the u-boot.bin file. In this case, you should use the file that would be installed as /usr/share/uboot/qemu-riscv64_smode/u-boot.bin by the uboot-images-riscv64 Fedora package.
Was any change made to libvirt / virt-install to make this possible?
Not yet. I would love to make booting a riscv64 VM a simple case of adding <os firmware='u-boot'/> to the domain XML, and I'm more than willing to write the necessary libvirt patches, but a couple of things need to happen first: * the uboot-images-riscv64 package needs to be made available on non-riscv64 architectures; * a JSON firmware descriptor needs to be added to said package, pointing to /usr/share/uboot/qemu-riscv64_smode/u-boot.bin. The first part might be tricky, as it will probably involve a cross-compiler, but other distributions seem to have managed to do it already so I don't expect Fedora to face significant roadblocks. For the second part, something along the lines of # /usr/share/qemu/firmware/30-uboot-riscv64-smode.json { "interface-types": [ "uboot" ], "mapping": { "device": "kernel", "filename": /usr/share/uboot/qemu-riscv64_smode/u-boot.bin" }, "targets": [ { "architecture": "riscv64", "machines": [ "virt" ] } ] } should do the trick, but note that I haven't tested it :) My only question is why the u-boot-spl.bin/u-boot.itb approach was chosen to be documented in the Fedora wiki? Is there something that I've missed and that makes it superior, or can we follow along with what everyone else is doing? A question for David, perhaps.
For the benefit of the search engine gods, this works for now:
# virt-install --import --memory 8192 -n fedora-37-riscv \ --arch riscv64 --vcpus 8 \ --disk fedora-37-riscv.qcow2,format=qcow2 \ --osinfo fedora37 \ --qemu-commandline=' -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 '
Based on the information above, using --boot kernel=/path/to/u-boot.bin should work...
'Course you have to disable SELinux ...
... without requiring this :) -- Andrea Bolognani / Red Hat / Virtualization
On Tue, May 23, 2023 at 3:18 PM Andrea Bolognani <abologna@redhat.com> wrote:
On Tue, May 23, 2023 at 11:59:41AM +0100, Richard W.M. Jones wrote:
I just came across this thread while trying to update the libvirt instructions on that page. Specifically I need to add these to the qemu command line:
-bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000
I recommend going through the entire thread if you haven't already, it's full of interesting details. And it would be useful to have another pair of eyes go over it, just in case something was missed :)
Anyway, we found that the -device loader incantation above is effectively equivalent to -kernel, so you can just use that instead. Both -bios and -kernel are exposed by libvirt through XML elements of the same names.
Further, since QEMU loads OpenSBI as -bios by default these days, you don't even need that part and can just use -kernel to point to the u-boot.bin file. In this case, you should use the file that would be installed as
We run QEMU in a similar way as any other board. So it's U-Boot SPL, which then loads U-Boot ITB (which typically contains U-Boot proper, OpenSBI FW_DYNAMIC generic binary, and DTB). U-Boot SPL transfers control to OpenSBI and tells it how to load the next stage (i.e U-Boot proper).
/usr/share/uboot/qemu-riscv64_smode/u-boot.bin
by the uboot-images-riscv64 Fedora package.
This is noarch package, and thus available on all arches. This package is only available in Fedora/RISCV Koji for now.
Was any change made to libvirt / virt-install to make this possible?
Not yet.
I would love to make booting a riscv64 VM a simple case of adding
<os firmware='u-boot'/>
to the domain XML, and I'm more than willing to write the necessary libvirt patches, but a couple of things need to happen first:
* the uboot-images-riscv64 package needs to be made available on non-riscv64 architectures;
* a JSON firmware descriptor needs to be added to said package, pointing to /usr/share/uboot/qemu-riscv64_smode/u-boot.bin.
The first part might be tricky, as it will probably involve a cross-compiler, but other distributions seem to have managed to do it already so I don't expect Fedora to face significant roadblocks.
For the second part, something along the lines of
# /usr/share/qemu/firmware/30-uboot-riscv64-smode.json { "interface-types": [ "uboot" ], "mapping": { "device": "kernel", "filename": /usr/share/uboot/qemu-riscv64_smode/u-boot.bin" }, "targets": [ { "architecture": "riscv64", "machines": [ "virt" ] } ] }
should do the trick, but note that I haven't tested it :)
My only question is why the u-boot-spl.bin/u-boot.itb approach was chosen to be documented in the Fedora wiki? Is there something that I've missed and that makes it superior, or can we follow along with what everyone else is doing? A question for David, perhaps.
Because that's how the boards boot, and testing on QEMU is more convenient.
For the benefit of the search engine gods, this works for now:
# virt-install --import --memory 8192 -n fedora-37-riscv \ --arch riscv64 --vcpus 8 \ --disk fedora-37-riscv.qcow2,format=qcow2 \ --osinfo fedora37 \ --qemu-commandline=' -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 '
This doesn't disable Sv48 and Sv57. I don't know the overall status, but at least Golang 1.20 has a fix to support anything above Sv39. Not sure about other runtimes that do pointer tagging. See: https://bugs.launchpad.net/ubuntu/+source/linux-riscv/+bug/1991790 Simply put, older disk images on newer QEMU versions might not work properly. Note that we might want to switch to EDK2 in the future for QEMU, and that probably will use two 32MiB pflash devices in virt machine. I have seen, but haven't tested QEMU + EDK2 patches. Cheers, david
Based on the information above, using
--boot kernel=/path/to/u-boot.bin
should work...
'Course you have to disable SELinux ...
... without requiring this :)
-- Andrea Bolognani / Red Hat / Virtualization
On Tue, May 23, 2023 at 04:05:04PM +0300, David Abdurachmanov wrote:
We run QEMU in a similar way as any other board.
So it's U-Boot SPL, which then loads U-Boot ITB (which typically contains U-Boot proper, OpenSBI FW_DYNAMIC generic binary, and DTB). U-Boot SPL transfers control to OpenSBI and tells it how to load the next stage (i.e U-Boot proper).
Is there any documentation on how RISC-V boots now? And about the eventual goal that we're aiming for?
For the benefit of the search engine gods, this works for now:
# virt-install --import --memory 8192 -n fedora-37-riscv \ --arch riscv64 --vcpus 8 \ --disk fedora-37-riscv.qcow2,format=qcow2 \ --osinfo fedora37 \ --qemu-commandline=' -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 '
This doesn't disable Sv48 and Sv57. I don't know the overall status, but at least Golang 1.20 has a fix to support anything above Sv39.
Not sure about other runtimes that do pointer tagging.
See: https://bugs.launchpad.net/ubuntu/+source/linux-riscv/+bug/1991790
Interesting and informative bug. I guess I'm not using any golang binaries, as the guest totally seems to work fine. I'm a bit confused how this kernel detail leaks into userspace though. I thought userspace can use all 64 bits in pointers.
Simply put, older disk images on newer QEMU versions might not work properly.
Note that we might want to switch to EDK2 in the future for QEMU, and that probably will use two 32MiB pflash devices in virt machine. I have seen, but haven't tested QEMU + EDK2 patches.
Rich.
Cheers, david
Based on the information above, using
--boot kernel=/path/to/u-boot.bin
should work...
'Course you have to disable SELinux ...
... without requiring this :)
-- Andrea Bolognani / Red Hat / Virtualization
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Tue, May 23, 2023 at 5:15 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Tue, May 23, 2023 at 04:05:04PM +0300, David Abdurachmanov wrote:
We run QEMU in a similar way as any other board.
So it's U-Boot SPL, which then loads U-Boot ITB (which typically contains U-Boot proper, OpenSBI FW_DYNAMIC generic binary, and DTB). U-Boot SPL transfers control to OpenSBI and tells it how to load the next stage (i.e U-Boot proper).
Is there any documentation on how RISC-V boots now? And about the eventual goal that we're aiming for?
For QEMU or the boards? By default QEMU ships OpenSBI FW_DYNAMIC generic image and that's default "BIOS". For us (Fedora/RISCV) we don't rely on what's shipped by QEMU. We build our own OpenSBI binary, and boot in a similar way as Unmatched boards. We don't load DTB from /boot, but take it from QEMU. That's the only major difference. I would expect us to switch to EDK2 once all the relevant bits land upstream. That work is mostly done by Ventana.
For the benefit of the search engine gods, this works for now:
# virt-install --import --memory 8192 -n fedora-37-riscv \ --arch riscv64 --vcpus 8 \ --disk fedora-37-riscv.qcow2,format=qcow2 \ --osinfo fedora37 \ --qemu-commandline=' -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 '
This doesn't disable Sv48 and Sv57. I don't know the overall status, but at least Golang 1.20 has a fix to support anything above Sv39.
Not sure about other runtimes that do pointer tagging.
See: https://bugs.launchpad.net/ubuntu/+source/linux-riscv/+bug/1991790
Interesting and informative bug. I guess I'm not using any golang binaries, as the guest totally seems to work fine. I'm a bit confused how this kernel detail leaks into userspace though. I thought userspace can use all 64 bits in pointers.
It's not leaked. Folks assume that N bits at the end are not used, and thus you can store extra information in there. That's "Pointer Tagging", but it's based on calculated assumptions. That got broken once QEMU gained Sv57 support. IIRC v8 and OpenJDK might also be affected, but I don't have details. OpenJDK: https://github.com/openjdk/jdk/pull/11388 Basically you get: err_msg("Unsupported satp mode: %s. Only satp modes up to sv48 are supported for now." .. I think v8, has: // Note that this assumes the use of SV48, the 48-bit virtual memory system. Note that RISCV has a "Pointer Masking" extension (WIP), Zjpm.
Simply put, older disk images on newer QEMU versions might not work properly.
Note that we might want to switch to EDK2 in the future for QEMU, and that probably will use two 32MiB pflash devices in virt machine. I have seen, but haven't tested QEMU + EDK2 patches.
Rich.
Cheers, david
Based on the information above, using
--boot kernel=/path/to/u-boot.bin
should work...
'Course you have to disable SELinux ...
... without requiring this :)
-- Andrea Bolognani / Red Hat / Virtualization
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Tue, May 23, 2023 at 04:05:04PM +0300, David Abdurachmanov wrote:
On Tue, May 23, 2023 at 3:18 PM Andrea Bolognani <abologna@redhat.com> wrote:
On Tue, May 23, 2023 at 11:59:41AM +0100, Richard W.M. Jones wrote:
I just came across this thread while trying to update the libvirt instructions on that page. Specifically I need to add these to the qemu command line:
-bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000
Anyway, we found that the -device loader incantation above is effectively equivalent to -kernel, so you can just use that instead. Both -bios and -kernel are exposed by libvirt through XML elements of the same names.
Further, since QEMU loads OpenSBI as -bios by default these days, you don't even need that part and can just use -kernel to point to the u-boot.bin file. In this case, you should use the file that would be installed as
We run QEMU in a similar way as any other board.
You mean physical boards?
So it's U-Boot SPL, which then loads U-Boot ITB (which typically contains U-Boot proper, OpenSBI FW_DYNAMIC generic binary, and DTB). U-Boot SPL transfers control to OpenSBI and tells it how to load the next stage (i.e U-Boot proper).
That sounds a bit roundabout when you consider that QEMU automatically loads OpenSBI, and that in turn knows to jump to a S-Mode payload such as the Linux kernel or an S-Mode build of U-Boot. Put it another way: do you have any objections to loading qemu-riscv64_smode/u-boot.bin via -kernel as the default boot strategy for riscv64 VMs? That's what every OS other than Fedora already recommends doing. Loading the SPL and ITB separately would still be possible, of course. I'm simply talking about the default experience.
/usr/share/uboot/qemu-riscv64_smode/u-boot.bin
by the uboot-images-riscv64 Fedora package.
This is noarch package, and thus available on all arches. This package is only available in Fedora/RISCV Koji for now.
You're right! I got confused because "riscv64" is in the name O:-) Is there ongoing work to move this package to Fedora proper? In terms of user experience, having to download disk images from a third-party is mostly fine, but installing third-party RPMs on the host... Not so much. Having this in Fedora proper would make it a fully self-contained host for RISC-V TCG VMs.
For the benefit of the search engine gods, this works for now:
# virt-install --import --memory 8192 -n fedora-37-riscv \ --arch riscv64 --vcpus 8 \ --disk fedora-37-riscv.qcow2,format=qcow2 \ --osinfo fedora37 \ --qemu-commandline=' -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 '
This doesn't disable Sv48 and Sv57. I don't know the overall status, but at least Golang 1.20 has a fix to support anything above Sv39.
Not sure about other runtimes that do pointer tagging.
See: https://bugs.launchpad.net/ubuntu/+source/linux-riscv/+bug/1991790
IIUC that's something that needs to be handled in the kernel? Or do we need to do something at the QEMU/libvirt/virt-install level to make things work?
Simply put, older disk images on newer QEMU versions might not work properly.
I'm not too concerned about this. In the long run, it will simply not matter.
Note that we might want to switch to EDK2 in the future for QEMU, and that probably will use two 32MiB pflash devices in virt machine. I have seen, but haven't tested QEMU + EDK2 patches.
Yup, there's some discussion about this very topic happening right now on the QEMU mailing list. I'm actively involved in it and it looks like the necessary changes might be merged soon. I'm not sure we can count on EFI-enabled RISC-V disk images popping up very quickly though, so I would like to improve the situation for the disk images that are out there right now and need U-Boot to run. -- Andrea Bolognani / Red Hat / Virtualization
On Tue, May 23, 2023 at 6:12 PM Andrea Bolognani <abologna@redhat.com> wrote:
On Tue, May 23, 2023 at 04:05:04PM +0300, David Abdurachmanov wrote:
On Tue, May 23, 2023 at 3:18 PM Andrea Bolognani <abologna@redhat.com> wrote:
On Tue, May 23, 2023 at 11:59:41AM +0100, Richard W.M. Jones wrote:
I just came across this thread while trying to update the libvirt instructions on that page. Specifically I need to add these to the qemu command line:
-bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000
Anyway, we found that the -device loader incantation above is effectively equivalent to -kernel, so you can just use that instead. Both -bios and -kernel are exposed by libvirt through XML elements of the same names.
Further, since QEMU loads OpenSBI as -bios by default these days, you don't even need that part and can just use -kernel to point to the u-boot.bin file. In this case, you should use the file that would be installed as
We run QEMU in a similar way as any other board.
You mean physical boards?
Yes
So it's U-Boot SPL, which then loads U-Boot ITB (which typically contains U-Boot proper, OpenSBI FW_DYNAMIC generic binary, and DTB). U-Boot SPL transfers control to OpenSBI and tells it how to load the next stage (i.e U-Boot proper).
That sounds a bit roundabout when you consider that QEMU automatically loads OpenSBI, and that in turn knows to jump to a S-Mode payload such as the Linux kernel or an S-Mode build of U-Boot.
Put it another way: do you have any objections to loading qemu-riscv64_smode/u-boot.bin via -kernel as the default boot strategy for riscv64 VMs? That's what every OS other than Fedora already recommends doing.
We typically use a non-release version of OpenSBI, because it has the latest hardware support and maybe some fixes. For libvirt users (out-of-the box experience) that's the correct approach.
Loading the SPL and ITB separately would still be possible, of course. I'm simply talking about the default experience.
Yeah, this default is sane.
/usr/share/uboot/qemu-riscv64_smode/u-boot.bin
by the uboot-images-riscv64 Fedora package.
This is noarch package, and thus available on all arches. This package is only available in Fedora/RISCV Koji for now.
You're right! I got confused because "riscv64" is in the name O:-)
Is there ongoing work to move this package to Fedora proper? In terms of user experience, having to download disk images from a third-party is mostly fine, but installing third-party RPMs on the host... Not so much. Having this in Fedora proper would make it a fully self-contained host for RISC-V TCG VMs.
It could go as-is, but most likely will need to change. So far we relied on a single OpenSBI FW_DYNAMIC generic image, but that changes starting with JH7110 which has a different (i.e. non default) load address. Thus we might need to build OpenSBI per board (or similar), and pick the right one while building U-Boot. Unknown (at least for me) how things will work on TH1520 based boards too. One thing I am sure about is that things in the SPEC file most likely will change once more boards from multiple vendors are supported. We can push the current thing as-is if you want.
For the benefit of the search engine gods, this works for now:
# virt-install --import --memory 8192 -n fedora-37-riscv \ --arch riscv64 --vcpus 8 \ --disk fedora-37-riscv.qcow2,format=qcow2 \ --osinfo fedora37 \ --qemu-commandline=' -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 '
This doesn't disable Sv48 and Sv57. I don't know the overall status, but at least Golang 1.20 has a fix to support anything above Sv39.
Not sure about other runtimes that do pointer tagging.
See: https://bugs.launchpad.net/ubuntu/+source/linux-riscv/+bug/1991790
IIUC that's something that needs to be handled in the kernel? Or do we need to do something at the QEMU/libvirt/virt-install level to make things work?
Basically two workarounds were created: - Ability to control stap mode in QEMU (basically options to enable/disable Sv39, Sv48, Sv57). - Ability to disable that on the kernel side. I think the 1st one to arrive was QEMU. IIRC x86_64 and aarch64 are smart in the way they give virtual addresses to user space. Which is not the case for riscv (yet). On the kernel side, that's part of 6.4 kernel (not released yet).
Simply put, older disk images on newer QEMU versions might not work properly.
I'm not too concerned about this. In the long run, it will simply not matter.
True.
Note that we might want to switch to EDK2 in the future for QEMU, and that probably will use two 32MiB pflash devices in virt machine. I have seen, but haven't tested QEMU + EDK2 patches.
Yup, there's some discussion about this very topic happening right now on the QEMU mailing list. I'm actively involved in it and it looks like the necessary changes might be merged soon.
Yup. I noticed your name on the emails. Good to know you are involved.
I'm not sure we can count on EFI-enabled RISC-V disk images popping up very quickly though, so I would like to improve the situation for the disk images that are out there right now and need U-Boot to run.
EFI works and some are already using it with U-Boot IIRC. Even smaller SBCs (e.g. based on StarFive JH7110) gained EDK2 port (probably not upstreamed yet). https://github.com/starfive-tech/edk2/releases So even sub-100 USD boards will get EDK2 :) Of course, that doesn't incl. ACPI. Cheers, david
-- Andrea Bolognani / Red Hat / Virtualization
On Tue, May 23, 2023 at 06:32:07PM +0300, David Abdurachmanov wrote:
On Tue, May 23, 2023 at 6:12 PM Andrea Bolognani <abologna@redhat.com> wrote:
On Tue, May 23, 2023 at 04:05:04PM +0300, David Abdurachmanov wrote:
On Tue, May 23, 2023 at 3:18 PM Andrea Bolognani <abologna@redhat.com> wrote:
On Tue, May 23, 2023 at 11:59:41AM +0100, Richard W.M. Jones wrote:
I just came across this thread while trying to update the libvirt instructions on that page. Specifically I need to add these to the qemu command line:
-bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000
Anyway, we found that the -device loader incantation above is effectively equivalent to -kernel, so you can just use that instead. Both -bios and -kernel are exposed by libvirt through XML elements of the same names.
Further, since QEMU loads OpenSBI as -bios by default these days, you don't even need that part and can just use -kernel to point to the u-boot.bin file. In this case, you should use the file that would be installed as
We run QEMU in a similar way as any other board.
You mean physical boards?
Yes
So it's U-Boot SPL, which then loads U-Boot ITB (which typically contains U-Boot proper, OpenSBI FW_DYNAMIC generic binary, and DTB). U-Boot SPL transfers control to OpenSBI and tells it how to load the next stage (i.e U-Boot proper).
That sounds a bit roundabout when you consider that QEMU automatically loads OpenSBI, and that in turn knows to jump to a S-Mode payload such as the Linux kernel or an S-Mode build of U-Boot.
Put it another way: do you have any objections to loading qemu-riscv64_smode/u-boot.bin via -kernel as the default boot strategy for riscv64 VMs? That's what every OS other than Fedora already recommends doing.
We typically use a non-release version of OpenSBI, because it has the latest hardware support and maybe some fixes.
For libvirt users (out-of-the box experience) that's the correct approach.
Loading the SPL and ITB separately would still be possible, of course. I'm simply talking about the default experience.
Yeah, this default is sane.
/usr/share/uboot/qemu-riscv64_smode/u-boot.bin
by the uboot-images-riscv64 Fedora package.
This is noarch package, and thus available on all arches. This package is only available in Fedora/RISCV Koji for now.
You're right! I got confused because "riscv64" is in the name O:-)
Is there ongoing work to move this package to Fedora proper? In terms of user experience, having to download disk images from a third-party is mostly fine, but installing third-party RPMs on the host... Not so much. Having this in Fedora proper would make it a fully self-contained host for RISC-V TCG VMs.
It could go as-is, but most likely will need to change. So far we relied on a single OpenSBI FW_DYNAMIC generic image, but that changes starting with JH7110 which has a different (i.e. non default) load address. Thus we might need to build OpenSBI per board (or similar), and pick the right one while building U-Boot.
Unknown (at least for me) how things will work on TH1520 based boards too.
One thing I am sure about is that things in the SPEC file most likely will change once more boards from multiple vendors are supported.
We can push the current thing as-is if you want.
The "hump" here is getting it through the Fedora approval process. Once in we can add additional features very easily. If you want me to do this let me know, I can start right away (still fiddling around with benchmarking that qemu instance so I'm not fully occupied right now). Rich.
For the benefit of the search engine gods, this works for now:
# virt-install --import --memory 8192 -n fedora-37-riscv \ --arch riscv64 --vcpus 8 \ --disk fedora-37-riscv.qcow2,format=qcow2 \ --osinfo fedora37 \ --qemu-commandline=' -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 '
This doesn't disable Sv48 and Sv57. I don't know the overall status, but at least Golang 1.20 has a fix to support anything above Sv39.
Not sure about other runtimes that do pointer tagging.
See: https://bugs.launchpad.net/ubuntu/+source/linux-riscv/+bug/1991790
IIUC that's something that needs to be handled in the kernel? Or do we need to do something at the QEMU/libvirt/virt-install level to make things work?
Basically two workarounds were created: - Ability to control stap mode in QEMU (basically options to enable/disable Sv39, Sv48, Sv57). - Ability to disable that on the kernel side.
I think the 1st one to arrive was QEMU.
IIRC x86_64 and aarch64 are smart in the way they give virtual addresses to user space. Which is not the case for riscv (yet).
On the kernel side, that's part of 6.4 kernel (not released yet).
Simply put, older disk images on newer QEMU versions might not work properly.
I'm not too concerned about this. In the long run, it will simply not matter.
True.
Note that we might want to switch to EDK2 in the future for QEMU, and that probably will use two 32MiB pflash devices in virt machine. I have seen, but haven't tested QEMU + EDK2 patches.
Yup, there's some discussion about this very topic happening right now on the QEMU mailing list. I'm actively involved in it and it looks like the necessary changes might be merged soon.
Yup. I noticed your name on the emails. Good to know you are involved.
I'm not sure we can count on EFI-enabled RISC-V disk images popping up very quickly though, so I would like to improve the situation for the disk images that are out there right now and need U-Boot to run.
EFI works and some are already using it with U-Boot IIRC. Even smaller SBCs (e.g. based on StarFive JH7110) gained EDK2 port (probably not upstreamed yet).
https://github.com/starfive-tech/edk2/releases
So even sub-100 USD boards will get EDK2 :) Of course, that doesn't incl. ACPI.
Cheers, david
-- Andrea Bolognani / Red Hat / Virtualization
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Tue, May 23, 2023 at 6:35 PM Richard W.M. Jones <rjones@redhat.com> wrote:
On Tue, May 23, 2023 at 06:32:07PM +0300, David Abdurachmanov wrote:
On Tue, May 23, 2023 at 6:12 PM Andrea Bolognani <abologna@redhat.com> wrote:
On Tue, May 23, 2023 at 04:05:04PM +0300, David Abdurachmanov wrote:
On Tue, May 23, 2023 at 3:18 PM Andrea Bolognani <abologna@redhat.com> wrote:
On Tue, May 23, 2023 at 11:59:41AM +0100, Richard W.M. Jones wrote:
I just came across this thread while trying to update the libvirt instructions on that page. Specifically I need to add these to the qemu command line:
-bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000
Anyway, we found that the -device loader incantation above is effectively equivalent to -kernel, so you can just use that instead. Both -bios and -kernel are exposed by libvirt through XML elements of the same names.
Further, since QEMU loads OpenSBI as -bios by default these days, you don't even need that part and can just use -kernel to point to the u-boot.bin file. In this case, you should use the file that would be installed as
We run QEMU in a similar way as any other board.
You mean physical boards?
Yes
So it's U-Boot SPL, which then loads U-Boot ITB (which typically contains U-Boot proper, OpenSBI FW_DYNAMIC generic binary, and DTB). U-Boot SPL transfers control to OpenSBI and tells it how to load the next stage (i.e U-Boot proper).
That sounds a bit roundabout when you consider that QEMU automatically loads OpenSBI, and that in turn knows to jump to a S-Mode payload such as the Linux kernel or an S-Mode build of U-Boot.
Put it another way: do you have any objections to loading qemu-riscv64_smode/u-boot.bin via -kernel as the default boot strategy for riscv64 VMs? That's what every OS other than Fedora already recommends doing.
We typically use a non-release version of OpenSBI, because it has the latest hardware support and maybe some fixes.
For libvirt users (out-of-the box experience) that's the correct approach.
Loading the SPL and ITB separately would still be possible, of course. I'm simply talking about the default experience.
Yeah, this default is sane.
/usr/share/uboot/qemu-riscv64_smode/u-boot.bin
by the uboot-images-riscv64 Fedora package.
This is noarch package, and thus available on all arches. This package is only available in Fedora/RISCV Koji for now.
You're right! I got confused because "riscv64" is in the name O:-)
Is there ongoing work to move this package to Fedora proper? In terms of user experience, having to download disk images from a third-party is mostly fine, but installing third-party RPMs on the host... Not so much. Having this in Fedora proper would make it a fully self-contained host for RISC-V TCG VMs.
It could go as-is, but most likely will need to change. So far we relied on a single OpenSBI FW_DYNAMIC generic image, but that changes starting with JH7110 which has a different (i.e. non default) load address. Thus we might need to build OpenSBI per board (or similar), and pick the right one while building U-Boot.
Unknown (at least for me) how things will work on TH1520 based boards too.
One thing I am sure about is that things in the SPEC file most likely will change once more boards from multiple vendors are supported.
We can push the current thing as-is if you want.
The "hump" here is getting it through the Fedora approval process. Once in we can add additional features very easily.
If you want me to do this let me know, I can start right away (still fiddling around with benchmarking that qemu instance so I'm not fully occupied right now).
There are a few items I posted today on Fedora RISC-V Matrix/IRC channel. There is plenty of work ;) Sure. Technically there are two opensbi packages we have in Fedora/RISCV: "opensbi" (stable, we never use it), and "opensbi-unstable" (basically OpenSBI master branch, which we always use [hardware support]). So look at those to figure out the best action plan. Technically QEMU ships with its own OpenSBI. Did we ever get that sorted out in Fedora too? I recall you working on it a year or more ago. david
Rich.
For the benefit of the search engine gods, this works for now:
# virt-install --import --memory 8192 -n fedora-37-riscv \ --arch riscv64 --vcpus 8 \ --disk fedora-37-riscv.qcow2,format=qcow2 \ --osinfo fedora37 \ --qemu-commandline=' -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 '
This doesn't disable Sv48 and Sv57. I don't know the overall status, but at least Golang 1.20 has a fix to support anything above Sv39.
Not sure about other runtimes that do pointer tagging.
See: https://bugs.launchpad.net/ubuntu/+source/linux-riscv/+bug/1991790
IIUC that's something that needs to be handled in the kernel? Or do we need to do something at the QEMU/libvirt/virt-install level to make things work?
Basically two workarounds were created: - Ability to control stap mode in QEMU (basically options to enable/disable Sv39, Sv48, Sv57). - Ability to disable that on the kernel side.
I think the 1st one to arrive was QEMU.
IIRC x86_64 and aarch64 are smart in the way they give virtual addresses to user space. Which is not the case for riscv (yet).
On the kernel side, that's part of 6.4 kernel (not released yet).
Simply put, older disk images on newer QEMU versions might not work properly.
I'm not too concerned about this. In the long run, it will simply not matter.
True.
Note that we might want to switch to EDK2 in the future for QEMU, and that probably will use two 32MiB pflash devices in virt machine. I have seen, but haven't tested QEMU + EDK2 patches.
Yup, there's some discussion about this very topic happening right now on the QEMU mailing list. I'm actively involved in it and it looks like the necessary changes might be merged soon.
Yup. I noticed your name on the emails. Good to know you are involved.
I'm not sure we can count on EFI-enabled RISC-V disk images popping up very quickly though, so I would like to improve the situation for the disk images that are out there right now and need U-Boot to run.
EFI works and some are already using it with U-Boot IIRC. Even smaller SBCs (e.g. based on StarFive JH7110) gained EDK2 port (probably not upstreamed yet).
https://github.com/starfive-tech/edk2/releases
So even sub-100 USD boards will get EDK2 :) Of course, that doesn't incl. ACPI.
Cheers, david
-- Andrea Bolognani / Red Hat / Virtualization
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
On Tue, May 23, 2023 at 06:32:07PM +0300, David Abdurachmanov wrote:
On Tue, May 23, 2023 at 6:12 PM Andrea Bolognani <abologna@redhat.com> wrote:
Put it another way: do you have any objections to loading qemu-riscv64_smode/u-boot.bin via -kernel as the default boot strategy for riscv64 VMs? That's what every OS other than Fedora already recommends doing.
We typically use a non-release version of OpenSBI, because it has the latest hardware support and maybe some fixes.
For libvirt users (out-of-the box experience) that's the correct approach.
Loading the SPL and ITB separately would still be possible, of course. I'm simply talking about the default experience.
Yeah, this default is sane.
Okay, I will start working on libvirt patches then!
Is there ongoing work to move this package to Fedora proper? In terms of user experience, having to download disk images from a third-party is mostly fine, but installing third-party RPMs on the host... Not so much. Having this in Fedora proper would make it a fully self-contained host for RISC-V TCG VMs.
It could go as-is, but most likely will need to change. So far we relied on a single OpenSBI FW_DYNAMIC generic image, but that changes starting with JH7110 which has a different (i.e. non default) load address. Thus we might need to build OpenSBI per board (or similar), and pick the right one while building U-Boot.
Unknown (at least for me) how things will work on TH1520 based boards too.
One thing I am sure about is that things in the SPEC file most likely will change once more boards from multiple vendors are supported.
These changes wouldn't affect the qemu-riscv64_smode U-Boot build though, would they? I don't want to discount the usefulness of other builds, but as far as my interests are concerned that's basically the only one I care about O:-)
We can push the current thing as-is if you want.
I understand that the process of getting packages accepted into Fedora can be lengthy, so starting it sooner rather than later would probably be a good idea.
This doesn't disable Sv48 and Sv57. I don't know the overall status, but at least Golang 1.20 has a fix to support anything above Sv39.
Not sure about other runtimes that do pointer tagging.
See: https://bugs.launchpad.net/ubuntu/+source/linux-riscv/+bug/1991790
IIUC that's something that needs to be handled in the kernel? Or do we need to do something at the QEMU/libvirt/virt-install level to make things work?
Basically two workarounds were created: - Ability to control stap mode in QEMU (basically options to enable/disable Sv39, Sv48, Sv57). - Ability to disable that on the kernel side.
I think the 1st one to arrive was QEMU.
IIRC x86_64 and aarch64 are smart in the way they give virtual addresses to user space. Which is not the case for riscv (yet).
On the kernel side, that's part of 6.4 kernel (not released yet).
Got it. I don't think we're quite ready to start dealing with RISC-V CPU features in libvirt yet. Luckily, Linux 6.4 should be right around the corner :) -- Andrea Bolognani / Red Hat / Virtualization
On Tue, May 23, 2023 at 08:12:17AM -0700, Andrea Bolognani wrote:
On Tue, May 23, 2023 at 04:05:04PM +0300, David Abdurachmanov wrote:
On Tue, May 23, 2023 at 3:18 PM Andrea Bolognani <abologna@redhat.com> wrote:
On Tue, May 23, 2023 at 11:59:41AM +0100, Richard W.M. Jones wrote:
I just came across this thread while trying to update the libvirt instructions on that page. Specifically I need to add these to the qemu command line:
-bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000
Anyway, we found that the -device loader incantation above is effectively equivalent to -kernel, so you can just use that instead. Both -bios and -kernel are exposed by libvirt through XML elements of the same names.
Further, since QEMU loads OpenSBI as -bios by default these days, you don't even need that part and can just use -kernel to point to the u-boot.bin file. In this case, you should use the file that would be installed as
We run QEMU in a similar way as any other board.
You mean physical boards?
So it's U-Boot SPL, which then loads U-Boot ITB (which typically contains U-Boot proper, OpenSBI FW_DYNAMIC generic binary, and DTB). U-Boot SPL transfers control to OpenSBI and tells it how to load the next stage (i.e U-Boot proper).
That sounds a bit roundabout when you consider that QEMU automatically loads OpenSBI, and that in turn knows to jump to a S-Mode payload such as the Linux kernel or an S-Mode build of U-Boot.
Put it another way: do you have any objections to loading qemu-riscv64_smode/u-boot.bin via -kernel as the default boot strategy for riscv64 VMs? That's what every OS other than Fedora already recommends doing.
Loading the SPL and ITB separately would still be possible, of course. I'm simply talking about the default experience.
/usr/share/uboot/qemu-riscv64_smode/u-boot.bin
by the uboot-images-riscv64 Fedora package.
This is noarch package, and thus available on all arches. This package is only available in Fedora/RISCV Koji for now.
You're right! I got confused because "riscv64" is in the name O:-)
Is there ongoing work to move this package to Fedora proper? In terms of user experience, having to download disk images from a third-party is mostly fine, but installing third-party RPMs on the host... Not so much. Having this in Fedora proper would make it a fully self-contained host for RISC-V TCG VMs.
In the interesting of being useful for once I could do this. What do you think David?
For the benefit of the search engine gods, this works for now:
# virt-install --import --memory 8192 -n fedora-37-riscv \ --arch riscv64 --vcpus 8 \ --disk fedora-37-riscv.qcow2,format=qcow2 \ --osinfo fedora37 \ --qemu-commandline=' -bios /path/to/u-boot-spl.bin -device loader,file=/path/to/u-boot.itb,addr=0x80200000 '
This doesn't disable Sv48 and Sv57. I don't know the overall status, but at least Golang 1.20 has a fix to support anything above Sv39.
Not sure about other runtimes that do pointer tagging.
See: https://bugs.launchpad.net/ubuntu/+source/linux-riscv/+bug/1991790
IIUC that's something that needs to be handled in the kernel? Or do we need to do something at the QEMU/libvirt/virt-install level to make things work?
David mentioned these CPU flags [qemu option]: -cpu rv64,sv57=off,sv48=off However it seems this is only a temporary workaround until various language runtimes are fixed.
Simply put, older disk images on newer QEMU versions might not work properly.
I'm not too concerned about this. In the long run, it will simply not matter.
Note that we might want to switch to EDK2 in the future for QEMU, and that probably will use two 32MiB pflash devices in virt machine. I have seen, but haven't tested QEMU + EDK2 patches.
Yup, there's some discussion about this very topic happening right now on the QEMU mailing list. I'm actively involved in it and it looks like the necessary changes might be merged soon.
I'm not sure we can count on EFI-enabled RISC-V disk images popping up very quickly though, so I would like to improve the situation for the disk images that are out there right now and need U-Boot to run.
Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit
All the code present in qemuFirmwareFillDomain() assumes that loader->path is always filled if using manual firmware selection. In the newly added "<loader type='none'/>" case, i.e. without using firmware autoselection, qemuFirmwareFillDomain() will call qemuFirmwareFillDomainModern(), which in turn will fetch the number of firmwares in the driver via qemuFirmwareFetchParsedConfigs(). If any firmware is found, qemuFirmwareFillDomainModern() will call qemuFirmwareMatchDomain(), and we'll SIGSEV in: STRNEQ(loader->path, fw->mapping.data.flash.executable.filename)) { Because we never checked if loader->path != NULL ever since the start of qemuFirmwareFillDomain(), 2 callers before. This doesn't happen in the field because, at this moment, there is no RISC-V firmwares set in the live driver. But the test driver from qemuxml2argvdata will populate the list with some firmwares, triggering the call to qemuFirmwareMatchDomain() that causes the seg fault. We'll hit this SIGSEV when adding a xml2xargv test that uses loader type='none'. One fix is to use STRNEQ_NULLABLE() in the forementioned line, , but doing that doesn't fix the loader->path != NULL assumption that we're making in qemuFirmwareFillDomain(). Let's instead exit early in that function if we're dealing with the loader type='none' scenario we're now supporting: no firmware autoselection, loader->type == none, loader->path == NULL. Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> --- src/qemu/qemu_firmware.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/qemu/qemu_firmware.c b/src/qemu/qemu_firmware.c index 9de4166772..8541a57bf6 100644 --- a/src/qemu/qemu_firmware.c +++ b/src/qemu/qemu_firmware.c @@ -1616,6 +1616,16 @@ qemuFirmwareFillDomain(virQEMUDriver *driver, return -1; } + /* If we're not autoselecting a firmware, and we have a loader + * element, and loader type is 'none', and we don't have a + * loader->path, consider that the user wants to explictly + * disable the firmware selection in QEMU (-bios none). */ + if (!autoSelection && loader && + loader->type == VIR_DOMAIN_LOADER_TYPE_NONE && + !loader->path) { + return 0; + } + /* If firmware autoselection is disabled and the loader is a ROM * instead of a PFLASH device, then we're using BIOS and we don't * need any information at all */ -- 2.39.2
Also add a qemuxml2argvtest to ensure that we're good. Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> --- src/qemu/qemu_command.c | 6 ++++ .../firmware-bios-none.riscv64-latest.args | 31 +++++++++++++++++++ tests/qemuxml2argvdata/firmware-bios-none.xml | 18 +++++++++++ tests/qemuxml2argvtest.c | 2 ++ 4 files changed, 57 insertions(+) create mode 100644 tests/qemuxml2argvdata/firmware-bios-none.riscv64-latest.args create mode 100644 tests/qemuxml2argvdata/firmware-bios-none.xml diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 7e75354902..97944dad55 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -9366,6 +9366,12 @@ qemuBuildDomainLoaderCommandLine(virCommand *cmd, break; case VIR_DOMAIN_LOADER_TYPE_NONE: + if (!loader->path) { + virCommandAddArg(cmd, "-bios"); + virCommandAddArg(cmd, "none"); + } + break; + case VIR_DOMAIN_LOADER_TYPE_LAST: /* nada */ break; diff --git a/tests/qemuxml2argvdata/firmware-bios-none.riscv64-latest.args b/tests/qemuxml2argvdata/firmware-bios-none.riscv64-latest.args new file mode 100644 index 0000000000..71efe0361e --- /dev/null +++ b/tests/qemuxml2argvdata/firmware-bios-none.riscv64-latest.args @@ -0,0 +1,31 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-guest \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-guest/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-guest/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-guest/.config \ +/usr/bin/qemu-system-riscv64 \ +-name guest=guest,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-guest/master-key.aes"}' \ +-machine virt,usb=off,dump-guest-core=off,memory-backend=riscv_virt_board.ram \ +-accel tcg \ +-bios none \ +-m 1024 \ +-object '{"qom-type":"memory-backend-ram","id":"riscv_virt_board.ram","size":1073741824}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid 63840878-0deb-4095-97e6-fc444d9bc9fa \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=1729,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/firmware-bios-none.xml b/tests/qemuxml2argvdata/firmware-bios-none.xml new file mode 100644 index 0000000000..b8604e6b8a --- /dev/null +++ b/tests/qemuxml2argvdata/firmware-bios-none.xml @@ -0,0 +1,18 @@ +<domain type='qemu'> + <name>guest</name> + <uuid>63840878-0deb-4095-97e6-fc444d9bc9fa</uuid> + <memory unit='KiB'>1048576</memory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='riscv64' machine='virt'>hvm</type> + <loader type='none'/> + </os> + <features> + <acpi/> + </features> + <devices> + <emulator>/usr/bin/qemu-system-riscv64</emulator> + <controller type='usb' model='none'/> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 293aea60d5..660d3ffca5 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1135,6 +1135,8 @@ mymain(void) DO_TEST_CAPS_ARCH_LATEST("firmware-auto-efi-format-loader-raw", "aarch64"); DO_TEST_CAPS_LATEST_PARSE_ERROR("firmware-auto-efi-format-mismatch"); + DO_TEST_CAPS_ARCH_LATEST("firmware-bios-none", "riscv64"); + DO_TEST_NOCAPS("clock-utc"); DO_TEST_NOCAPS("clock-localtime"); DO_TEST_NOCAPS("clock-localtime-basis-localtime"); -- 2.39.2
Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> --- docs/formatdomain.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 27f83e254d..3529c7a9c5 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -263,6 +263,13 @@ harddisk, cdrom, network) determining where to obtain/find the boot image. specific format. Supported values are ``raw`` and ``qcow2``. :since:`Since 9.2.0 (QEMU only)` + When firmware autoselection is not enabled, marking the ``type`` attribute + as ``none`` without any file path will set the QEMU BIOS file path as + ``none``. This is used by architectures that will always load the default + BIOS image otherwise, making them unable to start the domain when using + a guest kernel that overwrites the memory the default BIOS image uses. + :since:`Since 9.3.0 (QEMU only)` + ``nvram`` Some UEFI firmwares may want to use a non-volatile memory to store some variables. In the host, this is represented as a file and the absolute path -- 2.39.2
participants (6)
-
Andrea Bolognani -
Daniel Henrique Barboza -
Daniel P. Berrangé -
David Abdurachmanov -
Gerd Hoffmann -
Richard W.M. Jones