Hi Andrea:
I'm sorry for taking so long to reply you!
Hi Philippe:
When developing libvirt on loongarch, we encountered some problems
related to pflash.
libvirt and qemu met some difficulties in the coordination of UEFI loading.
I think we need your suggestions and opinions on the solution.
On Fri, Jan 12, 2024 at 02:32:43PM +0800, Xianglai Li wrote:
> Hello, Everyone:
> This patch series adds libvirt support for loongarch.Although the bios
> path and name has not been officially integrated into qemu and we think
> there are still many shortcomings, we try to push a version of patch to
> the community according to the opinions of the community, hoping to
> listen to everyone's opinions. Anyway we have a version of libvirt that
> supports loongarch.
>
> You can also get libvirt's patch from the link below:
>
https://gitlab.com/lixianglai/libvirt
> branch: loongarch
>
> Since the patch associated with loongarch has not yet been submitted to
> the virt-manager community, we are providing a temporary patch with
> loongarch for the time being patch's virt-manager, the open source work
> of virt-manager adding loongarch will be followed up later or
> synchronized with the open source libvirt.
>
> You can get the virt-manager code with loongarch patch from the link below:
>
https://github.com/lixianglai/virt-manager
> branch: loongarch
It's really great that you're considering the entire stack, all the
way up to virt-manager, while working on this! :)
Oh! Thanks!
:)
> loongarch's virtual machine bios is not yet available in
qemu, so you can get it from the following link
>
https://github.com/lixianglai/LoongarchVirtFirmware
> (Note: You should clone the repository using git instead of downloading the file via
wget or you'll get xml)
> We named the bios QEMU_EFI.fd, QEMU_VARS.fd is used to store pflash images of
non-volatile
> variables.After installing qemu-system-loongarch64,You can install the loongarch bios
by executing the script
> install-loongarch-virt-firmware.sh
To clarify, loongarch support has been merged into upstream edk2
already, right? And the contents of this repository are just for
convenience, since loongarch builds of edk2 have generally not made
it into distros yet? I think I might have asked about this already,
sorry O:-)
Don't mention it.
All right, The loongarch support has been merged into upstream edk2 already.
And you can see a detailed description of loongarch uefi in the link below:
https://github.com/tianocore/edk2-platforms/blob/master/Platform/Loongson...
this repository are just for convenience. :)
Anyway, I fetched and installed this. The firmware descriptor looks
like:
{
"interface-types": [
"uefi"
],
"mapping": {
"device": "memory",
"filename": "/usr/share/edk2/loongarch64/QEMU_EFI.fd"
},
"targets": [
{
"architecture": "loongarch64",
"machines": [
"virt",
"virt-*"
]
}
],
"features": [
"acpi"
]
}
This is not what I expected: specifically, it results in libvirt
generating
-bios /usr/share/edk2/loongarch64/QEMU_EFI.fd
So only one of the two files is used, in read-only mode, and there is
no persistent NVRAM storage that the guest can use.
This is what I expected instead:
{
"interface-types": [
"uefi"
],
"mapping": {
"device": "flash",
"mode": "split",
"executable": {
"filename": "/usr/share/edk2/loongarch64/QEMU_EFI.fd",
"format": "raw"
},
"nvram-template": {
"filename": "/usr/share/edk2/loongarch64/QEMU_VARS.fd",
"format": "raw"
}
},
"targets": [
{
"architecture": "loongarch64",
"machines": [
"virt",
"virt-*"
]
}
],
"features": [
"acpi"
]
}
I've tried installing such a descriptor and libvirt picks it up,
resulting in the following guest configuration:
<os firmware='efi'>
<type arch='loongarch64' machine='virt'>hvm</type>
<firmware>
<feature enabled='no' name='enrolled-keys'/>
<feature enabled='no' name='secure-boot'/>
</firmware>
<loader readonly='yes'
type='pflash'>/usr/share/edk2/loongarch64/QEMU_EFI.fd</loader>
<nvram
template='/usr/share/edk2/loongarch64/QEMU_VARS.fd'>/var/lib/libvirt/qemu/nvram/guest_VARS.fd</nvram>
<boot dev='hd'/>
</os>
which in turn produces the following QEMU command line options:
-blockdev
'{"driver":"file","filename":"/usr/share/edk2/loongarch64/QEMU_EFI.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}'
-blockdev
'{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}'
-blockdev
'{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/guest_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}'
-blockdev
'{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}'
Unfortunately, with this configuration the guest fails to start:
qemu-system-loongarch64: Property 'virt-machine.pflash0' not found
This error message looked familiar to me, as it is the same that I
hit when trying out UEFI support on RISC-V roughly a year ago[1]. In
this case, however, it seems that the issue runs deeper: it's not
just that the flash devices are not wired up to work as blockdevs,
but even the old -drive syntax doesn't work.
Looking at the QEMU code, it appears that the loongarch/virt machine
only creates a single pflash device and exposes it via -bios. So it
seems that there is simply no way to achieve the configuration that
we want.
I think that this is something that needs to be addressed as soon as
possible. In the long run, guest-accessible NVRAM storage is a must,
and I'm not sure it would make a lot of sense to merge loongarch
support into libvirt until the firmware situation has been sorted out
in the lower layers.
In the qemu code, loongarch virt machine does only create a pflash,
which is used for nvram, and uefi code is loaded by rom.
In summary, loongarch virt machine can use nvram with the following command:
-------------------------------------------------------------------------------------------------------
/qemu-system-loongarch64 \//
//-m 8G \//
//-smp 4 \//
//-cpu la464 \//
//-blockdev
'{"driver":"file","filename":"./QEMU_VARS-pflash.raw","node-name":"libvirt-pflash0-storage","auto-read-only":false,"discard":"unmap"}'
\//
//-blockdev
'{"node-name":"libvirt-pflash0-format","read-only":false,"driver":"raw","file":"libvirt-pflash0-storage"}'
\//
//-machine virt,pflash=libvirt-pflash0-format \//
//-snapshot \//
//-bios ./QEMU_EFI.fd \//
//-serial stdio/
-------------------------------------------------------------------------------------------------------
This is really a big difference from the following boot method, and it
still looks weird.
-------------------------------------------------------------------------------------------------------
/-blockdev
'{"driver":"file","filename":"/usr/share/edk2/loongarch64/QEMU_EFI.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}'
-blockdev
'{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}'
-blockdev
'{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/guest_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}'
-blockdev
'{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}'/
-------------------------------------------------------------------------------------------------------
However, during the development of qemu loongarch,
we also used a RISCV-like solution to create two pflash,
but the qemu community suggested that we put uefi code in rom for the
following reasons:
https://lore.kernel.org/qemu-devel/2f381d06-842f-ac8b-085c-0419675a4872@l...
"
Since you are starting a virtual machine from scratch, you should take
the opportunity to learn from other early mistakes. X86 ended that way
due to 1/ old firmwares back-compability and 2/ QEMU pflash block
protections not being implemented. IIUC if we were starting with a
UEFI firmware today, the layout design (still using QEMU) would be
to map the CODE area in a dumb ROM device, and the VARSTORE area
in a PFlash device. Since Virt machines don't need to use Capsule
update, having the CODE area in ROM drastically simplifies the design
and maintainance.
"
Well, anyway, now that we have an issue with qemu loongarch using nvram
that is incompatible with libvirt,
here I have come up with two solutions to solve this problem:
Option 1:
If the interface type "rom-uefi" is added and the device type
"rom-flash" is added, the json file should be written like this:
-------------------------------------------------------------------------------------------------------
/{//
// "interface-types": [//
////"rom-uefi"//
// ],//
// "mapping": {//
////"device": "rom-flash",//
// "executable": {//
// "filename": "/usr/share/edk2/loongarch64/QEMU_EFI.fd",//
// "format": "raw"//
// },//
// "nvram-template": {//
// "filename": "/usr/share/edk2/loongarch64/QEMU_VARS.fd",//
// "format": "raw"//
// }//
// },//
// "targets": [//
// {//
// "architecture": "loongarch64",//
// "machines": [//
// "virt",//
// "virt-*"//
// ]//
// }//
// ],//
// "features": [//
// "acpi"//
// ]/
-------------------------------------------------------------------------------------------------------
Then add the parsing of the new interface types in libvirt and load
QEMU_CODE.fd as -bios and QEMU_VARS.fd as nvram
when creating the command line, generating commands like the following:
-------------------------------------------------------------------------------------------------------
/qemu-system-loongarch64 \//
//-m 8G \//
//-smp 1 \//
//-cpu la464 \//
//-blockdev
'{"driver":"file","filename":"////usr/share/edk2/loongarch64/QEMU_VARS.fd/","node-name":"libvirt-pflash0-storage","auto-read-only":false,"discard":"unmap"}'
\//
//-blockdev
'{"node-name":"libvirt-pflash0-format","read-only":false,"driver":"raw","file":"libvirt-pflash0-storage"}'
\//
//-machine virt,pflash=libvirt-pflash0-format \//
//-snapshot \//
//-bios /////usr/share/edk2/loongarch64///QEMU_EFI.fd \//
//-serial stdio/
-------------------------------------------------------------------------------------------------------
Option 2:
Solution 2 mainly starts from qemu. Now the rom that bios is loaded into
is a memory region that cannot be configured with attributes,
so we imagine abstracting rom as a device, creating it during machine
initialization and setting "pflash0" attribute for it.
Then create a pflash and set its property to "pflash1", so our startup
command will look like this:
-------------------------------------------------------------------------------------------------------
/qemu-system-loongarch64 \//
//-m 8G \//
//-smp 1 \//
//-cpu la464 \//
//-blockdev
'{"driver":"file","filename":"/usr/share/edk2/loongarch64/QEMU_EFI.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}'
\//
//-blockdev
'{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}'
\//
//-blockdev
'{"driver":"file","filename":"/usr/share/edk2/loongarch64/QEMU_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}'
\//
//-blockdev
'{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}'
\//
//-machine
virt,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \//
//-snapshot \//
//-serial stdio/
-------------------------------------------------------------------------------------------------------
This way, without modifying libvirt, QEMU_CODE.fd can be loaded into the
rom,
but it is still a little strange that it is clearly rom but set a
"pflash0" attribute, which can be confusing.
> Since there is no fedora operating system that supports the
loongarch
> architecture, you can find an iso that supports loongarch at the link
> below for testing purposes:
>
https://github.com/fedora-remix-loongarch/releases-info
>
> Well, if you have completed the above steps I think you can now install loongarch
virtual machine,
> you can install it through the virt-manager graphical interface, or install it
through vrit-install,
> here is an example of installing it using virt-install:
>
> virt-install \
> --virt-type=qemu \
> --name loongarch-test \
> --memory 4096 \
> --vcpus=4 \
> --arch=loongarch64 \
> --boot cdrom \
> --disk device=cdrom,bus=scsi,path=/root/livecd-fedora-mate-4.loongarch64.iso \
> --disk
path=/var/lib/libvirt/images/debian12-loongarch64.qcow2,size=10,format=qcow2,bus=scsi \
> --network network=default \
> --osinfo archlinux \
> --video=virtio \
> --graphics=vnc,listen=0.0.0.0
Regardless of what I said above, this actually worked quite well! I
was able to boot the MATE image and, ignoring the expected slowness
caused by emulation, things were generally fairly smooth.
Note that I tried the LXDE image first, expecting it to be a bit
smaller and faster, but in that case I wasn't able to get a
successful boot: edk2 loaded up just fine, but the guest OS was never
started and I was dropped into the edk2 shell instead. Maybe that's a
known issue, but I thought I'd point it out.
Thank you very much for pointing out the problem,
lxde iso is missing efiboot.img file so that the virtual machine UEFI
can not recognize efi partition
caused by the problem has been notified to the relevant personnel,
I believe that the next version of libvirt patch will be shipped to use
lxde iso.
Thanks,
Xianglai.