Re: [libvirt PATCH V2 0/4] add loongarch support for libvirt

> Hi Philippe: > >     When developing libvirt on loongarch, we encountered some problems > related to pflash. > > libvirt and qemu met some difficulties in the coordination of UEFI loading. > > I think we need your suggestions and opinions on the solution. > >> Anyway, I fetched and installed this. The firmware descriptor looks >> like: >> >> { >> "interface-types": [ >> "uefi" >> ], >> "mapping": { >> "device": "memory", >> "filename": "/usr/share/edk2/loongarch64/QEMU_EFI.fd" >> }, >> "targets": [ >> { >> "architecture": "loongarch64", >> "machines": [ >> "virt", >> "virt-*" >> ] >> } >> ], >> "features": [ >> "acpi" >> ] >> } >> >> This is not what I expected: specifically, it results in libvirt >> generating >> >> -bios /usr/share/edk2/loongarch64/QEMU_EFI.fd >> >> So only one of the two files is used, in read-only mode, and there is >> no persistent NVRAM storage that the guest can use. >> >> This is what I expected instead: >> >> { >> "interface-types": [ >> "uefi" >> ], >> "mapping": { >> "device": "flash", >> "mode": "split", >> "executable": { >> "filename": "/usr/share/edk2/loongarch64/QEMU_EFI.fd", >> "format": "raw" >> }, >> "nvram-template": { >> "filename": "/usr/share/edk2/loongarch64/QEMU_VARS.fd", >> "format": "raw" >> } >> }, >> "targets": [ >> { >> "architecture": "loongarch64", >> "machines": [ >> "virt", >> "virt-*" >> ] >> } >> ], >> "features": [ >> "acpi" >> ] >> } >> >> I've tried installing such a descriptor and libvirt picks it up, >> resulting in the following guest configuration: >> >> <os firmware='efi'> >> <type arch='loongarch64' machine='virt'>hvm</type> >> <firmware> >> <feature enabled='no' name='enrolled-keys'/> >> <feature enabled='no' name='secure-boot'/> >> </firmware> >> <loader readonly='yes' >> type='pflash'>/usr/share/edk2/loongarch64/QEMU_EFI.fd</loader> >> <nvram template='/usr/share/edk2/loongarch64/QEMU_VARS.fd'>/var/lib/libvirt/qemu/nvram/guest_VARS.fd</nvram> >> <boot dev='hd'/> >> </os> >> >> which in turn produces the following QEMU command line options: >> >> -blockdev '{"driver":"file","filename":"/usr/share/edk2/loongarch64/QEMU_EFI.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' >> -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' >> -blockdev '{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/guest_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' >> -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' >> >> Unfortunately, with this configuration the guest fails to start: >> >> qemu-system-loongarch64: Property 'virt-machine.pflash0' not found >> >> This error message looked familiar to me, as it is the same that I >> hit when trying out UEFI support on RISC-V roughly a year ago[1]. In >> this case, however, it seems that the issue runs deeper: it's not >> just that the flash devices are not wired up to work as blockdevs, >> but even the old -drive syntax doesn't work. >> >> Looking at the QEMU code, it appears that the loongarch/virt machine >> only creates a single pflash device and exposes it via -bios. So it >> seems that there is simply no way to achieve the configuration that >> we want. >> >> I think that this is something that needs to be addressed as soon as >> possible. In the long run, guest-accessible NVRAM storage is a must, >> and I'm not sure it would make a lot of sense to merge loongarch >> support into libvirt until the firmware situation has been sorted out >> in the lower layers. > In the qemu code, loongarch virt machine does only create a pflash, > > which is used for nvram, and uefi code is loaded by rom. > > In summary, loongarch virt machine can use nvram with the following command: > > ------------------------------------------------------------------------------------------------------- > > qemu-system-loongarch64 \ > -m 8G \ > -smp 4 \ > -cpu la464 \ > -blockdev '{"driver":"file","filename":"./QEMU_VARS-pflash.raw","node-name":"libvirt-pflash0-storage","auto-read-only":false,"discard":"unmap"}' \ > -blockdev '{"node-name":"libvirt-pflash0-format","read-only":false,"driver":"raw","file":"libvirt-pflash0-storage"}' \ > -machine virt,pflash=libvirt-pflash0-format \ > -bios ./QEMU_EFI.fd > > ------------------------------------------------------------------------------------------------------- > > > This is really a big difference from the following boot method, and it still > looks weird. > > ------------------------------------------------------------------------------------------------------- > > -blockdev '{"driver":"file","filename":"/usr/share/edk2/loongarch64/QEMU_EFI.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' > -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' > -blockdev '{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/guest_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' > -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' > > ------------------------------------------------------------------------------------------------------- > > However, during the development of qemu loongarch, > > we also used a RISCV-like solution to create two pflash, > > but the qemu community suggested that we put uefi code in rom for the > following reasons: > > > https://lore.kernel.org/qemu-devel/2f381d06-842f-ac8b-085c-0419675a4872@l... > > " > > Since you are starting a virtual machine from scratch, you should take > the opportunity to learn from other early mistakes. X86 ended that way > due to 1/ old firmwares back-compability and 2/ QEMU pflash block > protections not being implemented. IIUC if we were starting with a > UEFI firmware today, the layout design (still using QEMU) would be > to map the CODE area in a dumb ROM device, and the VARSTORE area > in a PFlash device. Since Virt machines don't need to use Capsule > update, having the CODE area in ROM drastically simplifies the design > and maintainance. > > " > > Well, anyway, now that we have an issue with qemu loongarch using nvram that > is incompatible with libvirt, > > here I have come up with two solutions to solve this problem: > > > Option 1: > > If the interface type "rom-uefi" is added and the device type "rom-flash" is > added, the json file should be written like this: > > ------------------------------------------------------------------------------------------------------- > > { >    "interface-types": [ > "rom-uefi" >    ], >    "mapping": { > "device": "rom-flash", >      "executable": { >        "filename": "/usr/share/edk2/loongarch64/QEMU_EFI.fd", >        "format": "raw" >      }, >      "nvram-template": { >        "filename": "/usr/share/edk2/loongarch64/QEMU_VARS.fd", >        "format": "raw" >      } >    }, >    "targets": [ >      { >        "architecture": "loongarch64", >        "machines": [ >          "virt", >          "virt-*" >        ] >      } >    ], >    "features": [ >        "acpi" >    ] > >  ------------------------------------------------------------------------------------------------------- > > Then add the parsing of the new interface types in libvirt and load > QEMU_CODE.fd as -bios and QEMU_VARS.fd as nvram > > when creating the command line, generating commands like the following: > >  ------------------------------------------------------------------------------------------------------- > > -blockdev '{"driver":"file","filename":"/usr/share/edk2/loongarch64/QEMU_VARS.fd/","node-name":"libvirt-pflash0-storage","auto-read-only":false,"discard":"unmap"}' \ > -blockdev '{"node-name":"libvirt-pflash0-format","read-only":false,"driver":"raw","file":"libvirt-pflash0-storage"}' \ > -machine virt,pflash=libvirt-pflash0-format \ > -bios /usr/share/edk2/loongarch64///QEMU_EFI.fd \ > > ------------------------------------------------------------------------------------------------------- > > Option 2: > > Solution 2 mainly starts from qemu. Now the rom that bios is loaded into is > a memory region that cannot be configured with attributes, > > so we imagine abstracting rom as a device, creating it during machine > initialization and setting "pflash0" attribute for it. > > Then create a pflash and set its property to "pflash1", so our startup > command will look like this: > >  ------------------------------------------------------------------------------------------------------- > > -blockdev '{"driver":"file","filename":"/usr/share/edk2/loongarch64/QEMU_EFI.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \ > -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \ > -blockdev '{"driver":"file","filename":"/usr/share/edk2/loongarch64/QEMU_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \ > -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \ > -machine virt,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \ > >  ------------------------------------------------------------------------------------------------------- > > This way, without modifying libvirt, QEMU_CODE.fd can be loaded into the > rom, > > but it is still a little strange that it is clearly rom but set a "pflash0" > attribute, which can be confusing. We recently had a very similar discussion regarding EFI booting on RISC-V. Personally I would prefer to see the approach with two pflash devices, one read-only and one read/write, adopted. This is pretty much the de-facto standard across architectures: x86_64, aarch64 and riscv64 all boot edk2 this way. I understand the desire to simplify things where possible, and I am sympathetic towards it. If we could boot from just rom, without using pflash at all, I would definitely see the appeal. However, as noted earlier, in the case of EFI having some read/write storage space is necessary to expose the full functionality, so going without it is not really an option. With all the above in mind, the cost of loongarch64 doing things differently from other architectures seems like it would outweight the benefits, and I strongly advise against it.