Hi, With qemu using this itself in mind it probably makes sense to specify this as qapi schema. That'll simplify parsing and using these files in qemu, and possibly simplifies things on the libvirt side too. cheers, Gerd

On 03/08/18 11:17, Daniel P. Berrangé wrote: I've created an RFC-level "qapi/firmware.json" schema file, based on this discussion. It "builds", and the generated documentation looks acceptable, superficially speaking. Before I post "qapi/firmware.json" for getting comments, I'd like to write JSON text that (a) describes firmware that I use, and (b) conforms to the schema. IOW, I'd like to validate whether the schema is good enough for describing at least such firmware that I know. Is there a tool that generates example JSON objects from a given schema? I vaguely recall there used to be one. Otherwise, writing the examples manually looks arduous (and I wouldn't know how to verify them against the schema). Thanks! Laszlo

On 04/06/18 20:10, Eric Blake wrote: The top level structure is complex / nested, but that doesn't appear to be an issue. According to the script, # key=value pairs also support Python or JSON object literal subset notations, # without spaces. Dictionaries/objects {} are supported as are arrays []. # # example-command arg-name1={'key':'value','obj'={'prop':"value"}} # # Both JSON and Python formatting should work, including both styles of # string literal quotes. Both paradigms of literal values should work, # including null/true/false for JSON and None/True/False for Python. This looks awesome, because it should let me provide messy nested input (which I'll obviously compose in my $EDITOR and then paste it), and then the QMP shell will both validate and pretty print that. I'm going to try this. Thank you, Eric! Laszlo

On 04/09/18 11:02, Kashyap Chamarthy wrote: I noticed it used readline, but I didn't know about the dedicated history file. Nice! Thanks! Laszlo

How to load the binary (using -bios, -pflash, possibly also -kernel, for uboot @ arm). Also machine types. ovmf builds with smm don't boot on pc. coreboot has hardware-specific roms too, so the pc build wouldn't boot on q35 and visa versa. Same on arm, where the firmware typically is board-specific. Possibly a more generic "flags" or "properties" thing, I can easily imagine that simliar requirements show up on other platforms too. Also a "name" and a "description" field would be useful. cheers, Gerd

( Ard, the thread starts here: http://mid.mail-archive.com/20180307144951.d75lo5rgzi2vf27z@eukaryote ) On 03/07/18 15:49, Kashyap Chamarthy wrote: I've read Dan's message <20180307151836.GK20201(a)redhat.com&gt; and Gerd's messages <20180308075245.lgzredyhn2paawg4(a)sirius.home.kraxel.org&gt;, <20180308074507.nwho4tddsoxb3b7v(a)sirius.home.kraxel.org&gt;. Those seem to cover everything, I don't have anything to add wrt. purpose, use case, flexibility etc. I suggest (or agree) that the property list be composed of free-form name=value pairs (at least conceptually). I understand Gerd is proposing a QAPI schema for this, so maybe do { property_name : "foo", property_value : "bar" }, or similar. The registry of properties (names, possible values, meanings) should be kept separate (although possibly still under QEMU). For OVMF (x86), I guess the initial set of properties should come from the "-D FOO[=BAR]" build flags that OVMF currently supports. (The list might grow or change incompatibly over time, so this is just a raw starter idea.) We have: (0) ARCH (one of IA32, IA32X64, X64) -- the bitnesses of the PEI and DXE phases of the firmware. IA32 stands for "32-bit PEI and DXE". Such firmware is usable for booting 32-bit OSes only, and runs on both qemu-system-i386 and qemu-system-x86_64. IA32X64 stands for "32-bit PEI, 64-bit DXE". Needs qemu-system-x86_64 and runs 64-bit OSes only. X64 stands for "64-bit PEI and DXE". Needs qemu-system-x86_64 and runs 64-bit OSes only. (1) SECURE_BOOT_ENABLE (boolean) -- whether the Secure Boot UEFI feature is built into the firmware image. This decides whether the Secure Boot software interfaces will be available to the guest OS. Turning on just this flag does not imply that the guest OS cannot circumvent the SB software interfaces by direct (guest kernel) access to the pflash chip. (2) SMM_REQUIRE (boolean) -- whether the SMM driver stack is included in the firmware. If this flag is enabled, then SMM emulation is required from the board (implying Q35), otherwise the firmware will not boot. If enabled in combination with SECURE_BOOT_ENABLE, *and* the "secure" property of the "cfi.pflash01" driver is set to "on", then the circumvention under (1) is prevented by QEMU, and Secure Boot becomes actually secure. Regarding 32-bit x86 (qemu-system-i386), the compatible CPU models are strongly limited; one model that works is "coreduo", but the NX flag has to be disabled even on that. If ARCH (from under (0)) is X64, and SMM_REQUIRE is enabled, then ACPI S3 suspend/resume has to be disabled ("disable_s3" property of the "ICH9-LPC" driver), otherwise the firmware will not boot. IA32 and IA32X64 are not limited like this. (3) HTTP_BOOT_ENABLE (boolean) -- whether UEFI HTTP boot is available in the firmware image, in addition to the default PXE boot. (4) TLS_ENABLE (boolean) -- configurable independently of HTTP_BOOT_ENABLE, but only really makes sense in combination. Determines whether HTTPS boot is available in the firmware image. (5) NETWORK_IP6_ENABLE (boolean) -- determines whether IPv6 support is available. Orthogonal to all of: PXE, HTTP, HTTPS boot. (6) FD_SIZE_IN_KB (one of: 1024, 2048, 4096) -- the size of the combined firmware image (executable portion and variable store together), in KB. (Boolean shorthands are FD_SIZE_1MB, FD_SIZE_2MB, FD_SIZE_4MB.) Firmware images where this value is 1024 and 2048 are "compatible" with each other in the sense that the variable store files they use are identically structured. The variable store files are 128KB in size, and the actual variable space they offer is 56KB. This is generally sufficient for a low number of UEFI variables, and very basic key enrollment for Secure Boot. A firmware image where the value is 4096 is incompatible. The variable store file is 528KB in size, and the actual variable space it offers is 256KB. This is considered "generous" by recent-ish industry practice, and it is large enough for passing the Microsoft SVVP test cases related to Secure Boot. (Obviously there are other requirements presented by those test cases; I'm just saying that the size requirement is satisfied.) The variable store template files are pre-formatted in all cases (meaning "wire format" only -- on a logical level, they are "empty"). Under some circumstances, it could be desirable to provide varstore template files that are pre-populated with various certificates enrolled. It might make sense to describe such facts with another property. (7) DEBUG_ON_SERIAL_PORT (boolean) -- whether OVMF sends its debug messages to the QEMU debug IO port, or to the serial port. (8) SOURCE_DEBUG_ENABLE (boolean) -- whether OVMF includes the edk2 debug agent that allows it to be debugged from a proprietary debugger program, likely connected via the emulated serial port. (9) CSM_ENABLE (boolean) -- whether OVMF includes traditional BIOS support, by including the SeaBIOS Compatibility Support Module. Including the SeaBIOS CSM makes OVMF capable of booting traditional (not UEFI) OS-es. Such boots are never covered by Secure Boot. Given that the SeaBIOS CSM itself can be built with various configurations, it might make sense to list further properties when this property is enabled. (10) E1000_ENABLE (boolean) -- whether OVMF includes the non-redistributable, binary only E1000(E) UEFI driver module from Intel (previously known as "PROEFI", more recently known as "BootUtil"). This driver can drive QEMU's e1000 and e1000e cards for network booting, which is why the option exists at all. (11) USE_OLD_SHELL (boolean) -- whether the UEFI shell implementation built into OVMF is the old (EDK1 / EFI-1 style) shell which lives in a separate repository, or the new (EDK2 / UEFI-2 style) shell which lives within the edk2 project. (12) TOOLCHAIN (string): the edk2 toolchain identifier with which the firmware was built. (13) TARGET (one of NOOPT, DEBUG, RELEASE) -- the target for which the firmware image was built. NOOPT means "no optimization, DEBUG msgs/ASSERTs enabled". DEBUG means "optimization enabled, DEBUG msgs/ASSERTs enabled". RELEASE means "optimization enabled, DEBUG msgs/ASSERTs disabled". For ArmVirt, we have: (12) ARCH (one of ARM and AARCH64) -- 32-bit / 64-bit distinction. Mixed PEI and DXE bitness is not used, unlike on x86. (13, 14) TOOLCHAIN and TARGET -- see (12) and (13) above (15) SECURE_BOOT_ENABLE (boolean) -- see (1). Note that SMM_REQUIRE (or its architectural match on ARM/AARCH64) is currently not supported in the ArmVirt firmwares, hence the Secure Boot feature should only be included in the firmware for development / testing purposes. (16) HTTP_BOOT_ENABLE (boolean) -- see (3). (17) Well, what do I call this, let's call it ENTRY_POINT (one of Qemu, QemuKernel, Xen). This distinguishes what VMM the ArmVirt firmware was built for, and for QEMU, it also distinguishes "boot from flash" (Qemu) or "boot as payload for another, earlier boot firmware" (QemuKernel). In the above, I tried to be comprehensive; pick whatever you think makes sense to track in the metadata files. Since Dan mentions we should have a "just gimme EFI" shorthand, I can try suggesting specific combinations: - For x86: property_name property_value -------------------- -------------- ARCH IA32X64 SECURE_BOOT_ENABLE TRUE SMM_REQUIRE TRUE HTTP_BOOT_ENABLE TRUE TLS_ENABLE TRUE NETWORK_IP6_ENABLE TRUE FD_SIZE_IN_KB 4096 DEBUG_ON_SERIAL_PORT FALSE SOURCE_DEBUG_ENABLE FALSE CSM_ENABLE FALSE E1000_ENABLE FALSE USE_OLD_SHELL FALSE TOOLCHAIN GCC5 TARGET DEBUG This gives you a firmware image that requires Q35, with SMM emulation enabled. Secure Boot, when configured in the guest, is actually secure. Microsoft SVVP SB testing can be passed (with certificates properly enrolled, in addition), if you need that. ACPI S3 is supported. All kinds of netbooting are enabled (PXE, HTTP, HTTPS, over both IPv4 and IPv6). Debug messages are sent to the QEMU debug port (not messing up your serial port / terminal traffic). No proprietary or proprietary-interfacing components included. A pure UEFI build; if you need to boot a legacy OS, just use standalone SeaBIOS. Built with at least gcc-5, hence LTO (link time optimization) was utilized, allowing for better performance and better firmware space utilization. Debug messages are logged and assertions are enforced. Only 64-bit OSes are supported, qemu-system-x86_64 is required. - For arm: property_name property_value -------------------- -------------- ARCH AARCH64 TOOLCHAIN GCC5 TARGET DEBUG SECURE_BOOT_ENABLE FALSE HTTP_BOOT_ENABLE TRUE ENTRY_POINT QEMU qemu-system-aarch64 is required, only 64-bit OSes are supported. Toolchain/target: see above. Debug messages can only be sent to the PL011 UART (no separate debug port on qemu-system-aarch64 / "virt"). Secure Boot is not included, because at this point it cannot be secured in virtual hardware (no Management Mode equivalent just yet). Allow HTTP booting, and build to be booted on QEMU from pflash. Final note: variable store formats are entirely incompatible between OVMF and ArmVirt. Thanks, Laszlo

On 03/08/18 16:47, Daniel P. Berrangé wrote: This is a reasonable requirement from the libvirt side. Unfortunately (or not), it requires someone (or a tight group of people) to collect the features of all virtual firmwares in existence, and extract a common set of properties that maps back to each firmware one way or another. This is not unusual (basically this is how all standards bodies work that intend to codify existing practice), it just needs a bunch of work and coordination. We'll have to maintain a registry. Personally I can't comment on anything else than OVMF and the ArmVirt firmwares. Thanks, Laszlo

On 03/09/18 12:27, Kashyap Chamarthy wrote: That's for Daniel to decide. I can't sensibly generalize from OVMF & ArmVirt to other firmwares, without knowing them. Thanks Laszlo

Hi, I'd simply put that info the 'arch' bucket. IA32 gets 'i386' (assuming we simply use qemu target names), IA32X64 + X64 get 'x86_64'. Probably useful for upper management (above libvirt), for enabling secure boot key enrollment etc. in the UI. Needed (libvirt must enable smm). Do we actually need formal flags for these? Which application would use them? For humans this can go into the freeform "description" text field. Hmm, dunno. Might be useful as varstores are not portable between builds of different sizes, so libvirt could use this for sanity checks. Same question as for 3+4+5: Do we really need those? I doubt this is useful, established practice is that people just use seabios directly instead if using the ovmf+csm combo. Same question as for 3+4+5+7+8+9: Do we really need those? Likewise just use arch. Hmm, that looks simliar to u-boot, which I think can also be build as both flash image and kernel image (loadable via -kernel). So I guess we want that one, as how-to-load-this-blob option (-bios vs -pflash vs -kernel). cheers, Gerd

On Fri, Mar 09, 2018 at 04:18:45PM +0100, Laszlo Ersek wrote: The kernel config is typically shipped by kernel RPMs in Fedora, but it is purely there as informative text. No application is expected to read the /boot/config-XXXX files to apply functional logic. So with that analogy its fine if you want to ship a record of build config along with the firmware files, I just don't think it is something libvirt would interpret. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|