Daniel P. Berrangé wrote:
On Mon, Apr 14, 2025 at 02:36:11PM +0200, Roman Bogorodskiy wrote:
Hi,
I'd like to test the bhyve driver using libvirt-tck, which seems to be very useful both from the continuous integration perspective, and making the bhyve driver closer to other drivers to improve integration with other tooling.
As a small proof-of-concept I made just a single test 060-persistent-lifecycle.t work with bhyve, though it was enough to highlight a bunch of issues.
I've created a pull request on Gitlab:
https://gitlab.com/libvirt/libvirt-tck/-/merge_requests/58
but apparently it's not very active there, so I'll briefly repeat it here as well.
Issues I encountered:
* Network Filters not supported
It could be solved by implementing the network filters (obviously), but realistically it's probably not going to happen soon. I guess there are at least 3 options to solve that: - Add a test suite config knob to skip nwfilters - Don't fail the test suite on non-implemented list_nwfilters
Something along the lines of this is what we should do.
Ultimately the needs of the TCK are not that different from the needs of a regular mgmt app. There needs to be a way for clients to gracefully degrade when a feature is not implemented. If this isn't possible today we need to improve libvirt.
Sounds good.
- In the bhyve driver, add a minimal implementation which always returns 0 rules
* Hardcoded /dev/urandom RNG devices
As the bhyve driver now supports virtio-rnd with /dev/random backend, should there be a configuration knob for libvirt-tck to override the RNG device definition?
We shouldn't need to set any path at all these days IIRC. THe built-in default in QEMU should be fine, and ideally bhyve should hav a built-in default too, if it doesn't already
Yes, bhyve has a default too. I'll see if I can just remove this hardcoded path from the domain XML definition.
* Missing IDE device support
Bhyve does not (and likely never will) support IDE devices, but supports SATA devices. Similar to the previous item, should there be a config knob? Or maybe just change default to sata?
On the QEMU side i440fx machine uses IDE, while q35 uses SATA.
TCK needs to support both. This is an area where TCK should use domain capabilities to detect what's supported. Not sure off hand if we report IDE vs SATA in domain capabilities yet though.
Yes, it's reported. My qemu installation reports: <disk supported='yes'> <enum name='diskDevice'> <value>disk</value> <value>cdrom</value> <value>floppy</value> <value>lun</value> </enum> <enum name='bus'> <value>ide</value> <value>fdc</value> <value>scsi</value> <value>virtio</value> <value>usb</value> <value>sata</value> </enum> <enum name='model'> <value>virtio</value> <value>virtio-transitional</value> <value>virtio-non-transitional</value> </enum> </disk> And the bhyve installation reports: <disk supported='yes'> <enum name='diskDevice'> <value>disk</value> <value>cdrom</value> </enum> <enum name='bus'> <value>virtio</value> <value>sata</value> </enum> <enum name='model'/> </disk> I think I'll update the domain builder code to use either sata or ide, based on what is available as reported by domain capabilities. At least when the bus is not explicitly requested by the test.
* Missing pty console support
Bhyve does not support pty consoles. The bhyve driver currently supports the nmdm console, and bhyve also supports virtio-console and TCP socket connection which are not yet supported by the driver.
Don't have a particular suggestion for this without looking at the code in more detail.
Maybe it will make sense to report supported console configurations via domain capabilities as well?
* Firmware loader specification
This is probably one of the trickiest ones. When no loader specified, the bhyve driver uses the bhyveload(8) loader which allows to boot FreeBSD guests only. That means that any other guest OS requires specifying the BHYVE_UEFI.fd firmware (or use grub-bhyve). I see two options for that:
- A test suite/framework configuration knob (again)
This is probably simplest for now.
- Change the bhyve driver to default to BHYVE_UEFI.fd. I like this option for two reasons: it makes bhyve domain XMLs more compatible with other drivers, and also looks like a more reasonable default, because for example I don't run FreeBSD guests inside bhyve very often, so for most of my domains I have to set loader. It's a bit inconvenient that the firmware is not a part of bhyve and needs to be installed through the port, but we can make it possible to set the default firmware via the build option and allow to override that via the bhyve driver configuration, which shoudl provide enough flexibility to users, I guess.
On QEMU side, use of UEFI has always required manual config specification, though we have simplified the required XML now to just
<os firmware='efi'/>
where upon we auto-fill in the path to the UEFI loader.
Thanks, that sounds like a generally useful feature, even out of TCK context.
As long as bhyve can do the auto-fill of UEFI path from this attribute I think that's sufficiently simple.
When using bhyveload, is there any firmware at all ? Or is it fully paravirtualized in some way to avoid any firmware ? When using <type>hvm</type> conceptually we would expect a firmware to be present.
In my understanding, it's not paravirtualized, in a way that it doesn't need special guests preparations and can load any reasonable general FreeBSD image. It doesn't need any firmware. It's based on the FreeBSD loader and knows how to load the FreeBSD kernel into memory. Thanks, Roman