Re: RFC: libvirt-tck bhyve/FreeBSD support
On Mon, Apr 14, 2025 at 02:36:11PM +0200, Roman Bogorodskiy wrote:
Hi,
I'd like to test the bhyve driver using libvirt-tck, which seems to be
very useful both from the continuous integration perspective,
and making the bhyve driver closer to other drivers to improve
integration with other tooling.
As a small proof-of-concept I made just a single test
060-persistent-lifecycle.t work with bhyve, though it was enough to
highlight a bunch of issues.
I've created a pull request on Gitlab:
https://gitlab.com/libvirt/libvirt-tck/-/merge_requests/58
but apparently it's not very active there, so I'll briefly repeat it
here as well.
Issues I encountered:
* Network Filters not supported
It could be solved by implementing the network filters (obviously),
but realistically it's probably not going to happen soon.
I guess there are at least 3 options to solve that:
- Add a test suite config knob to skip nwfilters
- Don't fail the test suite on non-implemented list_nwfilters
Something along the lines of this is what we should do.
Ultimately the needs of the TCK are not that different from the
needs of a regular mgmt app. There needs to be a way for clients
to gracefully degrade when a feature is not implemented. If this
isn't possible today we need to improve libvirt.
- In the bhyve driver, add a minimal implementation which
always returns 0 rules
* Hardcoded /dev/urandom RNG devices
As the bhyve driver now supports virtio-rnd with /dev/random
backend, should there be a configuration knob for libvirt-tck to
override the RNG device definition?
We shouldn't need to set any path at all these days IIRC. THe
built-in default in QEMU should be fine, and ideally bhyve
should hav a built-in default too, if it doesn't already
* Missing IDE device support
Bhyve does not (and likely never will) support IDE devices, but supports
SATA devices. Similar to the previous item, should there be a config knob?
Or maybe just change default to sata?
On the QEMU side i440fx machine uses IDE, while q35 uses SATA.
TCK needs to support both. This is an area where TCK should
use domain capabilities to detect what's supported. Not sure
off hand if we report IDE vs SATA in domain capabilities yet
though.
* Missing pty console support
Bhyve does not support pty consoles. The bhyve driver currently supports
the nmdm console, and bhyve also supports virtio-console and TCP socket
connection which are not yet supported by the driver.
Don't have a particular suggestion for this without looking at the code
in more detail.
* Firmware loader specification
This is probably one of the trickiest ones. When no loader specified,
the bhyve driver uses the bhyveload(8) loader which allows to boot
FreeBSD guests only. That means that any other guest OS requires
specifying the BHYVE_UEFI.fd firmware (or use grub-bhyve). I see
two options for that:
- A test suite/framework configuration knob (again)
This is probably simplest for now.
- Change the bhyve driver to default to BHYVE_UEFI.fd. I like
this option
for two reasons: it makes bhyve domain XMLs more compatible with other
drivers, and also looks like a more reasonable default, because
for example I don't run FreeBSD guests inside bhyve very often, so for
most of my domains I have to set loader. It's a bit inconvenient that
the firmware is not a part of bhyve and needs to be installed through
the port, but we can make it possible to set the default firmware
via the build option and allow to override that via the bhyve driver
configuration, which shoudl provide enough flexibility to users, I guess.
On QEMU side, use of UEFI has always required manual config specification,
though we have simplified the required XML now to just
<os firmware='efi'/>
where upon we auto-fill in the path to the UEFI loader.
As long as bhyve can do the auto-fill of UEFI path from this attribute
I think that's sufficiently simple.
When using bhyveload, is there any firmware at all ? Or is it fully
paravirtualized in some way to avoid any firmware ? When using
<type>hvm</type> conceptually we would expect a firmware to be
present.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|