Hi, I'd like to introduce Fabian Freyer (CCed), he's taking part in Google Support of Code this year within FreeBSD organization and his project is called "Improving libvirt support for bhyve". Below I'm sharing a tentative plan we currently have. Or, to be more specific, it's a list of ideas for the bhyve driver with things were implementation is clear going first, followed by items were some additional research and working on approach needed. There's no goal to implement everything from this list though. This list was assembled by Fabian with some minor edits from me. Suggestions and ideas are very welcome. --- The primary aim of this project is to implement missing calls and functionality in the libvirt bhyve driver. According to the libvirt API Support Matrix [1], there are a large number of calls not yet implemented. While some missing API calls are not applicable to bhyve, a number of them are, among them the following calls: General calls: - virConnectDomainXMLFromNative This would mostly be an argument parser for a bhyve(8) and bhyvectl(8) command line. - virConnectGetCPUModelNames This needs a research: bhyve is not very flexible in configuration of what CPU model is exposed to the guest, need to figure out if that’s worth implementing now. Connection calls: Most of these include some form of authentication handling and are therefore not applicable. However, the following do apply to bhyve and are easy to implement: - virConnectGetType Trivially return “BHYVE” - virConnectIsAlive Trivially return 1, since “A connection will be classed as alive if it is [...] local” and /dev/vmm is local. - virConnectIsEncrypted Trivially return 0, since bhyve does not support encrypted interfaces. General Domain calls: - virDomainGetMaxMemory Since bhyve does not support memory ballooning, just return the amount of memory allocated here - virDomainGetMaxVcpus, virDomainGetVcpus Would use the approach described in this mailing list thread: “Until the vCPU state is exposed by bhyvectl, or we provide a sysctl, you can use heuristics: the number of vCPU threads for the bhyve process, or scan all vCPUs and only count those that have a non-zero RIP.” [2] - virDomainGetCPUStats - virDomainGetTime - virDomainInjectNMI Call bhyvectl --inject-nmi - virDomainReset Reset a bhyve VM with ‘bhyvectl --force-reset’ and then clean things up using bhyvectl --destroy; update bhyve monitor code to handle exit code 0 from bhyve(8) that’s corresponding to reset (0 - reset / reboot, 1 - shutdown, 2 - halt) to trigger re-starting of the VM. - virDomainReboot Block-Device level calls: These would implement access to the vdev block storage layer. The plan is implement support for both file-backed and zvol-backed virtual machines for the following API calls: - virDomainGetBlockInfo - virDomainBlockPeek - virDomainBlockCopy - virDomainBlockStats - virDomainBlockStatsFlags Going further, since zvols support snapshotting, I plan to implement the following for zvol-backed storage: - virDomainBlockCommit - virDomainBlockPull VirtFS layer I would like to create patches to support specifying filesystems when creating the domain as well as the following calls to be merged at a later time when VirtFS-9p support for bhyve becomes ready: - virDomainFSFreeze - virDomainFSThaw Memory inspection - virDomainMemoryPeek A guest’s memory space is exposed in /dev/vmm, so this call would have to read from that. - virDomainMemoryStats Virt-host-validate(1) bhyve support Add bhyve support to virt-host-validate: - Check support of CPU supports features required for bhyve - Check if the vmm module is available - Check if essential networking things are available (if_bridge(4), if_tap(4)) + nmdm(4) for console Networking support Currently the bhyve’s libvirt driver (as well as libxl/FreeBSD driver) only supports L2 interface bridging. There’s no support for upper level schemas like NAT for example. This is a huge task that involves research of what firefall is more applicable (ipfw or pf), designing of the firewall rules and the actual implemtation. PCI passthrough support I.e. the the "bhyve ... -s 7:0,passthru,4/0/0" thing. Probably that will involve the HAL nodedev driver modifications. --- PS For historic purposes, I've stashed the original proposal: https://people.freebsd.org/~novel/misc/FabianFreyer_GoogleSummerofCode201... 1: https://libvirt.org/hvsupport.html 2: https://lists.freebsd.org/pipermail/freebsd-virtualization/2014-April/002... Roman Bogorodskiy -- libvir-list mailing list libvir-list(a)redhat.com https://www.redhat.com/mailman/listinfo/libvir-list

On 04.05.2016 13:50, Roman Bogorodskiy wrote: > Hi, > > I'd like to introduce Fabian Freyer (CCed), he's taking part in Google > Support of Code this year within FreeBSD organization and his project is > called "Improving libvirt support for bhyve". Hey, that's awesome! I wasn't aware that somebody outside libvirt GSoC org is actually intending to work on libvirt. > > Below I'm sharing a tentative plan we currently have. Or, to be more > specific, it's a list of ideas for the bhyve driver with things were > implementation is clear going first, followed by items were some > additional research and working on approach needed. There's no goal to > implement everything from this list though. > > This list was assembled by Fabian with some minor edits from me. > > Suggestions and ideas are very welcome. > > --- > > The primary aim of this project is to implement missing calls and > functionality in the libvirt bhyve driver. According to the libvirt API > Support Matrix [1], there are a large number of calls not yet implemented. While > some missing API calls are not applicable to bhyve, a number of them are, > among them the following calls: > > General calls: > > - virConnectDomainXMLFromNative > This would mostly be an argument parser for a bhyve(8) and bhyvectl(8) > command line. > - virConnectGetCPUModelNames > This needs a research: bhyve is not very flexible in configuration of > what CPU model is exposed to the guest, need to figure out if that’s > worth implementing now. > > Connection calls: > > Most of these include some form of authentication handling and are > therefore not applicable. However, the following do apply to bhyve and are > easy to implement: > > - virConnectGetType > Trivially return “BHYVE” > - virConnectIsAlive > Trivially return 1, since “A connection will be classed as > alive if it is [...] local” and /dev/vmm is local. > - virConnectIsEncrypted > Trivially return 0, since bhyve does not support encrypted interfaces. > > General Domain calls: > > - virDomainGetMaxMemory > Since bhyve does not support memory ballooning, just > return the amount of memory allocated here > - virDomainGetMaxVcpus, virDomainGetVcpus > Would use the approach described in this mailing list > thread: “Until the vCPU state is exposed by bhyvectl, or we provide a > sysctl, you can use heuristics: the number of vCPU threads for the bhyve > process, or scan all vCPUs and only count those that have a non-zero RIP.” [2] > - virDomainGetCPUStats > - virDomainGetTime > - virDomainInjectNMI > Call bhyvectl --inject-nmi > - virDomainReset > Reset a bhyve VM with ‘bhyvectl --force-reset’ and then clean things up > using bhyvectl --destroy; update bhyve monitor code to handle exit > code 0 from bhyve(8) that’s corresponding to reset (0 - reset > / reboot, 1 - shutdown, 2 - halt) to trigger re-starting of the VM. My knowledge of the bhyve is very limited, but isn't --force-reset just enough? Or it is merely just sending the request to the hypervisor and then we have to lurk for actual request execution?

> - virDomainReboot Michal

  On that note, I have my first question: in the virHypervisorDriver struct for different drivers, fields are commented by a version number. For patches, what version number should be added there? I assume the upcoming release (1.3.5)?