7:10 a.m.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
docs/drvbhyve.html.in | 583 ------------------------------------------
docs/drvbhyve.rst | 582 +++++++++++++++++++++++++++++++++++++++++
docs/meson.build | 2 +-
3 files changed, 583 insertions(+), 584 deletions(-)
delete mode 100644 docs/drvbhyve.html.in
create mode 100644 docs/drvbhyve.rst
diff --git a/docs/drvbhyve.html.in b/docs/drvbhyve.html.in
deleted file mode 100644
index 228e8b2bd5..0000000000
--- a/docs/drvbhyve.html.in
+++ /dev/null
@@ -1,583 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE html>
-<html xmlns="http://www.w3.org/1999/xhtml">
- <body>
- <h1>Bhyve driver</h1>
-
- <ul id="toc"></ul>
-
-<p>
-Bhyve is a FreeBSD hypervisor. It first appeared in FreeBSD 10.0. However, it's
-recommended to keep tracking FreeBSD 10-STABLE to make sure all new features
-of bhyve are supported.
-
-In order to enable bhyve on your FreeBSD host, you'll need to load the
<code>vmm</code>
-kernel module. Additionally, <code>if_tap</code> and
<code>if_bridge</code> modules
-should be loaded for networking support. Also, <span class="since">since
3.2.0</span> the
-<code>virt-host-validate(1)</code> supports the bhyve host validation and
could be
-used like this:
-</p>
-
-<pre>
-$ virt-host-validate bhyve
- BHYVE: Checking for vmm module : PASS
- BHYVE: Checking for if_tap module : PASS
- BHYVE: Checking for if_bridge module : PASS
- BHYVE: Checking for nmdm module : PASS
-$
-</pre>
-
-<p>
-Additional information on bhyve could be obtained on <a
href="https://bhyve.org/">bhyve.org</a>;.
-</p>
-
-<h2><a id="uri">Connections to the Bhyve
driver</a></h2>
-<p>
-The libvirt bhyve driver is a single-instance privileged driver. Some sample
-connection URIs are:
-</p>
-
-<pre>
-bhyve:///system (local access)
-bhyve+unix:///system (local access)
-bhyve+ssh://root@example.com/system (remote access, SSH tunnelled)
-</pre>
-
-<h2><a id="exconfig">Example guest domain XML
configurations</a></h2>
-
-<h3>Example config</h3>
-<p>
-The bhyve driver in libvirt is in its early stage and under active development. So it
supports
-only limited number of features bhyve provides.
-</p>
-
-<p>
-Note: in older libvirt versions, only a single network device and a single
-disk device were supported per-domain. However,
-<span class="since">since 1.2.6</span> the libvirt bhyve driver
supports
-up to 31 PCI devices.
-</p>
-
-<p>
-Note: the Bhyve driver in libvirt will boot whichever device is first. If you
-want to install from CD, put the CD device first. If not, put the root HDD
-first.
-</p>
-
-<p>
-Note: Only the SATA bus is supported. Only <code>cdrom</code>- and
-<code>disk</code>-type disks are supported.
-</p>
-
-<pre>
-<domain type='bhyve'>
- <name>bhyve</name>
- <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid>
- <memory>219136</memory>
- <currentMemory>219136</currentMemory>
- <vcpu>1</vcpu>
- <os>
- <type>hvm</type>
- </os>
- <features>
- <apic/>
- <acpi/>
- </features>
- <clock offset='utc'/>
- <on_poweroff>destroy</on_poweroff>
- <on_reboot>restart</on_reboot>
- <on_crash>destroy</on_crash>
- <devices>
- <disk type='file'>
- <driver name='file' type='raw'/>
- <source file='/path/to/bhyve_freebsd.img'/>
- <target dev='hda' bus='sata'/>
- </disk>
- <disk type='file' device='cdrom'>
- <driver name='file' type='raw'/>
- <source file='/path/to/cdrom.iso'/>
- <target dev='hdc' bus='sata'/>
- <readonly/>
- </disk>
- <interface type='bridge'>
- <model type='virtio'/>
- <source bridge="virbr0"/>
- </interface>
- </devices>
-</domain>
-</pre>
-
-<p>(The <disk> sections may be swapped in order to install from
-<em>cdrom.iso</em>.)</p>
-
-<h3>Example config (Linux guest)</h3>
-
-<p>
-Note the addition of <bootloader>.
-</p>
-
-<pre>
-<domain type='bhyve'>
- <name>linux_guest</name>
- <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid>
- <memory>131072</memory>
- <currentMemory>131072</currentMemory>
- <vcpu>1</vcpu>
- <bootloader>/usr/local/sbin/grub-bhyve</bootloader>
- <os>
- <type>hvm</type>
- </os>
- <features>
- <apic/>
- <acpi/>
- </features>
- <clock offset='utc'/>
- <on_poweroff>destroy</on_poweroff>
- <on_reboot>restart</on_reboot>
- <on_crash>destroy</on_crash>
- <devices>
- <disk type='file' device='disk'>
- <driver name='file' type='raw'/>
- <source file='/path/to/guest_hdd.img'/>
- <target dev='hda' bus='sata'/>
- </disk>
- <disk type='file' device='cdrom'>
- <driver name='file' type='raw'/>
- <source file='/path/to/cdrom.iso'/>
- <target dev='hdc' bus='sata'/>
- <readonly/>
- </disk>
- <interface type='bridge'>
- <model type='virtio'/>
- <source bridge="virbr0"/>
- </interface>
- </devices>
-</domain>
-</pre>
-
-<h3>Example config (Linux UEFI guest, VNC, tablet)</h3>
-
-<p>This is an example to boot into Fedora 25 installation:</p>
-
-<pre>
-<domain type='bhyve'>
- <name>fedora_uefi_vnc_tablet</name>
- <memory unit='G'>4</memory>
- <vcpu>2</vcpu>
- <os>
- <type>hvm</type>
- <b><loader readonly="yes"
type="pflash">/usr/local/share/uefi-firmware/BHYVE_UEFI.fd</loader></b>
- </os>
- <features>
- <apic/>
- <acpi/>
- </features>
- <clock offset='utc'/>
- <on_poweroff>destroy</on_poweroff>
- <on_reboot>restart</on_reboot>
- <on_crash>destroy</on_crash>
- <devices>
- <disk type='file' device='cdrom'>
- <driver name='file' type='raw'/>
- <source
file='/path/to/Fedora-Workstation-Live-x86_64-25-1.3.iso'/>
- <target dev='hdc' bus='sata'/>
- <readonly/>
- </disk>
- <disk type='file' device='disk'>
- <driver name='file' type='raw'/>
- <source file='/path/to/linux_uefi.img'/>
- <target dev='hda' bus='sata'/>
- </disk>
- <interface type='bridge'>
- <model type='virtio'/>
- <source bridge="virbr0"/>
- </interface>
- <serial type="nmdm">
- <source master="/dev/nmdm0A"
slave="/dev/nmdm0B"/>
- </serial>
- <b><graphics type='vnc' port='5904'>
- <listen type='address' address='127.0.0.1'/>
- </graphics>
- <controller type='usb' model='nec-xhci'/>
- <input type='tablet' bus='usb'/></b>
- </devices>
-</domain>
-</pre>
-
-<p>Please refer to the <a href="#uefi">UEFI</a> section for a
more detailed explanation.</p>
-
-<h2><a id="usage">Guest usage / management</a></h2>
-
-<h3><a id="console">Connecting to a guest
console</a></h3>
-
-<p>
-Guest console connection is supported through the <code>nmdm</code> device.
It could be enabled by adding
-the following to the domain XML (<span class="since">Since
1.2.4</span>):
-</p>
-
-<pre>
-...
-<devices>
- <serial type="nmdm">
- <source master="/dev/nmdm0A" slave="/dev/nmdm0B"/>
- </serial>
-</devices>
-...</pre>
-
-
-<p>Make sure to load the <code>nmdm</code> kernel module if you plan to
use that.</p>
-
-<p>
-Then <code>virsh console</code> command can be used to connect to the text
console
-of a guest.</p>
-
-<p><b>NB:</b> Some versions of bhyve have a bug that prevents guests
from booting
-until the console is opened by a client. This bug was fixed in
-<a href="https://svnweb.freebsd.org/changeset/base/262884">FreeBSD
changeset r262884</a>. If
-an older version is used, one either has to open a console manually with
<code>virsh console</code>
-to let a guest boot or start a guest using:</p>
-
-<pre>start --console domname</pre>
-
-<p><b>NB:</b> A bootloader configured to require user interaction will
prevent
-the domain from starting (and thus <code>virsh console</code> or
<code>start
---console</code> from functioning) until the user interacts with it manually on
-the VM host. Because users typically do not have access to the VM host,
-interactive bootloaders are unsupported by libvirt. <em>However,</em> if you
happen to
-run into this scenario and also happen to have access to the Bhyve host
-machine, you may select a boot option and allow the domain to finish starting
-by using an alternative terminal client on the VM host to connect to the
-domain-configured null modem device. One example (assuming
-<code>/dev/nmdm0B</code> is configured as the slave end of the domain serial
-device) is:</p>
-
-<pre>cu -l /dev/nmdm0B</pre>
-
-<h3><a id="xmltonative">Converting from domain XML to Bhyve
args</a></h3>
-
-<p>
-The <code>virsh domxml-to-native</code> command can preview the actual
-<code>bhyve</code> commands that will be executed for a given domain.
-It outputs two lines, the first line is a <code>bhyveload</code> command and
-the second is a <code>bhyve</code> command.
-</p>
-
-<p>Please note that the <code>virsh domxml-to-native</code> doesn't
do any
-real actions other than printing the command, for example, it doesn't try to
-find a proper TAP interface and create it, like what is done when starting
-a domain; and always returns <code>tap0</code> for the network interface. So
-if you're going to run these commands manually, most likely you might want to
-tweak them.</p>
-
-<pre>
-# virsh -c "bhyve:///system" domxml-to-native --format bhyve-argv --xml
/path/to/bhyve.xml
-/usr/sbin/bhyveload -m 214 -d /home/user/vm1.img vm1
-/usr/sbin/bhyve -c 2 -m 214 -A -I -H -P -s 0:0,hostbridge \
- -s 3:0,virtio-net,tap0,mac=52:54:00:5d:74:e3 -s 2:0,virtio-blk,/home/user/vm1.img \
- -s 1,lpc -l com1,/dev/nmdm0A vm1
-</pre>
-
-<h3><a id="zfsvolume">Using ZFS volumes</a></h3>
-
-<p>It's possible to use ZFS volumes as disk devices <span
class="since">since 1.2.8</span>.
-An example of domain XML device entry for that will look like:</p>
-
-<pre>
-...
-<disk type='volume' device='disk'>
- <source pool='zfspool' volume='vol1'/>
- <target dev='vdb' bus='virtio'/>
-</disk>
-...</pre>
-
-<p>Please refer to the <a href="storage.html">Storage
documentation</a> for more details on storage
-management.</p>
-
-<h3><a id="grubbhyve">Using grub2-bhyve or Alternative
Bootloaders</a></h3>
-
-<p>It's possible to boot non-FreeBSD guests by specifying an explicit
-bootloader, e.g. <code>grub-bhyve(1)</code>. Arguments to the bootloader may
be
-specified as well. If the bootloader is <code>grub-bhyve</code> and
arguments
-are omitted, libvirt will try and infer boot ordering from user-supplied
-<boot order='N'> configuration in the domain. Failing that, it will
boot
-the first disk in the domain (either <code>cdrom</code>- or
-<code>disk</code>-type devices). If the disk type is
<code>disk</code>, it will
-attempt to boot from the first partition in the disk image.</p>
-
-<pre>
-...
-<bootloader>/usr/local/sbin/grub-bhyve</bootloader>
-<bootloader_args>...</bootloader_args>
-...
-</pre>
-
-<p>Caveat: <code>bootloader_args</code> does not support any quoting.
-Filenames, etc, must not have spaces or they will be tokenized incorrectly.</p>
-
-<h3><a id="uefi">Using UEFI bootrom, VNC, and USB
tablet</a></h3>
-
-<p><span class="since">Since 3.2.0</span>, in addition to
<a href="#grubbhyve">grub-bhyve</a>,
-non-FreeBSD guests could be also booted using an UEFI boot ROM, provided both guest OS
and
-installed <code>bhyve(1)</code> version support UEFI. To use that,
<code>loader</code>
-should be specified in the <code>os</code> section:</p>
-
-<pre>
-<domain type='bhyve'>
- ...
- <os>
- <type>hvm</type>
- <loader readonly="yes"
type="pflash">/usr/local/share/uefi-firmware/BHYVE_UEFI.fd</loader>
- </os>
- ...
-</pre>
-
-<p>This uses the UEFI firmware provided by
-the <a
href="https://www.freshports.org/sysutils/bhyve-firmware/">s...
-FreeBSD port.</p>
-
-<p>VNC and the tablet input device could be configured this way:</p>
-
-<pre>
-<domain type='bhyve'>
- <devices>
- ...
- <graphics type='vnc' port='5904'>
- <listen type='address' address='127.0.0.1'/>
- </graphics>
- <controller type='usb' model='nec-xhci'/>
- <input type='tablet' bus='usb'/>
- </devices>
- ...
-</domain>
-</pre>
-
-<p>This way, VNC will be accessible on
<code>127.0.0.1:5904</code>.</p>
-
-<p>Please note that the tablet device requires to have a USB controller
-of the <code>nec-xhci</code> model. Currently, only a single controller of
this
-type and a single tablet are supported per domain.</p>
-
-<p><span class="since">Since 3.5.0</span>, it's possible
to configure how the video device is exposed
-to the guest using the <code>vgaconf</code> attribute:</p>
-
-<pre>
-<domain type='bhyve'>
- <devices>
- ...
- <graphics type='vnc' port='5904'>
- <listen type='address' address='127.0.0.1'/>
- </graphics>
- <video>
- <driver vgaconf='on'/>
- <model type='gop' heads='1'
primary='yes'/>
- </video>
- ...
- </devices>
- ...
-</domain>
-</pre>
-
-<p>If not specified, bhyve's default mode for <code>vgaconf</code>
-will be used. Please refer to the
-<a
href="https://www.freebsd.org/cgi/man.cgi?query=bhyve&sektio...
-manual page and the <a href="https://wiki.freebsd.org/bhyve">bhyve
wiki</a> for more details on using
-the <code>vgaconf</code> option.</p>
-
-<p><span class="since">Since 3.7.0</span>, it's possible
to use <code>autoport</code>
-to let libvirt allocate VNC port automatically (instead of explicitly specifying
-it with the <code>port</code> attribute):</p>
-
-<pre>
- <graphics type='vnc' autoport='yes'>
-</pre>
-
-<p><span class="since">Since 6.8.0</span>, it's possible
to set framebuffer resolution
-using the <code>resolution</code> sub-element:</p>
-
-<pre>
- <video>
- <model type='gop' heads='1' primary='yes'>
- <resolution x='800' y='600'/>
- </model>
- </video>
-</pre>
-
-<p><span class="since">Since 6.8.0</span>, VNC server can be
configured to use
-password based authentication:</p>
-
-<pre>
- <graphics type='vnc' port='5904'
passwd='foobar'>
- <listen type='address' address='127.0.0.1'/>
- </graphics>
-</pre>
-
-<p>Note: VNC password authentication is known to be cryptographically weak.
-Additionally, the password is passed as a command line argument in clear text.
-Make sure you understand the risks associated with this feature before using
it.</p>
-
-<h3><a id="clockconfig">Clock configuration</a></h3>
-
-<p>Originally bhyve supported only localtime for RTC. Support for UTC time was
introduced in
-<a href="https://svnweb.freebsd.org/changeset/base/284894">FreeBSD
changeset r284894</a>
-for <i>10-STABLE</i> and
-in <a href="https://svnweb.freebsd.org/changeset/base/279225">chan...
r279225</a>
-for <i>-CURRENT</i>. It's possible to use this in libvirt <span
class="since">since 1.2.18</span>,
-just place the following to domain XML:</p>
-
-<pre>
-<domain type="bhyve">
- ...
- <clock offset='utc'/>
- ...
-</domain>
-</pre>
-
-<p>Please note that if you run the older bhyve version that doesn't support UTC
time, you'll
-fail to start a domain. As UTC is used as a default when you do not specify clock
settings,
-you'll need to explicitly specify 'localtime' in this case:</p>
-
-<pre>
-<domain type="bhyve">
- ...
- <clock offset='localtime'/>
- ...
-</domain>
-</pre>
-
-<h3><a id="e1000">e1000 NIC</a></h3>
-
-<p>As of <a
href="https://svnweb.freebsd.org/changeset/base/302504">FreeBSD changeset
r302504</a>
-bhyve supports Intel e1000 network adapter emulation. It's supported in libvirt
-<span class="since">since 3.1.0</span> and could be used as
follows:</p>
-
-<pre>
-...
- <interface type='bridge'>
- <source bridge='virbr0'/>
- <model type='<b>e1000</b>'/>
- </interface>
-...
-</pre>
-
-<h3><a id="sound">Sound device</a></h3>
-
-<p>As of <a
href="https://svnweb.freebsd.org/changeset/base/349355">FreeBSD changeset
r349355</a>
-bhyve supports sound device emulation. It's supported in libvirt
-<span class="since">since 6.7.0</span>.</p>
-
-<pre>
-...
- <sound model='ich7'>
- <audio id='1'/>
- </sound>
- <audio id='1' type='oss'>
- <input dev='/dev/dsp0'/>
- <output dev='/dev/dsp0'/>
- </audio>
-...
-</pre>
-
-<p>Here, the <code>sound</code> element specifies the sound device as
it's exposed
-to the guest, with <code>ich7</code> being the only supported model now,
-and the <code>audio</code> element specifies how the guest device is mapped
-to the host sound device.</p>
-
-<h3><a id="fs-9p">Virtio-9p filesystem</a></h3>
-
-<p>As of <a
href="https://svnweb.freebsd.org/changeset/base/366413">FreeBSD changeset
r366413</a>
-bhyve supports sharing arbitrary directory tree between the guest and the host.
-It's supported in libvirt <span class="since">since
6.9.0</span>.</p>
-
-<pre>
-...
- <filesystem>
- <source dir='/shared/dir'/>
- <target dir='shared_dir'/>
- </filesystem>
-...
-</pre>
-
-<p>This share could be made read only by adding the
<code><readonly/></code> sub-element.</p>
-
-<p>In the Linux guest, this could be mounted using:</p>
-
-<pre>mount -t 9p shared_dir /mnt/shared_dir</pre>
-
-<h3><a id="wired">Wiring guest memory</a></h3>
-
-<p><span class="since">Since 4.4.0</span>, it's possible
to specify that guest memory should
-be wired and cannot be swapped out as follows:</p>
-<pre>
-<domain type="bhyve">
- ...
- <memoryBacking>
- <locked/>
- </memoryBacking>
- ...
-</domain>
-</pre>
-
-<h3><a id="cputopology">CPU topology</a></h3>
-
-<p><span class="since">Since 4.5.0</span>, it's possible
to specify guest CPU topology, if bhyve
-supports that. Support for specifying guest CPU topology was added to bhyve in
-<a href="https://svnweb.freebsd.org/changeset/base/332298">FreeBSD
changeset r332298</a>
-for <i>-CURRENT</i>.
-Example:</p>
-<pre>
-<domain type="bhyve">
- ...
- <cpu>
- <topology sockets='1' cores='2'
threads='1'/>
- </cpu>
- ...
-</domain>
-</pre>
-
-<h3><a id="msrs">Ignoring unknown MSRs reads and
writes</a></h3>
-
-<p><span class="since">Since 5.1.0</span>, it's possible
to make bhyve
-ignore accesses to unimplemented Model Specific Registers (MSRs).
-Example:</p>
-
-<pre>
-<domain type="bhyve">
- ...
- <features>
- ...
- <msrs unknown='ignore'/>
- ...
- </features>
- ...
-</domain>
-</pre>
-
-<h3><a id="bhyvecommand">Pass-through of arbitrary bhyve
commands</a></h3>
-
-<p><span class="since">Since 5.1.0</span>, it's possible
to pass additional command-line
-arguments to the bhyve process when starting the domain using the
-<code><bhyve:commandline></code> element under
<code>domain</code>.
-To supply an argument, use the element <code><bhyve:arg></code>
with
-the attribute <code>value</code> set to additional argument to be added.
-The arg element may be repeated multiple times. To use this XML addition, it is
necessary
-to issue an XML namespace request (the special
<code>xmlns:<i>name</i></code> attribute)
-that pulls in <code>http://libvirt.org/schemas/domain/bhyve/1.0</code>;
-typically, the namespace is given the name of <code>bhyve</code>.
-</p>
-<p>Example:</p>
-<pre>
-<domain type="bhyve"
xmlns:bhyve="http://libvirt.org/schemas/domain/bhyve/1.0">
- ...
- <bhyve:commandline>
- <bhyve:arg value='-somebhyvearg'/>
- </bhyve:commandline>
-</domain>
-</pre>
-
-<p>Note that these extensions are for testing and development purposes only.
-They are <b>unsupported</b>, using them may result in inconsistent state,
-and upgrading either bhyve or libvirtd maybe break behavior of a domain that
-was relying on a specific commands pass-through.</p>
-
- </body>
-</html>
diff --git a/docs/drvbhyve.rst b/docs/drvbhyve.rst
new file mode 100644
index 0000000000..95ef4e9b49
--- /dev/null
+++ b/docs/drvbhyve.rst
@@ -0,0 +1,582 @@
+.. role:: since
+
+============
+Bhyve driver
+============
+
+.. contents::
+
+Bhyve is a FreeBSD hypervisor. It first appeared in FreeBSD 10.0. However, it's
+recommended to keep tracking FreeBSD 10-STABLE to make sure all new features of
+bhyve are supported. In order to enable bhyve on your FreeBSD host, you'll need
+to load the ``vmm`` kernel module. Additionally, ``if_tap`` and ``if_bridge``
+modules should be loaded for networking support. Also, :since:`since 3.2.0` the
+``virt-host-validate(1)`` supports the bhyve host validation and could be used
+like this:
+
+::
+
+ $ virt-host-validate bhyve
+ BHYVE: Checking for vmm module : PASS
+ BHYVE: Checking for if_tap module : PASS
+ BHYVE: Checking for if_bridge module : PASS
+ BHYVE: Checking for nmdm module : PASS
+ $
+
+Additional information on bhyve could be obtained on
+`bhyve.org <https://bhyve.org/>`__.
+
+Connections to the Bhyve driver
+-------------------------------
+
+The libvirt bhyve driver is a single-instance privileged driver. Some sample
+connection URIs are:
+
+::
+
+ bhyve:///system (local access)
+ bhyve+unix:///system (local access)
+ bhyve+ssh://root@example.com/system (remote access, SSH tunnelled)
+
+Example guest domain XML configurations
+---------------------------------------
+
+Example config
+~~~~~~~~~~~~~~
+
+The bhyve driver in libvirt is in its early stage and under active development.
+So it supports only limited number of features bhyve provides.
+
+Note: in older libvirt versions, only a single network device and a single disk
+device were supported per-domain. However, :since:`since 1.2.6` the libvirt
+bhyve driver supports up to 31 PCI devices.
+
+Note: the Bhyve driver in libvirt will boot whichever device is first. If you
+want to install from CD, put the CD device first. If not, put the root HDD
+first.
+
+Note: Only the SATA bus is supported. Only ``cdrom``- and ``disk``-type disks
+are supported.
+
+::
+
+ <domain type='bhyve'>
+ <name>bhyve</name>
+ <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid>
+ <memory>219136</memory>
+ <currentMemory>219136</currentMemory>
+ <vcpu>1</vcpu>
+ <os>
+ <type>hvm</type>
+ </os>
+ <features>
+ <apic/>
+ <acpi/>
+ </features>
+ <clock offset='utc'/>
+ <on_poweroff>destroy</on_poweroff>
+ <on_reboot>restart</on_reboot>
+ <on_crash>destroy</on_crash>
+ <devices>
+ <disk type='file'>
+ <driver name='file' type='raw'/>
+ <source file='/path/to/bhyve_freebsd.img'/>
+ <target dev='hda' bus='sata'/>
+ </disk>
+ <disk type='file' device='cdrom'>
+ <driver name='file' type='raw'/>
+ <source file='/path/to/cdrom.iso'/>
+ <target dev='hdc' bus='sata'/>
+ <readonly/>
+ </disk>
+ <interface type='bridge'>
+ <model type='virtio'/>
+ <source bridge="virbr0"/>
+ </interface>
+ </devices>
+ </domain>
+
+(The <disk> sections may be swapped in order to install from *cdrom.iso*.)
+
+Example config (Linux guest)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Note the addition of <bootloader>.
+
+::
+
+ <domain type='bhyve'>
+ <name>linux_guest</name>
+ <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid>
+ <memory>131072</memory>
+ <currentMemory>131072</currentMemory>
+ <vcpu>1</vcpu>
+ <bootloader>/usr/local/sbin/grub-bhyve</bootloader>
+ <os>
+ <type>hvm</type>
+ </os>
+ <features>
+ <apic/>
+ <acpi/>
+ </features>
+ <clock offset='utc'/>
+ <on_poweroff>destroy</on_poweroff>
+ <on_reboot>restart</on_reboot>
+ <on_crash>destroy</on_crash>
+ <devices>
+ <disk type='file' device='disk'>
+ <driver name='file' type='raw'/>
+ <source file='/path/to/guest_hdd.img'/>
+ <target dev='hda' bus='sata'/>
+ </disk>
+ <disk type='file' device='cdrom'>
+ <driver name='file' type='raw'/>
+ <source file='/path/to/cdrom.iso'/>
+ <target dev='hdc' bus='sata'/>
+ <readonly/>
+ </disk>
+ <interface type='bridge'>
+ <model type='virtio'/>
+ <source bridge="virbr0"/>
+ </interface>
+ </devices>
+ </domain>
+
+Example config (Linux UEFI guest, VNC, tablet)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is an example to boot into Fedora 25 installation:
+
+::
+
+ <domain type='bhyve'>
+ <name>fedora_uefi_vnc_tablet</name>
+ <memory unit='G'>4</memory>
+ <vcpu>2</vcpu>
+ <os>
+ <type>hvm</type>
+ <loader readonly="yes"
type="pflash">/usr/local/share/uefi-firmware/BHYVE_UEFI.fd</loader>
+ </os>
+ <features>
+ <apic/>
+ <acpi/>
+ </features>
+ <clock offset='utc'/>
+ <on_poweroff>destroy</on_poweroff>
+ <on_reboot>restart</on_reboot>
+ <on_crash>destroy</on_crash>
+ <devices>
+ <disk type='file' device='cdrom'>
+ <driver name='file' type='raw'/>
+ <source
file='/path/to/Fedora-Workstation-Live-x86_64-25-1.3.iso'/>
+ <target dev='hdc' bus='sata'/>
+ <readonly/>
+ </disk>
+ <disk type='file' device='disk'>
+ <driver name='file' type='raw'/>
+ <source file='/path/to/linux_uefi.img'/>
+ <target dev='hda' bus='sata'/>
+ </disk>
+ <interface type='bridge'>
+ <model type='virtio'/>
+ <source bridge="virbr0"/>
+ </interface>
+ <serial type="nmdm">
+ <source master="/dev/nmdm0A" slave="/dev/nmdm0B"/>
+ </serial>
+ <graphics type='vnc' port='5904'>
+ <listen type='address' address='127.0.0.1'/>
+ </graphics>
+ <controller type='usb' model='nec-xhci'/>
+ <input type='tablet' bus='usb'/>
+ </devices>
+ </domain>
+
+Please refer to the `UEFI <#uefi>`__ section for a more detailed explanation.
+
+Guest usage / management
+------------------------
+
+Connecting to a guest console
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Guest console connection is supported through the ``nmdm`` device. It could be
+enabled by adding the following to the domain XML ( :since:`Since 1.2.4` ):
+
+::
+
+ ...
+ <devices>
+ <serial type="nmdm">
+ <source master="/dev/nmdm0A" slave="/dev/nmdm0B"/>
+ </serial>
+ </devices>
+ ...
+
+Make sure to load the ``nmdm`` kernel module if you plan to use that.
+
+Then ``virsh console`` command can be used to connect to the text console of a
+guest.
+
+**NB:** Some versions of bhyve have a bug that prevents guests from booting
+until the console is opened by a client. This bug was fixed in `FreeBSD
+changeset r262884 <https://svnweb.freebsd.org/changeset/base/262884>`__. If an
+older version is used, one either has to open a console manually with
+``virsh console`` to let a guest boot or start a guest using:
+
+::
+
+ start --console domname
+
+**NB:** A bootloader configured to require user interaction will prevent the
+domain from starting (and thus ``virsh console`` or ``start --console`` from
+functioning) until the user interacts with it manually on the VM host. Because
+users typically do not have access to the VM host, interactive bootloaders are
+unsupported by libvirt. *However,* if you happen to run into this scenario and
+also happen to have access to the Bhyve host machine, you may select a boot
+option and allow the domain to finish starting by using an alternative terminal
+client on the VM host to connect to the domain-configured null modem device. One
+example (assuming ``/dev/nmdm0B`` is configured as the slave end of the domain
+serial device) is:
+
+::
+
+ cu -l /dev/nmdm0B
+
+Converting from domain XML to Bhyve args
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``virsh domxml-to-native`` command can preview the actual ``bhyve`` commands
+that will be executed for a given domain. It outputs two lines, the first line
+is a ``bhyveload`` command and the second is a ``bhyve`` command.
+
+Please note that the ``virsh domxml-to-native`` doesn't do any real actions
+other than printing the command, for example, it doesn't try to find a proper
+TAP interface and create it, like what is done when starting a domain; and
+always returns ``tap0`` for the network interface. So if you're going to run
+these commands manually, most likely you might want to tweak them.
+
+::
+
+ # virsh -c "bhyve:///system" domxml-to-native --format bhyve-argv --xml
/path/to/bhyve.xml
+ /usr/sbin/bhyveload -m 214 -d /home/user/vm1.img vm1
+ /usr/sbin/bhyve -c 2 -m 214 -A -I -H -P -s 0:0,hostbridge \
+ -s 3:0,virtio-net,tap0,mac=52:54:00:5d:74:e3 -s 2:0,virtio-blk,/home/user/vm1.img
\
+ -s 1,lpc -l com1,/dev/nmdm0A vm1
+
+Using ZFS volumes
+~~~~~~~~~~~~~~~~~
+
+It's possible to use ZFS volumes as disk devices :since:`since 1.2.8` . An
+example of domain XML device entry for that will look like:
+
+::
+
+ ...
+ <disk type='volume' device='disk'>
+ <source pool='zfspool' volume='vol1'/>
+ <target dev='vdb' bus='virtio'/>
+ </disk>
+ ...
+
+Please refer to the `Storage documentation <storage.html>`__ for more details on
+storage management.
+
+Using grub2-bhyve or Alternative Bootloaders
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It's possible to boot non-FreeBSD guests by specifying an explicit bootloader,
+e.g. ``grub-bhyve(1)``. Arguments to the bootloader may be specified as well. If
+the bootloader is ``grub-bhyve`` and arguments are omitted, libvirt will try and
+infer boot ordering from user-supplied <boot order='N'> configuration in
the
+domain. Failing that, it will boot the first disk in the domain (either
+``cdrom``- or ``disk``-type devices). If the disk type is ``disk``, it will
+attempt to boot from the first partition in the disk image.
+
+::
+
+ ...
+ <bootloader>/usr/local/sbin/grub-bhyve</bootloader>
+ <bootloader_args>...</bootloader_args>
+ ...
+
+Caveat: ``bootloader_args`` does not support any quoting. Filenames, etc, must
+not have spaces or they will be tokenized incorrectly.
+
+Using UEFI bootrom, VNC, and USB tablet
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:since:`Since 3.2.0` , in addition to `grub-bhyve <#grubbhyve>`__, non-FreeBSD
+guests could be also booted using an UEFI boot ROM, provided both guest OS and
+installed ``bhyve(1)`` version support UEFI. To use that, ``loader`` should be
+specified in the ``os`` section:
+
+::
+
+ <domain type='bhyve'>
+ ...
+ <os>
+ <type>hvm</type>
+ <loader readonly="yes"
type="pflash">/usr/local/share/uefi-firmware/BHYVE_UEFI.fd</loader>
+ </os>
+ ...
+
+This uses the UEFI firmware provided by the
+`sysutils/bhyve-firmware <https://www.freshports.org/sysutils/bhyve-firmware/>`__
+FreeBSD port.
+
+VNC and the tablet input device could be configured this way:
+
+::
+
+ <domain type='bhyve'>
+ <devices>
+ ...
+ <graphics type='vnc' port='5904'>
+ <listen type='address' address='127.0.0.1'/>
+ </graphics>
+ <controller type='usb' model='nec-xhci'/>
+ <input type='tablet' bus='usb'/>
+ </devices>
+ ...
+ </domain>
+
+This way, VNC will be accessible on ``127.0.0.1:5904``.
+
+Please note that the tablet device requires to have a USB controller of the
+``nec-xhci`` model. Currently, only a single controller of this type and a
+single tablet are supported per domain.
+
+:since:`Since 3.5.0` , it's possible to configure how the video device is
+exposed to the guest using the ``vgaconf`` attribute:
+
+::
+
+ <domain type='bhyve'>
+ <devices>
+ ...
+ <graphics type='vnc' port='5904'>
+ <listen type='address' address='127.0.0.1'/>
+ </graphics>
+ <video>
+ <driver vgaconf='on'/>
+ <model type='gop' heads='1' primary='yes'/>
+ </video>
+ ...
+ </devices>
+ ...
+ </domain>
+
+If not specified, bhyve's default mode for ``vgaconf`` will be used. Please
+refer to the
+`bhyve(8)
<https://www.freebsd.org/cgi/man.cgi?query=bhyve&sektion=8&manpath...
+manual page and the `bhyve wiki <https://wiki.freebsd.org/bhyve>`__ for more
+details on using the ``vgaconf`` option.
+
+:since:`Since 3.7.0` , it's possible to use ``autoport`` to let libvirt allocate
+VNC port automatically (instead of explicitly specifying it with the ``port``
+attribute):
+
+::
+
+ <graphics type='vnc' autoport='yes'>
+
+:since:`Since 6.8.0` , it's possible to set framebuffer resolution using the
+``resolution`` sub-element:
+
+::
+
+ <video>
+ <model type='gop' heads='1' primary='yes'>
+ <resolution x='800' y='600'/>
+ </model>
+ </video>
+
+:since:`Since 6.8.0` , VNC server can be configured to use password based
+authentication:
+
+::
+
+ <graphics type='vnc' port='5904' passwd='foobar'>
+ <listen type='address' address='127.0.0.1'/>
+ </graphics>
+
+Note: VNC password authentication is known to be cryptographically weak.
+Additionally, the password is passed as a command line argument in clear text.
+Make sure you understand the risks associated with this feature before using it.
+
+Clock configuration
+~~~~~~~~~~~~~~~~~~~
+
+Originally bhyve supported only localtime for RTC. Support for UTC time was
+introduced in `FreeBSD changeset
+r284894 <https://svnweb.freebsd.org/changeset/base/284894>`__ for *10-STABLE*
+and in `changeset r279225 <https://svnweb.freebsd.org/changeset/base/279225>`__
+for *-CURRENT*. It's possible to use this in libvirt :since:`since 1.2.18` ,
+just place the following to domain XML:
+
+::
+
+ <domain type="bhyve">
+ ...
+ <clock offset='utc'/>
+ ...
+ </domain>
+
+Please note that if you run the older bhyve version that doesn't support UTC
+time, you'll fail to start a domain. As UTC is used as a default when you do not
+specify clock settings, you'll need to explicitly specify 'localtime' in
this
+case:
+
+::
+
+ <domain type="bhyve">
+ ...
+ <clock offset='localtime'/>
+ ...
+ </domain>
+
+e1000 NIC
+~~~~~~~~~
+
+As of `FreeBSD changeset
+r302504 <https://svnweb.freebsd.org/changeset/base/302504>`__ bhyve supports
+Intel e1000 network adapter emulation. It's supported in libvirt :since:`since
+3.1.0` and could be used as follows:
+
+::
+
+ ...
+ <interface type='bridge'>
+ <source bridge='virbr0'/>
+ <model type='e1000'/>
+ </interface>
+ ...
+
+Sound device
+~~~~~~~~~~~~
+
+As of `FreeBSD changeset
+r349355 <https://svnweb.freebsd.org/changeset/base/349355>`__ bhyve supports
+sound device emulation. It's supported in libvirt :since:`since 6.7.0` .
+
+::
+
+ ...
+ <sound model='ich7'>
+ <audio id='1'/>
+ </sound>
+ <audio id='1' type='oss'>
+ <input dev='/dev/dsp0'/>
+ <output dev='/dev/dsp0'/>
+ </audio>
+ ...
+
+Here, the ``sound`` element specifies the sound device as it's exposed to the
+guest, with ``ich7`` being the only supported model now, and the ``audio``
+element specifies how the guest device is mapped to the host sound device.
+
+Virtio-9p filesystem
+~~~~~~~~~~~~~~~~~~~~
+
+As of `FreeBSD changeset
+r366413 <https://svnweb.freebsd.org/changeset/base/366413>`__ bhyve supports
+sharing arbitrary directory tree between the guest and the host. It's supported
+in libvirt :since:`since 6.9.0` .
+
+::
+
+ ...
+ <filesystem>
+ <source dir='/shared/dir'/>
+ <target dir='shared_dir'/>
+ </filesystem>
+ ...
+
+This share could be made read only by adding the ``<readonly/>`` sub-element.
+
+In the Linux guest, this could be mounted using:
+
+::
+
+ mount -t 9p shared_dir /mnt/shared_dir
+
+Wiring guest memory
+~~~~~~~~~~~~~~~~~~~
+
+:since:`Since 4.4.0` , it's possible to specify that guest memory should be
+wired and cannot be swapped out as follows:
+
+::
+
+ <domain type="bhyve">
+ ...
+ <memoryBacking>
+ <locked/>
+ </memoryBacking>
+ ...
+ </domain>
+
+CPU topology
+~~~~~~~~~~~~
+
+:since:`Since 4.5.0` , it's possible to specify guest CPU topology, if bhyve
+supports that. Support for specifying guest CPU topology was added to bhyve in
+`FreeBSD changeset r332298 <https://svnweb.freebsd.org/changeset/base/332298>`__
+for *-CURRENT*. Example:
+
+::
+
+ <domain type="bhyve">
+ ...
+ <cpu>
+ <topology sockets='1' cores='2' threads='1'/>
+ </cpu>
+ ...
+ </domain>
+
+Ignoring unknown MSRs reads and writes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:since:`Since 5.1.0` , it's possible to make bhyve ignore accesses to
+unimplemented Model Specific Registers (MSRs). Example:
+
+::
+
+ <domain type="bhyve">
+ ...
+ <features>
+ ...
+ <msrs unknown='ignore'/>
+ ...
+ </features>
+ ...
+ </domain>
+
+Pass-through of arbitrary bhyve commands
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:since:`Since 5.1.0` , it's possible to pass additional command-line arguments
+to the bhyve process when starting the domain using the ``<bhyve:commandline>``
+element under ``domain``. To supply an argument, use the element ``<bhyve:arg>``
+with the attribute ``value`` set to additional argument to be added. The arg
+element may be repeated multiple times. To use this XML addition, it is
+necessary to issue an XML namespace request (the special ``xmlns:name``
+attribute) that pulls in ``http://libvirt.org/schemas/domain/bhyve/1.0``;
+typically, the namespace is given the name of ``bhyve``.
+
+Example:
+
+::
+
+ <domain type="bhyve"
xmlns:bhyve="http://libvirt.org/schemas/domain/bhyve/1.0">
+ ...
+ <bhyve:commandline>
+ <bhyve:arg value='-somebhyvearg'/>
+ </bhyve:commandline>
+ </domain>
+
+Note that these extensions are for testing and development purposes only. They
+are **unsupported**, using them may result in inconsistent state, and upgrading
+either bhyve or libvirtd maybe break behavior of a domain that was relying on a
+specific commands pass-through.
diff --git a/docs/meson.build b/docs/meson.build
index bb7e27e031..a6c3077f25 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -22,7 +22,6 @@ docs_html_in_files = [
'csharp',
'dbus',
'docs',
- 'drvbhyve',
'drvesx',
'drvhyperv',
'drvlxc',
@@ -79,6 +78,7 @@ docs_rst_files = [
'daemons',
'downloads',
'drivers',
+ 'drvbhyve',
'drvch',
'drvqemu',
'errors',
--
2.35.1