[PATCH 0/6] qemu: Allow control of deprecation behaviour
For debugging purposes it's very useful to disable all deprecated commands and fields in qemu. This series implements a qemu.conf knob and a qemu namespace element to control this. The implementation tries to be very conservative to allow downgrades of qemu and such without breaking the startup of the VM. The intention is that developers and CI deployments use the 'crash' option to catch any unexpected qemu disappearance. Note that this applies on top of my series for -object QAPIfication. Based on Markus' -compat series which was now merged to upstream qemu. Peter Krempa (6): docs/drvqemu: Convert to RST qemu: capabilities: Introduce QEMU_CAPS_COMPAT_DEPRECATED qemu: conf: Add 'deprecation_behavior' setting to qemu.conf qemuxml2xmltest: Enable 'qemu-ns' case qemu: Add per-VM control of deprecation behavior qemu: command: Handle formatting of '-compat' options docs/drvqemu.html.in | 743 ------------------ docs/drvqemu.rst | 636 +++++++++++++++ docs/formatdomain.rst | 2 +- docs/manpages/virt-qemu-run.rst | 2 +- docs/meson.build | 2 +- docs/schemas/domaincommon.rng | 16 + src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf | 31 + src/qemu/qemu_capabilities.c | 8 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 80 ++ src/qemu/qemu_conf.c | 4 + src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.c | 10 +- src/qemu/qemu_domain.h | 5 + src/qemu/test_libvirtd_qemu.aug.in | 1 + .../caps_6.0.0.x86_64.xml | 1 + .../qemu-ns.x86_64-latest.args | 1 + tests/qemuxml2argvdata/qemu-ns.xml | 1 + .../qemu-ns.x86_64-latest.xml | 52 ++ tests/qemuxml2xmltest.c | 1 + 21 files changed, 853 insertions(+), 747 deletions(-) delete mode 100644 docs/drvqemu.html.in create mode 100644 docs/drvqemu.rst create mode 100644 tests/qemuxml2xmloutdata/qemu-ns.x86_64-latest.xml -- 2.29.2
There are two links to this document using anchors so they need to be updated as well. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/drvqemu.html.in | 743 -------------------------------- docs/drvqemu.rst | 588 +++++++++++++++++++++++++ docs/formatdomain.rst | 2 +- docs/manpages/virt-qemu-run.rst | 2 +- docs/meson.build | 2 +- 5 files changed, 591 insertions(+), 746 deletions(-) delete mode 100644 docs/drvqemu.html.in create mode 100644 docs/drvqemu.rst diff --git a/docs/drvqemu.html.in b/docs/drvqemu.html.in deleted file mode 100644 index fdf74979ad..0000000000 --- a/docs/drvqemu.html.in +++ /dev/null @@ -1,743 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE html> -<html xmlns="http://www.w3.org/1999/xhtml"> - <body> - <h1>KVM/QEMU hypervisor driver</h1> - - <ul id="toc"></ul> - - <p> - The libvirt KVM/QEMU driver can manage any QEMU emulator from - version 1.5.0 or later. - </p> - - <h2><a id="project">Project Links</a></h2> - - <ul> - <li> - The <a href="https://www.linux-kvm.org/">KVM</a> Linux - hypervisor - </li> - <li> - The <a href="https://wiki.qemu.org/Index.html">QEMU</a> emulator - </li> - </ul> - - <h2><a id="prereq">Deployment pre-requisites</a></h2> - - <ul> - <li> - <strong>QEMU emulators</strong>: The driver will probe <code>/usr/bin</code> - for the presence of <code>qemu</code>, <code>qemu-system-x86_64</code>, - <code>qemu-system-microblaze</code>, - <code>qemu-system-microblazeel</code>, - <code>qemu-system-mips</code>,<code>qemu-system-mipsel</code>, - <code>qemu-system-sparc</code>,<code>qemu-system-ppc</code>. The results - of this can be seen from the capabilities XML output. - </li> - <li> - <strong>KVM hypervisor</strong>: The driver will probe <code>/usr/bin</code> - for the presence of <code>qemu-kvm</code> and <code>/dev/kvm</code> device - node. If both are found, then KVM fully virtualized, hardware accelerated - guests will be available. - </li> - </ul> - - <h2><a id="uris">Connections to QEMU driver</a></h2> - - <p> - The libvirt QEMU driver is a multi-instance driver, providing a single - system wide privileged driver (the "system" instance), and per-user - unprivileged drivers (the "session" instance). The URI driver protocol - is "qemu". Some example connection URIs for the libvirt driver are: - </p> - -<pre> -qemu:///session (local access to per-user instance) -qemu+unix:///session (local access to per-user instance) - -qemu:///system (local access to system instance) -qemu+unix:///system (local access to system instance) -qemu://example.com/system (remote access, TLS/x509) -qemu+tcp://example.com/system (remote access, SASl/Kerberos) -qemu+ssh://root@example.com/system (remote access, SSH tunnelled) -</pre> - - <h3><a id="uriembedded">Embedded driver</a></h3> - - <p> - Since 6.1.0 the QEMU driver has experimental support for operating - in an embedded mode. In this scenario, rather than connecting to - the libvirtd daemon, the QEMU driver runs in the client application - process directly. To use this the client application must have - registered & be running an instance of the event loop. To open - the driver in embedded mode the app use the new URI path and specify - a virtual root directory under which the driver will create content. - The path to the root directory must be absolute. Passing a relative - path results in an error. - </p> - - <pre> - qemu:///embed?root=/some/dir - </pre> - - <p> - Broadly speaking the range of functionality is intended to be - on a par with that seen when using the traditional system or - session libvirt connections to QEMU. The features will of course - differ depending on whether the application using the embedded - driver is running privileged or unprivileged. For example PCI - device assignment or TAP based networking are only available - when running privileged. While the embedded mode is still classed - as experimental some features may change their default settings - between releases. - </p> - - <p> - By default if the application uses any APIs associated with - secondary drivers, these will result in a connection being - opened to the corresponding driver in libvirtd. For example, - this allows a virtual machine from the embedded QEMU to connect - its NIC to a virtual network or connect its disk to a storage - volume. Some of the secondary drivers will also be able to support - running in embedded mode. Currently this is supported by the - secrets driver, to allow for use of VMs with encrypted disks - </p> - - <h4><a id="embedTree">Directory tree</a></h4> - - <p> - Under the specified root directory the following locations will - be used - </p> - - <pre> -/some/dir - | - +- log - | | - | +- qemu - | +- swtpm - | - +- etc - | | - | +- qemu - | +- pki - | | - | +- qemu - | - +- run - | | - | +- qemu - | +- swtpm - | - +- cache - | | - | +- qemu - | - +- lib - | - +- qemu - +- swtpm - </pre> - - <p> - Note that UNIX domain sockets used for QEMU virtual machines had - a maximum filename length of 108 characters. Bear this in mind - when picking a root directory to avoid risk of exhausting the - filename space. The application is responsible for recursively - purging the contents of this directory tree once they no longer - require a connection, though it can also be left intact for reuse - when opening a future connection. - </p> - - <h4><a id="embedAPI">API usage with event loop</a></h4> - - <p> - To use the QEMU driver in embedded mode the application must - register an event loop with libvirt. Many of the QEMU driver - API calls will rely on the event loop processing data. With this - in mind, applications must <strong>NEVER</strong> invoke API - calls from the event loop thread itself, only other threads. - Not following this rule will lead to deadlocks in the API. - This restriction was lifted starting from 6.2.0 release, when - QMP processing moved to a dedicated thread. However, it is - important to let the event loop run after each API call, even - the ones made from the event loop thread itself. - </p> - - <h2><a id="configFiles">Location of configuration files</a></h2> - - <p> - The QEMU driver comes with sane default values. However, during its - initialization it reads a configuration file which offers system - administrator or an user to override some of that default. The location - of the file depends on the connection URI, as follows: - </p> - - <table> - <tr> - <td><code>qemu:///system</code></td> - <td><code>/etc/libvirt/qemu.conf</code></td> - </tr> - <tr> - <td><code>qemu:///session</code></td> - <td><code>$XDG_CONFIG_HOME/libvirt/qemu.conf</code></td> - </tr> - <tr> - <td><code>qemu:///embed</code></td> - <td><code>$rootdir/etc/qemu.conf</code></td> - </tr> - </table> - - <p> - If <code>$XDG_CONFIG_HOME</code> is not set in the environment, it - defaults to <code>$HOME/.config</code>. For the embed URI the - <code>$rootdir</code> represents the specified root directory from - the connection URI. - </p> - - <p> - Please note, that it is very likely that the only qemu.conf file that - will exist after installing libvirt is the - <code>/etc/libvirt/qemu.conf</code>, if users of the session daemon or - the embed driver want to override a built in value, then they need to - create the file before connecting to the respective URI. - </p> - - <h2><a id="security">Driver security architecture</a></h2> - - <p> - There are multiple layers to security in the QEMU driver, allowing for - flexibility in the use of QEMU based virtual machines. - </p> - - <h3><a id="securitydriver">Driver instances</a></h3> - - <p> - As explained above there are two ways to access the QEMU driver - in libvirt. The "qemu:///session" family of URIs connect to a - libvirtd instance running as the same user/group ID as the client - application. Thus the QEMU instances spawned from this driver will - share the same privileges as the client application. The intended - use case for this driver is desktop virtualization, with virtual - machines storing their disk images in the user's home directory and - being managed from the local desktop login session. - </p> - - <p> - The "qemu:///system" family of URIs connect to a - libvirtd instance running as the privileged system account 'root'. - Thus the QEMU instances spawned from this driver may have much - higher privileges than the client application managing them. - The intended use case for this driver is server virtualization, - where the virtual machines may need to be connected to host - resources (block, PCI, USB, network devices) whose access requires - elevated privileges. - </p> - - <h3><a id="securitydac">POSIX users/groups</a></h3> - - <p> - In the "session" instance, the POSIX users/groups model restricts QEMU - virtual machines (and libvirtd in general) to only have access to resources - with the same user/group ID as the client application. There is no - finer level of configuration possible for the "session" instances. - </p> - - <p> - In the "system" instance, libvirt releases from 0.7.0 onwards allow - control over the user/group that the QEMU virtual machines are run - as. A build of libvirt with no configuration parameters set will - still run QEMU processes as root:root. It is possible to change - this default by using the --with-qemu-user=$USERNAME and - --with-qemu-group=$GROUPNAME arguments to 'configure' during - build. It is strongly recommended that vendors build with both - of these arguments set to 'qemu'. Regardless of this build time - default, administrators can set a per-host default setting in - the <code>/etc/libvirt/qemu.conf</code> configuration file via - the <code>user=$USERNAME</code> and <code>group=$GROUPNAME</code> - parameters. When a non-root user or group is configured, the - libvirt QEMU driver will change uid/gid to match immediately - before executing the QEMU binary for a virtual machine. - </p> - - <p> - If QEMU virtual machines from the "system" instance are being - run as non-root, there will be greater restrictions on what - host resources the QEMU process will be able to access. The - libvirtd daemon will attempt to manage permissions on resources - to minimise the likelihood of unintentional security denials, - but the administrator / application developer must be aware of - some of the consequences / restrictions. - </p> - - <ul> - <li> - <p> - The directories <code>/var/run/libvirt/qemu/</code>, - <code>/var/lib/libvirt/qemu/</code> and - <code>/var/cache/libvirt/qemu/</code> must all have their - ownership set to match the user / group ID that QEMU - guests will be run as. If the vendor has set a non-root - user/group for the QEMU driver at build time, the - permissions should be set automatically at install time. - If a host administrator customizes user/group in - <code>/etc/libvirt/qemu.conf</code>, they will need to - manually set the ownership on these directories. - </p> - </li> - <li> - <p> - When attaching USB and PCI devices to a QEMU guest, - QEMU will need to access files in <code>/dev/bus/usb</code> - and <code>/sys/bus/pci/devices</code> respectively. The libvirtd daemon - will automatically set the ownership on specific devices - that are assigned to a guest at start time. There should - not be any need for administrator changes in this respect. - </p> - </li> - <li> - <p> - Any files/devices used as guest disk images must be - accessible to the user/group ID that QEMU guests are - configured to run as. The libvirtd daemon will automatically - set the ownership of the file/device path to the correct - user/group ID. Applications / administrators must be aware - though that the parent directory permissions may still - deny access. The directories containing disk images - must either have their ownership set to match the user/group - configured for QEMU, or their UNIX file permissions must - have the 'execute/search' bit enabled for 'others'. - </p> - <p> - The simplest option is the latter one, of just enabling - the 'execute/search' bit. For any directory to be used - for storing disk images, this can be achieved by running - the following command on the directory itself, and any - parent directories - </p> -<pre> -chmod o+x /path/to/directory -</pre> - <p> - In particular note that if using the "system" instance - and attempting to store disk images in a user home - directory, the default permissions on $HOME are typically - too restrictive to allow access. - </p> - </li> - </ul> - - <p> - The libvirt maintainers <strong>strongly recommend against</strong> - running QEMU as the root user/group. This should not be required - in most supported usage scenarios, as libvirt will generally do the - right thing to grant QEMU access to files it is permitted to - use when it is running non-root. - </p> - - <h3><a id="securitycap">Linux process capabilities</a></h3> - - <p> - In versions of libvirt prior to 6.0.0, even if QEMU was configured - to run as the root user / group, libvirt would strip all process - capabilities. This meant that QEMU could only read/write files - owned by root, or with open permissions. In reality, stripping - capabilities did not have any security benefit, as it was trivial - to get commands to run in another context with full capabilities, - for example, by creating a cronjob. - </p> - <p> - Thus since 6.0.0, if QEMU is running as root, it will keep all - process capabilities. Behaviour when QEMU is running non-root - is unchanged, it still has no capabilities. - </p> - - <h3><a id="securityselinux">SELinux basic confinement</a></h3> - - <p> - The basic SELinux protection for QEMU virtual machines is intended to - protect the host OS from a compromised virtual machine process. There - is no protection between guests. - </p> - - <p> - In the basic model, all QEMU virtual machines run under the confined - domain <code>root:system_r:qemu_t</code>. It is required that any - disk image assigned to a QEMU virtual machine is labelled with - <code>system_u:object_r:virt_image_t</code>. In a default deployment, - package vendors/distributor will typically ensure that the directory - <code>/var/lib/libvirt/images</code> has this label, such that any - disk images created in this directory will automatically inherit the - correct labelling. If attempting to use disk images in another - location, the user/administrator must ensure the directory has be - given this requisite label. Likewise physical block devices must - be labelled <code>system_u:object_r:virt_image_t</code>. - </p> - <p> - Not all filesystems allow for labelling of individual files. In - particular NFS, VFat and NTFS have no support for labelling. In - these cases administrators must use the 'context' option when - mounting the filesystem to set the default label to - <code>system_u:object_r:virt_image_t</code>. In the case of - NFS, there is an alternative option, of enabling the <code>virt_use_nfs</code> - SELinux boolean. - </p> - - <h3><a id="securitysvirt">SELinux sVirt confinement</a></h3> - - <p> - The SELinux sVirt protection for QEMU virtual machines builds to the - basic level of protection, to also allow individual guests to be - protected from each other. - </p> - - <p> - In the sVirt model, each QEMU virtual machine runs under its own - confined domain, which is based on <code>system_u:system_r:svirt_t:s0</code> - with a unique category appended, eg, <code>system_u:system_r:svirt_t:s0:c34,c44</code>. - The rules are setup such that a domain can only access files which are - labelled with the matching category level, eg - <code>system_u:object_r:svirt_image_t:s0:c34,c44</code>. This prevents one - QEMU process accessing any file resources that are prevent to another QEMU - process. - </p> - - <p> - There are two ways of assigning labels to virtual machines under sVirt. - In the default setup, if sVirt is enabled, guests will get an automatically - assigned unique label each time they are booted. The libvirtd daemon will - also automatically relabel exclusive access disk images to match this - label. Disks that are marked as <shared> will get a generic - label <code>system_u:system_r:svirt_image_t:s0</code> allowing all guests - read/write access them, while disks marked as <readonly> will - get a generic label <code>system_u:system_r:svirt_content_t:s0</code> - which allows all guests read-only access. - </p> - - <p> - With statically assigned labels, the application should include the - desired guest and file labels in the XML at time of creating the - guest with libvirt. In this scenario the application is responsible - for ensuring the disk images & similar resources are suitably - labelled to match, libvirtd will not attempt any relabelling. - </p> - - <p> - If the sVirt security model is active, then the node capabilities - XML will include its details. If a virtual machine is currently - protected by the security model, then the guest XML will include - its assigned labels. If enabled at compile time, the sVirt security - model will always be activated if SELinux is available on the host - OS. To disable sVirt, and revert to the basic level of SELinux - protection (host protection only), the <code>/etc/libvirt/qemu.conf</code> - file can be used to change the setting to <code>security_driver="none"</code> - </p> - - <h3><a id="securitysvirtaa">AppArmor sVirt confinement</a></h3> - - <p> - When using basic AppArmor protection for the libvirtd daemon and - QEMU virtual machines, the intention is to protect the host OS - from a compromised virtual machine process. There is no protection - between guests. - </p> - - <p> - The AppArmor sVirt protection for QEMU virtual machines builds on - this basic level of protection, to also allow individual guests to - be protected from each other. - </p> - - <p> - In the sVirt model, if a profile is loaded for the libvirtd daemon, - then each <code>qemu:///system</code> QEMU virtual machine will have - a profile created for it when the virtual machine is started if one - does not already exist. This generated profile uses a profile name - based on the UUID of the QEMU virtual machine and contains rules - allowing access to only the files it needs to run, such as its disks, - pid file and log files. Just before the QEMU virtual machine is - started, the libvirtd daemon will change into this unique profile, - preventing the QEMU process from accessing any file resources that - are present in another QEMU process or the host machine. - </p> - - <p> - The AppArmor sVirt implementation is flexible in that it allows an - administrator to customize the template file in - <code>/etc/apparmor.d/libvirt/TEMPLATE</code> for site-specific - access for all newly created QEMU virtual machines. Also, when a new - profile is generated, two files are created: - <code>/etc/apparmor.d/libvirt/libvirt-<uuid></code> and - <code>/etc/apparmor.d/libvirt/libvirt-<uuid>.files</code>. The - former can be fine-tuned by the administrator to allow custom access - for this particular QEMU virtual machine, and the latter will be - updated appropriately when required file access changes, such as when - a disk is added. This flexibility allows for situations such as - having one virtual machine in complain mode with all others in - enforce mode. - </p> - - <p> - While users can define their own AppArmor profile scheme, a typical - configuration will include a profile for <code>/usr/sbin/libvirtd</code>, - <code>/usr/lib/libvirt/virt-aa-helper</code> or - <code>/usr/libexec/virt-aa-helper</code>(a helper program which the - libvirtd daemon uses instead of manipulating AppArmor directly), and - an abstraction to be included by <code>/etc/apparmor.d/libvirt/TEMPLATE</code> - (typically <code>/etc/apparmor.d/abstractions/libvirt-qemu</code>). - An example profile scheme can be found in the examples/apparmor - directory of the source distribution. - </p> - - <p> - If the sVirt security model is active, then the node capabilities - XML will include its details. If a virtual machine is currently - protected by the security model, then the guest XML will include - its assigned profile name. If enabled at compile time, the sVirt - security model will be activated if AppArmor is available on the host - OS and a profile for the libvirtd daemon is loaded when libvirtd is - started. To disable sVirt, and revert to the basic level of AppArmor - protection (host protection only), the <code>/etc/libvirt/qemu.conf</code> - file can be used to change the setting to <code>security_driver="none"</code>. - </p> - - - <h3><a id="securityacl">Cgroups device ACLs</a></h3> - - <p> - Linux kernels have a capability known as "cgroups" which is used - for resource management. It is implemented via a number of "controllers", - each controller covering a specific task/functional area. One of the - available controllers is the "devices" controller, which is able to - setup access control lists of block/character devices that a cgroup - should be allowed to access. If the "devices" controller is mounted on a - host, then libvirt will automatically create a dedicated cgroup for each - QEMU virtual machine and setup the device access control list so that the - QEMU process can only access shared devices, and explicitly assigned disks - images backed by block devices. - </p> - - <p> - The list of shared devices a guest is allowed access to is - </p> - -<pre> -/dev/null, /dev/full, /dev/zero, -/dev/random, /dev/urandom, -/dev/ptmx, /dev/kvm, -</pre> - - <p> - In the event of unanticipated needs arising, this can be customized - via the <code>/etc/libvirt/qemu.conf</code> file. - To mount the cgroups device controller, the following command - should be run as root, prior to starting libvirtd - </p> - -<pre> -mkdir /dev/cgroup -mount -t cgroup none /dev/cgroup -o devices -</pre> - - <p> - libvirt will then place each virtual machine in a cgroup at - <code>/dev/cgroup/libvirt/qemu/$VMNAME/</code> - </p> - - <h2><a id="imex">Import and export of libvirt domain XML configs</a></h2> - - <p>The QEMU driver currently supports a single native - config format known as <code>qemu-argv</code>. The data for this format - is expected to be a single line first a list of environment variables, - then the QEMu binary name, finally followed by the QEMU command line - arguments</p> - - <h3><a id="xmlimport">Converting from QEMU args to domain XML</a></h3> - - <p> - <b>Note:</b> this operation is <span class="removed"> deleted as of - 5.5.0</span> and will return an error. - </p> - <p> - The <code>virsh domxml-from-native</code> provides a way to - convert an existing set of QEMU args into a guest description - using libvirt Domain XML that can then be used by libvirt. - Please note that this command is intended to be used to convert - existing qemu guests previously started from the command line to - be managed through libvirt. It should not be used a method of - creating new guests from scratch. New guests should be created - using an application calling the libvirt APIs (see - the <a href="apps.html">libvirt applications page</a> for some - examples) or by manually crafting XML to pass to virsh. - </p> - - <h3><a id="xmlexport">Converting from domain XML to QEMU args</a></h3> - - <p> - The <code>virsh domxml-to-native</code> provides a way to convert a - guest description using libvirt Domain XML, into a set of QEMU args - that can be run manually. Note that currently the command line formatted - by libvirt is no longer suited for manually running qemu as the - configuration expects various resources and open file descriptors passed - to the process which are usually prepared by libvirtd. - </p> - - <h2><a id="qemucommand">Pass-through of arbitrary qemu - commands</a></h2> - - <p>Libvirt provides an XML namespace and an optional - library <code>libvirt-qemu.so</code> for dealing specifically - with qemu. When used correctly, these extensions allow testing - specific qemu features that have not yet been ported to the - generic libvirt XML and API interfaces. However, they - are <b>unsupported</b>, in that the library is not guaranteed to - have a stable API, abusing the library or XML may result in - inconsistent state the crashes libvirtd, and upgrading either - qemu-kvm or libvirtd may break behavior of a domain that was - relying on a qemu-specific pass-through. If you find yourself - needing to use them to access a particular qemu feature, then - please post an RFE to the libvirt mailing list to get that - feature incorporated into the stable libvirt XML and API - interfaces. - </p> - <p>The library provides two - API: <code>virDomainQemuMonitorCommand</code>, for sending an - arbitrary monitor command (in either HMP or QMP format) to a - qemu guest (<span class="since">Since 0.8.3</span>), - and <code>virDomainQemuAttach</code>, for registering a qemu - domain that was manually started so that it can then be managed - by libvirtd (<span class="since">Since 0.9.4</span>, - <span class="removed">removed as of 5.5.0</span>). - </p> - <p>Additionally, the following XML additions allow fine-tuning of - the command line given to qemu when starting a domain - (<span class="since">Since 0.8.3</span>). In order to use the - XML additions, it is necessary to issue an XML namespace request - (the special <code>xmlns:<i>name</i></code> attribute) that - pulls in <code>http://libvirt.org/schemas/domain/qemu/1.0</code>; - typically, the namespace is given the name - of <code>qemu</code>. With the namespace in place, it is then - possible to add an element <code><qemu:commandline></code> - under <code>domain</code>, with the following sub-elements - repeated as often as needed: - </p> - <dl> - <dt><code>qemu:arg</code></dt> - <dd>Add an additional command-line argument to the qemu - process when starting the domain, given by the value of the - attribute <code>value</code>. - </dd> - <dt><code>qemu:env</code></dt> - <dd>Add an additional environment variable to the qemu - process when starting the domain, given with the name-value - pair recorded in the attributes <code>name</code> - and optional <code>value</code>.</dd> - </dl> - <p>Example:</p><pre> -<domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>; - <name>QEMU-fedora-i686</name> - <memory>219200</memory> - <os> - <type arch='i686' machine='pc'>hvm</type> - </os> - <devices> - <emulator>/usr/bin/qemu-system-x86_64</emulator> - </devices> - <qemu:commandline> - <qemu:arg value='-newarg'/> - <qemu:env name='QEMU_ENV' value='VAL'/> - </qemu:commandline> -</domain> -</pre> - - <h2><a id="xmlnsfeatures">QEMU feature configuration for testing</a></h2> - - <p> - In some cases e.g. when developing a new feature or for testing it may - be required to control a given qemu feature (or qemu capability) to test - it before it's complete or disable it for debugging purposes. - <span class="since">Since 5.5.0</span> it's possible to use the same - special qemu namespace as above - (<code>http://libvirt.org/schemas/domain/qemu/1.0</code>) and use - <code><qemu:capabilities></code> element to add - (<code><qemu:add capability="capname"/></code>) or remove - (<code><qemu:del capability="capname"/></code>) capability bits. - The naming of the feature bits is the same libvirt uses in the status - XML. Note that this feature is meant for experiments only and should - _not_ be used in production. - </p> - - <p>Example:</p><pre> -<domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>; - <name>testvm</name> - - [...] - - <qemu:capabilities> - <qemu:add capability='blockdev'/> - <qemu:del capability='drive'/> - </qemu:capabilities> -</domain> -</pre> - - <h2><a id="xmlconfig">Example domain XML config</a></h2> - - <h3>QEMU emulated guest on x86_64</h3> - - <pre><domain type='qemu'> - <name>QEMU-fedora-i686</name> - <uuid>c7a5fdbd-cdaf-9455-926a-d65c16db1809</uuid> - <memory>219200</memory> - <currentMemory>219200</currentMemory> - <vcpu>2</vcpu> - <os> - <type arch='i686' machine='pc'>hvm</type> - <boot dev='cdrom'/> - </os> - <devices> - <emulator>/usr/bin/qemu-system-x86_64</emulator> - <disk type='file' device='cdrom'> - <source file='/home/user/boot.iso'/> - <target dev='hdc'/> - <readonly/> - </disk> - <disk type='file' device='disk'> - <source file='/home/user/fedora.img'/> - <target dev='hda'/> - </disk> - <interface type='network'> - <source network='default'/> - </interface> - <graphics type='vnc' port='-1'/> - </devices> -</domain></pre> - - <h3>KVM hardware accelerated guest on i686</h3> - - <pre><domain type='kvm'> - <name>demo2</name> - <uuid>4dea24b3-1d52-d8f3-2516-782e98a23fa0</uuid> - <memory>131072</memory> - <vcpu>1</vcpu> - <os> - <type arch="i686">hvm</type> - </os> - <clock sync="localtime"/> - <devices> - <emulator>/usr/bin/qemu-kvm</emulator> - <disk type='file' device='disk'> - <source file='/var/lib/libvirt/images/demo2.img'/> - <target dev='hda'/> - </disk> - <interface type='network'> - <source network='default'/> - <mac address='24:42:53:21:52:45'/> - </interface> - <graphics type='vnc' port='-1' keymap='de'/> - </devices> -</domain></pre> - - </body> -</html> diff --git a/docs/drvqemu.rst b/docs/drvqemu.rst new file mode 100644 index 0000000000..0f8fd9e47d --- /dev/null +++ b/docs/drvqemu.rst @@ -0,0 +1,588 @@ +.. role:: since +.. role:: removed + +========================== +KVM/QEMU hypervisor driver +========================== + +The libvirt KVM/QEMU driver can manage any QEMU emulator from version 1.5.0 or +later. + +.. contents:: + +Project Links +------------- + +- The `KVM <https://www.linux-kvm.org/>`__ Linux hypervisor +- The `QEMU <https://wiki.qemu.org/Index.html>`__ emulator + +Deployment pre-requisites +------------------------- + +- **QEMU emulators**: The driver will probe ``/usr/bin`` for the presence of + ``qemu``, ``qemu-system-x86_64``, ``qemu-system-microblaze``, + ``qemu-system-microblazeel``, ``qemu-system-mips``,\ ``qemu-system-mipsel``, + ``qemu-system-sparc``,\ ``qemu-system-ppc``. The results of this can be seen + from the capabilities XML output. +- **KVM hypervisor**: The driver will probe ``/usr/bin`` for the presence of + ``qemu-kvm`` and ``/dev/kvm`` device node. If both are found, then KVM fully + virtualized, hardware accelerated guests will be available. + +Connections to QEMU driver +-------------------------- + +The libvirt QEMU driver is a multi-instance driver, providing a single system +wide privileged driver (the "system" instance), and per-user unprivileged +drivers (the "session" instance). The URI driver protocol is "qemu". Some +example connection URIs for the libvirt driver are: + +:: + + qemu:///session (local access to per-user instance) + qemu+unix:///session (local access to per-user instance) + + qemu:///system (local access to system instance) + qemu+unix:///system (local access to system instance) + qemu://example.com/system (remote access, TLS/x509) + qemu+tcp://example.com/system (remote access, SASl/Kerberos) + qemu+ssh://root@example.com/system (remote access, SSH tunnelled) + +Embedded driver +~~~~~~~~~~~~~~~ + +Since 6.1.0 the QEMU driver has experimental support for operating in an +embedded mode. In this scenario, rather than connecting to the libvirtd daemon, +the QEMU driver runs in the client application process directly. To use this the +client application must have registered & be running an instance of the event +loop. To open the driver in embedded mode the app use the new URI path and +specify a virtual root directory under which the driver will create content. The +path to the root directory must be absolute. Passing a relative path results in +an error. + +:: + + qemu:///embed?root=/some/dir + +Broadly speaking the range of functionality is intended to be on a par with that +seen when using the traditional system or session libvirt connections to QEMU. +The features will of course differ depending on whether the application using +the embedded driver is running privileged or unprivileged. For example PCI +device assignment or TAP based networking are only available when running +privileged. While the embedded mode is still classed as experimental some +features may change their default settings between releases. + +By default if the application uses any APIs associated with secondary drivers, +these will result in a connection being opened to the corresponding driver in +libvirtd. For example, this allows a virtual machine from the embedded QEMU to +connect its NIC to a virtual network or connect its disk to a storage volume. +Some of the secondary drivers will also be able to support running in embedded +mode. Currently this is supported by the secrets driver, to allow for use of VMs +with encrypted disks + +Directory tree +^^^^^^^^^^^^^^ + +Under the specified root directory the following locations will be used + +:: + + /some/dir + | + +- log + | | + | +- qemu + | +- swtpm + | + +- etc + | | + | +- qemu + | +- pki + | | + | +- qemu + | + +- run + | | + | +- qemu + | +- swtpm + | + +- cache + | | + | +- qemu + | + +- lib + | + +- qemu + +- swtpm + +Note that UNIX domain sockets used for QEMU virtual machines had a maximum +filename length of 108 characters. Bear this in mind when picking a root +directory to avoid risk of exhausting the filename space. The application is +responsible for recursively purging the contents of this directory tree once +they no longer require a connection, though it can also be left intact for reuse +when opening a future connection. + +API usage with event loop +^^^^^^^^^^^^^^^^^^^^^^^^^ + +To use the QEMU driver in embedded mode the application must register an event +loop with libvirt. Many of the QEMU driver API calls will rely on the event loop +processing data. With this in mind, applications must **NEVER** invoke API calls +from the event loop thread itself, only other threads. Not following this rule +will lead to deadlocks in the API. This restriction was lifted starting from +6.2.0 release, when QMP processing moved to a dedicated thread. However, it is +important to let the event loop run after each API call, even the ones made from +the event loop thread itself. + +Location of configuration files +------------------------------- + +The QEMU driver comes with sane default values. However, during its +initialization it reads a configuration file which offers system administrator +or an user to override some of that default. The location of the file depends on +the connection URI, as follows: + +=================== ====================================== +``qemu:///system`` ``/etc/libvirt/qemu.conf`` +``qemu:///session`` ``$XDG_CONFIG_HOME/libvirt/qemu.conf`` +``qemu:///embed`` ``$rootdir/etc/qemu.conf`` +=================== ====================================== + +If ``$XDG_CONFIG_HOME`` is not set in the environment, it defaults to +``$HOME/.config``. For the embed URI the ``$rootdir`` represents the specified +root directory from the connection URI. + +Please note, that it is very likely that the only qemu.conf file that will exist +after installing libvirt is the ``/etc/libvirt/qemu.conf``, if users of the +session daemon or the embed driver want to override a built in value, then they +need to create the file before connecting to the respective URI. + +Driver security architecture +---------------------------- + +There are multiple layers to security in the QEMU driver, allowing for +flexibility in the use of QEMU based virtual machines. + +Driver instances +~~~~~~~~~~~~~~~~ + +As explained above there are two ways to access the QEMU driver in libvirt. The +"qemu:///session" family of URIs connect to a libvirtd instance running as the +same user/group ID as the client application. Thus the QEMU instances spawned +from this driver will share the same privileges as the client application. The +intended use case for this driver is desktop virtualization, with virtual +machines storing their disk images in the user's home directory and being +managed from the local desktop login session. + +The "qemu:///system" family of URIs connect to a libvirtd instance running as +the privileged system account 'root'. Thus the QEMU instances spawned from this +driver may have much higher privileges than the client application managing +them. The intended use case for this driver is server virtualization, where the +virtual machines may need to be connected to host resources (block, PCI, USB, +network devices) whose access requires elevated privileges. + +POSIX users/groups +~~~~~~~~~~~~~~~~~~ + +In the "session" instance, the POSIX users/groups model restricts QEMU virtual +machines (and libvirtd in general) to only have access to resources with the +same user/group ID as the client application. There is no finer level of +configuration possible for the "session" instances. + +In the "system" instance, libvirt releases from 0.7.0 onwards allow control over +the user/group that the QEMU virtual machines are run as. A build of libvirt +with no configuration parameters set will still run QEMU processes as root:root. +It is possible to change this default by using the --with-qemu-user=$USERNAME +and --with-qemu-group=$GROUPNAME arguments to 'configure' during build. It is +strongly recommended that vendors build with both of these arguments set to +'qemu'. Regardless of this build time default, administrators can set a per-host +default setting in the ``/etc/libvirt/qemu.conf`` configuration file via the +``user=$USERNAME`` and ``group=$GROUPNAME`` parameters. When a non-root user or +group is configured, the libvirt QEMU driver will change uid/gid to match +immediately before executing the QEMU binary for a virtual machine. + +If QEMU virtual machines from the "system" instance are being run as non-root, +there will be greater restrictions on what host resources the QEMU process will +be able to access. The libvirtd daemon will attempt to manage permissions on +resources to minimise the likelihood of unintentional security denials, but the +administrator / application developer must be aware of some of the consequences +/ restrictions. + +- The directories ``/var/run/libvirt/qemu/``, ``/var/lib/libvirt/qemu/`` and + ``/var/cache/libvirt/qemu/`` must all have their ownership set to match the + user / group ID that QEMU guests will be run as. If the vendor has set a + non-root user/group for the QEMU driver at build time, the permissions should + be set automatically at install time. If a host administrator customizes + user/group in ``/etc/libvirt/qemu.conf``, they will need to manually set the + ownership on these directories. + +- When attaching USB and PCI devices to a QEMU guest, QEMU will need to access + files in ``/dev/bus/usb`` and ``/sys/bus/pci/devices`` respectively. The + libvirtd daemon will automatically set the ownership on specific devices that + are assigned to a guest at start time. There should not be any need for + administrator changes in this respect. + +- Any files/devices used as guest disk images must be accessible to the + user/group ID that QEMU guests are configured to run as. The libvirtd daemon + will automatically set the ownership of the file/device path to the correct + user/group ID. Applications / administrators must be aware though that the + parent directory permissions may still deny access. The directories + containing disk images must either have their ownership set to match the + user/group configured for QEMU, or their UNIX file permissions must have the + 'execute/search' bit enabled for 'others'. + + The simplest option is the latter one, of just enabling the 'execute/search' + bit. For any directory to be used for storing disk images, this can be + achieved by running the following command on the directory itself, and any + parent directories + + :: + + chmod o+x /path/to/directory + + In particular note that if using the "system" instance and attempting to + store disk images in a user home directory, the default permissions on $HOME + are typically too restrictive to allow access. + +The libvirt maintainers **strongly recommend against** running QEMU as the root +user/group. This should not be required in most supported usage scenarios, as +libvirt will generally do the right thing to grant QEMU access to files it is +permitted to use when it is running non-root. + +Linux process capabilities +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In versions of libvirt prior to 6.0.0, even if QEMU was configured to run as the +root user / group, libvirt would strip all process capabilities. This meant that +QEMU could only read/write files owned by root, or with open permissions. In +reality, stripping capabilities did not have any security benefit, as it was +trivial to get commands to run in another context with full capabilities, for +example, by creating a cronjob. + +Thus since 6.0.0, if QEMU is running as root, it will keep all process +capabilities. Behaviour when QEMU is running non-root is unchanged, it still has +no capabilities. + +SELinux basic confinement +~~~~~~~~~~~~~~~~~~~~~~~~~ + +The basic SELinux protection for QEMU virtual machines is intended to protect +the host OS from a compromised virtual machine process. There is no protection +between guests. + +In the basic model, all QEMU virtual machines run under the confined domain +``root:system_r:qemu_t``. It is required that any disk image assigned to a QEMU +virtual machine is labelled with ``system_u:object_r:virt_image_t``. In a +default deployment, package vendors/distributor will typically ensure that the +directory ``/var/lib/libvirt/images`` has this label, such that any disk images +created in this directory will automatically inherit the correct labelling. If +attempting to use disk images in another location, the user/administrator must +ensure the directory has be given this requisite label. Likewise physical block +devices must be labelled ``system_u:object_r:virt_image_t``. + +Not all filesystems allow for labelling of individual files. In particular NFS, +VFat and NTFS have no support for labelling. In these cases administrators must +use the 'context' option when mounting the filesystem to set the default label +to ``system_u:object_r:virt_image_t``. In the case of NFS, there is an +alternative option, of enabling the ``virt_use_nfs`` SELinux boolean. + +SELinux sVirt confinement +~~~~~~~~~~~~~~~~~~~~~~~~~ + +The SELinux sVirt protection for QEMU virtual machines builds to the basic level +of protection, to also allow individual guests to be protected from each other. + +In the sVirt model, each QEMU virtual machine runs under its own confined +domain, which is based on ``system_u:system_r:svirt_t:s0`` with a unique +category appended, eg, ``system_u:system_r:svirt_t:s0:c34,c44``. The rules are +setup such that a domain can only access files which are labelled with the +matching category level, eg ``system_u:object_r:svirt_image_t:s0:c34,c44``. This +prevents one QEMU process accessing any file resources that are prevent to +another QEMU process. + +There are two ways of assigning labels to virtual machines under sVirt. In the +default setup, if sVirt is enabled, guests will get an automatically assigned +unique label each time they are booted. The libvirtd daemon will also +automatically relabel exclusive access disk images to match this label. Disks +that are marked as <shared> will get a generic label +``system_u:system_r:svirt_image_t:s0`` allowing all guests read/write access +them, while disks marked as <readonly> will get a generic label +``system_u:system_r:svirt_content_t:s0`` which allows all guests read-only +access. + +With statically assigned labels, the application should include the desired +guest and file labels in the XML at time of creating the guest with libvirt. In +this scenario the application is responsible for ensuring the disk images & +similar resources are suitably labelled to match, libvirtd will not attempt any +relabelling. + +If the sVirt security model is active, then the node capabilities XML will +include its details. If a virtual machine is currently protected by the security +model, then the guest XML will include its assigned labels. If enabled at +compile time, the sVirt security model will always be activated if SELinux is +available on the host OS. To disable sVirt, and revert to the basic level of +SELinux protection (host protection only), the ``/etc/libvirt/qemu.conf`` file +can be used to change the setting to ``security_driver="none"`` + +AppArmor sVirt confinement +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When using basic AppArmor protection for the libvirtd daemon and QEMU virtual +machines, the intention is to protect the host OS from a compromised virtual +machine process. There is no protection between guests. + +The AppArmor sVirt protection for QEMU virtual machines builds on this basic +level of protection, to also allow individual guests to be protected from each +other. + +In the sVirt model, if a profile is loaded for the libvirtd daemon, then each +``qemu:///system`` QEMU virtual machine will have a profile created for it when +the virtual machine is started if one does not already exist. This generated +profile uses a profile name based on the UUID of the QEMU virtual machine and +contains rules allowing access to only the files it needs to run, such as its +disks, pid file and log files. Just before the QEMU virtual machine is started, +the libvirtd daemon will change into this unique profile, preventing the QEMU +process from accessing any file resources that are present in another QEMU +process or the host machine. + +The AppArmor sVirt implementation is flexible in that it allows an administrator +to customize the template file in ``/etc/apparmor.d/libvirt/TEMPLATE`` for +site-specific access for all newly created QEMU virtual machines. Also, when a +new profile is generated, two files are created: +``/etc/apparmor.d/libvirt/libvirt-<uuid>`` and +``/etc/apparmor.d/libvirt/libvirt-<uuid>.files``. The former can be fine-tuned +by the administrator to allow custom access for this particular QEMU virtual +machine, and the latter will be updated appropriately when required file access +changes, such as when a disk is added. This flexibility allows for situations +such as having one virtual machine in complain mode with all others in enforce +mode. + +While users can define their own AppArmor profile scheme, a typical +configuration will include a profile for ``/usr/sbin/libvirtd``, +``/usr/lib/libvirt/virt-aa-helper`` or ``/usr/libexec/virt-aa-helper``\ (a +helper program which the libvirtd daemon uses instead of manipulating AppArmor +directly), and an abstraction to be included by +``/etc/apparmor.d/libvirt/TEMPLATE`` (typically +``/etc/apparmor.d/abstractions/libvirt-qemu``). An example profile scheme can be +found in the examples/apparmor directory of the source distribution. + +If the sVirt security model is active, then the node capabilities XML will +include its details. If a virtual machine is currently protected by the security +model, then the guest XML will include its assigned profile name. If enabled at +compile time, the sVirt security model will be activated if AppArmor is +available on the host OS and a profile for the libvirtd daemon is loaded when +libvirtd is started. To disable sVirt, and revert to the basic level of AppArmor +protection (host protection only), the ``/etc/libvirt/qemu.conf`` file can be +used to change the setting to ``security_driver="none"``. + +Cgroups device ACLs +~~~~~~~~~~~~~~~~~~~ + +Linux kernels have a capability known as "cgroups" which is used for resource +management. It is implemented via a number of "controllers", each controller +covering a specific task/functional area. One of the available controllers is +the "devices" controller, which is able to setup access control lists of +block/character devices that a cgroup should be allowed to access. If the +"devices" controller is mounted on a host, then libvirt will automatically +create a dedicated cgroup for each QEMU virtual machine and setup the device +access control list so that the QEMU process can only access shared devices, and +explicitly assigned disks images backed by block devices. + +The list of shared devices a guest is allowed access to is + +:: + + /dev/null, /dev/full, /dev/zero, + /dev/random, /dev/urandom, + /dev/ptmx, /dev/kvm, + +In the event of unanticipated needs arising, this can be customized via the +``/etc/libvirt/qemu.conf`` file. To mount the cgroups device controller, the +following command should be run as root, prior to starting libvirtd + +:: + + mkdir /dev/cgroup + mount -t cgroup none /dev/cgroup -o devices + +libvirt will then place each virtual machine in a cgroup at +``/dev/cgroup/libvirt/qemu/$VMNAME/`` + +Import and export of libvirt domain XML configs +----------------------------------------------- + +The QEMU driver currently supports a single native config format known as +``qemu-argv``. The data for this format is expected to be a single line first a +list of environment variables, then the QEMu binary name, finally followed by +the QEMU command line arguments + +Converting from QEMU args to domain XML +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Note:** this operation is :removed:`deleted as of 5.5.0` and will return an +error. + +The ``virsh domxml-from-native`` provides a way to convert an existing set of +QEMU args into a guest description using libvirt Domain XML that can then be +used by libvirt. Please note that this command is intended to be used to convert +existing qemu guests previously started from the command line to be managed +through libvirt. It should not be used a method of creating new guests from +scratch. New guests should be created using an application calling the libvirt +APIs (see the `libvirt applications page <apps.html>`__ for some examples) or by +manually crafting XML to pass to virsh. + +Converting from domain XML to QEMU args +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``virsh domxml-to-native`` provides a way to convert a guest description +using libvirt Domain XML, into a set of QEMU args that can be run manually. Note +that currently the command line formatted by libvirt is no longer suited for +manually running qemu as the configuration expects various resources and open +file descriptors passed to the process which are usually prepared by libvirtd. + +Pass-through of arbitrary qemu commands +--------------------------------------- + +Libvirt provides an XML namespace and an optional library ``libvirt-qemu.so`` +for dealing specifically with qemu. When used correctly, these extensions allow +testing specific qemu features that have not yet been ported to the generic +libvirt XML and API interfaces. However, they are **unsupported**, in that the +library is not guaranteed to have a stable API, abusing the library or XML may +result in inconsistent state the crashes libvirtd, and upgrading either qemu-kvm +or libvirtd may break behavior of a domain that was relying on a qemu-specific +pass-through. If you find yourself needing to use them to access a particular +qemu feature, then please post an RFE to the libvirt mailing list to get that +feature incorporated into the stable libvirt XML and API interfaces. + +The library provides two API: ``virDomainQemuMonitorCommand``, for sending an +arbitrary monitor command (in either HMP or QMP format) to a qemu guest ( +:since:`Since 0.8.3` ), and ``virDomainQemuAttach``, for registering a qemu +domain that was manually started so that it can then be managed by libvirtd ( +:since:`Since 0.9.4` , :removed:`removed as of 5.5.0` ). + +Additionally, the following XML additions allow fine-tuning of the command line +given to qemu when starting a domain ( :since:`Since 0.8.3` ). In order to use +the XML additions, it is necessary to issue an XML namespace request (the +special ``xmlns:name`` attribute) that pulls in +``http://libvirt.org/schemas/domain/qemu/1.0``; typically, the namespace is +given the name of ``qemu``. With the namespace in place, it is then possible to +add an element ``<qemu:commandline>`` under ``domain``, with the following +sub-elements repeated as often as needed: + +``qemu:arg`` + Add an additional command-line argument to the qemu process when starting the + domain, given by the value of the attribute ``value``. +``qemu:env`` + Add an additional environment variable to the qemu process when starting the + domain, given with the name-value pair recorded in the attributes ``name`` + and optional ``value``. + +Example: + +:: + + <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> + <name>QEMU-fedora-i686</name> + <memory>219200</memory> + <os> + <type arch='i686' machine='pc'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-x86_64</emulator> + </devices> + <qemu:commandline> + <qemu:arg value='-newarg'/> + <qemu:env name='QEMU_ENV' value='VAL'/> + </qemu:commandline> + </domain> + +QEMU feature configuration for testing +-------------------------------------- + +In some cases e.g. when developing a new feature or for testing it may be +required to control a given qemu feature (or qemu capability) to test it before +it's complete or disable it for debugging purposes. :since:`Since 5.5.0` it's +possible to use the same special qemu namespace as above +(``http://libvirt.org/schemas/domain/qemu/1.0``) and use ``<qemu:capabilities>`` +element to add (``<qemu:add capability="capname"/>``) or remove +(``<qemu:del capability="capname"/>``) capability bits. The naming of the +feature bits is the same libvirt uses in the status XML. Note that this feature +is meant for experiments only and should _not_ be used in production. + +Example: + +:: + + <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> + <name>testvm</name> + + [...] + + <qemu:capabilities> + <qemu:add capability='blockdev'/> + <qemu:del capability='drive'/> + </qemu:capabilities> + </domain> + +Example domain XML config +------------------------- + +QEMU emulated guest on x86_64 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + <domain type='qemu'> + <name>QEMU-fedora-i686</name> + <uuid>c7a5fdbd-cdaf-9455-926a-d65c16db1809</uuid> + <memory>219200</memory> + <currentMemory>219200</currentMemory> + <vcpu>2</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='cdrom'/> + </os> + <devices> + <emulator>/usr/bin/qemu-system-x86_64</emulator> + <disk type='file' device='cdrom'> + <source file='/home/user/boot.iso'/> + <target dev='hdc'/> + <readonly/> + </disk> + <disk type='file' device='disk'> + <source file='/home/user/fedora.img'/> + <target dev='hda'/> + </disk> + <interface type='network'> + <source network='default'/> + </interface> + <graphics type='vnc' port='-1'/> + </devices> + </domain> + +KVM hardware accelerated guest on i686 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + <domain type='kvm'> + <name>demo2</name> + <uuid>4dea24b3-1d52-d8f3-2516-782e98a23fa0</uuid> + <memory>131072</memory> + <vcpu>1</vcpu> + <os> + <type arch="i686">hvm</type> + </os> + <clock sync="localtime"/> + <devices> + <emulator>/usr/bin/qemu-kvm</emulator> + <disk type='file' device='disk'> + <source file='/var/lib/libvirt/images/demo2.img'/> + <target dev='hda'/> + </disk> + <interface type='network'> + <source network='default'/> + <mac address='24:42:53:21:52:45'/> + </interface> + <graphics type='vnc' port='-1' keymap='de'/> + </devices> + </domain> diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 9392c80113..a7b1a6a0bd 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -8100,4 +8100,4 @@ Example configurations for each driver are provide on the driver specific pages listed below - `Xen examples <drvxen.html#xmlconfig>`__ -- `QEMU/KVM examples <drvqemu.html#xmlconfig>`__ +- `QEMU/KVM examples <drvqemu.html#example-domain-xml-config>`__ diff --git a/docs/manpages/virt-qemu-run.rst b/docs/manpages/virt-qemu-run.rst index b06c311b1d..470de93168 100644 --- a/docs/manpages/virt-qemu-run.rst +++ b/docs/manpages/virt-qemu-run.rst @@ -36,7 +36,7 @@ OS any opportunity to gracefully shutdown. **NOTE: this tool is currently considered experimental.** Its usage and behaviour is still subject to change in future libvirt releases. For further information on its usage consult the -`QEMU driver documentation <https://libvirt.org/drvqemu.html#uriembedded>`_. +`QEMU driver documentation <https://libvirt.org/drvqemu.html#embedded-driver>`_. OPTIONS ======= diff --git a/docs/meson.build b/docs/meson.build index d2e685f673..2b2f7879ed 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -49,7 +49,6 @@ docs_html_in_files = [ 'drvlxc', 'drvnodedev', 'drvopenvz', - 'drvqemu', 'drvremote', 'drvsecret', 'drvtest', @@ -113,6 +112,7 @@ docs_rst_files = [ 'compiling', 'daemons', 'developer-tooling', + 'drvqemu', 'formatbackup', 'formatcheckpoint', 'formatdomain', -- 2.29.2
The capability is asserted if qemu supports the -compat deprecated-input= and deprecated-output= settings to control what should happen if deprecated fields are used in QMP. This will be used for a developer/tester-oriented setting which will aid us in catching use of deprecated settings sooner. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_capabilities.c | 8 ++++++++ src/qemu/qemu_capabilities.h | 1 + tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml | 1 + 3 files changed, 10 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index dc1b10cd66..beea57caf6 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -624,6 +624,7 @@ VIR_ENUM_IMPL(virQEMUCaps, "audiodev", "blockdev-backup", "object.qapified", + "compat-deprecated", ); @@ -5187,6 +5188,13 @@ virQEMUCapsInitProcessCapsInterlock(virQEMUCapsPtr qemuCaps) if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_BLOCKDEV)) virQEMUCapsSet(qemuCaps, QEMU_CAPS_BLOCKDEV_HOSTDEV_SCSI); + + /* The -compat qemu command line argument is implemented using a newer + * method which doesn't show up in query-command-line-options. As we'll use + * it only for development and testing purposes we can base the capability + * on a not entirely related witness. */ + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_QAPIFIED)) + virQEMUCapsSet(qemuCaps, QEMU_CAPS_COMPAT_DEPRECATED); } diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index da51a788fa..a66a48a351 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -604,6 +604,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for syntax-check */ QEMU_CAPS_AUDIODEV, /* -audiodev instead of QEMU_AUDIO_DRV */ QEMU_CAPS_BLOCKDEV_BACKUP, /* qemu supports the blockdev-backup job */ QEMU_CAPS_OBJECT_QAPIFIED, /* parameters for object-add are formally described */ + QEMU_CAPS_COMPAT_DEPRECATED, /* -compat deprecated-(input|output) is supported */ QEMU_CAPS_LAST /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml index 555b6b5317..458ae66719 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml @@ -260,6 +260,7 @@ <flag name='audiodev'/> <flag name='blockdev-backup'/> <flag name='object.qapified'/> + <flag name='compat-deprecated'/> <version>5002050</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100242</microcodeVersion> -- 2.29.2
On Fri, Mar 19, 2021 at 07:33:50PM +0100, Peter Krempa wrote:
The capability is asserted if qemu supports the -compat deprecated-input= and deprecated-output= settings to control what should happen if deprecated fields are used in QMP.
This will be used for a developer/tester-oriented setting which will aid us in catching use of deprecated settings sooner.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_capabilities.c | 8 ++++++++ src/qemu/qemu_capabilities.h | 1 + tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml | 1 + 3 files changed, 10 insertions(+)
diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index dc1b10cd66..beea57caf6 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -624,6 +624,7 @@ VIR_ENUM_IMPL(virQEMUCaps, "audiodev", "blockdev-backup", "object.qapified", + "compat-deprecated", );
@@ -5187,6 +5188,13 @@ virQEMUCapsInitProcessCapsInterlock(virQEMUCapsPtr qemuCaps)
if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_BLOCKDEV)) virQEMUCapsSet(qemuCaps, QEMU_CAPS_BLOCKDEV_HOSTDEV_SCSI); + + /* The -compat qemu command line argument is implemented using a newer + * method which doesn't show up in query-command-line-options. As we'll use + * it only for development and testing purposes we can base the capability + * on a not entirely related witness. */ + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_QAPIFIED)) + virQEMUCapsSet(qemuCaps, QEMU_CAPS_COMPAT_DEPRECATED);
Or it could be just enabled by default since it should be used by devs and CI only.
}
diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index da51a788fa..a66a48a351 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -604,6 +604,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for syntax-check */ QEMU_CAPS_AUDIODEV, /* -audiodev instead of QEMU_AUDIO_DRV */ QEMU_CAPS_BLOCKDEV_BACKUP, /* qemu supports the blockdev-backup job */ QEMU_CAPS_OBJECT_QAPIFIED, /* parameters for object-add are formally described */ + QEMU_CAPS_COMPAT_DEPRECATED, /* -compat deprecated-(input|output) is supported */
QEMU_CAPS_LAST /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml index 555b6b5317..458ae66719 100644 --- a/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml @@ -260,6 +260,7 @@ <flag name='audiodev'/> <flag name='blockdev-backup'/> <flag name='object.qapified'/> + <flag name='compat-deprecated'/> <version>5002050</version> <kvmVersion>0</kvmVersion> <microcodeVersion>43100242</microcodeVersion> -- 2.29.2
On Thu, Apr 01, 2021 at 00:05:55 +0200, Martin Kletzander wrote:
On Fri, Mar 19, 2021 at 07:33:50PM +0100, Peter Krempa wrote:
The capability is asserted if qemu supports the -compat deprecated-input= and deprecated-output= settings to control what should happen if deprecated fields are used in QMP.
This will be used for a developer/tester-oriented setting which will aid us in catching use of deprecated settings sooner.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_capabilities.c | 8 ++++++++ src/qemu/qemu_capabilities.h | 1 + tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml | 1 + 3 files changed, 10 insertions(+)
diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index dc1b10cd66..beea57caf6 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -624,6 +624,7 @@ VIR_ENUM_IMPL(virQEMUCaps, "audiodev", "blockdev-backup", "object.qapified", + "compat-deprecated", );
@@ -5187,6 +5188,13 @@ virQEMUCapsInitProcessCapsInterlock(virQEMUCapsPtr qemuCaps)
if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_BLOCKDEV)) virQEMUCapsSet(qemuCaps, QEMU_CAPS_BLOCKDEV_HOSTDEV_SCSI); + + /* The -compat qemu command line argument is implemented using a newer + * method which doesn't show up in query-command-line-options. As we'll use + * it only for development and testing purposes we can base the capability + * on a not entirely related witness. */ + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_QAPIFIED)) + virQEMUCapsSet(qemuCaps, QEMU_CAPS_COMPAT_DEPRECATED);
Or it could be just enabled by default since it should be used by devs and CI only.
The idea here is that the capability will be used to hide the option from qemus which don't support it, so that you can e.g. downgrade your qemu or switch from git and non-git versions wihout having to touch this option. qemu-5.2.0: ./qemu-system-x86_64 -compat qemu-system-x86_64: -compat: invalid option
On Thu, Apr 01, 2021 at 10:07:15AM +0200, Peter Krempa wrote:
On Thu, Apr 01, 2021 at 00:05:55 +0200, Martin Kletzander wrote:
On Fri, Mar 19, 2021 at 07:33:50PM +0100, Peter Krempa wrote:
The capability is asserted if qemu supports the -compat deprecated-input= and deprecated-output= settings to control what should happen if deprecated fields are used in QMP.
This will be used for a developer/tester-oriented setting which will aid us in catching use of deprecated settings sooner.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/qemu_capabilities.c | 8 ++++++++ src/qemu/qemu_capabilities.h | 1 + tests/qemucapabilitiesdata/caps_6.0.0.x86_64.xml | 1 + 3 files changed, 10 insertions(+)
diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index dc1b10cd66..beea57caf6 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -624,6 +624,7 @@ VIR_ENUM_IMPL(virQEMUCaps, "audiodev", "blockdev-backup", "object.qapified", + "compat-deprecated", );
@@ -5187,6 +5188,13 @@ virQEMUCapsInitProcessCapsInterlock(virQEMUCapsPtr qemuCaps)
if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_BLOCKDEV)) virQEMUCapsSet(qemuCaps, QEMU_CAPS_BLOCKDEV_HOSTDEV_SCSI); + + /* The -compat qemu command line argument is implemented using a newer + * method which doesn't show up in query-command-line-options. As we'll use + * it only for development and testing purposes we can base the capability + * on a not entirely related witness. */ + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_QAPIFIED)) + virQEMUCapsSet(qemuCaps, QEMU_CAPS_COMPAT_DEPRECATED);
Or it could be just enabled by default since it should be used by devs and CI only.
The idea here is that the capability will be used to hide the option from qemus which don't support it, so that you can e.g. downgrade your qemu or switch from git and non-git versions wihout having to touch this option.
qemu-5.2.0:
./qemu-system-x86_64 -compat qemu-system-x86_64: -compat: invalid option
Oh, good point, I haven't thought of the option itself.
New QEMU supports an harsh, but hard to ignore way to notify that the QMP user used an deprecated command. This is useful e.g. for developers to see that something needs to be fixed. This patch introduces a qemu.conf option to enable the setting in cases when qemu supports it so that developers and continiuous integration efforts are notified about use of deprecated fields while it's not late. The option is deliberately stored as string and not validated to prevent failures when downgrading qemu or libvirt versions. While we don't support this, the knob isn't meant for public consumption anyways. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf | 31 ++++++++++++++++++++++++++++++ src/qemu/qemu_conf.c | 4 ++++ src/qemu/qemu_conf.h | 2 ++ src/qemu/test_libvirtd_qemu.aug.in | 1 + 5 files changed, 39 insertions(+) diff --git a/src/qemu/libvirtd_qemu.aug b/src/qemu/libvirtd_qemu.aug index 3c1045858b..0f18775121 100644 --- a/src/qemu/libvirtd_qemu.aug +++ b/src/qemu/libvirtd_qemu.aug @@ -131,6 +131,7 @@ module Libvirtd_qemu = let debug_level_entry = int_entry "gluster_debug_level" | bool_entry "virtiofsd_debug" + | str_entry "deprecation_behavior" let memory_entry = str_entry "memory_backing_dir" diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index 0c1054f198..086d7d2296 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -922,3 +922,34 @@ # may change across versions. # #capability_filters = [ "capname" ] + +# 'deprecation_behavior' setting controls how the qemu process behaves towards +# deprecated commands and arguments used by libvirt. +# +# This setting is meant for developers and CI efforts to make it obvious when +# libvirt relies on fields which are deprecated so that it can be fixes as soon +# as possible. +# +# Possible options are: +# "none" - (default) qemu is supposed to accept and output deprecated fields +# and commands +# "omit" - qemu is instructed to omit deprecated fields on output, behaviour +# towards fields and commadns from qemu is not changed +# "reject" - qemu is instructed to report an error if a deprecated command or +# field is used by libvirtd +# "crash" - qemu crashes when an deprecated command or field is used by libvirtd +# +# For both "reject" and "crash" qemu is instructed to omit any deprecated fields +# on output. +# +# The "reject" option is less harsh towards the VMs but some code paths ignore +# errors reported by qemu and thus it may not be obvious that a deprecated +# command/field was used, thus it's suggested to use the "crash" option instead. +# +# In cases when qemu doesn't support configuring the behaviour this setting is +# silently ignored to allow testing older qemu versions without having to +# reconfigure libvirtd. +# +# DO NOT use in production. +# +#deprecation_behavior = "none" diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 2bbc75024c..4f7c85cda1 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -380,6 +380,8 @@ static void virQEMUDriverConfigDispose(void *obj) g_free(cfg->swtpmStorageDir); g_strfreev(cfg->capabilityfilters); + + g_free(cfg->deprecationBehavior); } @@ -869,6 +871,8 @@ virQEMUDriverConfigLoadDebugEntry(virQEMUDriverConfigPtr cfg, return -1; if (virConfGetValueBool(conf, "virtiofsd_debug", &cfg->virtiofsdDebug) < 0) return -1; + if (virConfGetValueString(conf, "deprecation_behavior", &cfg->deprecationBehavior) < 0) + return -1; return 0; } diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 7025b5222e..e62cd88950 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -223,6 +223,8 @@ struct _virQEMUDriverConfig { gid_t swtpm_group; char **capabilityfilters; + + char *deprecationBehavior; }; G_DEFINE_AUTOPTR_CLEANUP_FUNC(virQEMUDriverConfig, virObjectUnref); diff --git a/src/qemu/test_libvirtd_qemu.aug.in b/src/qemu/test_libvirtd_qemu.aug.in index 9310dcec1c..20a89ade32 100644 --- a/src/qemu/test_libvirtd_qemu.aug.in +++ b/src/qemu/test_libvirtd_qemu.aug.in @@ -115,3 +115,4 @@ module Test_libvirtd_qemu = { "capability_filters" { "1" = "capname" } } +{ "deprecation_behavior" = "none" } -- 2.29.2
On Fri, Mar 19, 2021 at 07:33:51PM +0100, Peter Krempa wrote:
New QEMU supports an harsh, but hard to ignore way to notify that the
"a harsh" or just "harsh"
QMP user used an deprecated command. This is useful e.g. for developers
"a deprecated"
to see that something needs to be fixed.
This patch introduces a qemu.conf option to enable the setting in cases when qemu supports it so that developers and continiuous integration efforts are notified about use of deprecated fields while it's not late.
"before it's too late" sounds better
The option is deliberately stored as string and not validated to prevent failures when downgrading qemu or libvirt versions. While we don't support this, the knob isn't meant for public consumption anyways.
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf | 31 ++++++++++++++++++++++++++++++ src/qemu/qemu_conf.c | 4 ++++ src/qemu/qemu_conf.h | 2 ++ src/qemu/test_libvirtd_qemu.aug.in | 1 + 5 files changed, 39 insertions(+)
diff --git a/src/qemu/libvirtd_qemu.aug b/src/qemu/libvirtd_qemu.aug index 3c1045858b..0f18775121 100644 --- a/src/qemu/libvirtd_qemu.aug +++ b/src/qemu/libvirtd_qemu.aug @@ -131,6 +131,7 @@ module Libvirtd_qemu =
let debug_level_entry = int_entry "gluster_debug_level" | bool_entry "virtiofsd_debug" + | str_entry "deprecation_behavior"
let memory_entry = str_entry "memory_backing_dir"
diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index 0c1054f198..086d7d2296 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -922,3 +922,34 @@ # may change across versions. # #capability_filters = [ "capname" ] + +# 'deprecation_behavior' setting controls how the qemu process behaves towards +# deprecated commands and arguments used by libvirt. +# +# This setting is meant for developers and CI efforts to make it obvious when +# libvirt relies on fields which are deprecated so that it can be fixes as soon +# as possible. +# +# Possible options are: +# "none" - (default) qemu is supposed to accept and output deprecated fields +# and commands +# "omit" - qemu is instructed to omit deprecated fields on output, behaviour +# towards fields and commadns from qemu is not changed
s/commadns/commands/
The XML formatter validation was missing for this code path. Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- .../qemu-ns.x86_64-latest.xml | 51 +++++++++++++++++++ tests/qemuxml2xmltest.c | 1 + 2 files changed, 52 insertions(+) create mode 100644 tests/qemuxml2xmloutdata/qemu-ns.x86_64-latest.xml diff --git a/tests/qemuxml2xmloutdata/qemu-ns.x86_64-latest.xml b/tests/qemuxml2xmloutdata/qemu-ns.x86_64-latest.xml new file mode 100644 index 0000000000..53e21edfaf --- /dev/null +++ b/tests/qemuxml2xmloutdata/qemu-ns.x86_64-latest.xml @@ -0,0 +1,51 @@ +<domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <cpu mode='custom' match='exact' check='none'> + <model fallback='forbid'>qemu64</model> + </cpu> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <disk type='block' device='disk'> + <driver name='qemu' type='raw'/> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='ide'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='ide' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> + </controller> + <controller type='usb' index='0' model='piix3-uhci'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> + </controller> + <controller type='pci' index='0' model='pci-root'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <audio id='1' type='none'/> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </memballoon> + </devices> + <qemu:commandline> + <qemu:arg value='-unknown'/> + <qemu:arg value='parameter'/> + <qemu:env name='NS' value='ns'/> + <qemu:env name='BAR'/> + </qemu:commandline> + <qemu:capabilities> + <qemu:add capability='vnc-colon'/> + <qemu:add capability='blockdev'/> + <qemu:del capability='name'/> + </qemu:capabilities> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 4e7cce21c6..d4beb12e22 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -277,6 +277,7 @@ mymain(void) DO_TEST("restore-v2", NONE); DO_TEST("migrate", NONE); DO_TEST("qemu-ns-no-env", NONE); + DO_TEST_CAPS_LATEST("qemu-ns"); DO_TEST("disk-aio", NONE); DO_TEST_CAPS_LATEST("disk-aio-io_uring"); DO_TEST("disk-cdrom", NONE); -- 2.29.2
Similar to the qemu.conf knob 'deprecation_behavior' add a per-VM knob in the QEMU namespace: <qemu:deprecation behavior='...'/> Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/drvqemu.rst | 48 +++++++++++++++++++ docs/schemas/domaincommon.rng | 16 +++++++ src/qemu/qemu_domain.c | 10 +++- src/qemu/qemu_domain.h | 5 ++ tests/qemuxml2argvdata/qemu-ns.xml | 1 + .../qemu-ns.x86_64-latest.xml | 1 + 6 files changed, 80 insertions(+), 1 deletion(-) diff --git a/docs/drvqemu.rst b/docs/drvqemu.rst index 0f8fd9e47d..41c9c5d8e1 100644 --- a/docs/drvqemu.rst +++ b/docs/drvqemu.rst @@ -523,6 +523,54 @@ Example: </qemu:capabilities> </domain> +Control of QEMU deprecation warnings +------------------------------------ + +The following knob controls how QEMU behaves towards deprecated commands and +arguments used by libvirt: + +:: + + <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> + <name>testvm</name> + + [...] + + <qemu:deprecation behavior='crash'/> + +This setting is meant for developers and CI efforts to make it obvious when +libvirt relies on fields which are deprecated so that it can be fixes as soon +as possible. + +Possible options are: + +``none`` + (default) qemu is supposed to accept and output deprecated fields and commands + +``omit`` + qemu is instructed to omit deprecated fields on output, behaviour towards + fields and commadns from libvirtd is not changed + +``reject`` + qemu is instructed to report an error if a deprecated command or field is + used by libvirtd + +``crash`` + qemu crashes when an deprecated command or field is used by libvirtd + +For both "reject" and "crash" qemu is instructed to omit any deprecated fields +on output. + +The "reject" option is less harsh towards the VMs but some code paths ignore +errors reported by qemu and thus it may not be obvious that a deprecated +command/field was used, thus it's suggested to use the "crash" option instead. + +In cases when qemu doesn't support configuring the behaviour this setting is +silently ignored to allow testing older qemu versions without having to +reconfigure libvirtd. + +*DO NOT* use in production. + Example domain XML config ------------------------- diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 1dbfc68f18..ffd992852a 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -77,6 +77,9 @@ <optional> <ref name="qemucapabilities"/> </optional> + <optional> + <ref name="qemudeprecation"/> + </optional> <optional> <ref name="lxcsharens"/> </optional> @@ -7271,6 +7274,19 @@ </element> </define> + <define name="qemudeprecation"> + <element name="deprecation" ns="http://libvirt.org/schemas/domain/qemu/1.0"> + <attribute name="behavior"> + <choice> + <value>none</value> + <value>omit</value> + <value>reject</value> + <value>crash</value> + </choice> + </attribute> + </element> + </define> + <!-- Optional hypervisor extensions in their own namespace: diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 5c98f8ff1a..ec3cb84518 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -3316,6 +3316,8 @@ qemuDomainXmlNsDefFree(qemuDomainXmlNsDefPtr def) virStringListFreeCount(def->capsadd, def->ncapsadd); virStringListFreeCount(def->capsdel, def->ncapsdel); + g_free(def->deprecationBehavior); + g_free(def); } @@ -3468,8 +3470,11 @@ qemuDomainDefNamespaceParse(xmlXPathContextPtr ctxt, qemuDomainDefNamespaceParseCaps(nsdata, ctxt) < 0) goto cleanup; + nsdata->deprecationBehavior = virXPathString("string(./qemu:deprecation/@behavior)", ctxt); + if (nsdata->num_args > 0 || nsdata->num_env > 0 || - nsdata->ncapsadd > 0 || nsdata->ncapsdel > 0) + nsdata->ncapsadd > 0 || nsdata->ncapsdel > 0 || + nsdata->deprecationBehavior) *data = g_steal_pointer(&nsdata); ret = 0; @@ -3539,6 +3544,9 @@ qemuDomainDefNamespaceFormatXML(virBufferPtr buf, qemuDomainDefNamespaceFormatXMLCommandline(buf, cmd); qemuDomainDefNamespaceFormatXMLCaps(buf, cmd); + virBufferEscapeString(buf, "<qemu:deprecation behavior='%s'/>\n", + cmd->deprecationBehavior); + return 0; } diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 949307229b..1c80b8cfaa 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -485,6 +485,11 @@ struct _qemuDomainXmlNsDef { size_t ncapsdel; char **capsdel; + + /* We deliberatly keep this as a string so that it's parsed only when + * starting the VM to avoid any form of errors in the parser or when + * changing qemu versions. The knob is mainly for development/CI purposes */ + char *deprecationBehavior; }; diff --git a/tests/qemuxml2argvdata/qemu-ns.xml b/tests/qemuxml2argvdata/qemu-ns.xml index 515d4fabec..3e7590237e 100644 --- a/tests/qemuxml2argvdata/qemu-ns.xml +++ b/tests/qemuxml2argvdata/qemu-ns.xml @@ -32,4 +32,5 @@ <qemu:add capability="blockdev"/> <qemu:del capability="name"/> </qemu:capabilities> + <qemu:deprecation behavior='crash'/> </domain> diff --git a/tests/qemuxml2xmloutdata/qemu-ns.x86_64-latest.xml b/tests/qemuxml2xmloutdata/qemu-ns.x86_64-latest.xml index 53e21edfaf..e655e87cbf 100644 --- a/tests/qemuxml2xmloutdata/qemu-ns.x86_64-latest.xml +++ b/tests/qemuxml2xmloutdata/qemu-ns.x86_64-latest.xml @@ -48,4 +48,5 @@ <qemu:add capability='blockdev'/> <qemu:del capability='name'/> </qemu:capabilities> + <qemu:deprecation behavior='crash'/> </domain> -- 2.29.2
On Fri, Mar 19, 2021 at 07:33:53PM +0100, Peter Krempa wrote:
Similar to the qemu.conf knob 'deprecation_behavior' add a per-VM knob in the QEMU namespace:
<qemu:deprecation behavior='...'/>
Signed-off-by: Peter Krempa <pkrempa@redhat.com> --- docs/drvqemu.rst | 48 +++++++++++++++++++ docs/schemas/domaincommon.rng | 16 +++++++ src/qemu/qemu_domain.c | 10 +++- src/qemu/qemu_domain.h | 5 ++ tests/qemuxml2argvdata/qemu-ns.xml | 1 + .../qemu-ns.x86_64-latest.xml | 1 + 6 files changed, 80 insertions(+), 1 deletion(-)
diff --git a/docs/drvqemu.rst b/docs/drvqemu.rst index 0f8fd9e47d..41c9c5d8e1 100644 --- a/docs/drvqemu.rst +++ b/docs/drvqemu.rst @@ -523,6 +523,54 @@ Example: </qemu:capabilities> </domain>
+Control of QEMU deprecation warnings +------------------------------------ + +The following knob controls how QEMU behaves towards deprecated commands and +arguments used by libvirt: + +:: + + <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> + <name>testvm</name> + + [...] + + <qemu:deprecation behavior='crash'/> + +This setting is meant for developers and CI efforts to make it obvious when +libvirt relies on fields which are deprecated so that it can be fixes as soon +as possible. + +Possible options are: + +``none`` + (default) qemu is supposed to accept and output deprecated fields and commands + +``omit`` + qemu is instructed to omit deprecated fields on output, behaviour towards + fields and commadns from libvirtd is not changed
s/commadns/commands/
Enable '-compat' if requested in qemu.conf and supported by qemu to instruct qemu to crash when a deprecated command is used and stop returning deprecated fields. This setting is meant for libvirt developers and such. --- src/qemu/qemu_command.c | 80 +++++++++++++++++++ .../qemu-ns.x86_64-latest.args | 1 + 2 files changed, 81 insertions(+) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 1b4fa77867..2388b1a1a1 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -10325,6 +10325,84 @@ qemuBuildVsockCommandLine(virCommandPtr cmd, } +typedef enum { + QEMU_COMMAND_DEPRECATION_BEHAVIOR_NONE = 0, + QEMU_COMMAND_DEPRECATION_BEHAVIOR_OMIT, + QEMU_COMMAND_DEPRECATION_BEHAVIOR_REJECT, + QEMU_COMMAND_DEPRECATION_BEHAVIOR_CRASH, + + QEMU_COMMAND_DEPRECATION_BEHAVIOR_LAST +} qemuCommnadDeprecationBehavior; + + +VIR_ENUM_DECL(qemuCommnadDeprecationBehavior); +VIR_ENUM_IMPL(qemuCommnadDeprecationBehavior, + QEMU_COMMAND_DEPRECATION_BEHAVIOR_LAST, + "none", + "omit", + "reject", + "crash"); + +static void +qemuBuildCompatDeprecatedCommandLine(virCommand *cmd, + virQEMUDriverConfig *cfg, + virDomainDef *def, + virQEMUCaps *qemuCaps) +{ + g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER; + qemuDomainXmlNsDefPtr nsdata = def->namespaceData; + qemuCommnadDeprecationBehavior behavior = QEMU_COMMAND_DEPRECATION_BEHAVIOR_NONE; + const char *behaviorStr = cfg->deprecationBehavior; + int tmp; + + if (nsdata && nsdata->deprecationBehavior) + behaviorStr = nsdata->deprecationBehavior; + + if ((tmp = qemuCommnadDeprecationBehaviorTypeFromString(behaviorStr)) < 0) { + VIR_WARN("Unsupported deprecation behavior '%s' for VM '%s'", + behaviorStr, def->name); + return; + } + + behavior = tmp; + + if (behavior == QEMU_COMMAND_DEPRECATION_BEHAVIOR_NONE) + return; + + /* we don't try to enable this feature at all if qemu doesn't support it, + * so that a downgrade of qemu version doesn't impact startup of the VM */ + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_COMPAT_DEPRECATED)) { + VIR_DEBUG("-compat not supported for VM '%s'", def->name); + return; + } + + /* all active options hide output fields from qemu */ + virBufferAddLit(&buf, "deprecated-output=hide,"); + + switch (behavior) { + case QEMU_COMMAND_DEPRECATION_BEHAVIOR_OMIT: + case QEMU_COMMAND_DEPRECATION_BEHAVIOR_NONE: + case QEMU_COMMAND_DEPRECATION_BEHAVIOR_LAST: + default: + /* output field hiding is default for all cases */ + break; + + case QEMU_COMMAND_DEPRECATION_BEHAVIOR_REJECT: + virBufferAddLit(&buf, "deprecated-input=reject,"); + break; + + case QEMU_COMMAND_DEPRECATION_BEHAVIOR_CRASH: + virBufferAddLit(&buf, "deprecated-input=crash,"); + break; + } + + virBufferTrim(&buf, ","); + + virCommandAddArg(cmd, "-compat"); + virCommandAddArgBuffer(cmd, &buf); +} + + /* * Constructs a argv suitable for launching qemu with config defined * for a given virtual machine. @@ -10385,6 +10463,8 @@ qemuBuildCommandLine(virQEMUDriverPtr driver, if (qemuBuildNameCommandLine(cmd, cfg, def, qemuCaps) < 0) return NULL; + qemuBuildCompatDeprecatedCommandLine(cmd, cfg, def, qemuCaps); + if (!standalone) virCommandAddArg(cmd, "-S"); /* freeze CPU */ diff --git a/tests/qemuxml2argvdata/qemu-ns.x86_64-latest.args b/tests/qemuxml2argvdata/qemu-ns.x86_64-latest.args index 5a8136a8b7..71b83ca681 100644 --- a/tests/qemuxml2argvdata/qemu-ns.x86_64-latest.args +++ b/tests/qemuxml2argvdata/qemu-ns.x86_64-latest.args @@ -10,6 +10,7 @@ NS=ns \ BAR='' \ /usr/bin/qemu-system-i386 \ -name guest=QEMUGuest1,debug-threads=on \ +-compat deprecated-output=hide,deprecated-input=crash \ -S \ -object '{"qom-type":"secret","id":"masterKey0","format":"raw",\ "file":"/tmp/lib/domain--1-QEMUGuest1/master-key.aes"}' \ -- 2.29.2
On Fri, Mar 19, 2021 at 07:33:54PM +0100, Peter Krempa wrote:
Enable '-compat' if requested in qemu.conf and supported by qemu to instruct qemu to crash when a deprecated command is used and stop returning deprecated fields.
This setting is meant for libvirt developers and such. --- src/qemu/qemu_command.c | 80 +++++++++++++++++++ .../qemu-ns.x86_64-latest.args | 1 + 2 files changed, 81 insertions(+)
diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 1b4fa77867..2388b1a1a1 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -10325,6 +10325,84 @@ qemuBuildVsockCommandLine(virCommandPtr cmd, }
+typedef enum { + QEMU_COMMAND_DEPRECATION_BEHAVIOR_NONE = 0, + QEMU_COMMAND_DEPRECATION_BEHAVIOR_OMIT, + QEMU_COMMAND_DEPRECATION_BEHAVIOR_REJECT, + QEMU_COMMAND_DEPRECATION_BEHAVIOR_CRASH, + + QEMU_COMMAND_DEPRECATION_BEHAVIOR_LAST +} qemuCommnadDeprecationBehavior; + + +VIR_ENUM_DECL(qemuCommnadDeprecationBehavior); +VIR_ENUM_IMPL(qemuCommnadDeprecationBehavior, + QEMU_COMMAND_DEPRECATION_BEHAVIOR_LAST, + "none", + "omit", + "reject", + "crash"); + +static void +qemuBuildCompatDeprecatedCommandLine(virCommand *cmd, + virQEMUDriverConfig *cfg, + virDomainDef *def, + virQEMUCaps *qemuCaps) +{ + g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER; + qemuDomainXmlNsDefPtr nsdata = def->namespaceData; + qemuCommnadDeprecationBehavior behavior = QEMU_COMMAND_DEPRECATION_BEHAVIOR_NONE; + const char *behaviorStr = cfg->deprecationBehavior; + int tmp; + + if (nsdata && nsdata->deprecationBehavior) + behaviorStr = nsdata->deprecationBehavior; + + if ((tmp = qemuCommnadDeprecationBehaviorTypeFromString(behaviorStr)) < 0) { + VIR_WARN("Unsupported deprecation behavior '%s' for VM '%s'", + behaviorStr, def->name); + return; + } + + behavior = tmp; + + if (behavior == QEMU_COMMAND_DEPRECATION_BEHAVIOR_NONE) + return; + + /* we don't try to enable this feature at all if qemu doesn't support it, + * so that a downgrade of qemu version doesn't impact startup of the VM */ + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_COMPAT_DEPRECATED)) { + VIR_DEBUG("-compat not supported for VM '%s'", def->name); + return; + } + + /* all active options hide output fields from qemu */ + virBufferAddLit(&buf, "deprecated-output=hide,"); + + switch (behavior) { + case QEMU_COMMAND_DEPRECATION_BEHAVIOR_OMIT: + case QEMU_COMMAND_DEPRECATION_BEHAVIOR_NONE: + case QEMU_COMMAND_DEPRECATION_BEHAVIOR_LAST: + default: + /* output field hiding is default for all cases */ + break; + + case QEMU_COMMAND_DEPRECATION_BEHAVIOR_REJECT: + virBufferAddLit(&buf, "deprecated-input=reject,"); + break; + + case QEMU_COMMAND_DEPRECATION_BEHAVIOR_CRASH: + virBufferAddLit(&buf, "deprecated-input=crash,"); + break; + }
I wonder whether it would be beneficial to not hide the deprecated outputs (so that qemu output behaves just as it would without this option) but it would still crash when any deprecated field is used when instructing qemu. Probably does not make sense, just a thought.
On Fri, Mar 19, 2021 at 07:33:48PM +0100, Peter Krempa wrote:
For debugging purposes it's very useful to disable all deprecated commands and fields in qemu. This series implements a qemu.conf knob and a qemu namespace element to control this.
The implementation tries to be very conservative to allow downgrades of qemu and such without breaking the startup of the VM.
The intention is that developers and CI deployments use the 'crash' option to catch any unexpected qemu disappearance.
Note that this applies on top of my series for -object QAPIfication.
Based on Markus' -compat series which was now merged to upstream qemu.
Peter Krempa (6): docs/drvqemu: Convert to RST
I wanted to review the output of this first patch side-by-side so that I do not have to go through all the markup hell. But when I tried looking for the artefacts from any possible pipeline this could have run through I was greeted with a "get off my lawn" message. So if you are okay with me just skimming through the first patch (after reviewing the rest of da patch series), then with the mentioned fixes, this is for you (xkcd#276) Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
qemu: capabilities: Introduce QEMU_CAPS_COMPAT_DEPRECATED qemu: conf: Add 'deprecation_behavior' setting to qemu.conf qemuxml2xmltest: Enable 'qemu-ns' case qemu: Add per-VM control of deprecation behavior qemu: command: Handle formatting of '-compat' options
docs/drvqemu.html.in | 743 ------------------ docs/drvqemu.rst | 636 +++++++++++++++ docs/formatdomain.rst | 2 +- docs/manpages/virt-qemu-run.rst | 2 +- docs/meson.build | 2 +- docs/schemas/domaincommon.rng | 16 + src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf | 31 + src/qemu/qemu_capabilities.c | 8 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 80 ++ src/qemu/qemu_conf.c | 4 + src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.c | 10 +- src/qemu/qemu_domain.h | 5 + src/qemu/test_libvirtd_qemu.aug.in | 1 + .../caps_6.0.0.x86_64.xml | 1 + .../qemu-ns.x86_64-latest.args | 1 + tests/qemuxml2argvdata/qemu-ns.xml | 1 + .../qemu-ns.x86_64-latest.xml | 52 ++ tests/qemuxml2xmltest.c | 1 + 21 files changed, 853 insertions(+), 747 deletions(-) delete mode 100644 docs/drvqemu.html.in create mode 100644 docs/drvqemu.rst create mode 100644 tests/qemuxml2xmloutdata/qemu-ns.x86_64-latest.xml
-- 2.29.2
On Thu, Apr 01, 2021 at 00:36:52 +0200, Martin Kletzander wrote:
On Fri, Mar 19, 2021 at 07:33:48PM +0100, Peter Krempa wrote:
For debugging purposes it's very useful to disable all deprecated commands and fields in qemu. This series implements a qemu.conf knob and a qemu namespace element to control this.
The implementation tries to be very conservative to allow downgrades of qemu and such without breaking the startup of the VM.
The intention is that developers and CI deployments use the 'crash' option to catch any unexpected qemu disappearance.
Note that this applies on top of my series for -object QAPIfication.
Based on Markus' -compat series which was now merged to upstream qemu.
Peter Krempa (6): docs/drvqemu: Convert to RST
I wanted to review the output of this first patch side-by-side so that I do not have to go through all the markup hell. But when I tried looking for the artefacts from any possible pipeline this could have run through I was greeted with a "get off my lawn" message. So if you are okay with me just skimming through the first patch (after reviewing the rest of da patch series), then with the mentioned fixes, this is for you (xkcd#276)
Oops!, despite criticising the same thing last time [1] I actually didn't (forgot to?) push the branch to gitlab so that artifacts would be present. https://pipo.sk.gitlab.io/-/libvirt/-/jobs/1145757051/artifacts/website/drvq... [1] https://listman.redhat.com/archives/libvir-list/2021-March/msg00602.html
On Thu, Apr 01, 2021 at 10:37:31AM +0200, Peter Krempa wrote:
On Thu, Apr 01, 2021 at 00:36:52 +0200, Martin Kletzander wrote:
On Fri, Mar 19, 2021 at 07:33:48PM +0100, Peter Krempa wrote:
For debugging purposes it's very useful to disable all deprecated commands and fields in qemu. This series implements a qemu.conf knob and a qemu namespace element to control this.
The implementation tries to be very conservative to allow downgrades of qemu and such without breaking the startup of the VM.
The intention is that developers and CI deployments use the 'crash' option to catch any unexpected qemu disappearance.
Note that this applies on top of my series for -object QAPIfication.
Based on Markus' -compat series which was now merged to upstream qemu.
Peter Krempa (6): docs/drvqemu: Convert to RST
I wanted to review the output of this first patch side-by-side so that I do not have to go through all the markup hell. But when I tried looking for the artefacts from any possible pipeline this could have run through I was greeted with a "get off my lawn" message. So if you are okay with me just skimming through the first patch (after reviewing the rest of da patch series), then with the mentioned fixes, this is for you (xkcd#276)
Oops!, despite criticising the same thing last time [1] I actually didn't (forgot to?) push the branch to gitlab so that artifacts would be present.
https://pipo.sk.gitlab.io/-/libvirt/-/jobs/1145757051/artifacts/website/drvq...
=D looks good ;)
[1] https://listman.redhat.com/archives/libvir-list/2021-March/msg00602.html
participants (2)
-
Martin Kletzander -
Peter Krempa