[libvirt] [PATCH v2 00/15] docs: some refactoring and new docs about RPM deployment

This refactors existing docs related to the remote driver/daemon and URIs. It then also adds a kbase page about RPM package options. This introduces the use of RST for docs as a replacement for HTML. The intent is that all new docs should use RST from this point. The POD man pages will also be converted to RST. I'll post this as a different series. Note this series only touches the main HTML content, not the styling / templating using XSL. Tackling that is a separate job I've not attempted. My guess is that we'd use either Sphinx or Pelican to do the templating. Existing HTML docs are candidates for conversion to RST, however, people should *NOT* attempt todo this manually. In this series I've converted docs/kbase/ files, using the pandoc tool. This does a pretty good job in general but does need some manual cleanups. First I post-processed its output to change the heading highlighting to follow the documented style $ cat convert.pl my @in = <>; my @out; for (my $i ; $i <= $#in; $i++) { my $line = $in[$i]; if ($line =~ /^=+$/) { my @newout = splice @out, 0, $i - 1; push @newout, $line; push @newout, $out[$i-1]; push @newout, $out[$i]; push @newout, $line; push @newout, "\n"; push @newout, ".. contents::\n"; @out = @newout; } elsif ($line =~ /^(-|~|^|')+$/) { $line =~ s/-/=/g; $line =~ s/~/-/g; $line =~ s/\^/~/g; $line =~ s/'/^/g; push @out, $line; } else { push @out, $line; } } print @out; So I'm converting with pandoc -f html -t rst < foo.html.in | perl convert.pl > foo.rst $EDITOR foot.rst git rm -f foot.html.in git add foo.rst After pandoc & the headings convert, there's usually manual fixes needed to deal with a few oddities pandoc gets wrong, or looks ugly. This is still way simpler than converting the doc to rst manually. Changed in v2: - Added a style guide for RST docs - Probe for rst2html's different names - Added make rules for building rst - Fixed permalinks in generated docs - Misc CSS fixes - Auto-converted all of kbase/ directory Daniel P. Berrangé (15): docs: split TLS certificate setup into its own file docs: move docs about remote driver URIs into URI docs docs: introduce rst2html as a mandatory tool for building docs docs: adapt filling of <head> section for rst2html output docs: generate permalinks correctly for rst2html output docs: relax CSS context match for pretty tables docs: add styling for <tt> element docs: add a minimal style guide for writing RST docs docs: convert kbase/domainstatecapture.html.in to RST docs: convert kbase/launch_security_sev.html.in to RST docs: convert kbase/secureusage.html.in to RST docs: convert kbase/locking.html.in to RST docs: convert kbase/locking-lockd.html.in to RST docs: convert kbase/locking-sanlock.html.in to RST docs: add a kbase page about RPM packaging options docs/Makefile.am | 33 +- docs/aclpolkit.html.in | 20 +- docs/docs.html.in | 6 + docs/genaclperms.pl | 2 +- docs/generic.css | 4 +- docs/kbase.html.in | 4 + docs/kbase/domainstatecapture.html.in | 303 ----------- docs/kbase/domainstatecapture.rst | 255 +++++++++ docs/kbase/launch_security_sev.html.in | 533 ------------------- docs/kbase/launch_security_sev.rst | 529 +++++++++++++++++++ docs/kbase/locking-lockd.html.in | 160 ------ docs/kbase/locking-lockd.rst | 121 +++++ docs/kbase/locking-sanlock.html.in | 247 --------- docs/kbase/locking-sanlock.rst | 193 +++++++ docs/kbase/locking.html.in | 48 -- docs/kbase/locking.rst | 33 ++ docs/kbase/rpm-deployment.rst | 410 +++++++++++++++ docs/kbase/secureusage.html.in | 171 ------- docs/kbase/secureusage.rst | 131 +++++ docs/libvirt.css | 76 +-- docs/migration.html.in | 2 +- docs/newapi.xsl | 4 +- docs/page.xsl | 9 +- docs/remote.html.in | 684 +------------------------ docs/styleguide.rst | 66 +++ docs/tlscerts.html.in | 413 +++++++++++++++ docs/uri.html.in | 263 ++++++++-- libvirt.spec.in | 2 + m4/virt-external-programs.m4 | 5 + mingw-libvirt.spec.in | 1 + 30 files changed, 2478 insertions(+), 2250 deletions(-) delete mode 100644 docs/kbase/domainstatecapture.html.in create mode 100644 docs/kbase/domainstatecapture.rst delete mode 100644 docs/kbase/launch_security_sev.html.in create mode 100644 docs/kbase/launch_security_sev.rst delete mode 100644 docs/kbase/locking-lockd.html.in create mode 100644 docs/kbase/locking-lockd.rst delete mode 100644 docs/kbase/locking-sanlock.html.in create mode 100644 docs/kbase/locking-sanlock.rst delete mode 100644 docs/kbase/locking.html.in create mode 100644 docs/kbase/locking.rst create mode 100644 docs/kbase/rpm-deployment.rst delete mode 100644 docs/kbase/secureusage.html.in create mode 100644 docs/kbase/secureusage.rst create mode 100644 docs/styleguide.rst create mode 100644 docs/tlscerts.html.in -- 2.23.0

The generation and deployment of x509 certificates for TLS is complex and verbose and thus deserves its own standalone page. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/docs.html.in | 3 + docs/remote.html.in | 408 +---------------------------------------- docs/tlscerts.html.in | 413 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 417 insertions(+), 407 deletions(-) create mode 100644 docs/tlscerts.html.in diff --git a/docs/docs.html.in b/docs/docs.html.in index 268c16f3b3..f2721964b5 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -18,6 +18,9 @@ <dt><a href="remote.html">Remote access</a></dt> <dd>Enable remote access over TCP</dd> + <dt><a href="tlscerts.html">TLS certs</a></dt> + <dd>Generate and deploy x509 certificates for TLS</dd> + <dt><a href="auth.html">Authentication</a></dt> <dd>Configure authentication for the libvirt daemon</dd> diff --git a/docs/remote.html.in b/docs/remote.html.in index 78e071a898..5a0ebe4790 100644 --- a/docs/remote.html.in +++ b/docs/remote.html.in @@ -61,7 +61,7 @@ Remote libvirt supports a range of transports: <dd><a href="http://en.wikipedia.org/wiki/Transport_Layer_Security" title="Transport Layer Security">TLS</a> 1.0 (SSL 3.1) authenticated and encrypted TCP/IP socket, usually listening on a public port number. To use this you will need to - <a href="#Remote_certificates" title="Generating TLS certificates">generate client and + <a href="tlscerts.html" title="Generating TLS certificates">generate client and server certificates</a>. The standard port is 16514. </dd> @@ -382,412 +382,6 @@ Note that parameter values must be <td> Example: <code>sshauth=privkey,agent</code> </td> </tr> </table> - <h2> - <a id="Remote_certificates">Generating TLS certificates</a> - </h2> - <h3> - <a id="Remote_PKI">Public Key Infrastructure set up</a> - </h3> - <p> -If you are unsure how to create TLS certificates, skip to the -next section. -</p> - <table class="top_table"> - <tr> - <th> Location </th> - <th> Machine </th> - <th> Description </th> - <th> Required fields </th> - </tr> - <tr> - <td> - <code>/etc/pki/CA/cacert.pem</code> - </td> - <td> Installed on the client and server </td> - <td> CA's certificate (<a href="#Remote_TLS_CA">more info</a>)</td> - <td> n/a </td> - </tr> - <tr> - <td> - <code>$HOME/.pki/cacert.pem</code> - </td> - <td> Installed on the client </td> - <td> CA's certificate (<a href="#Remote_TLS_CA">more info</a>)</td> - <td> n/a </td> - </tr> - <tr> - <td> - <code>/etc/pki/libvirt/private/serverkey.pem</code> - </td> - <td> Installed on the server </td> - <td> Server's private key (<a href="#Remote_TLS_server_certificates">more info</a>)</td> - <td> n/a </td> - </tr> - <tr> - <td> - <code>/etc/pki/libvirt/servercert.pem</code> - </td> - <td> Installed on the server </td> - <td> Server's certificate signed by the CA. - (<a href="#Remote_TLS_server_certificates">more info</a>) </td> - <td> CommonName (CN) must be the hostname of the server as it - is seen by clients. All hostname and IP address variants that might - be used to reach the server should be listed in Subject Alt Name - fields.</td> - </tr> - <tr> - <td> - <code>/etc/pki/libvirt/private/clientkey.pem</code> - </td> - <td> Installed on the client </td> - <td> Client's private key. (<a href="#Remote_TLS_client_certificates">more info</a>) </td> - <td> n/a </td> - </tr> - <tr> - <td> - <code>/etc/pki/libvirt/clientcert.pem</code> - </td> - <td> Installed on the client </td> - <td> Client's certificate signed by the CA - (<a href="#Remote_TLS_client_certificates">more info</a>) </td> - <td> Distinguished Name (DN) can be checked against an access - control list (<code>tls_allowed_dn_list</code>). - </td> - </tr> - <tr> - <td> - <code>$HOME/.pki/libvirt/clientkey.pem</code> - </td> - <td> Installed on the client </td> - <td> Client's private key. (<a href="#Remote_TLS_client_certificates">more info</a>) </td> - <td> n/a </td> - </tr> - <tr> - <td> - <code>$HOME/.pki/libvirt/clientcert.pem</code> - </td> - <td> Installed on the client </td> - <td> Client's certificate signed by the CA - (<a href="#Remote_TLS_client_certificates">more info</a>) </td> - <td> Distinguished Name (DN) can be checked against an access - control list (<code>tls_allowed_dn_list</code>). - </td> - </tr> - </table> - <p> - If 'pkipath' is specified in URI, then all the client - certificates must be found in the path specified, otherwise the - connection will fail with a fatal error. If 'pkipath' is not - specified: - </p> - <ul> - <li> For a non-root user, libvirt tries to find the certificates - in $HOME/.pki/libvirt first. If the required CA certificate cannot - be found, then the global default location - (/etc/pki/CA/cacert.pem) will be used. - Likewise, if either the client certificate - or the client key cannot be found, then the global default - locations (/etc/pki/libvirt/clientcert.pem, - /etc/pki/libvirt/private/clientkey.pem) will be used. - </li> - <li> For the root user, the global default locations will always be used.</li> - </ul> - <h3> - <a id="Remote_TLS_background">Background to TLS certificates</a> - </h3> - <p> -Libvirt supports TLS certificates for verifying the identity -of the server and clients. There are two distinct checks involved: -</p> - <ul> - <li> The client should know that it is connecting to the right -server. Checking done by client by matching the certificate that -the server sends to the server's hostname. May be disabled by adding -<code>?no_verify=1</code> to the -<a href="#Remote_URI_parameters">remote URI</a>. -</li> - <li> The server should know that only permitted clients are -connecting. This can be done based on client's IP address, or on -client's IP address and client's certificate. Checking done by the -server. May be enabled and disabled in the <a href="#Remote_libvirtd_configuration">libvirtd.conf file</a>. -</li> - </ul> - <p> -For full certificate checking you will need to have certificates -issued by a recognised <a href="http://en.wikipedia.org/wiki/Certificate_authority">Certificate -Authority (CA)</a> for your server(s) and all clients. To avoid the -expense of getting certificates from a commercial CA, you can set up -your own CA and tell your server(s) and clients to trust certificates -issues by your own CA. Follow the instructions in the next section. -</p> - <p> -Be aware that the <a href="#Remote_libvirtd_configuration">default -configuration for libvirtd</a> allows any client to connect provided -they have a valid certificate issued by the CA for their own IP -address. You may want to change this to make it less (or more) -permissive, depending on your needs. -</p> - <h3> - <a id="Remote_TLS_CA">Setting up a Certificate Authority (CA)</a> - </h3> - <p> -You will need the <a href="http://www.gnu.org/software/gnutls/manual/html_node/Invoking-certtool.html">GnuTLS -certtool program documented here</a>. In Fedora, it is in the -<code>gnutls-utils</code> package. -</p> - <p> -Create a private key for your CA: -</p> - <pre> -certtool --generate-privkey > cakey.pem -</pre> - <p> -and self-sign it by creating a file with the -signature details called -<code>ca.info</code> containing: -</p> - <pre> -cn = <i>Name of your organization</i> -ca -cert_signing_key -</pre> - <pre> -certtool --generate-self-signed --load-privkey cakey.pem \ - --template ca.info --outfile cacert.pem -</pre> - <p> -(You can delete <code>ca.info</code> file now if you -want). -</p> - <p> -Now you have two files which matter: -</p> - <ul> - <li><code>cakey.pem</code> - Your CA's private key (keep this very secret!) -</li> - <li><code>cacert.pem</code> - Your CA's certificate (this is public). -</li> - </ul> - <p><code>cacert.pem</code> has to be installed on clients and -server(s) to let them know that they can trust certificates issued by -your CA. -</p> - <p> -The normal installation directory for <code>cacert.pem</code> -is <code>/etc/pki/CA/cacert.pem</code> on all clients and servers. -</p> - <p> -To see the contents of this file, do: -</p> - <pre><b>certtool -i --infile cacert.pem</b> - -X.509 certificate info: - -Version: 3 -Serial Number (hex): 00 -Subject: CN=Libvirt Project -Issuer: CN=Libvirt Project -Signature Algorithm: RSA-SHA -Validity: - Not Before: Mon Jun 18 16:22:18 2007 - Not After: Tue Jun 17 16:22:18 2008 -<i>[etc]</i> -</pre> - <p> -This is all that is required to set up your CA. Keep the CA's private -key carefully as you will need it when you come to issue certificates -for your clients and servers. -</p> - <h3> - <a id="Remote_TLS_server_certificates">Issuing server certificates</a> - </h3> - <p> -For each server (libvirtd) you need to issue a certificate -containing one or more hostnames and/or IP addresses. -Historically the CommonName (CN) field would contain the -hostname of the server and would match the hostname used -in the URI that clients pass to libvirt. In most TLS implementations -the CN field is considered legacy data. The preferential mechanism -is to use Subject Alt Name (SAN) extension fields to validate -against. In the future use of the CN field for validation may be -discontinued entirely, so it is strongly recommended to -include the SAN fields. -</p> - <p> -In the example below, clients will be connecting to the -server using a <a href="#Remote_URI_reference">URI</a> of -<code>qemu://compute1.libvirt.org/system</code>, so the CN -must be "<code>compute1.libvirt.org</code>". -</p> - <p> -Make a private key for the server: -</p> - <pre> -certtool --generate-privkey > serverkey.pem -</pre> - <p> -and sign that key with the CA's private key by first -creating a template file called <code>server.info</code>. -The template file will contain a number of fields to define -the server as follows: -</p> - <pre> -organization = <i>Name of your organization</i> -cn = compute1.libvirt.org -dns_name = compute1 -dns_name = compute1.libvirt.org -ip_address = 10.0.0.74 -ip_address = 192.168.1.24 -ip_address = 2001:cafe::74 -ip_address = fe20::24 -tls_www_server -encryption_key -signing_key -</pre> -<p> -The 'cn' field should refer to the fully qualified public -hostname of the server. For the SAN extension data, there -must also be one or more 'dns_name' fields that contain all -possible hostnames that can be reasonably used by clients -to reach the server, both with and without domain name -qualifiers. If clients are likely to connect to the server -by IP address, then one or more 'ip_address' fields should -also be added. -</p> - <p> -Use the template file as input to a <code>certtool</code> -command to sign the server certificate: -</p> - <pre> -certtool --generate-certificate --load-privkey serverkey.pem \ - --load-ca-certificate cacert.pem --load-ca-privkey cakey.pem \ - --template server.info --outfile servercert.pem -</pre> - <p> -This gives two files: -</p> - <ul> - <li><code>serverkey.pem</code> - The server's private key. -</li> - <li><code>servercert.pem</code> - The server's public key. -</li> - </ul> - <p> -We can examine this certificate and its signature: -</p> - <pre><b>certtool -i --infile servercert.pem</b> -X.509 certificate info: - -Version: 3 -Serial Number (hex): 00 -Subject: O=Libvirt Project,CN=compute1.libvirt.org -Issuer: CN=Libvirt Project -Signature Algorithm: RSA-SHA -Validity: - Not Before: Wed Oct 04 09:09:44 UTC 2017 - Not After: Thu Oct 04 09:09:44 UTC 2018 -Extensions: - Basic Constraints (critical): - Certificate Authority (CA): FALSE - Subject Alternative Name (not critical): - DNSname: compute1 - DNSname: compute1.libvirt.org - IPAddress: 10.0.0.74 - IPAddress: 192.168.1.24 - IPAddress: 2001:cafe::74 - IPAddress: fe20::24 -</pre> - <p> -Note the "Issuer" CN is "Libvirt Project" (the CA) and -the "Subject" CN is "compute1.libvirt.org" (the server). -Notice that the hostname listed in the CN must also -be duplicated as a DNSname entry -</p> - <p> -Finally we have two files to install: -</p> - <ul> - <li><code>serverkey.pem</code> is -the server's private key which should be copied to the -server <i>only</i> as -<code>/etc/pki/libvirt/private/serverkey.pem</code>. -</li> - <li><code>servercert.pem</code> is the server's certificate -which can be installed on the server as -<code>/etc/pki/libvirt/servercert.pem</code>. -</li> - </ul> - <h3> - <a id="Remote_TLS_client_certificates">Issuing client certificates</a> - </h3> - <p> -For each client (ie. any program linked with libvirt, such as -<a href="http://virt-manager.org/">virt-manager</a>) -you need to issue a certificate with the X.509 Distinguished Name (DN) -set to a suitable name. You can decide this on a company / organisation -policy. For example: -</p> - <pre> -C=GB,ST=London,L=London,O=Libvirt Project,CN=<i>name_of_client</i> -</pre> - <p> -The process is the same as for -<a href="#Remote_TLS_server_certificates">setting up the -server certificate</a> so here we just briefly cover the -steps. -</p> - <ol> - <li> -Make a private key: -<pre> -certtool --generate-privkey > clientkey.pem -</pre> -</li> - <li> -Act as CA and sign the certificate. Create client.info containing: -<pre> -country = GB -state = London -locality = London -organization = Libvirt Project -cn = client1 -tls_www_client -encryption_key -signing_key -</pre> -and sign by doing: -<pre> -certtool --generate-certificate --load-privkey clientkey.pem \ - --load-ca-certificate cacert.pem --load-ca-privkey cakey.pem \ - --template client.info --outfile clientcert.pem -</pre> -</li> - <li> -Install the certificates on the client machine: -<pre> -cp clientkey.pem /etc/pki/libvirt/private/clientkey.pem -cp clientcert.pem /etc/pki/libvirt/clientcert.pem -</pre> -</li> - </ol> - <h3> - <a id="Remote_TLS_troubleshooting">Troubleshooting TLS certificate problems</a> - </h3> - <dl> - <dt> failed to verify client's certificate </dt> - <dd> - <p> -On the server side, run the libvirtd server with -the '--listen' and '--verbose' options while the -client is connecting. The verbose log messages should -tell you enough to diagnose the problem. -</p> - </dd> - </dl> - <p> You can use the virt-pki-validate shell script -to analyze the setup on the client or server machines, preferably as root. -It will try to point out the possible problems and provide solutions to -fix the set up up to a point where you have secure remote access.</p> <h2> <a id="Remote_libvirtd_configuration">libvirtd configuration file</a> </h2> diff --git a/docs/tlscerts.html.in b/docs/tlscerts.html.in new file mode 100644 index 0000000000..3eea686daa --- /dev/null +++ b/docs/tlscerts.html.in @@ -0,0 +1,413 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml"> + <body> + <h1>TLS x509 certificate setup</h1> + + <ul id="toc"></ul> + + <h2> + <a id="Remote_PKI">Public Key Infrastructure set up</a> + </h2> + <p> +If you are unsure how to create TLS certificates, skip to the +next section. +</p> + <table class="top_table"> + <tr> + <th> Location </th> + <th> Machine </th> + <th> Description </th> + <th> Required fields </th> + </tr> + <tr> + <td> + <code>/etc/pki/CA/cacert.pem</code> + </td> + <td> Installed on the client and server </td> + <td> CA's certificate (<a href="#Remote_TLS_CA">more info</a>)</td> + <td> n/a </td> + </tr> + <tr> + <td> + <code>$HOME/.pki/cacert.pem</code> + </td> + <td> Installed on the client </td> + <td> CA's certificate (<a href="#Remote_TLS_CA">more info</a>)</td> + <td> n/a </td> + </tr> + <tr> + <td> + <code>/etc/pki/libvirt/private/serverkey.pem</code> + </td> + <td> Installed on the server </td> + <td> Server's private key (<a href="#Remote_TLS_server_certificates">more info</a>)</td> + <td> n/a </td> + </tr> + <tr> + <td> + <code>/etc/pki/libvirt/servercert.pem</code> + </td> + <td> Installed on the server </td> + <td> Server's certificate signed by the CA. + (<a href="#Remote_TLS_server_certificates">more info</a>) </td> + <td> CommonName (CN) must be the hostname of the server as it + is seen by clients. All hostname and IP address variants that might + be used to reach the server should be listed in Subject Alt Name + fields.</td> + </tr> + <tr> + <td> + <code>/etc/pki/libvirt/private/clientkey.pem</code> + </td> + <td> Installed on the client </td> + <td> Client's private key. (<a href="#Remote_TLS_client_certificates">more info</a>) </td> + <td> n/a </td> + </tr> + <tr> + <td> + <code>/etc/pki/libvirt/clientcert.pem</code> + </td> + <td> Installed on the client </td> + <td> Client's certificate signed by the CA + (<a href="#Remote_TLS_client_certificates">more info</a>) </td> + <td> Distinguished Name (DN) can be checked against an access + control list (<code>tls_allowed_dn_list</code>). + </td> + </tr> + <tr> + <td> + <code>$HOME/.pki/libvirt/clientkey.pem</code> + </td> + <td> Installed on the client </td> + <td> Client's private key. (<a href="#Remote_TLS_client_certificates">more info</a>) </td> + <td> n/a </td> + </tr> + <tr> + <td> + <code>$HOME/.pki/libvirt/clientcert.pem</code> + </td> + <td> Installed on the client </td> + <td> Client's certificate signed by the CA + (<a href="#Remote_TLS_client_certificates">more info</a>) </td> + <td> Distinguished Name (DN) can be checked against an access + control list (<code>tls_allowed_dn_list</code>). + </td> + </tr> + </table> + <p> + If 'pkipath' is specified in URI, then all the client + certificates must be found in the path specified, otherwise the + connection will fail with a fatal error. If 'pkipath' is not + specified: + </p> + <ul> + <li> For a non-root user, libvirt tries to find the certificates + in $HOME/.pki/libvirt first. If the required CA certificate cannot + be found, then the global default location + (/etc/pki/CA/cacert.pem) will be used. + Likewise, if either the client certificate + or the client key cannot be found, then the global default + locations (/etc/pki/libvirt/clientcert.pem, + /etc/pki/libvirt/private/clientkey.pem) will be used. + </li> + <li> For the root user, the global default locations will always be used.</li> + </ul> + <h2> + <a id="Remote_TLS_background">Background to TLS certificates</a> + </h2> + <p> +Libvirt supports TLS certificates for verifying the identity +of the server and clients. There are two distinct checks involved: +</p> + <ul> + <li> The client should know that it is connecting to the right +server. Checking done by client by matching the certificate that +the server sends to the server's hostname. May be disabled by adding +<code>?no_verify=1</code> to the +<a href="#Remote_URI_parameters">remote URI</a>. +</li> + <li> The server should know that only permitted clients are +connecting. This can be done based on client's IP address, or on +client's IP address and client's certificate. Checking done by the +server. May be enabled and disabled in the <a href="#Remote_libvirtd_configuration">libvirtd.conf file</a>. +</li> + </ul> + <p> +For full certificate checking you will need to have certificates +issued by a recognised <a href="http://en.wikipedia.org/wiki/Certificate_authority">Certificate +Authority (CA)</a> for your server(s) and all clients. To avoid the +expense of getting certificates from a commercial CA, you can set up +your own CA and tell your server(s) and clients to trust certificates +issues by your own CA. Follow the instructions in the next section. +</p> + <p> +Be aware that the <a href="#Remote_libvirtd_configuration">default +configuration for libvirtd</a> allows any client to connect provided +they have a valid certificate issued by the CA for their own IP +address. You may want to change this to make it less (or more) +permissive, depending on your needs. +</p> + <h2> + <a id="Remote_TLS_CA">Setting up a Certificate Authority (CA)</a> + </h2> + <p> +You will need the <a href="http://www.gnu.org/software/gnutls/manual/html_node/Invoking-certtool.html">GnuTLS +certtool program documented here</a>. In Fedora, it is in the +<code>gnutls-utils</code> package. +</p> + <p> +Create a private key for your CA: +</p> + <pre> +certtool --generate-privkey > cakey.pem +</pre> + <p> +and self-sign it by creating a file with the +signature details called +<code>ca.info</code> containing: +</p> + <pre> +cn = <i>Name of your organization</i> +ca +cert_signing_key +</pre> + <pre> +certtool --generate-self-signed --load-privkey cakey.pem \ + --template ca.info --outfile cacert.pem +</pre> + <p> +(You can delete <code>ca.info</code> file now if you +want). +</p> + <p> +Now you have two files which matter: +</p> + <ul> + <li><code>cakey.pem</code> - Your CA's private key (keep this very secret!) +</li> + <li><code>cacert.pem</code> - Your CA's certificate (this is public). +</li> + </ul> + <p><code>cacert.pem</code> has to be installed on clients and +server(s) to let them know that they can trust certificates issued by +your CA. +</p> + <p> +The normal installation directory for <code>cacert.pem</code> +is <code>/etc/pki/CA/cacert.pem</code> on all clients and servers. +</p> + <p> +To see the contents of this file, do: +</p> + <pre><b>certtool -i --infile cacert.pem</b> + +X.509 certificate info: + +Version: 3 +Serial Number (hex): 00 +Subject: CN=Libvirt Project +Issuer: CN=Libvirt Project +Signature Algorithm: RSA-SHA +Validity: + Not Before: Mon Jun 18 16:22:18 2007 + Not After: Tue Jun 17 16:22:18 2008 +<i>[etc]</i> +</pre> + <p> +This is all that is required to set up your CA. Keep the CA's private +key carefully as you will need it when you come to issue certificates +for your clients and servers. +</p> + <h2> + <a id="Remote_TLS_server_certificates">Issuing server certificates</a> + </h2> + <p> +For each server (libvirtd) you need to issue a certificate +containing one or more hostnames and/or IP addresses. +Historically the CommonName (CN) field would contain the +hostname of the server and would match the hostname used +in the URI that clients pass to libvirt. In most TLS implementations +the CN field is considered legacy data. The preferential mechanism +is to use Subject Alt Name (SAN) extension fields to validate +against. In the future use of the CN field for validation may be +discontinued entirely, so it is strongly recommended to +include the SAN fields. +</p> + <p> +In the example below, clients will be connecting to the +server using a <a href="#Remote_URI_reference">URI</a> of +<code>qemu://compute1.libvirt.org/system</code>, so the CN +must be "<code>compute1.libvirt.org</code>". +</p> + <p> +Make a private key for the server: +</p> + <pre> +certtool --generate-privkey > serverkey.pem +</pre> + <p> +and sign that key with the CA's private key by first +creating a template file called <code>server.info</code>. +The template file will contain a number of fields to define +the server as follows: +</p> + <pre> +organization = <i>Name of your organization</i> +cn = compute1.libvirt.org +dns_name = compute1 +dns_name = compute1.libvirt.org +ip_address = 10.0.0.74 +ip_address = 192.168.1.24 +ip_address = 2001:cafe::74 +ip_address = fe20::24 +tls_www_server +encryption_key +signing_key +</pre> +<p> +The 'cn' field should refer to the fully qualified public +hostname of the server. For the SAN extension data, there +must also be one or more 'dns_name' fields that contain all +possible hostnames that can be reasonably used by clients +to reach the server, both with and without domain name +qualifiers. If clients are likely to connect to the server +by IP address, then one or more 'ip_address' fields should +also be added. +</p> + <p> +Use the template file as input to a <code>certtool</code> +command to sign the server certificate: +</p> + <pre> +certtool --generate-certificate --load-privkey serverkey.pem \ + --load-ca-certificate cacert.pem --load-ca-privkey cakey.pem \ + --template server.info --outfile servercert.pem +</pre> + <p> +This gives two files: +</p> + <ul> + <li><code>serverkey.pem</code> - The server's private key. +</li> + <li><code>servercert.pem</code> - The server's public key. +</li> + </ul> + <p> +We can examine this certificate and its signature: +</p> + <pre><b>certtool -i --infile servercert.pem</b> +X.509 certificate info: + +Version: 3 +Serial Number (hex): 00 +Subject: O=Libvirt Project,CN=compute1.libvirt.org +Issuer: CN=Libvirt Project +Signature Algorithm: RSA-SHA +Validity: + Not Before: Wed Oct 04 09:09:44 UTC 2017 + Not After: Thu Oct 04 09:09:44 UTC 2018 +Extensions: + Basic Constraints (critical): + Certificate Authority (CA): FALSE + Subject Alternative Name (not critical): + DNSname: compute1 + DNSname: compute1.libvirt.org + IPAddress: 10.0.0.74 + IPAddress: 192.168.1.24 + IPAddress: 2001:cafe::74 + IPAddress: fe20::24 +</pre> + <p> +Note the "Issuer" CN is "Libvirt Project" (the CA) and +the "Subject" CN is "compute1.libvirt.org" (the server). +Notice that the hostname listed in the CN must also +be duplicated as a DNSname entry +</p> + <p> +Finally we have two files to install: +</p> + <ul> + <li><code>serverkey.pem</code> is +the server's private key which should be copied to the +server <i>only</i> as +<code>/etc/pki/libvirt/private/serverkey.pem</code>. +</li> + <li><code>servercert.pem</code> is the server's certificate +which can be installed on the server as +<code>/etc/pki/libvirt/servercert.pem</code>. +</li> + </ul> + <h2> + <a id="Remote_TLS_client_certificates">Issuing client certificates</a> + </h2> + <p> +For each client (ie. any program linked with libvirt, such as +<a href="http://virt-manager.org/">virt-manager</a>) +you need to issue a certificate with the X.509 Distinguished Name (DN) +set to a suitable name. You can decide this on a company / organisation +policy. For example: +</p> + <pre> +C=GB,ST=London,L=London,O=Libvirt Project,CN=<i>name_of_client</i> +</pre> + <p> +The process is the same as for +<a href="#Remote_TLS_server_certificates">setting up the +server certificate</a> so here we just briefly cover the +steps. +</p> + <ol> + <li> +Make a private key: +<pre> +certtool --generate-privkey > clientkey.pem +</pre> +</li> + <li> +Act as CA and sign the certificate. Create client.info containing: +<pre> +country = GB +state = London +locality = London +organization = Libvirt Project +cn = client1 +tls_www_client +encryption_key +signing_key +</pre> +and sign by doing: +<pre> +certtool --generate-certificate --load-privkey clientkey.pem \ + --load-ca-certificate cacert.pem --load-ca-privkey cakey.pem \ + --template client.info --outfile clientcert.pem +</pre> +</li> + <li> +Install the certificates on the client machine: +<pre> +cp clientkey.pem /etc/pki/libvirt/private/clientkey.pem +cp clientcert.pem /etc/pki/libvirt/clientcert.pem +</pre> +</li> + </ol> + <h2> + <a id="Remote_TLS_troubleshooting">Troubleshooting TLS certificate problems</a> + </h2> + <dl> + <dt> failed to verify client's certificate </dt> + <dd> + <p> +On the server side, run the libvirtd server with +the '--listen' and '--verbose' options while the +client is connecting. The verbose log messages should +tell you enough to diagnose the problem. +</p> + </dd> + </dl> + <p> You can use the virt-pki-validate shell script +to analyze the setup on the client or server machines, preferably as root. +It will try to point out the possible problems and provide solutions to +fix the set up up to a point where you have secure remote access.</p> + </body> +</html> -- 2.23.0

The docs about remote URIs in uri.html are somewhat sparse with the full docs being in remote.html. Move all the URI content from remote.html into uri.html so the user only needs to look in one place for URI info. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/remote.html.in | 278 +------------------------------------------- docs/uri.html.in | 263 ++++++++++++++++++++++++++++++++++++----- 2 files changed, 238 insertions(+), 303 deletions(-) diff --git a/docs/remote.html.in b/docs/remote.html.in index 5a0ebe4790..0b0dc87f6f 100644 --- a/docs/remote.html.in +++ b/docs/remote.html.in @@ -34,7 +34,7 @@ the system-wide QEMU daemon on a remote machine called <code>qemu://compute1.libvirt.org/system</code>. </p> <p> -The <a href="#Remote_URI_reference">section on remote URIs</a> +The <a href="uri.html#URI_remote">section on remote URIs</a> describes in more detail these remote URIs. </p> <p> @@ -109,279 +109,9 @@ even with graphical management applications. As with the classic ssh transport netcat is required on the remote side.</dd> </dl> <p> -The default transport, if no other is specified, is <code>tls</code>. -</p> - <h2> - <a id="Remote_URI_reference">Remote URIs</a> - </h2> - <p> -See also: <a href="uri.html">documentation on ordinary ("local") URIs</a>. -</p> - <p> -Remote URIs have the general form ("[...]" meaning an optional part): -</p> - <p><code>driver</code>[<code>+transport</code>]<code>://</code>[<code>username@</code>][<code>hostname</code>][<code>:port</code>]<code>/</code>[<code>path</code>][<code>?extraparameters</code>] -</p> - <p> -Either the transport or the hostname must be given in order -to distinguish this from a local URI. -</p> - <p> -Some examples: -</p> - <ul> - <li><code>xen+ssh://rjones@towada/system</code><br/> — Connect to a -remote Xen hypervisor on host <code>towada</code> using ssh transport and ssh -username <code>rjones</code>. -</li> - <li><code>xen://towada/system</code><br/> — Connect to a -remote Xen hypervisor on host <code>towada</code> using TLS. -</li> - <li><code>xen://towada/system?no_verify=1</code><br/> — Connect to a -remote Xen hypervisor on host <code>towada</code> using TLS. Do not verify -the server's certificate. -</li> - <li><code>qemu+unix:///system?socket=/opt/libvirt/run/libvirt/libvirt-sock</code><br/> — -Connect to the local qemu instances over a non-standard -Unix socket (the full path to the Unix socket is -supplied explicitly in this case). -</li> - <li><code>test+tcp://localhost:5000/default</code><br/> — -Connect to a libvirtd daemon offering unencrypted TCP/IP connections -on localhost port 5000 and use the test driver with default -settings. -</li> -<li><code>qemu+libssh2://user@host/system?known_hosts=/home/user/.ssh/known_hosts</code><br/> — -Connect to a remote host using a ssh connection with the libssh2 driver -and use a different known_hosts file.</li> -<li><code>qemu+libssh://user@host/system?known_hosts=/home/user/.ssh/known_hosts</code><br/> — -Connect to a remote host using a ssh connection with the libssh driver -and use a different known_hosts file.</li> - </ul> - <h3> - <a id="Remote_URI_parameters">Extra parameters</a> - </h3> - <p> -Extra parameters can be added to remote URIs as part -of the query string (the part following <q><code>?</code></q>). -Remote URIs understand the extra parameters shown below. -Any others are passed unmodified through to the back end. -Note that parameter values must be -<a href="http://xmlsoft.org/html/libxml-uri.html#xmlURIEscapeStr">URI-escaped</a>. -</p> - <table class="top_table"> - <tr> - <th> Name </th> - <th> Transports </th> - <th> Meaning </th> - </tr> - <tr> - <td> - <code>name</code> - </td> - <td> - <i>any transport</i> - </td> - <td> - The name passed to the remote virConnectOpen function. The - name is normally formed by removing transport, hostname, port - number, username and extra parameters from the remote URI, but in certain - very complex cases it may be better to supply the name explicitly. -</td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>name=qemu:///system</code> </td> - </tr> - <tr> - <td> - <code>tls_priority</code> - </td> - <td> tls </td> - <td> - A vaid GNUTLS priority string -</td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>tls_priority=NORMAL:-VERS-SSL3.0</code> </td> - </tr> - <tr> - <td> - <code>mode</code> - </td> - <td> unix, ssh, libssh, libssh2 </td> - <td> - <dl> - <dt><code>auto</code></dt><dd>automatically determine the daemon</dd> - <dt><code>direct</code></dt><dd>connect to per-driver daemons</dd> - <dt><code>legacy</code></dt><dd>connect to libvirtd</dd> - </dl> - Can also be set in <code>libvirt.conf</code> as <code>remote_mode</code> - </td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>mode=direct</code> </td> - </tr> - <tr> - <td> - <code>command</code> - </td> - <td> ssh, ext </td> - <td> - The external command. For ext transport this is required. - For ssh the default is <code>ssh</code>. - The PATH is searched for the command. -</td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>command=/opt/openssh/bin/ssh</code> </td> - </tr> - <tr> - <td> - <code>socket</code> - </td> - <td> unix, ssh, libssh2, libssh </td> - <td> - The path to the Unix domain socket, which overrides the - compiled-in default. For ssh transport, this is passed to - the remote netcat command (see next). -</td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>socket=/opt/libvirt/run/libvirt/libvirt-sock</code> </td> - </tr> - <tr> - <td> - <code>netcat</code> - </td> - <td> ssh, libssh2, libssh </td> - <td> - The name of the netcat command on the remote machine. - The default is <code>nc</code>. For ssh transport, libvirt - constructs an ssh command which looks like: - -<pre><i>command</i> -p <i>port</i> [-l <i>username</i>] <i>hostname</i> <i>netcat</i> -U <i>socket</i> -</pre> - - where <i>port</i>, <i>username</i>, <i>hostname</i> can be - specified as part of the remote URI, and <i>command</i>, <i>netcat</i> - and <i>socket</i> come from extra parameters (or - sensible defaults). - -</td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>netcat=/opt/netcat/bin/nc</code> </td> - </tr> - - <tr> - <td> - <code>keyfile</code> - </td> - <td> ssh, libssh2, libssh </td> - <td> - The name of the private key file to use to authentication to the remote - machine. If this option is not used the default keys are used. - </td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>keyfile=/root/.ssh/example_key</code> </td> - </tr> - - <tr> - <td> - <code>no_verify</code> - </td> - <td> ssh, tls </td> - <td> - SSH: If set to a non-zero value, this disables client's strict host key - checking making it auto-accept new host keys. Existing host keys will - still be validated. - <br/> - <br/> - TLS: If set to a non-zero value, this disables client checks of the - server's certificate. Note that to disable server checks of - the client's certificate or IP address you must - <a href="#Remote_libvirtd_configuration">change the libvirtd - configuration</a>. -</td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>no_verify=1</code> </td> - </tr> - <tr> - <td> - <code>no_tty</code> - </td> - <td> ssh </td> - <td> - If set to a non-zero value, this stops ssh from asking for - a password if it cannot log in to the remote machine automatically - (eg. using ssh-agent etc.). Use this when you don't have access - to a terminal - for example in graphical programs which use libvirt. -</td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>no_tty=1</code> </td> - </tr> - <tr> - <td> - <code>pkipath</code> - </td> - <td> tls</td> - <td> - Specifies x509 certificates path for the client. If any of - the CA certificate, client certificate, or client key is - missing, the connection will fail with a fatal error. - </td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>pkipath=/tmp/pki/client</code> </td> - </tr> - <tr> - <td> - <code>known_hosts</code> - </td> - <td> libssh2, libssh </td> - <td> - Path to the known_hosts file to verify the host key against. LibSSH2 and - libssh support OpenSSH-style known_hosts files, although LibSSH2 does not - support all key types, so using files created by the OpenSSH binary may - result into truncating the known_hosts file. Thus, with LibSSH2 it's - recommended to use the default known_hosts file is located in libvirt's - client local configuration directory e.g.: ~/.config/libvirt/known_hosts. - Note: Use absolute paths. -</td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>known_hosts=/root/.ssh/known_hosts</code> </td> - </tr> - <tr> - <td> - <code>sshauth</code> - </td> - <td> libssh2, libssh </td> - <td> - A comma separated list of authentication methods to use. Default (is - "agent,privkey,password,keyboard-interactive". The order of the methods - is preserved. Some methods may require additional parameters. -</td> - </tr> - <tr> - <td colspan="2"/> - <td> Example: <code>sshauth=privkey,agent</code> </td> - </tr> - </table> + The choice of transport is determined by the <a href="uri.html#URI_remote">URI scheme</a>, + with <code>tls</code> as the default if no explicit transport is requested. + </p> <h2> <a id="Remote_libvirtd_configuration">libvirtd configuration file</a> </h2> diff --git a/docs/uri.html.in b/docs/uri.html.in index 6da9eb9746..49f92773f8 100644 --- a/docs/uri.html.in +++ b/docs/uri.html.in @@ -153,65 +153,270 @@ here</a>. <a id="URI_remote">Remote URIs</a> </h2> <p> -Remote URIs are formed by taking ordinary local URIs and adding a -hostname and/or transport name. As a special case, using a URI -scheme of 'remote', will tell the remote libvirtd server to probe -for the optimal hypervisor driver. This is equivalent to passing -a NULL URI for a local connection. For example: +Remote URIs have the general form ("[...]" meaning an optional part): +</p> + <p><code>driver</code>[<code>+transport</code>]<code>://</code>[<code>username@</code>][<code>hostname</code>][<code>:port</code>]<code>/</code>[<code>path</code>][<code>?extraparameters</code>] +</p> + <p> +Either the transport or the hostname must be given in order +to distinguish this from a local URI. +</p> + <p> +Some examples: +</p> + <ul> + <li><code>xen+ssh://rjones@towada/system</code><br/> — Connect to a +remote Xen hypervisor on host <code>towada</code> using ssh transport and ssh +username <code>rjones</code>. +</li> + <li><code>xen://towada/system</code><br/> — Connect to a +remote Xen hypervisor on host <code>towada</code> using TLS. +</li> + <li><code>xen://towada/system?no_verify=1</code><br/> — Connect to a +remote Xen hypervisor on host <code>towada</code> using TLS. Do not verify +the server's certificate. +</li> + <li><code>qemu+unix:///system?socket=/opt/libvirt/run/libvirt/libvirt-sock</code><br/> — +Connect to the local qemu instances over a non-standard +Unix socket (the full path to the Unix socket is +supplied explicitly in this case). +</li> + <li><code>test+tcp://localhost:5000/default</code><br/> — +Connect to a libvirtd daemon offering unencrypted TCP/IP connections +on localhost port 5000 and use the test driver with default +settings. +</li> +<li><code>qemu+libssh2://user@host/system?known_hosts=/home/user/.ssh/known_hosts</code><br/> — +Connect to a remote host using a ssh connection with the libssh2 driver +and use a different known_hosts file.</li> +<li><code>qemu+libssh://user@host/system?known_hosts=/home/user/.ssh/known_hosts</code><br/> — +Connect to a remote host using a ssh connection with the libssh driver +and use a different known_hosts file.</li> + </ul> + <h3> + <a id="Remote_URI_parameters">Extra parameters</a> + </h3> + <p> +Extra parameters can be added to remote URIs as part +of the query string (the part following <q><code>?</code></q>). +Remote URIs understand the extra parameters shown below. +Any others are passed unmodified through to the back end. +Note that parameter values must be +<a href="http://xmlsoft.org/html/libxml-uri.html#xmlURIEscapeStr">URI-escaped</a>. </p> <table class="top_table"> <tr> - <th> Local URI </th> - <th> Remote URI </th> + <th> Name </th> + <th> Transports </th> <th> Meaning </th> </tr> <tr> <td> - <code>xen:///system</code> + <code>name</code> </td> <td> - <code>xen://oirase/system</code> + <i>any transport</i> </td> - <td> Connect to the Xen hypervisor running on host <code>oirase</code> - using TLS. </td> + <td> + The name passed to the remote virConnectOpen function. The + name is normally formed by removing transport, hostname, port + number, username and extra parameters from the remote URI, but in certain + very complex cases it may be better to supply the name explicitly. +</td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>name=qemu:///system</code> </td> </tr> <tr> <td> - <code>NULL</code> + <code>tls_priority</code> </td> + <td> tls </td> <td> - <code>remote://oirase/</code> + A vaid GNUTLS priority string +</td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>tls_priority=NORMAL:-VERS-SSL3.0</code> </td> + </tr> + <tr> + <td> + <code>mode</code> + </td> + <td> unix, ssh, libssh, libssh2 </td> + <td> + <dl> + <dt><code>auto</code></dt><dd>automatically determine the daemon</dd> + <dt><code>direct</code></dt><dd>connect to per-driver daemons</dd> + <dt><code>legacy</code></dt><dd>connect to libvirtd</dd> + </dl> + Can also be set in <code>libvirt.conf</code> as <code>remote_mode</code> </td> - <td> Connect to the "default" hypervisor running on host <code>oirase</code> - using TLS. </td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>mode=direct</code> </td> </tr> <tr> <td> - <code>xen:///system</code> + <code>command</code> </td> + <td> ssh, ext </td> <td> - <code>xen+ssh://oirase/system</code> + The external command. For ext transport this is required. + For ssh the default is <code>ssh</code>. + The PATH is searched for the command. +</td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>command=/opt/openssh/bin/ssh</code> </td> + </tr> + <tr> + <td> + <code>socket</code> </td> - <td> Connect to the Xen hypervisor running on host <code>oirase</code> - by going over an <code>ssh</code> connection. </td> + <td> unix, ssh, libssh2, libssh </td> + <td> + The path to the Unix domain socket, which overrides the + compiled-in default. For ssh transport, this is passed to + the remote netcat command (see next). +</td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>socket=/opt/libvirt/run/libvirt/libvirt-sock</code> </td> </tr> <tr> <td> - <code>test:///default</code> + <code>netcat</code> </td> + <td> ssh, libssh2, libssh </td> <td> - <code>test+tcp://oirase/default</code> + The name of the netcat command on the remote machine. + The default is <code>nc</code>. For ssh transport, libvirt + constructs an ssh command which looks like: + +<pre><i>command</i> -p <i>port</i> [-l <i>username</i>] <i>hostname</i> <i>netcat</i> -U <i>socket</i> +</pre> + + where <i>port</i>, <i>username</i>, <i>hostname</i> can be + specified as part of the remote URI, and <i>command</i>, <i>netcat</i> + and <i>socket</i> come from extra parameters (or + sensible defaults). + +</td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>netcat=/opt/netcat/bin/nc</code> </td> + </tr> + + <tr> + <td> + <code>keyfile</code> + </td> + <td> ssh, libssh2, libssh </td> + <td> + The name of the private key file to use to authentication to the remote + machine. If this option is not used the default keys are used. + </td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>keyfile=/root/.ssh/example_key</code> </td> + </tr> + + <tr> + <td> + <code>no_verify</code> + </td> + <td> ssh, tls </td> + <td> + SSH: If set to a non-zero value, this disables client's strict host key + checking making it auto-accept new host keys. Existing host keys will + still be validated. + <br/> + <br/> + TLS: If set to a non-zero value, this disables client checks of the + server's certificate. Note that to disable server checks of + the client's certificate or IP address you must + <a href="#Remote_libvirtd_configuration">change the libvirtd + configuration</a>. +</td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>no_verify=1</code> </td> + </tr> + <tr> + <td> + <code>no_tty</code> + </td> + <td> ssh </td> + <td> + If set to a non-zero value, this stops ssh from asking for + a password if it cannot log in to the remote machine automatically + (eg. using ssh-agent etc.). Use this when you don't have access + to a terminal - for example in graphical programs which use libvirt. +</td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>no_tty=1</code> </td> + </tr> + <tr> + <td> + <code>pkipath</code> + </td> + <td> tls</td> + <td> + Specifies x509 certificates path for the client. If any of + the CA certificate, client certificate, or client key is + missing, the connection will fail with a fatal error. + </td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>pkipath=/tmp/pki/client</code> </td> + </tr> + <tr> + <td> + <code>known_hosts</code> + </td> + <td> libssh2, libssh </td> + <td> + Path to the known_hosts file to verify the host key against. LibSSH2 and + libssh support OpenSSH-style known_hosts files, although LibSSH2 does not + support all key types, so using files created by the OpenSSH binary may + result into truncating the known_hosts file. Thus, with LibSSH2 it's + recommended to use the default known_hosts file is located in libvirt's + client local configuration directory e.g.: ~/.config/libvirt/known_hosts. + Note: Use absolute paths. +</td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>known_hosts=/root/.ssh/known_hosts</code> </td> + </tr> + <tr> + <td> + <code>sshauth</code> </td> - <td> Connect to the test driver on host <code>oirase</code> - using an unsecured TCP connection. </td> + <td> libssh2, libssh </td> + <td> + A comma separated list of authentication methods to use. Default (is + "agent,privkey,password,keyboard-interactive". The order of the methods + is preserved. Some methods may require additional parameters. +</td> + </tr> + <tr> + <td colspan="2"/> + <td> Example: <code>sshauth=privkey,agent</code> </td> </tr> </table> - <p> -Remote URIs in libvirt offer a rich syntax and many features. -We refer you to <a href="remote.html#Remote_URI_reference">the libvirt -remote URI reference</a> and <a href="remote.html">full documentation -for libvirt remote support</a>. -</p> <h2> <a id="URI_test">test:///... Test URIs</a> </h2> -- 2.23.0

The rst2html tool is provided by python docutils, and as the name suggests, it converts RST documents into HTML. Basic rules are added for integrating RST docs into the website build process. This enables us to start writing docs on our website in RST format instead of HTML, without changing the rest of our website templating system away from XSLT yet. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/Makefile.am | 33 +++++++++++++++++++++++++++------ libvirt.spec.in | 2 ++ m4/virt-external-programs.m4 | 5 +++++ mingw-libvirt.spec.in | 1 + 4 files changed, 35 insertions(+), 6 deletions(-) diff --git a/docs/Makefile.am b/docs/Makefile.am index 2a104bc837..7a8adbde2b 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -173,14 +173,26 @@ gif = \ internals_html_in = \ $(patsubst $(srcdir)/%,%,$(wildcard $(srcdir)/internals/*.html.in)) -internals_html = $(internals_html_in:%.html.in=%.html) +kbase_rst = \ + $(patsubst $(srcdir)/%,%,$(wildcard $(srcdir)/kbase/*.rst)) +kbase_rst_html_in = \ + $(kbase_rst:%.rst=%.html.in) +internals_html = \ + $(internals_html_in:%.html.in=%.html) \ + $(internals_rst_html_in:%.html.in=%.html) internalsdir = $(HTML_DIR)/internals internals_DATA = $(internals_html) kbase_html_in = \ $(patsubst $(srcdir)/%,%,$(wildcard $(srcdir)/kbase/*.html.in)) -kbase_html = $(kbase_html_in:%.html.in=%.html) +kbase_rst = \ + $(patsubst $(srcdir)/%,%,$(wildcard $(srcdir)/kbase/*.rst)) +kbase_rst_html_in = \ + $(kbase_rst:%.rst=%.html.in) +kbase_html = \ + $(kbase_html_in:%.html.in=%.html) \ + $(kbase_rst_html_in:%.html.in=%.html) kbasedir = $(HTML_DIR)/kbase kbase_DATA = $(kbase_html) @@ -191,9 +203,14 @@ dot_html_generated_in = \ news.html.in dot_html_in = \ $(notdir $(wildcard $(srcdir)/*.html.in)) +dot_rst = \ + $(notdir $(wildcard $(srcdir)/*.rst)) +dot_rst_html_in = \ + $(dot_rst:%.rst=%.html) dot_html = \ $(dot_html_generated_in:%.html.in=%.html) \ - $(dot_html_in:%.html.in=%.html) + $(dot_html_in:%.html.in=%.html) \ + $(dot_rst_html_in:%.html.in=%.html) htmldir = $(HTML_DIR) html_DATA = $(css) $(png) $(gif) $(dot_html) @@ -222,11 +239,11 @@ EXTRA_DIST= \ apibuild.py genaclperms.pl \ site.xsl subsite.xsl newapi.xsl page.xsl \ wrapstring.xsl \ - $(dot_html_in) $(gif) $(apipng) \ + $(dot_html_in) $(dot_rst) $(gif) $(apipng) \ $(fig) $(png) $(css) \ $(javascript) $(logofiles) \ - $(internals_html_in) $(fonts) \ - $(kbase_html_in) \ + $(internals_html_in) $(internals_rst) $(fonts) \ + $(kbase_html_in) $(kbase_rst) \ aclperms.htmlinc \ hvsupport.pl \ $(schema_DATA) @@ -281,6 +298,10 @@ EXTRA_DIST += \ %.png: %.fig convert -rotate 90 $< $@ +%.html.in: %.rst + $(AM_V_GEN)$(MKDIR_P) `dirname $@` && \ + $(RST2HTML) $< > $@ + %.html.tmp: %.html.in site.xsl subsite.xsl page.xsl \ $(acl_generated) $(AM_V_GEN)name=`echo $@ | sed -e 's/.tmp//'`; \ diff --git a/libvirt.spec.in b/libvirt.spec.in index a6219da604..5c47dd3d7c 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -265,6 +265,8 @@ BuildRequires: automake BuildRequires: gettext-devel BuildRequires: libtool BuildRequires: /usr/bin/pod2man +# Replace with python3-docutils when we drop py2 support +BuildRequires: /usr/bin/rst2html %endif BuildRequires: gcc BuildRequires: git diff --git a/m4/virt-external-programs.m4 b/m4/virt-external-programs.m4 index 0f995998c3..ed634a4c73 100644 --- a/m4/virt-external-programs.m4 +++ b/m4/virt-external-programs.m4 @@ -33,6 +33,11 @@ AC_DEFUN([LIBVIRT_CHECK_EXTERNAL_PROGRAMS], [ then AC_MSG_ERROR("xsltproc is required to build libvirt") fi + AC_PATH_PROGS([RST2HTML], [rst2html rst2html.py rst2html-3], []) + if test -z "$RST2HTML" + then + AC_MSG_ERROR("rst2html is required to build libvirt") + fi AC_PATH_PROG([AUGPARSE], [augparse], [/usr/bin/augparse]) AC_PROG_MKDIR_P AC_PROG_LN_S diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in index c29f3eeed2..35f1abc13d 100644 --- a/mingw-libvirt.spec.in +++ b/mingw-libvirt.spec.in @@ -82,6 +82,7 @@ BuildRequires: automake BuildRequires: gettext-devel BuildRequires: libtool %endif +BuildRequires: python3-docutils BuildRequires: mingw32-libssh2 BuildRequires: mingw64-libssh2 -- 2.23.0

The HTML from rst2html doesn't have <h1> immediately under the <body> tag, instead there is at least one <div> in between. There are also many things added in the <head> section that we don't want to have copied over, since our templating system already adds suitable <head> elements. We only need to copy the <script> to make index.html work. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/page.xsl | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/page.xsl b/docs/page.xsl index 6f429ae087..70dfec6df6 100644 --- a/docs/page.xsl +++ b/docs/page.xsl @@ -97,9 +97,9 @@ <link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png"/> <link rel="manifest" href="/manifest.json"/> <meta name="theme-color" content="#ffffff"/> - <title>libvirt: <xsl:value-of select="html:html/html:body/html:h1"/></title> + <title>libvirt: <xsl:value-of select="html:html/html:body//html:h1"/></title> <meta name="description" content="libvirt, virtualization, virtualization API"/> - <xsl:apply-templates select="/html:html/html:head/*" mode="content"/> + <xsl:apply-templates select="/html:html/html:head/html:script" mode="content"/> <script type="text/javascript" src="{$href_base}js/main.js"> <xsl:comment>// forces non-empty element</xsl:comment> -- 2.23.0

The rst2html output generates the links for headings in a slightly different way than we do for docs written in HTML, so we must match another scenario when generating back links. rst2html will also use <h1> tags for both the document title and the first level of section titles, so we must expand the matching to allow for this too. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/libvirt.css | 1 + docs/page.xsl | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/libvirt.css b/docs/libvirt.css index e927a084a7..399404ca54 100644 --- a/docs/libvirt.css +++ b/docs/libvirt.css @@ -419,6 +419,7 @@ a.headerlink { visibility: hidden; } +h1:hover > a.headerlink, h2:hover > a.headerlink, h3:hover > a.headerlink, h4:hover > a.headerlink, diff --git a/docs/page.xsl b/docs/page.xsl index 70dfec6df6..f8f7ff8cf9 100644 --- a/docs/page.xsl +++ b/docs/page.xsl @@ -176,12 +176,15 @@ <xsl:apply-templates select="exsl:node-set($inchtml)/html:html/html:body/*" mode="content"/> </xsl:template> - <xsl:template match="html:h2 | html:h3 | html:h4 | html:h5 | html:h6" mode="content"> + <xsl:template match="html:h1 | html:h2 | html:h3 | html:h4 | html:h5 | html:h6" mode="content"> <xsl:element name="{name()}"> <xsl:apply-templates mode="copy" /> <xsl:if test="./html:a/@id"> <a class="headerlink" href="#{html:a/@id}" title="Permalink to this headline">¶</a> </xsl:if> + <xsl:if test="./html:a[@class='toc-backref']"> + <a class="headerlink" href="#{../@id}" title="Permalink to this headline">¶</a> + </xsl:if> </xsl:element> </xsl:template> -- 2.23.0

We currently only render pretty tables if they have the "top_table" class set. All of our tables set this, except for the ACL & migration doc tables, which should have set it, and the API reference which does not want it. Simplify life by rendering all tables in a pretty style and remove the need for the "top_table" class entirely. A small rule turns off the pretty style for the API reference where tables are a hack used to render enums with horizontal alignment. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/aclpolkit.html.in | 20 +++++------ docs/genaclperms.pl | 2 +- docs/libvirt.css | 75 ++++++++++++------------------------------ docs/migration.html.in | 2 +- docs/newapi.xsl | 4 +-- 5 files changed, 35 insertions(+), 68 deletions(-) diff --git a/docs/aclpolkit.html.in b/docs/aclpolkit.html.in index 68e6d399b2..4a8877d5e7 100644 --- a/docs/aclpolkit.html.in +++ b/docs/aclpolkit.html.in @@ -64,7 +64,7 @@ </p> <h3><a id="object_connect">virConnectPtr</a></h3> - <table class="acl"> + <table> <thead> <tr> <th>Attribute</th> @@ -80,7 +80,7 @@ </table> <h3><a id="object_domain">virDomainPtr</a></h3> - <table class="acl"> + <table> <thead> <tr> <th>Attribute</th> @@ -104,7 +104,7 @@ </table> <h3><a id="object_interface">virInterfacePtr</a></h3> - <table class="acl"> + <table> <thead> <tr> <th>Attribute</th> @@ -128,7 +128,7 @@ </table> <h3><a id="object_network">virNetworkPtr</a></h3> - <table class="acl"> + <table> <thead> <tr> <th>Attribute</th> @@ -152,7 +152,7 @@ </table> <h3><a id="object_node_device">virNodeDevicePtr</a></h3> - <table class="acl"> + <table> <thead> <tr> <th>Attribute</th> @@ -172,7 +172,7 @@ </table> <h3><a id="object_nwfilter">virNWFilterPtr</a></h3> - <table class="acl"> + <table> <thead> <tr> <th>Attribute</th> @@ -196,7 +196,7 @@ </table> <h3><a id="object_secret">virSecretPtr</a></h3> - <table class="acl"> + <table> <thead> <tr> <th>Attribute</th> @@ -232,7 +232,7 @@ </table> <h3><a id="object_storage_pool">virStoragePoolPtr</a></h3> - <table class="acl"> + <table> <thead> <tr> <th>Attribute</th> @@ -256,7 +256,7 @@ </table> <h3><a id="object_storage_vol">virStorageVolPtr</a></h3> - <table class="acl"> + <table> <thead> <tr> <th>Attribute</th> @@ -317,7 +317,7 @@ </p> <h3><a id="object_connect_driver">Connection Driver Name</a></h3> - <table class="acl"> + <table> <thead> <tr> <th>Connection Driver</th> diff --git a/docs/genaclperms.pl b/docs/genaclperms.pl index e20b4c11c3..0de2cfad4d 100755 --- a/docs/genaclperms.pl +++ b/docs/genaclperms.pl @@ -85,7 +85,7 @@ foreach my $object (sort { $a cmp $b } keys %perms) { my $olink = lc "object_" . $object; print <<EOF; <h3><a id="$olink">$class</a></h3> -<table class="acl"> +<table> <thead> <tr> <th>Permission</th> diff --git a/docs/libvirt.css b/docs/libvirt.css index 399404ca54..d2e1842b62 100644 --- a/docs/libvirt.css +++ b/docs/libvirt.css @@ -161,37 +161,37 @@ p.image { text-align: center; } -.top_table { +table { border-collapse: collapse; min-width: 60%; margin-left: auto; margin-right: auto; } -.top_table th { +table th { background: rgb(0, 95, 97); color: rgb(255, 255, 255); padding: 0.5em; } -.top_table th a { +table th a { color: inherit; text-decoration: inherit; } -.top_table td, .top_table th { +table td, table th { border: 1px solid rgb(60, 133, 124); } -.top_table td { +table td { padding: 4px; } -.top_table tr:hover td, .top_table col:hover td { +table tr:hover td, table col:hover td { background: #eeeeee; } -.top_table tr td:hover { +table tr td:hover { background: #c5dbd8; } @@ -289,42 +289,12 @@ img.diagram { margin-right: auto; } -table.data th, table.data td { - padding: 0.3em; -} - -table.data { - border-spacing: 0px; -} - -table.data thead th { - background: rgb(178,178,178); - text-align: center; -} - -table.data { - border: 1px solid black; - border-collapse: collapse; -} - -table.data thead tr th { - border: 1px solid black; -} - -table.data tr.head th { - border-left: 1px solid black; - border-right: 1px solid black; -} - -table.data tbody td { - background: rgb(240,240,240); -} -table.data tbody td.y { +table tbody td.y { background: rgb(220,255,220); text-align: center; } -table.data tbody td.n { +table tbody td.n { background: rgb(255,220,220); text-align: center; } @@ -377,6 +347,18 @@ table.data tbody td.n { text-decoration: none; } +.api table td,.api table th { + border: 0px; +} + +.api table tr:hover td, .api table col:hover td { + background: inherit; +} + +.api table tr td:hover { + background: inherit; +} + dl.variablelist > dt { display: block; float: left; @@ -392,21 +374,6 @@ dl.variablelist > dt:after { content: ": "; } -table.acl { - margin: 1em; - border-spacing: 0px; - border: 1px solid #ccc; -} - -table.acl tr, table.acl td { - padding: 0.3em; - border: 1px solid #ccc; -} - -table.acl thead { - background: #ddd; -} - div.description pre.code { border: 1px dashed grey; background-color: inherit; diff --git a/docs/migration.html.in b/docs/migration.html.in index 7c345b65b7..355f0e89af 100644 --- a/docs/migration.html.in +++ b/docs/migration.html.in @@ -257,7 +257,7 @@ combinations. </p> - <table class="data"> + <table> <thead> <tr class="head"> <th colspan="3">Before migration</th> diff --git a/docs/newapi.xsl b/docs/newapi.xsl index 670879dc48..0dc4f7ae52 100644 --- a/docs/newapi.xsl +++ b/docs/newapi.xsl @@ -43,7 +43,7 @@ <xsl:if test="count(exsl:node-set($acls)/api[@name=$api]/check) > 0"> <h5>Access control parameter checks</h5> - <table class="acl"> + <table> <thead> <tr> <th>Object</th> @@ -56,7 +56,7 @@ </xsl:if> <xsl:if test="count(exsl:node-set($acls)/api[@name=$api]/filter) > 0"> <h5>Access control return value filters</h5> - <table class="acl"> + <table> <thead> <tr> <th>Object</th> -- 2.23.0

Although <tt> is deprecated in HTML5, the rst2html command will still emit it, in preference to <code> tags, so we must style it too. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/generic.css | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/generic.css b/docs/generic.css index a6b2354df0..c4092abc2b 100644 --- a/docs/generic.css +++ b/docs/generic.css @@ -72,11 +72,11 @@ h6 { font-size: 0.8em; } -code, pre { +code, pre, tt { font-family: LibvirtOverpassMono; } -dd code, p code { +dd code, p code, tt { background-color: #eeeeee; } -- 2.23.0

Most importantly we document the required heading markup so that we get consistency across the docs. Also mention that docs should have a table of contents if they have headings & are likely longer than one page of text. The 3-space indent rule may sound wierd, but that's what python has recommended and thus what tools like pandoc emit. Rather than try to reindent things to 4-space, just accept this RST norm. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/docs.html.in | 3 +++ docs/styleguide.rst | 66 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) create mode 100644 docs/styleguide.rst diff --git a/docs/docs.html.in b/docs/docs.html.in index f2721964b5..f441769617 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -135,6 +135,9 @@ <dt><a href="hacking.html">Contributor guidelines</a></dt> <dd>General hacking guidelines for contributors</dd> + <dt><a href="styleguide.html">Docs style guide</a></dt> + <dd>Style guidelines for reStructuredText docs</dd> + <dt><a href="strategy.html">Project strategy</a></dt> <dd>Sets a vision for future direction & technical choices</dd> diff --git a/docs/styleguide.rst b/docs/styleguide.rst new file mode 100644 index 0000000000..71f29320cb --- /dev/null +++ b/docs/styleguide.rst @@ -0,0 +1,66 @@ +========================= +Documentation style guide +========================= + +.. contents:: + +The following documents some specific libvirt rules for writing docs in +reStructuredText + +Table of contents +================= + +Any document which uses headings and whose content is long enough to cause +scrolling when viewed in the browser must start with a table of contents. +This should be created using the default formatting: + +:: + + .. contents:: + + +Whitespace +========== + +Blocks should be indented with 3 spaces, and no tabs + + +Headings +======== + +RST allows headings to be created simply by underlining with any punctuation +characters. Optionally the text may be overlined to. + +For the sake of consistency, libvirt defines the following style requirement +which allows for 6 levels of headings + +:: + + ========= + Heading 1 + ========= + + + + Heading 2 + ========= + + + + Heading 3 + --------- + + + + Heading 4 + ~~~~~~~~~ + + + + Heading 5 + ......... + + + + Heading 6 + ^^^^^^^^^ -- 2.23.0

This is a semi-automated conversion. The first conversion is done using "pandoc -f html -t rst". The result is then editted manually to apply the desired heading markup, and fix a few things that pandoc gets wrong. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/kbase/domainstatecapture.html.in | 303 -------------------------- docs/kbase/domainstatecapture.rst | 255 ++++++++++++++++++++++ 2 files changed, 255 insertions(+), 303 deletions(-) delete mode 100644 docs/kbase/domainstatecapture.html.in create mode 100644 docs/kbase/domainstatecapture.rst diff --git a/docs/kbase/domainstatecapture.html.in b/docs/kbase/domainstatecapture.html.in deleted file mode 100644 index f8c7394785..0000000000 --- a/docs/kbase/domainstatecapture.html.in +++ /dev/null @@ -1,303 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE html> -<html xmlns="http://www.w3.org/1999/xhtml"> - <body> - - <h1>Domain state capture using Libvirt</h1> - - <ul id="toc"></ul> - - <p> - In order to aid application developers to choose which - operations best suit their needs, this page compares the - different means for capturing state related to a domain managed - by libvirt. - </p> - - <p> - The information here is primarily geared towards capturing the - state of an active domain. Capturing the state of an inactive - domain essentially amounts to copying the contents of guest - disks, followed by a fresh boot of the same domain configuration - with disks restored back to that saved state. - </p> - - <h2><a id="definitions">State capture trade-offs</a></h2> - - <p>One of the features made possible with virtual machines is live - migration -- transferring all state related to the guest from - one host to another with minimal interruption to the guest's - activity. In this case, state includes domain memory (including - register and device contents), and domain storage (whether the - guest's view of the disks are backed by local storage on the - host, or by the hypervisor accessing shared storage over a - network). A clever observer will then note that if all state is - available for live migration, then there is nothing stopping a - user from saving some or all of that state at a given point of - time in order to be able to later rewind guest execution back to - the state it previously had. The astute reader will also realize - that state capture at any level requires that the data must be - stored and managed by some mechanism. This processing might fit - in a single file, or more likely require a chain of related - files, and may require synchronization with third-party tools - built around managing the amount of data resulting from - capturing the state of multiple guests that each use multiple - disks. - </p> - - <p> - There are several libvirt APIs associated with capturing the - state of a guest, which can later be used to rewind that guest - to the conditions it was in earlier. The following is a list of - trade-offs and differences between the various facets that - affect capturing domain state for active domains: - </p> - - <dl> - <dt>Duration</dt> - <dd>Capturing state can be a lengthy process, so while the - captured state ideally represents an atomic point in time - corresponding to something the guest was actually executing, - capturing state tends to focus on minimizing guest downtime - while performing the rest of the state capture in parallel - with guest execution. Some interfaces require up-front - preparation (the state captured is not complete until the API - ends, which may be some time after the command was first - started), while other interfaces track the state when the - command was first issued, regardless of the time spent in - capturing the rest of the state. Also, time spent in state - capture may be longer than the time required for live - migration, when state must be duplicated rather than shared. - </dd> - - <dt>Amount of state</dt> - <dd>For an online guest, there is a choice between capturing the - guest's memory (all that is needed during live migration when - the storage is already shared between source and destination), - the guest's disk state (all that is needed if there are no - pending guest I/O transactions that would be lost without the - corresponding memory state), or both together. Reverting to - partial state may still be viable, but typically, booting from - captured disk state without corresponding memory is comparable - to rebooting a machine that had power cut before I/O could be - flushed. Guests may need to use proper journaling methods to - avoid problems when booting from partial state. - </dd> - - <dt>Quiescing of data</dt> - <dd>Even if a guest has no pending I/O, capturing disk state may - catch the guest at a time when the contents of the disk are - inconsistent. Cooperating with the guest to perform data - quiescing is an optional step to ensure that captured disk - state is fully consistent without requiring additional memory - state, rather than just crash-consistent. But guest - cooperation may also have time constraints, where the guest - can rightfully panic if there is too much downtime while I/O - is frozen. - </dd> - - <dt>Quantity of files</dt> - <dd>When capturing state, some approaches store all state within - the same file (internal), while others expand a chain of - related files that must be used together (external), for more - files that a management application must track. - </dd> - - <dt>Impact to guest definition</dt> - <dd>Capturing state may require temporary changes to the guest - definition, such as associating new files into the domain - definition. While state capture should never impact the - running guest, a change to the domain's active XML may have - impact on other host operations being performed on the domain. - </dd> - - <dt>Third-party integration</dt> - <dd>When capturing state, there are tradeoffs to how much of the - process must be done directly by the hypervisor, and how much - can be off-loaded to third-party software. Since capturing - state is not instantaneous, it is essential that any - third-party integration see consistent data even if the - running guest continues to modify that data after the point in - time of the capture.</dd> - - <dt>Full vs. incremental</dt> - <dd>When periodically repeating the action of state capture, it - is useful to minimize the amount of state that must be - captured by exploiting the relation to a previous capture, - such as focusing only on the portions of the disk that the - guest has modified in the meantime. Some approaches are able - to take advantage of checkpoints to provide an incremental - backup, while others are only capable of a full backup even if - that means re-capturing unchanged portions of the disk.</dd> - - <dt>Local vs. remote</dt> - <dd>Domains that completely use remote storage may only need - some mechanism to keep track of guest memory state while using - external means to manage storage. Still, hypervisor and guest - cooperation to ensure points in time when no I/O is in flight - across the network can be important for properly capturing - disk state.</dd> - - <dt>Network latency</dt> - <dd>Whether it's domain storage or saving domain state into - remote storage, network latency has an impact on snapshot - data. Having dedicated network capacity, bandwidth, or quality - of service levels may play a role, as well as planning for how - much of the backup process needs to be local.</dd> - </dl> - - <p> - An example of the various facets in action is migration of a - running guest. In order for the guest to be able to resume on - the destination at the same place it left off at the source, the - hypervisor has to get to a point where execution on the source - is stopped, the last remaining changes occurring since the - migration started are then transferred, and the guest is started - on the target. The management software thus must keep track of - the starting point and any changes since the starting - point. These last changes are often referred to as dirty page - tracking or dirty disk block bitmaps. At some point in time - during the migration, the management software must freeze the - source guest, transfer the dirty data, and then start the guest - on the target. This period of time must be minimal. To minimize - overall migration time, one is advised to use a dedicated - network connection with a high quality of service. Alternatively - saving the current state of the running guest can just be a - point in time type operation which doesn't require updating the - "last vestiges" of state prior to writing out the saved state - file. The state file is the point in time of whatever is current - and may contain incomplete data which if used to restart the - guest could cause confusion or problems because some operation - wasn't completed depending upon where in time the operation was - commenced. - </p> - - <h2><a id="apis">State capture APIs</a></h2> - <p>With those definitions, the following libvirt APIs related to - state capture have these properties:</p> - <dl> - <dt><a href="html/libvirt-libvirt-domain.html#virDomainManagedSave"><code>virDomainManagedSave</code></a></dt> - <dd>This API saves guest memory, with libvirt managing all of - the saved state, then stops the guest. While stopped, the - disks can be copied by a third party. However, since any - subsequent restart of the guest by libvirt API will restore - the memory state (which typically only works if the disk state - is unchanged in the meantime), and since it is not possible to - get at the memory state that libvirt is managing, this is not - viable as a means for rolling back to earlier saved states, - but is rather more suited to situations such as suspending a - guest prior to rebooting the host in order to resume the guest - when the host is back up. This API also has a drawback of - potentially long guest downtime, and therefore does not lend - itself well to live backups.</dd> - - <dt><a href="html/libvirt-libvirt-domain.html#virDomainSave"><code>virDomainSave</code></a></dt> - <dd>This API is similar to virDomainManagedSave(), but moves the - burden on managing the stored memory state to the user. As - such, the user can now couple saved state with copies of the - disks to perform a revert to an arbitrary earlier saved state. - However, changing who manages the memory state does not change - the drawback of potentially long guest downtime when capturing - state.</dd> - - <dt><a href="html/libvirt-libvirt-domain-snapshot.html#virDomainSnapshotCreateXML"><code>virDomainSnapshotCreateXML</code></a></dt> - <dd>This API wraps several approaches for capturing guest state, - with a general premise of creating a snapshot (where the - current guest resources are frozen in time and a new wrapper - layer is opened for tracking subsequent guest changes). It - can operate on both offline and running guests, can choose - whether to capture the state of memory, disk, or both when - used on a running guest, and can choose between internal and - external storage for captured state. However, it is geared - towards post-event captures (when capturing both memory and - disk state, the disk state is not captured until all memory - state has been collected first). Using QEMU as the - hypervisor, internal snapshots currently have lengthy downtime - that is incompatible with freezing guest I/O, but external - snapshots are quick when memory contents are not also saved. - Since creating an external snapshot changes which disk image - resource is in use by the guest, this API can be coupled - with <a href="html/libvirt-libvirt-domain.html#virDomainBlockCommit"><code>virDomainBlockCommit()</code></a> - to restore things back to the guest using its original disk - image, where a third-party tool can read the backing file - prior to the live commit. See also - the <a href="formatsnapshot.html">XML details</a> used with - this command.</dd> - - <dt><a href="html/libvirt-libvirt-domain.html#virDomainFSFreeze"><code>virDomainFSFreeze</code></a>, <a href="html/libvirt-libvirt-domain.html#virDomainFSThaw"><code>virDomainFSThaw</code></a></dt> - <dd>This pair of APIs does not directly capture guest state, but - can be used to coordinate with a trusted live guest that state - capture is about to happen, and therefore guest I/O should be - quiesced so that the state capture is fully consistent, rather - than merely crash consistent. Some APIs are able to - automatically perform a freeze and thaw via a flags parameter, - rather than having to make separate calls to these - functions. Also, note that freezing guest I/O is only possible - with trusted guests running a guest agent, and that some - guests place maximum time limits on how long I/O can be - frozen.</dd> - - <dt><a href="html/libvirt-libvirt-domain-checkpoint.html#virDomainCheckpointCreateXML"><code>virDomainCheckpointCreateXML</code></a></dt> - <dd>This API does not actually capture guest state, rather it - makes it possible to track which portions of guest disks have - changed between a checkpoint and the current live execution of - the guest. However, while it is possible use this API to - create checkpoints in isolation, it is more typical to create - a checkpoint as a side-effect of starting a new incremental - backup with <code>virDomainBackupBegin()</code> or at the - creation of an external snapshot - with <code>virDomainSnapshotCreateXML2()</code>, since a - second incremental backup is most useful when using the - checkpoint created during the first. See also - the <a href="formatcheckpoint.html">XML details</a> used with - this command.</dd> - - <dt><a href="html/libvirt-libvirt-domain.html#virDomainBackupBegin"><code>virDomainBackupBegin</code></a>, <a href="html/libvirt-libvirt-domain.html#virDomainBackupEnd"><code>virDomainBackupEnd</code></a></dt> - <dd>This API wraps approaches for capturing the state of disks - of a running guest, but does not track accompanying guest - memory state. The capture is consistent to the start of the - operation, where the captured state is stored independently - from the disk image in use with the guest and where it can be - easily integrated with a third-party for capturing the disk - state. Since the backup operation is stored externally from - the guest resources, there is no need to commit data back in - at the completion of the operation. When coupled with - checkpoints, this can be used to capture incremental backups - instead of full.</dd> - </dl> - - <h2><a id="examples">Examples</a></h2> - <p>The following two sequences both accomplish the task of - capturing the disk state of a running guest, then wrapping - things up so that the guest is still running with the same file - as its disk image as before the sequence of operations began. - The difference between the two sequences boils down to the - impact of an unexpected interruption made at any point in the - middle of the sequence: with such an interruption, the first - example leaves the guest tied to a temporary wrapper file rather - than the original disk, and requires manual clean up of the - domain definition; while the second example has no impact to the - domain definition.</p> - - <p>1. Backup via temporary snapshot - <pre> -virDomainFSFreeze() -virDomainSnapshotCreateXML(VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY) -virDomainFSThaw() -third-party copy the backing file to backup storage # most time spent here -virDomainBlockCommit(VIR_DOMAIN_BLOCK_COMMIT_ACTIVE) per disk -wait for commit ready event per disk -virDomainBlockJobAbort() per disk - </pre></p> - - <p>2. Direct backup - <pre> -virDomainFSFreeze() -virDomainBackupBegin() -virDomainFSThaw() -wait for push mode event, or pull data over NBD # most time spent here -virDomainBackupEnd() - </pre></p> - - </body> -</html> diff --git a/docs/kbase/domainstatecapture.rst b/docs/kbase/domainstatecapture.rst new file mode 100644 index 0000000000..cefc381d2e --- /dev/null +++ b/docs/kbase/domainstatecapture.rst @@ -0,0 +1,255 @@ +================================== +Domain state capture using Libvirt +================================== + +.. contents:: + +In order to aid application developers to choose which operations best +suit their needs, this page compares the different means for capturing +state related to a domain managed by libvirt. + +The information here is primarily geared towards capturing the state of +an active domain. Capturing the state of an inactive domain essentially +amounts to copying the contents of guest disks, followed by a fresh boot +of the same domain configuration with disks restored back to that saved +state. + +State capture trade-offs +======================== + +One of the features made possible with virtual machines is live +migration -- transferring all state related to the guest from one host +to another with minimal interruption to the guest's activity. In this +case, state includes domain memory (including register and device +contents), and domain storage (whether the guest's view of the disks are +backed by local storage on the host, or by the hypervisor accessing +shared storage over a network). A clever observer will then note that if +all state is available for live migration, then there is nothing +stopping a user from saving some or all of that state at a given point +of time in order to be able to later rewind guest execution back to the +state it previously had. The astute reader will also realize that state +capture at any level requires that the data must be stored and managed +by some mechanism. This processing might fit in a single file, or more +likely require a chain of related files, and may require synchronization +with third-party tools built around managing the amount of data +resulting from capturing the state of multiple guests that each use +multiple disks. + +There are several libvirt APIs associated with capturing the state of a +guest, which can later be used to rewind that guest to the conditions it +was in earlier. The following is a list of trade-offs and differences +between the various facets that affect capturing domain state for active +domains: + +Duration + Capturing state can be a lengthy process, so while the captured state + ideally represents an atomic point in time corresponding to something + the guest was actually executing, capturing state tends to focus on + minimizing guest downtime while performing the rest of the state + capture in parallel with guest execution. Some interfaces require + up-front preparation (the state captured is not complete until the + API ends, which may be some time after the command was first + started), while other interfaces track the state when the command was + first issued, regardless of the time spent in capturing the rest of + the state. Also, time spent in state capture may be longer than the + time required for live migration, when state must be duplicated + rather than shared. +Amount of state + For an online guest, there is a choice between capturing the guest's + memory (all that is needed during live migration when the storage is + already shared between source and destination), the guest's disk + state (all that is needed if there are no pending guest I/O + transactions that would be lost without the corresponding memory + state), or both together. Reverting to partial state may still be + viable, but typically, booting from captured disk state without + corresponding memory is comparable to rebooting a machine that had + power cut before I/O could be flushed. Guests may need to use proper + journaling methods to avoid problems when booting from partial state. +Quiescing of data + Even if a guest has no pending I/O, capturing disk state may catch + the guest at a time when the contents of the disk are inconsistent. + Cooperating with the guest to perform data quiescing is an optional + step to ensure that captured disk state is fully consistent without + requiring additional memory state, rather than just crash-consistent. + But guest cooperation may also have time constraints, where the guest + can rightfully panic if there is too much downtime while I/O is + frozen. +Quantity of files + When capturing state, some approaches store all state within the same + file (internal), while others expand a chain of related files that + must be used together (external), for more files that a management + application must track. +Impact to guest definition + Capturing state may require temporary changes to the guest + definition, such as associating new files into the domain definition. + While state capture should never impact the running guest, a change + to the domain's active XML may have impact on other host operations + being performed on the domain. +Third-party integration + When capturing state, there are tradeoffs to how much of the process + must be done directly by the hypervisor, and how much can be + off-loaded to third-party software. Since capturing state is not + instantaneous, it is essential that any third-party integration see + consistent data even if the running guest continues to modify that + data after the point in time of the capture. +Full vs. incremental + When periodically repeating the action of state capture, it is useful + to minimize the amount of state that must be captured by exploiting + the relation to a previous capture, such as focusing only on the + portions of the disk that the guest has modified in the meantime. + Some approaches are able to take advantage of checkpoints to provide + an incremental backup, while others are only capable of a full backup + even if that means re-capturing unchanged portions of the disk. +Local vs. remote + Domains that completely use remote storage may only need some + mechanism to keep track of guest memory state while using external + means to manage storage. Still, hypervisor and guest cooperation to + ensure points in time when no I/O is in flight across the network can + be important for properly capturing disk state. +Network latency + Whether it's domain storage or saving domain state into remote + storage, network latency has an impact on snapshot data. Having + dedicated network capacity, bandwidth, or quality of service levels + may play a role, as well as planning for how much of the backup + process needs to be local. + +An example of the various facets in action is migration of a running +guest. In order for the guest to be able to resume on the destination at +the same place it left off at the source, the hypervisor has to get to a +point where execution on the source is stopped, the last remaining +changes occurring since the migration started are then transferred, and +the guest is started on the target. The management software thus must +keep track of the starting point and any changes since the starting +point. These last changes are often referred to as dirty page tracking +or dirty disk block bitmaps. At some point in time during the migration, +the management software must freeze the source guest, transfer the dirty +data, and then start the guest on the target. This period of time must +be minimal. To minimize overall migration time, one is advised to use a +dedicated network connection with a high quality of service. +Alternatively saving the current state of the running guest can just be +a point in time type operation which doesn't require updating the "last +vestiges" of state prior to writing out the saved state file. The state +file is the point in time of whatever is current and may contain +incomplete data which if used to restart the guest could cause confusion +or problems because some operation wasn't completed depending upon where +in time the operation was commenced. + +State capture APIs +================== + +With those definitions, the following libvirt APIs related to state +capture have these properties: + +`virDomainManagedSave <../html/libvirt-libvirt-domain.html#virDomainManagedSave>`__ + This API saves guest memory, with libvirt managing all of the saved + state, then stops the guest. While stopped, the disks can be copied + by a third party. However, since any subsequent restart of the guest + by libvirt API will restore the memory state (which typically only + works if the disk state is unchanged in the meantime), and since it + is not possible to get at the memory state that libvirt is managing, + this is not viable as a means for rolling back to earlier saved + states, but is rather more suited to situations such as suspending a + guest prior to rebooting the host in order to resume the guest when + the host is back up. This API also has a drawback of potentially long + guest downtime, and therefore does not lend itself well to live + backups. +`virDomainSave <../html/libvirt-libvirt-domain.html#virDomainSave>`__ + This API is similar to virDomainManagedSave(), but moves the burden + on managing the stored memory state to the user. As such, the user + can now couple saved state with copies of the disks to perform a + revert to an arbitrary earlier saved state. However, changing who + manages the memory state does not change the drawback of potentially + long guest downtime when capturing state. +`virDomainSnapshotCreateXML <../html/libvirt-libvirt-domain-snapshot.html#virDomainSnapshotCreateXML>`__ + This API wraps several approaches for capturing guest state, with a + general premise of creating a snapshot (where the current guest + resources are frozen in time and a new wrapper layer is opened for + tracking subsequent guest changes). It can operate on both offline + and running guests, can choose whether to capture the state of + memory, disk, or both when used on a running guest, and can choose + between internal and external storage for captured state. However, it + is geared towards post-event captures (when capturing both memory and + disk state, the disk state is not captured until all memory state has + been collected first). Using QEMU as the hypervisor, internal + snapshots currently have lengthy downtime that is incompatible with + freezing guest I/O, but external snapshots are quick when memory + contents are not also saved. Since creating an external snapshot + changes which disk image resource is in use by the guest, this API + can be coupled with + `virDomainBlockCommit() <html/libvirt-libvirt-domain.html#virDomainBlockCommit>`__ + to restore things back to the guest using its original disk image, + where a third-party tool can read the backing file prior to the live + commit. See also the `XML details <formatsnapshot.html>`__ used with + this command. +`virDomainFSFreeze <html/libvirt-libvirt-domain.html#virDomainFSFreeze>`__, `virDomainFSThaw <html/libvirt-libvirt-domain.html#virDomainFSThaw>`__ + This pair of APIs does not directly capture guest state, but can be + used to coordinate with a trusted live guest that state capture is + about to happen, and therefore guest I/O should be quiesced so that + the state capture is fully consistent, rather than merely crash + consistent. Some APIs are able to automatically perform a freeze and + thaw via a flags parameter, rather than having to make separate calls + to these functions. Also, note that freezing guest I/O is only + possible with trusted guests running a guest agent, and that some + guests place maximum time limits on how long I/O can be frozen. +`virDomainCheckpointCreateXML <html/libvirt-libvirt-domain-checkpoint.html#virDomainCheckpointCreateXML>`__ + This API does not actually capture guest state, rather it makes it + possible to track which portions of guest disks have changed between + a checkpoint and the current live execution of the guest. However, + while it is possible use this API to create checkpoints in isolation, + it is more typical to create a checkpoint as a side-effect of + starting a new incremental backup with ``virDomainBackupBegin()`` or + at the creation of an external snapshot with + ``virDomainSnapshotCreateXML2()``, since a second incremental backup + is most useful when using the checkpoint created during the first. + See also the `XML details <formatcheckpoint.html>`__ used with this + command. +`virDomainBackupBegin <html/libvirt-libvirt-domain.html#virDomainBackupBegin>`__, `virDomainBackupEnd <html/libvirt-libvirt-domain.html#virDomainBackupEnd>`__ + This API wraps approaches for capturing the state of disks of a + running guest, but does not track accompanying guest memory state. + The capture is consistent to the start of the operation, where the + captured state is stored independently from the disk image in use + with the guest and where it can be easily integrated with a + third-party for capturing the disk state. Since the backup operation + is stored externally from the guest resources, there is no need to + commit data back in at the completion of the operation. When coupled + with checkpoints, this can be used to capture incremental backups + instead of full. + +Examples +======== + +The following two sequences both accomplish the task of capturing the +disk state of a running guest, then wrapping things up so that the guest +is still running with the same file as its disk image as before the +sequence of operations began. The difference between the two sequences +boils down to the impact of an unexpected interruption made at any point +in the middle of the sequence: with such an interruption, the first +example leaves the guest tied to a temporary wrapper file rather than +the original disk, and requires manual clean up of the domain +definition; while the second example has no impact to the domain +definition. + +Backup via temporary snapshot +----------------------------- + +:: + + virDomainFSFreeze() + virDomainSnapshotCreateXML(VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY) + virDomainFSThaw() + third-party copy the backing file to backup storage # most time spent here + virDomainBlockCommit(VIR_DOMAIN_BLOCK_COMMIT_ACTIVE) per disk + wait for commit ready event per disk + virDomainBlockJobAbort() per disk + + +Direct backup +------------- + +:: + + virDomainFSFreeze() + virDomainBackupBegin() + virDomainFSThaw() + wait for push mode event, or pull data over NBD # most time spent here + virDomainBackupEnd() -- 2.23.0

This is a semi-automated conversion. The first conversion is done using "pandoc -f html -t rst". The result is then editted manually to apply the desired heading markup, and fix a few things that pandoc gets wrong. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/kbase/launch_security_sev.html.in | 533 ------------------------- docs/kbase/launch_security_sev.rst | 529 ++++++++++++++++++++++++ 2 files changed, 529 insertions(+), 533 deletions(-) delete mode 100644 docs/kbase/launch_security_sev.html.in create mode 100644 docs/kbase/launch_security_sev.rst diff --git a/docs/kbase/launch_security_sev.html.in b/docs/kbase/launch_security_sev.html.in deleted file mode 100644 index 985c11a47b..0000000000 --- a/docs/kbase/launch_security_sev.html.in +++ /dev/null @@ -1,533 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE html> -<html xmlns="http://www.w3.org/1999/xhtml"> - <body> - <h1>Launch security with AMD SEV</h1> - - <ul id="toc"></ul> - - <p> - Storage encryption in modern public cloud computing is a common practice. - However, from the point of view of a user of these cloud workloads, a - significant amount of trust needs to be put in the cloud platform security as - well as integrity (was the hypervisor tampered?). For this reason there's ever - rising demand for securing data in use, i.e. memory encryption. - One of the solutions addressing this matter is AMD SEV. - </p> - - <h2>AMD SEV</h2> - <p> - SEV (Secure Encrypted Virtualization) is a feature extension of AMD's SME (Secure - Memory Encryption) intended for KVM virtual machines which is supported - primarily on AMD's EPYC CPU line. In contrast to SME, SEV uses a unique memory encryption - key for each VM. The whole encryption of memory pages is completely transparent - to the hypervisor and happens inside dedicated hardware in the on-die memory controller. - Each controller includes a high-performance Advanced Encryption Standard - (AES) engine that encrypts data when it is written to DRAM and decrypts it - when read. - - For more details about the technology itself, you can visit - <a href="https://developer.amd.com/sev/">AMD's developer portal</a>. - </p> - - <h2><a id="Host">Enabling SEV on the host</a></h2> - <p> - Before VMs can make use of the SEV feature you need to make sure your - AMD CPU does support SEV. You can check whether SEV is among the CPU - flags with: - </p> - - <pre> -$ cat /proc/cpuinfo | grep sev -... -sme ssbd sev ibpb</pre> - - <p> - Next step is to enable SEV in the kernel, because it is disabled by default. - This is done by putting the following onto the kernel command line: - </p> - - <pre> -mem_encrypt=on kvm_amd.sev=1 - </pre> - - <p> - To make the changes persistent, append the above to the variable holding - parameters of the kernel command line in - <code>/etc/default/grub</code> to preserve SEV settings across reboots - </p> - - <pre> -$ cat /etc/default/grub -... -GRUB_CMDLINE_LINUX="... mem_encrypt=on kvm_amd.sev=1" -$ grub2-mkconfig -o /boot/efi/EFI/<distro>/grub.cfg</pre> - - <p> - <code>mem_encrypt=on</code> turns on the SME memory encryption feature on - the host which protects against the physical attack on the hypervisor - memory. The <code>kvm_amd.sev</code> parameter actually enables SEV in - the kvm module. It can be set on the command line alongside - <code>mem_encrypt</code> like shown above, or it can be put into a - module config under <code>/etc/modprobe.d/</code> - </p> - - <pre> -$ cat /etc/modprobe.d/sev.conf -options kvm_amd sev=1 - </pre> - - <p> - After rebooting the host, you should see SEV being enabled in the kernel: - </p> - - <pre> -$ cat /sys/module/kvm_amd/parameters/sev -1 - </pre> - - <h2><a id="Virt">Checking SEV support in the virt stack</a></h2> - <p> - <b>Note: All of the commands bellow need to be run with root privileges.</b> - </p> - - <p> - First make sure you have the following packages in the specified versions: - </p> - - <ul> - <li> - libvirt >= 4.5.0 (>5.1.0 recommended due to additional SEV bugfixes) - </li> - <li> - QEMU >= 2.12.0 - </li> - </ul> - <p> - To confirm that the virtualization stack supports SEV, run the following: - </p> - - <pre> -# virsh domcapabilities -<domainCapabilities> -... - <features> - ... - <sev supported='yes'> - <cbitpos>47</cbitpos> - <reducedPhysBits>1</reducedPhysBits> - </sev> - ... - </features> -</domainCapabilities></pre> - <p> - Note that if libvirt was already installed and libvirtd running before enabling SEV in the kernel followed by the host reboot you need to force libvirtd - to re-probe both the host and QEMU capabilities. First stop libvirtd: - </p> - - <pre> -# systemctl stop libvirtd.service - </pre> - - <p> - Now you need to clean the capabilities cache: - </p> - - <pre> -# rm -f /var/cache/libvirt/qemu/capabilities/* - </pre> - - <p> - If you now restart libvirtd, it will re-probe the capabilities and if - you now run: - </p> - - <pre> -# virsh domcapabilities - </pre> - - <p> - SEV should be listed as supported. If you still see: - </p> - - <pre> -<sev supported='no'/> - </pre> - - <p> - it means one of two things: - <ol> - <li> - libvirt does support SEV, but either QEMU or the host does not - </li> - <li> - you have libvirt <=5.1.0 which suffered from getting a - <code>'Permission denied'</code> on <code>/dev/sev</code> because - of the default permissions on the character device which prevented - QEMU from opening it during capabilities probing - you can either - manually tweak the permissions so that QEMU has access to it or - preferably install libvirt 5.1.0 or higher - </li> - </ol> - </p> - - <h2><a id="Configuration">VM Configuration</a></h2> - <p> - SEV is enabled in the XML by specifying the - <a href="https://libvirt.org/formatdomain.html#launchSecurity"><launchSecurity> </a> element. However, specifying <code>launchSecurity</code> isn't - enough to boot an SEV VM. Further configuration requirements are discussed - below. - </p> - - <h3><a id="Machine">Machine type</a></h3> - <p> - Even though both Q35 and legacy PC machine types (for PC see also - "virtio") can be used with SEV, usage of the legacy PC machine type is - strongly discouraged, since depending on how your OVMF package was - built (e.g. including features like SecureBoot or SMM) Q35 may even be - required. - </p> - - <h5>Q35</h5> -<pre> -... -<os> - <type arch='x86_64' machine='pc-q35-3.0'>hvm</type> - ... -</os> -...</pre> - - <h5>i440fx (discouraged)</h5> - <pre> -... -<os> - <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type> - ... -</os> -... - </pre> - - <h3><a id="Boot">Boot loader</a></h3> - <p> - SEV is only going to work with OVMF (UEFI), so you'll need to point libvirt to - the correct OVMF binary. - </p> - <pre> -... -<os> - <type arch='x86_64' machine='pc-q35-3.0'>hvm</type> - <loader readonly='yes' type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.fd</loader> -</os> -...</pre> - - <h3><a id="Memory">Memory</a></h3> - <p> - Internally, SEV expects that the encrypted memory pages won't be swapped out or move - around so the VM memory needs to be pinned in physical RAM which will be - handled by QEMU. Apart from that, certain memory regions allocated by QEMU - itself (UEFI pflash, device ROMs, video RAM, etc.) have to be encrypted as - well. This causes a conflict in how libvirt tries to protect the host. - By default, libvirt enforces a memory hard limit on each VM's cgroup in order - to protect the host from malicious QEMU to allocate and lock all the available - memory. This limit corresponds to the total memory allocation for the VM given - by <code><currentMemory></code> element. However, trying to account for the additional - memory regions QEMU allocates when calculating the limit in an automated manner - is non-deterministic. One way to resolve this is to set the hard limit manually. - - <p> - Note: Figuring out the right number so that your guest boots and isn't killed is - challenging, but 256MiB extra memory over the total guest RAM should suffice for - most workloads and may serve as a good starting point. - - For example, a domain with 4GB memory with a 256MiB extra hard limit would look - like this: - </p> - </p> - - <pre> -# virsh edit <domain> -<domain> - ... - <currentMemory unit='KiB'>4194304</currentMemory> - <memtune> - <hard_limit unit='KiB'>4456448</hard_limit> - </memtune> - ... -</domain></pre> - <p> - There's another, preferred method of taking care of the limits by - using the<code><memoryBacking></code> element along with the - <code><locked/></code> subelement: - </p> - - <pre> -<domain> - ... - <memoryBacking> - <locked/> - </memoryBacking> - ... -</domain></pre> - - <p> - What that does is that it tells libvirt not to force any hard limit (well, - unlimited) upon the VM cgroup. The obvious advantage is that one doesn't need - to determine the hard limit for every single SEV-enabled VM. However, there is - a significant security-related drawback to this approach. Since no hard limit - is applied, a malicious QEMU could perform a DoS attack by locking all of the - host's available memory. The way to avoid this issue and to protect the host is - to enforce a bigger hard limit on the master cgroup containing all of the VMs - - on systemd this is <code>machine.slice</code>. - </p> - - <pre> -# systemctl set-property machine.slice MemoryHigh=<value></pre> - - <p> - To put even stricter measures in place which would involve the OOM killer, use - <pre> -# systemctl set-property machine.slice MemoryMax=<value></pre> - instead. Alternatively, you can create a systemd config (don't forget - to reload systemd configuration in this case): - <pre> -# cat << EOF > /etc/systemd/system.control/machine.slice.d/90-MemoryMax.conf -MemoryMax=<value> -EOF</pre> - The trade-off to keep in mind with the second approach is that the VMs - can still perform DoS on each other. - </p> - - <h3><a id="Virtio">Virtio</a></h3> - <p> - In order to make virtio devices work, we need to enable emulated IOMMU - on the devices so that virtual DMA can work. - </p> - - <pre> -# virsh edit <domain> -<domain> - ... - <controller type='virtio-serial' index='0'> - <driver iommu='on'/> - </controller> - <controller type='scsi' index='0' model='virtio-scsi'> - <driver iommu='on'/> - </controller> - ... - <memballoon model='virtio'> - <driver iommu='on'/> - </memballoon> - <rng model='virtio'> - <backend model='random'>/dev/urandom</backend> - <driver iommu='on'/> - </rng> - ... -<domain></pre> - - <p> - If you for some reason want to use the legacy PC machine type, further changes - to the virtio - configuration is required, because SEV will not work with Virtio <1.0. In - libvirt, this is handled by using the virtio-non-transitional device model - (libvirt >= 5.2.0 required). - - <p> - Note: some devices like video devices don't - support non-transitional model, which means that virtio GPU cannot be used. - </p> - </p> - - <pre> -<domain> - ... - <devices> - ... - <memballoon model='virtio-non-transitional'> - <driver iommu='on'/> - </memballoon> - </devices> - ... -</domain></pre> - - <h2><a id="Guest">Checking SEV from within the guest</a></h2> - <p> - After making the necessary adjustments discussed in - <a href="#Configuration">Configuration</a>, the VM should now boot - successfully with SEV enabled. You can then verify that the guest has - SEV enabled by running: - </p> - - <pre> -# dmesg | grep -i sev -AMD Secure Encrypted Virtualization (SEV) active</pre> - - <h2><a id="Limitations">Limitations</a></h2> - <p> - Currently, the boot disk cannot be of type virtio-blk, instead, virtio-scsi - needs to be used if virtio is desired. This limitation is expected to be lifted - with future releases of kernel (the kernel used at the time of writing the - article is 5.0.14). - If you still cannot start an SEV VM, it could be because of wrong SELinux label on the <code>/dev/sev</code> device with selinux-policy <3.14.2.40 which prevents QEMU from touching the device. This can be resolved by upgrading the package, tuning the selinux policy rules manually to allow svirt_t to access the device (see <code>audit2allow</code> on how to do that) or putting SELinux into permissive mode (discouraged). - </p> - - <h2><a id="Examples">Full domain XML examples</a></h2> - - <h5>Q35 machine</h5> - <pre> -<domain type='kvm'> - <name>sev-dummy</name> - <memory unit='KiB'>4194304</memory> - <currentMemory unit='KiB'>4194304</currentMemory> - <memoryBacking> - <locked/> - </memoryBacking> - <vcpu placement='static'>4</vcpu> - <os> - <type arch='x86_64' machine='pc-q35-3.0'>hvm</type> - <loader readonly='yes' type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.fd</loader> - <nvram>/var/lib/libvirt/qemu/nvram/sev-dummy_VARS.fd</nvram> - </os> - <features> - <acpi/> - <apic/> - <vmport state='off'/> - </features> - <cpu mode='host-model' check='partial'> - <model fallback='allow'/> - </cpu> - <clock offset='utc'> - <timer name='rtc' tickpolicy='catchup'/> - <timer name='pit' tickpolicy='delay'/> - <timer name='hpet' present='no'/> - </clock> - <on_poweroff>destroy</on_poweroff> - <on_reboot>restart</on_reboot> - <on_crash>destroy</on_crash> - <pm> - <suspend-to-mem enabled='no'/> - <suspend-to-disk enabled='no'/> - </pm> - <devices> - <emulator>/usr/bin/qemu-kvm</emulator> - <disk type='file' device='disk'> - <driver name='qemu' type='qcow2'/> - <source file='/var/lib/libvirt/images/sev-dummy.qcow2'/> - <target dev='sda' bus='scsi'/> - <boot order='1'/> - </disk> - <controller type='virtio-serial' index='0'> - <driver iommu='on'/> - </controller> - <controller type='scsi' index='0' model='virtio-scsi'> - <driver iommu='on'/> - </controller> - <interface type='network'> - <mac address='52:54:00:cc:56:90'/> - <source network='default'/> - <model type='virtio'/> - <driver iommu='on'/> - </interface> - <graphics type='spice' autoport='yes'> - <listen type='address'/> - <gl enable='no'/> - </graphics> - <video> - <model type='qxl'/> - </video> - <memballoon model='virtio'> - <driver iommu='on'/> - </memballoon> - <rng model='virtio'> - <driver iommu='on'/> - </rng> - </devices> - <launchSecurity type='sev'> - <cbitpos>47</cbitpos> - <reducedPhysBits>1</reducedPhysBits> - <policy>0x0003</policy> - </launchSecurity> -</domain></pre> - - <h5>PC-i440fx machine:</h5> - <pre> -<domain type='kvm'> - <name>sev-dummy-legacy</name> - <memory unit='KiB'>4194304</memory> - <currentMemory unit='KiB'>4194304</currentMemory> - <memtune> - <hard_limit unit='KiB'>5242880</hard_limit> - </memtune> - <vcpu placement='static'>4</vcpu> - <os> - <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type> - <loader readonly='yes' type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.fd</loader> - <nvram>/var/lib/libvirt/qemu/nvram/sev-dummy_VARS.fd</nvram> - <boot dev='hd'/> - </os> - <features> - <acpi/> - <apic/> - <vmport state='off'/> - </features> - <cpu mode='host-model' check='partial'> - <model fallback='allow'/> - </cpu> - <clock offset='utc'> - <timer name='rtc' tickpolicy='catchup'/> - <timer name='pit' tickpolicy='delay'/> - <timer name='hpet' present='no'/> - </clock> - <on_poweroff>destroy</on_poweroff> - <on_reboot>restart</on_reboot> - <on_crash>destroy</on_crash> - <pm> - <suspend-to-mem enabled='no'/> - <suspend-to-disk enabled='no'/> - </pm> - <devices> - <emulator>/usr/bin/qemu-kvm</emulator> - <disk type='file' device='disk'> - <driver name='qemu' type='qcow2'/> - <source file='/var/lib/libvirt/images/sev-dummy-seabios.qcow2'/> - <target dev='sda' bus='sata'/> - </disk> - <interface type='network'> - <mac address='52:54:00:d8:96:c8'/> - <source network='default'/> - <model type='virtio-non-transitional'/> - </interface> - <serial type='pty'> - <target type='isa-serial' port='0'> - <model name='isa-serial'/> - </target> - </serial> - <console type='pty'> - <target type='serial' port='0'/> - </console> - <input type='tablet' bus='usb'> - <address type='usb' bus='0' port='1'/> - </input> - <input type='mouse' bus='ps2'/> - <input type='keyboard' bus='ps2'/> - <graphics type='spice' autoport='yes'> - <listen type='address'/> - <gl enable='no'/> - </graphics> - <video> - <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> - </video> - <memballoon model='virtio-non-transitional'> - <driver iommu='on'/> - </memballoon> - <rng model='virtio-non-transitional'> - <driver iommu='on'/> - </rng> - </devices> - <launchSecurity type='sev'> - <cbitpos>47</cbitpos> - <reducedPhysBits>1</reducedPhysBits> - <policy>0x0003</policy> - </launchSecurity> -</domain></pre> - </body> -</html> diff --git a/docs/kbase/launch_security_sev.rst b/docs/kbase/launch_security_sev.rst new file mode 100644 index 0000000000..4387ae64b0 --- /dev/null +++ b/docs/kbase/launch_security_sev.rst @@ -0,0 +1,529 @@ +============================ +Launch security with AMD SEV +============================ + +.. contents:: + +Storage encryption in modern public cloud computing is a common +practice. However, from the point of view of a user of these cloud +workloads, a significant amount of trust needs to be put in the cloud +platform security as well as integrity (was the hypervisor tampered?). +For this reason there's ever rising demand for securing data in use, +i.e. memory encryption. One of the solutions addressing this matter is +AMD SEV. + +AMD SEV +======= + +SEV (Secure Encrypted Virtualization) is a feature extension of AMD's +SME (Secure Memory Encryption) intended for KVM virtual machines which +is supported primarily on AMD's EPYC CPU line. In contrast to SME, SEV +uses a unique memory encryption key for each VM. The whole encryption of +memory pages is completely transparent to the hypervisor and happens +inside dedicated hardware in the on-die memory controller. Each +controller includes a high-performance Advanced Encryption Standard +(AES) engine that encrypts data when it is written to DRAM and decrypts +it when read. For more details about the technology itself, you can +visit `AMD's developer portal <https://developer.amd.com/sev/>`__. + +Enabling SEV on the host +======================== + +Before VMs can make use of the SEV feature you need to make sure your +AMD CPU does support SEV. You can check whether SEV is among the CPU +flags with: + +:: + + $ cat /proc/cpuinfo | grep sev + ... + sme ssbd sev ibpb + +Next step is to enable SEV in the kernel, because it is disabled by +default. This is done by putting the following onto the kernel command +line: + +:: + + mem_encrypt=on kvm_amd.sev=1 + +To make the changes persistent, append the above to the variable holding +parameters of the kernel command line in ``/etc/default/grub`` to +preserve SEV settings across reboots + +:: + + $ cat /etc/default/grub + ... + GRUB_CMDLINE_LINUX="... mem_encrypt=on kvm_amd.sev=1" + $ grub2-mkconfig -o /boot/efi/EFI/<distro>/grub.cfg + +``mem_encrypt=on`` turns on the SME memory encryption feature on the +host which protects against the physical attack on the hypervisor +memory. The ``kvm_amd.sev`` parameter actually enables SEV in the kvm +module. It can be set on the command line alongside ``mem_encrypt`` like +shown above, or it can be put into a module config under +``/etc/modprobe.d/`` + +:: + + $ cat /etc/modprobe.d/sev.conf + options kvm_amd sev=1 + +After rebooting the host, you should see SEV being enabled in the +kernel: + +:: + + $ cat /sys/module/kvm_amd/parameters/sev + 1 + + +Checking SEV support in the virt stack +====================================== + +**Note: All of the commands bellow need to be run with root +privileges.** + +First make sure you have the following packages in the specified +versions: + +- libvirt >= 4.5.0 (>5.1.0 recommended due to additional SEV bugfixes) +- QEMU >= 2.12.0 + +To confirm that the virtualization stack supports SEV, run the +following: + +:: + + # virsh domcapabilities + <domainCapabilities> + ... + <features> + ... + <sev supported='yes'> + <cbitpos>47</cbitpos> + <reducedPhysBits>1</reducedPhysBits> + </sev> + ... + </features> + </domainCapabilities> + +Note that if libvirt was already installed and libvirtd running before +enabling SEV in the kernel followed by the host reboot you need to force +libvirtd to re-probe both the host and QEMU capabilities. First stop +libvirtd: + +:: + + # systemctl stop libvirtd.service + +Now you need to clean the capabilities cache: + +:: + + # rm -f /var/cache/libvirt/qemu/capabilities/* + +If you now restart libvirtd, it will re-probe the capabilities and if +you now run: + +:: + + # virsh domcapabilities + +SEV should be listed as supported. If you still see: + +:: + + <sev supported='no'/> + +it means one of two things: + +#. libvirt does support SEV, but either QEMU or the host does not +#. you have libvirt <=5.1.0 which suffered from getting a + ``'Permission denied'`` on ``/dev/sev`` because of the default + permissions on the character device which prevented QEMU from opening + it during capabilities probing - you can either manually tweak the + permissions so that QEMU has access to it or preferably install + libvirt 5.1.0 or higher + +VM Configuration +================ + +SEV is enabled in the XML by specifying the +`<launchSecurity> <https://libvirt.org/formatdomain.html#launchSecurity>`__ +element. However, specifying ``launchSecurity`` isn't enough to boot an +SEV VM. Further configuration requirements are discussed below. + +Machine type +------------ + +Even though both Q35 and legacy PC machine types (for PC see also +"virtio") can be used with SEV, usage of the legacy PC machine type is +strongly discouraged, since depending on how your OVMF package was built +(e.g. including features like SecureBoot or SMM) Q35 may even be +required. + +Q35 +~~~ + +:: + + ... + <os> + <type arch='x86_64' machine='pc-q35-3.0'>hvm</type> + ... + </os> + ... + +i440fx (discouraged) +~~~~~~~~~~~~~~~~~~~~ + +:: + + ... + <os> + <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type> + ... + </os> + ... + +Boot loader +----------- + +SEV is only going to work with OVMF (UEFI), so you'll need to point +libvirt to the correct OVMF binary. + +:: + + ... + <os> + <type arch='x86_64' machine='pc-q35-3.0'>hvm</type> + <loader readonly='yes' type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.fd</loader> + </os> + ... + +Memory +------ + +Internally, SEV expects that the encrypted memory pages won't be swapped +out or move around so the VM memory needs to be pinned in physical RAM +which will be handled by QEMU. Apart from that, certain memory regions +allocated by QEMU itself (UEFI pflash, device ROMs, video RAM, etc.) +have to be encrypted as well. This causes a conflict in how libvirt +tries to protect the host. By default, libvirt enforces a memory hard +limit on each VM's cgroup in order to protect the host from malicious +QEMU to allocate and lock all the available memory. This limit +corresponds to the total memory allocation for the VM given by +``<currentMemory>`` element. However, trying to account for the +additional memory regions QEMU allocates when calculating the limit in +an automated manner is non-deterministic. One way to resolve this is to +set the hard limit manually. + +Note: Figuring out the right number so that your guest boots and isn't +killed is challenging, but 256MiB extra memory over the total guest RAM +should suffice for most workloads and may serve as a good starting +point. For example, a domain with 4GB memory with a 256MiB extra hard +limit would look like this: + +:: + + # virsh edit <domain> + <domain> + ... + <currentMemory unit='KiB'>4194304</currentMemory> + <memtune> + <hard_limit unit='KiB'>4456448</hard_limit> + </memtune> + ... + </domain> + +There's another, preferred method of taking care of the limits by using +the\ ``<memoryBacking>`` element along with the ``<locked/>`` +subelement: + +:: + + <domain> + ... + <memoryBacking> + <locked/> + </memoryBacking> + ... + </domain> + +What that does is that it tells libvirt not to force any hard limit +(well, unlimited) upon the VM cgroup. The obvious advantage is that one +doesn't need to determine the hard limit for every single SEV-enabled +VM. However, there is a significant security-related drawback to this +approach. Since no hard limit is applied, a malicious QEMU could perform +a DoS attack by locking all of the host's available memory. The way to +avoid this issue and to protect the host is to enforce a bigger hard +limit on the master cgroup containing all of the VMs - on systemd this +is ``machine.slice``. + +:: + + # systemctl set-property machine.slice MemoryHigh=<value> + +To put even stricter measures in place which would involve the OOM +killer, use + +:: + + # systemctl set-property machine.slice MemoryMax=<value> + +instead. Alternatively, you can create a systemd config (don't forget to +reload systemd configuration in this case): + +:: + + # cat << EOF > /etc/systemd/system.control/machine.slice.d/90-MemoryMax.conf + MemoryMax=<value> + EOF + +The trade-off to keep in mind with the second approach is that the VMs +can still perform DoS on each other. + +Virtio +------ + +In order to make virtio devices work, we need to enable emulated IOMMU +on the devices so that virtual DMA can work. + +:: + + # virsh edit <domain> + <domain> + ... + <controller type='virtio-serial' index='0'> + <driver iommu='on'/> + </controller> + <controller type='scsi' index='0' model='virtio-scsi'> + <driver iommu='on'/> + </controller> + ... + <memballoon model='virtio'> + <driver iommu='on'/> + </memballoon> + <rng model='virtio'> + <backend model='random'>/dev/urandom</backend> + <driver iommu='on'/> + </rng> + ... + <domain> + +If you for some reason want to use the legacy PC machine type, further +changes to the virtio configuration is required, because SEV will not +work with Virtio <1.0. In libvirt, this is handled by using the +virtio-non-transitional device model (libvirt >= 5.2.0 required). + +Note: some devices like video devices don't support non-transitional +model, which means that virtio GPU cannot be used. + +:: + + <domain> + ... + <devices> + ... + <memballoon model='virtio-non-transitional'> + <driver iommu='on'/> + </memballoon> + </devices> + ... + </domain> + +Checking SEV from within the guest +================================== + +After making the necessary adjustments discussed in +`Configuration <#Configuration>`__, the VM should now boot successfully +with SEV enabled. You can then verify that the guest has SEV enabled by +running: + +:: + + # dmesg | grep -i sev + AMD Secure Encrypted Virtualization (SEV) active + +Limitations +=========== + +Currently, the boot disk cannot be of type virtio-blk, instead, +virtio-scsi needs to be used if virtio is desired. This limitation is +expected to be lifted with future releases of kernel (the kernel used at +the time of writing the article is 5.0.14). If you still cannot start an +SEV VM, it could be because of wrong SELinux label on the ``/dev/sev`` +device with selinux-policy <3.14.2.40 which prevents QEMU from touching +the device. This can be resolved by upgrading the package, tuning the +selinux policy rules manually to allow svirt_t to access the device (see +``audit2allow`` on how to do that) or putting SELinux into permissive +mode (discouraged). + +Full domain XML examples +======================== + +Q35 machine +----------- + +:: + + <domain type='kvm'> + <name>sev-dummy</name> + <memory unit='KiB'>4194304</memory> + <currentMemory unit='KiB'>4194304</currentMemory> + <memoryBacking> + <locked/> + </memoryBacking> + <vcpu placement='static'>4</vcpu> + <os> + <type arch='x86_64' machine='pc-q35-3.0'>hvm</type> + <loader readonly='yes' type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.fd</loader> + <nvram>/var/lib/libvirt/qemu/nvram/sev-dummy_VARS.fd</nvram> + </os> + <features> + <acpi/> + <apic/> + <vmport state='off'/> + </features> + <cpu mode='host-model' check='partial'> + <model fallback='allow'/> + </cpu> + <clock offset='utc'> + <timer name='rtc' tickpolicy='catchup'/> + <timer name='pit' tickpolicy='delay'/> + <timer name='hpet' present='no'/> + </clock> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <pm> + <suspend-to-mem enabled='no'/> + <suspend-to-disk enabled='no'/> + </pm> + <devices> + <emulator>/usr/bin/qemu-kvm</emulator> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2'/> + <source file='/var/lib/libvirt/images/sev-dummy.qcow2'/> + <target dev='sda' bus='scsi'/> + <boot order='1'/> + </disk> + <controller type='virtio-serial' index='0'> + <driver iommu='on'/> + </controller> + <controller type='scsi' index='0' model='virtio-scsi'> + <driver iommu='on'/> + </controller> + <interface type='network'> + <mac address='52:54:00:cc:56:90'/> + <source network='default'/> + <model type='virtio'/> + <driver iommu='on'/> + </interface> + <graphics type='spice' autoport='yes'> + <listen type='address'/> + <gl enable='no'/> + </graphics> + <video> + <model type='qxl'/> + </video> + <memballoon model='virtio'> + <driver iommu='on'/> + </memballoon> + <rng model='virtio'> + <driver iommu='on'/> + </rng> + </devices> + <launchSecurity type='sev'> + <cbitpos>47</cbitpos> + <reducedPhysBits>1</reducedPhysBits> + <policy>0x0003</policy> + </launchSecurity> + </domain> + +PC-i440fx machine +----------------- + +:: + + <domain type='kvm'> + <name>sev-dummy-legacy</name> + <memory unit='KiB'>4194304</memory> + <currentMemory unit='KiB'>4194304</currentMemory> + <memtune> + <hard_limit unit='KiB'>5242880</hard_limit> + </memtune> + <vcpu placement='static'>4</vcpu> + <os> + <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type> + <loader readonly='yes' type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.fd</loader> + <nvram>/var/lib/libvirt/qemu/nvram/sev-dummy_VARS.fd</nvram> + <boot dev='hd'/> + </os> + <features> + <acpi/> + <apic/> + <vmport state='off'/> + </features> + <cpu mode='host-model' check='partial'> + <model fallback='allow'/> + </cpu> + <clock offset='utc'> + <timer name='rtc' tickpolicy='catchup'/> + <timer name='pit' tickpolicy='delay'/> + <timer name='hpet' present='no'/> + </clock> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <pm> + <suspend-to-mem enabled='no'/> + <suspend-to-disk enabled='no'/> + </pm> + <devices> + <emulator>/usr/bin/qemu-kvm</emulator> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2'/> + <source file='/var/lib/libvirt/images/sev-dummy-seabios.qcow2'/> + <target dev='sda' bus='sata'/> + </disk> + <interface type='network'> + <mac address='52:54:00:d8:96:c8'/> + <source network='default'/> + <model type='virtio-non-transitional'/> + </interface> + <serial type='pty'> + <target type='isa-serial' port='0'> + <model name='isa-serial'/> + </target> + </serial> + <console type='pty'> + <target type='serial' port='0'/> + </console> + <input type='tablet' bus='usb'> + <address type='usb' bus='0' port='1'/> + </input> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <graphics type='spice' autoport='yes'> + <listen type='address'/> + <gl enable='no'/> + </graphics> + <video> + <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> + </video> + <memballoon model='virtio-non-transitional'> + <driver iommu='on'/> + </memballoon> + <rng model='virtio-non-transitional'> + <driver iommu='on'/> + </rng> + </devices> + <launchSecurity type='sev'> + <cbitpos>47</cbitpos> + <reducedPhysBits>1</reducedPhysBits> + <policy>0x0003</policy> + </launchSecurity> + </domain> -- 2.23.0

This is a semi-automated conversion. The first conversion is done using "pandoc -f html -t rst". The result is then editted manually to apply the desired heading markup, and fix a few things that pandoc gets wrong. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/kbase/secureusage.html.in | 171 --------------------------------- docs/kbase/secureusage.rst | 131 +++++++++++++++++++++++++ 2 files changed, 131 insertions(+), 171 deletions(-) delete mode 100644 docs/kbase/secureusage.html.in create mode 100644 docs/kbase/secureusage.rst diff --git a/docs/kbase/secureusage.html.in b/docs/kbase/secureusage.html.in deleted file mode 100644 index c60187fab3..0000000000 --- a/docs/kbase/secureusage.html.in +++ /dev/null @@ -1,171 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE html> -<html xmlns="http://www.w3.org/1999/xhtml"> - <body> - - <h1>Secure Usage of Libvirt</h1> - - <ul id="toc"></ul> - - <p> - This page details information that application developers and - administrators of libvirt should be aware of when working with - libvirt, that may have a bearing on security of the system. - </p> - - - <h2><a id="diskimage">Disk image handling</a></h2> - - <h3><a id="diskimageformat">Disk image format probing</a></h3> - - <p> - Historically there have been multiple flaws in QEMU and most - projects using QEMU, related to handling of disk formats. - The problems occur when a guest is given a virtual disk backed - by raw disk format on the host. If the management application - on the host tries to auto-detect / probe the disk format, it - is vulnerable to a malicious guest which can write a qcow2 - file header into its raw disk. If the management application - subsequently probes the disk, it will see it as a 'qcow2' disk - instead of a 'raw' disk. Since 'qcow2' disks can have a copy - on write backing file, such flaw can be leveraged to read - arbitrary files on the host. The same type of flaw may occur - if the management application allows users to upload pre-created - raw images. - </p> - - <p> - <strong>Recommendation:</strong> never attempt to automatically - detect the format of a disk image based on file contents which - are accessible to / originate from an untrusted source. - </p> - - <h3><a id="diskimagebacking">Disk image backing files</a></h3> - - <p> - If a management application allows users to upload pre-created - disk images in non-raw formats, it can be tricked into giving - the user access to arbitrary host files via the copy-on-write - backing file feature. This is because the qcow2 disk format - header contains a filename field which can point to any location. - It can also point to network protocols such as NBD, HTTP, GlusterFS, - RBD and more. This could allow for compromise of almost arbitrary - data accessible on the LAN/WAN. - </p> - - <p> - <strong>Recommendation:</strong> always validate that a disk - image originating from an untrusted source has no backing - file set. If a backing file is seen, reject the image. - </p> - - <h3><a id="diskimagesize">Disk image size validation</a></h3> - - <p> - If an application allows users to upload pre-created disk - images in non-raw formats, it is essential to validate the - logical disk image size, rather than the physical disk - image size. Non-raw disk images have a grow-on-demand - capability, so a user can provide a qcow2 image that may - be only 1 MB in size, but is configured to grow to many - TB in size. - </p> - - <p> - <strong>Recommendation:</strong> if receiving a non-raw disk - image from an untrusted source, validate the logical image - size stored in the disk image metadata against some finite - limit. - </p> - - <h3><a id="diskimageaccess">Disk image data access</a></h3> - - <p> - If an untrusted disk image is ever mounted on the host OS by - a management application or administrator, this opens an - avenue of attack with which to potentially compromise the - host kernel. Filesystem drivers in OS kernels are often very - complex code and thus may have bugs lurking in them. With - Linux, there are a large number of filesystem drivers, many - of which attract little security analysis attention. Linux - will helpfully probe filesystem formats if not told to use an - explicit format, allowing an attacker the ability to target - specific weak filesystem drivers. Even commonly used and - widely audited filesystems such as <code>ext4</code> have had - <a href="https://lwn.net/Articles/538898/">bugs lurking in them</a> - undetected for years at a time. - </p> - - <p> - <strong>Recommendation:</strong> if there is a need to access - the content of a disk image, use a single-use throwaway virtual - machine to access the data. Never mount disk images on the host - OS. Ideally make use of the <a href="http://libguestfs.org">libguestfs</a> - tools and APIs for accessing disks - </p> - - <h2><a id="migration">Guest migration network</a></h2> - - <p> - Most hypervisors with support for guest migration between hosts - make use of one (or more) network connections. Typically the source - host will connect to some port on the target host to initiate the - migration. There may be separate connections for co-ordinating the - migration, transferring memory state and transferring storage. - If the network over which migration takes place is accessible the - guest, or client applications, there is potential for data leakage - via packet snooping/capture. It is also possible for a malicious - guest or client to make attempts to connect to the target host - to trigger bogus migration operations, or at least inflict a denial - of service attack. - </p> - - <p> - <strong>Recommendations:</strong> there are several things to consider - when performing migration - </p> - - <ul> - <li>Use a specific address for establishing the migration - connection which is accessible only to the virtualization - hosts themselves, not libvirt clients or virtual guests. - Most hypervisors allow the management application to provide - the IP address of the target host as a way to - determine which network migration takes place on. This is - effectively the connect() socket address for the source host.</li> - <li>Use a specific address for listening for incoming migration - connections which is accessible only to the virtualization - hosts themselves, not libvirt clients or virtual guests. - Most hypervisors allow the management application to configure - the IP address on which the target host listens. This is - the bind() socket address for the target host.</li> - <li>Use an encrypted migration protocol. Some hypervisors - have support for encrypting the migration memory/storage - data. In other cases it can be tunnelled over the libvirtd - RPC protocol connections.</li> - </ul> - - <h2><a id="storage">Storage encryption</a></h2> - - <p> - Virtual disk images will typically contain confidential data - belonging to the owner of the virtual machine. It is desirable - to protect this against data center administrators as much as - possible. For example, a rogue storage administrator may attempt - to access disk contents directly from a storage host, or a network - administrator/attack may attempt to snoop on data packets relating - to storage access. Use of disk encryption on the virtualization - host can ensure that only the virtualization host administrator - can see the plain text contents of disk images. - </p> - - <p> - <strong>Recommendation:</strong> make use of storage encryption - to protect non-local storage from attack by rogue network / - storage administrators or external attackers. This is particularly - important if the storage protocol itself does not offer any kind - of encryption capabilities. - </p> - - </body> -</html> diff --git a/docs/kbase/secureusage.rst b/docs/kbase/secureusage.rst new file mode 100644 index 0000000000..5a631193d1 --- /dev/null +++ b/docs/kbase/secureusage.rst @@ -0,0 +1,131 @@ +======================= +Secure Usage of Libvirt +======================= + +.. contents:: + +This page details information that application developers and +administrators of libvirt should be aware of when working with libvirt, +that may have a bearing on security of the system. + +Disk image handling +=================== + +Disk image format probing +------------------------- + +Historically there have been multiple flaws in QEMU and most projects +using QEMU, related to handling of disk formats. The problems occur when +a guest is given a virtual disk backed by raw disk format on the host. +If the management application on the host tries to auto-detect / probe +the disk format, it is vulnerable to a malicious guest which can write a +qcow2 file header into its raw disk. If the management application +subsequently probes the disk, it will see it as a 'qcow2' disk instead +of a 'raw' disk. Since 'qcow2' disks can have a copy on write backing +file, such flaw can be leveraged to read arbitrary files on the host. +The same type of flaw may occur if the management application allows +users to upload pre-created raw images. + +**Recommendation:** never attempt to automatically detect the format of +a disk image based on file contents which are accessible to / originate +from an untrusted source. + +Disk image backing files +------------------------ + +If a management application allows users to upload pre-created disk +images in non-raw formats, it can be tricked into giving the user access +to arbitrary host files via the copy-on-write backing file feature. This +is because the qcow2 disk format header contains a filename field which +can point to any location. It can also point to network protocols such +as NBD, HTTP, GlusterFS, RBD and more. This could allow for compromise +of almost arbitrary data accessible on the LAN/WAN. + +**Recommendation:** always validate that a disk image originating from +an untrusted source has no backing file set. If a backing file is seen, +reject the image. + +Disk image size validation +-------------------------- + +If an application allows users to upload pre-created disk images in +non-raw formats, it is essential to validate the logical disk image +size, rather than the physical disk image size. Non-raw disk images have +a grow-on-demand capability, so a user can provide a qcow2 image that +may be only 1 MB in size, but is configured to grow to many TB in size. + +**Recommendation:** if receiving a non-raw disk image from an untrusted +source, validate the logical image size stored in the disk image +metadata against some finite limit. + +Disk image data access +---------------------- + +If an untrusted disk image is ever mounted on the host OS by a +management application or administrator, this opens an avenue of attack +with which to potentially compromise the host kernel. Filesystem drivers +in OS kernels are often very complex code and thus may have bugs lurking +in them. With Linux, there are a large number of filesystem drivers, +many of which attract little security analysis attention. Linux will +helpfully probe filesystem formats if not told to use an explicit +format, allowing an attacker the ability to target specific weak +filesystem drivers. Even commonly used and widely audited filesystems +such as ``ext4`` have had `bugs lurking in +them <https://lwn.net/Articles/538898/>`__ undetected for years at a +time. + +**Recommendation:** if there is a need to access the content of a disk +image, use a single-use throwaway virtual machine to access the data. +Never mount disk images on the host OS. Ideally make use of the +`libguestfs <http://libguestfs.org>`__ tools and APIs for accessing +disks + +Guest migration network +======================= + +Most hypervisors with support for guest migration between hosts make use +of one (or more) network connections. Typically the source host will +connect to some port on the target host to initiate the migration. There +may be separate connections for co-ordinating the migration, +transferring memory state and transferring storage. If the network over +which migration takes place is accessible the guest, or client +applications, there is potential for data leakage via packet +snooping/capture. It is also possible for a malicious guest or client to +make attempts to connect to the target host to trigger bogus migration +operations, or at least inflict a denial of service attack. + +**Recommendations:** there are several things to consider when +performing migration + +- Use a specific address for establishing the migration connection + which is accessible only to the virtualization hosts themselves, not + libvirt clients or virtual guests. Most hypervisors allow the + management application to provide the IP address of the target host + as a way to determine which network migration takes place on. This is + effectively the connect() socket address for the source host. +- Use a specific address for listening for incoming migration + connections which is accessible only to the virtualization hosts + themselves, not libvirt clients or virtual guests. Most hypervisors + allow the management application to configure the IP address on which + the target host listens. This is the bind() socket address for the + target host. +- Use an encrypted migration protocol. Some hypervisors have support + for encrypting the migration memory/storage data. In other cases it + can be tunnelled over the libvirtd RPC protocol connections. + +Storage encryption +================== + +Virtual disk images will typically contain confidential data belonging +to the owner of the virtual machine. It is desirable to protect this +against data center administrators as much as possible. For example, a +rogue storage administrator may attempt to access disk contents directly +from a storage host, or a network administrator/attack may attempt to +snoop on data packets relating to storage access. Use of disk encryption +on the virtualization host can ensure that only the virtualization host +administrator can see the plain text contents of disk images. + +**Recommendation:** make use of storage encryption to protect non-local +storage from attack by rogue network / storage administrators or +external attackers. This is particularly important if the storage +protocol itself does not offer any kind of encryption capabilities. -- 2.23.0

This is a semi-automated conversion. The first conversion is done using "pandoc -f html -t rst". The result is then editted manually to apply the desired heading markup, and fix a few things that pandoc gets wrong. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/kbase/locking.html.in | 48 -------------------------------------- docs/kbase/locking.rst | 33 ++++++++++++++++++++++++++ 2 files changed, 33 insertions(+), 48 deletions(-) delete mode 100644 docs/kbase/locking.html.in create mode 100644 docs/kbase/locking.rst diff --git a/docs/kbase/locking.html.in b/docs/kbase/locking.html.in deleted file mode 100644 index 4532dbddf9..0000000000 --- a/docs/kbase/locking.html.in +++ /dev/null @@ -1,48 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE html> -<html xmlns="http://www.w3.org/1999/xhtml"> - <body> - <h1>Virtual machine lock manager</h1> - - <ul id="toc"></ul> - - <p> - Libvirt includes a framework for ensuring mutual exclusion - between virtual machines using host resources. Typically - this is used to prevent two VM processes from having concurrent - write access to the same disk image, as this would result in - data corruption if the guest was not using a cluster - aware filesystem. - </p> - - <h2><a id="plugins">Lock manager plugins</a></h2> - - <p> - The lock manager framework has a pluggable architecture, - to allow different locking technologies to be used. - </p> - - <dl> - <dt><code>nop</code></dt> - <dd>This is a "no op" implementation which does absolutely - nothing. This can be used if mutual exclusion between - virtual machines is not required, or if it is being - solved at another level in the management stack.</dd> - <dt><code><a href="locking-lockd.html">lockd</a></code></dt> - <dd>This is the current preferred implementation shipped - with libvirt. It uses the <code>virtlockd</code> daemon - to manage locks using the POSIX fcntl() advisory locking - capability. As such it requires a shared filesystem of - some kind be accessible to all hosts which share the - same image storage.</dd> - <dt><code><a href="locking-sanlock.html">sanlock</a></code></dt> - <dd>This is an alternative implementation preferred by - the oVirt project. It uses a disk paxos algorithm for - maintaining continuously renewed leases. In the default - setup it requires some shared filesystem, but it is - possible to use it in a manual mode where the management - application creates leases in SAN storage volumes. - </dd> - </dl> - </body> -</html> diff --git a/docs/kbase/locking.rst b/docs/kbase/locking.rst new file mode 100644 index 0000000000..c0d5e39b2d --- /dev/null +++ b/docs/kbase/locking.rst @@ -0,0 +1,33 @@ +============================ +Virtual machine lock manager +============================ + +Libvirt includes a framework for ensuring mutual exclusion between +virtual machines using host resources. Typically this is used to prevent +two VM processes from having concurrent write access to the same disk +image, as this would result in data corruption if the guest was not +using a cluster aware filesystem. + +Lock manager plugins +==================== + +The lock manager framework has a pluggable architecture, to allow +different locking technologies to be used. + +nop + This is a "no op" implementation which does absolutely nothing. This + can be used if mutual exclusion between virtual machines is not + required, or if it is being solved at another level in the management + stack. +`lockd <locking-lockd.html>`__ + This is the current preferred implementation shipped with libvirt. It + uses the ``virtlockd`` daemon to manage locks using the POSIX fcntl() + advisory locking capability. As such it requires a shared filesystem + of some kind be accessible to all hosts which share the same image + storage. +`sanlock <locking-sanlock.html>`__ + This is an alternative implementation preferred by the oVirt project. + It uses a disk paxos algorithm for maintaining continuously renewed + leases. In the default setup it requires some shared filesystem, but + it is possible to use it in a manual mode where the management + application creates leases in SAN storage volumes. -- 2.23.0

This is a semi-automated conversion. The first conversion is done using "pandoc -f html -t rst". The result is then editted manually to apply the desired heading markup, and fix a few things that pandoc gets wrong. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/kbase/locking-lockd.html.in | 160 ------------------------------- docs/kbase/locking-lockd.rst | 121 +++++++++++++++++++++++ 2 files changed, 121 insertions(+), 160 deletions(-) delete mode 100644 docs/kbase/locking-lockd.html.in create mode 100644 docs/kbase/locking-lockd.rst diff --git a/docs/kbase/locking-lockd.html.in b/docs/kbase/locking-lockd.html.in deleted file mode 100644 index 855404ac97..0000000000 --- a/docs/kbase/locking-lockd.html.in +++ /dev/null @@ -1,160 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE html> -<html xmlns="http://www.w3.org/1999/xhtml"> - <body> - <h1>Virtual machine lock manager, virtlockd plugin</h1> - - <ul id="toc"></ul> - - <p> - This page describes use of the <code>virtlockd</code> - service as a <a href="locking.html">lock driver</a> - plugin for virtual machine disk mutual exclusion. - </p> - - <h2><a id="background">virtlockd background</a></h2> - - <p> - The virtlockd daemon is a single purpose binary which - focuses exclusively on the task of acquiring and holding - locks on behalf of running virtual machines. It is - designed to offer a low overhead, portable locking - scheme can be used out of the box on virtualization - hosts with minimal configuration overheads. It makes - use of the POSIX fcntl advisory locking capability - to hold locks, which is supported by the majority of - commonly used filesystems. - </p> - - <h2><a id="sanlock">virtlockd daemon setup</a></h2> - - <p> - In most OS, the virtlockd daemon itself will not require - any upfront configuration work. It is installed by default - when libvirtd is present, and a systemd socket unit is - registered such that the daemon will be automatically - started when first required. With OS that predate systemd - though, it will be necessary to start it at boot time, - prior to libvirtd being started. On RHEL/Fedora distros, - this can be achieved as follows - </p> - - <pre> -# chkconfig virtlockd on -# service virtlockd start - </pre> - - <p> - The above instructions apply to the instance of virtlockd - that runs privileged, and is used by the libvirtd daemon - that runs privileged. If running libvirtd as an unprivileged - user, it will always automatically spawn an instance of - the virtlockd daemon unprivileged too. This requires no - setup at all. - </p> - - <h2><a id="lockdplugin">libvirt lockd plugin configuration</a></h2> - - <p> - Once the virtlockd daemon is running, or setup to autostart, - the next step is to configure the libvirt lockd plugin. - There is a separate configuration file for each libvirt - driver that is using virtlockd. For QEMU, we will edit - <code>/etc/libvirt/qemu-lockd.conf</code> - </p> - - <p> - The default behaviour of the lockd plugin is to acquire locks - directly on the virtual disk images associated with the guest - <disk> elements. This ensures it can run out of the box - with no configuration, providing locking for disk images on - shared filesystems such as NFS. It does not provide any cross - host protection for storage that is backed by block devices, - since locks acquired on device nodes in /dev only apply within - the host. It may also be the case that the filesystem holding - the disk images is not capable of supporting fcntl locks. - </p> - - <p> - To address these problems it is possible to tell lockd to - acquire locks on an indirect file. Essentially lockd will - calculate the SHA256 checksum of the fully qualified path, - and create a zero length file in a given directory whose - filename is the checksum. It will then acquire a lock on - that file. Assuming the block devices assigned to the guest - are using stable paths (eg /dev/disk/by-path/XXXXXXX) then - this will allow for locks to apply across hosts. This - feature can be enabled by setting a configuration setting - that specifies the directory in which to create the lock - files. The directory referred to should of course be - placed on a shared filesystem (eg NFS) that is accessible - to all hosts which can see the shared block devices. - </p> - - <pre> -$ su - root -# augtool -s set \ - /files/etc/libvirt/qemu-lockd.conf/file_lockspace_dir \ - "/var/lib/libvirt/lockd/files" - </pre> - - <p> - If the guests are using either LVM and SCSI block devices - for their virtual disks, there is a unique identifier - associated with each device. It is possible to tell lockd - to use this UUID as the basis for acquiring locks, rather - than the SHA256 sum of the filename. The benefit of this - is that the locking protection will work even if the file - paths to the given block device are different on each - host. - </p> - - <pre> -$ su - root -# augtool -s set \ - /files/etc/libvirt/qemu-lockd.conf/scsi_lockspace_dir \ - "/var/lib/libvirt/lockd/scsi" -# augtool -s set \ - /files/etc/libvirt/qemu-lockd.conf/lvm_lockspace_dir \ - "/var/lib/libvirt/lockd/lvm" - </pre> - - <p> - It is important to remember that the changes made to the - <code>/etc/libvirt/qemu-lockd.conf</code> file must be - propagated to all hosts before any virtual machines are - launched on them. This ensures that all hosts are using - the same locking mechanism - </p> - - <h2><a id="qemuconfig">QEMU/KVM driver configuration</a></h2> - - <p> - The QEMU driver is capable of using the virtlockd plugin - since the release <span>1.0.2</span>. - The out of the box configuration, however, currently - uses the <strong>nop</strong> lock manager plugin. - To get protection for disks, it is thus necessary - to reconfigure QEMU to activate the <strong>lockd</strong> - driver. This is achieved by editing the QEMU driver - configuration file (<code>/etc/libvirt/qemu.conf</code>) - and changing the <code>lock_manager</code> configuration - tunable. - </p> - - <pre> -$ su - root -# augtool -s set /files/etc/libvirt/qemu.conf/lock_manager lockd -# service libvirtd restart - </pre> - - <p> - Every time you start a guest, the virtlockd daemon will acquire - locks on the disk files directly, or in one of the configured - lookaside directories based on SHA256 sum. To check that locks - are being acquired as expected, the <code>lslocks</code> tool - can be run. - </p> - - </body> -</html> diff --git a/docs/kbase/locking-lockd.rst b/docs/kbase/locking-lockd.rst new file mode 100644 index 0000000000..70e742b77c --- /dev/null +++ b/docs/kbase/locking-lockd.rst @@ -0,0 +1,121 @@ +============================================== +Virtual machine lock manager, virtlockd plugin +============================================== + +.. contents:: + +This page describes use of the ``virtlockd`` service as a `lock +driver <locking.html>`__ plugin for virtual machine disk mutual +exclusion. + +virtlockd background +==================== + +The virtlockd daemon is a single purpose binary which focuses +exclusively on the task of acquiring and holding locks on behalf of +running virtual machines. It is designed to offer a low overhead, +portable locking scheme can be used out of the box on virtualization +hosts with minimal configuration overheads. It makes use of the POSIX +fcntl advisory locking capability to hold locks, which is supported by +the majority of commonly used filesystems. + +virtlockd daemon setup +====================== + +In most OS, the virtlockd daemon itself will not require any upfront +configuration work. It is installed by default when libvirtd is present, +and a systemd socket unit is registered such that the daemon will be +automatically started when first required. With OS that predate systemd +though, it will be necessary to start it at boot time, prior to libvirtd +being started. On RHEL/Fedora distros, this can be achieved as follows + +:: + + # chkconfig virtlockd on + # service virtlockd start + +The above instructions apply to the instance of virtlockd that runs +privileged, and is used by the libvirtd daemon that runs privileged. If +running libvirtd as an unprivileged user, it will always automatically +spawn an instance of the virtlockd daemon unprivileged too. This +requires no setup at all. + +libvirt lockd plugin configuration +================================== + +Once the virtlockd daemon is running, or setup to autostart, the next +step is to configure the libvirt lockd plugin. There is a separate +configuration file for each libvirt driver that is using virtlockd. For +QEMU, we will edit ``/etc/libvirt/qemu-lockd.conf`` + +The default behaviour of the lockd plugin is to acquire locks directly +on the virtual disk images associated with the guest <disk> elements. +This ensures it can run out of the box with no configuration, providing +locking for disk images on shared filesystems such as NFS. It does not +provide any cross host protection for storage that is backed by block +devices, since locks acquired on device nodes in /dev only apply within +the host. It may also be the case that the filesystem holding the disk +images is not capable of supporting fcntl locks. + +To address these problems it is possible to tell lockd to acquire locks +on an indirect file. Essentially lockd will calculate the SHA256 +checksum of the fully qualified path, and create a zero length file in a +given directory whose filename is the checksum. It will then acquire a +lock on that file. Assuming the block devices assigned to the guest are +using stable paths (eg /dev/disk/by-path/XXXXXXX) then this will allow +for locks to apply across hosts. This feature can be enabled by setting +a configuration setting that specifies the directory in which to create +the lock files. The directory referred to should of course be placed on +a shared filesystem (eg NFS) that is accessible to all hosts which can +see the shared block devices. + +:: + + $ su - root + # augtool -s set \ + /files/etc/libvirt/qemu-lockd.conf/file_lockspace_dir \ + "/var/lib/libvirt/lockd/files" + +If the guests are using either LVM and SCSI block devices for their +virtual disks, there is a unique identifier associated with each device. +It is possible to tell lockd to use this UUID as the basis for acquiring +locks, rather than the SHA256 sum of the filename. The benefit of this +is that the locking protection will work even if the file paths to the +given block device are different on each host. + +:: + + $ su - root + # augtool -s set \ + /files/etc/libvirt/qemu-lockd.conf/scsi_lockspace_dir \ + "/var/lib/libvirt/lockd/scsi" + # augtool -s set \ + /files/etc/libvirt/qemu-lockd.conf/lvm_lockspace_dir \ + "/var/lib/libvirt/lockd/lvm" + +It is important to remember that the changes made to the +``/etc/libvirt/qemu-lockd.conf`` file must be propagated to all hosts +before any virtual machines are launched on them. This ensures that all +hosts are using the same locking mechanism + +QEMU/KVM driver configuration +============================= + +The QEMU driver is capable of using the virtlockd plugin since the +release 1.0.2. The out of the box configuration, however, currently uses +the **nop** lock manager plugin. To get protection for disks, it is thus +necessary to reconfigure QEMU to activate the **lockd** driver. This is +achieved by editing the QEMU driver configuration file +(``/etc/libvirt/qemu.conf``) and changing the ``lock_manager`` +configuration tunable. + +:: + + $ su - root + # augtool -s set /files/etc/libvirt/qemu.conf/lock_manager lockd + # service libvirtd restart + +Every time you start a guest, the virtlockd daemon will acquire locks on +the disk files directly, or in one of the configured lookaside +directories based on SHA256 sum. To check that locks are being acquired +as expected, the ``lslocks`` tool can be run. -- 2.23.0

This is a semi-automated conversion. The first conversion is done using "pandoc -f html -t rst". The result is then editted manually to apply the desired heading markup, and fix a few things that pandoc gets wrong. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/kbase/locking-sanlock.html.in | 247 ----------------------------- docs/kbase/locking-sanlock.rst | 193 ++++++++++++++++++++++ 2 files changed, 193 insertions(+), 247 deletions(-) delete mode 100644 docs/kbase/locking-sanlock.html.in create mode 100644 docs/kbase/locking-sanlock.rst diff --git a/docs/kbase/locking-sanlock.html.in b/docs/kbase/locking-sanlock.html.in deleted file mode 100644 index 3ee9849e87..0000000000 --- a/docs/kbase/locking-sanlock.html.in +++ /dev/null @@ -1,247 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE html> -<html xmlns="http://www.w3.org/1999/xhtml"> - <body> - <h1>Virtual machine lock manager, sanlock plugin</h1> - - <ul id="toc"></ul> - - <p> - This page describes use of the - <a href="https://fedorahosted.org/sanlock/">sanlock</a> - service as a <a href="locking.html">lock driver</a> - plugin for virtual machine disk mutual exclusion. - </p> - - <h2><a id="sanlock">Sanlock daemon setup</a></h2> - - <p> - On many operating systems, the <strong>sanlock</strong> plugin - is distributed in a sub-package which needs to be installed - separately from the main libvirt RPM. On a Fedora/RHEL host - this can be done with the <code>yum</code> command - </p> - - <pre> -$ su - root -# yum install libvirt-lock-sanlock - </pre> - - <p> - The next step is to start the sanlock daemon. For maximum - safety sanlock prefers to have a connection to a watchdog - daemon. This will cause the entire host to be rebooted in - the event that sanlock crashes / terminates abnormally. - To start the watchdog daemon on a Fedora/RHEL host - the following commands can be run: - </p> - - <pre> -$ su - root -# chkconfig wdmd on -# service wdmd start - </pre> - - <p> - Once the watchdog is running, sanlock can be started - as follows - </p> - - <pre> -# chkconfig sanlock on -# service sanlock start - </pre> - - <p> - <em>Note:</em> if you wish to avoid the use of the - watchdog, add the following line to <code>/etc/sysconfig/sanlock</code> - before starting it - </p> - - <pre> -SANLOCKOPTS="-w 0" - </pre> - - <p> - The sanlock daemon must be started on every single host - that will be running virtual machines. So repeat these - steps as necessary. - </p> - - <h2><a id="sanlockplugin">libvirt sanlock plugin configuration</a></h2> - - <p> - Once the sanlock daemon is running, the next step is to - configure the libvirt sanlock plugin. There is a separate - configuration file for each libvirt driver that is using - sanlock. For QEMU, we will edit <code>/etc/libvirt/qemu-sanlock.conf</code> - There is one mandatory parameter that needs to be set, - the <code>host_id</code>. This is an integer between - 1 and 2000, which must be set to a <strong>unique</strong> - value on each host running virtual machines. - </p> - - <pre> -$ su - root -# augtool -s set /files/etc/libvirt/qemu-sanlock.conf/host_id 1 - </pre> - - <p> - Repeat this on every host, changing <strong>1</strong> to a - unique value for the host. - </p> - - <h2><a id="sanlockstorage">libvirt sanlock storage configuration</a></h2> - - <p> - The sanlock plugin needs to create leases in a directory - that is on a filesystem shared between all hosts running - virtual machines. Obvious choices for this include NFS - or GFS2. The libvirt sanlock plugin expects its lease - directory be at <code>/var/lib/libvirt/sanlock</code> - so update the host's <code>/etc/fstab</code> to mount - a suitable shared/cluster filesystem at that location - </p> - - <pre> -$ su - root -# echo "some.nfs.server:/export/sanlock /var/lib/libvirt/sanlock nfs hard,nointr 0 0" >> /etc/fstab -# mount /var/lib/libvirt/sanlock - </pre> - - <p> - If your sanlock daemon happen to run under non-root - privileges, you need to tell this to libvirt so it - chowns created files correctly. This can be done by - setting <code>user</code> and/or <code>group</code> - variables in the configuration file. Accepted values - range is specified in description to the same - variables in <code>/etc/libvirt/qemu.conf</code>. For - example: - </p> - - <pre> -augtool -s set /files/etc/libvirt/qemu-sanlock.conf/user sanlock -augtool -s set /files/etc/libvirt/qemu-sanlock.conf/group sanlock - </pre> - - <p> - But remember, that if this is NFS share, you need a - no_root_squash-ed one for chown (and chmod possibly) - to succeed. - </p> - - <p> - In terms of storage requirements, if the filesystem - uses 512 byte sectors, you need to allow for <code>1MB</code> - of storage for each guest disk. So if you have a network - with 20 virtualization hosts, each running 50 virtual - machines and an average of 2 disks per guest, you will - need <code>20*50*2 == 2000 MB</code> of storage for - sanlock. - </p> - - - <p> - On one of the hosts on the network is it wise to setup - a cron job which runs the <code>virt-sanlock-cleanup</code> - script periodically. This scripts deletes any lease - files which are not currently in use by running virtual - machines, freeing up disk space on the shared filesystem. - Unless VM disks are very frequently created + deleted - it should be sufficient to run the cleanup once a week. - </p> - - <h2><a id="qemuconfig">QEMU/KVM driver configuration</a></h2> - - <p> - The QEMU/KVM driver is fully integrated with the lock - manager framework as of release <span>0.9.3</span>. - The out of the box configuration, however, currently - uses the <strong>nop</strong> lock manager plugin. - To get protection for disks, it is thus necessary - to reconfigure QEMU to activate the <strong>sanlock</strong> - driver. This is achieved by editing the QEMU driver - configuration file (<code>/etc/libvirt/qemu.conf</code>) - and changing the <code>lock_manager</code> configuration - tunable. - </p> - - <pre> -$ su - root -# augtool -s set /files/etc/libvirt/qemu.conf/lock_manager sanlock -# service libvirtd restart - </pre> - - <p> - If all went well, libvirtd will have talked to sanlock - and created the basic lockspace. This can be checked - by looking for existence of the following file - </p> - - <pre> -# ls /var/lib/libvirt/sanlock/ -__LIBVIRT__DISKS__ - </pre> - - <p> - Every time you start a guest, additional lease files will appear - in this directory, one for each virtual disk. The lease - files are named based on the MD5 checksum of the fully qualified - path of the virtual disk backing file. So if the guest is given - a disk backed by <code>/var/lib/libvirt/images/demo.img</code> - expect to see a lease <code>/var/lib/libvirt/sanlock/bfa0240911bc17753e0b473688822159</code> - </p> - - <p> - It should be obvious that for locking to work correctly, every - host running virtual machines should have storage configured - in the same way. The easiest way to do this is to use the libvirt - storage pool capability to configure any NFS volumes, iSCSI targets, - or SCSI HBAs used for guest storage. Simply replicate the same - storage pool XML across every host. It is important that any - storage pools exposing block devices are configured to create - volume paths under <code>/dev/disks/by-path</code> to ensure - stable paths across hosts. An example iSCSI configuration - which ensures this is: - </p> - - <pre> -<pool type='iscsi'> - <name>myiscsipool</name> - <source> - <host name='192.168.254.8'/> - <device path='your-iscsi-target-iqn'/> - </source> - <target> - <path>/dev/disk/by-path</path> - </target> -</pool> - </pre> - - <h2><a id="domainconfig">Domain configuration</a></h2> - - <p> - In case sanlock loses access to disk locks for some reason, it will - kill all domains that lost their locks. This default behavior may - be changed using - <a href="formatdomain.html#elementsEvents">on_lockfailure - element</a> in domain XML. When this element is present, sanlock - will call <code>sanlock_helper</code> (provided by libvirt) with - the specified action. This helper binary will connect to libvirtd - and thus it may need to authenticate if libvirtd was configured to - require that on the read-write UNIX socket. To provide the - appropriate credentials to sanlock_helper, a - <a href="auth.html#Auth_client_config">client authentication - file</a> needs to contain something like the following: - </p> - <pre> -[auth-libvirt-localhost] -credentials=sanlock - -[credentials-sanlock] -authname=login -password=password - </pre> - </body> -</html> diff --git a/docs/kbase/locking-sanlock.rst b/docs/kbase/locking-sanlock.rst new file mode 100644 index 0000000000..fece9df80e --- /dev/null +++ b/docs/kbase/locking-sanlock.rst @@ -0,0 +1,193 @@ +============================================ +Virtual machine lock manager, sanlock plugin +============================================ + +.. contents:: + +This page describes use of the +`sanlock <https://fedorahosted.org/sanlock/>`__ service as a `lock +driver <locking.html>`__ plugin for virtual machine disk mutual +exclusion. + +Sanlock daemon setup +==================== + +On many operating systems, the **sanlock** plugin is distributed in a +sub-package which needs to be installed separately from the main libvirt +RPM. On a Fedora/RHEL host this can be done with the ``yum`` command + +:: + + $ su - root + # yum install libvirt-lock-sanlock + +The next step is to start the sanlock daemon. For maximum safety sanlock +prefers to have a connection to a watchdog daemon. This will cause the +entire host to be rebooted in the event that sanlock crashes / +terminates abnormally. To start the watchdog daemon on a Fedora/RHEL +host the following commands can be run: + +:: + + $ su - root + # chkconfig wdmd on + # service wdmd start + +Once the watchdog is running, sanlock can be started as follows + +:: + + # chkconfig sanlock on + # service sanlock start + +*Note:* if you wish to avoid the use of the watchdog, add the following +line to ``/etc/sysconfig/sanlock`` before starting it + +:: + + SANLOCKOPTS="-w 0" + +The sanlock daemon must be started on every single host that will be +running virtual machines. So repeat these steps as necessary. + +libvirt sanlock plugin configuration +==================================== + +Once the sanlock daemon is running, the next step is to configure the +libvirt sanlock plugin. There is a separate configuration file for each +libvirt driver that is using sanlock. For QEMU, we will edit +``/etc/libvirt/qemu-sanlock.conf`` There is one mandatory parameter that +needs to be set, the ``host_id``. This is an integer between 1 and 2000, +which must be set to a **unique** value on each host running virtual +machines. + +:: + + $ su - root + # augtool -s set /files/etc/libvirt/qemu-sanlock.conf/host_id 1 + +Repeat this on every host, changing **1** to a unique value for the +host. + +libvirt sanlock storage configuration +===================================== + +The sanlock plugin needs to create leases in a directory that is on a +filesystem shared between all hosts running virtual machines. Obvious +choices for this include NFS or GFS2. The libvirt sanlock plugin expects +its lease directory be at ``/var/lib/libvirt/sanlock`` so update the +host's ``/etc/fstab`` to mount a suitable shared/cluster filesystem at +that location + +:: + + $ su - root + # echo "some.nfs.server:/export/sanlock /var/lib/libvirt/sanlock nfs hard,nointr 0 0" >> /etc/fstab + # mount /var/lib/libvirt/sanlock + +If your sanlock daemon happen to run under non-root privileges, you need +to tell this to libvirt so it chowns created files correctly. This can +be done by setting ``user`` and/or ``group`` variables in the +configuration file. Accepted values range is specified in description to +the same variables in ``/etc/libvirt/qemu.conf``. For example: + +:: + + augtool -s set /files/etc/libvirt/qemu-sanlock.conf/user sanlock + augtool -s set /files/etc/libvirt/qemu-sanlock.conf/group sanlock + +But remember, that if this is NFS share, you need a no_root_squash-ed +one for chown (and chmod possibly) to succeed. + +In terms of storage requirements, if the filesystem uses 512 byte +sectors, you need to allow for ``1MB`` of storage for each guest disk. +So if you have a network with 20 virtualization hosts, each running 50 +virtual machines and an average of 2 disks per guest, you will need +``20*50*2 == 2000 MB`` of storage for sanlock. + +On one of the hosts on the network is it wise to setup a cron job which +runs the ``virt-sanlock-cleanup`` script periodically. This scripts +deletes any lease files which are not currently in use by running +virtual machines, freeing up disk space on the shared filesystem. Unless +VM disks are very frequently created + deleted it should be sufficient +to run the cleanup once a week. + +QEMU/KVM driver configuration +============================= + +The QEMU/KVM driver is fully integrated with the lock manager framework +as of release 0.9.3. The out of the box configuration, however, +currently uses the **nop** lock manager plugin. To get protection for +disks, it is thus necessary to reconfigure QEMU to activate the +**sanlock** driver. This is achieved by editing the QEMU driver +configuration file (``/etc/libvirt/qemu.conf``) and changing the +``lock_manager`` configuration tunable. + +:: + + $ su - root + # augtool -s set /files/etc/libvirt/qemu.conf/lock_manager sanlock + # service libvirtd restart + +If all went well, libvirtd will have talked to sanlock and created the +basic lockspace. This can be checked by looking for existence of the +following file + +:: + + # ls /var/lib/libvirt/sanlock/ + __LIBVIRT__DISKS__ + +Every time you start a guest, additional lease files will appear in this +directory, one for each virtual disk. The lease files are named based on +the MD5 checksum of the fully qualified path of the virtual disk backing +file. So if the guest is given a disk backed by +``/var/lib/libvirt/images/demo.img`` expect to see a lease +``/var/lib/libvirt/sanlock/bfa0240911bc17753e0b473688822159`` + +It should be obvious that for locking to work correctly, every host +running virtual machines should have storage configured in the same way. +The easiest way to do this is to use the libvirt storage pool capability +to configure any NFS volumes, iSCSI targets, or SCSI HBAs used for guest +storage. Simply replicate the same storage pool XML across every host. +It is important that any storage pools exposing block devices are +configured to create volume paths under ``/dev/disks/by-path`` to ensure +stable paths across hosts. An example iSCSI configuration which ensures +this is: + +:: + + <pool type='iscsi'> + <name>myiscsipool</name> + <source> + <host name='192.168.254.8'/> + <device path='your-iscsi-target-iqn'/> + </source> + <target> + <path>/dev/disk/by-path</path> + </target> + </pool> + +Domain configuration +==================== + +In case sanlock loses access to disk locks for some reason, it will kill +all domains that lost their locks. This default behavior may be changed +using `on_lockfailure element <formatdomain.html#elementsEvents>`__ in +domain XML. When this element is present, sanlock will call +``sanlock_helper`` (provided by libvirt) with the specified action. This +helper binary will connect to libvirtd and thus it may need to +authenticate if libvirtd was configured to require that on the +read-write UNIX socket. To provide the appropriate credentials to +sanlock_helper, a `client authentication +file <auth.html#Auth_client_config>`__ needs to contain something like +the following: + +:: + + [auth-libvirt-localhost] + credentials=sanlock + + [credentials-sanlock] + authname=login + password=password -- 2.23.0

The libvirt RPM packaging is quite fine grained but it is not obvious to users which package is best to install. Add a kbase doc that describes the different RPMs, and illustrates some example deployment use cases. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- docs/kbase.html.in | 4 + docs/kbase/rpm-deployment.rst | 410 ++++++++++++++++++++++++++++++++++ 2 files changed, 414 insertions(+) create mode 100644 docs/kbase/rpm-deployment.rst diff --git a/docs/kbase.html.in b/docs/kbase.html.in index 97d3f4c384..a5504a540f 100644 --- a/docs/kbase.html.in +++ b/docs/kbase.html.in @@ -21,6 +21,10 @@ capture</a></dt> <dd>Comparison between different methods of capturing domain state</dd> + + <dt><a href="kbase/rpm-deployment.html">RPM deployment</a></dt> + <dd>Explanation of the different RPM packages and illustration of + which to pick for installation</dd> </dl> </div> diff --git a/docs/kbase/rpm-deployment.rst b/docs/kbase/rpm-deployment.rst new file mode 100644 index 0000000000..8f1584d7ea --- /dev/null +++ b/docs/kbase/rpm-deployment.rst @@ -0,0 +1,410 @@ +======================= +RPM Deployment Guidance +======================= + +.. contents:: + +A complete libvirt build includes a wide range of features, many of which are +dynamically loadable at runtime. Applications using libvirt typically only +need to use a subset of these features, and so do not require a full install +of all libvirt RPM packages. + +This document provides some guidance on the RPM packages available with libvirt +on Fedora and related distributions, to enable applications and administrators +to pick the optimal set for their needs. + +The RHEL and CentOS distributions use the same RPM packaging split, but many +of the drivers will be disabled at build time, so not all of the packages +listed on this page will exist. + + +RPM packages +============ + +* libvirt + + This is an empty package that exists solely as a convenient way to install + every other libvirt RPM package. Almost every deployment scenario would be + better served by picking one of the other RPMs listed below. + +* libvirt-admin + + The virt-admin tool, used for administrative operations on any libvirt + daemons. Most usefully it allows for logging filters and outputs to be + reconfigured on a running daemon without a restart. This is recommended + to be installed on any host running a libvirt daemon. + + +* libvirt-bash-completion + + Argument auto-completion support for the Bash shell. This is shared code that + is pulled in by either the libvirt-admin or libvirt-clients RPMs, so there is + no need to explicitly ask for this package to be installed. + + +* libvirt-client + + The virsh tool, used for interacting with any libvirt driver, both primary + virt drivers and secondary drivers for storage, networking, etc. All libvirt + installs should have this installed as it provides a useful way to view and + debug what is being done by other applications using libvirt. + + +* libvirt-daemon + + The monolithic libvirtd daemon, traditionally used for running all the + stateful drivers. This package does not contain any drivers, so further + packages need to be installed to provide the desired drivers. + + +* libvirt-daemon-config-network + + The sample configuration file providing the 'default' virtual network that + enables outbound NAT based connectivity for virtual machines. This is useful + on desktop installations, but is not typically desired on server + installations where VMs will use full bridged connectivity. + + +* libvirt-daemon-config-nwfilter + + The sample configuration files providing the network filters for protecting + against common malicious guest traffic. This includes protection against ARP, + MAC and IP spoofing. This is typically desired on server installations, if + the mgmt app is using libvirt's network filtering features. + + +* libvirt-daemon-driver-interface + + The dynamically loadable driver providing an implementation of the host + network interface management APIs, as well as the virtinterfaced daemon + binary. + + +* libvirt-daemon-driver-libxl + + The dynamically loadable driver providing an implementation of the hypervisor + APIs for Xen using the libxl library, as well as the virtxend daemon + binary. + + Note that this is a minimal package so does not actually pull in the full + Xen hypervisor package set. This be must requested separately. + + +* libvirt-daemon-driver-lxc + + The dynamically loadable driver providing an implementation of the hypervisor + APIs for Linux containers, as well as the virtlxcd daemon binary. + + +* libvirt-daemon-driver-network + + The dynamically loadable driver providing an implementation of the virtual + network interface management APIs, as well as the virtinterfaced daemon + binary. Typically the libvirt-daemon-config-network RPM will also be desired + when this is installed. + + +* libvirt-daemon-driver-nodedev + + The dynamically loadable driver providing an implementation of the host + device management APIs, as well as the virtnodedevd daemon binary. + + +* libvirt-daemon-driver-nwfilter + + The dynamically loadable driver providing an implementation of the host + network firewall management APIs, as well as the virtnwfilterd daemon + binary. + + +* libvirt-daemon-driver-qemu + + The dynamically loadable driver providing an implementation of the hypervisor + network interface management APIs, as well as the virtqemud daemon + binary. + + Note that this is a minimal package so does not actually pull in the full + QEMU or KVM package set. This be must requested separately. + + +* libvirt-daemon-driver-secret + The dynamically loadable driver providing an implementation of the secret + data management APIs, as well as the virtsecretd daemon binary. + + +* libvirt-daemon-driver-storage + + This is an empty package that exists only as a convenient way to request + installation of all the storage pool drivers. + + If the application only supports a subset of storage pool types, then + a smaller install footprint can be obtained by requesting the individual + drivers. + + +* libvirt-daemon-driver-storage-core + + The dynamically loadable driver providing an implementation of the host + storage pool/volume management APIs, as well as the virtstoraged daemon + binary. + + Note that this is a minimal package so does not actually pull in any pool + implementations. + + +* libvirt-daemon-driver-storage-disk + + The dynamically loadable driver providing an implementation of the disk + partition storage pool type, for the storage pool management APIs. + + +* libvirt-daemon-driver-storage-gluster + + The dynamically loadable driver providing an implementation of the GlusterFS + file storage pool type, for the storage pool management APIs. + + +* libvirt-daemon-driver-storage-iscsi + + The dynamically loadable driver providing an implementation of the ISCSI + disk storage pool type, for the storage pool management APIs. + + +* libvirt-daemon-driver-storage-iscsi-direct + + The dynamically loadable driver providing an implementation of the ISCSI + network storage pool type, for the storage pool management APIs. + + +* libvirt-daemon-driver-storage-logical + + The dynamically loadable driver providing an implementation of the LVM + storage pool type, for the storage pool management APIs. + + +* libvirt-daemon-driver-storage-mpath + + The dynamically loadable driver providing an implementation of the multipath + disk storage pool type, for the storage pool management APIs. + + +* libvirt-daemon-driver-storage-rbd + + The dynamically loadable driver providing an implementation of the RBD + network storage pool type, for the storage pool management APIs. + + +* libvirt-daemon-driver-storage-scsi + + The dynamically loadable driver providing an implementation of the SCSI + disk storage pool type, for the storage pool management APIs. + + +* libvirt-daemon-driver-storage-sheepdog + + The dynamically loadable driver providing an implementation of the SheepDog + network storage pool type, for the storage pool management APIs. + + +* libvirt-daemon-driver-storage-zfs + + The dynamically loadable driver providing an implementation of the ZFS + file storage pool type, for the storage pool management APIs. + + +* libvirt-daemon-driver-vbox + + The dynamically loadable driver providing an implementation of the host + network interface management APIs, as well as the virtinterfaced daemon + binary. + + +* libvirt-daemon-kvm + + This is an empty package that exists only as a convenient way to request + installation of all the libvirt features that are relevant to the management + of KVM guests. This includes the QEMU driver, and the secondary drivers for + secrets, storage pools, virtual networks, host interfaces, host devices + and network filtering. + + It will also pull in the full set of QEMU features that can be utilized with + native architecture KVM guests. + + This is a good default for an installation to use KVM if the specific set of + required features is not known. To have finer grained control over the + features, the subset of libvirt-daemon-driver-XXX packages should be used + instead. + + +* libvirt-daemon-lxc + + This is an empty package that exists only as a convenient way to request + installation of all the libvirt features that are relevant to the management + of Linux containers. This includes the LXC driver, and the secondary drivers + for secrets, storage pools, virtual networks, host interfaces, host devices + and network filtering. + + This is a good default for an installation to use LXC if the specific set of + required features is not known. To have finer grained control over the + features, the subset of libvirt-daemon-driver-XXX packages should be used + instead. + + +* libvirt-daemon-qemu + + This is an empty package that exists only as a convenient way to request + installation of all the libvirt features that are relevant to the management + of QEMU guests. This includes the QEMU driver, and the secondary drivers for + secrets, storage pools, virtual networks, host interfaces, host devices + and network filtering. + + It will also pull in the full set of QEMU features that can be utilized to + emulate any guests architecture supported by QEMU. + + This is a good default for an installation to use QEMU if the specific set of + required features is not known. To have finer grained control over the + features, the subset of libvirt-daemon-driver-XXX packages should be used + instead. + + +* libvirt-daemon-vbox + + This is an empty package that exists only as a convenient way to request + installation of all the libvirt features that are relevant to the management + of KVM guests. This includes the QEMU driver, and the secondary drivers for + secrets, storage pools, virtual networks, host interfaces, host devices + and network filtering. + + This is a good default for an installation to use VirtualBox if the specific + set of required features is not known. To have finer grained control over the + features, the subset of libvirt-daemon-driver-XXX packages should be used + instead. + + +* libvirt-daemon-xen + + This is an empty package that exists only as a convenient way to request + installation of all the libvirt features that are relevant to the management + of KVM guests. This includes the QEMU driver, and the secondary drivers for + secrets, storage pools, virtual networks, host interfaces, host devices + and network filtering. + + It will also pull in the full set of Xen features that can be utilized with + Xen guests. + + This is a good default for an installation to use Xen if the specific set of + required features is not known. To have finer grained control over the + features, the subset of libvirt-daemon-driver-XXX packages should be used + instead. + + +* libvirt-devel + + The header files required to build applications, or language bindings against + the libvirt C library. This should never be required on a production host, + only development hosts. + +* libvirt-docs + + A local copy of the `libvirt website <https://libvirt.org>`_ website content + that matches the deployed version of libvirt. + +* libvirt-libs + + The ELF libraries providing the main application interface to libvirt. These + have stateless drivers (VMWare ESX, HyperV, etc) built-in, and are able to + take to the libvirt daemons to utilize stateful drivers (QEMU, Xen, BHyve, + LXC, VZ, etc). This is needed on all libvirt hosts, both client and server. + +* libvirt-lock-sanlock + + A plugin for locking disks that communicates with the sanlock daemon. It is + optional and only relevant to hosts with the QEMU driver and oVirt management + application. + +* libvirt-login-shell + + A simple login shell that automatically spawns an LXC container for the user + logging in and places them in a shell inside that container. + + +* libvirt-nss + + A NSS plugin that provides hostname resolution for guests attached to a + libvirt virtual network. It is recommended to be installed on any host with + guests using the libvirt virtual network connectivity. + + +* libvirt-wireshark + + A wireshark plugin that allows for dissecting the XDR based RPC protocol used + between libvirt and its daemons. Since production deployments should all be + using a TLS encrypted, this only useful for development hosts with a libvirt + daemon configured without encryption. + + +Deployment choices +================== + +Client only install +------------------- + +If an application is capable of using multiple different virtualization drivers +it is undesirable to force the installation of a specific set of drivers. In +this case the application will merely wish to request a client only install + +Alternatively if an application is intended to communicate with a hypervisor on +a remote host there is no need to install drivers locally, only a client is +needed + +The only required package is the `libvirt-libs`, however, it is useful to +also install `libvirt-client`. + + +Every possible virt driver +-------------------------- + +There is rarely a need to install every virt driver at once on a given host. +In the unlikely event that this is needed, however, the `libvirt` package +should be installed. + +Note that this doesn't actually pull in the hypervisors, only the libvirt +code to talk to the hypervisors. + + +Full features for one virt driver +--------------------------------- + +This is a common default installation profile when there is no need to minimise +the on-disk footprint. + +This is achieved by installing the `libvirt-daemon-XXXX` package for the +virtualization driver that is desired. This will also pull in the default +set of hypervisor packages too. + +Since this installs every possible libvirt feature for the virtualization +driver in question, the on-disk footprint is quite large. The in-memory +footprint of the daemons is also relatively large since alot of code is +loaded. + + +Minimal features for one virt driver +------------------------------------ + +This is the best installation profile when it is desired to minimize the +on-disk footprint. + +This is achieved by installing the individual `libvirt-daemon-driver-XXX` +packages needed for the features that will be used. This will not pull in the +hypervisor packages, allowing a fine grained set of hypervisor features to be +chosen separately. + +Since this allows fine grained installation of individual libvirt drivers, +this results in the lowest on-disk footprint. The in-memory footprint of +the daemons is also minimized by reducing the code loaded. + +As an example, the smallest possible installation for running KVM guests can +be achieved by installing `libvirt-daemon-driver-qemu` and `qemu-kvm-core`. +This will exclude all the secondary libvirt drivers for storage, networking +and host devices, leaving only the bare minimum functionality for managing +KVM guests. -- 2.23.0

ping On Fri, Nov 22, 2019 at 02:46:47PM +0000, Daniel P. Berrangé wrote:
This refactors existing docs related to the remote driver/daemon and URIs. It then also adds a kbase page about RPM package options.
This introduces the use of RST for docs as a replacement for HTML. The intent is that all new docs should use RST from this point. The POD man pages will also be converted to RST. I'll post this as a different series.
Note this series only touches the main HTML content, not the styling / templating using XSL. Tackling that is a separate job I've not attempted. My guess is that we'd use either Sphinx or Pelican to do the templating.
Existing HTML docs are candidates for conversion to RST, however, people should *NOT* attempt todo this manually.
In this series I've converted docs/kbase/ files, using the pandoc tool. This does a pretty good job in general but does need some manual cleanups. First I post-processed its output to change the heading highlighting to follow the documented style
$ cat convert.pl
my @in = <>;
my @out; for (my $i ; $i <= $#in; $i++) { my $line = $in[$i]; if ($line =~ /^=+$/) { my @newout = splice @out, 0, $i - 1; push @newout, $line; push @newout, $out[$i-1]; push @newout, $out[$i]; push @newout, $line; push @newout, "\n"; push @newout, ".. contents::\n"; @out = @newout; } elsif ($line =~ /^(-|~|^|')+$/) { $line =~ s/-/=/g; $line =~ s/~/-/g; $line =~ s/\^/~/g; $line =~ s/'/^/g; push @out, $line; } else { push @out, $line; } }
print @out;
So I'm converting with
pandoc -f html -t rst < foo.html.in | perl convert.pl > foo.rst $EDITOR foot.rst git rm -f foot.html.in git add foo.rst
After pandoc & the headings convert, there's usually manual fixes needed to deal with a few oddities pandoc gets wrong, or looks ugly. This is still way simpler than converting the doc to rst manually.
Changed in v2:
- Added a style guide for RST docs - Probe for rst2html's different names - Added make rules for building rst - Fixed permalinks in generated docs - Misc CSS fixes - Auto-converted all of kbase/ directory
Daniel P. Berrangé (15): docs: split TLS certificate setup into its own file docs: move docs about remote driver URIs into URI docs docs: introduce rst2html as a mandatory tool for building docs docs: adapt filling of <head> section for rst2html output docs: generate permalinks correctly for rst2html output docs: relax CSS context match for pretty tables docs: add styling for <tt> element docs: add a minimal style guide for writing RST docs docs: convert kbase/domainstatecapture.html.in to RST docs: convert kbase/launch_security_sev.html.in to RST docs: convert kbase/secureusage.html.in to RST docs: convert kbase/locking.html.in to RST docs: convert kbase/locking-lockd.html.in to RST docs: convert kbase/locking-sanlock.html.in to RST docs: add a kbase page about RPM packaging options
docs/Makefile.am | 33 +- docs/aclpolkit.html.in | 20 +- docs/docs.html.in | 6 + docs/genaclperms.pl | 2 +- docs/generic.css | 4 +- docs/kbase.html.in | 4 + docs/kbase/domainstatecapture.html.in | 303 ----------- docs/kbase/domainstatecapture.rst | 255 +++++++++ docs/kbase/launch_security_sev.html.in | 533 ------------------- docs/kbase/launch_security_sev.rst | 529 +++++++++++++++++++ docs/kbase/locking-lockd.html.in | 160 ------ docs/kbase/locking-lockd.rst | 121 +++++ docs/kbase/locking-sanlock.html.in | 247 --------- docs/kbase/locking-sanlock.rst | 193 +++++++ docs/kbase/locking.html.in | 48 -- docs/kbase/locking.rst | 33 ++ docs/kbase/rpm-deployment.rst | 410 +++++++++++++++ docs/kbase/secureusage.html.in | 171 ------- docs/kbase/secureusage.rst | 131 +++++ docs/libvirt.css | 76 +-- docs/migration.html.in | 2 +- docs/newapi.xsl | 4 +- docs/page.xsl | 9 +- docs/remote.html.in | 684 +------------------------ docs/styleguide.rst | 66 +++ docs/tlscerts.html.in | 413 +++++++++++++++ docs/uri.html.in | 263 ++++++++-- libvirt.spec.in | 2 + m4/virt-external-programs.m4 | 5 + mingw-libvirt.spec.in | 1 + 30 files changed, 2478 insertions(+), 2250 deletions(-) delete mode 100644 docs/kbase/domainstatecapture.html.in create mode 100644 docs/kbase/domainstatecapture.rst delete mode 100644 docs/kbase/launch_security_sev.html.in create mode 100644 docs/kbase/launch_security_sev.rst delete mode 100644 docs/kbase/locking-lockd.html.in create mode 100644 docs/kbase/locking-lockd.rst delete mode 100644 docs/kbase/locking-sanlock.html.in create mode 100644 docs/kbase/locking-sanlock.rst delete mode 100644 docs/kbase/locking.html.in create mode 100644 docs/kbase/locking.rst create mode 100644 docs/kbase/rpm-deployment.rst delete mode 100644 docs/kbase/secureusage.html.in create mode 100644 docs/kbase/secureusage.rst create mode 100644 docs/styleguide.rst create mode 100644 docs/tlscerts.html.in
-- 2.23.0
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 11/22/19 3:46 PM, Daniel P. Berrangé wrote:
This refactors existing docs related to the remote driver/daemon and URIs. It then also adds a kbase page about RPM package options.
This introduces the use of RST for docs as a replacement for HTML. The intent is that all new docs should use RST from this point. The POD man pages will also be converted to RST. I'll post this as a different series.
Note this series only touches the main HTML content, not the styling / templating using XSL. Tackling that is a separate job I've not attempted. My guess is that we'd use either Sphinx or Pelican to do the templating.
Existing HTML docs are candidates for conversion to RST, however, people should *NOT* attempt todo this manually.
In this series I've converted docs/kbase/ files, using the pandoc tool. This does a pretty good job in general but does need some manual cleanups. First I post-processed its output to change the heading highlighting to follow the documented style
$ cat convert.pl
my @in = <>;
my @out; for (my $i ; $i <= $#in; $i++) { my $line = $in[$i]; if ($line =~ /^=+$/) { my @newout = splice @out, 0, $i - 1; push @newout, $line; push @newout, $out[$i-1]; push @newout, $out[$i]; push @newout, $line; push @newout, "\n"; push @newout, ".. contents::\n"; @out = @newout; } elsif ($line =~ /^(-|~|^|')+$/) { $line =~ s/-/=/g; $line =~ s/~/-/g; $line =~ s/\^/~/g; $line =~ s/'/^/g; push @out, $line; } else { push @out, $line; } }
print @out;
So I'm converting with
pandoc -f html -t rst < foo.html.in | perl convert.pl > foo.rst $EDITOR foot.rst git rm -f foot.html.in git add foo.rst
After pandoc & the headings convert, there's usually manual fixes needed to deal with a few oddities pandoc gets wrong, or looks ugly. This is still way simpler than converting the doc to rst manually.
Changed in v2:
- Added a style guide for RST docs - Probe for rst2html's different names - Added make rules for building rst - Fixed permalinks in generated docs - Misc CSS fixes - Auto-converted all of kbase/ directory
Daniel P. Berrangé (15): docs: split TLS certificate setup into its own file docs: move docs about remote driver URIs into URI docs docs: introduce rst2html as a mandatory tool for building docs docs: adapt filling of <head> section for rst2html output docs: generate permalinks correctly for rst2html output docs: relax CSS context match for pretty tables docs: add styling for <tt> element docs: add a minimal style guide for writing RST docs docs: convert kbase/domainstatecapture.html.in to RST docs: convert kbase/launch_security_sev.html.in to RST docs: convert kbase/secureusage.html.in to RST docs: convert kbase/locking.html.in to RST docs: convert kbase/locking-lockd.html.in to RST docs: convert kbase/locking-sanlock.html.in to RST docs: add a kbase page about RPM packaging options
docs/Makefile.am | 33 +- docs/aclpolkit.html.in | 20 +- docs/docs.html.in | 6 + docs/genaclperms.pl | 2 +- docs/generic.css | 4 +- docs/kbase.html.in | 4 + docs/kbase/domainstatecapture.html.in | 303 ----------- docs/kbase/domainstatecapture.rst | 255 +++++++++ docs/kbase/launch_security_sev.html.in | 533 ------------------- docs/kbase/launch_security_sev.rst | 529 +++++++++++++++++++ docs/kbase/locking-lockd.html.in | 160 ------ docs/kbase/locking-lockd.rst | 121 +++++ docs/kbase/locking-sanlock.html.in | 247 --------- docs/kbase/locking-sanlock.rst | 193 +++++++ docs/kbase/locking.html.in | 48 -- docs/kbase/locking.rst | 33 ++ docs/kbase/rpm-deployment.rst | 410 +++++++++++++++ docs/kbase/secureusage.html.in | 171 ------- docs/kbase/secureusage.rst | 131 +++++ docs/libvirt.css | 76 +-- docs/migration.html.in | 2 +- docs/newapi.xsl | 4 +- docs/page.xsl | 9 +- docs/remote.html.in | 684 +------------------------ docs/styleguide.rst | 66 +++ docs/tlscerts.html.in | 413 +++++++++++++++ docs/uri.html.in | 263 ++++++++-- libvirt.spec.in | 2 + m4/virt-external-programs.m4 | 5 + mingw-libvirt.spec.in | 1 + 30 files changed, 2478 insertions(+), 2250 deletions(-) delete mode 100644 docs/kbase/domainstatecapture.html.in create mode 100644 docs/kbase/domainstatecapture.rst delete mode 100644 docs/kbase/launch_security_sev.html.in create mode 100644 docs/kbase/launch_security_sev.rst delete mode 100644 docs/kbase/locking-lockd.html.in create mode 100644 docs/kbase/locking-lockd.rst delete mode 100644 docs/kbase/locking-sanlock.html.in create mode 100644 docs/kbase/locking-sanlock.rst delete mode 100644 docs/kbase/locking.html.in create mode 100644 docs/kbase/locking.rst create mode 100644 docs/kbase/rpm-deployment.rst delete mode 100644 docs/kbase/secureusage.html.in create mode 100644 docs/kbase/secureusage.rst create mode 100644 docs/styleguide.rst create mode 100644 docs/tlscerts.html.in
Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Michal
participants (2)
-
Daniel P. Berrangé
-
Michal Privoznik