On Mon, Mar 12, 2018 at 13:31:23 +0000, Daniel Berrange wrote:

On Thu, Mar 08, 2018 at 11:12:00AM -0600, Brijesh Singh wrote:

...

QEMU version >= 2.12 provides support for launching an encrypted VMs on AMD x86 platform using Secure Encrypted Virtualization (SEV) feature. This patch adds support to query the SEV capability from the qemu.

Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> --- src/conf/domain_capabilities.h | 13 ++++++++ src/qemu/qemu_capabilities.c | 43 +++++++++++++++++++++++++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_capspriv.h | 4 +++ src/qemu/qemu_monitor.c | 9 ++++++ src/qemu/qemu_monitor.h | 3 ++ src/qemu/qemu_monitor_json.c | 73 ++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 3 ++ 8 files changed, 149 insertions(+)

[...]

...

@@ -4896,6 +4927,12 @@ virQEMUCapsInitQMPMonitor(virQEMUCapsPtr qemuCaps, virQEMUCapsSet(qemuCaps, QEMU_CAPS_MACHINE_PSERIES_MAX_CPU_COMPAT); }

+ /* no way to query -object sev-guest */ + if (ARCH_IS_X86(qemuCaps->arch) && + qemuCaps->version >= 2012000) { + virQEMUCapsSet(qemuCaps, QEMU_CAPS_SEV); + }

Sigh, we really need to fix introspection of -object types one day...

Quick grep-ing found that we are able to probe for 'memory-backend-ram' or 'secret' objects so the 'sev' object should be possible to probe too. You should add test data with the qemu patches applied so that we can verify it. Anyways, we should not push this until it's in upstream qemu.

On 03/12/2018 08:31 AM, Daniel P. Berrangé wrote:

On Thu, Mar 08, 2018 at 11:12:00AM -0600, Brijesh Singh wrote:

...

QEMU version >= 2.12 provides support for launching an encrypted VMs on AMD x86 platform using Secure Encrypted Virtualization (SEV) feature. This patch adds support to query the SEV capability from the qemu.

Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> --- src/conf/domain_capabilities.h | 13 ++++++++ src/qemu/qemu_capabilities.c | 43 +++++++++++++++++++++++++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_capspriv.h | 4 +++ src/qemu/qemu_monitor.c | 9 ++++++ src/qemu/qemu_monitor.h | 3 ++ src/qemu/qemu_monitor_json.c | 73 ++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 3 ++ 8 files changed, 149 insertions(+)

diff --git a/src/conf/domain_capabilities.h b/src/conf/domain_capabilities.h index fa4c1e442f57..83d04d4c8506 100644 --- a/src/conf/domain_capabilities.h +++ b/src/conf/domain_capabilities.h @@ -137,6 +137,19 @@ struct _virDomainCapsCPU { virDomainCapsCPUModelsPtr custom; };

+/* + * SEV capabilities + */ +typedef struct _virSEVCapability virSEVCapability; +typedef virSEVCapability *virSEVCapabilityPtr; +struct _virSEVCapability { + char *pdh; + char *cert_chain; + int cbitpos; + int reduced_phys_bits;

If you have any reason to re-spin this patch series, lets make these two be unsigned int, since IIUC -ve values are not possible.

Will make the changes in next rev.

...

+ /* no way to query -object sev-guest */ + if (ARCH_IS_X86(qemuCaps->arch) && + qemuCaps->version >= 2012000) { + virQEMUCapsSet(qemuCaps, QEMU_CAPS_SEV); + }

Sigh, we really need to fix introspection of -object types one day...

I will take Peter's suggestion and look at -secret object and see how its probed and will do similar thing for sev-guest. Thanks

On Thu, Mar 08, 2018 at 11:12:01AM -0600, Brijesh Singh wrote:

Extend hypervisor capabilities to include sev feature. When available, hypervisor supports launching an encrypted VM on AMD platform. The sev feature tag provides additional details like platform diffie-hellman key and certificate chain which can be used by the guest owner to establish a cryptographic session with the SEV firmware to negotiate keys used for attestation or to provide secret during launch.

Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> --- docs/formatdomaincaps.html.in | 40 ++++++++++++++++++++++++++++++++++++++++ docs/schemas/domaincaps.rng | 20 ++++++++++++++++++++ src/conf/domain_capabilities.c | 20 ++++++++++++++++++++ src/conf/domain_capabilities.h | 1 + src/qemu/qemu_capabilities.c | 2 ++ 5 files changed, 83 insertions(+)

diff --git a/docs/formatdomaincaps.html.in b/docs/formatdomaincaps.html.in index 6bfcaf61caae..f38314166ac3 100644 --- a/docs/formatdomaincaps.html.in +++ b/docs/formatdomaincaps.html.in @@ -417,6 +417,12 @@ <value>3</value> </enum> </gic> + <sev> + <pdh> </pdh> + <cert-chain> </cert-chain> + <cbitpos> </cbitpos> + <reduced-phys-bits> </reduced-phys-bits> + </sev> </features> </domainCapabilities> </pre> @@ -441,5 +447,39 @@ <code>gic</code> element.</dd> </dl>

+ <h4><a id="elementsSEV">SEV capabilities</a></h4> + + <p>AMD Secure Encrypted Virtualization (SEV) capabilities are exposed under + the <code>sev</code> element. + SEV is an extension to the AMD-V architecture which supports running + virtual machines (VMs) under the control of a hypervisor. When supported, + guest owner can create a VM whose memory contents will be transparently + encrypted with a key unique to that VM. + + For more details on SEV feature see: + <a href="https://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf"> + SEV API spec</a> and <a href="http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf"> + SEV White Paper</a> + + </p> + + <dl> + <dt><code>pdh</code></dt> + <dd>Platform diffie-hellman key, which can be exported to remote entities + which wish to establish a secure transport context with the SEV platform + in order to transmit data securely. The key is encoded in base64</dd> + <dt><code>cert-chain</code></dt> + <dd> Platform certificate chain -- which includes platform endorsement key + (PEK), owners certificate authory (OCA) and chip endorsement key (CEK). + The certificate chain is encoded in base64.</dd> + <dt><code>cbitpos</code></dt> + <dd>When memory encryption is enabled, one of the physical address bit + (aka the C-bit) is utilized to mark if a memory page is protected. The + C-bit position is Hypervisor dependent.</dd> + <dt><code>reduced-phys-bits</code></dt> + <dd>When memory encryption is enabled, we loose certain bits in physical + address space. The number of bits we loose is hypervisor dependent.</dd> + </dl> + </body> </html> diff --git a/docs/schemas/domaincaps.rng b/docs/schemas/domaincaps.rng index 39053181eb9a..53b33eb83c1f 100644 --- a/docs/schemas/domaincaps.rng +++ b/docs/schemas/domaincaps.rng @@ -173,6 +173,9 @@ <element name='features'> <interleave> <ref name='gic'/> + <optional> + <ref name='sev'/> + </optional> </interleave> </element> </define> @@ -184,6 +187,23 @@ </element> </define>

+ <define name='sev'> + <element name='sev'> + <element name='pdh'> + <data type='string'/> + </element> + <element name='cert-chain'> + <data type='string'/> + </element> + <element name='cbitpos'> + <data type='unsignedInt'/> + </element> + <element name='reduced-phys-bits'> + <data type='unsignedInt'/> + </element> + </element> + </define> + <define name='value'> <zeroOrMore> <element name='value'> diff --git a/src/conf/domain_capabilities.c b/src/conf/domain_capabilities.c index f7d9be50f82d..082065fb4733 100644 --- a/src/conf/domain_capabilities.c +++ b/src/conf/domain_capabilities.c @@ -549,6 +549,25 @@ virDomainCapsFeatureGICFormat(virBufferPtr buf, FORMAT_EPILOGUE(gic); }

+static void +virDomainCapsFeatureSEVFormat(virBufferPtr buf, + virSEVCapabilityPtr const sev) +{ + if (!sev) + return; + + virBufferAddLit(buf, "<sev supported='yes'>\n"); + virBufferAdjustIndent(buf, 2); + virBufferAsprintf(buf, "<cbitpos>%d</cbitpos>\n", sev->cbitpos); + virBufferAsprintf(buf, "<reduced-phys-bits>%d</reduced-phys-bits>\n", + sev->reduced_phys_bits); + virBufferAsprintf(buf, "<pdh>%s</pdh>\n", sev->pdh); + virBufferAsprintf(buf, "<cert-chain>%s</cert-chain>\n", + sev->cert_chain);

I wonder what length 'cert_chain' is going to be typically ? I hope we don't cause ourselves any trouble by having the data inline in the main capabilities XML ?

+ virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "</sev>\n"); +} +

char * virDomainCapsFormat(virDomainCapsPtr const caps) @@ -587,6 +606,7 @@ virDomainCapsFormat(virDomainCapsPtr const caps) virBufferAdjustIndent(&buf, 2);

virDomainCapsFeatureGICFormat(&buf, &caps->gic); + virDomainCapsFeatureSEVFormat(&buf, caps->sev);

virBufferAdjustIndent(&buf, -2); virBufferAddLit(&buf, "</features>\n"); diff --git a/src/conf/domain_capabilities.h b/src/conf/domain_capabilities.h index 83d04d4c8506..2239566105fd 100644 --- a/src/conf/domain_capabilities.h +++ b/src/conf/domain_capabilities.h @@ -170,6 +170,7 @@ struct _virDomainCaps { /* add new domain devices here */

virDomainCapsFeatureGIC gic; + virSEVCapabilityPtr sev; /* add new domain features here */ };

diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 68e3622a3963..f56e88032c06 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -5975,6 +5975,8 @@ virQEMUCapsFillDomainCaps(virCapsPtr caps, virQEMUCapsFillDomainDeviceHostdevCaps(qemuCaps, hostdev) < 0 || virQEMUCapsFillDomainFeatureGICCaps(qemuCaps, domCaps) < 0) return -1; + + domCaps->sev = qemuCaps->sevCapabilities; return 0; }

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, Mar 12, 2018 at 10:25:44AM -0500, Brijesh Singh wrote:

On 03/12/2018 08:33 AM, Daniel P. Berrangé wrote:

...

On Thu, Mar 08, 2018 at 11:12:01AM -0600, Brijesh Singh wrote:

...

Extend hypervisor capabilities to include sev feature. When available, hypervisor supports launching an encrypted VM on AMD platform. The sev feature tag provides additional details like platform diffie-hellman key and certificate chain which can be used by the guest owner to establish a cryptographic session with the SEV firmware to negotiate keys used for attestation or to provide secret during launch.

Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> --- docs/formatdomaincaps.html.in | 40 ++++++++++++++++++++++++++++++++++++++++ docs/schemas/domaincaps.rng | 20 ++++++++++++++++++++ src/conf/domain_capabilities.c | 20 ++++++++++++++++++++ src/conf/domain_capabilities.h | 1 + src/qemu/qemu_capabilities.c | 2 ++ 5 files changed, 83 insertions(+)

diff --git a/docs/formatdomaincaps.html.in b/docs/formatdomaincaps.html.in index 6bfcaf61caae..f38314166ac3 100644 --- a/docs/formatdomaincaps.html.in +++ b/docs/formatdomaincaps.html.in @@ -417,6 +417,12 @@ <value>3</value> </enum> </gic> + <sev> + <pdh> </pdh> + <cert-chain> </cert-chain> + <cbitpos> </cbitpos> + <reduced-phys-bits> </reduced-phys-bits> + </sev> </features> </domainCapabilities> </pre> @@ -441,5 +447,39 @@ <code>gic</code> element.</dd> </dl> + <h4><a id="elementsSEV">SEV capabilities</a></h4> + + <p>AMD Secure Encrypted Virtualization (SEV) capabilities are exposed under + the <code>sev</code> element. + SEV is an extension to the AMD-V architecture which supports running + virtual machines (VMs) under the control of a hypervisor. When supported, + guest owner can create a VM whose memory contents will be transparently + encrypted with a key unique to that VM. + + For more details on SEV feature see: + <a href="https://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf"> + SEV API spec</a> and <a href="http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf"> + SEV White Paper</a> + + </p> + + <dl> + <dt><code>pdh</code></dt> + <dd>Platform diffie-hellman key, which can be exported to remote entities + which wish to establish a secure transport context with the SEV platform + in order to transmit data securely. The key is encoded in base64</dd> + <dt><code>cert-chain</code></dt> + <dd> Platform certificate chain -- which includes platform endorsement key + (PEK), owners certificate authory (OCA) and chip endorsement key (CEK). + The certificate chain is encoded in base64.</dd> + <dt><code>cbitpos</code></dt> + <dd>When memory encryption is enabled, one of the physical address bit + (aka the C-bit) is utilized to mark if a memory page is protected. The + C-bit position is Hypervisor dependent.</dd> + <dt><code>reduced-phys-bits</code></dt> + <dd>When memory encryption is enabled, we loose certain bits in physical + address space. The number of bits we loose is hypervisor dependent.</dd> + </dl> + </body> </html> diff --git a/docs/schemas/domaincaps.rng b/docs/schemas/domaincaps.rng index 39053181eb9a..53b33eb83c1f 100644 --- a/docs/schemas/domaincaps.rng +++ b/docs/schemas/domaincaps.rng @@ -173,6 +173,9 @@ <element name='features'> <interleave> <ref name='gic'/> + <optional> + <ref name='sev'/> + </optional> </interleave> </element> </define> @@ -184,6 +187,23 @@ </element> </define> + <define name='sev'> + <element name='sev'> + <element name='pdh'> + <data type='string'/> + </element> + <element name='cert-chain'> + <data type='string'/> + </element> + <element name='cbitpos'> + <data type='unsignedInt'/> + </element> + <element name='reduced-phys-bits'> + <data type='unsignedInt'/> + </element> + </element> + </define> + <define name='value'> <zeroOrMore> <element name='value'> diff --git a/src/conf/domain_capabilities.c b/src/conf/domain_capabilities.c index f7d9be50f82d..082065fb4733 100644 --- a/src/conf/domain_capabilities.c +++ b/src/conf/domain_capabilities.c @@ -549,6 +549,25 @@ virDomainCapsFeatureGICFormat(virBufferPtr buf, FORMAT_EPILOGUE(gic); } +static void +virDomainCapsFeatureSEVFormat(virBufferPtr buf, + virSEVCapabilityPtr const sev) +{ + if (!sev) + return; + + virBufferAddLit(buf, "<sev supported='yes'>\n"); + virBufferAdjustIndent(buf, 2); + virBufferAsprintf(buf, "<cbitpos>%d</cbitpos>\n", sev->cbitpos); + virBufferAsprintf(buf, "<reduced-phys-bits>%d</reduced-phys-bits>\n", + sev->reduced_phys_bits); + virBufferAsprintf(buf, "<pdh>%s</pdh>\n", sev->pdh); + virBufferAsprintf(buf, "<cert-chain>%s</cert-chain>\n", + sev->cert_chain);

I wonder what length 'cert_chain' is going to be typically ? I hope we don't cause ourselves any trouble by having the data inline in the main capabilities XML ?

The PDH is 2K and cert-chain length is 8K. A quick google indicates that a maximum data length in XML can be up to 64000 bytes.

64kb is a fairly arbitrary cap we place on it in the RPC protocol, but we have freedom to extend it. I wonder if putting 8k of certs into the domain capabilities XML is good idea though, even if possible. Does anyone else have an opinion about that, or should we perhaps have a separate API to fetch the PDH / Cert data, and use the domain capabilities merely to indicate existance of the SEV feature (and the bit field stuff) Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Thu, Mar 08, 2018 at 11:12:02AM -0600, Brijesh Singh wrote:

The launch-security element can be used to define the security model to use when launching a domain. Currently we support 'sev'.

When 'sev' is used, the VM will be launched with AMD SEV feature enabled. SEV feature supports running encrypted VM under the control of KVM. Encrypted VMs have their pages (code and data) secured such that only the guest itself has access to the unencrypted version. Each encrypted VM is associated with a unique encryption key; if its data is accessed to a different entity using a different key the encrypted guests data will be incorrectly decrypted, leading to unintelligible data.

Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> --- docs/formatdomain.html.in | 120 ++++++++++++++++++++++++++++++++++++++++++ docs/schemas/domaincommon.rng | 39 ++++++++++++++ src/conf/domain_conf.c | 111 ++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 27 ++++++++++ 4 files changed, 297 insertions(+)

diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 6fd2189cd2f4..830d2a3c59be 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -8195,6 +8195,126 @@ qemu-kvm -net nic,model=? /dev/null

<p>Note: DEA/TDEA is synonymous with DES/TDES.</p>

+ <h3><a id="sev">Secure Encrypted Virtualization (SEV)</a></h3> + + <p> + The contents of the <code><launch-security type='sev'></code> element + is used to provide the guest owners input used for creating an encrypted + VM using the AMD SEV feature. + + SEV is an extension to the AMD-V architecture which supports running + encrypted virtual machine (VMs) under the control of KVM. Encrypted + VMs have their pages (code and data) secured such that only the guest + itself has access to the unencrypted version. Each encrypted VM is + associated with a unique encryption key; if its data is accessed to a + different entity using a different key the encrypted guests data will + be incorrectly decrypted, leading to unintelligible data. + + For more information see various input parameters and its format see SEV API spec + <a href="https://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf"> https://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf </a> + <span class="since">Since 4.2.0</span> + </p> + <pre> +<domain> + ... + <launch-security type='sev'> + <policy> 0 </policy> + <cbitpos> 47 </cbitpos> + <reduced-phys-bits> 5 </reduced-phys-bits> + <session> ... </session> + <dh-cert> ... </dh> + </sev> + ... +</domain> +</pre> + + <p> + A least <code>cbitpos</code> and <code>reduced-phys-bits</code> must be + nested within the <code>launch-security</code> element. + </p> + <dl> + <dt><code>cbitpos</code></dt> + <dd>The <code>cbitpos</code> element provides the C-bit (aka encryption bit) + location in guest page table entry. The value of <code>cbitpos</code> is + hypervisor dependent and can be obtained through the <code>sev</code> element + from domaincapabilities. + </dd> + <dt><code>reduced-phys-bits</code></dt> + <dd>The <code>reduced-phys-bits</code> element provides the physical + address bit reducation. Similar to <code>cbitpos</code> the value of <code> + reduced-phys-bit</code> is hypervisor dependent and can be obtained + through the <code>sev</code> element from domaincapabilities. + </dd> + <dt><code>policy</code></dt> + <dd>The optional <code>policy</code> element provides the guest policy + which must be maintained by the SEV firmware. This policy is enforced by + the firmware and restricts what configuration and operational commands + can be performed on this guest by the hypervisor. The guest policy + provided during guest launch is bound to the guest and cannot be changed + throughout the lifetime of the guest. The policy is also transmitted + during snapshot and migration flows and enforced on the destination platform. + + The guest policy is a 4-byte structure with the fields shown in Table: + + <table class="top_table"> + <tr> + <th> Bit(s) </th> + <th> Description </th> + </tr> + <tr> + <td> 0 </td> + <td> Debugging of the guest is disallowed when set </td> + </tr> + <tr> + <td> 1 </td> + <td> Sharing keys with other guests is disallowed when set </td> + </tr> + <tr> + <td> 2 </td> + <td> SEV-ES is required when set</td> + </tr> + <tr> + <td> 3 </td> + <td> Sending the guest to another platform is disallowed when set</td> + </tr> + <tr> + <td> 4 </td> + <td> The guest must not be transmitted to another platform that is + not in the domain when set. </td> + </tr> + <tr> + <td> 5 </td> + <td> The guest must not be transmitted to another platform that is + not SEV capable when set. </td> + </tr> + <tr> + <td> 15:6 </td> + <td> reserved </td> + </tr> + <tr> + <td> 16:32 </td> + <td> The guest must not be transmitted to another platform with a + lower firmware version. </td> + </tr> + </table> + Default value is 0x1 + + </dd> + <dt><code>dh-cert</code></dt> + <dd>The optional <code>dh-cert</code> element provides the guest owners public + Diffie-Hellman (DH) key. The key is used to negotiate a master secret + key between the SEV firmware and guest owner. This master secret key is + then used to establish a trusted channel between SEV firmware and guest + owner. The value must be encoded in base64. + </dd> + <dt><code>session</code></dt> + <dd>The optional <code>session</code> element provides the guest owners + session blob defined in SEV API spec. The value must be encoded in base64. + + See SEV spec LAUNCH_START section for session blob format. + </dd> + </dl> + <h2><a id="examples">Example configs</a></h2>

<p> diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 8165e699d67e..e3dcc69067c2 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -77,6 +77,9 @@ <optional> <ref name='keywrap'/> </optional> + <optional> + <ref name='launch-security'/> + </optional> </interleave> </element> </define> @@ -436,6 +439,42 @@ </element> </define>

+ <define name="launch-security"> + <element name="launch-security"> + <attribute name="type"> + <value>sev</value> + </attribute> + <interleave> + <element name="cbitpos"> + <data type='unsignedInt'/> + </element> + <element name="reduced-phys-bits"> + <data type='unsignedInt'/> + </element> + <optional> + <element name="policy"> + <ref name='hexuint'/> + </element> + </optional> + <optional> + <element name="handle"> + <ref name='unsignedInt'/> + </element> + </optional> + <optional> + <element name="dh-cert"> + <data type="string"/> + </element> + </optional> + <optional> + <element name="session"> + <data type="string"/> + </element> + </optional> + </interleave> + </element> + </define> + <!-- Enable or disable perf events for the domain. For each of the events the following rules apply: diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index fcafc8b2fafe..52d34de69d45 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -929,6 +929,10 @@ VIR_ENUM_IMPL(virDomainShmemModel, VIR_DOMAIN_SHMEM_MODEL_LAST, "ivshmem-plain", "ivshmem-doorbell")

+VIR_ENUM_IMPL(virDomainLaunchSecurity, VIR_DOMAIN_LAUNCH_SECURITY_LAST, + "", + "sev") + static virClassPtr virDomainObjClass; static virClassPtr virDomainXMLOptionClass; static void virDomainObjDispose(void *obj); @@ -2897,6 +2901,14 @@ virDomainCachetuneDefFree(virDomainCachetuneDefPtr cachetune) VIR_FREE(cachetune); }

+static void +virDomainSevDefFree(virDomainSevDefPtr def) +{ + VIR_FREE(def->dh_cert); + VIR_FREE(def->session); + + VIR_FREE(def); +}

void virDomainDefFree(virDomainDefPtr def) { @@ -3079,6 +3091,9 @@ void virDomainDefFree(virDomainDefPtr def) if (def->namespaceData && def->ns.free) (def->ns.free)(def->namespaceData);

+ if (def->sev) + virDomainSevDefFree(def->sev); + xmlFreeNode(def->metadata);

VIR_FREE(def); @@ -15539,6 +15554,71 @@ virDomainMemoryTargetDefParseXML(xmlNodePtr node, return ret; }

+static virDomainSevDefPtr +virDomainSevDefParseXML(xmlNodePtr sevNode, + xmlXPathContextPtr ctxt) +{ + char *tmp = NULL, *type = NULL; + xmlNodePtr save = ctxt->node; + virDomainSevDefPtr def; + unsigned long policy; + + ctxt->node = sevNode; + + if (VIR_ALLOC(def) < 0) + return NULL; + + if (!(type = virXMLPropString(sevNode, "type"))) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("missing launch-security type")); + goto error; + } + + if (virDomainLaunchSecurityTypeFromString(type) != + VIR_DOMAIN_LAUNCH_SECURITY_SEV) { + goto error; + } + + if (virXPathInt("string(./cbitpos)", ctxt, &def->cbitpos) < 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", _("failed to get cbitpos")); + goto error; + } + + if (virXPathInt("string(./reduced-phys-bits)", ctxt, + &def->reduced_phys_bits) < 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("failed to get reduced-phys-bits")); + goto error; + } + + if (virXPathULongHex("string(./policy)", ctxt, &policy) < 0) + policy = 0x1; + + def->policy = policy; + + if ((tmp = virXPathString("string(./dh-cert)", ctxt))) { + if (VIR_STRDUP(def->dh_cert, tmp) < 0) + goto error; + + VIR_FREE(tmp); + } + + if ((tmp = virXPathString("string(./session)", ctxt))) { + if (VIR_STRDUP(def->session, tmp) < 0) + goto error; + + VIR_FREE(tmp); + } + + ctxt->node = save; + return def; + + error: + VIR_FREE(tmp); + virDomainSevDefFree(def); + ctxt->node = save; + return NULL; +}

static virDomainMemoryDefPtr virDomainMemoryDefParseXML(virDomainXMLOptionPtr xmlopt, @@ -20212,6 +20292,13 @@ virDomainDefParseXML(xmlDocPtr xml, ctxt->node = node; VIR_FREE(nodes);

+ /* Check for SEV feature */ + if ((node = virXPathNode("./launch-security", ctxt)) != NULL) { + def->sev = virDomainSevDefParseXML(node, ctxt); + if (!def->sev) + goto error; + } + /* analysis of memory devices */ if ((n = virXPathNodeSet("./devices/memory", ctxt, &nodes)) < 0) goto error; @@ -26102,6 +26189,27 @@ virDomainKeyWrapDefFormat(virBufferPtr buf, virDomainKeyWrapDefPtr keywrap) virBufferAddLit(buf, "</keywrap>\n"); }

+static void +virDomainSevDefFormat(virBufferPtr buf, virDomainSevDefPtr sev) +{ + virBufferAddLit(buf, "<launch-security type='sev'>\n"); + virBufferAdjustIndent(buf, 2); + + virBufferAsprintf(buf, "<cbitpos>%d</cbitpos>\n", sev->cbitpos); + virBufferAsprintf(buf, "<reduced-phys-bits>%d</reduced-phys-bits>\n", + sev->reduced_phys_bits); + virBufferAsprintf(buf, "<policy>%d</policy>\n", sev->policy); + if (sev->dh_cert) + virBufferAsprintf(buf, "<dh_cert>%s</dh_cert>\n", sev->dh_cert); + + if (sev->session) + virBufferAsprintf(buf, "<session>%s</session>\n", sev->session); + + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "</launch-security>\n"); +} + + static void virDomainPerfDefFormat(virBufferPtr buf, virDomainPerfDefPtr perf) { @@ -27274,6 +27382,9 @@ virDomainDefFormatInternal(virDomainDefPtr def, if (def->keywrap) virDomainKeyWrapDefFormat(buf, def->keywrap);

+ if (def->sev) + virDomainSevDefFormat(buf, def->sev); + virBufferAdjustIndent(buf, -2); virBufferAddLit(buf, "</domain>\n");

diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 368f16f3fbf9..ff625a0aff1b 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -142,6 +142,9 @@ typedef virDomainPanicDef *virDomainPanicDefPtr; typedef struct _virDomainMemoryDef virDomainMemoryDef; typedef virDomainMemoryDef *virDomainMemoryDefPtr;

+typedef struct _virDomainSevDef virDomainSevDef; +typedef virDomainSevDef *virDomainSevDefPtr; + /* forward declarations virDomainChrSourceDef, required by * virDomainNetDef */ @@ -2289,6 +2292,26 @@ struct _virDomainKeyWrapDef { int dea; /* enum virTristateSwitch */ };

+typedef enum { + VIR_DOMAIN_LAUNCH_SECURITY_NONE, + VIR_DOMAIN_LAUNCH_SECURITY_SEV, + + VIR_DOMAIN_LAUNCH_SECURITY_LAST, +} virDomainLaunchSecurity; + +typedef struct _virDomainSevDef virDomainSevDef; +typedef virDomainSevDef *virDomainSevDefPtr; + +struct _virDomainSevDef { + char *dh_cert; + char *session; + char *configDir; + int policy; + int cbitpos; + int reduced_phys_bits; +};

I'd suggested 'unsigned int' for these 3 variables, assuming they don't ever need -ve values. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Thu, Mar 08, 2018 at 11:12:03AM -0600, Brijesh Singh wrote:

QEMU >= 2.12 provides 'sev-guest' object which is used to launch encrypted VMs on AMD platform using SEV feature. The various inputs required to launch SEV guest is provided through the <launch-security> tag. A typical SEV guest launch command line looks like this:

# $QEMU ...\ -object sev-guest,id=sev0,cbitpos=47,reduced-phys-bits=5 ...\ -machine memory-encryption=sev0 \

Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> --- src/qemu/qemu_command.c | 33 ++++++++++++++++++ src/qemu/qemu_process.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 124 insertions(+)

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index fa0aa5d5c3d4..39f136a389cb 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -9663,6 +9663,36 @@ qemuBuildTPMCommandLine(virCommandPtr cmd, return 0; }

+static void +qemuBuildSevCommandLine(virCommandPtr cmd, + virDomainSevDefPtr sev) +{ + virBuffer obj = VIR_BUFFER_INITIALIZER; + char *path = NULL; + + VIR_DEBUG("policy=0x%x cbitpos=%d reduced_phys_bits=%d", + sev->policy, sev->cbitpos, sev->reduced_phys_bits); + + virCommandAddArgList(cmd, "-machine", "memory-encryption=sev0", NULL);

Hmm, so this will result in us passing '-machine' many times, and relying on QEMU to acculate the args. It should work, but IME it could be a little surprising to people debuggnig qemu to see repeated args for -machine. Could we just call this method from the place that constructs the current -machine arg so we append to that.

+ + virBufferAsprintf(&obj, "sev-guest,id=sev0,cbitpos=%d", sev->cbitpos); + virBufferAsprintf(&obj, ",reduced-phys-bits=%d", sev->reduced_phys_bits); + virBufferAsprintf(&obj, ",policy=0x%x", sev->policy); + + if (sev->dh_cert) { + ignore_value(virAsprintf(&path, "%s/dh_cert.base64", sev->configDir)); + virBufferAsprintf(&obj, ",dh-cert-file=%s", path); + VIR_FREE(path); + } + + if (sev->session) { + ignore_value(virAsprintf(&path, "%s/session.base64", sev->configDir)); + virBufferAsprintf(&obj, ",session-file=%s", path); + VIR_FREE(path); + } + + virCommandAddArgList(cmd, "-object", virBufferContentAndReset(&obj), NULL); +}

static int qemuBuildVMCoreInfoCommandLine(virCommandPtr cmd, @@ -10108,6 +10138,9 @@ qemuBuildCommandLine(virQEMUDriverPtr driver, if (qemuBuildVMCoreInfoCommandLine(cmd, def, qemuCaps) < 0) goto error;

+ if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_SEV) && def->sev) + qemuBuildSevCommandLine(cmd, def->sev); + if (snapshot) virCommandAddArgList(cmd, "-loadvm", snapshot->def->name, NULL);

diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 57c06c7c1550..349e12b6dc12 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -3457,6 +3457,16 @@ qemuProcessBuildDestroyMemoryPathsImpl(virQEMUDriverPtr driver, }

+static void +qemuProcessDestroySevPaths(virDomainSevDefPtr sev) +{ + if (!sev) + return; + + virFileDeleteTree(sev->configDir);

Just delete the individual files we created, no recursive delete please.

+ VIR_FREE(sev->configDir); +} + int qemuProcessBuildDestroyMemoryPaths(virQEMUDriverPtr driver, virDomainObjPtr vm, @@ -5741,6 +5751,83 @@ qemuProcessPrepareDomain(virQEMUDriverPtr driver, return ret; }

+static int +qemuBuildSevCreateFile(const char *configDir, const char *name, + const char *data) +{ + char *configFile; + + if (!(configFile = virFileBuildPath(configDir, name, ".base64"))) + return -1; + + if (virFileRewriteStr(configFile, S_IRUSR | S_IWUSR, data) < 0) { + virReportSystemError(errno, _("failed to write data to config '%s'"), + configFile); + goto error; + } + + VIR_FREE(configFile); + return 0; + + error: + VIR_FREE(configFile); + return -1; +} + +static int +qemuProcessPrepareSevGuestInput(virQEMUDriverPtr driver, + virDomainObjPtr vm) +{ + qemuDomainObjPrivatePtr priv = vm->privateData; + virDomainDefPtr def = vm->def; + virQEMUCapsPtr qemuCaps = priv->qemuCaps; + virDomainSevDefPtr sev = def->sev; + char *configDir = NULL; + char *domPath = virDomainDefGetShortName(def); + virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver); + + if (!sev) + return 0; + + VIR_DEBUG("Prepare SEV guest"); + + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_SEV)) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Domain %s asked for 'sev' launch but " + "QEMU does not support SEV feature"), vm->def->name); + return -1; + } + + if (virAsprintf(&configDir, "%s/sev/%s", cfg->configDir, domPath) < 0) + goto error;

We already have a directory where we put transient things like the QEMU monitor socket, pid file, etc. This is defineed by 'priv->libDir', so just use that directory and don't create extra sub-dirs.

+ if (virFileMakePathWithMode(configDir, S_IRWXU) < 0) { + virReportSystemError(errno, _("cannot create config directory '%s'"), + configDir); + goto error; + } + + if (sev->dh_cert) { + if (qemuBuildSevCreateFile(configDir, "dh_cert", sev->dh_cert) < 0) + goto error1; + } + + if (sev->session) { + if (qemuBuildSevCreateFile(configDir, "session", sev->session) < 0) + goto error1; + } + + VIR_FREE(domPath); + sev->configDir = configDir; + return 0; + + error1: + virFileDeleteTree(configDir); + error: + VIR_FREE(configDir); + VIR_FREE(domPath); + return -1; +}

static int qemuProcessPrepareHostStorage(virQEMUDriverPtr driver, @@ -5866,6 +5953,9 @@ qemuProcessPrepareHost(virQEMUDriverPtr driver, if (qemuProcessPrepareHostStorage(driver, vm, flags) < 0) goto cleanup;

+ if (qemuProcessPrepareSevGuestInput(driver, vm) < 0) + goto cleanup; + ret = 0; cleanup: virObjectUnref(cfg); @@ -6535,6 +6625,7 @@ void qemuProcessStop(virQEMUDriverPtr driver, }

qemuProcessBuildDestroyMemoryPaths(driver, vm, NULL, false); + qemuProcessDestroySevPaths(vm->def->sev);

vm->def->id = -1;

-- 2.14.3

Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 03/12/2018 07:01 AM, Daniel P. Berrangé wrote:

On Thu, Mar 08, 2018 at 11:12:04AM -0600, Brijesh Singh wrote:

...

The API can be used outside the libvirt to get the launch security information. When SEV is enabled, the API can be used to get the measurement of the launch process.

Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> --- include/libvirt/libvirt-domain.h | 17 ++++++++++++++ src/driver-hypervisor.h | 7 ++++++ src/libvirt-domain.c | 50 ++++++++++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 5 ++++ 4 files changed, 79 insertions(+)

diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 4048acf38aaf..11c3fec92bfa 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -4756,4 +4756,21 @@ int virDomainSetLifecycleAction(virDomainPtr domain, unsigned int action, unsigned int flags);

+/** + * Launch Security API + */ + +/** + * VIR_DOMAIN_LAUNCH_SECURITY_SEV_MEASUREMENT: + * + * Macro represents the launch measurement of the SEV guest, + * as VIR_TYPED_PARAM_STRING. + */ +#define VIR_DOMAIN_LAUNCH_SECURITY_SEV_MEASUREMENT "sev-measurement" + +int virDomainGetLaunchSecurityInfo(virDomainPtr domain, + virTypedParameterPtr params, + int *nparams, + unsigned int flags);

These days we prefer new APIs to use

virTypedParameterPtr *params,

and have the API implementation allocate the right number of elements for the array, so the caller doesn't have to allocate anything itself - only free the returned memory.

See virDomainGetJobStats for an example.

Ah okay, I will take a look and change the API to allocate the array elements instead of forcing caller to allocate the arrays.

...

diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index ce0e2b252552..dc4873a8ad1c 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -1283,6 +1283,12 @@ typedef int unsigned int action, unsigned int flags);

+typedef int +(*virDrvDomainGetLaunchSecurityInfo)(virDomainPtr domain, + virTypedParameterPtr params, + int *nparams, + unsigned int flags); +

typedef struct _virHypervisorDriver virHypervisorDriver; typedef virHypervisorDriver *virHypervisorDriverPtr; @@ -1528,6 +1534,7 @@ struct _virHypervisorDriver { virDrvDomainSetVcpu domainSetVcpu; virDrvDomainSetBlockThreshold domainSetBlockThreshold; virDrvDomainSetLifecycleAction domainSetLifecycleAction; + virDrvDomainGetLaunchSecurityInfo domainGetLaunchSecurityInfo; };

diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index eaec0979ad49..21356bb92894 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -12095,3 +12095,53 @@ int virDomainSetLifecycleAction(virDomainPtr domain, virDispatchError(domain->conn); return -1; } + +/** + * virDomainGetLaunchSecurityInfo: + * @domain: a domain object + * @params: where to store security info + * @nparams: number of items in @params + * @flags: bitwise-OR of virDomainModificationImpact

This API doesn't use virDomainModificationImpact. So just say

@flags currently used, set to 0

Noted. thanks

...

+ * + * Get the launch security info. In case of the SEV guest, this will + * return the launch measurement. + * + * Returns -1 in case of failure, 0 in case of success. + */ +int virDomainGetLaunchSecurityInfo(virDomainPtr domain, + virTypedParameterPtr params, + int *nparams, + unsigned int flags) +{ + virConnectPtr conn; + + VIR_DOMAIN_DEBUG(domain, "params=%p, nparams=%p flags=0x%x", + params, nparams, flags); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + virCheckNonNegativeArgGoto(*nparams, error); + if (*nparams != 0) + virCheckNonNullArgGoto(params, error);

We should require both args to be non-null, when we do allocation ourselves.

I think we probably want to forbid this method for read-only users, so add

virCheckReadOnlyGoto(conn->flags, error);

Okay, got it.

...

+ + if (VIR_DRV_SUPPORTS_FEATURE(domain->conn->driver, domain->conn, + VIR_DRV_FEATURE_TYPED_PARAM_STRING)) + flags |= VIR_TYPED_PARAM_STRING_OKAY; + + conn = domain->conn; + + if (conn->driver->domainGetLaunchSecurityInfo) { + int ret; + ret = conn->driver->domainGetLaunchSecurityInfo(domain, params, + nparams, flags); + if (ret < 0) + goto error; + return ret; + } + virReportUnsupportedError(); + + error: + virDispatchError(domain->conn); + return -1; +} diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 95df3a0dbc7b..caba2862d371 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -785,4 +785,9 @@ LIBVIRT_4.1.0 { virStoragePoolLookupByTargetPath; } LIBVIRT_3.9.0;

+LIBVIRT_4.2.0 { + global: + virDomainGetLaunchSecurityInfo; +} LIBVIRT_4.1.0; + # .... define new API here using predicted next version number .... -- 2.14.3

Regards, Daniel