On a Wednesday in 2022, Daniel P. Berrangé wrote: [...] --tk [...] The license here says GPLv2+, but LGPL-2.1+ in the SPDX identifier below. [...] *minor Reviewed-by: Ján Tomko <jtomko(a)redhat.com&gt; Jano

When connected to libvirt we can validate that the guest configuration has the kernel hashes property enabled, otherwise including the kernel GUID table in our expected measurements is not likely to match the actual measurement. When running locally we can also automatically detect the kernel/initrd paths, along with the cmdline string from the XML. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- docs/manpages/virt-qemu-sev-validate.rst | 9 ++++ tools/virt-qemu-sev-validate | 59 ++++++++++++++++++++++++ 2 files changed, 68 insertions(+) diff --git a/docs/manpages/virt-qemu-sev-validate.rst b/docs/manpages/virt-qemu-sev-validate.rst index 0aee1db76d..8fa1452c5c 100644 --- a/docs/manpages/virt-qemu-sev-validate.rst +++ b/docs/manpages/virt-qemu-sev-validate.rst @@ -265,6 +265,15 @@ Validate the measurement of a SEV guest booting from disk: --tk this-guest-tk.bin \ --domain fedora34x86_64 +Validate the measurement of a SEV guest with direct kernel boot: + +:: + + # virt-dom-sev-validate \ + --insecure \ + --tk this-guest-tk.bin \ + --domain fedora34x86_64 + EXIT STATUS =========== diff --git a/tools/virt-qemu-sev-validate b/tools/virt-qemu-sev-validate index deef34628b..2c696ba04c 100755 --- a/tools/virt-qemu-sev-validate +++ b/tools/virt-qemu-sev-validate @@ -295,6 +295,35 @@ class LibvirtConfidentialVM(ConfidentialVM): raise IncorrectConfigException( "Domain must have one firmware path") + measure_kernel_nodes = doc.xpath( + "/domain/launchSecurity[@type='sev']/@kernelHashes") + measure_kernel = False + if len(measure_kernel_nodes) == 1: + if measure_kernel_nodes[0] == "yes": + measure_kernel = True + + xp_kernel = "/domain/os/kernel" + xp_initrd = "/domain/os/initrd" + xp_cmdline = "/domain/os/cmdline" + kern_nodes = (doc.xpath(xp_kernel) + + doc.xpath(xp_initrd) + + doc.xpath(xp_cmdline)) + if not measure_kernel: + if len(self.kernel_table.entries()) != 0: + raise UnsupportedUsageException( + "kernel/initrd/cmdline provided but kernel " + "measurement not enabled") + + # Check for an insecure scenario + if len(kern_nodes) != 0 and secure: + raise InsecureUsageException( + "direct kernel boot present without measurement") + else: + if len(kern_nodes) == 0: + raise IncorrectConfigException( + "kernel/initrd/cmdline not provided but kernel " + "measurement is enabled") + def load_domain(self, uri, id_name_uuid, secure, ignore_config): self.conn = libvirt.open(uri) @@ -363,6 +392,36 @@ class LibvirtConfidentialVM(ConfidentialVM): self.load_firmware(loadernodes[0].text) + if self.kernel_table.kernel is None: + kernelnodes = doc.xpath("/domain/os/kernel") + if len(kernelnodes) != 0: + if remote: + raise UnsupportedUsageException( + "Cannot access kernel path remotely") + if secure: + raise InsecureUsageException( + "Using kernel path from XML is not secure") + self.kernel_table.load_kernel(kernelnodes[0].text) + + if self.kernel_table.initrd is None: + initrdnodes = doc.xpath("/domain/os/initrd") + if len(initrdnodes) != 0: + if remote: + raise UnsupportedUsageException( + "Cannot access initrd path remotely") + if secure: + raise InsecureUsageException( + "Using initrd path from XML is not secure") + self.kernel_table.load_initrd(initrdnodes[0].text) + + if self.kernel_table.cmdline is None: + cmdlinenodes = doc.xpath("/domain/os/cmdline") + if len(cmdlinenodes) != 0: + if secure: + raise InsecureUsageException( + "Using cmdline string from XML is not secure") + self.kernel_table.load_cmdline(cmdlinenodes[0].text) + def parse_command_line(): parser = argparse.ArgumentParser( -- 2.37.3

With the SEV-ES policy the VMSA state of each vCPU must be included in the measured data. The VMSA state can be generated using the 'sevctl' tool, by telling it a QEMU VMSA is required, and passing the hypevisor's CPU SKU (family, model, stepping). Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- docs/manpages/virt-qemu-sev-validate.rst | 58 ++++++++++++++++++++ tools/virt-qemu-sev-validate | 69 ++++++++++++++++++++++-- 2 files changed, 124 insertions(+), 3 deletions(-) diff --git a/docs/manpages/virt-qemu-sev-validate.rst b/docs/manpages/virt-qemu-sev-validate.rst index 8fa1452c5c..9f86212cb9 100644 --- a/docs/manpages/virt-qemu-sev-validate.rst +++ b/docs/manpages/virt-qemu-sev-validate.rst @@ -116,6 +116,23 @@ content if omitted. String containing any kernel command line parameters used during boot of the domain. Defaults to the empty string if omitted. +``-n COUNT``, ``--num-cpus=COUNT`` + +The number of virtual CPUs for the domain. This is required when the +domain policy is set to require SEV-ES. + +``-0 PATH``, ``--vmsa-cpu0=PATH`` + +Path to the VMSA initial state for the boot CPU. This is required when +the domain policy is set to require SEV-ES. The file contents must be +exactly 4096 bytes in length. + +``-1 PATH``, ``--vmsa-cpu1=PATH`` + +Path to the VMSA initial state for the non-boot CPU. This is required when +the domain policy is set to require SEV-ES and the domain has more than one +CPU present. The file contents must be exactly 4096 bytes in length. + ``--tik PATH`` TIK file for domain. This file must be exactly 16 bytes in size and contains the @@ -212,6 +229,22 @@ Validate the measurement of a SEV guest with direct kernel boot: --build-id 13 \ --policy 3 +Validate the measurement of a SEV-ES SMP guest booting from disk: + +:: + + # virt-dom-sev-validate \ + --firmware OVMF.sev.fd \ + --num-cpus 2 \ + --vmsa-cpu0 vmsa0.bin \ + --vmsa-cpu1 vmsa1.bin \ + --tk this-guest-tk.bin \ + --measurement Zs2pf19ubFSafpZ2WKkwquXvACx9Wt/BV+eJwQ/taO8jhyIj/F8swFrybR1fZ2ID \ + --api-major 0 \ + --api-minor 24 \ + --build-id 13 \ + --policy 7 + Fetch from remote libvirt ------------------------- @@ -245,6 +278,19 @@ Validate the measurement of a SEV guest with direct kernel boot: --tk this-guest-tk.bin \ --domain fedora34x86_64 +Validate the measurement of a SEV-ES SMP guest booting from disk: + +:: + + # virt-dom-sev-validate \ + --connect qemu+ssh://root@some.remote.host/system \ + --firmware OVMF.sev.fd \ + --num-cpus 2 \ + --vmsa-cpu0 vmsa0.bin \ + --vmsa-cpu1 vmsa1.bin \ + --tk this-guest-tk.bin \ + --domain fedora34x86_64 + Fetch from local libvirt ------------------------ @@ -274,6 +320,18 @@ Validate the measurement of a SEV guest with direct kernel boot: --tk this-guest-tk.bin \ --domain fedora34x86_64 +Validate the measurement of a SEV-ES SMP guest booting from disk: + +:: + + # virt-dom-sev-validate \ + --insecure \ + --num-cpus 2 \ + --vmsa-cpu0 vmsa0.bin \ + --vmsa-cpu1 vmsa1.bin \ + --tk this-guest-tk.bin \ + --domain fedora34x86_64 + EXIT STATUS =========== diff --git a/tools/virt-qemu-sev-validate b/tools/virt-qemu-sev-validate index 2c696ba04c..5abcedfb1f 100755 --- a/tools/virt-qemu-sev-validate +++ b/tools/virt-qemu-sev-validate @@ -153,13 +153,16 @@ class KernelTable(GUIDTable): class ConfidentialVM(object): + POLICY_BIT_SEV_ES = 2 + POLICY_VAL_SEV_ES = (1 << POLICY_BIT_SEV_ES) def __init__(self, measurement=None, api_major=None, api_minor=None, build_id=None, - policy=None): + policy=None, + num_cpus=None): self.measurement = measurement self.api_major = api_major self.api_minor = api_minor @@ -170,8 +173,15 @@ class ConfidentialVM(object): self.tik = None self.tek = None + self.num_cpus = num_cpus + self.vmsa_cpu0 = None + self.vmsa_cpu1 = None + self.kernel_table = KernelTable() + def is_sev_es(self): + return self.policy & self.POLICY_VAL_SEV_ES + def load_tik_tek(self, tik_path, tek_path): with open(tik_path, 'rb') as fh: self.tik = fh.read() @@ -207,6 +217,43 @@ class ConfidentialVM(object): self.firmware = fh.read() log.debug("Loader(sha256): %s", sha256(self.firmware).hexdigest()) + @staticmethod + def _load_vmsa(path): + with open(path, 'rb') as fh: + vmsa = fh.read() + + if len(vmsa) != 4096: + raise UnsupportedUsageException( + "VMSA must be 4096 bytes in length") + return vmsa + + def load_vmsa_cpu0(self, path): + self.vmsa_cpu0 = self._load_vmsa(path) + log.debug("VMSA CPU 0(sha256): %s", + sha256(self.vmsa_cpu0).hexdigest()) + + def load_vmsa_cpu1(self, path): + self.vmsa_cpu1 = self._load_vmsa(path) + log.debug("VMSA CPU 1(sha256): %s", + sha256(self.vmsa_cpu1).hexdigest()) + + def get_cpu_state(self): + if self.num_cpus is None: + raise UnsupportedUsageException( + "Number of virtual CPUs must be specified for SEV-ES domain") + + if self.vmsa_cpu0 is None: + raise UnsupportedUsageException( + "VMSA for boot CPU required for SEV-ES domain") + + if self.num_cpus > 1 and self.vmsa_cpu1 is None: + raise UnsupportedUsageException( + "VMSA for additional CPUs required for SEV-ES domain with SMP") + + vmsa = self.vmsa_cpu0 + (self.vmsa_cpu1 * (self.num_cpus - 1)) + log.debug("VMSA(sha256): %s", sha256(vmsa).hexdigest()) + return vmsa + # Get the full set of measured launch data for the domain # # The measured data that the guest is initialized with is the concatenation @@ -217,6 +264,8 @@ class ConfidentialVM(object): def get_measured_data(self): measured_data = (self.firmware + self.kernel_table.build()) + if self.is_sev_es(): + measured_data += self.get_cpu_state() log.debug("Measured-data(sha256): %s", sha256(measured_data).hexdigest()) return measured_data @@ -454,6 +503,12 @@ def parse_command_line(): help='Path to the initrd binary') vmconfig.add_argument('--cmdline', '-e', help='Cmdline string booted with') + vmconfig.add_argument('--num-cpus', '-n', type=int, + help='Number of virtual CPUs') + vmconfig.add_argument('--vmsa-cpu0', '-0', + help='VMSA state for the boot CPU') + vmconfig.add_argument('--vmsa-cpu1', '-1', + help='VMSA state for the additional CPUs') vmconfig.add_argument('--tik', help='TIK file for domain') vmconfig.add_argument('--tek', @@ -524,13 +579,15 @@ def attest(args): api_major=args.api_major, api_minor=args.api_minor, build_id=args.build_id, - policy=args.policy) + policy=args.policy, + num_cpus=args.num_cpus) else: cvm = LibvirtConfidentialVM(measurement=args.measurement, api_major=args.api_major, api_minor=args.api_minor, build_id=args.build_id, - policy=args.policy) + policy=args.policy, + num_cpus=args.num_cpus) if args.firmware is not None: cvm.load_firmware(args.firmware) @@ -549,6 +606,12 @@ def attest(args): if args.cmdline is not None: cvm.kernel_table.load_cmdline(args.cmdline) + if args.vmsa_cpu0 is not None: + cvm.load_vmsa_cpu0(args.vmsa_cpu0) + + if args.vmsa_cpu1 is not None: + cvm.load_vmsa_cpu1(args.vmsa_cpu1) + if args.domain is not None: cvm.load_domain(args.connect, args.domain, -- 2.37.3

The VMSA files contain the expected CPU register state for the VM. Their content varies based on a few pieces of the stack - AMD CPU architectural initial state - KVM hypervisor VM CPU initialization - QEMU userspace VM CPU initialization - AMD CPU SKU (family/model/stepping) The first three pieces of information we can obtain through code inspection. The last piece of information we can take on the command line. This allows a user to validate a SEV-ES guest merely by providing the CPU SKU information, using --cpu-family, --cpu-model, --cpu-stepping. This avoids the need to obtain or construct VMSA files directly. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- docs/manpages/virt-qemu-sev-validate.rst | 45 +++ tools/virt-qemu-sev-validate | 467 +++++++++++++++++++++++ 2 files changed, 512 insertions(+) diff --git a/docs/manpages/virt-qemu-sev-validate.rst b/docs/manpages/virt-qemu-sev-validate.rst index 9f86212cb9..4aebbb3c9b 100644 --- a/docs/manpages/virt-qemu-sev-validate.rst +++ b/docs/manpages/virt-qemu-sev-validate.rst @@ -245,6 +245,24 @@ Validate the measurement of a SEV-ES SMP guest booting from disk: --build-id 13 \ --policy 7 +Validate the measurement of a SEV-ES SMP guest booting from disk, with +automatically constructed VMSA: + +:: + + # virt-dom-sev-validate \ + --firmware OVMF.sev.fd \ + --num-cpus 2 \ + --cpu-family 23 \ + --cpu-model 49 \ + --cpu-stepping 0 \ + --tk this-guest-tk.bin \ + --measurement Zs2pf19ubFSafpZ2WKkwquXvACx9Wt/BV+eJwQ/taO8jhyIj/F8swFrybR1fZ2ID \ + --api-major 0 \ + --api-minor 24 \ + --build-id 13 \ + --policy 7 + Fetch from remote libvirt ------------------------- @@ -291,6 +309,20 @@ Validate the measurement of a SEV-ES SMP guest booting from disk: --tk this-guest-tk.bin \ --domain fedora34x86_64 +Validate the measurement of a SEV-ES SMP guest booting from disk, with +automatically constructed VMSA: + +:: + + # virt-dom-sev-validate \ + --connect qemu+ssh://root@some.remote.host/system \ + --firmware OVMF.sev.fd \ + --cpu-family 23 \ + --cpu-model 49 \ + --cpu-stepping 0 \ + --tk this-guest-tk.bin \ + --domain fedora34x86_64 + Fetch from local libvirt ------------------------ @@ -332,6 +364,19 @@ Validate the measurement of a SEV-ES SMP guest booting from disk: --tk this-guest-tk.bin \ --domain fedora34x86_64 +Validate the measurement of a SEV-ES SMP guest booting from disk, with +automatically constructed VMSA: + +:: + + # virt-dom-sev-validate \ + --insecure \ + --cpu-family 23 \ + --cpu-model 49 \ + --cpu-stepping 0 \ + --tk this-guest-tk.bin \ + --domain fedora34x86_64 + EXIT STATUS =========== diff --git a/tools/virt-qemu-sev-validate b/tools/virt-qemu-sev-validate index 5abcedfb1f..cdf6a24d46 100755 --- a/tools/virt-qemu-sev-validate +++ b/tools/virt-qemu-sev-validate @@ -42,6 +42,7 @@ import hmac import logging import re import socket +from struct import pack import sys import traceback from uuid import UUID @@ -72,6 +73,427 @@ class InvalidStateException(Exception): pass +class Field(object): + U8 = 0 + U16 = 2 + U32 = 4 + U64 = 8 + + SCALAR = 0 + BITMASK = 1 + ARRAY = 2 + + def __init__(self, name, size, fmt, value, order): + self.name = name + self.size = size + self.value = value + self.fmt = fmt + self.order = order + + +class Struct(object): + def __init__(self, size): + self._fields = {} + self.size = size + + def register_field(self, name, size, fmt=Field.SCALAR, defvalue=0): + self._fields[name] = Field(name, size, fmt, + defvalue, len(self.fields)) + + @property + def fields(self): + return sorted(self._fields.values(), key=lambda f: f.order) + + def __getattr__(self, name): + return self._fields[name] + + def __setattr__(self, name, value): + if name in ["_fields", "size"]: + super().__setattr__(name, value) + else: + self._fields[name].value = value + + def binary_format(self): + fmt = ["<"] + datalen = 0 + for field in self.fields: + if field.size == Field.U8: + if field.fmt == Field.ARRAY: + datalen += len(field.value) + fmt += ["%dB" % len(field.value)] + else: + datalen += 1 + fmt += ["B"] + elif field.size == Field.U16: + datalen += 2 + fmt += ["H"] + elif field.size == Field.U32: + datalen += 4 + fmt += ["L"] + elif field.size == Field.U64: + datalen += 8 + fmt += ["Q"] + + pad = self.size - datalen + assert self.size >= 1 + fmt += ["%dB" % pad] + + return "".join(fmt), pad + + def pack(self): + fmt, pad = self.binary_format() + + values = [] + for field in self.fields: + if field.size == Field.U8 and field.fmt == Field.ARRAY: + for _, k in enumerate(field.value): + values.append(k) + else: + values.append(field.value) + values.extend([0] * pad) + + return pack(fmt, *values) + + +class VMSA(Struct): + ATTR_G_SHIFT = 23 + ATTR_G_MASK = (1 << ATTR_G_SHIFT) + ATTR_B_SHIFT = 22 + ATTR_B_MASK = (1 << ATTR_B_SHIFT) + ATTR_L_SHIFT = 21 + ATTR_L_MASK = (1 << ATTR_L_SHIFT) + ATTR_AVL_SHIFT = 20 + ATTR_AVL_MASK = (1 << ATTR_AVL_SHIFT) + ATTR_P_SHIFT = 15 + ATTR_P_MASK = (1 << ATTR_P_SHIFT) + ATTR_DPL_SHIFT = 13 + ATTR_DPL_MASK = (3 << ATTR_DPL_SHIFT) + ATTR_S_SHIFT = 12 + ATTR_S_MASK = (1 << ATTR_S_SHIFT) + ATTR_TYPE_SHIFT = 8 + ATTR_TYPE_MASK = (15 << ATTR_TYPE_SHIFT) + ATTR_A_MASK = (1 << 8) + + ATTR_CS_MASK = (1 << 11) + ATTR_C_MASK = (1 << 10) + ATTR_R_MASK = (1 << 9) + + ATTR_E_MASK = (1 << 10) + ATTR_W_MASK = (1 << 9) + + def __init__(self): + super().__init__(4096) + + # From Linux arch/x86/include/asm/svm.h, we're unpacking the + # struct vmcb_save_area + + self.register_field("es_selector", Field.U16) + self.register_field("es_attrib", Field.U16, Field.BITMASK) + self.register_field("es_limit", Field.U32) + self.register_field("es_base", Field.U64) + + self.register_field("cs_selector", Field.U16) + self.register_field("cs_attrib", Field.U16, Field.BITMASK) + self.register_field("cs_limit", Field.U32) + self.register_field("cs_base", Field.U64) + + self.register_field("ss_selector", Field.U16) + self.register_field("ss_attrib", Field.U16, Field.BITMASK) + self.register_field("ss_limit", Field.U32) + self.register_field("ss_base", Field.U64) + + self.register_field("ds_selector", Field.U16) + self.register_field("ds_attrib", Field.U16, Field.BITMASK) + self.register_field("ds_limit", Field.U32) + self.register_field("ds_base", Field.U64) + + self.register_field("fs_selector", Field.U16) + self.register_field("fs_attrib", Field.U16, Field.BITMASK) + self.register_field("fs_limit", Field.U32) + self.register_field("fs_base", Field.U64) + + self.register_field("gs_selector", Field.U16) + self.register_field("gs_attrib", Field.U16, Field.BITMASK) + self.register_field("gs_limit", Field.U32) + self.register_field("gs_base", Field.U64) + + self.register_field("gdtr_selector", Field.U16) + self.register_field("gdtr_attrib", Field.U16, Field.BITMASK) + self.register_field("gdtr_limit", Field.U32) + self.register_field("gdtr_base", Field.U64) + + self.register_field("ldtr_selector", Field.U16) + self.register_field("ldtr_attrib", Field.U16, Field.BITMASK) + self.register_field("ldtr_limit", Field.U32) + self.register_field("ldtr_base", Field.U64) + + self.register_field("idtr_selector", Field.U16) + self.register_field("idtr_attrib", Field.U16, Field.BITMASK) + self.register_field("idtr_limit", Field.U32) + self.register_field("idtr_base", Field.U64) + + self.register_field("tr_selector", Field.U16) + self.register_field("tr_attrib", Field.U16, Field.BITMASK) + self.register_field("tr_limit", Field.U32) + self.register_field("tr_base", Field.U64) + + self.register_field("reserved_1", + Field.U8, Field.ARRAY, bytearray([0] * 43)) + + self.register_field("cpl", Field.U8) + + self.register_field("reserved_2", + Field.U8, Field.ARRAY, bytearray([0] * 4)) + + self.register_field("efer", Field.U64) + + self.register_field("reserved_3", + Field.U8, Field.ARRAY, bytearray([0] * 104)) + + self.register_field("xss", Field.U64) + self.register_field("cr4", Field.U64) + self.register_field("cr3", Field.U64) + self.register_field("cr0", Field.U64) + self.register_field("dr7", Field.U64) + self.register_field("dr6", Field.U64) + self.register_field("rflags", Field.U64) + self.register_field("rip", Field.U64) + + self.register_field("reserved_4", + Field.U8, Field.ARRAY, bytearray([0] * 88)) + + self.register_field("rsp", Field.U64) + + self.register_field("reserved_5", + Field.U8, Field.ARRAY, bytearray([0] * 24)) + + self.register_field("rax", Field.U64) + self.register_field("star", Field.U64) + self.register_field("lstar", Field.U64) + self.register_field("cstar", Field.U64) + self.register_field("sfmask", Field.U64) + self.register_field("kernel_gs_base", Field.U64) + self.register_field("sysenter_cs", Field.U64) + self.register_field("sysenter_esp", Field.U64) + self.register_field("sysenter_eip", Field.U64) + self.register_field("cr2", Field.U64) + + self.register_field("reserved_6", + Field.U8, Field.ARRAY, bytearray([0] * 32)) + + self.register_field("g_pat", Field.U64) + self.register_field("dbgctl", Field.U64) + self.register_field("br_from", Field.U64) + self.register_field("br_to", Field.U64) + self.register_field("last_excp_from", Field.U64) + self.register_field("last_excp_to", Field.U64) + + self.register_field("reserved_7", + Field.U8, Field.ARRAY, bytearray([0] * 72)) + + self.register_field("spec_ctrl", Field.U32) + + self.register_field("reserved_7b", + Field.U8, Field.ARRAY, bytearray([0] * 4)) + + self.register_field("pkru", Field.U32) + + self.register_field("reserved_7a", + Field.U8, Field.ARRAY, bytearray([0] * 20)) + + self.register_field("reserved_8", Field.U64) # rax duplicate + + self.register_field("rcx", Field.U64) + self.register_field("rdx", Field.U64, Field.BITMASK) + self.register_field("rbx", Field.U64) + + self.register_field("reserved_9", Field.U64) # rsp duplicate + + self.register_field("rbp", Field.U64) + self.register_field("rsi", Field.U64) + self.register_field("rdi", Field.U64) + self.register_field("r8", Field.U64) + self.register_field("r9", Field.U64) + self.register_field("r10", Field.U64) + self.register_field("r11", Field.U64) + self.register_field("r12", Field.U64) + self.register_field("r13", Field.U64) + self.register_field("r14", Field.U64) + self.register_field("r15", Field.U64) + + self.register_field("reserved_10", + Field.U8, Field.ARRAY, bytearray([0] * 16)) + + self.register_field("sw_exit_code", Field.U64) + self.register_field("sw_exit_info_1", Field.U64) + self.register_field("sw_exit_info_2", Field.U64) + self.register_field("sw_scratch", Field.U64) + + self.register_field("reserved_11", + Field.U8, Field.ARRAY, bytearray([0] * 56)) + + self.register_field("xcr0", Field.U64) + self.register_field("valid_bitmap", + Field.U8, Field.ARRAY, bytearray([0] * 16)) + self.register_field("x87_state_gpa", + Field.U64) + + def amd64_cpu_init(self): + # AMD64 Architecture Programmer’s Manual + # Volume 2: System Programming. + # + # 14.1.3 Processor Initialization State + # + # Values after INIT + + self.cr0 = (1 << 4) + self.rip = 0xfff0 + + self.cs_selector = 0xf000 + self.cs_base = 0xffff0000 + self.cs_limit = 0xffff + + self.ds_limit = 0xffff + + self.es_limit = 0xffff + self.fs_limit = 0xffff + self.gs_limit = 0xffff + self.ss_limit = 0xffff + + self.gdtr_limit = 0xffff + self.idtr_limit = 0xffff + + self.ldtr_limit = 0xffff + self.tr_limit = 0xffff + + self.dr6 = 0xffff0ff0 + self.dr7 = 0x0400 + self.rflags = 0x2 + self.xcr0 = 0x1 + + def kvm_cpu_init(self): + # svm_set_cr4() sets guest X86_CR4_MCE bit if host + # has X86_CR4_MCE enabled + self.cr4 = 0x40 + + # svm_set_efer sets guest EFER_SVME (Secure Virtual Machine enable) + self.efer = 0x1000 + + # init_vmcb + init_sys_seg() sets + # SVM_SELECTOR_P_MASK | SEG_TYPE_LDT + self.ldtr_attrib = 0x0082 + + # init_vmcb + init_sys_seg() sets + # SVM_SELECTOR_P_MASK | SEG_TYPE_BUSY_TSS16 + self.tr_attrib = 0x0083 + + # kvm_arch_vcpu_create() in arch/x86/kvm/x86.c + self.g_pat = 0x0007040600070406 + + def qemu_cpu_init(self): + # Based on logic in x86_cpu_reset() + # + # file target/i386/cpu.c + + def attr(mask): + return (mask >> VMSA.ATTR_TYPE_SHIFT) + + self.ldtr_attrib = attr(VMSA.ATTR_P_MASK | + (2 << VMSA.ATTR_TYPE_SHIFT)) + self.tr_attrib = attr(VMSA.ATTR_P_MASK | + (11 << VMSA.ATTR_TYPE_SHIFT)) + self.cs_attrib = attr(VMSA.ATTR_P_MASK | + VMSA.ATTR_S_MASK | + VMSA.ATTR_CS_MASK | + VMSA.ATTR_R_MASK | + VMSA.ATTR_A_MASK) + self.ds_attrib = attr(VMSA.ATTR_P_MASK | + VMSA.ATTR_S_MASK | + VMSA.ATTR_W_MASK | + VMSA.ATTR_A_MASK) + self.es_attrib = attr(VMSA.ATTR_P_MASK | + VMSA.ATTR_S_MASK | + VMSA.ATTR_W_MASK | + VMSA.ATTR_A_MASK) + self.ss_attrib = attr(VMSA.ATTR_P_MASK | + VMSA.ATTR_S_MASK | + VMSA.ATTR_W_MASK | + VMSA.ATTR_A_MASK) + self.fs_attrib = attr(VMSA.ATTR_P_MASK | + VMSA.ATTR_S_MASK | + VMSA.ATTR_W_MASK | + VMSA.ATTR_A_MASK) + self.gs_attrib = attr(VMSA.ATTR_P_MASK | + VMSA.ATTR_S_MASK | + VMSA.ATTR_W_MASK | + VMSA.ATTR_A_MASK) + + self.g_pat = 0x0007040600070406 + + def cpu_sku(self, family, model, stepping): + stepping &= 0xf + model &= 0xff + family &= 0xfff + + self.rdx.value = stepping + + if family > 0xf: + self.rdx.value |= 0xf00 | ((family - 0x0f) << 20) + else: + self.rdx.value |= family << 8 + + self.rdx.value |= ((model & 0xf) << 4) | ((model >> 4) << 16) + + def reset_addr(self, reset_addr): + reset_cs = reset_addr & 0xffff0000 + reset_ip = reset_addr & 0x0000ffff + + self.rip.value = reset_ip + self.cs_base.value = reset_cs + + +class OVMF(object): + + OVMF_TABLE_FOOTER_GUID = UUID("96b582de-1fb2-45f7-baea-a366c55a082d") + SEV_INFO_BLOCK_GUID = UUID("00f771de-1a7e-4fcb-890e-68c77e2fb44e") + + def __init__(self): + self.entries = {} + + def load(self, content): + expect = OVMF.OVMF_TABLE_FOOTER_GUID.bytes_le + actual = content[-48:-32] + if expect != actual: + raise Exception("OVMF footer GUID not found") + + tablelen = int.from_bytes(content[-50:-48], byteorder='little') + + if tablelen == 0: + raise Exception("OVMF tables zero length") + + table = content[-(32 + tablelen):-50] + + self.parse_table(table) + + def parse_table(self, data): + while len(data) > 0: + entryuuid = UUID(bytes_le=data[-16:]) + entrylen = int.from_bytes(data[-18:-16], byteorder='little') + entrydata = data[-entrylen:-18] + + self.entries[str(entryuuid)] = entrydata + + data = data[0:-entrylen] + + def reset_addr(self): + if str(OVMF.SEV_INFO_BLOCK_GUID) not in self.entries: + raise Exception("SEV info block GUID not found") + + reset_addr = int.from_bytes( + self.entries[str(OVMF.SEV_INFO_BLOCK_GUID)], "little") + return reset_addr + + class GUIDTable(abc.ABC): GUID_LEN = 16 @@ -237,6 +659,26 @@ class ConfidentialVM(object): log.debug("VMSA CPU 1(sha256): %s", sha256(self.vmsa_cpu1).hexdigest()) + def build_vmsas(self, family, model, stepping): + ovmf = OVMF() + ovmf.load(self.firmware) + + vmsa = VMSA() + vmsa.amd64_cpu_init() + vmsa.kvm_cpu_init() + vmsa.qemu_cpu_init() + + vmsa.cpu_sku(family, model, stepping) + + self.vmsa_cpu0 = vmsa.pack() + log.debug("VMSA CPU 0(sha256): %s", + sha256(self.vmsa_cpu0).hexdigest()) + + vmsa.reset_addr(ovmf.reset_addr()) + self.vmsa_cpu1 = vmsa.pack() + log.debug("VMSA CPU 1(sha256): %s", + sha256(self.vmsa_cpu1).hexdigest()) + def get_cpu_state(self): if self.num_cpus is None: raise UnsupportedUsageException( @@ -509,6 +951,12 @@ def parse_command_line(): help='VMSA state for the boot CPU') vmconfig.add_argument('--vmsa-cpu1', '-1', help='VMSA state for the additional CPUs') + vmconfig.add_argument('--cpu-family', type=int, + help='Hypervisor host CPU family number') + vmconfig.add_argument('--cpu-model', type=int, + help='Hypervisor host CPU model number') + vmconfig.add_argument('--cpu-stepping', type=int, + help='Hypervisor host CPU stepping number') vmconfig.add_argument('--tik', help='TIK file for domain') vmconfig.add_argument('--tek', @@ -572,6 +1020,20 @@ def check_usage(args): raise UnsupportedUsageException( "--initrd/--cmdline require --kernel") + sku = [args.cpu_family, args.cpu_model, args.cpu_stepping] + if sku.count(None) == len(sku): + if args.vmsa_cpu1 is not None and args.vmsa_cpu0 is None: + raise UnsupportedUsageException( + "VMSA for additional CPU also requires VMSA for boot CPU") + else: + if args.vmsa_cpu0 is not None or args.vmsa_cpu1 is not None: + raise UnsupportedUsageException( + "VMSA files are mutually exclusive with CPU SKU") + + if sku.count(None) != 0: + raise UnsupportedUsageException( + "CPU SKU needs family, model and stepping for SEV-ES domain") + def attest(args): if args.domain is None: @@ -612,6 +1074,11 @@ def attest(args): if args.vmsa_cpu1 is not None: cvm.load_vmsa_cpu1(args.vmsa_cpu1) + if args.cpu_family is not None: + cvm.build_vmsas(args.cpu_family, + args.cpu_model, + args.cpu_stepping) + if args.domain is not None: cvm.load_domain(args.connect, args.domain, -- 2.37.3

When validating a SEV-ES guest, we need to know the CPU count and VMSA state. We can get the CPU count directly from libvirt's guest info. The VMSA state can be constructed automatically if we query the CPU SKU from host capabilities XML. Neither of these is secure, however, so this behaviour is restricted. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- docs/manpages/virt-qemu-sev-validate.rst | 4 ---- tools/virt-qemu-sev-validate | 26 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/docs/manpages/virt-qemu-sev-validate.rst b/docs/manpages/virt-qemu-sev-validate.rst index 4aebbb3c9b..e1e290f125 100644 --- a/docs/manpages/virt-qemu-sev-validate.rst +++ b/docs/manpages/virt-qemu-sev-validate.rst @@ -358,7 +358,6 @@ Validate the measurement of a SEV-ES SMP guest booting from disk: # virt-dom-sev-validate \ --insecure \ - --num-cpus 2 \ --vmsa-cpu0 vmsa0.bin \ --vmsa-cpu1 vmsa1.bin \ --tk this-guest-tk.bin \ @@ -371,9 +370,6 @@ automatically constructed VMSA: # virt-dom-sev-validate \ --insecure \ - --cpu-family 23 \ - --cpu-model 49 \ - --cpu-stepping 0 \ --tk this-guest-tk.bin \ --domain fedora34x86_64 diff --git a/tools/virt-qemu-sev-validate b/tools/virt-qemu-sev-validate index cdf6a24d46..9eb9d54258 100755 --- a/tools/virt-qemu-sev-validate +++ b/tools/virt-qemu-sev-validate @@ -868,6 +868,14 @@ class LibvirtConfidentialVM(ConfidentialVM): if self.policy is None: self.policy = sevinfo["sev-policy"] + if self.is_sev_es() and self.num_cpus is None: + if secure: + raise InsecureUsageException( + "Using CPU count from guest is not secure") + + info = self.dom.info() + self.num_cpus = info[3] + if self.firmware is None: if remote: raise UnsupportedUsageException( @@ -913,6 +921,24 @@ class LibvirtConfidentialVM(ConfidentialVM): "Using cmdline string from XML is not secure") self.kernel_table.load_cmdline(cmdlinenodes[0].text) + capsxml = self.conn.getCapabilities() + capsdoc = etree.fromstring(capsxml) + + if self.is_sev_es() and self.vmsa_cpu0 is None: + if secure: + raise InsecureUsageException( + "Using CPU SKU from capabilities is not secure") + + sig = capsdoc.xpath("/capabilities/host/cpu/signature") + if len(sig) != 1: + raise UnsupportedUsageException( + "Libvirt is too old to report host CPU signature") + + cpu_family = int(sig[0].get("family")) + cpu_model = int(sig[0].get("model")) + cpu_stepping = int(sig[0].get("stepping")) + self.build_vmsas(cpu_family, cpu_model, cpu_stepping) + def parse_command_line(): parser = argparse.ArgumentParser( -- 2.37.3

It is possible to build OVMF for SEV with an embedded Grub that can fetch LUKS disk secrets. This adds support for injecting secrets in the required format. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- docs/manpages/virt-qemu-sev-validate.rst | 83 ++++++++++ tools/virt-qemu-sev-validate | 183 +++++++++++++++++++++-- 2 files changed, 257 insertions(+), 9 deletions(-) diff --git a/docs/manpages/virt-qemu-sev-validate.rst b/docs/manpages/virt-qemu-sev-validate.rst index e1e290f125..db1b43a775 100644 --- a/docs/manpages/virt-qemu-sev-validate.rst +++ b/docs/manpages/virt-qemu-sev-validate.rst @@ -189,6 +189,46 @@ understand any configuration mistakes that have been made. If the will be skipped. The result is that the validation will likely be reported as failed. +Secret injection options +------------------------ + +These options provide a way to inject a secret if validation of the +launch measurement passes. + +``--inject-secret ALIAS-OR-GUID:PATH`` + +Path to a file containing a secret to inject into the guest OS. Typical +usage would be to supply a password for unlocking the root filesystem +full disk encryption. ``ALIAS`` can be one of the well known secrets: + +* ``luks-key`` - bytes to use as a key for unlocking a LUKS key slot. + GUID of ``736869e5-84f0-4973-92ec-06879ce3da0b``. + +Alternatively ``GUID`` refers to an arbitrary UUID of the callers +choosing. The contents of ``PATH`` are defined by the requirements +of the associated GUID, and will used as-is without modification. +In particular be aware: + + * Avoid unwanted trailing newline characters in ``PATH`` unless + mandated by the ``GUID``. + * Any trailing ``NUL`` byte must be explicitly included in ``PATH`` + if mandated by the ``GUID``. + +This argument can be repeated multiple times, provided a different +``GUID`` is given for each instance. + +``--secret-header PATH`` + +Path to a file in which the injected secret header will be written in base64 +format and later injected into the domain. This is required if there is no +connection to libvirt, otherwise the secret will be directly injected. + +``--secret-payload PATH`` + +Path to a file in which the injected secret payload will be written in base64 +format and later injected into the domain. This is required if there is no +connection to libvirt, otherwise the secret will be directly injected. + EXAMPLES ======== @@ -263,6 +303,26 @@ automatically constructed VMSA: --build-id 13 \ --policy 7 +Validate the measurement of a SEV guest booting from disk and +inject a disk password on success: + +:: + + # virt-dom-sev-validate \ + --loader OVMF.sev.fd \ + --tk this-guest-tk.bin \ + --measurement Zs2pf19ubFSafpZ2WKkwquXvACx9Wt/BV+eJwQ/taO8jhyIj/F8swFrybR1fZ2ID \ + --api-major 0 \ + --api-minor 24 \ + --build-id 13 \ + --policy 3 \ + --disk-password passwd.txt \ + --secret-header secret-header.b64 \ + --secret-payload secret-payload.b64 + +The ``secret-header.b64`` and ``secret-payload.b64`` files can now be sent to +the virtualization host for injection. + Fetch from remote libvirt ------------------------- @@ -323,6 +383,18 @@ automatically constructed VMSA: --tk this-guest-tk.bin \ --domain fedora34x86_64 +Validate the measurement of a SEV guest booting from disk and +inject a disk password on success: + +:: + + # virt-dom-sev-validate \ + --connect qemu+ssh://root@some.remote.host/system \ + --loader OVMF.sev.fd \ + --tk this-guest-tk.bin \ + --domain fedora34x86_64 \ + --disk-password passwd.txt + Fetch from local libvirt ------------------------ @@ -373,6 +445,17 @@ automatically constructed VMSA: --tk this-guest-tk.bin \ --domain fedora34x86_64 +Validate the measurement of a SEV guest booting from disk and +inject a disk password on success: + +:: + + # virt-dom-sev-validate \ + --insecure \ + --tk this-guest-tk.bin \ + --domain fedora34x86_64 \ + --disk-password passwd.txt + EXIT STATUS =========== diff --git a/tools/virt-qemu-sev-validate b/tools/virt-qemu-sev-validate index 9eb9d54258..ac1e4c8d66 100755 --- a/tools/virt-qemu-sev-validate +++ b/tools/virt-qemu-sev-validate @@ -36,16 +36,19 @@ import abc import argparse -from base64 import b64decode +from base64 import b64decode, b64encode from hashlib import sha256 import hmac import logging +import os import re import socket from struct import pack import sys import traceback from uuid import UUID +from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes + from lxml import etree import libvirt @@ -574,7 +577,46 @@ class KernelTable(GUIDTable): return entries -class ConfidentialVM(object): +class SecretsTable(GUIDTable): + + TABLE_GUID = UUID('{1e74f542-71dd-4d66-963e-ef4287ff173b}').bytes_le + + GUID_ALIASES = { + "luks-key": UUID('{736869e5-84f0-4973-92ec-06879ce3da0b}') + } + + def __init__(self): + super().__init__(guid=self.TABLE_GUID, + lenlen=4) + self.secrets = {} + + def load_secret(self, alias_or_guid, path): + guid = None + if re.match(r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$", + alias_or_guid): + guid = UUID(alias_or_guid) + else: + if alias_or_guid not in self.GUID_ALIASES: + raise UnsupportedUsageException( + "Secret alias '%s' is not known" % alias_or_guid) + + guid = self.GUID_ALIASES[alias_or_guid] + + if guid in self.secrets: + raise UnsupportedUsageException( + "Secret for GUID %s already loaded" % guid) + + with open(path, 'rb') as fh: + self.secrets[guid] = fh.read() + + def entries(self): + entries = bytes([]) + for guid, value in self.secrets.items(): + entries += self.build_entry(guid.bytes_le, value, 4) + return entries + + +class ConfidentialVM(abc.ABC): POLICY_BIT_SEV_ES = 2 POLICY_VAL_SEV_ES = (1 << POLICY_BIT_SEV_ES) @@ -600,6 +642,7 @@ class ConfidentialVM(object): self.vmsa_cpu1 = None self.kernel_table = KernelTable() + self.secrets_table = SecretsTable() def is_sev_es(self): return self.policy & self.POLICY_VAL_SEV_ES @@ -752,6 +795,82 @@ class ConfidentialVM(object): raise AttestationFailedException( "Measurement does not match, VM is not trustworthy") + def build_secrets(self): + measurement, _ = self.get_measurements() + + iv = os.urandom(16) + + secret_table = self.secrets_table.build() + + cipher = Cipher(algorithms.AES(self.tek), modes.CTR(iv)) + enc = cipher.encryptor() + secret_table_ciphertext = (enc.update(secret_table) + + enc.finalize()) + + flags = 0 + + ## + # Table 55. LAUNCH_SECRET Packet Header Buffer + ## + header = ( + flags.to_bytes(4, byteorder='little') + + iv + ) + + # AMD Secure Encrypted Virtualization API , section 6.6 + # + # hdrmac = HMAC(0x01 || FLAGS || IV || GUEST_LENGTH || + # TRANS_LENGTH || DATA || + # MEASURE; GCTX.TIK) + # + msg = ( + bytes([0x01]) + + flags.to_bytes(4, byteorder='little') + + iv + + len(secret_table).to_bytes(4, byteorder='little') + + len(secret_table).to_bytes(4, byteorder='little') + + secret_table_ciphertext + + measurement + ) + + h = hmac.new(self.tik, msg, 'sha256') + header = ( + flags.to_bytes(4, byteorder='little') + + iv + + h.digest() + ) + + header64 = b64encode(header).decode('utf8') + secret64 = b64encode(secret_table_ciphertext).decode('utf8') + log.debug("Header: %s (%d bytes)", header64, len(header)) + log.debug("Secret: %s (%d bytes)", + secret64, len(secret_table_ciphertext)) + + return header64, secret64 + + @abc.abstractmethod + def inject_secrets(self): + pass + + +class OfflineConfidentialVM(ConfidentialVM): + def __init__(self, + secret_header=None, + secret_payload=None, + **kwargs): + super().__init__(**kwargs) + + self.secret_header = secret_header + self.secret_payload = secret_payload + + def inject_secrets(self): + header64, secret64 = self.build_secrets() + + with open(self.secret_header, "wb") as fh: + fh.write(header64.encode('utf8')) + with open(self.secret_payload, "wb") as fh: + fh.write(secret64.encode('utf8')) + class LibvirtConfidentialVM(ConfidentialVM): def __init__(self, **kwargs): @@ -939,6 +1058,14 @@ class LibvirtConfidentialVM(ConfidentialVM): cpu_stepping = int(sig[0].get("stepping")) self.build_vmsas(cpu_family, cpu_model, cpu_stepping) + def inject_secrets(self): + header64, secret64 = self.build_secrets() + + params = {"sev-secret": secret64, + "sev-secret-header": header64} + self.dom.setLaunchSecurityState(params, 0) + self.dom.resume() + def parse_command_line(): parser = argparse.ArgumentParser( @@ -1001,6 +1128,15 @@ def parse_command_line(): vmconn.add_argument('--ignore-config', '-g', action='store_true', help='Do not attempt to sanity check the guest config') + # Arguments related to secret injection + inject = parser.add_argument_group("Secret injection parameters") + inject.add_argument('--inject-secret', '-s', action='append', default=[], + help='ALIAS-OR-GUID:PATH file containing secret to inject') + inject.add_argument('--secret-payload', + help='Path to file to write secret data payload to') + inject.add_argument('--secret-header', + help='Path to file to write secret data header to') + return parser.parse_args() @@ -1041,6 +1177,15 @@ def check_usage(args): raise UnsupportedUsageException( "Either --firmware or --domain is required") + if len(args.inject_secret) > 0: + if args.secret_header is None: + raise UnsupportedUsageException( + "Either --secret-header or --domain is required") + + if args.secret_payload is None: + raise UnsupportedUsageException( + "Either --secret-payload or --domain is required") + if args.kernel is None: if args.initrd is not None or args.cmdline is not None: raise UnsupportedUsageException( @@ -1060,15 +1205,22 @@ def check_usage(args): raise UnsupportedUsageException( "CPU SKU needs family, model and stepping for SEV-ES domain") + secret = [args.secret_payload, args.secret_header] + if secret.count(None) > 0 and secret.count(None) != len(secret): + raise UnsupportedUsageException( + "Both --secret-payload and --secret-header are required") + def attest(args): if args.domain is None: - cvm = ConfidentialVM(measurement=args.measurement, - api_major=args.api_major, - api_minor=args.api_minor, - build_id=args.build_id, - policy=args.policy, - num_cpus=args.num_cpus) + cvm = OfflineConfidentialVM(measurement=args.measurement, + api_major=args.api_major, + api_minor=args.api_minor, + build_id=args.build_id, + policy=args.policy, + num_cpus=args.num_cpus, + secret_header=args.secret_header, + secret_payload=args.secret_payload) else: cvm = LibvirtConfidentialVM(measurement=args.measurement, api_major=args.api_major, @@ -1112,10 +1264,23 @@ def attest(args): args.ignore_config) cvm.attest() - if not args.quiet: print("OK: Looks good to me") + for secret in args.inject_secret: + bits = secret.split(":") + if len(bits) != 2: + raise UnsupportedUsageException( + "Expecting ALIAS-OR-GUID:PATH for injected secret") + + cvm.secrets_table.load_secret(bits[0], bits[1]) + + if len(args.inject_secret) > 0: + cvm.inject_secrets() + if not args.quiet: + print("OK: Injected %d secrets" % len(args.inject_secret)) + + def main(): args = parse_command_line() if args.debug: -- 2.37.3

Expand the SEV guest kbase guide with information about how to configure a SEV/SEV-ES guest when attestation is required, and mention the use of virt-qemu-sev-validate as a way to confirm it. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- docs/kbase/launch_security_sev.rst | 105 +++++++++++++++++++++++++++++ 1 file changed, 105 insertions(+) diff --git a/docs/kbase/launch_security_sev.rst b/docs/kbase/launch_security_sev.rst index 2734832487..7f692af748 100644 --- a/docs/kbase/launch_security_sev.rst +++ b/docs/kbase/launch_security_sev.rst @@ -206,6 +206,20 @@ libvirt to the correct OVMF binary. </os> ... +If intending to attest the boot measurement, it is required to use a +firmware binary that is stateless, as persistent NVRAM can undermine +the trust of the secure guest. This is achieved by telling libvirt +that a stateless binary is required + +:: + + ... + <os type='efi'> + <type arch='x86_64' machine='q35'>hvm</type> + <loader stateless='yes'/> + </os> + ... + Memory ------ @@ -373,6 +387,97 @@ running: # dmesg | grep -i sev AMD Secure Encrypted Virtualization (SEV) active +Guest attestation for SEV/SEV-ES from a trusted host +==================================================== + +Before a confidential guest is used, it may be desirable to attest the boot +measurement. To be trustworthy the attestation process needs to be performed +from a machine that is already trusted. This would typically be a physical +machine that the guest owner controls, or could be a previously launched +confidential guest that has already itself been attested. Most notably, it is +**not** possible to securely attest a guest from the hypervisor host itself, +as the goal of the attestation process is to detect whether the hypervisor is +malicious. + +Performing an attestation requires that the ``<launchSecurity>`` element is +configured with a guest owner Diffie-Hellman (DH) certificate, and a session +data blob. These must be unique for every guest launch attempt. Any reuse will +open avenues of attack for the hypervisor to fake the measurement. Unique data +can be generated using the `sevctl <https://github.com/virtee/sevctl>`_ tool. + +First of all the Platform Diffie-Hellman key (PDH) for the hypervisor host +needs to be obtained. The PDH is used to negotiate a master secret between +the SEV firmware and external entities. + +The admin of the hypervisor can extract the PDH using:: + + $ sevctl export --full ${hostname}.pdh + +Upon receiving the PDH associated with the hypervisor, the guest owner should +validate its integrity:: + + $ sevctl verify --sev ${hostname}.pdh + PDH EP384 D256 008cec87d6bd9df67a35e7d6057a933463cd8a02440f60c5df150821b5662ee0 + ⬑ PEK EP384 E256 431ba88424378200d58b6fb5db9657268c599b1be25f8047ac2e2981eff667e6 + •⬑ OCA EP384 E256 b4f1d0a2144186d1aa9c63f19039834e729f508000aa05a76ba044f8e1419765 + ⬑ CEK EP384 E256 22c27ee3c1c33287db24d3c06869a5ae933eb44148fdb70838019e267077c6b8 + ⬑ ASK R4096 R384 d8cd9d1798c311c96e009a91552f17b4ddc4886a064ec933697734965b9ab29db803c79604e2725658f0861bfaf09ad4 + •⬑ ARK R4096 R384 3d2c1157c29ef7bd4207fc0c8b08db080e579ceba267f8c93bec8dce73f5a5e2e60d959ac37ea82176c1a0c61ae203ed + + • = self signed, ⬑ = signs, •̷ = invalid self sign, ⬑̸ = invalid signs + +Assuming this is successful, it is now possible to generate a unique launch +data for the guest boot attempt:: + + $ sevctl session --name ${myvmname} ${hostname}.pdh ${policy} + +This will generate four files + + * ``${myvmname}_tik.bin`` + * ``${myvmname}_tek.bin`` + * ``${myvmname}_godh.bin`` + * ``${myvmname}_session.bin`` + +The ``tik.bin`` and ``tek.bin`` files will be needed to perform the boot +attestation, and must be kept somewhere secure, away from the hypervisor +host. + +The ``godh.bin`` file contents should be copied into the ``<dhCert>`` field +in the ``<launchSecurity>`` configuration, while the ``session.bin`` file +contents should be copied into the ``<session>`` field. + +When launching the guest, it should be set to remain in the paused state with +no vCPUs running:: + + $ virsh start --paused ${myvmname} + +With it launched, it is possible to query the launch measurement:: + + $ virsh domlaunchsecinfo ${myvmname} + sev-measurement: LMnv8i8N2QejezMPkscShF0cyPYCslgUoCxGWRqQuyt0Q0aUjVkH/T6NcmkwZkWp + sev-api-major : 0 + sev-api-minor : 24 + sev-build-id : 15 + sev-policy : 3 + +The techiques required to validate the measurement reported are beyond the +scope of this document. Fortunately, libvirt provides a tool that can be used +to perform this validation:: + + $ virt-qemu-sev-validate \ + --measurement LMnv8i8N2QejezMPkscShF0cyPYCslgUoCxGWRqQuyt0Q0aUjVkH/T6NcmkwZkWp + --api-major 0 + --api-minor 24 + --build-id 15 + --policy 3 + --tik ${myvmname}_tik.bin + --tek ${myvmname}_tek.bin + OK: Looks good to me + +The `man page <../manpages/virt-qemu-sev-validate.html>`__ for +``virt-qemu-sev-validate`` outlines a great many other ways to invoke this +tool. + Limitations =========== -- 2.37.3

In general we expect to be able to construct a SEV-ES VMSA blob from knowledge about the AMD achitectural CPU register defaults, KVM setup and QEMU setup. If any of this unexpectedly changes, figuring out what's wrong could be horrible. This systemtap script demonstrates how to capture the real VMSA that is used for a SEV-ES as it is booted. The captured data can be fed into the 'sevctl vmsa show' command in order to produce formatted info with named registers, allowing a 'diff' to be performed. This script will need updating for any kernel version that is not 6.0, to set the correct line numbers. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- examples/systemtap/amd-sev-es-vmsa.stp | 48 ++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 examples/systemtap/amd-sev-es-vmsa.stp diff --git a/examples/systemtap/amd-sev-es-vmsa.stp b/examples/systemtap/amd-sev-es-vmsa.stp new file mode 100644 index 0000000000..551ed739b7 --- /dev/null +++ b/examples/systemtap/amd-sev-es-vmsa.stp @@ -0,0 +1,48 @@ +#!/usr/bin/stap +# +# Copyright (C) 2022 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# <http://www.gnu.org/licenses/>;. +# +# A script that captures the VMSA blob for the boot vCPU and +# first additional vCPU, when a KVM guest is booted with SEV-ES +# +# The captured VMSA will be printed to the console in hex format, +# and can be converted to the required binary format by feeding +# it through +# +# perl -e 'while (<>) { print pack("C64", map { hex($_) } ( $_ =~ m/../g )); }' > vmsa.bin +# + +probe begin { + printf("Running\n") +} + +function dump_vmsa(addr:long) { + printf("VMSA\n") + for (i = 0; i < 4096 ; i+= 64) { + printf("%.64M\n", addr + i); + } +} + +# This line number will need to be updated for the specific kernel +# version that is being probed. The line that needs to be targetted +# is the one beween the call to clflush_cache_range(...) and the +# call to sev_issue_cmd(kvm, SEV_CMD_LAUNCH_UPDATE...). +# +# Line 632 is correct for Linux v6.0 +probe module("kvm_amd").statement("__sev_launch_update_vmsa@arch/x86/kvm/svm/sev.c:632") { + dump_vmsa($svm->vmsa) +} -- 2.37.3

Despite efforts to make the virt-qemu-sev-validate tool friendly, it is a certainty that almost everyone who tries it will hit false negative results, getting a failure despite the VM being trustworthy. Diagnosing these problems is no easy matter, especially for those not familiar with SEV/SEV-ES in general. This extra docs text attempts to set out a checklist of items to look at to identify what went wrong. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com&gt; --- docs/manpages/virt-qemu-sev-validate.rst | 116 +++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/docs/manpages/virt-qemu-sev-validate.rst b/docs/manpages/virt-qemu-sev-validate.rst index db1b43a775..331264074e 100644 --- a/docs/manpages/virt-qemu-sev-validate.rst +++ b/docs/manpages/virt-qemu-sev-validate.rst @@ -456,6 +456,122 @@ inject a disk password on success: --domain fedora34x86_64 \ --disk-password passwd.txt +COMMON MISTAKES CHECKLIST +========================= + +The complexity of configuring a guest and validating its boot measurement +means it is very likely to see the failure:: + + ERROR: Measurement does not match, VM is not trustworthy + +This error message assumes the worst, but in most cases will failure will be +a result of either mis-configuring the guest, or passing the wrong information +when trying to validate it. The following information is a guide for what +items to check in order to stand the best chance of diagnosing the problem + +* Check the VM configuration for the DH certificate and session + blob in the libvirt guest XML. + + The content for these fields should be in base64 format, which is + what ``sevctl session`` generates. Other tools may generate the files + in binary format, so ensure it has been correctly converted to base64. + +* Check the VM configuration policy value matches the session blob + + The ``<policy>`` value in libvirt guest XML has to match the value + passed to the ``sevctl session`` command. If this is mismatched + then the guest will not even start, and QEMU will show an error + such as:: + + sev_launch_start: LAUNCH_START ret=1 fw_error=11 'Bad measurement' + +* Check the correct TIK/TEK keypair are passed + + The TIK/TEK keypair are uniquely tied to each DH cert and session + blob. Make sure that the TIK/TEK keypair passed to this program + the ones matched to the DH cert and session blob configured for + the libvirt guest XML. This is one of the most common mistakes. + Further ensure that the TIK and TEK files are not swapped. + +* Check the firmware binary matches the one used to boot + + The firmware binary content is part of the data covered by the + launch measurement. Ensure that the firmware binary passed to + this program matches the one used to launch the guest. The + hypervisor host will periodically get software updates which + introduce a new firmware binary version. + +* Check the kernel, initrd and cmdline match the one used to boot + + If the guest is configured to use direct kernel boot, check that + the kernel, initrd and cmdline passed to this program match the + ones used to boot the guest. In the kernel cmdline whitespace + must be preserved exactly, including any leading or trailing + spaces. + +* Check whether the kernel hash measurement is enabled + + The ``kernelHashes`` property in the libvirt guest XML controls + whether hashes of the kernel, initrd and cmdline content are + covered by the boot measurement. If enabled, then the matching + content must be passed to this program. UIf disabled, then + the content must **NOT** be passed. + +* Check that the correct measurement hash is passed + + The measurement hash includes a nonce, so it will be different + on every boot attempt. Thus when validating the measuremnt it + is important ensure the most recent measurement is used. + +* Check the correct VMSA blobs / CPU SKU values for the host are used + + The VMSA blobs provide the initial register state for the + boot CPU and any additional CPUs. One of the registers + encodes the CPU SKU (family, model, stepping) of the physical + host CPU. Make sure that the VMSA blob used for validation + is one that matches the SKU of the host the guest is booted + on. Passing the CPU SKU values directly to the tool can + reduce the likelihood of using the wrong ones. + +* Check the CPU count is correct + + When passing VMSA blobs for SEV-ES guests, the number of CPUs + present will influence the measurement result. Ensure that the + correct vCPU count is used corresponding to the guest boot + attempt. + + +Best practice is to run this tool in completely offline mode and pass +all information as explicit command line parameters. When debugging +failures, however, it can be useful to tell it to connect to libvirt +and fetch information. If connecting to a remote libvirt instance, +it will fetch any information that can be trusted, which is the basic +VM launch state data. It will also sanity check the XML configuration +to identify some common mistakes. If the ``--insecure`` flag is passed +it can extract some configuration information and use that for the +attestation process. + +If the mistake still can't be identified, then this tool can be run +on the virtualization host. In that scenario the only three command +line parameters required are for the TIK, TEK and libvirt domain +name. It should be able to automatically determine all the other +information required. If it still reports a failure, this points +very strongly to the TIK/TEK pair not maching the configured +DH certificate and session blob. + +The ``--debug`` flag will display hashes and/or hex dumps for various +pieces of information used in the attestation process. Comparing the +``--debug`` output from running on the hypervisor host, against that +obtained when running in offline mode can give further guidance to +which parameter is inconsistent. + +As mentioned earlier in this document, bear in mind that in general +any attestation answers obtained from running on the hypervisor host +should not be trusted. So if a configuration mistake is identified +it is strongly recommended to re-run the attestation in offline mode +on a trusted machine. + + EXIT STATUS =========== -- 2.37.3