Re: REST service for libvirt to simplify SEV(ES) launch measurement

On Wed, Feb 23, 2022 at 03:33:22PM -0500, Tobin Feldman-Fitzthum wrote: > > > On 2/23/22 1:38 PM, Dov Murik wrote: >> +cc Tobin, James >> >> On 23/02/2022 19:28, Daniel P. Berrangé wrote: >>> Extending management apps using libvirt to support measured launch of >>> QEMU guests with SEV/SEV-ES is unreasonably complicated today, both for >>> the guest owner and for the cloud management apps. We have APIs for >>> exposing info about the SEV host, the SEV guest, guest measurements >>> and secret injections. This is a "bags of bits" solution. We expect >>> apps to them turn this into a user facting solution. It is possible >>> but we're heading to a place where every cloud mgmt app essentially >>> needs to reinvent the same wheel and the guest owner will need to >>> learn custom APIs for dealing with SEV/SEV-ES for each cloud mgmt >>> app. This is pretty awful. We need to do a better job at providing >>> a solution that is more general purpose IMHO. >>> > Note in particular that we provide a client script called LaunchVM.py > that uses libvirt to start an SEV VM in conjunction with the attestation > server. This is basically a stand in for a management app or cloud > control plane. The modifications needed to launch an SEV VM are not > particularly extensive. I agree with some of your comments though. In > some ways it might be nice to have libvirt take care of things and hide > the complexity from the management app. LaunchVM.py nicely illustrates my concerns. Every application that uses libvirt that knows how to start VMs, now needs to be changed to support the series of operations shown in LaunchVM.py. THe guest owner probably can't use LaunchVM.py except for demoware, as they'll need a equivalent that talks to the API of their cloud mgmt app, of which there are many. > When we started working on our attestation server, our initial plan was > to make PRs to libvirt that would add one end of the attestation API to > libvirt, which would directly query the KBS. This is basically what you > are proposing. We decided against this for a couple of reasons. > > First, we were concerned that libvirt might not have network > connectivity to an arbitrary attestation server in a cloud environment. > We had envisioned that the guest owner would provide a URI for their > attestation server as part of the XML. This assumes that the node where > the VM is going to run can connect to an attestation server living > somewhere on the internet. I think that this might be challenging in > some cloud environments. By having the management app connect to libvirt > and the attestation server, we add some flexibility. Agreed, we can't assume that libvirt will always have ability to connect to an arbitrary service on the internet. That said, it does not neccessarily need this ability. If the user gives a URL of 'https://myhost.com/attest', the cloud doesn't have to give that straight to libvirt. The cloud software could have a attestation proxy server. So they could tell libvirt to use the URI https://10.0.0.5/attest, and then libvirt connects to that, it will proxy the calls through the guest owner's real attestation server.

If even that isn't possible though, there's still the fallback option of ignoring libvirt's native support for talking to an attestation server, and doing it manually as per LaunchVM.py illustration. > Second, we were worried that it would be difficult to settle on and > maintain a standard. Fortunately this discussion is only relevant for > SEV(-ES), given that SNP measurements are reported from inside the > guest, but nonetheless there are already a number of approaches for > handling things. By using a management app, each CSP can easily adapt > the standard libvirt api into whatever attestation API they want. This > does put a burden on the management apps, but it might sidestep a tricky > problem for libvirt and like I said, we found it pretty easy to write > our LaunchVM script (except for the CEK issue mentioned elsewhere). The attestation server is ultimately something that the guest owner needs to control / use. Whether the cloud mgmt app conects to it, or if libvirt connects to it, it feels like we would benefit from having a standard that can be used from either approach. I don't think we want to end up with IBM's cloud requiring one attestation server design and OpenStack requiring another and KubeVirt requiring yet another, etc. Guest owners souldn't be given the burden of using different services depending on which cloud they deploy on each time, as that would effectively become a form of vendor lockin.

The needs of all the apps look similar enough, because they are all ultimately constrained by the functionality made available by SEV(ES). Thus at least wrt apps doing traditional VM management using QEMU, it feels like it should be practical to come up with a common solution. I am all for common ground here. I had basically given up on it, but

I can understand if it is harder to achieve commonality with tech like libkrun though, since that's consuming virt in quite a different way at the userspace level.

> Finally, we didn't think that there would be any interest in the libvirt > community. It seems like we might have been wrong about this. Like I > said, our first instinct was to extend libvirt, and if there is interest > in doing this, we could dust off those ideas. I certainly have a lot of > ideas about how to design an API for attestation. Of course we now have > an API for attestation that we think is pretty good. It is gRPC, but we > are thinking about also supporting a REST interface. If an attestation > api is added to libvirt, I will definitely try to be involved, although > honestly I think it's fine, and in some ways maybe better, to have the > management app take care of things. Thanks for the feedback so far. Regards, Daniel