[libvirt PATCH v2 00/10] make internal only secrets work with split daemons
If you define a secret with private="yes", then libvirt won't let any client query the secret value after it is set. Only other libvirt drivers inside the daemon can query it by passing a special internal only flag to the virSecretGetValue API. The remote driver/daemon refuses to let this internal flag go over the wire preventing normal clients from using it This doesn't work with the split daemons because the virSecretGetValue API done by virqemud / virtstoraged has to go over the wire to reach the virsecretd. We need to come up with an alternative way to "prove" that the caller of virSecretGetValue is a libvirt daemon, as opposed to a general libvirt client. Note with if only traditional POSIX DAC permissions are in effect then we could consider it pointless trying to restrict access to clients running the same user/group as the libvirt daemon. We ought to take into account that the client might be confined by SELinux though, so the "private secret" concept isn't entirely pointless. Thus doing a simple uid of client == uid of daemon check is a bit too weak. The UID check might also not fly if the modular daemons are run inside containers with user namespaces, as the container for virtsecretd and virtqemud might have different user mappings in theory. This series adds a concept of a "token" which is known only to the libvirt daemons. The first daemon to use it writes a random hex string to /var/run/libvirt/common/system.token. Other daemons can read and compare this. Unless a MAC system is present this is still largely security theatre, but that's not really worse than the historical behaviour. When an API call is made the virIdentity by default reflects the identity of the UNIX process that initiated it. When connecting to virtproxyd, the client apps' identity is forwarded to the next virtNNNNd daemon. When libvirt drivers, however, initiate an API call we never set any identity. With monolithic libvirtd, they'd inherit the current client identity automagically since it was all in the same thread local. With modular daemons the othe driver would see the identity of the other libvirt daemon which is bad as this gives elevated privileges in the ACL check. Thus we fix the code which drivers use to open a connection to other daemons, such that it applies the current caller's identity. It does this using an "elevated" identity though, which means, we have added in the system token. Thus the virtsecretd daemon getting the call virSecretGetValue sees the virIdentity reflecting the client application which originally called the virDomainCreate() API, but with the system token set. Thus virsecretd can see that the virSecretGetValue was invoked by another daemon, not a libvirt client app. Changed in v2... We can't set the elevated identity only when opening the virConnect for the secret driver. This works for modular daemons, as the identity is passed to the virsecretd at time of opening and thus applies to the later virSecretGetValue call on that connection. For monolithic daemon, the identity present at virConnectOpen is irrelevant. The virSecretGetValue call will just directly query the current thread's identity. IOW, to work in both deployment scenarios we need to have the elevated identity set across both virConnectOpen and virSecretGetValue Daniel P. Berrangé (10): util: add virRandomToken API util: introduce concept of a system token into identities util: generate a persistent system token util: set system token for system identity util: add API for copying identity objects util: helper to temporary elevate privileges of the current identity src: add API to determine if current identity is a system identity src: set identity when opening secondary drivers src: elevate current identity privilege when fetching secret secret: rework handling of private secrets src/driver-secret.h | 9 +- src/driver.c | 27 +++ src/libvirt-secret.c | 2 +- src/libvirt_private.syms | 7 + src/libxl/libxl_conf.c | 5 + src/qemu/qemu_domain.c | 11 +- src/qemu/qemu_tpm.c | 5 + src/remote/remote_driver.c | 8 +- src/secret/secret_driver.c | 34 ++- src/storage/storage_backend_iscsi.c | 5 + src/storage/storage_backend_iscsi_direct.c | 5 + src/storage/storage_backend_rbd.c | 5 + src/storage/storage_util.c | 5 + src/util/viridentity.c | 239 +++++++++++++++++++++ src/util/viridentity.h | 11 + src/util/virrandom.c | 18 ++ src/util/virrandom.h | 1 + src/util/virsecret.c | 3 +- tests/qemuxml2argvtest.c | 3 +- 19 files changed, 375 insertions(+), 28 deletions(-) -- 2.31.1
A random token is simply a string of random bytes formatted in hexidecimal. Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/libvirt_private.syms | 1 + src/util/virrandom.c | 18 ++++++++++++++++++ src/util/virrandom.h | 1 + 3 files changed, 20 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 1b12c49018..23621fcfd0 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -3081,6 +3081,7 @@ virRandomBits; virRandomBytes; virRandomGenerateWWN; virRandomInt; +virRandomToken; # util/virresctrl.h diff --git a/src/util/virrandom.c b/src/util/virrandom.c index 3ae1297e6b..c3f3aa1fa6 100644 --- a/src/util/virrandom.c +++ b/src/util/virrandom.c @@ -161,3 +161,21 @@ virRandomGenerateWWN(char **wwn, (unsigned long long)virRandomBits(36)); return 0; } + +char *virRandomToken(size_t len) +{ + g_autofree unsigned char *data = g_new0(unsigned char, len); + g_autofree char *token = g_new0(char, (len * 2) + 1); + static const char hex[] = "0123456789abcdef"; + size_t i; + + if (virRandomBytes(data, len) < 0) + return NULL; + + for (i = 0; i < len; i++) { + token[(i*2)] = hex[data[i] & 0xf]; + token[(i*2)+1] = hex[(data[i] >> 4) & 0xf]; + } + + return g_steal_pointer(&token); +} diff --git a/src/util/virrandom.h b/src/util/virrandom.h index 297721f912..aac684ada9 100644 --- a/src/util/virrandom.h +++ b/src/util/virrandom.h @@ -26,3 +26,4 @@ uint32_t virRandomInt(uint32_t max); int virRandomBytes(unsigned char *buf, size_t buflen) ATTRIBUTE_NONNULL(1) G_GNUC_WARN_UNUSED_RESULT G_GNUC_NO_INLINE; int virRandomGenerateWWN(char **wwn, const char *virt_type) G_GNUC_NO_INLINE; +char *virRandomToken(size_t len); -- 2.31.1
We want a way to distinguish between calls from a libvirt daemon, and a regular client application when both are running as the same user account. This is not possible with the current set of attributes recorded against an identity, as there is nothing that is common to all of the modular libvirt daemons, while distinct to all other processes. We thus introduce the idea of a system token, which is simply a random hex string that is only known by the libvirt daemons, to be recored against the system identity. Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/libvirt_private.syms | 2 ++ src/util/viridentity.c | 34 ++++++++++++++++++++++++++++++++++ src/util/viridentity.h | 4 ++++ 3 files changed, 40 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 23621fcfd0..aaae1c8002 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2404,6 +2404,7 @@ virIdentityGetProcessTime; virIdentityGetSASLUserName; virIdentityGetSELinuxContext; virIdentityGetSystem; +virIdentityGetSystemToken; virIdentityGetUNIXGroupID; virIdentityGetUNIXUserID; virIdentityGetUserName; @@ -2416,6 +2417,7 @@ virIdentitySetProcessID; virIdentitySetProcessTime; virIdentitySetSASLUserName; virIdentitySetSELinuxContext; +virIdentitySetSystemToken; virIdentitySetUNIXGroupID; virIdentitySetUNIXUserID; virIdentitySetUserName; diff --git a/src/util/viridentity.c b/src/util/viridentity.c index 7edb6a171a..7da4ea12f5 100644 --- a/src/util/viridentity.c +++ b/src/util/viridentity.c @@ -40,6 +40,8 @@ #define VIR_FROM_THIS VIR_FROM_IDENTITY +#define VIR_CONNECT_IDENTITY_SYSTEM_TOKEN "system.token" + VIR_LOG_INIT("util.identity"); struct _virIdentity { @@ -382,6 +384,17 @@ int virIdentityGetSELinuxContext(virIdentity *ident, } +int virIdentityGetSystemToken(virIdentity *ident, + const char **token) +{ + *token = NULL; + return virTypedParamsGetString(ident->params, + ident->nparams, + VIR_CONNECT_IDENTITY_SYSTEM_TOKEN, + token); +} + + int virIdentitySetUserName(virIdentity *ident, const char *username) { @@ -554,6 +567,25 @@ int virIdentitySetSELinuxContext(virIdentity *ident, } +int virIdentitySetSystemToken(virIdentity *ident, + const char *token) +{ + if (virTypedParamsGet(ident->params, + ident->nparams, + VIR_CONNECT_IDENTITY_SYSTEM_TOKEN)) { + virReportError(VIR_ERR_OPERATION_DENIED, "%s", + _("Identity attribute is already set")); + return -1; + } + + return virTypedParamsAddString(&ident->params, + &ident->nparams, + &ident->maxparams, + VIR_CONNECT_IDENTITY_SYSTEM_TOKEN, + token); +} + + int virIdentitySetParameters(virIdentity *ident, virTypedParameterPtr params, int nparams) @@ -577,6 +609,8 @@ int virIdentitySetParameters(virIdentity *ident, VIR_TYPED_PARAM_STRING, VIR_CONNECT_IDENTITY_SELINUX_CONTEXT, VIR_TYPED_PARAM_STRING, + VIR_CONNECT_IDENTITY_SYSTEM_TOKEN, + VIR_TYPED_PARAM_STRING, NULL) < 0) return -1; diff --git a/src/util/viridentity.h b/src/util/viridentity.h index fa3f46788c..640a7ba2e4 100644 --- a/src/util/viridentity.h +++ b/src/util/viridentity.h @@ -52,6 +52,8 @@ int virIdentityGetX509DName(virIdentity *ident, const char **dname); int virIdentityGetSELinuxContext(virIdentity *ident, const char **context); +int virIdentityGetSystemToken(virIdentity *ident, + const char **token); int virIdentitySetUserName(virIdentity *ident, @@ -72,6 +74,8 @@ int virIdentitySetX509DName(virIdentity *ident, const char *dname); int virIdentitySetSELinuxContext(virIdentity *ident, const char *context); +int virIdentitySetSystemToken(virIdentity *ident, + const char *token); int virIdentitySetParameters(virIdentity *ident, virTypedParameterPtr params, -- 2.31.1
On 5/7/21 11:24 AM, Daniel P. Berrangé wrote:
We want a way to distinguish between calls from a libvirt daemon, and a regular client application when both are running as the same user account. This is not possible with the current set of attributes recorded against an identity, as there is nothing that is common to all of the modular libvirt daemons, while distinct to all other processes.
We thus introduce the idea of a system token, which is simply a random hex string that is only known by the libvirt daemons, to be recored
recorded
against the system identity.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
When creating the system identity set the system token. The system token is currently stored in a local path /var/run/libvirt/common/system.token Obviously with only traditional UNIX DAC in effect, this is largely security through obscurity, if the client is running at the same privilege level as the daemon. It does, however, reliably distinguish an unprivilegd client from the system daemons. With a MAC system like SELinux though, or possible use of containers, access can be further restricted. A possible future improvement for Linux would be to populate the kernel keyring with a secret for libvirt daemons to share. Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/util/viridentity.c | 102 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) diff --git a/src/util/viridentity.c b/src/util/viridentity.c index 7da4ea12f5..8c939a507e 100644 --- a/src/util/viridentity.c +++ b/src/util/viridentity.c @@ -22,6 +22,7 @@ #include <config.h> #include <unistd.h> +#include <fcntl.h> #if WITH_SELINUX # include <selinux/selinux.h> #endif @@ -32,11 +33,14 @@ #include "viridentity.h" #include "virlog.h" #include "virobject.h" +#include "virrandom.h" #include "virthread.h" #include "virutil.h" #include "virstring.h" #include "virprocess.h" #include "virtypedparam.h" +#include "virfile.h" +#include "configmake.h" #define VIR_FROM_THIS VIR_FROM_IDENTITY @@ -54,7 +58,10 @@ struct _virIdentity { G_DEFINE_TYPE(virIdentity, vir_identity, G_TYPE_OBJECT) +static char *virIdentityEnsureSystemToken(void); + static virThreadLocal virIdentityCurrent; +static char *systemToken; static void virIdentityFinalize(GObject *obj); @@ -73,6 +80,9 @@ static int virIdentityOnceInit(void) return -1; } + if (!(systemToken = virIdentityEnsureSystemToken())) + return -1; + return 0; } @@ -144,6 +154,98 @@ int virIdentitySetCurrent(virIdentity *ident) } +#define TOKEN_BYTES 16 +#define TOKEN_STRLEN (TOKEN_BYTES * 2) + +static char * +virIdentityConstructSystemTokenPath(void) +{ + g_autofree char *commondir = NULL; + if (geteuid() == 0) { + commondir = g_strdup(RUNSTATEDIR "/libvirt/common"); + } else { + g_autofree char *rundir = virGetUserRuntimeDirectory(); + commondir = g_strdup_printf("%s/common", rundir); + } + + if (g_mkdir_with_parents(commondir, 0700) < 0) { + virReportSystemError(errno, + _("Cannot create daemon common directory '%s'"), + commondir); + return NULL; + } + + return g_strdup_printf("%s/system.token", commondir); +} + + +static char * +virIdentityEnsureSystemToken(void) +{ + g_autofree char *tokenfile = virIdentityConstructSystemTokenPath(); + g_autofree char *token = NULL; + VIR_AUTOCLOSE fd = -1; + struct stat st; + + fd = open(tokenfile, O_RDWR|O_APPEND|O_CREAT, 0600); + if (fd < 0) { + virReportSystemError(errno, + _("Unable to open system token %s"), + tokenfile); + return NULL; + } + + if (virSetCloseExec(fd) < 0) { + virReportSystemError(errno, + _("Failed to set close-on-exec flag '%s'"), + tokenfile); + return NULL; + } + + if (virFileLock(fd, false, 0, 1, true) < 0) { + virReportSystemError(errno, + _("Failed to lock system token '%s'"), + tokenfile); + return NULL; + } + + if (fstat(fd, &st) < 0) { + virReportSystemError(errno, + _("Failed to check system token '%s'"), + tokenfile); + return NULL; + } + + /* Ok, we're the first one here, so we must populate it */ + if (st.st_size == 0) { + if (!(token = virRandomToken(TOKEN_BYTES))) { + return NULL; + } + if (safewrite(fd, token, TOKEN_STRLEN) != TOKEN_STRLEN) { + virReportSystemError(errno, + _("Failed to write system token '%s'"), + tokenfile); + return NULL; + } + } else { + if (virFileReadLimFD(fd, TOKEN_STRLEN, &token) < 0) { + virReportSystemError(errno, + _("Failed to write system token '%s'"), + tokenfile); + return NULL; + } + if (strlen(token) != TOKEN_STRLEN) { + virReportSystemError(errno, + _("System token in %s was corrupt"), + tokenfile); + return NULL; + } + } + + return g_steal_pointer(&token); +} + + /** * virIdentityGetSystem: * -- 2.31.1
On 5/7/21 11:24 AM, Daniel P. Berrangé wrote:
When creating the system identity set the system token. The system token is currently stored in a local path
/var/run/libvirt/common/system.token
Obviously with only traditional UNIX DAC in effect, this is largely security through obscurity, if the client is running at the same privilege level as the daemon. It does, however, reliably distinguish an unprivilegd client from the system daemons.
unprivileged
With a MAC system like SELinux though, or possible use of containers, access can be further restricted.
A possible future improvement for Linux would be to populate the kernel keyring with a secret for libvirt daemons to share.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/util/viridentity.c | 102 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+)
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 5/7/21 6:24 PM, Daniel P. Berrangé wrote:
When creating the system identity set the system token. The system token is currently stored in a local path
/var/run/libvirt/common/system.token
Obviously with only traditional UNIX DAC in effect, this is largely security through obscurity, if the client is running at the same privilege level as the daemon. It does, however, reliably distinguish an unprivilegd client from the system daemons.
With a MAC system like SELinux though, or possible use of containers, access can be further restricted.
A possible future improvement for Linux would be to populate the kernel keyring with a secret for libvirt daemons to share.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/util/viridentity.c | 102 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+)
diff --git a/src/util/viridentity.c b/src/util/viridentity.c index 7da4ea12f5..8c939a507e 100644 --- a/src/util/viridentity.c +++ b/src/util/viridentity.c
@@ -144,6 +154,98 @@ int virIdentitySetCurrent(virIdentity *ident) }
+#define TOKEN_BYTES 16 +#define TOKEN_STRLEN (TOKEN_BYTES * 2) + +static char * +virIdentityConstructSystemTokenPath(void) +{ + g_autofree char *commondir = NULL; + if (geteuid() == 0) { + commondir = g_strdup(RUNSTATEDIR "/libvirt/common"); + } else { + g_autofree char *rundir = virGetUserRuntimeDirectory(); + commondir = g_strdup_printf("%s/common", rundir); + } + + if (g_mkdir_with_parents(commondir, 0700) < 0) { + virReportSystemError(errno, + _("Cannot create daemon common directory '%s'"), + commondir); + return NULL; + } + + return g_strdup_printf("%s/system.token", commondir); +} + + +static char * +virIdentityEnsureSystemToken(void) +{ + g_autofree char *tokenfile = virIdentityConstructSystemTokenPath(); + g_autofree char *token = NULL; + VIR_AUTOCLOSE fd = -1; + struct stat st; +
Sorry for not spotting this in v1, but @tokenfile can be NULL here, in which case virIdentityConstructSystemTokenPath() already reported accurate error. Something like: if (!tokenfile) return NULL; is sufficient.
+ fd = open(tokenfile, O_RDWR|O_APPEND|O_CREAT, 0600); + if (fd < 0) { + virReportSystemError(errno, + _("Unable to open system token %s"), + tokenfile); + return NULL; + } +
Also, I believe you will want to mock this function, because if you don't then the viridentitytest starts failing after next patch. Something among these lines: https://gitlab.com/MichalPrivoznik/libvirt/-/commit/5594631df3b3512adecea0f5... Michal
Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/util/viridentity.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/util/viridentity.c b/src/util/viridentity.c index 8c939a507e..dabe416037 100644 --- a/src/util/viridentity.c +++ b/src/util/viridentity.c @@ -263,6 +263,7 @@ virIdentity *virIdentityGetSystem(void) #if WITH_SELINUX char *con; #endif + g_autofree char *token = NULL; if (!(ret = virIdentityNew())) return NULL; @@ -305,6 +306,12 @@ virIdentity *virIdentityGetSystem(void) } #endif + if (!(token = virIdentityEnsureSystemToken())) + return NULL; + + if (virIdentitySetSystemToken(ret, token) < 0) + return NULL; + return g_steal_pointer(&ret); } -- 2.31.1
Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/libvirt_private.syms | 1 + src/util/viridentity.c | 21 +++++++++++++++++++++ src/util/viridentity.h | 1 + 3 files changed, 23 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index aaae1c8002..de5123aaa9 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2410,6 +2410,7 @@ virIdentityGetUNIXUserID; virIdentityGetUserName; virIdentityGetX509DName; virIdentityNew; +virIdentityNewCopy; virIdentitySetCurrent; virIdentitySetGroupName; virIdentitySetParameters; diff --git a/src/util/viridentity.c b/src/util/viridentity.c index dabe416037..9ffaf57da9 100644 --- a/src/util/viridentity.c +++ b/src/util/viridentity.c @@ -330,6 +330,27 @@ virIdentity *virIdentityNew(void) } +/** + * virIdentityNewCopy: + * + * Creates a new identity object that is a deep copy of an + * existing identity. + * + * Returns: a copy of the source identity + */ +virIdentity *virIdentityNewCopy(virIdentity *src) +{ + g_autoptr(virIdentity) ident = virIdentityNew(); + + if (virTypedParamsCopy(&ident->params, src->params, src->nparams) < 0) + return NULL; + ident->nparams = src->nparams; + ident->maxparams = src->nparams; + + return g_steal_pointer(&ident); +} + + static void virIdentityFinalize(GObject *object) { virIdentity *ident = VIR_IDENTITY(object); diff --git a/src/util/viridentity.h b/src/util/viridentity.h index 640a7ba2e4..512bca286d 100644 --- a/src/util/viridentity.h +++ b/src/util/viridentity.h @@ -33,6 +33,7 @@ int virIdentitySetCurrent(virIdentity *ident); virIdentity *virIdentityGetSystem(void); virIdentity *virIdentityNew(void); +virIdentity *virIdentityNewCopy(virIdentity *src); int virIdentityGetUserName(virIdentity *ident, const char **username); -- 2.31.1
When talking to the secret driver, the callers inside libvirt daemons need to be able to run with an elevated privileges that prove the API calls are made by a libvirt daemon, not an end user application. The virIdentityElevateCurrent method will take the current identity and, if not already present, add the system token. The old current identity is returned to the caller. With the VIR_IDENTITY_AUTORESTORE annotation, the old current identity will be restored upon leaving the codeblock scope. ... early work with regular privileges ... if (something needing elevated privs) { VIR_IDENTITY_AUTORESTORE virIdentity *oldident = virIdentityElevateCurrent(); if (!oldident) return -1; ... do something with elevated privileges ... } ... later work with regular privileges ... Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/libvirt_private.syms | 2 ++ src/util/viridentity.c | 47 ++++++++++++++++++++++++++++++++++++++++ src/util/viridentity.h | 5 +++++ 3 files changed, 54 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index de5123aaa9..7db04d3d3b 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2396,6 +2396,7 @@ virHostGetBootTime; # util/viridentity.h +virIdentityElevateCurrent; virIdentityGetCurrent; virIdentityGetGroupName; virIdentityGetParameters; @@ -2411,6 +2412,7 @@ virIdentityGetUserName; virIdentityGetX509DName; virIdentityNew; virIdentityNewCopy; +virIdentityRestoreHelper; virIdentitySetCurrent; virIdentitySetGroupName; virIdentitySetParameters; diff --git a/src/util/viridentity.c b/src/util/viridentity.c index 9ffaf57da9..a9f54232b9 100644 --- a/src/util/viridentity.c +++ b/src/util/viridentity.c @@ -154,6 +154,53 @@ int virIdentitySetCurrent(virIdentity *ident) } +/** + * virIdentityElevateCurrent: + * + * Set the new identity to be associated with this thread, + * to an elevated copy of the current identity. The old + * current identity is returned and should be released by + * the caller when no longer required. + * + * Returns the previous identity, or NULL on error + */ +virIdentity *virIdentityElevateCurrent(void) +{ + g_autoptr(virIdentity) ident = virIdentityGetCurrent(); + const char *token; + int rc; + + if (!ident) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("No current identity to elevate")); + return NULL; + } + + if ((rc = virIdentityGetSystemToken(ident, &token)) < 0) + return NULL; + + if (rc == 0) { + g_autoptr(virIdentity) identel = virIdentityNewCopy(ident); + + if (virIdentitySetSystemToken(identel, systemToken) < 0) + return NULL; + + if (virIdentitySetCurrent(identel) < 0) + return NULL; + } + + return g_steal_pointer(&ident); +} + + +void virIdentityRestoreHelper(virIdentity **identptr) +{ + virIdentity *ident = *identptr; + + if (ident != NULL) + virIdentitySetCurrent(ident); +} + #define TOKEN_BYTES 16 #define TOKEN_STRLEN (TOKEN_BYTES * 2) diff --git a/src/util/viridentity.h b/src/util/viridentity.h index 512bca286d..848e5b2056 100644 --- a/src/util/viridentity.h +++ b/src/util/viridentity.h @@ -27,8 +27,13 @@ #define VIR_TYPE_IDENTITY vir_identity_get_type() G_DECLARE_FINAL_TYPE(virIdentity, vir_identity, VIR, IDENTITY, GObject); +#define VIR_IDENTITY_AUTORESTORE __attribute__((cleanup(virIdentityRestoreHelper))) + virIdentity *virIdentityGetCurrent(void); int virIdentitySetCurrent(virIdentity *ident); +virIdentity *virIdentityElevateCurrent(void); + +void virIdentityRestoreHelper(virIdentity **identptr); virIdentity *virIdentityGetSystem(void); -- 2.31.1
This is essentially a way to determine if the current identity is that of another libvirt daemon. Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/libvirt_private.syms | 1 + src/util/viridentity.c | 28 ++++++++++++++++++++++++++++ src/util/viridentity.h | 1 + 3 files changed, 30 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 7db04d3d3b..aecb803369 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2410,6 +2410,7 @@ virIdentityGetUNIXGroupID; virIdentityGetUNIXUserID; virIdentityGetUserName; virIdentityGetX509DName; +virIdentityIsCurrentElevated; virIdentityNew; virIdentityNewCopy; virIdentityRestoreHelper; diff --git a/src/util/viridentity.c b/src/util/viridentity.c index a9f54232b9..d98a7d77d1 100644 --- a/src/util/viridentity.c +++ b/src/util/viridentity.c @@ -363,6 +363,34 @@ virIdentity *virIdentityGetSystem(void) } +/** + * virIdentityIsCurrentElevated: + * + * Determine if the current identity has elevated privileges. + * This indicates that it was invoked on behalf of the + * user by a libvirt daemon. + * + * Returns: true if elevated + */ +int virIdentityIsCurrentElevated(void) +{ + g_autoptr(virIdentity) current = virIdentityGetCurrent(); + const char *currentToken = NULL; + int rv; + + if (!current) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("No current identity")); + return -1; + } + + rv = virIdentityGetSystemToken(current, ¤tToken); + if (rv <= 0) + return rv; + + return STREQ_NULLABLE(currentToken, systemToken); +} + /** * virIdentityNew: * diff --git a/src/util/viridentity.h b/src/util/viridentity.h index 848e5b2056..6da6d0c557 100644 --- a/src/util/viridentity.h +++ b/src/util/viridentity.h @@ -35,6 +35,7 @@ virIdentity *virIdentityElevateCurrent(void); void virIdentityRestoreHelper(virIdentity **identptr); +int virIdentityIsCurrentElevated(void); virIdentity *virIdentityGetSystem(void); virIdentity *virIdentityNew(void); -- 2.31.1
The drivers can all call virGetConnectXXX to open a connection to a secondary driver. For example, when creating a encrypted storage volume, the storage driver has to open a secret driver connection, or when starting a guest, the QEMU driver has to open the network driver to lookup a virtual network. When using monolithic libvirtd, the connection has the same effective identity as the client, since everything is still in the same process. When using the modular daemons, however, the remote daemon sees the identity of the calling daemon. This is a mistake as it results in the modular daemons seeing the client with elevated privileges. We need to pass on the current identity explicitly when opening the secondary drivers. This is the same thing that is done by daemon RPC dispatcher code when it is directly forwarding top level API calls from virtproxyd and other daemons. Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/driver.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/src/driver.c b/src/driver.c index f8022d2522..227bb56e48 100644 --- a/src/driver.c +++ b/src/driver.c @@ -33,6 +33,8 @@ #include "virstring.h" #include "virthread.h" #include "virutil.h" +#include "viridentity.h" +#include "datatypes.h" #include "configmake.h" VIR_LOG_INIT("driver"); @@ -136,6 +138,7 @@ static virConnectPtr virGetConnectGeneric(virThreadLocal *threadPtr, const char *name) { virConnectPtr conn; + virErrorPtr saved; if (virConnectCacheInitialize() < 0) return NULL; @@ -153,8 +156,32 @@ virGetConnectGeneric(virThreadLocal *threadPtr, const char *name) conn = virConnectOpen(uri); VIR_DEBUG("Opened new %s connection %p", name, conn); + if (!conn) + return NULL; + + if (conn->driver->connectSetIdentity != NULL) { + g_autoptr(virIdentity) ident = NULL; + virTypedParameterPtr identparams = NULL; + int nidentparams = 0; + + VIR_DEBUG("Attempting to delegate current identity"); + if (!(ident = virIdentityGetCurrent())) + goto error; + + if (virIdentityGetParameters(ident, &identparams, &nidentparams) < 0) + goto error; + + if (virConnectSetIdentity(conn, identparams, nidentparams, 0) < 0) + goto error; + } } return conn; + + error: + saved = virSaveLastError(); + virConnectClose(conn); + virSetError(saved); + return NULL; } -- 2.31.1
When fetching the value of a private secret, we need to use an elevated identity otherwise the secret driver will deny access. When using the modular daemons, the elevated identity needs to be active before the secret driver connection is opened, and it will apply to all APIs calls made on that conncetion. When using the monolithic daemon, the identity at time of opening the connection is ignored, and the elevated identity needs to be active precisely at the time the virSecretGetValue API call is made. After acquiring the secret value, the elevated identity should be cleared. This sounds complex, but is fairly straightfoward with the automatic cleanup callbacks. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/libxl/libxl_conf.c | 5 +++++ src/qemu/qemu_domain.c | 11 ++++++++++- src/qemu/qemu_tpm.c | 5 +++++ src/storage/storage_backend_iscsi.c | 5 +++++ src/storage/storage_backend_iscsi_direct.c | 5 +++++ src/storage/storage_backend_rbd.c | 5 +++++ src/storage/storage_util.c | 5 +++++ 7 files changed, 40 insertions(+), 1 deletion(-) diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c index 4de2158bea..e33297a9ba 100644 --- a/src/libxl/libxl_conf.c +++ b/src/libxl/libxl_conf.c @@ -31,6 +31,7 @@ #include "datatypes.h" #include "virconf.h" #include "virfile.h" +#include "viridentity.h" #include "virstring.h" #include "viralloc.h" #include "viruuid.h" @@ -1001,6 +1002,10 @@ libxlMakeNetworkDiskSrc(virStorageSource *src, char **srcstr) if (src->auth && src->protocol == VIR_STORAGE_NET_PROTOCOL_RBD) { g_autofree uint8_t *secret = NULL; size_t secretlen = 0; + VIR_IDENTITY_AUTORESTORE virIdentity *oldident = virIdentityElevateCurrent(); + + if (!oldident) + goto cleanup; username = src->auth->username; if (!(conn = virConnectOpen("xen:///system"))) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index fe56d17486..10641846b3 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -41,6 +41,7 @@ #include "viralloc.h" #include "virlog.h" #include "virerror.h" +#include "viridentity.h" #include "cpu/cpu.h" #include "viruuid.h" #include "virfile.h" @@ -1116,9 +1117,13 @@ qemuDomainSecretPlainSetup(qemuDomainSecretInfo *secinfo, const char *username, virSecretLookupTypeDef *seclookupdef) { + VIR_IDENTITY_AUTORESTORE virIdentity *oldident = virIdentityElevateCurrent(); g_autoptr(virConnect) conn = virGetConnectSecret(); int ret = -1; + if (!oldident) + return -1; + if (!conn) return -1; @@ -1213,11 +1218,15 @@ qemuDomainSecretAESSetupFromSecret(qemuDomainObjPrivate *priv, const char *username, virSecretLookupTypeDef *seclookupdef) { - g_autoptr(virConnect) conn = virGetConnectSecret(); qemuDomainSecretInfo *secinfo; g_autofree char *alias = qemuAliasForSecret(srcalias, secretuse); g_autofree uint8_t *secret = NULL; size_t secretlen = 0; + VIR_IDENTITY_AUTORESTORE virIdentity *oldident = virIdentityElevateCurrent(); + g_autoptr(virConnect) conn = virGetConnectSecret(); + + if (!oldident) + return NULL; if (!conn) return NULL; diff --git a/src/qemu/qemu_tpm.c b/src/qemu/qemu_tpm.c index 9ae7e5f94b..477a26dc69 100644 --- a/src/qemu/qemu_tpm.c +++ b/src/qemu/qemu_tpm.c @@ -33,6 +33,7 @@ #include "vircommand.h" #include "viralloc.h" #include "virkmod.h" +#include "viridentity.h" #include "virlog.h" #include "virutil.h" #include "viruuid.h" @@ -366,6 +367,10 @@ qemuTPMSetupEncryption(const unsigned char *secretuuid, virSecretLookupTypeDef seclookupdef = { .type = VIR_SECRET_LOOKUP_TYPE_UUID, }; + VIR_IDENTITY_AUTORESTORE virIdentity *oldident = virIdentityElevateCurrent(); + + if (!oldident) + return -1; conn = virGetConnectSecret(); if (!conn) diff --git a/src/storage/storage_backend_iscsi.c b/src/storage/storage_backend_iscsi.c index 67e59e856c..ed17ed11a6 100644 --- a/src/storage/storage_backend_iscsi.c +++ b/src/storage/storage_backend_iscsi.c @@ -34,6 +34,7 @@ #include "virerror.h" #include "virfile.h" #include "viriscsi.h" +#include "viridentity.h" #include "virlog.h" #include "virobject.h" #include "virstring.h" @@ -263,6 +264,7 @@ virStorageBackendISCSISetAuth(const char *portal, virStorageAuthDef *authdef = source->auth; int ret = -1; virConnectPtr conn = NULL; + VIR_IDENTITY_AUTORESTORE virIdentity *oldident = NULL; if (!authdef || authdef->authType == VIR_STORAGE_AUTH_TYPE_NONE) return 0; @@ -275,6 +277,9 @@ virStorageBackendISCSISetAuth(const char *portal, return -1; } + if (!(oldident = virIdentityElevateCurrent())) + return -1; + conn = virGetConnectSecret(); if (!conn) return -1; diff --git a/src/storage/storage_backend_iscsi_direct.c b/src/storage/storage_backend_iscsi_direct.c index cb5b39baf4..0bff1882b9 100644 --- a/src/storage/storage_backend_iscsi_direct.c +++ b/src/storage/storage_backend_iscsi_direct.c @@ -29,6 +29,7 @@ #include "storage_util.h" #include "viralloc.h" #include "virerror.h" +#include "viridentity.h" #include "virlog.h" #include "virobject.h" #include "virstring.h" @@ -94,6 +95,7 @@ virStorageBackendISCSIDirectSetAuth(struct iscsi_context *iscsi, virStorageAuthDef *authdef = source->auth; int ret = -1; virConnectPtr conn = NULL; + VIR_IDENTITY_AUTORESTORE virIdentity *oldident = NULL; if (!authdef || authdef->authType == VIR_STORAGE_AUTH_TYPE_NONE) return 0; @@ -107,6 +109,9 @@ virStorageBackendISCSIDirectSetAuth(struct iscsi_context *iscsi, return ret; } + if (!(oldident = virIdentityElevateCurrent())) + return -1; + if (!(conn = virGetConnectSecret())) return ret; diff --git a/src/storage/storage_backend_rbd.c b/src/storage/storage_backend_rbd.c index 9fbb2464d1..ce3ab11dd6 100644 --- a/src/storage/storage_backend_rbd.c +++ b/src/storage/storage_backend_rbd.c @@ -27,6 +27,7 @@ #include "storage_backend_rbd.h" #include "storage_conf.h" #include "viralloc.h" +#include "viridentity.h" #include "virlog.h" #include "viruuid.h" #include "virstring.h" @@ -196,6 +197,7 @@ virStorageBackendRBDOpenRADOSConn(virStorageBackendRBDState *ptr, g_autofree char *mon_buff = NULL; if (authdef) { + VIR_IDENTITY_AUTORESTORE virIdentity *oldident = NULL; g_autofree char *rados_key = NULL; int rc; @@ -206,6 +208,9 @@ virStorageBackendRBDOpenRADOSConn(virStorageBackendRBDState *ptr, goto cleanup; } + if (!(oldident = virIdentityElevateCurrent())) + goto cleanup; + conn = virGetConnectSecret(); if (!conn) return -1; diff --git a/src/storage/storage_util.c b/src/storage/storage_util.c index 7efadc2197..2b0d08c65d 100644 --- a/src/storage/storage_util.c +++ b/src/storage/storage_util.c @@ -68,6 +68,7 @@ #include "storage_source_conf.h" #include "virlog.h" #include "virfile.h" +#include "viridentity.h" #include "virjson.h" #include "virqemu.h" #include "virstring.h" @@ -1265,6 +1266,7 @@ storageBackendCreateQemuImgSecretPath(virStoragePoolObj *pool, size_t secretlen = 0; virConnectPtr conn = NULL; VIR_AUTOCLOSE fd = -1; + VIR_IDENTITY_AUTORESTORE virIdentity *oldident = NULL; if (!enc) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", @@ -1279,6 +1281,9 @@ storageBackendCreateQemuImgSecretPath(virStoragePoolObj *pool, return NULL; } + if (!(oldident = virIdentityElevateCurrent())) + return NULL; + conn = virGetConnectSecret(); if (!conn) return NULL; -- 2.31.1
On 5/7/21 6:24 PM, Daniel P. Berrangé wrote:
When fetching the value of a private secret, we need to use an elevated identity otherwise the secret driver will deny access.
When using the modular daemons, the elevated identity needs to be active before the secret driver connection is opened, and it will apply to all APIs calls made on that conncetion.
When using the monolithic daemon, the identity at time of opening the connection is ignored, and the elevated identity needs to be active precisely at the time the virSecretGetValue API call is made.
After acquiring the secret value, the elevated identity should be cleared.
This sounds complex, but is fairly straightfoward with the automatic cleanup callbacks.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/libxl/libxl_conf.c | 5 +++++ src/qemu/qemu_domain.c | 11 ++++++++++- src/qemu/qemu_tpm.c | 5 +++++ src/storage/storage_backend_iscsi.c | 5 +++++ src/storage/storage_backend_iscsi_direct.c | 5 +++++ src/storage/storage_backend_rbd.c | 5 +++++ src/storage/storage_util.c | 5 +++++ 7 files changed, 40 insertions(+), 1 deletion(-)
After this, I see qemuxml2argv test crash (because of NULL passed to open() in the area I'm raising in 03/10). With the fix I'm suggesting I see a different error: internal error: No current identity to elevate That's because we failed to initialize identity. Unfortunately, I will have to leave this up to you. Michal
On Mon, May 10, 2021 at 01:32:20PM +0200, Michal Prívozník wrote:
On 5/7/21 6:24 PM, Daniel P. Berrangé wrote:
When fetching the value of a private secret, we need to use an elevated identity otherwise the secret driver will deny access.
When using the modular daemons, the elevated identity needs to be active before the secret driver connection is opened, and it will apply to all APIs calls made on that conncetion.
When using the monolithic daemon, the identity at time of opening the connection is ignored, and the elevated identity needs to be active precisely at the time the virSecretGetValue API call is made.
After acquiring the secret value, the elevated identity should be cleared.
This sounds complex, but is fairly straightfoward with the automatic cleanup callbacks.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/libxl/libxl_conf.c | 5 +++++ src/qemu/qemu_domain.c | 11 ++++++++++- src/qemu/qemu_tpm.c | 5 +++++ src/storage/storage_backend_iscsi.c | 5 +++++ src/storage/storage_backend_iscsi_direct.c | 5 +++++ src/storage/storage_backend_rbd.c | 5 +++++ src/storage/storage_util.c | 5 +++++ 7 files changed, 40 insertions(+), 1 deletion(-)
After this, I see qemuxml2argv test crash (because of NULL passed to open() in the area I'm raising in 03/10). With the fix I'm suggesting I see a different error:
internal error: No current identity to elevate
That's because we failed to initialize identity. Unfortunately, I will have to leave this up to you.
Yep, the test suite needs to call virIdentitySetCurrent now we have a dependancy on the identity APIs for internal secret access. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
A secret can be marked with the "private" attribute. The intent was that it is not possible for any libvirt client to be able to read the secret value, it would only be accesible from within libvirtd. eg the QEMU driver can read the value to launch a guest. With the modular daemons, the QEMU, storage and secret drivers are all running in separate daemons. The QEMU and storage drivers thus appear to be normal libvirt client's from the POV of the secret driver, and thus they are not able to read a private secret. This is unhelpful. With the previous patches that introduced a "system token" to the identity object, we can now distinguish APIs invoked by libvirt daemons from those invoked by client applications. Reviewed-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- src/driver-secret.h | 9 +-------- src/libvirt-secret.c | 2 +- src/remote/remote_driver.c | 8 +------- src/secret/secret_driver.c | 34 +++++++++++++++++++++++++++------- src/util/virsecret.c | 3 +-- tests/qemuxml2argvtest.c | 3 +-- 6 files changed, 32 insertions(+), 27 deletions(-) diff --git a/src/driver-secret.h b/src/driver-secret.h index eb6e82478c..1d21f62bb3 100644 --- a/src/driver-secret.h +++ b/src/driver-secret.h @@ -24,12 +24,6 @@ # error "Don't include this file directly, only use driver.h" #endif -enum { - /* This getValue call is inside libvirt, override the "private" flag. - This flag cannot be set by outside callers. */ - VIR_SECRET_GET_VALUE_INTERNAL_CALL = 1 << 0, -}; - typedef virSecretPtr (*virDrvSecretLookupByUUID)(virConnectPtr conn, const unsigned char *uuid); @@ -57,8 +51,7 @@ typedef int typedef unsigned char * (*virDrvSecretGetValue)(virSecretPtr secret, size_t *value_size, - unsigned int flags, - unsigned int internalFlags); + unsigned int flags); typedef int (*virDrvSecretUndefine)(virSecretPtr secret); diff --git a/src/libvirt-secret.c b/src/libvirt-secret.c index 75d40f53dc..a427805c7a 100644 --- a/src/libvirt-secret.c +++ b/src/libvirt-secret.c @@ -585,7 +585,7 @@ virSecretGetValue(virSecretPtr secret, size_t *value_size, unsigned int flags) if (conn->secretDriver != NULL && conn->secretDriver->secretGetValue != NULL) { unsigned char *ret; - ret = conn->secretDriver->secretGetValue(secret, value_size, flags, 0); + ret = conn->secretDriver->secretGetValue(secret, value_size, flags); if (ret == NULL) goto error; return ret; diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index 0c72d69933..eed99af127 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -5382,7 +5382,7 @@ remoteDomainBuildQemuMonitorEvent(virNetClientProgram *prog G_GNUC_UNUSED, static unsigned char * remoteSecretGetValue(virSecretPtr secret, size_t *value_size, - unsigned int flags, unsigned int internalFlags) + unsigned int flags) { unsigned char *rv = NULL; remote_secret_get_value_args args; @@ -5391,12 +5391,6 @@ remoteSecretGetValue(virSecretPtr secret, size_t *value_size, remoteDriverLock(priv); - /* internalFlags intentionally do not go over the wire */ - if (internalFlags) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("no internalFlags support")); - goto done; - } - make_nonnull_secret(&args.secret, secret); args.flags = flags; diff --git a/src/secret/secret_driver.c b/src/secret/secret_driver.c index 6ea8cc8ce9..d2175de8ed 100644 --- a/src/secret/secret_driver.c +++ b/src/secret/secret_driver.c @@ -36,6 +36,7 @@ #include "viruuid.h" #include "virerror.h" #include "virfile.h" +#include "viridentity.h" #include "virpidfile.h" #include "configmake.h" #include "virstring.h" @@ -352,8 +353,7 @@ secretSetValue(virSecretPtr secret, static unsigned char * secretGetValue(virSecretPtr secret, size_t *value_size, - unsigned int flags, - unsigned int internalFlags) + unsigned int flags) { unsigned char *ret = NULL; virSecretObj *obj; @@ -368,11 +368,31 @@ secretGetValue(virSecretPtr secret, if (virSecretGetValueEnsureACL(secret->conn, def) < 0) goto cleanup; - if ((internalFlags & VIR_SECRET_GET_VALUE_INTERNAL_CALL) == 0 && - def->isprivate) { - virReportError(VIR_ERR_INVALID_SECRET, "%s", - _("secret is private")); - goto cleanup; + /* + * For historical compat we want to deny access to + * private secrets, even if no ACL driver is + * present. + * + * We need to validate the identity requesting + * the secret value is running as the same user + * credentials as this driver. + * + * ie a non-root libvirt client should not be + * able to request the value from privileged + * libvirt driver. + * + * To apply restrictions to processes running under + * the same user account is out of scope. + */ + if (def->isprivate) { + int rv = virIdentityIsCurrentElevated(); + if (rv < 0) + goto cleanup; + if (rv == 0) { + virReportError(VIR_ERR_INVALID_SECRET, "%s", + _("secret is private")); + goto cleanup; + } } if (!(ret = virSecretObjGetValue(obj))) diff --git a/src/util/virsecret.c b/src/util/virsecret.c index 0695288229..604d900f77 100644 --- a/src/util/virsecret.c +++ b/src/util/virsecret.c @@ -174,8 +174,7 @@ virSecretGetSecretString(virConnectPtr conn, goto cleanup; } - *secret = conn->secretDriver->secretGetValue(sec, secret_size, 0, - VIR_SECRET_GET_VALUE_INTERNAL_CALL); + *secret = conn->secretDriver->secretGetValue(sec, secret_size, 0); if (!*secret) goto cleanup; diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index a9dafe226e..3591b7b9f0 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -42,8 +42,7 @@ static virQEMUDriver driver; static unsigned char * fakeSecretGetValue(virSecretPtr obj G_GNUC_UNUSED, size_t *value_size, - unsigned int fakeflags G_GNUC_UNUSED, - unsigned int internalFlags G_GNUC_UNUSED) + unsigned int fakeflags G_GNUC_UNUSED) { char *secret; secret = g_strdup("AQCVn5hO6HzFAhAAq0NCv8jtJcIcE+HOBlMQ1A"); -- 2.31.1
participants (3)
-
Daniel P. Berrangé -
Eric Blake -
Michal Prívozník