[libvirt] [PATCH v2 0/2] security: Deal with stale XATTRs

v2 of: https://www.redhat.com/archives/libvir-list/2019-August/msg00520.html diff to v1: - use virOnce to obtain host boot time - switched to configure time check of getutxid - dropped host's UUID from timestamp Michal Prívozník (2): util: Introduce virhostuptime security_util: Remove stale XATTRs configure.ac | 1 + src/libvirt_private.syms | 4 + src/security/security_util.c | 196 +++++++++++++++++++++++++++++++- src/util/Makefile.inc.am | 2 + src/util/virhostuptime.c | 81 +++++++++++++ src/util/virhostuptime.h | 27 +++++ tests/qemusecuritymock.c | 12 ++ tools/libvirt_recover_xattrs.sh | 2 +- 8 files changed, 323 insertions(+), 2 deletions(-) create mode 100644 src/util/virhostuptime.c create mode 100644 src/util/virhostuptime.h -- 2.21.0

This module contains function to get host boot time. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- configure.ac | 1 + src/libvirt_private.syms | 4 ++ src/util/Makefile.inc.am | 2 + src/util/virhostuptime.c | 81 ++++++++++++++++++++++++++++++++++++++++ src/util/virhostuptime.h | 27 ++++++++++++++ 5 files changed, 115 insertions(+) create mode 100644 src/util/virhostuptime.c create mode 100644 src/util/virhostuptime.h diff --git a/configure.ac b/configure.ac index f41c6d5d86..6744ace578 100644 --- a/configure.ac +++ b/configure.ac @@ -337,6 +337,7 @@ AC_CHECK_FUNCS_ONCE([\ getpwuid_r \ getrlimit \ getuid \ + getutxid \ if_indextoname \ mmap \ newlocale \ diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 9db4ac7933..c230a852e7 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2135,6 +2135,10 @@ virHostMemGetStats; virHostMemSetParameters; +# util/virhostuptime.h +virHostGetBootTime; + + # util/viridentity.h virIdentityGetAttr; virIdentityGetCurrent; diff --git a/src/util/Makefile.inc.am b/src/util/Makefile.inc.am index a47f333a98..46866cf213 100644 --- a/src/util/Makefile.inc.am +++ b/src/util/Makefile.inc.am @@ -91,6 +91,8 @@ UTIL_SOURCES = \ util/virhostdev.h \ util/virhostmem.c \ util/virhostmem.h \ + util/virhostuptime.c \ + util/virhostuptime.h \ util/viridentity.c \ util/viridentity.h \ util/virinitctl.c \ diff --git a/src/util/virhostuptime.c b/src/util/virhostuptime.c new file mode 100644 index 0000000000..62b781acd5 --- /dev/null +++ b/src/util/virhostuptime.c @@ -0,0 +1,81 @@ +/* + * virhostuptime.c: helper APIs for host uptime + * + * Copyright (C) 2019 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +#include <config.h> + +#ifdef HAVE_GETUTXID +# include <utmpx.h> +#endif + +#include "virhostuptime.h" +#include "virthread.h" + +static unsigned long long bootTime; +static int bootTimeErrno; +static virOnceControl virHostGetBootTimeOnce = VIR_ONCE_CONTROL_INITIALIZER; + +#ifdef HAVE_GETUTXID +static void +virHostGetBootTimeOnceInit(void) +{ + struct utmpx id = {.ut_type = BOOT_TIME}; + struct utmpx *res = NULL; + + if (!(res = getutxid(&id))) { + bootTimeErrno = errno; + } else { + bootTime = res->ut_tv.tv_sec; + } + + endutxent(); +} + +#else /* !HAVE_GETUTXID */ + +static void +virHostGetBootTimeOnceInit(void) +{ + bootTimeErrno = ENOSYS; +} +#endif /* HAVE_GETUTXID */ + +/** + * virHostGetBootTime: + * @when: UNIX timestamp of boot time + * + * Get a UNIX timestamp of host boot time and store it at @when. + * + * Return: 0 on success, + * -1 otherwise. + */ +int +virHostGetBootTime(unsigned long long *when) +{ + if (virOnce(&virHostGetBootTimeOnce, virHostGetBootTimeOnceInit) < 0) + return -1; + + if (bootTimeErrno) { + errno = bootTimeErrno; + return -1; + } + + *when = bootTime; + return 0; +} diff --git a/src/util/virhostuptime.h b/src/util/virhostuptime.h new file mode 100644 index 0000000000..03c1517a64 --- /dev/null +++ b/src/util/virhostuptime.h @@ -0,0 +1,27 @@ +/* + * virhostuptime.h: helper APIs for host uptime + * + * Copyright (C) 2019 Red Hat, Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library. If not, see + * <http://www.gnu.org/licenses/>. + */ + +#pragma once + +#include "internal.h" + +int +virHostGetBootTime(unsigned long long *when) + ATTRIBUTE_NOINLINE; -- 2.21.0

On Thu, Aug 22, 2019 at 13:15:32 +0200, Michal Privoznik wrote:
This module contains function to get host boot time.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- configure.ac | 1 + src/libvirt_private.syms | 4 ++ src/util/Makefile.inc.am | 2 + src/util/virhostuptime.c | 81 ++++++++++++++++++++++++++++++++++++++++ src/util/virhostuptime.h | 27 ++++++++++++++ 5 files changed, 115 insertions(+) create mode 100644 src/util/virhostuptime.c create mode 100644 src/util/virhostuptime.h
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>

It may happen that we leave some XATTRs behind. For instance, on a sudden power loss, the host just shuts down without calling restore on domain paths. This creates a problem, because when the host starts up again, the XATTRs are there but they don't reflect the true state and this may result in libvirt denying start of a domain. To solve this, save a unique timestamp (host boot time) among with our XATTRs. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/security/security_util.c | 196 +++++++++++++++++++++++++++++++- tests/qemusecuritymock.c | 12 ++ tools/libvirt_recover_xattrs.sh | 2 +- 3 files changed, 208 insertions(+), 2 deletions(-) diff --git a/src/security/security_util.c b/src/security/security_util.c index 365b2dd2d6..c65e27e6d4 100644 --- a/src/security/security_util.c +++ b/src/security/security_util.c @@ -22,11 +22,16 @@ #include "virfile.h" #include "virstring.h" #include "virerror.h" +#include "virlog.h" +#include "viruuid.h" +#include "virhostuptime.h" #include "security_util.h" #define VIR_FROM_THIS VIR_FROM_SECURITY +VIR_LOG_INIT("security.security_util"); + /* There are four namespaces available on Linux (xattr(7)): * * user - can be modified by anybody, @@ -83,6 +88,151 @@ virSecurityGetRefCountAttrName(const char *name ATTRIBUTE_UNUSED) } +#ifdef XATTR_NAMESPACE +static char * +virSecurityGetTimestampAttrName(const char *name) +{ + char *ret = NULL; + ignore_value(virAsprintf(&ret, XATTR_NAMESPACE".libvirt.security.timestamp_%s", name)); + return ret; +} +#else /* !XATTR_NAMESPACE */ +static char * +virSecurityGetTimestampAttrName(const char *name ATTRIBUTE_UNUSED) +{ + errno = ENOSYS; + virReportSystemError(errno, "%s", + _("Extended attributes are not supported on this system")); + return NULL; +} +#endif /* !XATTR_NAMESPACE */ + + +static char * +virSecurityGetTimestamp(void) +{ + unsigned long long boottime = 0; + char *ret = NULL; + + if (virHostGetBootTime(&boottime) < 0) { + virReportSystemError(errno, "%s", + _("Unable to get host boot time")); + return NULL; + } + + ignore_value(virAsprintf(&ret, "%llu", boottime)); + return ret; +} + + +/** + * virSecurityValidateTimestamp: + * @name: security driver name + * @path: file name + * + * Check if remembered label on @path for security driver @name + * is valid, i.e. the label has been set since the last boot. If + * the label was set in previous runs, all XATTRs related to + * @name are removed so that clean slate is restored. + * + * This is done having extra attribute timestamp_$SECDRIVER which + * contains the host boot time. Its value is then compared to + * actual host boot time. If these two values don't match then + * XATTRs are considered as stale and thus invalid. + * + * In ideal world, where there network file systems have XATTRs + * using plain host boot time is not enough as it may lead to a + * situation where a freshly started host sees XATTRs, sees the + * timestamp put there by some longer running host and considers + * the XATTRs invalid. Well, there is not an easy way out. We + * would need to somehow check if the longer running host is + * still there and is the @path (how?). + * Fortunately, there is only one network file system which + * supports XATTRs currently (GlusterFS via FUSE) and it is used + * so rarely that it's almost a corner case. + * The worst thing that happens there is that we remove XATTRs + * and thus return @path to the default label for $SECDRIVER. + * + * Returns: 0 if remembered label is valid, + * 1 if remembered label was not valid, + * -1 otherwise. + */ +static int +virSecurityValidateTimestamp(const char *name, + const char *path) +{ + VIR_AUTOFREE(char *) expected_timestamp = NULL; + VIR_AUTOFREE(char *) timestamp_name = NULL; + VIR_AUTOFREE(char *) value = NULL; + + if (!(expected_timestamp = virSecurityGetTimestamp()) || + !(timestamp_name = virSecurityGetTimestampAttrName(name))) + return -1; + + errno = 0; + if (virFileGetXAttrQuiet(path, timestamp_name, &value) < 0) { + if (errno == ENOSYS || errno == ENOTSUP) { + return -1; + } else if (errno != ENODATA) { + virReportSystemError(errno, + _("Unable to get XATTR %s on %s"), + timestamp_name, + path); + return -1; + } + + /* Timestamp is missing. We can continue and claim a valid timestamp. + * But then we would never remove stale XATTRs. Therefore, claim it + * invalid and have the code below remove all XATTRs. This of course + * means that we will not restore the original owner, but the plus side + * is that we reset refcounter which will represent the true state. + */ + } + + if (STREQ_NULLABLE(value, expected_timestamp)) { + VIR_DEBUG("XATTRs on %s secdriver=%s are valid", path, name); + return 0; + } + + VIR_WARN("Invalid XATTR timestamp detected on %s secdriver=%s", path, name); + + if (virSecurityMoveRememberedLabel(name, path, NULL) < 0) + return -1; + + return 1; +} + + +static int +virSecurityAddTimestamp(const char *name, + const char *path) +{ + VIR_AUTOFREE(char *) timestamp_name = NULL; + VIR_AUTOFREE(char *) timestamp_value = NULL; + + if (!(timestamp_name = virSecurityGetTimestampAttrName(name))) + return -1; + + return virFileSetXAttr(path, timestamp_name, timestamp_value); +} + + +static int +virSecurityRemoveTimestamp(const char *name, + const char *path) +{ + VIR_AUTOFREE(char *) timestamp_name = NULL; + + if (!(timestamp_name = virSecurityGetTimestampAttrName(name))) + return -1; + + if (virFileRemoveXAttr(path, timestamp_name) < 0 && errno != ENOENT) + return -1; + + return 0; +} + + /** * virSecurityGetRememberedLabel: * @name: security driver name @@ -120,6 +270,12 @@ virSecurityGetRememberedLabel(const char *name, *label = NULL; + if (virSecurityValidateTimestamp(name, path) < 0) { + if (errno == ENOSYS || errno == ENOTSUP) + return -2; + return -1; + } + if (!(ref_name = virSecurityGetRefCountAttrName(name))) return -1; @@ -163,6 +319,9 @@ virSecurityGetRememberedLabel(const char *name, if (virFileRemoveXAttr(path, attr_name) < 0) return -1; + + if (virSecurityRemoveTimestamp(name, path) < 0) + return -1; } return 0; @@ -199,6 +358,12 @@ virSecuritySetRememberedLabel(const char *name, VIR_AUTOFREE(char *) value = NULL; unsigned int refcount = 0; + if (virSecurityValidateTimestamp(name, path) < 0) { + if (errno == ENOSYS || errno == ENOTSUP) + return -2; + return -1; + } + if (!(ref_name = virSecurityGetRefCountAttrName(name))) return -1; @@ -232,6 +397,9 @@ virSecuritySetRememberedLabel(const char *name, if (virFileSetXAttr(path, attr_name, label) < 0) return -1; + + if (virSecurityAddTimestamp(name, path) < 0) + return -1; } if (virAsprintf(&value, "%u", refcount) < 0) @@ -266,9 +434,12 @@ virSecurityMoveRememberedLabel(const char *name, VIR_AUTOFREE(char *) ref_value = NULL; VIR_AUTOFREE(char *) attr_name = NULL; VIR_AUTOFREE(char *) attr_value = NULL; + VIR_AUTOFREE(char *) timestamp_name = NULL; + VIR_AUTOFREE(char *) timestamp_value = NULL; if (!(ref_name = virSecurityGetRefCountAttrName(name)) || - !(attr_name = virSecurityGetAttrName(name))) + !(attr_name = virSecurityGetAttrName(name)) || + !(timestamp_name = virSecurityGetTimestampAttrName(name))) return -1; if (virFileGetXAttrQuiet(src, ref_name, &ref_value) < 0) { @@ -293,6 +464,17 @@ virSecurityMoveRememberedLabel(const char *name, } } + if (virFileGetXAttrQuiet(src, timestamp_name, ×tamp_value) < 0) { + if (errno == ENOSYS || errno == ENOTSUP) { + return -2; + } else if (errno != ENODATA) { + virReportSystemError(errno, + _("Unable to get XATTR %s on %s"), + attr_name, src); + return -1; + } + } + if (ref_value && virFileRemoveXAttr(src, ref_name) < 0) { return -1; @@ -303,6 +485,11 @@ virSecurityMoveRememberedLabel(const char *name, return -1; } + if (timestamp_value && + virFileRemoveXAttr(src, timestamp_name) < 0) { + return -1; + } + if (dst) { if (ref_value && virFileSetXAttr(dst, ref_name, ref_value) < 0) { @@ -314,6 +501,13 @@ virSecurityMoveRememberedLabel(const char *name, ignore_value(virFileRemoveXAttr(dst, ref_name)); return -1; } + + if (timestamp_value && + virFileSetXAttr(dst, timestamp_name, timestamp_value) < 0) { + ignore_value(virFileRemoveXAttr(dst, ref_name)); + ignore_value(virFileRemoveXAttr(dst, attr_name)); + return -1; + } } return 0; diff --git a/tests/qemusecuritymock.c b/tests/qemusecuritymock.c index a15eef29c9..373d64305a 100644 --- a/tests/qemusecuritymock.c +++ b/tests/qemusecuritymock.c @@ -32,6 +32,7 @@ #include "viralloc.h" #include "qemusecuritytest.h" #include "security/security_manager.h" +#include "virhostuptime.h" #define VIR_FROM_THIS VIR_FROM_NONE @@ -488,3 +489,14 @@ virProcessRunInFork(virProcessForkCallback cb, { return cb(-1, opaque); } + + +/* We don't really need to mock this function. The qemusecuritytest doesn't + * care about the actual value. However, travis runs build and tests in a + * container where utmp is missing and thus this function fails. */ +int +virHostGetBootTime(unsigned long long *when) +{ + *when = 1234567890; + return 0; +} diff --git a/tools/libvirt_recover_xattrs.sh b/tools/libvirt_recover_xattrs.sh index 58f02f8dfb..3907413c63 100755 --- a/tools/libvirt_recover_xattrs.sh +++ b/tools/libvirt_recover_xattrs.sh @@ -74,7 +74,7 @@ fi declare -a XATTRS for i in "dac" "selinux"; do for p in ${LIBVIRT_XATTR_PREFIXES[@]}; do - XATTRS+=("$p.$i" "$p.ref_$i") + XATTRS+=("$p.$i" "$p.ref_$i" "$p.timestamp_$i") done done -- 2.21.0

On Thu, Aug 22, 2019 at 13:15:33 +0200, Michal Privoznik wrote:
It may happen that we leave some XATTRs behind. For instance, on a sudden power loss, the host just shuts down without calling restore on domain paths. This creates a problem, because when the host starts up again, the XATTRs are there but they don't reflect the true state and this may result in libvirt denying start of a domain.
To solve this, save a unique timestamp (host boot time) among with our XATTRs.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- src/security/security_util.c | 196 +++++++++++++++++++++++++++++++- tests/qemusecuritymock.c | 12 ++ tools/libvirt_recover_xattrs.sh | 2 +- 3 files changed, 208 insertions(+), 2 deletions(-)
diff --git a/src/security/security_util.c b/src/security/security_util.c index 365b2dd2d6..c65e27e6d4 100644 --- a/src/security/security_util.c +++ b/src/security/security_util.c @@ -22,11 +22,16 @@ #include "virfile.h" #include "virstring.h" #include "virerror.h" +#include "virlog.h" +#include "viruuid.h" +#include "virhostuptime.h"
#include "security_util.h"
#define VIR_FROM_THIS VIR_FROM_SECURITY
+VIR_LOG_INIT("security.security_util"); + /* There are four namespaces available on Linux (xattr(7)): * * user - can be modified by anybody, @@ -83,6 +88,151 @@ virSecurityGetRefCountAttrName(const char *name ATTRIBUTE_UNUSED) }
+#ifdef XATTR_NAMESPACE +static char * +virSecurityGetTimestampAttrName(const char *name) +{ + char *ret = NULL; + ignore_value(virAsprintf(&ret, XATTR_NAMESPACE".libvirt.security.timestamp_%s", name));
I'd put a space between XATTR_NAMESPACE and ".
+ return ret; +} +#else /* !XATTR_NAMESPACE */ +static char * +virSecurityGetTimestampAttrName(const char *name ATTRIBUTE_UNUSED) +{ + errno = ENOSYS; + virReportSystemError(errno, "%s", + _("Extended attributes are not supported on this system")); + return NULL; +} +#endif /* !XATTR_NAMESPACE */ + + +static char * +virSecurityGetTimestamp(void) +{ + unsigned long long boottime = 0; + char *ret = NULL; + + if (virHostGetBootTime(&boottime) < 0) { + virReportSystemError(errno, "%s", + _("Unable to get host boot time")); + return NULL; + } + + ignore_value(virAsprintf(&ret, "%llu", boottime)); + return ret; +} + + +/** + * virSecurityValidateTimestamp: + * @name: security driver name + * @path: file name + * + * Check if remembered label on @path for security driver @name + * is valid, i.e. the label has been set since the last boot. If + * the label was set in previous runs, all XATTRs related to + * @name are removed so that clean slate is restored. + * + * This is done having extra attribute timestamp_$SECDRIVER which + * contains the host boot time. Its value is then compared to + * actual host boot time. If these two values don't match then + * XATTRs are considered as stale and thus invalid. + * + * In ideal world, where there network file systems have XATTRs + * using plain host boot time is not enough as it may lead to a + * situation where a freshly started host sees XATTRs, sees the + * timestamp put there by some longer running host and considers + * the XATTRs invalid. Well, there is not an easy way out. We + * would need to somehow check if the longer running host is + * still there and is the @path (how?).
Looks like there is a missing word in the sentence above.
+ * Fortunately, there is only one network file system which + * supports XATTRs currently (GlusterFS via FUSE) and it is used + * so rarely that it's almost a corner case. + * The worst thing that happens there is that we remove XATTRs + * and thus return @path to the default label for $SECDRIVER. + * + * Returns: 0 if remembered label is valid, + * 1 if remembered label was not valid, + * -1 otherwise. + */ +static int +virSecurityValidateTimestamp(const char *name, + const char *path) +{ + VIR_AUTOFREE(char *) expected_timestamp = NULL; + VIR_AUTOFREE(char *) timestamp_name = NULL; + VIR_AUTOFREE(char *) value = NULL; + + if (!(expected_timestamp = virSecurityGetTimestamp()) || + !(timestamp_name = virSecurityGetTimestampAttrName(name))) + return -1; + + errno = 0; + if (virFileGetXAttrQuiet(path, timestamp_name, &value) < 0) { + if (errno == ENOSYS || errno == ENOTSUP) { + return -1;
This is the only path which returns -1 without setting an error. I know it's because the caller will check errno and do something else then with other paths that return -1, but it's weird. I think it would be much better to just return -2 here and let the caller use only the return value without looking at errno.
+ } else if (errno != ENODATA) { + virReportSystemError(errno, + _("Unable to get XATTR %s on %s"), + timestamp_name, + path); + return -1; + } + + /* Timestamp is missing. We can continue and claim a valid timestamp.
s/can/could/
+ * But then we would never remove stale XATTRs. Therefore, claim it + * invalid and have the code below remove all XATTRs. This of course + * means that we will not restore the original owner, but the plus side + * is that we reset refcounter which will represent the true state. + */ + } + + if (STREQ_NULLABLE(value, expected_timestamp)) { + VIR_DEBUG("XATTRs on %s secdriver=%s are valid", path, name); + return 0; + } + + VIR_WARN("Invalid XATTR timestamp detected on %s secdriver=%s", path, name); + + if (virSecurityMoveRememberedLabel(name, path, NULL) < 0) + return -1; + + return 1; +} + + +static int +virSecurityAddTimestamp(const char *name, + const char *path) +{ + VIR_AUTOFREE(char *) timestamp_name = NULL; + VIR_AUTOFREE(char *) timestamp_value = NULL; + + if (!(timestamp_name = virSecurityGetTimestampAttrName(name)))
Missing timestamp_value = virSecurityGetTimestamp()
+ return -1; + + return virFileSetXAttr(path, timestamp_name, timestamp_value); +} + + +static int +virSecurityRemoveTimestamp(const char *name, + const char *path) +{ + VIR_AUTOFREE(char *) timestamp_name = NULL; + + if (!(timestamp_name = virSecurityGetTimestampAttrName(name))) + return -1; + + if (virFileRemoveXAttr(path, timestamp_name) < 0 && errno != ENOENT) + return -1; + + return 0; +} + + /** * virSecurityGetRememberedLabel: * @name: security driver name @@ -120,6 +270,12 @@ virSecurityGetRememberedLabel(const char *name,
*label = NULL;
+ if (virSecurityValidateTimestamp(name, path) < 0) {
Here you could just return what virSecurityValidateTimestamp returned if you follow what I suggested above.
+ if (errno == ENOSYS || errno == ENOTSUP) + return -2; + return -1; + } + if (!(ref_name = virSecurityGetRefCountAttrName(name))) return -1;
@@ -163,6 +319,9 @@ virSecurityGetRememberedLabel(const char *name,
if (virFileRemoveXAttr(path, attr_name) < 0) return -1; + + if (virSecurityRemoveTimestamp(name, path) < 0) + return -1; }
return 0; @@ -199,6 +358,12 @@ virSecuritySetRememberedLabel(const char *name, VIR_AUTOFREE(char *) value = NULL; unsigned int refcount = 0;
+ if (virSecurityValidateTimestamp(name, path) < 0) {
Here you could just return what virSecurityValidateTimestamp returned if you follow what I suggested above.
+ if (errno == ENOSYS || errno == ENOTSUP) + return -2; + return -1; + } + if (!(ref_name = virSecurityGetRefCountAttrName(name))) return -1;
... Jirka

It may happen that we leave some XATTRs behind. For instance, on a sudden power loss, the host just shuts down without calling restore on domain paths. This creates a problem, because when the host starts up again, the XATTRs are there but they don't reflect the true state and this may result in libvirt denying start of a domain. To solve this, save a unique timestamp (host boot time) among with our XATTRs. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1741140 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> --- diff to v2: - Jirka's suggestions worked in. src/security/security_util.c | 194 +++++++++++++++++++++++++++++++- tests/qemusecuritymock.c | 12 ++ tools/libvirt_recover_xattrs.sh | 2 +- 3 files changed, 206 insertions(+), 2 deletions(-) diff --git a/src/security/security_util.c b/src/security/security_util.c index 365b2dd2d6..31f41cedfd 100644 --- a/src/security/security_util.c +++ b/src/security/security_util.c @@ -22,11 +22,16 @@ #include "virfile.h" #include "virstring.h" #include "virerror.h" +#include "virlog.h" +#include "viruuid.h" +#include "virhostuptime.h" #include "security_util.h" #define VIR_FROM_THIS VIR_FROM_SECURITY +VIR_LOG_INIT("security.security_util"); + /* There are four namespaces available on Linux (xattr(7)): * * user - can be modified by anybody, @@ -83,6 +88,153 @@ virSecurityGetRefCountAttrName(const char *name ATTRIBUTE_UNUSED) } +#ifdef XATTR_NAMESPACE +static char * +virSecurityGetTimestampAttrName(const char *name) +{ + char *ret = NULL; + ignore_value(virAsprintf(&ret, XATTR_NAMESPACE ".libvirt.security.timestamp_%s", name)); + return ret; +} +#else /* !XATTR_NAMESPACE */ +static char * +virSecurityGetTimestampAttrName(const char *name ATTRIBUTE_UNUSED) +{ + errno = ENOSYS; + virReportSystemError(errno, "%s", + _("Extended attributes are not supported on this system")); + return NULL; +} +#endif /* !XATTR_NAMESPACE */ + + +static char * +virSecurityGetTimestamp(void) +{ + unsigned long long boottime = 0; + char *ret = NULL; + + if (virHostGetBootTime(&boottime) < 0) { + virReportSystemError(errno, "%s", + _("Unable to get host boot time")); + return NULL; + } + + ignore_value(virAsprintf(&ret, "%llu", boottime)); + return ret; +} + + +/** + * virSecurityValidateTimestamp: + * @name: security driver name + * @path: file name + * + * Check if remembered label on @path for security driver @name + * is valid, i.e. the label has been set since the last boot. If + * the label was set in previous runs, all XATTRs related to + * @name are removed so that clean slate is restored. + * + * This is done having extra attribute timestamp_$SECDRIVER which + * contains the host boot time. Its value is then compared to + * actual host boot time. If these two values don't match then + * XATTRs are considered as stale and thus invalid. + * + * In ideal world, where there network file systems have XATTRs + * using plain host boot time is not enough as it may lead to a + * situation where a freshly started host sees XATTRs, sees the + * timestamp put there by some longer running host and considers + * the XATTRs invalid. Well, there is not an easy way out. We + * would need to somehow check if the longer running host is + * still there and uses the @path (how?). + * Fortunately, there is only one network file system which + * supports XATTRs currently (GlusterFS via FUSE) and it is used + * so rarely that it's almost a corner case. + * The worst thing that happens there is that we remove XATTRs + * and thus return @path to the default label for $SECDRIVER. + * + * Returns: 0 if remembered label is valid, + * 1 if remembered label was not valid, + * -2 if underlying file system doesn't support XATTRs, + * -1 otherwise. + */ +static int +virSecurityValidateTimestamp(const char *name, + const char *path) +{ + VIR_AUTOFREE(char *) expected_timestamp = NULL; + VIR_AUTOFREE(char *) timestamp_name = NULL; + VIR_AUTOFREE(char *) value = NULL; + + if (!(expected_timestamp = virSecurityGetTimestamp()) || + !(timestamp_name = virSecurityGetTimestampAttrName(name))) + return -1; + + errno = 0; + if (virFileGetXAttrQuiet(path, timestamp_name, &value) < 0) { + if (errno == ENOSYS || errno == ENOTSUP) { + return -2; + } else if (errno != ENODATA) { + virReportSystemError(errno, + _("Unable to get XATTR %s on %s"), + timestamp_name, + path); + return -1; + } + + /* Timestamp is missing. We could continue and claim a valid timestamp. + * But then we would never remove stale XATTRs. Therefore, claim it + * invalid and have the code below remove all XATTRs. This of course + * means that we will not restore the original owner, but the plus side + * is that we reset refcounter which will represent the true state. + */ + } + + if (STREQ_NULLABLE(value, expected_timestamp)) { + VIR_DEBUG("XATTRs on %s secdriver=%s are valid", path, name); + return 0; + } + + VIR_WARN("Invalid XATTR timestamp detected on %s secdriver=%s", path, name); + + if (virSecurityMoveRememberedLabel(name, path, NULL) < 0) + return -1; + + return 1; +} + + +static int +virSecurityAddTimestamp(const char *name, + const char *path) +{ + VIR_AUTOFREE(char *) timestamp_name = NULL; + VIR_AUTOFREE(char *) timestamp_value = NULL; + + if (!(timestamp_value = virSecurityGetTimestamp()) || + !(timestamp_name = virSecurityGetTimestampAttrName(name))) + return -1; + + return virFileSetXAttr(path, timestamp_name, timestamp_value); +} + + +static int +virSecurityRemoveTimestamp(const char *name, + const char *path) +{ + VIR_AUTOFREE(char *) timestamp_name = NULL; + + if (!(timestamp_name = virSecurityGetTimestampAttrName(name))) + return -1; + + if (virFileRemoveXAttr(path, timestamp_name) < 0 && errno != ENOENT) + return -1; + + return 0; +} + + /** * virSecurityGetRememberedLabel: * @name: security driver name @@ -117,9 +269,13 @@ virSecurityGetRememberedLabel(const char *name, VIR_AUTOFREE(char *) attr_name = NULL; VIR_AUTOFREE(char *) value = NULL; unsigned int refcount = 0; + int rc; *label = NULL; + if ((rc = virSecurityValidateTimestamp(name, path)) < 0) + return rc; + if (!(ref_name = virSecurityGetRefCountAttrName(name))) return -1; @@ -163,6 +319,9 @@ virSecurityGetRememberedLabel(const char *name, if (virFileRemoveXAttr(path, attr_name) < 0) return -1; + + if (virSecurityRemoveTimestamp(name, path) < 0) + return -1; } return 0; @@ -198,6 +357,10 @@ virSecuritySetRememberedLabel(const char *name, VIR_AUTOFREE(char *) attr_name = NULL; VIR_AUTOFREE(char *) value = NULL; unsigned int refcount = 0; + int rc; + + if ((rc = virSecurityValidateTimestamp(name, path)) < 0) + return rc; if (!(ref_name = virSecurityGetRefCountAttrName(name))) return -1; @@ -232,6 +395,9 @@ virSecuritySetRememberedLabel(const char *name, if (virFileSetXAttr(path, attr_name, label) < 0) return -1; + + if (virSecurityAddTimestamp(name, path) < 0) + return -1; } if (virAsprintf(&value, "%u", refcount) < 0) @@ -266,9 +432,12 @@ virSecurityMoveRememberedLabel(const char *name, VIR_AUTOFREE(char *) ref_value = NULL; VIR_AUTOFREE(char *) attr_name = NULL; VIR_AUTOFREE(char *) attr_value = NULL; + VIR_AUTOFREE(char *) timestamp_name = NULL; + VIR_AUTOFREE(char *) timestamp_value = NULL; if (!(ref_name = virSecurityGetRefCountAttrName(name)) || - !(attr_name = virSecurityGetAttrName(name))) + !(attr_name = virSecurityGetAttrName(name)) || + !(timestamp_name = virSecurityGetTimestampAttrName(name))) return -1; if (virFileGetXAttrQuiet(src, ref_name, &ref_value) < 0) { @@ -293,6 +462,17 @@ virSecurityMoveRememberedLabel(const char *name, } } + if (virFileGetXAttrQuiet(src, timestamp_name, ×tamp_value) < 0) { + if (errno == ENOSYS || errno == ENOTSUP) { + return -2; + } else if (errno != ENODATA) { + virReportSystemError(errno, + _("Unable to get XATTR %s on %s"), + attr_name, src); + return -1; + } + } + if (ref_value && virFileRemoveXAttr(src, ref_name) < 0) { return -1; @@ -303,6 +483,11 @@ virSecurityMoveRememberedLabel(const char *name, return -1; } + if (timestamp_value && + virFileRemoveXAttr(src, timestamp_name) < 0) { + return -1; + } + if (dst) { if (ref_value && virFileSetXAttr(dst, ref_name, ref_value) < 0) { @@ -314,6 +499,13 @@ virSecurityMoveRememberedLabel(const char *name, ignore_value(virFileRemoveXAttr(dst, ref_name)); return -1; } + + if (timestamp_value && + virFileSetXAttr(dst, timestamp_name, timestamp_value) < 0) { + ignore_value(virFileRemoveXAttr(dst, ref_name)); + ignore_value(virFileRemoveXAttr(dst, attr_name)); + return -1; + } } return 0; diff --git a/tests/qemusecuritymock.c b/tests/qemusecuritymock.c index a15eef29c9..373d64305a 100644 --- a/tests/qemusecuritymock.c +++ b/tests/qemusecuritymock.c @@ -32,6 +32,7 @@ #include "viralloc.h" #include "qemusecuritytest.h" #include "security/security_manager.h" +#include "virhostuptime.h" #define VIR_FROM_THIS VIR_FROM_NONE @@ -488,3 +489,14 @@ virProcessRunInFork(virProcessForkCallback cb, { return cb(-1, opaque); } + + +/* We don't really need to mock this function. The qemusecuritytest doesn't + * care about the actual value. However, travis runs build and tests in a + * container where utmp is missing and thus this function fails. */ +int +virHostGetBootTime(unsigned long long *when) +{ + *when = 1234567890; + return 0; +} diff --git a/tools/libvirt_recover_xattrs.sh b/tools/libvirt_recover_xattrs.sh index 58f02f8dfb..3907413c63 100755 --- a/tools/libvirt_recover_xattrs.sh +++ b/tools/libvirt_recover_xattrs.sh @@ -74,7 +74,7 @@ fi declare -a XATTRS for i in "dac" "selinux"; do for p in ${LIBVIRT_XATTR_PREFIXES[@]}; do - XATTRS+=("$p.$i" "$p.ref_$i") + XATTRS+=("$p.$i" "$p.ref_$i" "$p.timestamp_$i") done done -- 2.21.0

On Thu, Aug 22, 2019 at 16:56:02 +0200, Michal Privoznik wrote:
It may happen that we leave some XATTRs behind. For instance, on a sudden power loss, the host just shuts down without calling restore on domain paths. This creates a problem, because when the host starts up again, the XATTRs are there but they don't reflect the true state and this may result in libvirt denying start of a domain.
To solve this, save a unique timestamp (host boot time) among with our XATTRs.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1741140
Signed-off-by: Michal Privoznik <mprivozn@redhat.com> ---
diff to v2: - Jirka's suggestions worked in.
src/security/security_util.c | 194 +++++++++++++++++++++++++++++++- tests/qemusecuritymock.c | 12 ++ tools/libvirt_recover_xattrs.sh | 2 +- 3 files changed, 206 insertions(+), 2 deletions(-)
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
participants (2)
-
Jiri Denemark
-
Michal Privoznik