[PATCH v3 0/7] Add TLS-PSK support for migration
QEMU provides the capability to encrypt the migration data stream using two transport layer security (TLS) authentication schemes: X.509 certificates and pre-shared keys (PSK). Currently, Libvirt only supports the X.509-based TLS authentication scheme. In TLS X.509 certificates, a set of live migrations utilize a fixed set of static certificates for encrypted migration. In this authentication scheme, users require to deploy a certificate authority and monitor the certificate expiration window. In case certificates are compromised all the future live migrations are vulnerable. To resolve this, this patch series introduce the support for pre-shared key-based authentication scheme. This mechanism relies on symmetric pre-shared keys (a secret key that is known to both sender and receiver prior to secure communication) for providing secure transfer of data. Libvirt solely manages the lifecycle of the ephemeral pre-shared keys, including, generation, persistent storage, and cleanup. Libvirt generates the key on the source machine, then transfers it to the destination machine using the migration cookie. To allow users to configure the size of the key, Libvirt provides the migrate_tls_psk_length configuration parameter in qemu.conf. To avoid introduction of an additional VIR_MIGRATE_* flag, we rely on existing the VIR_MIGRATE_TLS flag. If the VIR_MIGRATE_TLS flag is set but the necessary X.509 credential files are missing on the destination, then we fallback to using PSK-based authentication scheme during migration. v3: 1. Destination host decides which TLS authentication scheme to use. 2. The directory of the key file is set to <tls_psk_state_dir>/$ID-$VMNAME. 3. Use the same alias for both tls-creds-x509 and tls-creds-psk objects. 4. Validate the length of the pre-shared key. 5. Unit test to validate the pre-shared key in the migration cookie. v2: 1. Libvirt manages the lifecycle of pre-shared keys. 2. Transfer of keys to the destination via the migration cookie 3. Remove the VIR_MIGRATE_TLS_PSK flag instead rely on VIR_MIGRATE_TLS and availability of ca-cert.pem on source. 4. Drop VIR_MIGRATE_PARAM_TLS_PSK_DIRECTORY, Libvirt solely manages the pre-shared keys. Abhisek Panda (7): conf: Add a configuration param for TLS-PSK qemu: Manage a pre-shared key's lifecycle qemu: Add support to build the tls-creds-psk object qemu: rename tls-creds-x509 obj related functions qemu: Manage tls-creds-psk object lifecycle qemu: Set up the migrate TLS-PSK objects tests: Add testing of pre-shared key lifecycle include/libvirt/libvirt-domain.h | 11 +- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 8 ++ src/qemu/qemu_command.c | 26 ++++ src/qemu/qemu_command.h | 7 + src/qemu/qemu_conf.c | 22 ++++ src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 6 + src/qemu/qemu_hotplug.c | 40 +++--- src/qemu/qemu_hotplug.h | 24 ++-- src/qemu/qemu_migration.c | 202 +++++++++++++++++++++++++---- src/qemu/qemu_migration_cookie.c | 94 +++++++++++++- src/qemu/qemu_migration_cookie.h | 5 + src/qemu/qemu_migration_params.c | 98 +++++++++++--- src/qemu/qemu_migration_params.h | 22 +++- src/qemu/test_libvirtd_qemu.aug.in | 1 + tests/qemumigrationcookiexmltest.c | 141 +++++++++++++++++++- tests/testutilsqemu.c | 12 ++ 20 files changed, 631 insertions(+), 93 deletions(-) -- 2.43.7
For encrypted migration of VMs, QEMU provides the TLS-PSK authentication apart from TLS certificates. This mechanism relies on pre-shared keys (a secret key that is known to both sender and receiver prior to secure communication) for providing secure transfer of data. Libvirt handles the lifecycle of pre-shared keys, managing their generation, persistent storage, and cleanup. Add the "migrate_tls_psk_length" configuration attribute to qemu.conf to allow users to define the size of the pre-shared key. Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 8 ++++++++ src/qemu/qemu_conf.c | 18 ++++++++++++++++++ src/qemu/qemu_conf.h | 1 + src/qemu/test_libvirtd_qemu.aug.in | 1 + 5 files changed, 29 insertions(+) diff --git a/src/qemu/libvirtd_qemu.aug b/src/qemu/libvirtd_qemu.aug index 311992e441..d58f995282 100644 --- a/src/qemu/libvirtd_qemu.aug +++ b/src/qemu/libvirtd_qemu.aug @@ -68,6 +68,7 @@ module Libvirtd_qemu = | str_entry "migrate_tls_x509_secret_uuid" | str_entry "migrate_tls_priority" | bool_entry "migrate_tls_force" + | int_entry "migrate_tls_psk_length" let backup_entry = str_entry "backup_tls_x509_cert_dir" | bool_entry "backup_tls_x509_verify" diff --git a/src/qemu/qemu.conf.in b/src/qemu/qemu.conf.in index 97b0141cf6..7f36bd1a68 100644 --- a/src/qemu/qemu.conf.in +++ b/src/qemu/qemu.conf.in @@ -437,6 +437,14 @@ #migrate_tls_force = 0 +# The TLS-PSK authentication relies on pre-shared keys for providing secure transfer of data. +# When TLS-PSK is enabled for the migration operation, Libvirt manages the lifecycle of the +# pre-shared key files. For the key generation process, users can specify the pre-shared +# key size in bytes. The default value is set to 32 bytes. +# +#migrate_tls_psk_length = 32 + + # In order to override the default TLS certificate location for backup NBD # server certificates, supply a valid path to the certificate directory. If the # provided path does not exist, libvirtd will fail to start. If the path is diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index e30b146634..986f01ddcd 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -77,6 +77,9 @@ VIR_LOG_INIT("qemu.qemu_conf"); #define QEMU_BACKUP_PORT_MIN 10809 #define QEMU_BACKUP_PORT_MAX 10872 +#define QEMU_MIGRATE_TLS_PSK_LENGTH 32 +#define QEMU_MIGRATE_TLS_PSK_LENGTH_MAX 512 + VIR_ENUM_IMPL(virQEMUSchedCore, QEMU_SCHED_CORE_LAST, "none", @@ -616,6 +619,17 @@ virQEMUDriverConfigLoadSpecificTLSEntry(virQEMUDriverConfig *cfg, #undef GET_CONFIG_TLS_CERTINFO_COMMON #undef GET_CONFIG_TLS_CERTINFO_SERVER + + if (virConfGetValueUInt(conf, "migrate_tls_psk_length", &cfg->migrateTLSPSKLength) < 0) + return -1; + + if (cfg->migrateTLSPSKLength > QEMU_MIGRATE_TLS_PSK_LENGTH_MAX) { + virReportError(VIR_ERR_CONF_SYNTAX, + _("migrate_tls_psk_length must not be greater than %1$d"), + QEMU_MIGRATE_TLS_PSK_LENGTH_MAX); + return -1; + } + return 0; } @@ -1594,6 +1608,10 @@ virQEMUDriverConfigSetDefaults(virQEMUDriverConfig *cfg) #undef SET_TLS_VERIFY_DEFAULT + if (cfg->migrateTLSPSKLength == 0) { + cfg->migrateTLSPSKLength = QEMU_MIGRATE_TLS_PSK_LENGTH; + } + return 0; } diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 1d29f35c5d..c18aedf59c 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -170,6 +170,7 @@ struct _virQEMUDriverConfig { char *migrateTLSx509secretUUID; char *migrateTLSpriority; bool migrateTLSForce; + unsigned int migrateTLSPSKLength; char *backupTLSx509certdir; bool backupTLSx509verify; diff --git a/src/qemu/test_libvirtd_qemu.aug.in b/src/qemu/test_libvirtd_qemu.aug.in index c4cf9cf634..7e337825fd 100644 --- a/src/qemu/test_libvirtd_qemu.aug.in +++ b/src/qemu/test_libvirtd_qemu.aug.in @@ -45,6 +45,7 @@ module Test_libvirtd_qemu = { "migrate_tls_x509_secret_uuid" = "00000000-0000-0000-0000-000000000000" } { "migrate_tls_priority" = "@SYSTEM" } { "migrate_tls_force" = "0" } +{ "migrate_tls_psk_length" = "32" } { "backup_tls_x509_cert_dir" = "/etc/pki/libvirt-backup" } { "backup_tls_x509_verify" = "1" } { "backup_tls_x509_secret_uuid" = "00000000-0000-0000-0000-000000000000" } -- 2.43.7
On Wed, Jul 29, 2026 at 08:58:53 +0000, Abhisek Panda wrote:
For encrypted migration of VMs, QEMU provides the TLS-PSK authentication apart from TLS certificates. This mechanism relies on pre-shared keys (a secret key that is known to both sender and receiver prior to secure communication) for providing secure transfer of data. Libvirt handles the lifecycle of pre-shared keys, managing their generation, persistent storage, and cleanup.
Add the "migrate_tls_psk_length" configuration attribute to qemu.conf to allow users to define the size of the pre-shared key.
Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 8 ++++++++ src/qemu/qemu_conf.c | 18 ++++++++++++++++++ src/qemu/qemu_conf.h | 1 + src/qemu/test_libvirtd_qemu.aug.in | 1 + 5 files changed, 29 insertions(+)
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Libvirt falls back to the TLS-PSK-enabled VM migration, if the VIR_MIGRATE_TLS flag is set but the destination host lacks necessary X.509 credentials (ca-cert.pem, server-cert.pem and server-key.pem). The source host unconditionally adds the pre-shared key in the migration cookie if the VIR_MIGRATE_TLS flag is set. Upon parsing the migration cookie, the destination host checks for the presence of X.509 credentials and informs the source host whether to use TLS X.509 or TLS PSK during VM migration via the migration cookie. For a migration session, Libvirt generates a random key of the specified length, and then stores the content, "qemu:<random key>", at <tls_psk_state_dir>/$ID-$VMNAME/keys.psk on the source host. This is because QEMU's tls-creds-psk object does not accept a raw key string as a parameter, it only accepts a dir argument pointing to a directory from which it can read the key file. Subsequently, it sends the key to destination by embedding it within the migration cookie. The destination's Libvirt extracts the key from the migration cookie. Upon migration completion or any failure, both source and destination Libvirt must delete the directory containing the session's keys.psk. Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/qemu_conf.c | 4 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 6 ++ src/qemu/qemu_migration.c | 119 +++++++++++++++++++++++++++++ src/qemu/qemu_migration_cookie.c | 94 ++++++++++++++++++++++- src/qemu/qemu_migration_cookie.h | 5 ++ tests/qemumigrationcookiexmltest.c | 18 +++-- 9 files changed, 241 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index 986f01ddcd..86571e8da6 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -168,6 +168,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privileged, cfg->cacheDir = g_strdup_printf("%s/cache/qemu", root); cfg->libDir = g_strdup_printf("%s/lib/qemu", root); cfg->swtpmStorageDir = g_strdup_printf("%s/lib/swtpm", root); + cfg->tlsPSKStateDir = g_strdup_printf("%s/run/psk", root); cfg->saveDir = g_strdup_printf("%s/save", cfg->libDir); cfg->snapshotDir = g_strdup_printf("%s/snapshot", cfg->libDir); @@ -187,6 +188,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privileged, cfg->stateDir = g_strdup_printf("%s/libvirt/qemu", RUNSTATEDIR); cfg->swtpmStateDir = g_strdup_printf("%s/swtpm", cfg->stateDir); cfg->channelTargetDir = g_strdup_printf("%s/channel", cfg->stateDir); + cfg->tlsPSKStateDir = g_strdup_printf("%s/psk", cfg->stateDir); cfg->cacheDir = g_strdup_printf("%s/cache/libvirt/qemu", LOCALSTATEDIR); @@ -214,6 +216,7 @@ virQEMUDriverConfig *virQEMUDriverConfigNew(bool privileged, cfg->stateDir = g_strdup_printf("%s/qemu/run", rundir); cfg->swtpmStateDir = g_strdup_printf("%s/swtpm", cfg->stateDir); cfg->channelTargetDir = g_strdup_printf("%s/channel", cfg->stateDir); + cfg->tlsPSKStateDir = g_strdup_printf("%s/psk", cfg->stateDir); cfg->configBaseDir = virGetUserConfigDirectory(); @@ -375,6 +378,7 @@ static void virQEMUDriverConfigDispose(void *obj) g_free(cfg->dbusStateDir); g_free(cfg->rdpStateDir); g_free(cfg->vncStateDir); + g_free(cfg->tlsPSKStateDir); g_free(cfg->libDir); g_free(cfg->cacheDir); diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index c18aedf59c..06a0b0770c 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -113,6 +113,7 @@ struct _virQEMUDriverConfig { char *dbusStateDir; char *rdpStateDir; char *vncStateDir; + char *tlsPSKStateDir; /* These two directories are ones QEMU processes use (so must match * the QEMU user/group */ char *libDir; diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index e1b805d906..41421214df 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1992,6 +1992,7 @@ qemuDomainObjPrivateFree(void *data) virObjectUnref(priv->monConfig); g_free(priv->lockState); g_free(priv->origname); + g_free(priv->migTLSPSK); virChrdevFree(priv->devs); diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 50ab492023..1feeb11bcd 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -148,6 +148,7 @@ struct _qemuDomainObjPrivate { char *origname; int nbdPort; /* Port used for migration with NBD */ unsigned short migrationPort; + char *migTLSPSK; /* Hex-encoded pre-shared key for TLS-PSK-enabled VM migration session */ unsigned short backupNBDPort; int preMigrationState; unsigned long long preMigrationMemlock; /* Original RLIMIT_MEMLOCK in case diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index bdc0cff66a..84b017b342 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -664,6 +664,12 @@ qemuStateInitialize(bool privileged, cfg->vncStateDir); goto error; } + if (virDirCreate(cfg->tlsPSKStateDir, 0700, cfg->user, cfg->group, + VIR_DIR_CREATE_ALLOW_EXIST) < 0) { + virReportSystemError(errno, _("Failed to create TLS PSK state dir %1$s"), + cfg->tlsPSKStateDir); + goto error; + } qemu_driver->inhibitor = virInhibitorNew( VIR_INHIBITOR_WHAT_SHUTDOWN, diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 4a43ab83b0..3d6e472443 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -59,6 +59,7 @@ #include "virprocess.h" #include "virdomainsnapshotobjlist.h" #include "virutil.h" +#include "virsecureerase.h" #define VIR_FROM_THIS VIR_FROM_QEMU @@ -1503,6 +1504,91 @@ qemuMigrationSrcIsAllowedHostdev(const virDomainDef *def) } +static void +qemuMigrationDeletePSKDir(virQEMUDriver *driver, virDomainObj *vm) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + qemuDomainObjPrivate *priv = vm->privateData; + g_autofree char *dir_path = NULL; + g_autofree char *shortName = NULL; + + if (priv->migTLSPSK) { + virSecureEraseString(priv->migTLSPSK); + g_clear_pointer(&priv->migTLSPSK, g_free); + } + + if (!vm || !vm->def || !(shortName = virDomainDefGetShortName(vm->def))) + return; + + dir_path = g_strdup_printf("%s/%s", cfg->tlsPSKStateDir, shortName); + + if (virFileIsDir(dir_path) && + virFileDeleteTree(dir_path) < 0) + VIR_WARN("Failed to delete the directory %s containing the pre-shared keys for migration of domain %s", + dir_path, vm->def->name); +} + + +static int +qemuPersistTLSPSKHelper(int pskFD, + const char *pskPath, + const void *opaque) +{ + const char *key = opaque; + + if (safewrite(pskFD, "qemu:", 5) < 0 || + safewrite(pskFD, key, strlen(key)) < 0) { + virReportSystemError(errno, + _("Unable to write the pre-shared key to file '%1$s'"), + pskPath); + return -1; + } + + return 0; +} + + +static int +qemuMigrationPersistPSK(virQEMUDriver *driver, virDomainObj *vm, const char *tlsPSK) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autofree char *dir_path = NULL; + g_autofree char *key_path = NULL; + g_autofree char *shortName = NULL; + + if (!vm || !vm->def || !(shortName = virDomainDefGetShortName(vm->def))) + return -1; + + dir_path = g_strdup_printf("%s/%s", cfg->tlsPSKStateDir, shortName); + key_path = g_strdup_printf("%s/keys.psk", dir_path); + + if (virDirCreate(dir_path, 0700, cfg->user, cfg->group, + VIR_DIR_CREATE_ALLOW_EXIST) < 0) { + virReportSystemError(errno, + _("Could not create the directory %1$s for storing PSKs"), + dir_path); + goto error; + } + + if (tlsPSK) { + if (virFileRewrite(key_path, S_IRUSR, cfg->user, + cfg->group, qemuPersistTLSPSKHelper, + tlsPSK) < 0) + goto error; + } else { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("The pre-shared key for TLS-PSK migration is not provided")); + goto error; + } + + return 0; + + error: + qemuMigrationDeletePSKDir(driver, vm); + return -1; +} + + static int qemuDomainGetMigrationBlockers(virDomainObj *vm, int asyncJob, @@ -2718,6 +2804,7 @@ qemuMigrationSrcBeginXML(virDomainObj *vm, qemuDomainObjPrivate *priv = vm->privateData; virQEMUDriver *driver = priv->driver; g_autoptr(qemuMigrationCookie) mig = NULL; + int ret; if (priv->origCPU) cookieFlags |= QEMU_MIGRATION_COOKIE_CPU; @@ -2725,6 +2812,9 @@ qemuMigrationSrcBeginXML(virDomainObj *vm, if (!(flags & VIR_MIGRATE_OFFLINE)) cookieFlags |= QEMU_MIGRATION_COOKIE_CAPS; + if (flags & VIR_MIGRATE_TLS) + cookieFlags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + if (!(mig = qemuMigrationCookieNew(vm->def, priv->origname))) return NULL; @@ -2738,6 +2828,15 @@ qemuMigrationSrcBeginXML(virDomainObj *vm, cookieFlags) < 0) return NULL; + if ((flags & VIR_MIGRATE_TLS) && mig->tlsPSK) { + ret = qemuMigrationPersistPSK(driver, vm, mig->tlsPSK); + if (ret < 0) { + virSecureEraseString(mig->tlsPSK); + g_clear_pointer(&mig->tlsPSK, g_free); + return NULL; + } + } + if (xmlin) { g_autoptr(virDomainDef) def = NULL; @@ -4232,6 +4331,9 @@ qemuMigrationSrcConfirmPhase(virQEMUDriver *driver, privJob->stats.mig.downtime = privMigJob->stats.mig.downtime; } + if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm); + if (flags & VIR_MIGRATE_OFFLINE) return 0; @@ -5275,6 +5377,9 @@ qemuMigrationSrcRun(virQEMUDriver *driver, error: virErrorPreserveLast(&orig_err); + if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm); + if (qemuDomainObjIsActive(vm)) { int reason; virDomainState state = virDomainObjGetState(vm, &reason); @@ -7029,6 +7134,9 @@ qemuMigrationDstFinishActive(virQEMUDriver *driver, QEMU_MIGRATION_COOKIE_STATS) < 0) VIR_WARN("Unable to encode migration cookie"); + if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm); + qemuMigrationDstComplete(driver, vm, inPostCopy, VIR_ASYNC_JOB_MIGRATION_IN, vm->job); @@ -7039,6 +7147,9 @@ qemuMigrationDstFinishActive(virQEMUDriver *driver, * overwrites it. */ virErrorPreserveLast(&orig_err); + if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm); + if (qemuDomainObjIsActive(vm)) { if (doKill) { qemuProcessStop(vm, VIR_DOMAIN_SHUTOFF_FAILED, @@ -7197,6 +7308,14 @@ qemuMigrationProcessUnattended(virQEMUDriver *driver, else qemuMigrationSrcComplete(driver, vm, job); + /* + * Attempt to clean up the directory containing the pre-shared keys + * for the domain. Since, we cannot determine if the migration has + * enabled the VIR_MIGRATE_TLS flag with pre-shared keys, we clean up + * the directory unconditionally. + */ + qemuMigrationDeletePSKDir(driver, vm); + qemuMigrationJobFinish(vm); if (!virDomainObjIsActive(vm)) diff --git a/src/qemu/qemu_migration_cookie.c b/src/qemu/qemu_migration_cookie.c index 7311a8294b..1a7b9361da 100644 --- a/src/qemu/qemu_migration_cookie.c +++ b/src/qemu/qemu_migration_cookie.c @@ -20,13 +20,16 @@ #include <gnutls/gnutls.h> #include <gnutls/x509.h> +#include <inttypes.h> #include "locking/domain_lock.h" #include "virerror.h" +#include "virfile.h" #include "virlog.h" #include "virnetdevopenvswitch.h" #include "virstring.h" #include "virutil.h" +#include "virsecureerase.h" #include "qemu_domain.h" #include "qemu_migration_cookie.h" @@ -52,6 +55,7 @@ VIR_ENUM_IMPL(qemuMigrationCookieFlag, "allowReboot", "capabilities", "block-dirty-bitmaps", + "psk", ); @@ -149,6 +153,17 @@ G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuMigrationBlockDirtyBitmapsDisk, qemuMigrationBlockDirtyBitmapsDiskFree); +static bool +qemuMigrationServerCertsExists(virQEMUDriver *driver) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autofree char *cacert_path = g_strdup_printf("%s/ca-cert.pem", cfg->migrateTLSx509certdir); + g_autofree char *key_path = g_strdup_printf("%s/server-key.pem", cfg->migrateTLSx509certdir); + g_autofree char *cert_path = g_strdup_printf("%s/server-cert.pem", cfg->migrateTLSx509certdir); + return virFileExists(cacert_path) && virFileExists(key_path) && virFileExists(cert_path); +} + + void qemuMigrationCookieFree(qemuMigrationCookie *mig) { @@ -165,6 +180,9 @@ qemuMigrationCookieFree(qemuMigrationCookie *mig) g_free(mig->name); g_free(mig->lockState); g_free(mig->lockDriver); + if (mig->tlsPSK) + virSecureEraseString(mig->tlsPSK); + g_free(mig->tlsPSK); g_clear_pointer(&mig->jobData, virDomainJobDataFree); virCPUDefFree(mig->cpu); qemuMigrationCookieCapsFree(mig->caps); @@ -575,6 +593,51 @@ qemuMigrationCookieAddCaps(qemuMigrationCookie *mig, } +static int +qemuMigrationCookieAddTLSPSK(qemuMigrationCookie *mig, + virQEMUDriver *driver, + virDomainObj *vm) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + qemuDomainObjPrivate *priv = vm->privateData; + gnutls_datum_t psk_key = {NULL, 0}; + g_autofree char *key = NULL; + size_t key_len; + int ret; + + /* Generate the pre-shared key exactly once for a migration session*/ + if (priv->migTLSPSK) { + mig->tlsPSK = g_strdup(priv->migTLSPSK); + mig->flags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + return 0; + } + + ret = gnutls_key_generate(&psk_key, cfg->migrateTLSPSKLength); + if (ret < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Generation of a pre-shared key failed")); + return -1; + } + key_len = (psk_key.size*2) + 1; + key = g_new0(char, key_len); + + ret = gnutls_hex_encode(&psk_key, key, &key_len); + if (ret < 0) { + gnutls_free(psk_key.data); + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Hex encoding of a PSK key failed")); + return -1; + } + + priv->migTLSPSK = g_strdup(key); + mig->tlsPSK = g_steal_pointer(&key); + mig->flags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + + gnutls_free(psk_key.data); + return 0; +} + + static void qemuMigrationCookieGraphicsXMLFormat(virBuffer *buf, qemuMigrationCookieGraphics *grap) @@ -890,6 +953,9 @@ qemuMigrationCookieXMLFormat(virQEMUDriver *driver, if (mig->flags & QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS) qemuMigrationCookieBlockDirtyBitmapsFormat(buf, mig->blockDirtyBitmaps); + if ((mig->flags & QEMU_MIGRATION_COOKIE_TLS_PSK) && mig->tlsPSK) + virBufferAsprintf(buf, "<migration-key>%s</migration-key>\n", mig->tlsPSK); + virBufferAdjustIndent(buf, -2); virBufferAddLit(buf, "</qemu-migration>\n"); return 0; @@ -1396,6 +1462,12 @@ qemuMigrationCookieXMLParse(qemuMigrationCookie *mig, qemuMigrationCookieBlockDirtyBitmapsParse(ctxt, mig) < 0) return -1; + if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK) { + mig->tlsPSK = virXPathString("string(./migration-key[1])", ctxt); + if (mig->tlsPSK) + mig->flags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + } + return 0; } @@ -1471,14 +1543,17 @@ qemuMigrationCookieFormat(qemuMigrationCookie *mig, qemuMigrationCookieAddCaps(mig, dom, party) < 0) return -1; + if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK && + party == QEMU_MIGRATION_SOURCE && + qemuMigrationCookieAddTLSPSK(mig, driver, dom) < 0) + return -1; + if (qemuMigrationCookieXMLFormat(driver, priv->qemuCaps, &buf, mig) < 0) return -1; *cookieoutlen = virBufferUse(&buf) + 1; *cookieout = virBufferContentAndReset(&buf); - VIR_DEBUG("cookielen=%d cookie=%s", *cookieoutlen, *cookieout); - return 0; } @@ -1494,6 +1569,7 @@ qemuMigrationCookieParse(virQEMUDriver *driver, unsigned int flags) { g_autoptr(qemuMigrationCookie) mig = NULL; + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); /* Parse & validate incoming cookie (if any) */ if (cookiein && cookieinlen && @@ -1537,6 +1613,20 @@ qemuMigrationCookieParse(virQEMUDriver *driver, } } + if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK) { + if (!qemuMigrationServerCertsExists(driver)) { + if (!mig->tlsPSK) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("destination host has no X.509 certificates configured for migration and source host did not provide a pre-shared key")); + return NULL; + } + } else { + virSecureEraseString(mig->tlsPSK); + mig->flags &= ~QEMU_MIGRATION_COOKIE_TLS_PSK; + g_clear_pointer(&mig->tlsPSK, g_free); + } + } + if (vm && flags & QEMU_MIGRATION_COOKIE_STATS && mig->jobData && vm->job->current) mig->jobData->operation = vm->job->current->operation; diff --git a/src/qemu/qemu_migration_cookie.h b/src/qemu/qemu_migration_cookie.h index 254372234d..fd3b4c5a56 100644 --- a/src/qemu/qemu_migration_cookie.h +++ b/src/qemu/qemu_migration_cookie.h @@ -35,6 +35,7 @@ typedef enum { QEMU_MIGRATION_COOKIE_FLAG_ALLOW_REBOOT, QEMU_MIGRATION_COOKIE_FLAG_CAPS, QEMU_MIGRATION_COOKIE_FLAG_BLOCK_DIRTY_BITMAPS, + QEMU_MIGRATION_COOKIE_FLAG_TLS_PSK, QEMU_MIGRATION_COOKIE_FLAG_LAST } qemuMigrationCookieFlags; @@ -53,6 +54,7 @@ typedef enum { QEMU_MIGRATION_COOKIE_CPU = (1 << QEMU_MIGRATION_COOKIE_FLAG_CPU), QEMU_MIGRATION_COOKIE_CAPS = (1 << QEMU_MIGRATION_COOKIE_FLAG_CAPS), QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS = (1 << QEMU_MIGRATION_COOKIE_FLAG_BLOCK_DIRTY_BITMAPS), + QEMU_MIGRATION_COOKIE_TLS_PSK = (1 << QEMU_MIGRATION_COOKIE_FLAG_TLS_PSK), } qemuMigrationCookieFeatures; typedef struct _qemuMigrationCookieGraphics qemuMigrationCookieGraphics; @@ -171,6 +173,9 @@ struct _qemuMigrationCookie { /* If flags & QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS */ GSList *blockDirtyBitmaps; + + /* If flags & QEMU_MIGRATION_COOKIE_TLS_PSK */ + char *tlsPSK; }; diff --git a/tests/qemumigrationcookiexmltest.c b/tests/qemumigrationcookiexmltest.c index bc0f68b8c5..adf2eb4d08 100644 --- a/tests/qemumigrationcookiexmltest.c +++ b/tests/qemumigrationcookiexmltest.c @@ -160,8 +160,12 @@ testQemuMigrationCookieParse(const void *opaque) return -1; } - /* set all flags so that formatter attempts to format everything */ - data->cookie->flags = ~0; + /* Set all flags except QEMU_MIGRATION_COOKIE_TLS_PSK so that formatter + * attempts to format everything except the migration-key element. This is + * because the value of the migration-key element is randomly generated every time + * the migration cookie is constructed. + */ + data->cookie->flags = ~QEMU_MIGRATION_COOKIE_TLS_PSK; if (qemuMigrationCookieXMLFormat(&driver, priv->qemuCaps, @@ -225,15 +229,17 @@ testQemuMigrationCookieDom2XML(const char *namesuffix, * - lockstate: internals are NULL in tests, causes crash * - nbd: monitor not present * - dirty bitmaps: monitor not present + * - tls-psk: a new key is generated every time the migration cookie is constructed, so we can't test it */ unsigned int cookiePopulateFlagMask = QEMU_MIGRATION_COOKIE_LOCKSTATE | QEMU_MIGRATION_COOKIE_NBD | - QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS; + QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS | + QEMU_MIGRATION_COOKIE_TLS_PSK; data->cookiePopulateFlags = ~cookiePopulateFlagMask; } if (cookieParseFlags == 0) - data->cookieParseFlags = ~0; + data->cookieParseFlags = ~QEMU_MIGRATION_COOKIE_TLS_PSK; data->inStatus = g_strconcat(abs_srcdir, "/", domxml, NULL); @@ -279,7 +285,7 @@ testQemuMigrationCookieXML2XML(const char *name, int ret = 0; if (cookieParseFlags == 0) - data->cookieParseFlags = ~0; + data->cookieParseFlags = ~QEMU_MIGRATION_COOKIE_TLS_PSK; data->inStatus = g_strconcat(abs_srcdir, "/", statusxml, NULL); data->infile = g_strconcat(abs_srcdir, "/qemumigrationcookiexmldata/", @@ -381,7 +387,7 @@ testQemuMigrationCookieXML2XMLBitmaps(const char *name, int ret = 0; if (cookieParseFlags == 0) - data->cookieParseFlags = ~0; + data->cookieParseFlags = ~QEMU_MIGRATION_COOKIE_TLS_PSK; data->inStatus = g_strconcat(abs_srcdir, "/", statusxml, NULL); data->infile = g_strconcat(abs_srcdir, "/qemumigrationcookiexmldata/", -- 2.43.7
On Wed, Jul 29, 2026 at 08:58:54 +0000, Abhisek Panda wrote:
Libvirt falls back to the TLS-PSK-enabled VM migration, if the VIR_MIGRATE_TLS flag is set but the destination host lacks necessary X.509 credentials (ca-cert.pem, server-cert.pem and server-key.pem). The source host unconditionally adds the pre-shared key in the migration cookie if the VIR_MIGRATE_TLS flag is set. Upon parsing the migration cookie, the destination host checks for the presence of X.509 credentials and informs the source host whether to use TLS X.509 or TLS PSK during VM migration via the migration cookie.
For a migration session, Libvirt generates a random key of the specified length, and then stores the content, "qemu:<random key>", at <tls_psk_state_dir>/$ID-$VMNAME/keys.psk on the source host. This is because QEMU's tls-creds-psk object does not accept a raw key string as a parameter, it only accepts a dir argument pointing to a directory from which it can read the key file. Subsequently, it sends the key to destination by embedding it within the migration cookie. The destination's Libvirt extracts the key from the migration cookie. Upon migration completion or any failure, both source and destination Libvirt must delete the directory containing the session's keys.psk.
Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/qemu_conf.c | 4 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 6 ++ src/qemu/qemu_migration.c | 119 +++++++++++++++++++++++++++++ src/qemu/qemu_migration_cookie.c | 94 ++++++++++++++++++++++- src/qemu/qemu_migration_cookie.h | 5 ++ tests/qemumigrationcookiexmltest.c | 18 +++-- 9 files changed, 241 insertions(+), 8 deletions(-)
[...]
diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 4a43ab83b0..3d6e472443 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c
[...]
+static int +qemuMigrationPersistPSK(virQEMUDriver *driver, virDomainObj *vm, const char *tlsPSK) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autofree char *dir_path = NULL; + g_autofree char *key_path = NULL; + g_autofree char *shortName = NULL; + + if (!vm || !vm->def || !(shortName = virDomainDefGetShortName(vm->def))) + return -1;
This error path mixes situations where a libvirt error is raised (below) and when no error is reported. We don't allow that because the caller can't then know if an error was reported, thus all code paths must report error or all must not report error. How can 'vm' or 'vm->def' even be NULL here? Does this check even make sense?
+ + dir_path = g_strdup_printf("%s/%s", cfg->tlsPSKStateDir, shortName); + key_path = g_strdup_printf("%s/keys.psk", dir_path); + + if (virDirCreate(dir_path, 0700, cfg->user, cfg->group, + VIR_DIR_CREATE_ALLOW_EXIST) < 0) { + virReportSystemError(errno, + _("Could not create the directory %1$s for storing PSKs"), + dir_path); + goto error; + } + /> + if (tlsPSK) { + if (virFileRewrite(key_path, S_IRUSR, cfg->user, + cfg->group, qemuPersistTLSPSKHelper, + tlsPSK) < 0) + goto error; + } else { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("The pre-shared key for TLS-PSK migration is not provided"));
Can this happen? Why would the caller then call this function? And why would you create the directory first before reporting this?
+ goto error; + } + + return 0; + + error: + qemuMigrationDeletePSKDir(driver, vm); + return -1; +} + + static int qemuDomainGetMigrationBlockers(virDomainObj *vm, int asyncJob, @@ -2718,6 +2804,7 @@ qemuMigrationSrcBeginXML(virDomainObj *vm, qemuDomainObjPrivate *priv = vm->privateData; virQEMUDriver *driver = priv->driver; g_autoptr(qemuMigrationCookie) mig = NULL; + int ret;
if (priv->origCPU) cookieFlags |= QEMU_MIGRATION_COOKIE_CPU; @@ -2725,6 +2812,9 @@ qemuMigrationSrcBeginXML(virDomainObj *vm, if (!(flags & VIR_MIGRATE_OFFLINE)) cookieFlags |= QEMU_MIGRATION_COOKIE_CAPS;
+ if (flags & VIR_MIGRATE_TLS) + cookieFlags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + if (!(mig = qemuMigrationCookieNew(vm->def, priv->origname))) return NULL;
@@ -2738,6 +2828,15 @@ qemuMigrationSrcBeginXML(virDomainObj *vm, cookieFlags) < 0) return NULL;
+ if ((flags & VIR_MIGRATE_TLS) && mig->tlsPSK) { + ret = qemuMigrationPersistPSK(driver, vm, mig->tlsPSK);
So, since this is the only caller and it's guaranteed that tlsPSK exists and 'vm' is non, null here, all the nonsense checks I pointed out above can be removed. Also the PSK should be written to the disk at any point when it will be used, which is decided before, so the flag check of VIR_MIGRATE_TLS doesn't make sense.
+ if (ret < 0) { + virSecureEraseString(mig->tlsPSK); + g_clear_pointer(&mig->tlsPSK, g_free); + return NULL;
This belongs to a common cleanup path after failed/completed migration, not this random location.
+ } + } + if (xmlin) { g_autoptr(virDomainDef) def = NULL;
@@ -4232,6 +4331,9 @@ qemuMigrationSrcConfirmPhase(virQEMUDriver *driver, privJob->stats.mig.downtime = privMigJob->stats.mig.downtime; }
+ if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm); + if (flags & VIR_MIGRATE_OFFLINE) return 0;
@@ -5275,6 +5377,9 @@ qemuMigrationSrcRun(virQEMUDriver *driver, error: virErrorPreserveLast(&orig_err);
+ if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm);
IMO this should't be gated by the flag check but rather by whether the PSK dir was set up and all of it inside qemuMigrationDeletePSKDir so that any furher change doesn't need to fix all callers.
+ if (qemuDomainObjIsActive(vm)) { int reason; virDomainState state = virDomainObjGetState(vm, &reason); @@ -7029,6 +7134,9 @@ qemuMigrationDstFinishActive(virQEMUDriver *driver, QEMU_MIGRATION_COOKIE_STATS) < 0) VIR_WARN("Unable to encode migration cookie");
+ if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm);
ditto
+ qemuMigrationDstComplete(driver, vm, inPostCopy, VIR_ASYNC_JOB_MIGRATION_IN, vm->job);
@@ -7039,6 +7147,9 @@ qemuMigrationDstFinishActive(virQEMUDriver *driver, * overwrites it. */ virErrorPreserveLast(&orig_err);
+ if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm);
same here
+ if (qemuDomainObjIsActive(vm)) { if (doKill) { qemuProcessStop(vm, VIR_DOMAIN_SHUTOFF_FAILED, @@ -7197,6 +7308,14 @@ qemuMigrationProcessUnattended(virQEMUDriver *driver, else qemuMigrationSrcComplete(driver, vm, job);
+ /* + * Attempt to clean up the directory containing the pre-shared keys + * for the domain. Since, we cannot determine if the migration has + * enabled the VIR_MIGRATE_TLS flag with pre-shared keys, we clean up + * the directory unconditionally. + */ + qemuMigrationDeletePSKDir(driver, vm);
And here you then don't need the comment.
+ qemuMigrationJobFinish(vm);
if (!virDomainObjIsActive(vm)) diff --git a/src/qemu/qemu_migration_cookie.c b/src/qemu/qemu_migration_cookie.c index 7311a8294b..1a7b9361da 100644 --- a/src/qemu/qemu_migration_cookie.c +++ b/src/qemu/qemu_migration_cookie.c
@@ -149,6 +153,17 @@ G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuMigrationBlockDirtyBitmapsDisk, qemuMigrationBlockDirtyBitmapsDiskFree);
+static bool +qemuMigrationServerCertsExists(virQEMUDriver *driver) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autofree char *cacert_path = g_strdup_printf("%s/ca-cert.pem", cfg->migrateTLSx509certdir); + g_autofree char *key_path = g_strdup_printf("%s/server-key.pem", cfg->migrateTLSx509certdir); + g_autofree char *cert_path = g_strdup_printf("%s/server-cert.pem", cfg->migrateTLSx509certdir); + return virFileExists(cacert_path) && virFileExists(key_path) && virFileExists(cert_path); +} + + void qemuMigrationCookieFree(qemuMigrationCookie *mig) { @@ -165,6 +180,9 @@ qemuMigrationCookieFree(qemuMigrationCookie *mig) g_free(mig->name); g_free(mig->lockState); g_free(mig->lockDriver); + if (mig->tlsPSK) + virSecureEraseString(mig->tlsPSK); + g_free(mig->tlsPSK);
So you do have a common cleanup path.
g_clear_pointer(&mig->jobData, virDomainJobDataFree); virCPUDefFree(mig->cpu); qemuMigrationCookieCapsFree(mig->caps); @@ -575,6 +593,51 @@ qemuMigrationCookieAddCaps(qemuMigrationCookie *mig, }
+static int +qemuMigrationCookieAddTLSPSK(qemuMigrationCookie *mig, + virQEMUDriver *driver, + virDomainObj *vm) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + qemuDomainObjPrivate *priv = vm->privateData; + gnutls_datum_t psk_key = {NULL, 0}; + g_autofree char *key = NULL; + size_t key_len; + int ret; + + /* Generate the pre-shared key exactly once for a migration session*/ + if (priv->migTLSPSK) { + mig->tlsPSK = g_strdup(priv->migTLSPSK); + mig->flags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + return 0; + } + + ret = gnutls_key_generate(&psk_key, cfg->migrateTLSPSKLength); + if (ret < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Generation of a pre-shared key failed")); + return -1; + } + key_len = (psk_key.size*2) + 1; + key = g_new0(char, key_len); + + ret = gnutls_hex_encode(&psk_key, key, &key_len); + if (ret < 0) { + gnutls_free(psk_key.data); + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Hex encoding of a PSK key failed")); + return -1; + } + + priv->migTLSPSK = g_strdup(key); + mig->tlsPSK = g_steal_pointer(&key); + mig->flags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + + gnutls_free(psk_key.data); + return 0; +} + + static void qemuMigrationCookieGraphicsXMLFormat(virBuffer *buf, qemuMigrationCookieGraphics *grap) @@ -890,6 +953,9 @@ qemuMigrationCookieXMLFormat(virQEMUDriver *driver, if (mig->flags & QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS) qemuMigrationCookieBlockDirtyBitmapsFormat(buf, mig->blockDirtyBitmaps);
+ if ((mig->flags & QEMU_MIGRATION_COOKIE_TLS_PSK) && mig->tlsPSK) + virBufferAsprintf(buf, "<migration-key>%s</migration-key>\n", mig->tlsPSK); + virBufferAdjustIndent(buf, -2); virBufferAddLit(buf, "</qemu-migration>\n"); return 0; @@ -1396,6 +1462,12 @@ qemuMigrationCookieXMLParse(qemuMigrationCookie *mig, qemuMigrationCookieBlockDirtyBitmapsParse(ctxt, mig) < 0) return -1;
+ if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK) { + mig->tlsPSK = virXPathString("string(./migration-key[1])", ctxt); + if (mig->tlsPSK) + mig->flags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + } + return 0; }
@@ -1471,14 +1543,17 @@ qemuMigrationCookieFormat(qemuMigrationCookie *mig, qemuMigrationCookieAddCaps(mig, dom, party) < 0) return -1;
+ if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK && + party == QEMU_MIGRATION_SOURCE && + qemuMigrationCookieAddTLSPSK(mig, driver, dom) < 0) + return -1; + if (qemuMigrationCookieXMLFormat(driver, priv->qemuCaps, &buf, mig) < 0) return -1;
*cookieoutlen = virBufferUse(&buf) + 1; *cookieout = virBufferContentAndReset(&buf);
- VIR_DEBUG("cookielen=%d cookie=%s", *cookieoutlen, *cookieout); -
Why is this debug statement deleted?
return 0; }
@@ -1537,6 +1613,20 @@ qemuMigrationCookieParse(virQEMUDriver *driver, } }
+ if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK) { + if (!qemuMigrationServerCertsExists(driver)) { + if (!mig->tlsPSK) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("destination host has no X.509 certificates configured for migration and source host did not provide a pre-shared key")); + return NULL; + } + } else {
So, if the destination has x509 certs this signals that PSK is not supported? What if the source doesn't have them. IMO we want to use PSK if available.
+ virSecureEraseString(mig->tlsPSK); + mig->flags &= ~QEMU_MIGRATION_COOKIE_TLS_PSK; + g_clear_pointer(&mig->tlsPSK, g_free); + } + } + if (vm && flags & QEMU_MIGRATION_COOKIE_STATS && mig->jobData && vm->job->current) mig->jobData->operation = vm->job->current->operation;
On 20 Aug 2026, at 6:25 PM, Peter Krempa <pkrempa@redhat.com> wrote:
!-------------------------------------------------------------------| CAUTION: External Email
|-------------------------------------------------------------------!
On Wed, Jul 29, 2026 at 08:58:54 +0000, Abhisek Panda wrote:
Libvirt falls back to the TLS-PSK-enabled VM migration, if the VIR_MIGRATE_TLS flag is set but the destination host lacks necessary X.509 credentials (ca-cert.pem, server-cert.pem and server-key.pem). The source host unconditionally adds the pre-shared key in the migration cookie if the VIR_MIGRATE_TLS flag is set. Upon parsing the migration cookie, the destination host checks for the presence of X.509 credentials and informs the source host whether to use TLS X.509 or TLS PSK during VM migration via the migration cookie.
For a migration session, Libvirt generates a random key of the specified length, and then stores the content, "qemu:<random key>", at <tls_psk_state_dir>/$ID-$VMNAME/keys.psk on the source host. This is because QEMU's tls-creds-psk object does not accept a raw key string as a parameter, it only accepts a dir argument pointing to a directory from which it can read the key file. Subsequently, it sends the key to destination by embedding it within the migration cookie. The destination's Libvirt extracts the key from the migration cookie. Upon migration completion or any failure, both source and destination Libvirt must delete the directory containing the session's keys.psk.
Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/qemu_conf.c | 4 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 6 ++ src/qemu/qemu_migration.c | 119 +++++++++++++++++++++++++++++ src/qemu/qemu_migration_cookie.c | 94 ++++++++++++++++++++++- src/qemu/qemu_migration_cookie.h | 5 ++ tests/qemumigrationcookiexmltest.c | 18 +++-- 9 files changed, 241 insertions(+), 8 deletions(-)
[...]
diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 4a43ab83b0..3d6e472443 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c
[...]
+static int +qemuMigrationPersistPSK(virQEMUDriver *driver, virDomainObj *vm, const char *tlsPSK) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autofree char *dir_path = NULL; + g_autofree char *key_path = NULL; + g_autofree char *shortName = NULL; + + if (!vm || !vm->def || !(shortName = virDomainDefGetShortName(vm->def))) + return -1;
This error path mixes situations where a libvirt error is raised (below) and when no error is reported. We don't allow that because the caller can't then know if an error was reported, thus all code paths must report error or all must not report error.
How can 'vm' or 'vm->def' even be NULL here? Does this check even make sense?
Acknowledged. Since ‘vm’ or ‘vm->def’ are guaranteed to be non NULL, I will drop the unnecessary checks. If virDomainDefGetShortName returns NULL, I will explicitly report VIR_ERR_INTERNAL_ERROR to keep error reporting consistent.
+ + dir_path = g_strdup_printf("%s/%s", cfg->tlsPSKStateDir, shortName); + key_path = g_strdup_printf("%s/keys.psk", dir_path); + + if (virDirCreate(dir_path, 0700, cfg->user, cfg->group, + VIR_DIR_CREATE_ALLOW_EXIST) < 0) { + virReportSystemError(errno, + _("Could not create the directory %1$s for storing PSKs"), + dir_path); + goto error; + } + /> + if (tlsPSK) { + if (virFileRewrite(key_path, S_IRUSR, cfg->user, + cfg->group, qemuPersistTLSPSKHelper, + tlsPSK) < 0) + goto error; + } else { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("The pre-shared key for TLS-PSK migration is not provided"));
Can this happen? Why would the caller then call this function? And why would you create the directory first before reporting this?
The caller guarantees tlsPSK is present when invoking this, so I have removed the tlsPSK check entirely.
+ goto error; + } + + return 0; + + error: + qemuMigrationDeletePSKDir(driver, vm); + return -1; +} + + static int qemuDomainGetMigrationBlockers(virDomainObj *vm, int asyncJob, @@ -2718,6 +2804,7 @@ qemuMigrationSrcBeginXML(virDomainObj *vm, qemuDomainObjPrivate *priv = vm->privateData; virQEMUDriver *driver = priv->driver; g_autoptr(qemuMigrationCookie) mig = NULL; + int ret;
if (priv->origCPU) cookieFlags |= QEMU_MIGRATION_COOKIE_CPU; @@ -2725,6 +2812,9 @@ qemuMigrationSrcBeginXML(virDomainObj *vm, if (!(flags & VIR_MIGRATE_OFFLINE)) cookieFlags |= QEMU_MIGRATION_COOKIE_CAPS;
+ if (flags & VIR_MIGRATE_TLS) + cookieFlags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + if (!(mig = qemuMigrationCookieNew(vm->def, priv->origname))) return NULL;
@@ -2738,6 +2828,15 @@ qemuMigrationSrcBeginXML(virDomainObj *vm, cookieFlags) < 0) return NULL;
+ if ((flags & VIR_MIGRATE_TLS) && mig->tlsPSK) { + ret = qemuMigrationPersistPSK(driver, vm, mig->tlsPSK);
So, since this is the only caller and it's guaranteed that tlsPSK exists and 'vm' is non, null here, all the nonsense checks I pointed out above can be removed.
Acknowledged.
Also the PSK should be written to the disk at any point when it will be used, which is decided before, so the flag check of VIR_MIGRATE_TLS doesn't make sense.
Acknowledged. I will update this to check ‘if mig->tlsPSK’ directly instead of Re-checking the flag.
+ if (ret < 0) { + virSecureEraseString(mig->tlsPSK); + g_clear_pointer(&mig->tlsPSK, g_free); + return NULL;
This belongs to a common cleanup path after failed/completed migration, not this random location.
Acknowledged. The cookie cleanup path will handle freeing and erasing sensitive string.
+ } + } + if (xmlin) { g_autoptr(virDomainDef) def = NULL;
@@ -4232,6 +4331,9 @@ qemuMigrationSrcConfirmPhase(virQEMUDriver *driver, privJob->stats.mig.downtime = privMigJob->stats.mig.downtime; }
+ if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm); + if (flags & VIR_MIGRATE_OFFLINE) return 0;
@@ -5275,6 +5377,9 @@ qemuMigrationSrcRun(virQEMUDriver *driver, error: virErrorPreserveLast(&orig_err);
+ if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm);
IMO this should't be gated by the flag check but rather by whether the PSK dir was set up and all of it inside qemuMigrationDeletePSKDir so that any furher change doesn't need to fix all callers.
Acknowledged. qemuMigrationDeletePSKDir will check for the existence of the PSKDir. If it is present then we clean it up else we just return. Therefore, we do not need any explicit check on the flag.
+ if (qemuDomainObjIsActive(vm)) { int reason; virDomainState state = virDomainObjGetState(vm, &reason); @@ -7029,6 +7134,9 @@ qemuMigrationDstFinishActive(virQEMUDriver *driver, QEMU_MIGRATION_COOKIE_STATS) < 0) VIR_WARN("Unable to encode migration cookie");
+ if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm);
ditto
Acknowledged
+ qemuMigrationDstComplete(driver, vm, inPostCopy, VIR_ASYNC_JOB_MIGRATION_IN, vm->job);
@@ -7039,6 +7147,9 @@ qemuMigrationDstFinishActive(virQEMUDriver *driver, * overwrites it. */ virErrorPreserveLast(&orig_err);
+ if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm);
same here
Acknowledged
+ if (qemuDomainObjIsActive(vm)) { if (doKill) { qemuProcessStop(vm, VIR_DOMAIN_SHUTOFF_FAILED, @@ -7197,6 +7308,14 @@ qemuMigrationProcessUnattended(virQEMUDriver *driver, else qemuMigrationSrcComplete(driver, vm, job);
+ /* + * Attempt to clean up the directory containing the pre-shared keys + * for the domain. Since, we cannot determine if the migration has + * enabled the VIR_MIGRATE_TLS flag with pre-shared keys, we clean up + * the directory unconditionally. + */ + qemuMigrationDeletePSKDir(driver, vm);
And here you then don't need the comment.
Acknowledged
+ qemuMigrationJobFinish(vm);
if (!virDomainObjIsActive(vm)) diff --git a/src/qemu/qemu_migration_cookie.c b/src/qemu/qemu_migration_cookie.c index 7311a8294b..1a7b9361da 100644 --- a/src/qemu/qemu_migration_cookie.c +++ b/src/qemu/qemu_migration_cookie.c
@@ -149,6 +153,17 @@ G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuMigrationBlockDirtyBitmapsDisk, qemuMigrationBlockDirtyBitmapsDiskFree);
+static bool +qemuMigrationServerCertsExists(virQEMUDriver *driver) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autofree char *cacert_path = g_strdup_printf("%s/ca-cert.pem", cfg->migrateTLSx509certdir); + g_autofree char *key_path = g_strdup_printf("%s/server-key.pem", cfg->migrateTLSx509certdir); + g_autofree char *cert_path = g_strdup_printf("%s/server-cert.pem", cfg->migrateTLSx509certdir); + return virFileExists(cacert_path) && virFileExists(key_path) && virFileExists(cert_path); +} + + void qemuMigrationCookieFree(qemuMigrationCookie *mig) { @@ -165,6 +180,9 @@ qemuMigrationCookieFree(qemuMigrationCookie *mig) g_free(mig->name); g_free(mig->lockState); g_free(mig->lockDriver); + if (mig->tlsPSK) + virSecureEraseString(mig->tlsPSK); + g_free(mig->tlsPSK);
So you do have a common cleanup path.
g_clear_pointer(&mig->jobData, virDomainJobDataFree); virCPUDefFree(mig->cpu); qemuMigrationCookieCapsFree(mig->caps); @@ -575,6 +593,51 @@ qemuMigrationCookieAddCaps(qemuMigrationCookie *mig, }
+static int +qemuMigrationCookieAddTLSPSK(qemuMigrationCookie *mig, + virQEMUDriver *driver, + virDomainObj *vm) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + qemuDomainObjPrivate *priv = vm->privateData; + gnutls_datum_t psk_key = {NULL, 0}; + g_autofree char *key = NULL; + size_t key_len; + int ret; + + /* Generate the pre-shared key exactly once for a migration session*/ + if (priv->migTLSPSK) { + mig->tlsPSK = g_strdup(priv->migTLSPSK); + mig->flags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + return 0; + } + + ret = gnutls_key_generate(&psk_key, cfg->migrateTLSPSKLength); + if (ret < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Generation of a pre-shared key failed")); + return -1; + } + key_len = (psk_key.size*2) + 1; + key = g_new0(char, key_len); + + ret = gnutls_hex_encode(&psk_key, key, &key_len); + if (ret < 0) { + gnutls_free(psk_key.data); + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Hex encoding of a PSK key failed")); + return -1; + } + + priv->migTLSPSK = g_strdup(key); + mig->tlsPSK = g_steal_pointer(&key); + mig->flags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + + gnutls_free(psk_key.data); + return 0; +} + + static void qemuMigrationCookieGraphicsXMLFormat(virBuffer *buf, qemuMigrationCookieGraphics *grap) @@ -890,6 +953,9 @@ qemuMigrationCookieXMLFormat(virQEMUDriver *driver, if (mig->flags & QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS) qemuMigrationCookieBlockDirtyBitmapsFormat(buf, mig->blockDirtyBitmaps);
+ if ((mig->flags & QEMU_MIGRATION_COOKIE_TLS_PSK) && mig->tlsPSK) + virBufferAsprintf(buf, "<migration-key>%s</migration-key>\n", mig->tlsPSK); + virBufferAdjustIndent(buf, -2); virBufferAddLit(buf, "</qemu-migration>\n"); return 0; @@ -1396,6 +1462,12 @@ qemuMigrationCookieXMLParse(qemuMigrationCookie *mig, qemuMigrationCookieBlockDirtyBitmapsParse(ctxt, mig) < 0) return -1;
+ if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK) { + mig->tlsPSK = virXPathString("string(./migration-key[1])", ctxt); + if (mig->tlsPSK) + mig->flags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + } + return 0; }
@@ -1471,14 +1543,17 @@ qemuMigrationCookieFormat(qemuMigrationCookie *mig, qemuMigrationCookieAddCaps(mig, dom, party) < 0) return -1;
+ if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK && + party == QEMU_MIGRATION_SOURCE && + qemuMigrationCookieAddTLSPSK(mig, driver, dom) < 0) + return -1; + if (qemuMigrationCookieXMLFormat(driver, priv->qemuCaps, &buf, mig) < 0) return -1;
*cookieoutlen = virBufferUse(&buf) + 1; *cookieout = virBufferContentAndReset(&buf);
- VIR_DEBUG("cookielen=%d cookie=%s", *cookieoutlen, *cookieout); -
Why is this debug statement deleted?
My intention was to prevent the raw PSK key string embedded in XML from being written to debug logs.
return 0; }
@@ -1537,6 +1613,20 @@ qemuMigrationCookieParse(virQEMUDriver *driver, } }
+ if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK) { + if (!qemuMigrationServerCertsExists(driver)) { + if (!mig->tlsPSK) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("destination host has no X.509 certificates configured for migration and source host did not provide a pre-shared key")); + return NULL; + } + } else {
So, if the destination has x509 certs this signals that PSK is not supported? What if the source doesn't have them. IMO we want to use PSK if available.
I have the following understanding of the pre-shared key: 1. QEMU added support for tls-creds-psk objects back in v3.0. Since libvirt now targets QEMU >= 7.2, we can safely rely on tls-creds-psk support being present unconditionally on both src and dst without needing feature capability probes. 2. Following our previous discussion in v2 on letting the destination decide whether to use PSK or certificates, the current patch had destination prefer x509 whenever x509 certs were available (qemuMigrationServerCertsExists), falling back to PSK otherwise. However, as you rightly noted, this assumes src has certs ready, which isn't guaranteed. So moving forward we have 2 design choices? 1. The source sends mig->tlsPSK along with an x509_present flag in the migration cookie. The destination uses x509 only if both sides have certs; otherwise, it falls back to PSK. 2. If `mig->tlsPSK` is present in the cookie, the destination always selects PSK for the session, bypassing the X.509 check entirely. Since, both the source and destination supports tls-creds-psk, I am confused with the statement: “IMO we want to use PSK If available“. Can you please explain this in detail?
+ virSecureEraseString(mig->tlsPSK); + mig->flags &= ~QEMU_MIGRATION_COOKIE_TLS_PSK; + g_clear_pointer(&mig->tlsPSK, g_free); + } + } + if (vm && flags & QEMU_MIGRATION_COOKIE_STATS && mig->jobData && vm->job->current) mig->jobData->operation = vm->job->current->operation;
On Mon, Aug 31, 2026 at 19:26:31 +0000, Abhisek Panda wrote:
On 20 Aug 2026, at 6:25 PM, Peter Krempa <pkrempa@redhat.com> wrote:
+static int +qemuMigrationPersistPSK(virQEMUDriver *driver, virDomainObj *vm, const char *tlsPSK) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autofree char *dir_path = NULL; + g_autofree char *key_path = NULL; + g_autofree char *shortName = NULL; + + if (!vm || !vm->def || !(shortName = virDomainDefGetShortName(vm->def))) + return -1;
This error path mixes situations where a libvirt error is raised (below) and when no error is reported. We don't allow that because the caller can't then know if an error was reported, thus all code paths must report error or all must not report error.
How can 'vm' or 'vm->def' even be NULL here? Does this check even make sense?
Acknowledged. Since ‘vm’ or ‘vm->def’ are guaranteed to be non NULL, I will drop the unnecessary checks. If virDomainDefGetShortName returns NULL, I will explicitly report VIR_ERR_INTERNAL_ERROR to keep error reporting consistent.
[...]
The caller guarantees tlsPSK is present when invoking this, so I have removed the tlsPSK check entirely.
[...]
So, since this is the only caller and it's guaranteed that tlsPSK exists and 'vm' is non, null here, all the nonsense checks I pointed out above can be removed.
Acknowledged.
[...]
Also the PSK should be written to the disk at any point when it will be used, which is decided before, so the flag check of VIR_MIGRATE_TLS doesn't make sense.
Acknowledged. I will update this to check ‘if mig->tlsPSK’ directly instead of Re-checking the flag.
[...]
This belongs to a common cleanup path after failed/completed migration, not this random location.
Acknowledged. The cookie cleanup path will handle freeing and erasing sensitive string.
[..]
IMO this should't be gated by the flag check but rather by whether the PSK dir was set up and all of it inside qemuMigrationDeletePSKDir so that any furher change doesn't need to fix all callers.
Acknowledged. qemuMigrationDeletePSKDir will check for the existence of the PSKDir. If it is present then we clean it up else we just return. Therefore, we do not need any explicit check on the flag.
[...]
+ if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm);
ditto
Acknowledged
[...]
same here
Acknowledged
[...]
And here you then don't need the comment.
Acknowledged
[...] Please avoid the acknowledgement of requested changes as seen above unless you have a followup needing discussion. I almost stopped reading this message after the fifth acknowledgement, which would mean I'd miss your question below.
@@ -1537,6 +1613,20 @@ qemuMigrationCookieParse(virQEMUDriver *driver, } }
+ if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK) { + if (!qemuMigrationServerCertsExists(driver)) { + if (!mig->tlsPSK) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("destination host has no X.509 certificates configured for migration and source host did not provide a pre-shared key")); + return NULL; + } + } else {
So, if the destination has x509 certs this signals that PSK is not supported? What if the source doesn't have them. IMO we want to use PSK if available.
I have the following understanding of the pre-shared key: 1. QEMU added support for tls-creds-psk objects back in v3.0. Since libvirt now targets QEMU >= 7.2, we can safely rely on tls-creds-psk support being present unconditionally on both src and dst without needing feature capability probes.
yup But you need to keep in mind that either the source or destination can be an older libvirt version which doesn't support PSK.
2. Following our previous discussion in v2 on letting the destination decide whether to use PSK or certificates, the current patch had destination prefer x509 whenever x509 certs were available (qemuMigrationServerCertsExists), falling back to PSK otherwise. However, as you rightly noted, this assumes src has certs ready, which isn't guaranteed.
So I'd normally prefer to use x509 if it was already set up. The problem is that it's hard to know if it is set up properly before trying. Since the setup of PSK is much simpler and much more likely to work the simplest option is to use PSK as default and use x509 only if one of the sides doesn't support PSK. Security-wise this will not be a downgrade because the libvirt connection needs to be secured too. As mentioned earlier I want to also enable TLS+PSK always (when supported) even when the _TLS flag wasn't supplied so that we provide security by default (that's why I've asked for removing some of the checks and depending on the actual setup).
1. The source sends mig->tlsPSK along with an x509_present flag in the migration cookie. The destination uses x509 only if both sides have certs; otherwise, it falls back to PSK.
The issue is that both hosts can have x509 set up incompatibly which wouldn't work, but with PSK it will.
2. If `mig->tlsPSK` is present in the cookie, the destination always selects PSK for the session, bypassing the X.509 check entirely.
Since, both the source and destination supports tls-creds-psk, I am confused with the statement: “IMO we want to use PSK If available“. Can you please explain this in detail?
So I think an advanced version of 2 is the correct approach. We should: 1) If both sides support PSK, use PSK. Even when VIR_MIGRATE_TLS is not specified. 2) If one of the sides doesn't support PSK and VIR_MIGRATE_TLS is specified, try x509 3) Otherwise - though luck The use of VIR_MIGRATE_TLS will then gate that the migration will fail if neither PSK nor x509 is supported.
On 1 Sep 2026, at 12:48 PM, Peter Krempa <pkrempa@redhat.com> wrote:
!-------------------------------------------------------------------| CAUTION: External Email
|-------------------------------------------------------------------!
On Mon, Aug 31, 2026 at 19:26:31 +0000, Abhisek Panda wrote:
On 20 Aug 2026, at 6:25 PM, Peter Krempa <pkrempa@redhat.com> wrote:
+static int +qemuMigrationPersistPSK(virQEMUDriver *driver, virDomainObj *vm, const char *tlsPSK) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autofree char *dir_path = NULL; + g_autofree char *key_path = NULL; + g_autofree char *shortName = NULL; + + if (!vm || !vm->def || !(shortName = virDomainDefGetShortName(vm->def))) + return -1;
This error path mixes situations where a libvirt error is raised (below) and when no error is reported. We don't allow that because the caller can't then know if an error was reported, thus all code paths must report error or all must not report error.
How can 'vm' or 'vm->def' even be NULL here? Does this check even make sense?
Acknowledged. Since ‘vm’ or ‘vm->def’ are guaranteed to be non NULL, I will drop the unnecessary checks. If virDomainDefGetShortName returns NULL, I will explicitly report VIR_ERR_INTERNAL_ERROR to keep error reporting consistent.
[...]
The caller guarantees tlsPSK is present when invoking this, so I have removed the tlsPSK check entirely.
[...]
So, since this is the only caller and it's guaranteed that tlsPSK exists and 'vm' is non, null here, all the nonsense checks I pointed out above can be removed.
Acknowledged.
[...]
Also the PSK should be written to the disk at any point when it will be used, which is decided before, so the flag check of VIR_MIGRATE_TLS doesn't make sense.
Acknowledged. I will update this to check ‘if mig->tlsPSK’ directly instead of Re-checking the flag.
[...]
This belongs to a common cleanup path after failed/completed migration, not this random location.
Acknowledged. The cookie cleanup path will handle freeing and erasing sensitive string.
[..]
IMO this should't be gated by the flag check but rather by whether the PSK dir was set up and all of it inside qemuMigrationDeletePSKDir so that any furher change doesn't need to fix all callers.
Acknowledged. qemuMigrationDeletePSKDir will check for the existence of the PSKDir. If it is present then we clean it up else we just return. Therefore, we do not need any explicit check on the flag.
[...]
+ if (flags & VIR_MIGRATE_TLS) + qemuMigrationDeletePSKDir(driver, vm);
ditto
Acknowledged
[...]
same here
Acknowledged
[...]
And here you then don't need the comment.
Acknowledged
[...]
Please avoid the acknowledgement of requested changes as seen above unless you have a followup needing discussion. I almost stopped reading this message after the fifth acknowledgement, which would mean I'd miss your question below.
@@ -1537,6 +1613,20 @@ qemuMigrationCookieParse(virQEMUDriver *driver, } }
+ if (flags & QEMU_MIGRATION_COOKIE_TLS_PSK) { + if (!qemuMigrationServerCertsExists(driver)) { + if (!mig->tlsPSK) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("destination host has no X.509 certificates configured for migration and source host did not provide a pre-shared key")); + return NULL; + } + } else {
So, if the destination has x509 certs this signals that PSK is not supported? What if the source doesn't have them. IMO we want to use PSK if available.
I have the following understanding of the pre-shared key: 1. QEMU added support for tls-creds-psk objects back in v3.0. Since libvirt now targets QEMU >= 7.2, we can safely rely on tls-creds-psk support being present unconditionally on both src and dst without needing feature capability probes.
yup
But you need to keep in mind that either the source or destination can be an older libvirt version which doesn't support PSK.
2. Following our previous discussion in v2 on letting the destination decide whether to use PSK or certificates, the current patch had destination prefer x509 whenever x509 certs were available (qemuMigrationServerCertsExists), falling back to PSK otherwise. However, as you rightly noted, this assumes src has certs ready, which isn't guaranteed.
So I'd normally prefer to use x509 if it was already set up. The problem is that it's hard to know if it is set up properly before trying.
Since the setup of PSK is much simpler and much more likely to work the simplest option is to use PSK as default and use x509 only if one of the sides doesn't support PSK.
Security-wise this will not be a downgrade because the libvirt connection needs to be secured too.
As mentioned earlier I want to also enable TLS+PSK always (when supported) even when the _TLS flag wasn't supplied so that we provide security by default (that's why I've asked for removing some of the checks and depending on the actual setup).
1. The source sends mig->tlsPSK along with an x509_present flag in the migration cookie. The destination uses x509 only if both sides have certs; otherwise, it falls back to PSK.
The issue is that both hosts can have x509 set up incompatibly which wouldn't work, but with PSK it will.
2. If `mig->tlsPSK` is present in the cookie, the destination always selects PSK for the session, bypassing the X.509 check entirely.
Since, both the source and destination supports tls-creds-psk, I am confused with the statement: “IMO we want to use PSK If available“. Can you please explain this in detail?
So I think an advanced version of 2 is the correct approach. We should:
1) If both sides support PSK, use PSK. Even when VIR_MIGRATE_TLS is not specified. 2) If one of the sides doesn't support PSK and VIR_MIGRATE_TLS is specified, try x509 3) Otherwise - though luck
The use of VIR_MIGRATE_TLS will then gate that the migration will fail if neither PSK nor x509 is supported.
I have pushed v4 of this patch series for review with the advanced version of 2nd design. Given that with encrypted migration, there will be a performance impact to the live migrations of the VMs, due to auxiliary encrypt and decrypt operations at the source and destination, and also due to a lack of MSG_ZEROCOPY support for encrypted migrations in QEMU. Some users might be running VM migration within a cluster of trusted nodes, for which encrypted migration might not be a performant solution. Can we gate this behaviour of by-default enablement of PSK behind a configuration parameter in qemu.conf. This design is as follows: 1. If the configuration parameter is set we do advanced version of 2nd design. 2. If the configuration parameter is not set, then we attempt PSK if VIR_MIGRATE_TLS flag is specified and both source and destination supports it. If either of the nodes doesn’t support it we fallback to x509. Would you please let me know your opinion on this design?
Build the tls-creds-psk object with the following params: id, dir, endpoint. Note: Libvirt generates a keys.psk file for each migration session that only contains the pre-shared key for the "qemu" user. Because QEMU defaults to using "qemu" as the username if the username attribute of the tls-creds-psk object is undefined. We intentionally do not set this attribute. Suggested-by: Tejus GK <tejus.gk@nutanix.com> Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/qemu_command.c | 26 ++++++++++++++++++++++++++ src/qemu/qemu_command.h | 7 +++++++ 2 files changed, 33 insertions(+) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 95e2ea9a6b..b08dca4b66 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1293,6 +1293,32 @@ qemuBuildTLSx509CommandLine(virCommand *cmd, } +/* qemuBuildTLSPSKBackendProps: + * @tlsPSKPath: path to the TLS-PSK credentials file + * @isListen: boolean listen for client or server setting + * @alias: alias for the TLS-PSK object + * @propsret: json properties to return + * + * Create a backend string for the tls-creds-psk object. + * + * Returns 0 on success, -1 on failure with error set. + */ +int +qemuBuildTLSPSKBackendProps(const char *tlsPSKPath, + bool isListen, + const char *alias, + virJSONValue **propsret) +{ + if (qemuMonitorCreateObjectProps(propsret, "tls-creds-psk", alias, + "s:dir", tlsPSKPath, + "s:endpoint", (isListen ? "server": "client"), + NULL) < 0) + return -1; + + return 0; +} + + static int qemuBuildChardevCommand(virCommand *cmd, const virDomainChrSourceDef *dev, diff --git a/src/qemu/qemu_command.h b/src/qemu/qemu_command.h index e34172fbff..92ae32d285 100644 --- a/src/qemu/qemu_command.h +++ b/src/qemu/qemu_command.h @@ -72,6 +72,13 @@ qemuBuildTLSx509BackendProps(const char *tlspath, const char *secalias, virJSONValue **propsret); +/* Generate the object properties for a tls-creds-psk */ +int +qemuBuildTLSPSKBackendProps(const char *tlsPSKPath, + bool isListen, + const char *alias, + virJSONValue **propsret); + /* Open a UNIX socket for chardev FD passing */ int qemuOpenChrChardevUNIXSocket(const virDomainChrSourceDef *dev); -- 2.43.7
On Wed, Jul 29, 2026 at 08:58:55 +0000, Abhisek Panda wrote:
Build the tls-creds-psk object with the following params: id, dir, endpoint.
Note: Libvirt generates a keys.psk file for each migration session that only contains the pre-shared key for the "qemu" user. Because QEMU defaults to using "qemu" as the username if the username attribute of the tls-creds-psk object is undefined. We intentionally do not set this attribute.
Suggested-by: Tejus GK <tejus.gk@nutanix.com> Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/qemu_command.c | 26 ++++++++++++++++++++++++++ src/qemu/qemu_command.h | 7 +++++++ 2 files changed, 33 insertions(+)
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Append 'x509' to the function identifiers managing the tls-creds-x509 objects. This defines the functions' scope and prevents naming conflicts with the introduction of functions related to tls-creds-psk in subsequent commits. Suggested-by: Tejus GK <tejus.gk@nutanix.com> Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/qemu_hotplug.c | 40 ++++++++++++++++---------------- src/qemu/qemu_hotplug.h | 24 +++++++++---------- src/qemu/qemu_migration.c | 16 ++++++------- src/qemu/qemu_migration_params.c | 32 ++++++++++++------------- src/qemu/qemu_migration_params.h | 14 +++++------ 5 files changed, 63 insertions(+), 63 deletions(-) diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 5be567b510..a192ab1a0b 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -1729,10 +1729,10 @@ qemuDomainDelTLSObjects(virDomainObj *vm, int -qemuDomainAddTLSObjects(virDomainObj *vm, - virDomainAsyncJob asyncJob, - virJSONValue **secProps, - virJSONValue **tlsProps) +qemuDomainAddTLSx509Objects(virDomainObj *vm, + virDomainAsyncJob asyncJob, + virJSONValue **secProps, + virJSONValue **tlsProps) { qemuDomainObjPrivate *priv = vm->privateData; virErrorPtr orig_err; @@ -1766,14 +1766,14 @@ qemuDomainAddTLSObjects(virDomainObj *vm, int -qemuDomainGetTLSObjects(qemuDomainSecretInfo *secinfo, - const char *tlsCertdir, - bool tlsListen, - bool tlsVerify, - const char *tlsPriority, - const char *alias, - virJSONValue **tlsProps, - virJSONValue **secProps) +qemuDomainGetTLSx509Objects(qemuDomainSecretInfo *secinfo, + const char *tlsCertdir, + bool tlsListen, + bool tlsVerify, + const char *tlsPriority, + const char *alias, + virJSONValue **tlsProps, + virJSONValue **secProps) { const char *secAlias = NULL; @@ -1824,18 +1824,18 @@ qemuDomainAddChardevTLSObjects(virQEMUDriver *driver, if (!(*tlsAlias = qemuAliasTLSObjFromSrcAlias(charAlias))) return -1; - if (qemuDomainGetTLSObjects(secinfo, - cfg->chardevTLSx509certdir, - dev->data.tcp.listen, - cfg->chardevTLSx509verify, - cfg->chardevTLSpriority, - *tlsAlias, &tlsProps, &secProps) < 0) + if (qemuDomainGetTLSx509Objects(secinfo, + cfg->chardevTLSx509certdir, + dev->data.tcp.listen, + cfg->chardevTLSx509verify, + cfg->chardevTLSpriority, + *tlsAlias, &tlsProps, &secProps) < 0) return -1; dev->data.tcp.tlscreds = true; - if (qemuDomainAddTLSObjects(vm, VIR_ASYNC_JOB_NONE, - &secProps, &tlsProps) < 0) + if (qemuDomainAddTLSx509Objects(vm, VIR_ASYNC_JOB_NONE, + &secProps, &tlsProps) < 0) return -1; return 0; diff --git a/src/qemu/qemu_hotplug.h b/src/qemu/qemu_hotplug.h index 60ed0e174c..0c969ab7f0 100644 --- a/src/qemu/qemu_hotplug.h +++ b/src/qemu/qemu_hotplug.h @@ -31,20 +31,20 @@ qemuDomainDelTLSObjects(virDomainObj *vm, const char *tlsAlias); int -qemuDomainAddTLSObjects(virDomainObj *vm, - virDomainAsyncJob asyncJob, - virJSONValue **secProps, - virJSONValue **tlsProps); +qemuDomainAddTLSx509Objects(virDomainObj *vm, + virDomainAsyncJob asyncJob, + virJSONValue **secProps, + virJSONValue **tlsProps); int -qemuDomainGetTLSObjects(qemuDomainSecretInfo *secinfo, - const char *tlsCertdir, - bool tlsListen, - bool tlsVerify, - const char *tlsPriority, - const char *alias, - virJSONValue **tlsProps, - virJSONValue **secProps); +qemuDomainGetTLSx509Objects(qemuDomainSecretInfo *secinfo, + const char *tlsCertdir, + bool tlsListen, + bool tlsVerify, + const char *tlsPriority, + const char *alias, + virJSONValue **tlsProps, + virJSONValue **secProps); int qemuDomainAttachDiskGeneric(virDomainObj *vm, diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 3d6e472443..943478fe68 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3511,10 +3511,10 @@ qemuMigrationDstPrepareActive(virQEMUDriver *driver, /* Migrations using TLS need to add the "tls-creds-x509" object and * set the migration TLS parameters */ if (flags & VIR_MIGRATE_TLS) { - if (qemuMigrationParamsEnableTLS(driver, vm, true, - VIR_ASYNC_JOB_MIGRATION_IN, - &tlsAlias, NULL, - migParams) < 0) + if (qemuMigrationParamsEnableTLSx509(driver, vm, true, + VIR_ASYNC_JOB_MIGRATION_IN, + &tlsAlias, NULL, + migParams) < 0) goto error; } else { if (qemuMigrationParamsDisableTLS(vm, migParams) < 0) @@ -5171,10 +5171,10 @@ qemuMigrationSrcRun(virQEMUDriver *driver, spec->destType == MIGRATION_DEST_FD) hostname = spec->dest.host.name; - if (qemuMigrationParamsEnableTLS(driver, vm, false, - VIR_ASYNC_JOB_MIGRATION_OUT, - &tlsAlias, hostname, - migParams) < 0) + if (qemuMigrationParamsEnableTLSx509(driver, vm, false, + VIR_ASYNC_JOB_MIGRATION_OUT, + &tlsAlias, hostname, + migParams) < 0) goto error; } else { if (qemuMigrationParamsDisableTLS(vm, migParams) < 0) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index dd47516742..d551ab9216 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -1150,12 +1150,12 @@ qemuMigrationParamsSetString(qemuMigrationParams *migParams, } -/* qemuMigrationParamsEnableTLS +/* qemuMigrationParamsEnableTLSx509 * @driver: pointer to qemu driver * @vm: domain object * @tlsListen: server or client * @asyncJob: Migration job to join - * @tlsAlias: alias to be generated for TLS object + * @tlsAlias: alias to be generated for TLS X.509 object * @hostname: hostname of the migration destination * @migParams: migration parameters to set * @@ -1166,17 +1166,17 @@ qemuMigrationParamsSetString(qemuMigrationParams *migParams, * Returns 0 on success, -1 on failure */ int -qemuMigrationParamsEnableTLS(virQEMUDriver *driver, - virDomainObj *vm, - bool tlsListen, - int asyncJob, - char **tlsAlias, - const char *hostname, - qemuMigrationParams *migParams) +qemuMigrationParamsEnableTLSx509(virQEMUDriver *driver, + virDomainObj *vm, + bool tlsListen, + int asyncJob, + char **tlsAlias, + const char *hostname, + qemuMigrationParams *migParams) { qemuDomainObjPrivate *priv = vm->privateData; qemuDomainJobPrivate *jobPriv = vm->job->privateData; - g_autoptr(virJSONValue) tlsProps = NULL; + g_autoptr(virJSONValue) tlsx509Props = NULL; g_autoptr(virJSONValue) secProps = NULL; g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); const char *secAlias = NULL; @@ -1205,11 +1205,11 @@ qemuMigrationParamsEnableTLS(virQEMUDriver *driver, if (!(*tlsAlias = qemuAliasTLSObjFromSrcAlias(QEMU_MIGRATION_TLS_ALIAS_BASE))) return -1; - if (qemuDomainGetTLSObjects(priv->migSecinfo, - cfg->migrateTLSx509certdir, tlsListen, - cfg->migrateTLSx509verify, - cfg->migrateTLSpriority, - *tlsAlias, &tlsProps, &secProps) < 0) + if (qemuDomainGetTLSx509Objects(priv->migSecinfo, + cfg->migrateTLSx509certdir, tlsListen, + cfg->migrateTLSx509verify, + cfg->migrateTLSpriority, + *tlsAlias, &tlsx509Props, &secProps) < 0) return -1; /* Ensure the domain doesn't already have the TLS objects defined... @@ -1218,7 +1218,7 @@ qemuMigrationParamsEnableTLS(virQEMUDriver *driver, * some other error path between now and perform . */ qemuDomainDelTLSObjects(vm, asyncJob, secAlias, *tlsAlias); - if (qemuDomainAddTLSObjects(vm, asyncJob, &secProps, &tlsProps) < 0) + if (qemuDomainAddTLSx509Objects(vm, asyncJob, &secProps, &tlsx509Props) < 0) return -1; if (qemuMigrationParamsSetString(migParams, diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index b7a829b85a..14e20fb71e 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -115,13 +115,13 @@ qemuMigrationParamsApply(virDomainObj *vm, unsigned int apiFlags); int -qemuMigrationParamsEnableTLS(virQEMUDriver *driver, - virDomainObj *vm, - bool tlsListen, - int asyncJob, - char **tlsAlias, - const char *hostname, - qemuMigrationParams *migParams); +qemuMigrationParamsEnableTLSx509(virQEMUDriver *driver, + virDomainObj *vm, + bool tlsListen, + int asyncJob, + char **tlsAlias, + const char *hostname, + qemuMigrationParams *migParams); int qemuMigrationParamsDisableTLS(virDomainObj *vm, -- 2.43.7
On Wed, Jul 29, 2026 at 08:58:56 +0000, Abhisek Panda wrote:
Append 'x509' to the function identifiers managing the tls-creds-x509 objects. This defines the functions' scope and prevents naming conflicts with the introduction of functions related to tls-creds-psk in subsequent commits.
Suggested-by: Tejus GK <tejus.gk@nutanix.com> Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/qemu_hotplug.c | 40 ++++++++++++++++---------------- src/qemu/qemu_hotplug.h | 24 +++++++++---------- src/qemu/qemu_migration.c | 16 ++++++------- src/qemu/qemu_migration_params.c | 32 ++++++++++++------------- src/qemu/qemu_migration_params.h | 14 +++++------ 5 files changed, 63 insertions(+), 63 deletions(-)
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
To enable TLS-PSK-based authentication scheme, add support for instantiating the tls-creds-psk object through QEMU monitor. Suggested-by: Tejus GK <tejus.gk@nutanix.com> Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/qemu_migration_params.c | 66 ++++++++++++++++++++++++++++++++ src/qemu/qemu_migration_params.h | 8 ++++ 2 files changed, 74 insertions(+) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index d551ab9216..129f6540ec 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -31,6 +31,7 @@ #define LIBVIRT_QEMU_MIGRATION_PARAMSPRIV_H_ALLOW #include "qemu_migration_paramspriv.h" #include "qemu_monitor.h" +#include "qemu_command.h" #define VIR_FROM_THIS VIR_FROM_QEMU @@ -1237,6 +1238,71 @@ qemuMigrationParamsEnableTLSx509(virQEMUDriver *driver, } +/* qemuMigrationParamsEnableTLSPSK + * @driver: pointer to qemu driver + * @vm: domain object + * @tlsListen: server or client + * @asyncJob: Migration job to join + * @tlsPSKAlias: alias to be generated for TLS-PSK object + * @migParams: migration parameters to set + * + * Create the TLS PSK objects for the migration and set the migParams value. + * + * Returns 0 on success, -1 on failure + */ +int +qemuMigrationParamsEnableTLSPSK(virQEMUDriver *driver, + virDomainObj *vm, + bool tlsListen, + int asyncJob, + char **tlsAlias, + qemuMigrationParams *migParams) +{ + qemuDomainObjPrivate *priv = vm->privateData; + g_autoptr(virJSONValue) tlsPSKProps = NULL; + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autofree char *key_dir_path = NULL; + g_autofree char *shortName = NULL; + virErrorPtr orig_err = NULL; + + if (!vm || !vm->def || !(shortName = virDomainDefGetShortName(vm->def))) + return -1; + + key_dir_path = g_strdup_printf("%s/%s", cfg->tlsPSKStateDir, shortName); + + if (!(*tlsAlias = qemuAliasTLSObjFromSrcAlias(QEMU_MIGRATION_TLS_ALIAS_BASE))) + return -1; + + if (qemuBuildTLSPSKBackendProps(key_dir_path, tlsListen, *tlsAlias, &tlsPSKProps) < 0) + return -1; + + /* Ensure the domain doesn't already have the TLS-PSK objects defined. + * This should prevent any issues just in case some cleanup wasn't + * properly completed (both src and dst use the same alias) or + * some other error path. */ + qemuDomainDelTLSObjects(vm, asyncJob, NULL, *tlsAlias); + + + /* Add the tls-creds-psk object to QEMU */ + if (qemuDomainObjEnterMonitorAsync(vm, asyncJob) < 0) + return -1; + + if (qemuMonitorAddObject(priv->mon, &tlsPSKProps, NULL) < 0) { + virErrorPreserveLast(&orig_err); + qemuDomainObjExitMonitor(vm); + virErrorRestore(&orig_err); + return -1; + } + qemuDomainObjExitMonitor(vm); + + if (qemuMigrationParamsSetString(migParams, QEMU_MIGRATION_PARAM_TLS_CREDS, + *tlsAlias) < 0) + return -1; + + return 0; +} + + /* qemuMigrationParamsDisableTLS * @vm: domain object * @migParams: Pointer to a migration parameters block diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_params.h index 14e20fb71e..8774b4596c 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -123,6 +123,14 @@ qemuMigrationParamsEnableTLSx509(virQEMUDriver *driver, const char *hostname, qemuMigrationParams *migParams); +int +qemuMigrationParamsEnableTLSPSK(virQEMUDriver *driver, + virDomainObj *vm, + bool tlsListen, + int asyncJob, + char **tlsAlias, + qemuMigrationParams *migParams); + int qemuMigrationParamsDisableTLS(virDomainObj *vm, qemuMigrationParams *migParams); -- 2.43.7
On Wed, Jul 29, 2026 at 08:58:57 +0000, Abhisek Panda wrote:
To enable TLS-PSK-based authentication scheme, add support for instantiating the tls-creds-psk object through QEMU monitor.
Suggested-by: Tejus GK <tejus.gk@nutanix.com> Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- src/qemu/qemu_migration_params.c | 66 ++++++++++++++++++++++++++++++++ src/qemu/qemu_migration_params.h | 8 ++++ 2 files changed, 74 insertions(+)
diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index d551ab9216..129f6540ec 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -31,6 +31,7 @@ #define LIBVIRT_QEMU_MIGRATION_PARAMSPRIV_H_ALLOW #include "qemu_migration_paramspriv.h" #include "qemu_monitor.h" +#include "qemu_command.h"
#define VIR_FROM_THIS VIR_FROM_QEMU
@@ -1237,6 +1238,71 @@ qemuMigrationParamsEnableTLSx509(virQEMUDriver *driver, }
+/* qemuMigrationParamsEnableTLSPSK + * @driver: pointer to qemu driver + * @vm: domain object + * @tlsListen: server or client + * @asyncJob: Migration job to join + * @tlsPSKAlias: alias to be generated for TLS-PSK object + * @migParams: migration parameters to set + * + * Create the TLS PSK objects for the migration and set the migParams value. + * + * Returns 0 on success, -1 on failure + */ +int +qemuMigrationParamsEnableTLSPSK(virQEMUDriver *driver, + virDomainObj *vm, + bool tlsListen, + int asyncJob, + char **tlsAlias, + qemuMigrationParams *migParams) +{ + qemuDomainObjPrivate *priv = vm->privateData; + g_autoptr(virJSONValue) tlsPSKProps = NULL; + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver); + g_autofree char *key_dir_path = NULL; + g_autofree char *shortName = NULL; + virErrorPtr orig_err = NULL; + + if (!vm || !vm->def || !(shortName = virDomainDefGetShortName(vm->def))) + return -1;
Same as before; this would create a code path with no error. I'm fairly certain that at the point where this will be called, both vm and vm->def are guaranteed to exist.
+ + key_dir_path = g_strdup_printf("%s/%s", cfg->tlsPSKStateDir, shortName); + + if (!(*tlsAlias = qemuAliasTLSObjFromSrcAlias(QEMU_MIGRATION_TLS_ALIAS_BASE))) + return -1; + + if (qemuBuildTLSPSKBackendProps(key_dir_path, tlsListen, *tlsAlias, &tlsPSKProps) < 0) + return -1; + + /* Ensure the domain doesn't already have the TLS-PSK objects defined. + * This should prevent any issues just in case some cleanup wasn't + * properly completed (both src and dst use the same alias) or + * some other error path. */ + qemuDomainDelTLSObjects(vm, asyncJob, NULL, *tlsAlias); + + + /* Add the tls-creds-psk object to QEMU */ + if (qemuDomainObjEnterMonitorAsync(vm, asyncJob) < 0) + return -1; + + if (qemuMonitorAddObject(priv->mon, &tlsPSKProps, NULL) < 0) { + virErrorPreserveLast(&orig_err); + qemuDomainObjExitMonitor(vm); + virErrorRestore(&orig_err); + return -1;
Block indented incorrectly.
+ } + qemuDomainObjExitMonitor(vm); + + if (qemuMigrationParamsSetString(migParams, QEMU_MIGRATION_PARAM_TLS_CREDS, + *tlsAlias) < 0) + return -1; + + return 0; +} + + /* qemuMigrationParamsDisableTLS * @vm: domain object * @migParams: Pointer to a migration parameters blocka
With the 2 things above addressed: Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Enable TLS-PSK based secure migration, if and only if the VIR_MIGRATE_TLS flag is set and the destination host lacks necessary X.509 credentials (ca-cert.pem, server-cert.pem and server-key.pem). Suggested-by: Tejus GK <tejus.gk@nutanix.com> Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- include/libvirt/libvirt-domain.h | 11 +++-- src/qemu/qemu_migration.c | 83 +++++++++++++++++++++----------- 2 files changed, 64 insertions(+), 30 deletions(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 5b67f8f897..70d955fd31 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1089,9 +1089,14 @@ typedef enum { VIR_MIGRATE_POSTCOPY = (1 << 15), /* Setting the VIR_MIGRATE_TLS flag will cause the migration to attempt - * to use the TLS environment configured by the hypervisor in order to - * perform the migration. If incorrectly configured on either source or - * destination, the migration will fail. + * to use either the TLS X.509 or TLS pre-shared key (PSK) authentication + * mechanisms. If valid X.509 certificates and keys exist on the destination host, + * then Libvirt will use TLS X.509 authentication. Otherwise, Libvirt falls back to + * TLS PSK authentication. If the certificate or the key file is corrupted or not + * properly configured on either source or destination, the migration will fail. + * + * Note: Since 12.6.0, Libvirt supports the TLS pre-shared key (PSK) authentication + * mechanism along with the TLS X.509 authentication scheme. * * Since: 3.2.0 */ diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 943478fe68..5e381b96e9 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3508,14 +3508,28 @@ qemuMigrationDstPrepareActive(virQEMUDriver *driver, /* Save original migration parameters */ qemuDomainSaveStatus(vm); - /* Migrations using TLS need to add the "tls-creds-x509" object and + /* Migrations using TLS need to add the "tls-creds-x509" object if the cert files are + * present on the host, else fallback to adding the "tls-creds-psk" object. Additionally, * set the migration TLS parameters */ if (flags & VIR_MIGRATE_TLS) { - if (qemuMigrationParamsEnableTLSx509(driver, vm, true, - VIR_ASYNC_JOB_MIGRATION_IN, - &tlsAlias, NULL, - migParams) < 0) - goto error; + if (!mig->tlsPSK) { + if (qemuMigrationParamsEnableTLSx509(driver, vm, true, + VIR_ASYNC_JOB_MIGRATION_IN, + &tlsAlias, NULL, + migParams) < 0) + goto error; + } else { + ret = qemuMigrationPersistPSK(driver, vm, mig->tlsPSK); + if (ret < 0) { + virSecureEraseString(mig->tlsPSK); + g_clear_pointer(&mig->tlsPSK, g_free); + goto error; + } + if (qemuMigrationParamsEnableTLSPSK(driver, vm, true, + VIR_ASYNC_JOB_MIGRATION_IN, + &tlsAlias, migParams) < 0) + goto error; + } } else { if (qemuMigrationParamsDisableTLS(vm, migParams) < 0) goto error; @@ -3622,6 +3636,13 @@ qemuMigrationDstPrepareFresh(virQEMUDriver *driver, g_autofree char *xmlout = NULL; unsigned int cookieFlags = 0; bool taint_hook = false; + unsigned int parseCookieFlags = QEMU_MIGRATION_COOKIE_LOCKSTATE | + QEMU_MIGRATION_COOKIE_NBD | + QEMU_MIGRATION_COOKIE_MEMORY_HOTPLUG | + QEMU_MIGRATION_COOKIE_CPU_HOTPLUG | + QEMU_MIGRATION_COOKIE_CPU | + QEMU_MIGRATION_COOKIE_CAPS | + QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS; VIR_DEBUG("name=%s, origname=%s, protocol=%s, port=%hu, " "listenAddress=%s, nbdPort=%d, nbdURI=%s, flags=0x%x", @@ -3633,6 +3654,11 @@ qemuMigrationDstPrepareFresh(virQEMUDriver *driver, QEMU_MIGRATION_COOKIE_CAPS; } + if (flags & VIR_MIGRATE_TLS) { + parseCookieFlags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + cookieFlags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + } + /* Let migration hook filter domain XML */ if (virHookPresent(VIR_HOOK_DRIVER_QEMU)) { g_autofree char *xml = NULL; @@ -3679,14 +3705,7 @@ qemuMigrationDstPrepareFresh(virQEMUDriver *driver, * domain list. Parsing/validation may fail and there's no * point in having the domain in the list at that point. */ if (!(mig = qemuMigrationCookieParse(driver, NULL, *def, origname, NULL, - cookiein, cookieinlen, - QEMU_MIGRATION_COOKIE_LOCKSTATE | - QEMU_MIGRATION_COOKIE_NBD | - QEMU_MIGRATION_COOKIE_MEMORY_HOTPLUG | - QEMU_MIGRATION_COOKIE_CPU_HOTPLUG | - QEMU_MIGRATION_COOKIE_CPU | - QEMU_MIGRATION_COOKIE_CAPS | - QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS))) + cookiein, cookieinlen, parseCookieFlags))) goto cleanup; if (!(vm = virDomainObjListAdd(driver->domains, def, @@ -5112,6 +5131,9 @@ qemuMigrationSrcRun(virQEMUDriver *driver, if (storageMigration) cookieFlags |= QEMU_MIGRATION_COOKIE_NBD; + if (flags & VIR_MIGRATE_TLS) + cookieFlags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + if (virLockManagerPluginUsesState(driver->lockManager) && !cookieout) { virReportError(VIR_ERR_INTERNAL_ERROR, @@ -5163,19 +5185,26 @@ qemuMigrationSrcRun(virQEMUDriver *driver, qemuDomainSaveStatus(vm); if (flags & VIR_MIGRATE_TLS) { - const char *hostname = NULL; - - /* We need to add tls-hostname whenever QEMU itself does not - * connect directly to the destination. */ - if (spec->destType == MIGRATION_DEST_CONNECT_HOST || - spec->destType == MIGRATION_DEST_FD) - hostname = spec->dest.host.name; - - if (qemuMigrationParamsEnableTLSx509(driver, vm, false, - VIR_ASYNC_JOB_MIGRATION_OUT, - &tlsAlias, hostname, - migParams) < 0) - goto error; + if (mig->flags & QEMU_MIGRATION_COOKIE_TLS_PSK) { + if (qemuMigrationParamsEnableTLSPSK(driver, vm, false, + VIR_ASYNC_JOB_MIGRATION_OUT, + &tlsAlias, migParams) < 0) + goto error; + } else { + const char *hostname = NULL; + + /* We need to add tls-hostname whenever QEMU itself does not + * connect directly to the destination. */ + if (spec->destType == MIGRATION_DEST_CONNECT_HOST || + spec->destType == MIGRATION_DEST_FD) + hostname = spec->dest.host.name; + + if (qemuMigrationParamsEnableTLSx509(driver, vm, false, + VIR_ASYNC_JOB_MIGRATION_OUT, + &tlsAlias, hostname, + migParams) < 0) + goto error; + } } else { if (qemuMigrationParamsDisableTLS(vm, migParams) < 0) goto error; -- 2.43.7
On Wed, Jul 29, 2026 at 08:58:58 +0000, Abhisek Panda wrote:
Enable TLS-PSK based secure migration, if and only if the VIR_MIGRATE_TLS flag is set and the destination host lacks necessary X.509 credentials (ca-cert.pem, server-cert.pem and server-key.pem).
Suggested-by: Tejus GK <tejus.gk@nutanix.com> Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- include/libvirt/libvirt-domain.h | 11 +++-- src/qemu/qemu_migration.c | 83 +++++++++++++++++++++----------- 2 files changed, 64 insertions(+), 30 deletions(-)
diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 5b67f8f897..70d955fd31 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1089,9 +1089,14 @@ typedef enum { VIR_MIGRATE_POSTCOPY = (1 << 15),
/* Setting the VIR_MIGRATE_TLS flag will cause the migration to attempt - * to use the TLS environment configured by the hypervisor in order to - * perform the migration. If incorrectly configured on either source or - * destination, the migration will fail. + * to use either the TLS X.509 or TLS pre-shared key (PSK) authentication + * mechanisms. If valid X.509 certificates and keys exist on the destination host, + * then Libvirt will use TLS X.509 authentication. Otherwise, Libvirt falls back to + * TLS PSK authentication. If the certificate or the key file is corrupted or not + * properly configured on either source or destination, the migration will fail. + * + * Note: Since 12.6.0, Libvirt supports the TLS pre-shared key (PSK) authentication + * mechanism along with the TLS X.509 authentication scheme.
Per <https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/PLXLRIV33ZGREYCL745JXCF34TOLARGE/> freeze is on monday, so the version statement might need to be bumped. Once again I'm sorry about the review delay, but there's a lot of other stuff I need to deal with.
* * Since: 3.2.0 */ diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 943478fe68..5e381b96e9 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3508,14 +3508,28 @@ qemuMigrationDstPrepareActive(virQEMUDriver *driver, /* Save original migration parameters */ qemuDomainSaveStatus(vm);
- /* Migrations using TLS need to add the "tls-creds-x509" object and + /* Migrations using TLS need to add the "tls-creds-x509" object if the cert files are + * present on the host, else fallback to adding the "tls-creds-psk" object. Additionally, * set the migration TLS parameters */ if (flags & VIR_MIGRATE_TLS) { - if (qemuMigrationParamsEnableTLSx509(driver, vm, true, - VIR_ASYNC_JOB_MIGRATION_IN, - &tlsAlias, NULL, - migParams) < 0) - goto error; + if (!mig->tlsPSK) { + if (qemuMigrationParamsEnableTLSx509(driver, vm, true, + VIR_ASYNC_JOB_MIGRATION_IN, + &tlsAlias, NULL, + migParams) < 0) + goto error; + } else { + ret = qemuMigrationPersistPSK(driver, vm, mig->tlsPSK); + if (ret < 0) { + virSecureEraseString(mig->tlsPSK); + g_clear_pointer(&mig->tlsPSK, g_free); + goto error;
As noted before this belongs to the common cleanup. Also at a failure at this point the key was not used yet so there's no need to consider it a secret.
+ } + if (qemuMigrationParamsEnableTLSPSK(driver, vm, true, + VIR_ASYNC_JOB_MIGRATION_IN, + &tlsAlias, migParams) < 0) + goto error; + }
For future proofing (enabling PSK tls even without explicit flag) the logic should be: 1) if PSK exists enable TLS psk 2) if VIR_MIGRATE_TLS and x509 certs exist enable x509 3) otherwise do nothing
} else { if (qemuMigrationParamsDisableTLS(vm, migParams) < 0) goto error; @@ -3622,6 +3636,13 @@ qemuMigrationDstPrepareFresh(virQEMUDriver *driver, g_autofree char *xmlout = NULL; unsigned int cookieFlags = 0; bool taint_hook = false; + unsigned int parseCookieFlags = QEMU_MIGRATION_COOKIE_LOCKSTATE | + QEMU_MIGRATION_COOKIE_NBD | + QEMU_MIGRATION_COOKIE_MEMORY_HOTPLUG | + QEMU_MIGRATION_COOKIE_CPU_HOTPLUG | + QEMU_MIGRATION_COOKIE_CPU | + QEMU_MIGRATION_COOKIE_CAPS | + QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS;
VIR_DEBUG("name=%s, origname=%s, protocol=%s, port=%hu, " "listenAddress=%s, nbdPort=%d, nbdURI=%s, flags=0x%x", @@ -3633,6 +3654,11 @@ qemuMigrationDstPrepareFresh(virQEMUDriver *driver, QEMU_MIGRATION_COOKIE_CAPS; }
+ if (flags & VIR_MIGRATE_TLS) { + parseCookieFlags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + cookieFlags |= QEMU_MIGRATION_COOKIE_TLS_PSK;
Always parse PSK on destination. There's no need to gate this on VIR_MIGRATE_TLS
+ } + /* Let migration hook filter domain XML */ if (virHookPresent(VIR_HOOK_DRIVER_QEMU)) { g_autofree char *xml = NULL; @@ -3679,14 +3705,7 @@ qemuMigrationDstPrepareFresh(virQEMUDriver *driver, * domain list. Parsing/validation may fail and there's no * point in having the domain in the list at that point. */ if (!(mig = qemuMigrationCookieParse(driver, NULL, *def, origname, NULL, - cookiein, cookieinlen, - QEMU_MIGRATION_COOKIE_LOCKSTATE | - QEMU_MIGRATION_COOKIE_NBD | - QEMU_MIGRATION_COOKIE_MEMORY_HOTPLUG | - QEMU_MIGRATION_COOKIE_CPU_HOTPLUG | - QEMU_MIGRATION_COOKIE_CPU | - QEMU_MIGRATION_COOKIE_CAPS | - QEMU_MIGRATION_COOKIE_BLOCK_DIRTY_BITMAPS))) + cookiein, cookieinlen, parseCookieFlags))) goto cleanup;
if (!(vm = virDomainObjListAdd(driver->domains, def, @@ -5112,6 +5131,9 @@ qemuMigrationSrcRun(virQEMUDriver *driver, if (storageMigration) cookieFlags |= QEMU_MIGRATION_COOKIE_NBD;
+ if (flags & VIR_MIGRATE_TLS) + cookieFlags |= QEMU_MIGRATION_COOKIE_TLS_PSK; + if (virLockManagerPluginUsesState(driver->lockManager) && !cookieout) { virReportError(VIR_ERR_INTERNAL_ERROR, @@ -5163,19 +5185,26 @@ qemuMigrationSrcRun(virQEMUDriver *driver, qemuDomainSaveStatus(vm);
if (flags & VIR_MIGRATE_TLS) { - const char *hostname = NULL; - - /* We need to add tls-hostname whenever QEMU itself does not - * connect directly to the destination. */ - if (spec->destType == MIGRATION_DEST_CONNECT_HOST || - spec->destType == MIGRATION_DEST_FD) - hostname = spec->dest.host.name; - - if (qemuMigrationParamsEnableTLSx509(driver, vm, false, - VIR_ASYNC_JOB_MIGRATION_OUT, - &tlsAlias, hostname, - migParams) < 0) - goto error; + if (mig->flags & QEMU_MIGRATION_COOKIE_TLS_PSK) { + if (qemuMigrationParamsEnableTLSPSK(driver, vm, false, + VIR_ASYNC_JOB_MIGRATION_OUT, + &tlsAlias, migParams) < 0) + goto error; + } else { + const char *hostname = NULL; + + /* We need to add tls-hostname whenever QEMU itself does not + * connect directly to the destination. */ + if (spec->destType == MIGRATION_DEST_CONNECT_HOST || + spec->destType == MIGRATION_DEST_FD) + hostname = spec->dest.host.name; + + if (qemuMigrationParamsEnableTLSx509(driver, vm, false, + VIR_ASYNC_JOB_MIGRATION_OUT, + &tlsAlias, hostname, + migParams) < 0) + goto error; + }
Same note as before. If the PSK got through the handshake, you can always enable it rather than gating it behind VIR_MIGRATE_TLS. x509 needs to be gated behind VIR_MIGRATE_TLS but only if PSK is not agreed upon
} else { if (qemuMigrationParamsDisableTLS(vm, migParams) < 0) goto error; -- 2.43.7
In the case of TLS-PSK-enabled migration, Libvirt manages the lifecycle of the pre-shared key. Add a new test scenario to the migration cookie test suite to validate the generation of pre-shared key when the QEMU_MIGRATION_COOKIE_TLS_PSK flag is set. In addition, the test also checks the length and the hexadecimal encoding of the pre-shared key. Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- tests/qemumigrationcookiexmltest.c | 123 +++++++++++++++++++++++++++++ tests/testutilsqemu.c | 12 +++ 2 files changed, 135 insertions(+) diff --git a/tests/qemumigrationcookiexmltest.c b/tests/qemumigrationcookiexmltest.c index adf2eb4d08..0914793974 100644 --- a/tests/qemumigrationcookiexmltest.c +++ b/tests/qemumigrationcookiexmltest.c @@ -16,6 +16,7 @@ #include <config.h> +#include <stddef.h> #include <unistd.h> #include <sys/types.h> @@ -92,6 +93,99 @@ testQemuMigrationCookieDataFree(struct testQemuMigrationCookieData *data) } +static int +testQemuMigrationCookiePSKValidate(const char *key) +{ + g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(&driver); + size_t expected_key_len = 2 * cfg->migrateTLSPSKLength; /* The pre-shared key's length is 2 * PSK length */ + size_t i; + + if (!key) { + VIR_TEST_DEBUG("\nThe pre-shared key is not present in the migration cookie."); + return -1; + } + + if (strlen(key) != expected_key_len) { + VIR_TEST_DEBUG("\nThe length of the pre-shared key is incorrect."); + return -1; + } + + /* Check the characters of the key */ + for (i = 0; i < expected_key_len; i++) { + if (!g_ascii_isxdigit(key[i])) { + VIR_TEST_DEBUG("\nThe key contains non-hex characters"); + return -1; + } + } + + return 0; +} + + +static int +testQemuMigrationCookiePSKAttribute(const void *opaque) +{ + struct testQemuMigrationCookieData *data = (struct testQemuMigrationCookieData *) opaque; + g_autoptr(qemuMigrationCookie) cookie = NULL; + qemuDomainObjPrivate *priv; + + /* if the VM object parsing step failed there's nothing this test can do */ + if (!data->vm) { + VIR_TEST_DEBUG("\nmissing VM object\n"); + return -1; + } + priv = data->vm->privateData; + + if (!(cookie = qemuMigrationCookieNew(data->vm->def, NULL))) + return -1; + + g_free(cookie->localHostname); + cookie->localHostname = g_strdup("hostname2"); + + if (virUUIDParse("8b3f4dc4-6a8e-5f9b-94a5-4c35babd8d95", cookie->localHostuuid) < 0) { + VIR_TEST_DEBUG("\nfailed to parse fake UUID"); + return -1; + } + + /* Format the migration cookie at the source */ + if (qemuMigrationCookieFormat(cookie, + &driver, + data->vm, + data->cookiePopulateParty, + &data->xmlstr, + &data->xmlstrlen, + data->cookiePopulateFlags) < 0) { + VIR_TEST_DEBUG("\nfailed to populate and format qemu migration cookie"); + return -1; + } + + /* Check for the <migration-key> element in the generated migration cookie XML */ + if (!strstr(data->xmlstr, "<migration-key>")) { + VIR_TEST_DEBUG("\nthe migration cookie is missing the migration-key element:\n%s\n", + data->xmlstr); + return -1; + } + + /* Parse the migration cookie at the destination */ + if (!(data->cookie = qemuMigrationCookieParse(&driver, + data->vm, + data->vm->def, + NULL, + priv->qemuCaps, + data->xmlstr, + data->xmlstrlen, + data->cookieParseFlags))) { + VIR_TEST_DEBUG("\nfailed to parse qemu migration cookie:\n%s\n", data->xmlstr); + return -1; + } + + if (testQemuMigrationCookiePSKValidate(data->cookie->tlsPSK) < 0) + return -1; + + return 0; +} + + static int testQemuMigrationCookiePopulate(const void *opaque) { @@ -418,6 +512,32 @@ testQemuMigrationCookieXML2XMLBitmaps(const char *name, return ret; } +/* tests the generation and persistence of the pre-shared keys */ +static int +testQemuMigrationCookiePSK(const char *domxml) +{ + struct testQemuMigrationCookieData *data = g_new0(struct testQemuMigrationCookieData, 1); + int ret = 0; + + data->cookiePopulateFlags = QEMU_MIGRATION_COOKIE_TLS_PSK; + data->cookiePopulateParty = QEMU_MIGRATION_SOURCE; + data->cookieParseFlags = QEMU_MIGRATION_COOKIE_TLS_PSK; + data->inStatus = g_strconcat(abs_srcdir, "/", domxml, NULL); + + /* load status XML as domain object */ + if (testQemuMigrationCookieDomInit(data) < 0) + ret = -1; + + /* test PSK generation and persistence */ + if (virTestRun("qemumigrationcookie-psk-lifecycle", + testQemuMigrationCookiePSKAttribute, data) < 0) + ret = -1; + + testQemuMigrationCookieDataFree(data); + + return ret; +} + static int mymain(void) @@ -448,6 +568,9 @@ mymain(void) if (testQemuMigrationCookieXML2XMLBitmaps("nbd-bitmaps", "qemustatusxml2xmldata/migration-out-nbd-bitmaps-in.xml", 0) < 0) ret = -1; + if (testQemuMigrationCookiePSK("qemustatusxml2xmldata/modern-in.xml") < 0) + ret = -1; + virBufferFreeAndReset(&testnamebuf); cleanup: diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index e5e3342b53..b0e0d41895 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -297,6 +297,7 @@ qemuTestCapsCacheInsert(virFileCache *cache, #define STATEDIRTEMPLATE abs_builddir "/qemustatedir-XXXXXX" #define CONFIGDIRTEMPLATE abs_builddir "/qemuconfigdir-XXXXXX" +#define TLSPSKDIRTEMPLATE abs_builddir "/pskstatedir-XXXXXX" int qemuTestDriverInit(virQEMUDriver *driver) { @@ -304,6 +305,7 @@ int qemuTestDriverInit(virQEMUDriver *driver) virSecurityManager *mgr = NULL; char statedir[] = STATEDIRTEMPLATE; char configdir[] = CONFIGDIRTEMPLATE; + char tlspskdir[] = TLSPSKDIRTEMPLATE; memset(driver, 0, sizeof(*driver)); @@ -328,6 +330,7 @@ int qemuTestDriverInit(virQEMUDriver *driver) * dirs. */ VIR_FREE(cfg->stateDir); VIR_FREE(cfg->configDir); + VIR_FREE(cfg->tlsPSKStateDir); /* Override paths to ensure predictable output * @@ -367,6 +370,13 @@ int qemuTestDriverInit(virQEMUDriver *driver) cfg->configDir = g_strdup(configdir); + if (!g_mkdtemp(tlspskdir)) { + fprintf(stderr, "Cannot create fake tlsPSKStateDir"); + goto error; + } + + cfg->tlsPSKStateDir = g_strdup(tlspskdir); + driver->caps = testQemuCapsInit(); if (!driver->caps) goto error; @@ -433,6 +443,8 @@ int qemuTestDriverInit(virQEMUDriver *driver) cfg->dumpGuestCore = false; + cfg->migrateTLSPSKLength = 32; + driver->privileged = true; return 0; -- 2.43.7
On Wed, Jul 29, 2026 at 08:58:59 +0000, Abhisek Panda wrote:
In the case of TLS-PSK-enabled migration, Libvirt manages the lifecycle of the pre-shared key. Add a new test scenario to the migration cookie test suite to validate the generation of pre-shared key when the QEMU_MIGRATION_COOKIE_TLS_PSK flag is set. In addition, the test also checks the length and the hexadecimal encoding of the pre-shared key.
Signed-off-by: Abhisek Panda <abhisek.panda1@nutanix.com> --- tests/qemumigrationcookiexmltest.c | 123 +++++++++++++++++++++++++++++ tests/testutilsqemu.c | 12 +++ 2 files changed, 135 insertions(+)
Looks reasonable; but I'll do a once-over once other things are solved
On 29 Jul 2026, at 2:28 PM, Abhisek Panda <abhisek.panda1@nutanix.com> wrote:
QEMU provides the capability to encrypt the migration data stream using two transport layer security (TLS) authentication schemes: X.509 certificates and pre-shared keys (PSK). Currently, Libvirt only supports the X.509-based TLS authentication scheme. In TLS X.509 certificates, a set of live migrations utilize a fixed set of static certificates for encrypted migration. In this authentication scheme, users require to deploy a certificate authority and monitor the certificate expiration window. In case certificates are compromised all the future live migrations are vulnerable.
To resolve this, this patch series introduce the support for pre-shared key-based authentication scheme. This mechanism relies on symmetric pre-shared keys (a secret key that is known to both sender and receiver prior to secure communication) for providing secure transfer of data. Libvirt solely manages the lifecycle of the ephemeral pre-shared keys, including, generation, persistent storage, and cleanup. Libvirt generates the key on the source machine, then transfers it to the destination machine using the migration cookie. To allow users to configure the size of the key, Libvirt provides the migrate_tls_psk_length configuration parameter in qemu.conf.
To avoid introduction of an additional VIR_MIGRATE_* flag, we rely on existing the VIR_MIGRATE_TLS flag. If the VIR_MIGRATE_TLS flag is set but the necessary X.509 credential files are missing on the destination, then we fallback to using PSK-based authentication scheme during migration.
v3: 1. Destination host decides which TLS authentication scheme to use. 2. The directory of the key file is set to <tls_psk_state_dir>/$ID-$VMNAME. 3. Use the same alias for both tls-creds-x509 and tls-creds-psk objects. 4. Validate the length of the pre-shared key. 5. Unit test to validate the pre-shared key in the migration cookie.
v2: 1. Libvirt manages the lifecycle of pre-shared keys. 2. Transfer of keys to the destination via the migration cookie 3. Remove the VIR_MIGRATE_TLS_PSK flag instead rely on VIR_MIGRATE_TLS and availability of ca-cert.pem on source. 4. Drop VIR_MIGRATE_PARAM_TLS_PSK_DIRECTORY, Libvirt solely manages the pre-shared keys.
Abhisek Panda (7): conf: Add a configuration param for TLS-PSK qemu: Manage a pre-shared key's lifecycle qemu: Add support to build the tls-creds-psk object qemu: rename tls-creds-x509 obj related functions qemu: Manage tls-creds-psk object lifecycle qemu: Set up the migrate TLS-PSK objects tests: Add testing of pre-shared key lifecycle
include/libvirt/libvirt-domain.h | 11 +- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 8 ++ src/qemu/qemu_command.c | 26 ++++ src/qemu/qemu_command.h | 7 + src/qemu/qemu_conf.c | 22 ++++ src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 6 + src/qemu/qemu_hotplug.c | 40 +++--- src/qemu/qemu_hotplug.h | 24 ++-- src/qemu/qemu_migration.c | 202 +++++++++++++++++++++++++---- src/qemu/qemu_migration_cookie.c | 94 +++++++++++++- src/qemu/qemu_migration_cookie.h | 5 + src/qemu/qemu_migration_params.c | 98 +++++++++++--- src/qemu/qemu_migration_params.h | 22 +++- src/qemu/test_libvirtd_qemu.aug.in | 1 + tests/qemumigrationcookiexmltest.c | 141 +++++++++++++++++++- tests/testutilsqemu.c | 12 ++ 20 files changed, 631 insertions(+), 93 deletions(-)
-- 2.43.7
Ping for the series. Checking in to see if anyone has feedback on this patch set. Thanks and warm regards Dr. Abhisek Panda
On 6 Aug 2026, at 9:30 PM, Abhisek Panda <abhisek.panda1@nutanix.com> wrote:
On 29 Jul 2026, at 2:28 PM, Abhisek Panda <abhisek.panda1@nutanix.com> wrote:
QEMU provides the capability to encrypt the migration data stream using two transport layer security (TLS) authentication schemes: X.509 certificates and pre-shared keys (PSK). Currently, Libvirt only supports the X.509-based TLS authentication scheme. In TLS X.509 certificates, a set of live migrations utilize a fixed set of static certificates for encrypted migration. In this authentication scheme, users require to deploy a certificate authority and monitor the certificate expiration window. In case certificates are compromised all the future live migrations are vulnerable.
To resolve this, this patch series introduce the support for pre-shared key-based authentication scheme. This mechanism relies on symmetric pre-shared keys (a secret key that is known to both sender and receiver prior to secure communication) for providing secure transfer of data. Libvirt solely manages the lifecycle of the ephemeral pre-shared keys, including, generation, persistent storage, and cleanup. Libvirt generates the key on the source machine, then transfers it to the destination machine using the migration cookie. To allow users to configure the size of the key, Libvirt provides the migrate_tls_psk_length configuration parameter in qemu.conf.
To avoid introduction of an additional VIR_MIGRATE_* flag, we rely on existing the VIR_MIGRATE_TLS flag. If the VIR_MIGRATE_TLS flag is set but the necessary X.509 credential files are missing on the destination, then we fallback to using PSK-based authentication scheme during migration.
v3: 1. Destination host decides which TLS authentication scheme to use. 2. The directory of the key file is set to <tls_psk_state_dir>/$ID-$VMNAME. 3. Use the same alias for both tls-creds-x509 and tls-creds-psk objects. 4. Validate the length of the pre-shared key. 5. Unit test to validate the pre-shared key in the migration cookie.
v2: 1. Libvirt manages the lifecycle of pre-shared keys. 2. Transfer of keys to the destination via the migration cookie 3. Remove the VIR_MIGRATE_TLS_PSK flag instead rely on VIR_MIGRATE_TLS and availability of ca-cert.pem on source. 4. Drop VIR_MIGRATE_PARAM_TLS_PSK_DIRECTORY, Libvirt solely manages the pre-shared keys.
Abhisek Panda (7): conf: Add a configuration param for TLS-PSK qemu: Manage a pre-shared key's lifecycle qemu: Add support to build the tls-creds-psk object qemu: rename tls-creds-x509 obj related functions qemu: Manage tls-creds-psk object lifecycle qemu: Set up the migrate TLS-PSK objects tests: Add testing of pre-shared key lifecycle
include/libvirt/libvirt-domain.h | 11 +- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 8 ++ src/qemu/qemu_command.c | 26 ++++ src/qemu/qemu_command.h | 7 + src/qemu/qemu_conf.c | 22 ++++ src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 6 + src/qemu/qemu_hotplug.c | 40 +++--- src/qemu/qemu_hotplug.h | 24 ++-- src/qemu/qemu_migration.c | 202 +++++++++++++++++++++++++---- src/qemu/qemu_migration_cookie.c | 94 +++++++++++++- src/qemu/qemu_migration_cookie.h | 5 + src/qemu/qemu_migration_params.c | 98 +++++++++++--- src/qemu/qemu_migration_params.h | 22 +++- src/qemu/test_libvirtd_qemu.aug.in | 1 + tests/qemumigrationcookiexmltest.c | 141 +++++++++++++++++++- tests/testutilsqemu.c | 12 ++ 20 files changed, 631 insertions(+), 93 deletions(-)
-- 2.43.7
Ping for the series.
Checking in to see if anyone has feedback on this patch set.
Thanks and warm regards Dr. Abhisek Panda
Ping for the series. Checking in to see if anyone has feedback on this patch set. Thanks and warm regards Dr. Abhisek Panda
On Fri, Aug 14, 2026 at 06:43:46 +0000, Abhisek Panda wrote:
Ping for the series.
Checking in to see if anyone has feedback on this patch set.
Thanks and warm regards Dr. Abhisek Panda
Ping for the series.
Checking in to see if anyone has feedback on this patch set.
I'm working myself through my backlog of patches for review that accumulated. I'll hopefully get to it today or on Monday
participants (2)
-
Abhisek Panda -
Peter Krempa