1:38 p.m.
v2: http://www.redhat.com/archives/libvir-list/2017-February/msg01374.html
NB: Parts of the original series were split out, patches posted, ACK'd,
and committed separately, see:
http://www.redhat.com/archives/libvir-list/2017-March/msg00037.html
Changes since v2:
First and foremost after some gentle encouragement and lots of hours cursing
at no one in particular, I was finally able to configure a couple of nested
host guests, create a guest on each, and actually migrate that guest without
TLS back and forth between the two nested hosts. Suffice to say there are
dribs and drabs of "howto"'s that help, but it is quite a confusing setup
and I was certainly left wondering how customers do this. OK beyond paying
someone set things up for them!
Now that I had the basics, the next task was creating the TLS environment.
MANY more curse words uttered, including some I didn't know I knew, but I
was actually able to test the changes in my environment. Suffice to say it's
"somewhat confusing" because the TLS environment that QEMU expects (ok, the
filenames) differ slightly than those that a libvirt TLS environment would
expects. What really got frustrating is that all I'd get back is the
following error message after 'migrate' started on the client and after
the server was essentially waiting for the migration:
error: operation failed: migration job: unexpectedly failed
Turns out that I was missing the '/etc/pki/qemu/client-cert.pem' and
'/etc/pki/qemu/client-key.pem' files, but got no message. Of course I
did have something in my /etc/pki/libvirt directory, but of course those
weren't being used. Still my brain wasn't exactly telling me that. The
other piece I was missing was the "details" about when "tls-hostname"
must be provided. In the previous patches, I would search on "fd" or
"spec" in spec->dest.host.protocol; however, that wasn't quite right.
What I needed to search on was MIGRATION_DEST_CONNECT_HOST or
MIGRATION_DEST_FD in spec->destType (and _CONNECT_HOST only because it
turns into _FD 'eventually').
Anyway, so happy days are here again and I can now liberally celebrate
St Patrick's Day (not that I wasn't going to before, but now moreso!).
Beyond the above description from my testing the real "key" behind all
of this is ensuring that the 'right' QEMU bits are installed by fetching
the "tls-creds" via qemuMonitorGetMigrationParams and as long as *something*
is returned, then we are assured that we can proceed. This relates to QEMU
commit '4af245dc3' (after v2.9.0-rc0) which sets a default for tls-creds
and more importantly for our purposes allows us to "clear" the tls-creds
and tls-hostname migration parameters by passing a "" (empty string).
With that in place the rest of the logic falls into place with regard
to qemuMigrationCheckSetupTLS ensuring that TLS for migration is available
for libvirt to use.
Once we know we can use TLS, we'll use qemuMigrationAddTLSObjects to
add the "secret" and "tls-creds-x509" objects to the domain for
either
server (target) or client (source). Doing this also required an adjustment
to the existing code to allow the Async monitor usage.
Once those objects are created, we'll set the "tls-creds" and
"tls-hostname"
parameters appropriate. If TLS isn't being used, we'll be sure to clear
them. This is only made tricky by needing to ensure the qemu that's running
can support setting them to "", hence the qemuMigrationCheckTLSCreds prior
to that clearing. If this wasn't done, older QEMU would be less than pleased
to find "" in tls-creds and tls-hostname.
Finally when the migration is successful, call qemuMigrationDeconstructTLS
(naming credit goes to my wife!). Failures will call just qemuMigrationResetTLS
in order to achieve the same call; however, for those paths we do not still
have the original 'tlsAlias' and 'secAlias' available, so we have to
recreate
it on the fly'
Have I won the prize for the longest cover letter yet?
John Ferlan (7):
qemu: Create #define for TLS configuration setup.
conf: Introduce migrate_tls_x509_cert_dir
Add new migration flag VIR_MIGRATE_TLS
qemu: Add TLS params to _qemuMonitorMigrationParams
qemu: Add job for qemuDomain{Add|Del}TLSObjects
qemu: Set up the migrate TLS objects for target
qemu: Set up the migration TLS objects for source
include/libvirt/libvirt-domain.h | 8 +
src/qemu/libvirtd_qemu.aug | 5 +
src/qemu/qemu.conf | 37 ++++
src/qemu/qemu_conf.c | 45 ++--
src/qemu/qemu_conf.h | 4 +
src/qemu/qemu_domain.c | 7 +-
src/qemu/qemu_domain.h | 91 ++++----
src/qemu/qemu_driver.c | 4 +-
src/qemu/qemu_hotplug.c | 24 ++-
src/qemu/qemu_hotplug.h | 2 +
src/qemu/qemu_migration.c | 423 ++++++++++++++++++++++++++++++++++++-
src/qemu/qemu_migration.h | 9 +-
src/qemu/qemu_monitor.c | 11 +-
src/qemu/qemu_monitor.h | 3 +
src/qemu/qemu_monitor_json.c | 28 +++
src/qemu/test_libvirtd_qemu.aug.in | 3 +
tests/qemumonitorjsontest.c | 25 ++-
tools/virsh-domain.c | 7 +
18 files changed, 668 insertions(+), 68 deletions(-)
--
2.9.3