[libvirt] qemu+tls server certificate validation failure (The certificate is not trusted)

I'm having a problem with remote TLS libvirt connections from an Ubuntu Jaunty client. I've reported the bug here[1] but haven't had any hits yet so I thought I'd come to the source. Let me know if ya'll have any ideas or know of any bugs in the versions I'm using (see below). I just upgraded my client to Jaunty from Intrepid and I can no longer connect to Hardy or Intrepid libvirt servers that have TLS enabled. I get the following errors: $ virt-viewer -c qemu+tls://example.com/system virt.example.com libvir: Remote error : server certificate failed validation: The certificate is not trusted. libvir: Remote error : unable to connect to 'example.com': Invalid argument unable to connect to libvirt qemu+tls://example.com/system $ In the past (ie hardy, intrepid) I was able to use the following command. Now I get an error: $ virt-viewer -c qemu://example.com/system virt.example.com libvir: error : could not connect to qemu://example.com/system unable to connect to libvirt qemu://example.com/system $ The server's config has not changed (I've tested against libvirt-bin versions 0.4.4-3ubuntu3.1 and 0.4.0-2ubuntu8.1 on the server side). I have the CA certificate installed on both server and client (in /etc/pki/CA/cacert.pem). That cert signed both my x509 client cert and the server cert. Here is some proof that it *should* work: $ openssl s_client -CAfile /etc/pki/CA/cacert.pem -cert /etc/pki/libvirt/clientcert.pem -key /etc/pki/libvirt/private/clientkey.pem -connect example.com:16514 2>/dev/null|sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'
servercert.pem $ openssl verify -CAfile /etc/pki/CA/cacert.pem servercert.pem servercert.pem: OK $ openssl verify -CAfile /etc/pki/CA/cacert.pem /etc/pki/libvirt/clientcert.pem /etc/pki/libvirt/clientcert.pem: OK $
When I run strace against virt-viewer I can see that it is accessing and (successfully opening) the correct certs/keys: $ grep /etc/pki /tmp/out stat64("/etc/pki/CA/cacert.pem", {st_mode=S_IFREG|0644, st_size=1716, ...}) = 0 stat64("/etc/pki/libvirt/private/clientkey.pem", {st_mode=S_IFREG|0644, st_size=887, ...}) = 0 stat64("/etc/pki/libvirt/clientcert.pem", {st_mode=S_IFREG|0644, st_size=1172, ...}) = 0 open("/etc/pki/CA/cacert.pem", O_RDONLY) = 5 open("/etc/pki/libvirt/private/clientkey.pem", O_RDONLY) = 5 open("/etc/pki/libvirt/clientcert.pem", O_RDONLY) = 5 $ Thanks in advance, Scott ------------ [1] https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/366455

On Mon, Apr 27, 2009 at 02:37:28PM -0700, Scott Beardsley wrote:
I'm having a problem with remote TLS libvirt connections from an Ubuntu Jaunty client. I've reported the bug here[1] but haven't had any hits yet so I thought I'd come to the source. Let me know if ya'll have any ideas or know of any bugs in the versions I'm using (see below). I just upgraded my client to Jaunty from Intrepid and I can no longer connect to Hardy or Intrepid libvirt servers that have TLS enabled. I get the following errors:
$ virt-viewer -c qemu+tls://example.com/system virt.example.com libvir: Remote error : server certificate failed validation: The certificate is not trusted. libvir: Remote error : unable to connect to 'example.com': Invalid argument unable to connect to libvirt qemu+tls://example.com/system
This error message comes from gnutls_certificate_verify_peers2() and maps to the annoyingly generic GNUTLS_CERT_INVALID error code.
In the past (ie hardy, intrepid) I was able to use the following command. Now I get an error: $ virt-viewer -c qemu://example.com/system virt.example.com libvir: error : could not connect to qemu://example.com/system unable to connect to libvirt qemu://example.com/system $
The server's config has not changed (I've tested against libvirt-bin versions 0.4.4-3ubuntu3.1 and 0.4.0-2ubuntu8.1 on the server side). I have the CA certificate installed on both server and client (in /etc/pki/CA/cacert.pem). That cert signed both my x509 client cert and the server cert. Here is some proof that it *should* work:
I'd run some checks with the gnutls 'certtool' instead of openssl, so you can be sure you're running the same SSL code as libvirt uses. One random idea is that perhaps the newer GNUTLS in Jaunty has stopped supporting some feature used in your certificates. eg perhaps they finally disabled md5 algorithm for cert signing or similar ideas. certtool may give you info if this is the case Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

This error message comes from gnutls_certificate_verify_peers2() and maps to the annoyingly generic GNUTLS_CERT_INVALID error code.
indeed
The server's config has not changed (I've tested against libvirt-bin versions 0.4.4-3ubuntu3.1 and 0.4.0-2ubuntu8.1 on the server side). I have the CA certificate installed on both server and client (in /etc/pki/CA/cacert.pem). That cert signed both my x509 client cert and the server cert. Here is some proof that it *should* work:
I'd run some checks with the gnutls 'certtool' instead of openssl, so you can be sure you're running the same SSL code as libvirt uses. One random idea is that perhaps the newer GNUTLS in Jaunty has stopped supporting some feature used in your certificates. eg perhaps they finally disabled md5 algorithm for cert signing or similar ideas. certtool may give you info if this is the case
I just verified that our self-signed CA uses MD5 (boo). I'll have to look into whether a SHA CA fixes the problem. I'm using gnutls v2.4.2-6 (on the client side, 2.4.1-1ubuntu0.2 on the server side). There is a changelog here[1]. According to that log: "Verifying untrusted X.509 certificates signed with RSA-MD2 or RSA-MD5 will now fail with a GNUTLS_CERT_INSECURE_ALGORITHM verification output." I'm curious if there is a different problem. Or, perhaps virt-viewer is detecting GNUTLS_CERT_INSECURE_ALGORITHM as GNUTLS_CERT_INVALID ? Either way, we should fix our CA. BTW, will certtool verify certs ala "openssl verify" ? Scott --------- [1] http://changelogs.ubuntu.com/changelogs/pool/main/g/gnutls26/gnutls26_2.4.2-...

BTW, will certtool verify certs ala "openssl verify" ?
$ certtool --verify-chain --infile /etc/pki/CA/cacert.pem |grep Verification Verification output: Verified. I found the verify-chain option but it doesn't like it when I add my x509 client cert.
participants (2)
-
Daniel P. Berrange
-
Scott Beardsley