[libvirt] debugging daemon-conf failure
Ever since I first started playing with libvirt last month, 'make check' has been failing for me on the same test: $ VIR_TEST_DEBUG=2 make -C tests check TESTS=daemon-conf ... 1) corrupted config listen_tls ... OK 2) corrupted config listen_tcp ... OK 3) corrupted config tls_port ... OK 4) corrupted config tcp_port ... OK 5) corrupted config listen_addr ... OK 6) corrupted config mdns_adv ... FAILED 7) corrupted config mdns_name ... FAILED 8) corrupted config unix_sock_group ... FAILED 9) corrupted config unix_sock_ro_perms ... FAILED 10) corrupted config unix_sock_rw_perms ... FAILED 11) corrupted config unix_sock_dir ... FAILED 12) corrupted config auth_unix_ro ... OK 13) corrupted config auth_unix_rw ... OK 14) corrupted config auth_tcp ... OK 15) corrupted config auth_tls ... FAILED 16) corrupted config key_file ... FAILED 17) corrupted config cert_file ... FAILED 18) corrupted config ca_file ... FAILED 19) corrupted config crl_file ... FAILED 20) corrupted config tls_no_verify_certificate ... FAILED 21) corrupted config tls_allowed_dn_list ... FAILED 22) corrupted config sasl_allowed_username_list ... FAILED 23) corrupted config max_clients ... FAILED 24) corrupted config min_workers ... FAILED 25) corrupted config max_workers ... FAILED 26) corrupted config max_requests ... FAILED 27) corrupted config max_client_requests ... FAILED ./daemon-conf: line 81: kill: (3145) - No such process 28) valid config file (sleeping 2 seconds) ... FAILED FAIL: daemon-conf ================== 1 of 1 test failed ================== Any hints on what I should do to help find the root cause for the failure? Am I missing a package? Does the test need to be made more robust to skip if a prereq is missing? -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Eric Blake wrote:
Ever since I first started playing with libvirt last month, 'make check' has been failing for me on the same test:
$ VIR_TEST_DEBUG=2 make -C tests check TESTS=daemon-conf ... 1) corrupted config listen_tls ... OK 2) corrupted config listen_tcp ... OK 3) corrupted config tls_port ... OK 4) corrupted config tcp_port ... OK 5) corrupted config listen_addr ... OK 6) corrupted config mdns_adv ... FAILED 7) corrupted config mdns_name ... FAILED 8) corrupted config unix_sock_group ... FAILED 9) corrupted config unix_sock_ro_perms ... FAILED 10) corrupted config unix_sock_rw_perms ... FAILED 11) corrupted config unix_sock_dir ... FAILED 12) corrupted config auth_unix_ro ... OK 13) corrupted config auth_unix_rw ... OK 14) corrupted config auth_tcp ... OK 15) corrupted config auth_tls ... FAILED 16) corrupted config key_file ... FAILED 17) corrupted config cert_file ... FAILED 18) corrupted config ca_file ... FAILED 19) corrupted config crl_file ... FAILED 20) corrupted config tls_no_verify_certificate ... FAILED 21) corrupted config tls_allowed_dn_list ... FAILED 22) corrupted config sasl_allowed_username_list ... FAILED 23) corrupted config max_clients ... FAILED 24) corrupted config min_workers ... FAILED 25) corrupted config max_workers ... FAILED 26) corrupted config max_requests ... FAILED 27) corrupted config max_client_requests ... FAILED ./daemon-conf: line 81: kill: (3145) - No such process 28) valid config file (sleeping 2 seconds) ... FAILED FAIL: daemon-conf ================== 1 of 1 test failed ==================
Any hints on what I should do to help find the root cause for the failure? Am I missing a package? Does the test need to be made more robust to skip if a prereq is missing?
What type of system are you using? Parts of that test are a little obscure-looking (you can probably blame me for that). It starts with the sample config file and iterates through it corrupting one line at a time. If running libvirtd with each resulting corrupted config file does not yield the expected result, its a failure. A good place to start would be to save one of the failing config files and to invoke libvirtd manually to see exactly what it's printing and thus get a hint as to why the expected diagnostic is missing.
----- "Jim Meyering" <jim@meyering.net> wrote:
Eric Blake wrote:
Ever since I first started playing with libvirt last month, 'make check' has been failing for me on the same test:
$ VIR_TEST_DEBUG=2 make -C tests check TESTS=daemon-conf
What type of system are you using?
This is on a machine that is 4 years old, running F12. $ uname -a Linux eblake 2.6.31.12-174.2.22.fc12.i686 #1 SMP Fri Feb 19 19:26:06 UTC 2010 i686 i686 i386 GNU/Linux
Parts of that test are a little obscure-looking (you can probably blame me for that). It starts with the sample config file and iterates through it corrupting one line at a time. If running libvirtd with each resulting corrupted config file does not yield the expected result, its a failure.
A good place to start would be to save one of the failing config files and to invoke libvirtd manually to see exactly what it's printing and thus get a hint as to why the expected diagnostic is missing.
Or even trying to run successfully. I created the following tmp.conf: listen_tls = 0 listen_tcp = 1 tls_port = "16514" tcp_port = "16509" listen_addr = "192.168.0.1" mdns_adv = 0 mdns_name = "Virtualization Host Joe Demo" unix_sock_group = "libvirt" unix_sock_ro_perms = "0777" unix_sock_rw_perms = "0770" unix_sock_dir = "/home/eblake/libvirt" auth_unix_ro = "none" auth_unix_rw = "none" auth_tcp = "sasl" auth_tls = "none" key_file = "/etc/pki/libvirt/private/serverkey.pem" cert_file = "/etc/pki/libvirt/servercert.pem" ca_file = "/etc/pki/CA/cacert.pem" crl_file = "/etc/pki/CA/crl.pem" tls_no_verify_certificate = 1 tls_allowed_dn_list = ["DN1", "DN2"] sasl_allowed_username_list = ["joe@EXAMPLE.COM", "fred@EXAMPLE.COM" ] max_clients = 20 min_workers = 5 max_workers = 20 max_requests = 20 max_client_requests = 5 log_level = 3 then ran: $ daemon/libvirtd --pid-file=pid-file --config=tmp.conf 16:17:16.754: error : remoteConfigGetAuth:2564 : remoteReadConfigFile: tmp.conf: auth_tcp: unsupported auth sasl Ideas of what to do next? Maybe the test should be reordered to first attempt testing an unmodified conf file, and only if that succeeds, then rip through the list of one-per-line corruptions? Am I missing some setup or packages to get SASL support enabled? -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Eric Blake wrote:
----- "Jim Meyering" <jim@meyering.net> wrote:
Eric Blake wrote:
Ever since I first started playing with libvirt last month, 'make check' has been failing for me on the same test:
$ VIR_TEST_DEBUG=2 make -C tests check TESTS=daemon-conf
What type of system are you using?
This is on a machine that is 4 years old, running F12. $ uname -a Linux eblake 2.6.31.12-174.2.22.fc12.i686 #1 SMP Fri Feb 19 19:26:06 UTC 2010 i686 i686 i386 GNU/Linux
Parts of that test are a little obscure-looking (you can probably blame me for that). It starts with the sample config file and iterates through it corrupting one line at a time. If running libvirtd with each resulting corrupted config file does not yield the expected result, its a failure.
A good place to start would be to save one of the failing config files and to invoke libvirtd manually to see exactly what it's printing and thus get a hint as to why the expected diagnostic is missing.
Or even trying to run successfully. I created the following tmp.conf:
listen_tls = 0 listen_tcp = 1 ...
then ran:
$ daemon/libvirtd --pid-file=pid-file --config=tmp.conf 16:17:16.754: error : remoteConfigGetAuth:2564 : remoteReadConfigFile: tmp.conf: auth_tcp: unsupported auth sasl
Ideas of what to do next? Maybe the test should be reordered to first attempt testing an unmodified conf file, and only if that succeeds, then rip through the list of one-per-line corruptions? Am I missing some setup or packages to get SASL support enabled?
I'll bet you're right. Note this one on the list below: cyrus-sasl-devel Here's the list of packages you'll want to install before configuring. $ grep '^BuildRe' libvirt.spec BuildRequires: python-devel BuildRequires: util-linux BuildRequires: nfs-utils BuildRequires: xen-devel BuildRequires: xmlrpc-c-devel >= 1.14.0 BuildRequires: libxml2-devel BuildRequires: xhtml1-dtds BuildRequires: readline-devel BuildRequires: ncurses-devel BuildRequires: gettext BuildRequires: gnutls-devel BuildRequires: hal-devel BuildRequires: libudev-devel >= 145 BuildRequires: libpciaccess-devel >= 0.10.9 BuildRequires: yajl-devel BuildRequires: avahi-devel BuildRequires: libselinux-devel BuildRequires: dnsmasq BuildRequires: bridge-utils BuildRequires: cyrus-sasl-devel BuildRequires: polkit >= 0.93 BuildRequires: PolicyKit-devel >= 0.6 BuildRequires: util-linux BuildRequires: /usr/bin/qemu-img BuildRequires: /usr/sbin/qcow-create BuildRequires: lvm2 BuildRequires: iscsi-initiator-utils BuildRequires: parted-devel BuildRequires: e2fsprogs-devel BuildRequires: device-mapper BuildRequires: device-mapper-devel BuildRequires: numactl-devel BuildRequires: libcap-ng-devel >= 0.5.0 BuildRequires: libssh2-devel BuildRequires: netcf-devel >= 0.1.4 BuildRequires: gawk It'd be nice if "make check" would fail right away with an explanation (overridable if you promise not to file bugs about resulting test failures ;-) when libvirt has been built without certain essential-for-proper-testing packages. Or maybe a library analog of the existing bootstrap-time check for minimal program version requirements: # Build prerequisites buildreq="\ autoconf 2.59 automake 1.9.6 autopoint - gettext - git 1.5.5 gzip - libtool - perl 5.5 rsync - tar - "
On Wed, Mar 03, 2010 at 06:20:31PM -0500, Eric Blake wrote:
----- "Jim Meyering" <jim@meyering.net> wrote:
Eric Blake wrote:
Ever since I first started playing with libvirt last month, 'make check' has been failing for me on the same test:
$ VIR_TEST_DEBUG=2 make -C tests check TESTS=daemon-conf
What type of system are you using?
This is on a machine that is 4 years old, running F12. $ uname -a Linux eblake 2.6.31.12-174.2.22.fc12.i686 #1 SMP Fri Feb 19 19:26:06 UTC 2010 i686 i686 i386 GNU/Linux
Parts of that test are a little obscure-looking (you can probably blame me for that). It starts with the sample config file and iterates through it corrupting one line at a time. If running libvirtd with each resulting corrupted config file does not yield the expected result, its a failure.
A good place to start would be to save one of the failing config files and to invoke libvirtd manually to see exactly what it's printing and thus get a hint as to why the expected diagnostic is missing.
Or even trying to run successfully. I created the following tmp.conf:
listen_tls = 0 listen_tcp = 1 tls_port = "16514" tcp_port = "16509" listen_addr = "192.168.0.1" mdns_adv = 0 mdns_name = "Virtualization Host Joe Demo" unix_sock_group = "libvirt" unix_sock_ro_perms = "0777" unix_sock_rw_perms = "0770" unix_sock_dir = "/home/eblake/libvirt" auth_unix_ro = "none" auth_unix_rw = "none" auth_tcp = "sasl" auth_tls = "none" key_file = "/etc/pki/libvirt/private/serverkey.pem" cert_file = "/etc/pki/libvirt/servercert.pem" ca_file = "/etc/pki/CA/cacert.pem" crl_file = "/etc/pki/CA/crl.pem" tls_no_verify_certificate = 1 tls_allowed_dn_list = ["DN1", "DN2"] sasl_allowed_username_list = ["joe@EXAMPLE.COM", "fred@EXAMPLE.COM" ] max_clients = 20 min_workers = 5 max_workers = 20 max_requests = 20 max_client_requests = 5 log_level = 3
then ran:
$ daemon/libvirtd --pid-file=pid-file --config=tmp.conf 16:17:16.754: error : remoteConfigGetAuth:2564 : remoteReadConfigFile: tmp.conf: auth_tcp: unsupported auth sasl
Ideas of what to do next? Maybe the test should be reordered to first attempt testing an unmodified conf file, and only if that succeeds, then rip through the list of one-per-line corruptions? Am I missing some setup or packages to get SASL support enabled?
To be honest this whole test is complete overkill for checking the config file. The config file syntax allow 3 data types, integer, string and array. There is no need for us to be running the libvirtd daemon, and munging each possible setting in turn - we just need to test the edge cases for the different data types. This test also completely misses testing of the config file writing code, and things like escaping of quotes, etc. I'd rather we just kill this test and write a better unit test directly against the virConfPtr APIs in src/util/conf.c to validate all the parser edge cases Daniel. -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Daniel P. Berrange wrote: ...
To be honest this whole test is complete overkill for checking the config file. The config file syntax allow 3 data types, integer, string and array. There is no need for us to be running the libvirtd daemon, and munging each possible setting in turn - we just need to test the edge
On the contrary, I think it is useful to get test coverage of as much of the code as possible, and especially of as many *failing* cases as possible. Each of those failures goes through a slightly different error path in the code. When I wrote it initially, it triggered a few problems, and that's why I extended it to cover all of the cases. Since then, the presence of that test has helped me avoid introducing at least one bug, that would have been a double free.
cases for the different data types. This test also completely misses testing of the config file writing code, and things like escaping of quotes, etc. I'd rather we just kill this test and write a better unit test directly against the virConfPtr APIs in src/util/conf.c to validate all the parser edge cases
Please don't remove it until there's a replacement with better coverage.
On 02/26/2010 01:29 PM, Eric Blake wrote:
Ever since I first started playing with libvirt last month, 'make check' has been failing for me on the same test:
$ VIR_TEST_DEBUG=2 make -C tests check TESTS=daemon-conf ... 1) corrupted config listen_tls ... OK 2) corrupted config listen_tcp ... OK 3) corrupted config tls_port ... OK 4) corrupted config tcp_port ... OK 5) corrupted config listen_addr ... OK 6) corrupted config mdns_adv ... FAILED 7) corrupted config mdns_name ... FAILED 8) corrupted config unix_sock_group ... FAILED 9) corrupted config unix_sock_ro_perms ... FAILED 10) corrupted config unix_sock_rw_perms ... FAILED 11) corrupted config unix_sock_dir ... FAILED 12) corrupted config auth_unix_ro ... OK 13) corrupted config auth_unix_rw ... OK 14) corrupted config auth_tcp ... OK 15) corrupted config auth_tls ... FAILED 16) corrupted config key_file ... FAILED 17) corrupted config cert_file ... FAILED 18) corrupted config ca_file ... FAILED 19) corrupted config crl_file ... FAILED 20) corrupted config tls_no_verify_certificate ... FAILED 21) corrupted config tls_allowed_dn_list ... FAILED 22) corrupted config sasl_allowed_username_list ... FAILED 23) corrupted config max_clients ... FAILED 24) corrupted config min_workers ... FAILED 25) corrupted config max_workers ... FAILED 26) corrupted config max_requests ... FAILED 27) corrupted config max_client_requests ... FAILED ./daemon-conf: line 81: kill: (3145) - No such process 28) valid config file (sleeping 2 seconds) ... FAILED FAIL: daemon-conf ================== 1 of 1 test failed ==================
Any hints on what I should do to help find the root cause for the failure? Am I missing a package? Does the test need to be made more robust to skip if a prereq is missing?
Yep, this one also fails for me. I don't think it is lack of a package, though I could be wrong. It's probably worthwhile to dig into the test deeper and see what is going on. -- Chris Lalancette
On 02/26/2010 11:49 AM, Chris Lalancette wrote:
On 02/26/2010 01:29 PM, Eric Blake wrote:
Ever since I first started playing with libvirt last month, 'make check' has been failing for me on the same test:
$ VIR_TEST_DEBUG=2 make -C tests check TESTS=daemon-conf ... 1) corrupted config listen_tls ... OK 2) corrupted config listen_tcp ... OK 3) corrupted config tls_port ... OK 4) corrupted config tcp_port ... OK 5) corrupted config listen_addr ... OK 6) corrupted config mdns_adv ... FAILED
Any hints on what I should do to help find the root cause for the failure? Am I missing a package? Does the test need to be made more robust to skip if a prereq is missing?
Yep, this one also fails for me. I don't think it is lack of a package, though I could be wrong. It's probably worthwhile to dig into the test deeper and see what is going on.
Thanks to migrating to a new machine, I've just had the opportunity to confirm that installing cyrus-sasl-devel, then rerunning configure, was sufficient to get past this failure. But the absence of cyrus-sasl-devel did not stop building the rest of the package. So now the trick is whether to skip this test if the prereq is not met, or whether to make configure more insistent that the package exist. Furthermore, configure barfed on me three separate times, before I got to the point of reproducing this failure; each solved by installing more packages: libxml2-devel gnutls-devel device-mapper-devel Again, it would be nice to have a configure patch that recognizes failure for these mandatory packages with only one failed run, rather than the current three separate configure runs to identify missing prereqs one at a time. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
participants (4)
-
Chris Lalancette -
Daniel P. Berrange -
Eric Blake -
Jim Meyering