Re: [libvirt] libvirt TCK wrapper for autotest review

Hi guys, I was here looking at the autotest wrapper for libvirt TCK and then decided to work on it, as I had the review fresh on my mind. Things that I've worked on: * Fixed some download links, that were already sent to upstream tck and applied (thanks Dan Berrange) * Instead of making all tests output to the same DEBUG log, make them output to separate .tap files on the results directory * Run all tests available for a given item, rather than stopping the test on the first failure * removed capitalization on the wrapper name, since it's project policy * Use os.environ, and some features of the subcommand execution API to execute the tests * Remove usages of error.JobError, as the problems there are more error.TestError, since they are restricted to the libvirt_tck test, not the entire job (in autotest, a job can do more stuff than just a sequence of job.runtest() calls). * Made the error messages more descriptive, with info of all failed tests So, the current output of the tests is like this: $ sudo client/bin/autotest run libvirt_tck 18:29:27 INFO | Writing results to /home/lmr/Code/autotest.lmr/client/results/default 18:29:27 INFO | START ---- ---- timestamp=1329942567 localtime=Feb 22 18:29:27 18:29:27 INFO | START libvirt_tck.domain libvirt_tck.domain timestamp=1329942567 localtime=Feb 22 18:29:27 18:30:19 ERROR| child process failed 18:30:19 INFO | FAIL libvirt_tck.domain libvirt_tck.domain timestamp=1329942619 localtime=Feb 22 18:30:19 FAIL: ['120-disks-stats.t', '205-disk-hotplug-ordering.t'] 18:30:19 INFO | END FAIL libvirt_tck.domain libvirt_tck.domain timestamp=1329942619 localtime=Feb 22 18:30:19 18:30:19 INFO | START libvirt_tck.hooks libvirt_tck.hooks timestamp=1329942619 localtime=Feb 22 18:30:19 18:30:19 ERROR| child process failed 18:30:19 INFO | FAIL libvirt_tck.hooks libvirt_tck.hooks timestamp=1329942619 localtime=Feb 22 18:30:19 FAIL: ['051-daemon-hook.t', '052-domain-hook.t'] 18:30:19 INFO | END FAIL libvirt_tck.hooks libvirt_tck.hooks timestamp=1329942619 localtime=Feb 22 18:30:19 18:30:19 INFO | START libvirt_tck.networks libvirt_tck.networks timestamp=1329942619 localtime=Feb 22 18:30:19 18:30:28 INFO | GOOD libvirt_tck.networks libvirt_tck.networks timestamp=1329942628 localtime=Feb 22 18:30:28 completed successfully 18:30:28 INFO | END GOOD libvirt_tck.networks libvirt_tck.networks timestamp=1329942628 localtime=Feb 22 18:30:28 18:30:28 INFO | START libvirt_tck.nwfilter libvirt_tck.nwfilter timestamp=1329942628 localtime=Feb 22 18:30:28 18:30:32 ERROR| child process failed 18:30:32 INFO | FAIL libvirt_tck.nwfilter libvirt_tck.nwfilter timestamp=1329942632 localtime=Feb 22 18:30:32 FAIL: ['090-install-image.t', '100-ping-still-working.t', '210-no-mac-spoofing.t', '220-no-ip-spoofing.t', '230-no-mac-broadcast.t', '240-no-arp-spoofing.t', '300-vsitype.t'] 18:30:32 INFO | END FAIL libvirt_tck.nwfilter libvirt_tck.nwfilter timestamp=1329942632 localtime=Feb 22 18:30:32 18:30:32 INFO | START libvirt_tck.qemu libvirt_tck.qemu timestamp=1329942632 localtime=Feb 22 18:30:32 18:30:40 ERROR| child process failed 18:30:40 INFO | FAIL libvirt_tck.qemu libvirt_tck.qemu timestamp=1329942640 localtime=Feb 22 18:30:40 FAIL: ['205-qcow2-double-backing-file.t'] 18:30:40 INFO | END FAIL libvirt_tck.qemu libvirt_tck.qemu timestamp=1329942640 localtime=Feb 22 18:30:40 18:30:40 INFO | START libvirt_tck.selinux libvirt_tck.selinux timestamp=1329942640 localtime=Feb 22 18:30:40 18:30:49 ERROR| child process failed 18:30:49 INFO | FAIL libvirt_tck.selinux libvirt_tck.selinux timestamp=1329942649 localtime=Feb 22 18:30:49 FAIL: ['055-dynamic-base-label.t', '100-static-relabel-no.t'] 18:30:49 INFO | END FAIL libvirt_tck.selinux libvirt_tck.selinux timestamp=1329942649 localtime=Feb 22 18:30:49 18:30:49 INFO | START libvirt_tck.storage libvirt_tck.storage timestamp=1329942649 localtime=Feb 22 18:30:49 18:31:24 INFO | GOOD libvirt_tck.storage libvirt_tck.storage timestamp=1329942684 localtime=Feb 22 18:31:24 completed successfully 18:31:24 INFO | END GOOD libvirt_tck.storage libvirt_tck.storage timestamp=1329942684 localtime=Feb 22 18:31:24 18:31:24 INFO | END GOOD ---- ---- timestamp=1329942684 localtime=Feb 22 18:31:24