When the test suites fail, we have a DEBUG_TESTS=1 environment variable which can be set to show the full XML doc or string pair that failed to match. Unfortunately these docs can be quite large, so it is hard to spot the difference between the actual and expected output. So this patch introduces a virtTestDifference(actual, expect) method which will trim the head & tail of the actual & expected strings until the point at which they differ. This gives immediate clear feedback on where the bug is Second, it will enumerate each test condition & pretty print align things a little better. When I apply this to a test which fails, i now see output like this: 19) Xen SEXPR-2-XML fv-usbmouse -> fv-usbmouse ... Expect [localtime'/> <devices> <emulator>/usr/lib64/xen/bin/qemu-dm</emulator> <disk type='file' device='disk'> <driver name='file'/> <source file='/root/foo.img'/> <target dev='hda'/> </disk> <interface type='network] Actual [utc'/> <devices> <emulator>/usr/lib64/xen/bin/qemu-dm</emulator> <disk type='file' device='disk'> <driver name='file'/> <source file='/root/foo.img'/> <target dev='hda'/> </disk> <interface type='bridge] ... FAILED So you can see immediately that they start to differ in the 'localtime' vs 'utc' value, and the last place at which they differ is the network interface which has type of 'network' vs 'bridge'. This has dramatically eased my debugging of tests when adding the serial device options :-) This patch is merely the helper routines. I'll include the actualy test suite changes later. Dan. Index: testutils.h =================================================================== RCS file: /data/cvs/libvirt/tests/testutils.h,v retrieving revision 1.6 diff -u -p -r1.6 testutils.h --- testutils.h 10 Apr 2008 16:53:29 -0000 1.6 +++ testutils.h 18 Apr 2008 00:22:55 -0000 @@ -32,6 +32,11 @@ extern "C" { char **buf, int buflen); + + int virtTestDifference(FILE *stream, + const char *expect, + const char *actual); + #ifdef __cplusplus } #endif Index: testutils.c =================================================================== RCS file: /data/cvs/libvirt/tests/testutils.c,v retrieving revision 1.11 diff -u -p -r1.11 testutils.c --- testutils.c 10 Apr 2008 16:53:29 -0000 1.11 +++ testutils.c 18 Apr 2008 00:22:55 -0000 @@ -19,6 +19,7 @@ #include <sys/stat.h> #include <sys/wait.h> #include <unistd.h> +#include <string.h> #include <fcntl.h> #include <limits.h> #include "testutils.h" @@ -58,6 +59,12 @@ virtTestRun(const char *title, int nloop { int i, ret = 0; double *ts = NULL; + static int counter = 0; + + counter++; + + fprintf(stderr, "%2d) %-65s ... ", counter, title); + fflush(stderr); if (nloops > 1 && (ts = calloc(nloops, sizeof(double)))==NULL) @@ -76,12 +83,12 @@ virtTestRun(const char *title, int nloop } } if (ret == 0 && ts) - fprintf(stderr, "%-50s ... OK [%.5f ms]\n", title, + fprintf(stderr, "OK [%.5f ms]\n", virtTestCountAverage(ts, nloops)); else if (ret == 0) - fprintf(stderr, "%-50s ... OK\n", title); + fprintf(stderr, "OK\n"); else - fprintf(stderr, "%-50s ... FAILED\n", title); + fprintf(stderr, "FAILED\n"); free(ts); return ret; @@ -206,3 +213,57 @@ int virtTestCaptureProgramOutput(const c } } } + + +/** + * @param stream: output stream write to differences to + * @param expect: expected output text + * @param actual: actual output text + * + * Display expected and actual output text, trimmed to + * first and last characters at which differences occur + */ +int virtTestDifference(FILE *stream, + const char *expect, + const char *actual) +{ + const char *expectStart = expect; + const char *expectEnd = expect + (strlen(expect)-1); + const char *actualStart = actual; + const char *actualEnd = actual + (strlen(actual)-1); + + if (getenv("DEBUG_TESTS") == NULL) + return 0; + + /* Skip to first character where they differ */ + while (*expectStart && *actualStart && + *actualStart == *expectStart) { + actualStart++; + expectStart++; + } + + /* Work backwards to last character where they differ */ + while (actualEnd > actualStart && + expectEnd > expectStart && + *actualEnd == *expectEnd) { + actualEnd--; + expectEnd--; + } + + /* Show the trimmed differences */ + fprintf(stream, "\nExpect ["); + if ((expectEnd - expectStart + 1) && + fwrite(expectStart, (expectEnd-expectStart+1), 1, stream) != 1) + return -1; + fprintf(stream, "]\n"); + fprintf(stream, "Actual ["); + if ((actualEnd - actualStart + 1) && + fwrite(actualStart, (actualEnd-actualStart+1), 1, stream) != 1) + return -1; + fprintf(stream, "]\n"); + + /* Pad to line up with test name ... in virTestRun */ + fprintf(stream, " ... "); + + return 0; +} -- |: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|