Dan Smith wrote:
DV> Basically the Fedora-9 tree is frozen, but you should be able to DV> push update for it after it has been released
Right, which I definitely plan to do. I'd like the ability to look at a test report and see "X failures that have been fixed in the tree but are not fixed on the current platform". That will help indicate when a refresh to Fedora or a new release is a good idea. Might be a bit of a lofty goal, though :)
For future releases, would it be a good idea to archive the current state of the cimtest tree at the same time a release is snapped of libvirt-cim? This would allow us to preserve the state of the tests and tie them to a given release.
This is a good point. The Fedora 9 RPM has been unchanged for a while now, so none of the changes that you mention are included. I think that the test development should proceed to track the libvirt-cim development tree, but perhaps there is a way that we can map FAIL to XFAIL for older versions of the provider.
Does anyone have a suggestion for a reasonable way we could do this?
Have the test suite check for the libvirt-cim rpm. If it exists, grab version. Otherwise, assume the providers are the most recent version upstream. If a test case needs to be updated due to a provider behavior change, use an if condition based on the rpm version (or lack there of) to determine the pass/fail/xfail behavior. It's not a very elegant idea, but something along these lines might work. And of course, such an idea wouldn't work on environments that don't support rpm / that we currently don't build rpms for. -- Kaitlin Rupert IBM Linux Technology Center kaitlin@linux.vnet.ibm.com