Dan Smith wrote:
DV> Basically the Fedora-9 tree is frozen, but you should be able
to
DV> push update for it after it has been released
Right, which I definitely plan to do. I'd like the ability to look at
a test report and see "X failures that have been fixed in the tree but
are not fixed on the current platform". That will help indicate when
a refresh to Fedora or a new release is a good idea. Might be a bit
of a lofty goal, though :)
For future releases, would it be a good idea to archive the current
state of the cimtest tree at the same time a release is snapped of
libvirt-cim? This would allow us to preserve the state of the tests and
tie them to a given release.
This is a good point. The Fedora 9 RPM has been unchanged for a
while
now, so none of the changes that you mention are included. I think
that the test development should proceed to track the libvirt-cim
development tree, but perhaps there is a way that we can map FAIL to
XFAIL for older versions of the provider.
Does anyone have a suggestion for a reasonable way we could do this?
Have the test suite check for the libvirt-cim rpm. If it exists, grab
version. Otherwise, assume the providers are the most recent version
upstream.
If a test case needs to be updated due to a provider behavior change,
use an if condition based on the rpm version (or lack there of) to
determine the pass/fail/xfail behavior.
It's not a very elegant idea, but something along these lines might
work. And of course, such an idea wouldn't work on environments that
don't support rpm / that we currently don't build rpms for.
--
Kaitlin Rupert
IBM Linux Technology Center
kaitlin(a)linux.vnet.ibm.com