Re: [libvirt] [PATCH v2 3/7] tests: qemuxml2argv: Add infrastructure for testing with real qemuCaps

On Wed, Apr 18, 2018 at 13:09:52 +0200, Ján Tomko wrote: > On Wed, Apr 18, 2018 at 11:38:43AM +0200, Peter Krempa wrote:

> I do not find DO_TEST_CAPS_NEW that beneficial - if QEMU introduced a > new capability that needs to be handled by libvirt, the code author > should introduce a new DO_TEST_CAPS_VER test for that. In such reason, the author adding the code should fork the test by adding a DO_TEST_CAPS_VER along with the existing DO_TEST_CAPS_NEW and then add the new capability bit. With such approach it will be even visible which options changed.

> Otherwise it would just be checking whether QEMU did not break something > for us. And since the soonest we update capabilities is at the time of Actually no. It will also check that something other will not break: https://www.redhat.com/archives/libvir-list/2018-April/msg01066.html tried to introduce a new property for the memory-backing-file object. This object is used in multiple places. The property is gated by a capability bit. Our tests were not able to catch, that this would add the property to the NVDIMM device which needs to persist the data, which would actually break it.

While I agree that DO_TEST_CAPS_VER is very useful for checking old command line, I think that the true value is also in DO_TEST_CAPS_NEW when used together with DO_TEST_CAPS_VER, so when a new addition would change some tests using DO_TEST_CAPS_NEW we should fork them to cover both recent and old versions. > QEMU freeze, that might be later than needed. > > > With the comment fixed to encourage DO_TEST_CAPS_VER: I can reword it but I disagree that DO_TEST_CAPS_VER should be used primarily.