[libvirt PATCH 0/3] tests: virtimetest: Skip more tests near year's end
Failed pipeline: https://gitlab.com/libvirt/libvirt/-/pipelines/439750804 Fixed pipeline: https://gitlab.com/abologna/libvirt/-/pipelines/439847251 Andrea Bolognani (3): tests: virtimetest: Move comment tests: virtimetest: Skip more tests near year's end tests: virtimetest: Mention GLib version containing fix tests/virtimetest.c | 58 ++++++++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 30 deletions(-) -- 2.31.1
Will keep things reasonable as we perform further code movements. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/virtimetest.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/tests/virtimetest.c b/tests/virtimetest.c index 5736245722..fffd189612 100644 --- a/tests/virtimetest.c +++ b/tests/virtimetest.c @@ -197,19 +197,18 @@ mymain(void) TEST_LOCALOFFSET("VIR-12:00VID-13:00,1/00:00:00,364/23:59:59", ((13 * 60) + 0) * 60); + /* experiments have shown that the following tests will fail + * during certain hours of Dec 31 or Jan 1 (depending on the + * TZ setting in the shell running the test, but in general + * for a period that apparently starts at 00:00:00 UTC Jan 1 + * and continues for 1 - 2 hours). We've determined this is + * due to our inability to specify a timezone with DST on/off + * settings that make it truly *always* on DST - i.e. it is a + * failing of the test data, *not* of the function we are + * testing. So to test as much as possible, we still run these + * tests, except on Dec 31 and Jan 1. + */ if (!isNearYearEnd()) { - /* experiments have shown that the following tests will fail - * during certain hours of Dec 31 or Jan 1 (depending on the - * TZ setting in the shell running the test, but in general - * for a period that apparently starts at 00:00:00 UTC Jan 1 - * and continues for 1 - 2 hours). We've determined this is - * due to our inability to specify a timezone with DST on/off - * settings that make it truly *always* on DST - i.e. it is a - * failing of the test data, *not* of the function we are - * testing. So to test as much as possible, we still run these - * tests, except on Dec 31 and Jan 1. - */ - TEST_LOCALOFFSET("VIR02:45VID00:45,1/00:00:00,364/23:59:59", -45 * 60); TEST_LOCALOFFSET("VIR05:00VID04:00,1/00:00:00,364/23:59:59", -- 2.31.1
All of these are affected by the same issue as the ones that we're already skipping during this specific time interval. Suggested-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/virtimetest.c | 39 +++++++++++++++++++-------------------- 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/tests/virtimetest.c b/tests/virtimetest.c index fffd189612..5447ba5721 100644 --- a/tests/virtimetest.c +++ b/tests/virtimetest.c @@ -177,26 +177,6 @@ mymain(void) TEST_LOCALOFFSET("VIR-00:30", 30 * 60); TEST_LOCALOFFSET("VIR-01:30", 90 * 60); - /* test DST processing with timezones that always - * have DST in effect; what's more, cover a zone with - * with an unusual DST different than a usual one hour - * - * These tests originally used '0' as the first day, - * but changed to '1' due to GLib GTimeZone parsing bug: - * https://gitlab.gnome.org/GNOME/glib/issues/1999 - * - * Once we depend on a new enough GLib, we can put then - * back to 0 again. - */ - TEST_LOCALOFFSET("VIR-00:30VID,1/00:00:00,364/23:59:59", - ((1 * 60) + 30) * 60); - TEST_LOCALOFFSET("VIR-02:30VID,1/00:00:00,364/23:59:59", - ((3 * 60) + 30) * 60); - TEST_LOCALOFFSET("VIR-02:30VID-04:30,1/00:00:00,364/23:59:59", - ((4 * 60) + 30) * 60); - TEST_LOCALOFFSET("VIR-12:00VID-13:00,1/00:00:00,364/23:59:59", - ((13 * 60) + 0) * 60); - /* experiments have shown that the following tests will fail * during certain hours of Dec 31 or Jan 1 (depending on the * TZ setting in the shell running the test, but in general @@ -209,6 +189,25 @@ mymain(void) * tests, except on Dec 31 and Jan 1. */ if (!isNearYearEnd()) { + /* test DST processing with timezones that always + * have DST in effect; what's more, cover a zone with + * with an unusual DST different than a usual one hour + * + * These tests originally used '0' as the first day, + * but changed to '1' due to GLib GTimeZone parsing bug: + * https://gitlab.gnome.org/GNOME/glib/issues/1999 + * + * Once we depend on a new enough GLib, we can put then + * back to 0 again. + */ + TEST_LOCALOFFSET("VIR-00:30VID,1/00:00:00,364/23:59:59", + ((1 * 60) + 30) * 60); + TEST_LOCALOFFSET("VIR-02:30VID,1/00:00:00,364/23:59:59", + ((3 * 60) + 30) * 60); + TEST_LOCALOFFSET("VIR-02:30VID-04:30,1/00:00:00,364/23:59:59", + ((4 * 60) + 30) * 60); + TEST_LOCALOFFSET("VIR-12:00VID-13:00,1/00:00:00,364/23:59:59", + ((13 * 60) + 0) * 60); TEST_LOCALOFFSET("VIR02:45VID00:45,1/00:00:00,364/23:59:59", -45 * 60); TEST_LOCALOFFSET("VIR05:00VID04:00,1/00:00:00,364/23:59:59", -- 2.31.1
We now know exactly which GLib version we need to depend on for the workaround to no longer be necessary. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/virtimetest.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/virtimetest.c b/tests/virtimetest.c index 5447ba5721..5d0c0717ca 100644 --- a/tests/virtimetest.c +++ b/tests/virtimetest.c @@ -197,8 +197,8 @@ mymain(void) * but changed to '1' due to GLib GTimeZone parsing bug: * https://gitlab.gnome.org/GNOME/glib/issues/1999 * - * Once we depend on a new enough GLib, we can put then - * back to 0 again. + * Once we depend on a new enough GLib (>= 2.63.4), we + * can put then back to 0 again. */ TEST_LOCALOFFSET("VIR-00:30VID,1/00:00:00,364/23:59:59", ((1 * 60) + 30) * 60); -- 2.31.1
On 1/1/22 2:07 PM, Andrea Bolognani wrote:
Failed pipeline: https://gitlab.com/libvirt/libvirt/-/pipelines/439750804 Fixed pipeline: https://gitlab.com/abologna/libvirt/-/pipelines/439847251
Andrea Bolognani (3): tests: virtimetest: Move comment tests: virtimetest: Skip more tests near year's end
Ugh, I had managed to repress the memories of this, and now it's come back :-O
tests: virtimetest: Mention GLib version containing fix
tests/virtimetest.c | 58 ++++++++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 30 deletions(-)
Reviewed-by: Laine Stump <laine@redhat.com> *if* you can explain to me why (at 11:55PM local time on Jan 1) all 28 of virtimetest cases were run by "ninja -C build test", with test 27 & 28 failing, *but* when I just ran build/tests/virtimetest directly, it succeeded - running it under gdb I found that isNearYearEnd() properly returned true, so the final 8 tests aren't run (as you would expect). Not only that, but when I ran "ninja -C build test" at 12:01AM Jan 2, test 28 failed; it also failed when I directly ran virtimetest. For the first problem - is it possible that ninja is setting the timezone to something different during the build? My machine is in EST (UTC -5). For the 2nd problem - maybe we need to widen the definition of "isNearYearEnd()"? If I make the following change then the final tests are skipped by both ninja -C build test and virtimetest directly when the date is 12;14AM Jan 2. Oh, wait. That solves the 1st problem too :-) diff --git a/tests/virtimetest.c b/tests/virtimetest.c index 5d0c0717ca..383a4cfa59 100644 --- a/tests/virtimetest.c +++ b/tests/virtimetest.c @@ -104,7 +104,7 @@ isNearYearEnd(void) g_autoptr(GDateTime) now = g_date_time_new_now_local(); return ((g_date_time_get_month(now) == 1 && - g_date_time_get_day_of_month(now) == 1) || + g_date_time_get_day_of_month(now) <= 2) || (g_date_time_get_month(now) == 12 && g_date_time_get_day_of_month(now) == 31)); } (I was just kidding about the "*if*" above - you have my Reviewed-by: in any case. But unless you want to spend the time figuring out why my tests failed on Jan 1/early Jan 2, I'm going to send the above diff as a patch)
On Sun, Jan 02, 2022 at 12:17:31AM -0500, Laine Stump wrote:
On 1/1/22 2:07 PM, Andrea Bolognani wrote:
tests: virtimetest: Skip more tests near year's end
Ugh, I had managed to repress the memories of this, and now it's come back :-O
Same :)
*if* you can explain to me why (at 11:55PM local time on Jan 1) all 28 of virtimetest cases were run by "ninja -C build test", with test 27 & 28 failing, *but* when I just ran build/tests/virtimetest directly, it succeeded - running it under gdb I found that isNearYearEnd() properly returned true, so the final 8 tests aren't run (as you would expect).
Not only that, but when I ran "ninja -C build test" at 12:01AM Jan 2, test 28 failed; it also failed when I directly ran virtimetest.
For the first problem - is it possible that ninja is setting the timezone to something different during the build? My machine is in EST (UTC -5).
I don't think that's the case: I hacked virtimetest to print out the timezone, and I get the same output whether I run it through 'meson test' (or 'ninja test') or as a standalone program. While doing so, I also realized something that should probably have been obvious: since each call to TEST_LOCALOFFSET() sets the TZ environment variable and invokes tzset(), depending on where in the test program it is called isNearYearEnd() might return a different result. So perhaps we should call isNearYearEnd() once, before the first call to TEST_LOCALOFFSET(), and use that result to decide which tests to skip? Honestly I'm still unclear on the interactions between the data we're testing and the local timezone of the machine we're running tests on. I basically shamelessly cargo-culted the existing approach :) -- Andrea Bolognani / Red Hat / Virtualization
participants (2)
-
Andrea Bolognani -
Laine Stump