[libvirt PATCH v2 0/8] meson: Improve handling of tests
Test pipeline: https://gitlab.com/abologna/libvirt/-/pipelines/1049326523 Changes from [v1] * fix test suite on macOS and ensure it is run as part of the pipeline (with the previous version it would just always fail); * disable a couple of tests (check-html, check-html-references) that I had missed the first time around. [v1] https://listman.redhat.com/archives/libvir-list/2023-October/242491.html Andrea Bolognani (8): tests: Fix some test cases on macOS ci: Disable optimizations on macOS meson: Do less when not building from git meson: Move all handling of test options together meson: Handle -Dtests=enabled with Clang meson: Make -Dexpensive_tests depend on -Dtests meson: Disable all tests when tests are disabled meson: Rename build_tests -> tests_enabled build-aux/meson.build | 99 +++++---- ci/cirrus/build.yml | 2 +- docs/html/meson.build | 20 +- docs/meson.build | 24 ++- meson.build | 75 ++++--- meson_options.txt | 2 +- src/access/meson.build | 16 +- src/meson.build | 204 +++++++++--------- .../disk-source-fd.x86_64-latest.args | 10 +- .../disk-vhostvdpa.x86_64-latest.args | 2 +- tests/qemuxml2argvtest.c | 8 +- 11 files changed, 242 insertions(+), 220 deletions(-) -- 2.41.0
Test cases that depend on duplicating fds are using fairly big values as targets. This works fine on Linux, where RLIMIT_NOFILE is 1024 by default, but fails on macOS which uses 256 as the default. Decrease the values so that they're valid across all platforms. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- .../qemuxml2argvdata/disk-source-fd.x86_64-latest.args | 10 +++++----- .../qemuxml2argvdata/disk-vhostvdpa.x86_64-latest.args | 2 +- tests/qemuxml2argvtest.c | 8 ++++---- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/tests/qemuxml2argvdata/disk-source-fd.x86_64-latest.args b/tests/qemuxml2argvdata/disk-source-fd.x86_64-latest.args index 9d8109a8f4..1341b7d032 100644 --- a/tests/qemuxml2argvdata/disk-source-fd.x86_64-latest.args +++ b/tests/qemuxml2argvdata/disk-source-fd.x86_64-latest.args @@ -27,18 +27,18 @@ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ -no-shutdown \ -boot strict=on \ -device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ --add-fd set=2,fd=700,opaque=libvirt-4-storage0 \ --add-fd set=2,fd=705,opaque=libvirt-4-storage1 \ +-add-fd set=2,fd=200,opaque=libvirt-4-storage0 \ +-add-fd set=2,fd=205,opaque=libvirt-4-storage1 \ -blockdev '{"driver":"file","filename":"/dev/fdset/2","node-name":"libvirt-4-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"qcow2","file":"libvirt-4-storage"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"libvirt-4-format","id":"virtio-disk4","bootindex":1}' \ -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/rhel7.1484071876","node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"qcow2","file":"libvirt-3-storage","backing":null}' \ --add-fd set=1,fd=777,opaque=libvirt-2-storage0 \ --add-fd set=1,fd=778,opaque=libvirt-2-storage1 \ +-add-fd set=1,fd=247,opaque=libvirt-2-storage0 \ +-add-fd set=1,fd=248,opaque=libvirt-2-storage1 \ -blockdev '{"driver":"file","filename":"/dev/fdset/1","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"qcow2","file":"libvirt-2-storage","backing":"libvirt-3-format"}' \ --add-fd set=0,fd=704,opaque=libvirt-1-storage0 \ +-add-fd set=0,fd=204,opaque=libvirt-1-storage0 \ -blockdev '{"driver":"file","filename":"/dev/fdset/0","node-name":"libvirt-1-storage","read-only":false,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-2-format"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"libvirt-1-format","id":"virtio-disk5"}' \ diff --git a/tests/qemuxml2argvdata/disk-vhostvdpa.x86_64-latest.args b/tests/qemuxml2argvdata/disk-vhostvdpa.x86_64-latest.args index b987455ee4..27035184ad 100644 --- a/tests/qemuxml2argvdata/disk-vhostvdpa.x86_64-latest.args +++ b/tests/qemuxml2argvdata/disk-vhostvdpa.x86_64-latest.args @@ -27,7 +27,7 @@ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ -no-shutdown \ -boot strict=on \ -device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ --add-fd set=0,fd=801,opaque=libvirt-1-storage-vdpa \ +-add-fd set=0,fd=201,opaque=libvirt-1-storage-vdpa \ -blockdev '{"driver":"virtio-blk-vhost-vdpa","path":"/dev/fdset/0","node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1,"write-cache":"on"}' \ diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 48058cb924..4fda68a4ce 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1186,7 +1186,7 @@ mymain(void) DO_TEST_CAPS_LATEST("disk-vhostuser-numa"); DO_TEST_CAPS_LATEST("disk-vhostuser"); DO_TEST_CAPS_ARCH_LATEST_FULL("disk-vhostvdpa", "x86_64", - ARG_VDPA_FD, "/dev/vhost-vdpa-0", 801); + ARG_VDPA_FD, "/dev/vhost-vdpa-0", 201); DO_TEST_CAPS_LATEST_PARSE_ERROR("disk-device-lun-type-invalid"); DO_TEST_CAPS_LATEST_PARSE_ERROR("disk-attaching-partition-nosupport"); DO_TEST_CAPS_LATEST("disk-usb-device"); @@ -1226,9 +1226,9 @@ mymain(void) DO_TEST_CAPS_LATEST("disk-backing-chains-index"); DO_TEST_CAPS_LATEST("disk-backing-chains-noindex"); DO_TEST_CAPS_ARCH_LATEST_FULL("disk-source-fd", "x86_64", - ARG_FD_GROUP, "testgroup2", 2, 700, 705, - ARG_FD_GROUP, "testgroup5", 1, 704, - ARG_FD_GROUP, "testgroup6", 2, 777, 778); + ARG_FD_GROUP, "testgroup2", 2, 200, 205, + ARG_FD_GROUP, "testgroup5", 1, 204, + ARG_FD_GROUP, "testgroup6", 2, 247, 248); DO_TEST_CAPS_LATEST("disk-slices"); DO_TEST_CAPS_LATEST("disk-rotation"); -- 2.41.0
Clang can be too aggressive at optimizations, which can end up breaking our test suite. See f9f5ab57189b for details. As a result of this, since 7944700b4037 we are automatically disabling tests when Clang is used unless it supports the -fsemantic-interposition compiler flag. Since the version of Clang included in macOS doesn't support that compiler flag, we end up always disabling the test suite on that platform. This is already far from ideal, considering that it was just last year when we finally managed to get the test suite to successfully pass on macOS, and it would be a real shame if the situation regressed again. With the upcoming changes, which will turn running 'meson test' into a hard failure if tests are disabled, this behavior will result in every single pipeline failing. Work around the problem the only way we can: disabling optimizations entirely for the macOS CI jobs. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- ci/cirrus/build.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/cirrus/build.yml b/ci/cirrus/build.yml index 519e5ae144..60ac90eee0 100644 --- a/ci/cirrus/build.yml +++ b/ci/cirrus/build.yml @@ -24,7 +24,7 @@ build_task: - git fetch origin "${CI_MERGE_REQUEST_REF_PATH:-$CI_COMMIT_REF_NAME}" - git reset --hard "$CI_COMMIT_SHA" build_script: - - meson setup build + - if test "$(uname)" = "Darwin"; then meson setup build --optimization 0; else meson setup build; fi - meson dist -C build --no-tests - meson compile -C build - meson test -C build --no-suite syntax-check --print-errorlogs || (cat ~/Library/Logs/DiagnosticReports/*.crash && exit 1) -- 2.41.0
As explained in the comment, the syntax-check machinery uses git to figure out the list of files it should operate on, so we can only enable it when building from git. Despite only registering the various tests with meson in that case, however, we unconditionally perform a bunch of preparation that is only useful for the purpose of registering and running the tests. If we're not going to do that, we can skip a few steps and save a bit of time. Best viewed with 'git show -w'. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com> --- build-aux/meson.build | 99 +++++++++++++++++++++---------------------- 1 file changed, 49 insertions(+), 50 deletions(-) diff --git a/build-aux/meson.build b/build-aux/meson.build index 16d085505d..b5d88a4c44 100644 --- a/build-aux/meson.build +++ b/build-aux/meson.build @@ -1,63 +1,62 @@ -flake8_path = '' -if flake8_prog.found() - flake8_path = flake8_prog.full_path() -endif +# Skip syntax-check if not building from git because we get the list of files +# to check using git commands and it fails if we are not in git repository. +if git + flake8_path = '' + if flake8_prog.found() + flake8_path = flake8_prog.full_path() + endif -if host_machine.system() == 'freebsd' or host_machine.system() == 'darwin' - make_prog = find_program('gmake') - sed_prog = find_program('gsed') -else - make_prog = find_program('make') - sed_prog = find_program('sed') -endif + if host_machine.system() == 'freebsd' or host_machine.system() == 'darwin' + make_prog = find_program('gmake') + sed_prog = find_program('gsed') + else + make_prog = find_program('make') + sed_prog = find_program('sed') + endif -if host_machine.system() == 'freebsd' - grep_prog = find_program('grep') - grep_cmd = run_command(grep_prog, '--version', check: true) - if grep_cmd.stdout().startswith('grep (BSD grep') - grep_prog = find_program('/usr/local/bin/grep', required: false) - if not grep_prog.found() - error('GNU grep not found') + if host_machine.system() == 'freebsd' + grep_prog = find_program('grep') + grep_cmd = run_command(grep_prog, '--version', check: true) + if grep_cmd.stdout().startswith('grep (BSD grep') + grep_prog = find_program('/usr/local/bin/grep', required: false) + if not grep_prog.found() + error('GNU grep not found') + endif endif + elif host_machine.system() == 'darwin' + grep_prog = find_program('ggrep') + else + grep_prog = find_program('grep') endif -elif host_machine.system() == 'darwin' - grep_prog = find_program('ggrep') -else - grep_prog = find_program('grep') -endif - -awk_prog = find_program('awk') -syntax_check_conf = configuration_data({ - 'top_srcdir': meson.project_source_root(), - 'top_builddir': meson.project_build_root(), - 'flake8_path': flake8_path, - 'runutf8': ' '.join(runutf8), - 'PYTHON3': python3_prog.full_path(), - 'GREP': grep_prog.full_path(), - 'SED': sed_prog.full_path(), - 'AWK': awk_prog.full_path(), -}) + awk_prog = find_program('awk') -configure_file( - input: 'Makefile.in', - output: '@BASENAME@', - configuration: syntax_check_conf, -) + syntax_check_conf = configuration_data({ + 'top_srcdir': meson.project_source_root(), + 'top_builddir': meson.project_build_root(), + 'flake8_path': flake8_path, + 'runutf8': ' '.join(runutf8), + 'PYTHON3': python3_prog.full_path(), + 'GREP': grep_prog.full_path(), + 'SED': sed_prog.full_path(), + 'AWK': awk_prog.full_path(), + }) -rc = run_command( - 'sed', '-n', - 's/^sc_\\([a-zA-Z0-9_-]*\\):.*/\\1/p', - meson.current_source_dir() / 'syntax-check.mk', - check: true, -) + configure_file( + input: 'Makefile.in', + output: '@BASENAME@', + configuration: syntax_check_conf, + ) -sc_tests = rc.stdout().strip().split() + rc = run_command( + 'sed', '-n', + 's/^sc_\\([a-zA-Z0-9_-]*\\):.*/\\1/p', + meson.current_source_dir() / 'syntax-check.mk', + check: true, + ) + sc_tests = rc.stdout().strip().split() -# Skip syntax-check if not building from git because we get the list of files -# to check using git commands and it fails if we are not in git repository. -if git foreach target : sc_tests test( target, -- 2.41.0
This will make future patches nicer. Note that we need to handle these somewhat late because of the dependency on information about the compiler and the flags it supports. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com> --- meson.build | 57 +++++++++++++++++++++++++++-------------------------- 1 file changed, 29 insertions(+), 28 deletions(-) diff --git a/meson.build b/meson.build index 47748febb8..a372f99c21 100644 --- a/meson.build +++ b/meson.build @@ -151,23 +151,6 @@ if packager_version != '' endif -# test options - -if get_option('expensive_tests').auto() - use_expensive_tests = not git -else - use_expensive_tests = get_option('expensive_tests').enabled() -endif - -coverage_flags = [] -if get_option('test_coverage') - coverage_flags = [ - '-fprofile-arcs', - '-ftest-coverage', - ] -endif - - # Add RPATH information when building for a non-standard prefix, or # when explicitly requested to do so @@ -2041,6 +2024,35 @@ endif conf.set_quoted('TLS_PRIORITY', get_option('tls_priority')) + +# test options + +build_tests = [ not get_option('tests').disabled() ] +if build_tests[0] and \ + cc.get_id() == 'clang' and \ + not supported_cc_flags.contains('-fsemantic-interposition') \ + and get_option('optimization') != '0' + # If CLang doesn't support -fsemantic-interposition then our + # mocking doesn't work. The best we can do is to not run the + # test suite. + build_tests = [ false, '!!! Forcibly disabling tests because CLang lacks -fsemantic-interposition. Update CLang or disable optimization !!!' ] +endif + +if get_option('expensive_tests').auto() + use_expensive_tests = not git +else + use_expensive_tests = get_option('expensive_tests').enabled() +endif + +coverage_flags = [] +if get_option('test_coverage') + coverage_flags = [ + '-fprofile-arcs', + '-ftest-coverage', + ] +endif + + # Various definitions # Python3 < 3.7 treats the C locale as 7-bit only. We must force env vars so @@ -2064,17 +2076,6 @@ subdir('src') subdir('tools') -build_tests = [ not get_option('tests').disabled() ] -if build_tests[0] and \ - cc.get_id() == 'clang' and \ - not supported_cc_flags.contains('-fsemantic-interposition') \ - and get_option('optimization') != '0' - # If CLang doesn't support -fsemantic-interposition then our - # mocking doesn't work. The best we can do is to not run the - # test suite. - build_tests = [ false, '!!! Forcibly disabling tests because CLang lacks -fsemantic-interposition. Update CLang or disable optimization !!!' ] -endif - if build_tests[0] subdir('tests') endif -- 2.41.0
There are some cases in which we automatically disable tests when using Clang as the compiler. If the user has explicitly asked for tests to be enabled, however, we should error out instead of silently disabling things. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com> --- meson.build | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/meson.build b/meson.build index a372f99c21..ea8ee84ba0 100644 --- a/meson.build +++ b/meson.build @@ -2035,7 +2035,11 @@ if build_tests[0] and \ # If CLang doesn't support -fsemantic-interposition then our # mocking doesn't work. The best we can do is to not run the # test suite. - build_tests = [ false, '!!! Forcibly disabling tests because CLang lacks -fsemantic-interposition. Update CLang or disable optimization !!!' ] + msg = 'Forcibly disabling tests because CLang lacks -fsemantic-interposition. Update CLang or disable optimization' + if get_option('tests').enabled() + error(msg) + endif + build_tests = [ false, '!!! @0@ !!!'.format(msg) ] endif if get_option('expensive_tests').auto() -- 2.41.0
It only makes sense to enable expensive tests when tests are enabled. Disallow invalid configurations. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com> --- meson.build | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/meson.build b/meson.build index ea8ee84ba0..33027404f6 100644 --- a/meson.build +++ b/meson.build @@ -2043,9 +2043,12 @@ if build_tests[0] and \ endif if get_option('expensive_tests').auto() - use_expensive_tests = not git + use_expensive_tests = not git and build_tests[0] else use_expensive_tests = get_option('expensive_tests').enabled() + if use_expensive_tests and not build_tests[0] + error('cannot enable expensive tests when tests are disabled') + endif endif coverage_flags = [] -- 2.41.0
Currently, passing -Dtests=disabled only disables a subset of tests: those that are written in C and thus require compilation. Other tests, such as the syntax-check ones and those that are implemented as scripts, are always enabled. There's a potentially dangerous consequence of this behavior: when tests are disabled, 'meson test' will succeed as if they had been enabled. No indication of this will be shown, so the user will likely make the reasonable assumption that everything is fine when in fact the significantly reduced coverage might be hiding failures. To solve this issues, disable *all* tests when asked to do so, and inject an intentionally failing test to ensure that 'meson test' doesn't succeed. Best viewed with 'git show -w'. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- build-aux/meson.build | 2 +- docs/html/meson.build | 20 ++-- docs/meson.build | 24 ++--- meson.build | 7 ++ src/access/meson.build | 16 ++-- src/meson.build | 204 +++++++++++++++++++++-------------------- 6 files changed, 144 insertions(+), 129 deletions(-) diff --git a/build-aux/meson.build b/build-aux/meson.build index b5d88a4c44..84405c5ec8 100644 --- a/build-aux/meson.build +++ b/build-aux/meson.build @@ -1,6 +1,6 @@ # Skip syntax-check if not building from git because we get the list of files # to check using git commands and it fails if we are not in git repository. -if git +if git and build_tests[0] flake8_path = '' if flake8_prog.found() flake8_path = flake8_prog.full_path() diff --git a/docs/html/meson.build b/docs/html/meson.build index c0a666f4e1..b4e81f8501 100644 --- a/docs/html/meson.build +++ b/docs/html/meson.build @@ -119,12 +119,14 @@ html_xslt_gen = [] # --- end of XSLT processing --- -test( - 'check-html', - xmllint_prog, - args: [ - '--nonet', '--noout', docs_html_paths, - ], - depends: docs_html_dep, - suite: 'script' -) +if build_tests[0] + test( + 'check-html', + xmllint_prog, + args: [ + '--nonet', '--noout', docs_html_paths, + ], + depends: docs_html_dep, + suite: 'script' + ) +endif diff --git a/docs/meson.build b/docs/meson.build index b20ef1c926..52763a8597 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -351,14 +351,16 @@ run_target( depends: install_web_deps, ) -test( - 'check-html-references', - python3_prog, - args: [ - check_html_references_prog.full_path(), - '--webroot', - meson.project_build_root() / 'docs' - ], - env: runutf8, - suite: 'script' -) +if build_tests[0] + test( + 'check-html-references', + python3_prog, + args: [ + check_html_references_prog.full_path(), + '--webroot', + meson.project_build_root() / 'docs' + ], + env: runutf8, + suite: 'script' + ) +endif diff --git a/meson.build b/meson.build index 33027404f6..b30150d605 100644 --- a/meson.build +++ b/meson.build @@ -2085,6 +2085,13 @@ subdir('tools') if build_tests[0] subdir('tests') +else + # Ensure that 'meson test' fails when tests are disabled, as opposed to + # misleadingly succeeding at doing absolutely nothing + test( + 'tests-are-disabled', + python3_prog, args: [ '-c', 'raise Exception("tests are disabled")' ], + ) endif subdir('examples') diff --git a/src/access/meson.build b/src/access/meson.build index e65f17c0a2..6ca953c932 100644 --- a/src/access/meson.build +++ b/src/access/meson.build @@ -105,10 +105,12 @@ access_dep = declare_dependency( generated_sym_files += access_gen_sym -test( - 'check-aclperms', - python3_prog, - args: [ check_aclperms_prog.full_path(), access_perm_h, files('viraccessperm.c') ], - env: runutf8, - suite: 'script' -) +if build_tests[0] + test( + 'check-aclperms', + python3_prog, + args: [ check_aclperms_prog.full_path(), access_perm_h, files('viraccessperm.c') ], + env: runutf8, + suite: 'script' + ) +endif diff --git a/src/meson.build b/src/meson.build index 43146fe3c3..e25b3e5980 100644 --- a/src/meson.build +++ b/src/meson.build @@ -946,121 +946,123 @@ meson.add_install_script( # Check driver files -if host_machine.system() == 'linux' - test( - 'check-symfile', - python3_prog, - args: [ check_symfile_prog.full_path(), libvirt_syms, libvirt_lib ], - env: runutf8, - suite: 'script' - ) - - if conf.has('WITH_REMOTE') +if build_tests[0] + if host_machine.system() == 'linux' test( - 'check-admin-symfile', + 'check-symfile', python3_prog, - args: [ check_symfile_prog.full_path(), libvirt_admin_syms, libvirt_admin_lib ], + args: [ check_symfile_prog.full_path(), libvirt_syms, libvirt_lib ], env: runutf8, suite: 'script' ) + + if conf.has('WITH_REMOTE') + test( + 'check-admin-symfile', + python3_prog, + args: [ check_symfile_prog.full_path(), libvirt_admin_syms, libvirt_admin_lib ], + env: runutf8, + suite: 'script' + ) + endif endif -endif -test( - 'check-symsorting', - python3_prog, - args: [ - check_symsorting_prog.full_path(), - meson.current_source_dir(), - files(sym_files, used_sym_files), - ], - env: runutf8, - suite: 'script' -) + test( + 'check-symsorting', + python3_prog, + args: [ + check_symsorting_prog.full_path(), + meson.current_source_dir(), + files(sym_files, used_sym_files), + ], + env: runutf8, + suite: 'script' + ) -test( - 'check-admin-symsorting', - python3_prog, - args: [ - check_symsorting_prog.full_path(), - meson.current_source_dir(), - libvirt_admin_private_syms, - ], - env: runutf8, - suite: 'script' -) + test( + 'check-admin-symsorting', + python3_prog, + args: [ + check_symsorting_prog.full_path(), + meson.current_source_dir(), + libvirt_admin_private_syms, + ], + env: runutf8, + suite: 'script' + ) -test( - 'check-drivername', - python3_prog, - args: [ - check_drivername_prog.full_path(), files(driver_headers), - files('libvirt_public.syms'), libvirt_qemu_syms, libvirt_lxc_syms, - ], - env: runutf8, - suite: 'script' -) + test( + 'check-drivername', + python3_prog, + args: [ + check_drivername_prog.full_path(), files(driver_headers), + files('libvirt_public.syms'), libvirt_qemu_syms, libvirt_lxc_syms, + ], + env: runutf8, + suite: 'script' + ) -test( - 'check-admin-drivername', - python3_prog, - args: [ - check_drivername_prog.full_path(), libvirt_admin_public_syms, - ], - env: runutf8, - suite: 'script' -) + test( + 'check-admin-drivername', + python3_prog, + args: [ + check_drivername_prog.full_path(), libvirt_admin_public_syms, + ], + env: runutf8, + suite: 'script' + ) -test( - 'check-driverimpls', - python3_prog, - args: [ check_driverimpls_prog.full_path(), driver_source_files ], - env: runutf8, - suite: 'script' -) + test( + 'check-driverimpls', + python3_prog, + args: [ check_driverimpls_prog.full_path(), driver_source_files ], + env: runutf8, + suite: 'script' + ) -test( - 'check-aclrules', - python3_prog, - args: [ check_aclrules_prog.full_path(), files('remote/remote_protocol.x'), stateful_driver_source_files ], - env: runutf8, - suite: 'script' -) + test( + 'check-aclrules', + python3_prog, + args: [ check_aclrules_prog.full_path(), files('remote/remote_protocol.x'), stateful_driver_source_files ], + env: runutf8, + suite: 'script' + ) -if augparse_prog.found() - foreach data : augeas_test_data - test( - 'check-augeas-@0@'.format(data['name']), - augparse_prog, - args: [ - '-I', data['srcdir'], - '-I', data['builddir'], - data['file'].full_path(), - ], - suite: 'script' - ) - endforeach -endif + if augparse_prog.found() + foreach data : augeas_test_data + test( + 'check-augeas-@0@'.format(data['name']), + augparse_prog, + args: [ + '-I', data['srcdir'], + '-I', data['builddir'], + data['file'].full_path(), + ], + suite: 'script' + ) + endforeach + endif -if pdwtags_prog.found() and cc.get_id() != 'clang' - foreach proto : check_protocols - lib = proto['lib'] - test( - 'check-@0@'.format(proto['name']), - python3_prog, - args: [ - check_remote_protocol_prog.full_path(), - proto['name'], - lib.name(), - lib.full_path(), - pdwtags_prog.full_path(), - files('@0@-structs'.format(proto['name'])), - ], - env: runutf8, - depends: [ lib ], - suite: 'script' - ) - endforeach + if pdwtags_prog.found() and cc.get_id() != 'clang' + foreach proto : check_protocols + lib = proto['lib'] + test( + 'check-@0@'.format(proto['name']), + python3_prog, + args: [ + check_remote_protocol_prog.full_path(), + proto['name'], + lib.name(), + lib.full_path(), + pdwtags_prog.full_path(), + files('@0@-structs'.format(proto['name'])), + ], + env: runutf8, + depends: [ lib ], + suite: 'script' + ) + endforeach + endif endif # configure pkg-config files for run script -- 2.41.0
Given that this variable now controls not just whether C tests are built, but also whether any test at all is executed, the new name is more appropriate. Update the description for the corresponding meson option accordingly. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com> --- build-aux/meson.build | 2 +- docs/html/meson.build | 2 +- docs/meson.build | 2 +- meson.build | 14 +++++++------- meson_options.txt | 2 +- src/access/meson.build | 2 +- src/meson.build | 2 +- 7 files changed, 13 insertions(+), 13 deletions(-) diff --git a/build-aux/meson.build b/build-aux/meson.build index 84405c5ec8..f96d46c445 100644 --- a/build-aux/meson.build +++ b/build-aux/meson.build @@ -1,6 +1,6 @@ # Skip syntax-check if not building from git because we get the list of files # to check using git commands and it fails if we are not in git repository. -if git and build_tests[0] +if git and tests_enabled[0] flake8_path = '' if flake8_prog.found() flake8_path = flake8_prog.full_path() diff --git a/docs/html/meson.build b/docs/html/meson.build index b4e81f8501..e2758ed177 100644 --- a/docs/html/meson.build +++ b/docs/html/meson.build @@ -119,7 +119,7 @@ html_xslt_gen = [] # --- end of XSLT processing --- -if build_tests[0] +if tests_enabled[0] test( 'check-html', xmllint_prog, diff --git a/docs/meson.build b/docs/meson.build index 52763a8597..87d728213c 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -351,7 +351,7 @@ run_target( depends: install_web_deps, ) -if build_tests[0] +if tests_enabled[0] test( 'check-html-references', python3_prog, diff --git a/meson.build b/meson.build index b30150d605..e51ba9231e 100644 --- a/meson.build +++ b/meson.build @@ -2027,8 +2027,8 @@ conf.set_quoted('TLS_PRIORITY', get_option('tls_priority')) # test options -build_tests = [ not get_option('tests').disabled() ] -if build_tests[0] and \ +tests_enabled = [ not get_option('tests').disabled() ] +if tests_enabled[0] and \ cc.get_id() == 'clang' and \ not supported_cc_flags.contains('-fsemantic-interposition') \ and get_option('optimization') != '0' @@ -2039,14 +2039,14 @@ if build_tests[0] and \ if get_option('tests').enabled() error(msg) endif - build_tests = [ false, '!!! @0@ !!!'.format(msg) ] + tests_enabled = [ false, '!!! @0@ !!!'.format(msg) ] endif if get_option('expensive_tests').auto() - use_expensive_tests = not git and build_tests[0] + use_expensive_tests = not git and tests_enabled[0] else use_expensive_tests = get_option('expensive_tests').enabled() - if use_expensive_tests and not build_tests[0] + if use_expensive_tests and not tests_enabled[0] error('cannot enable expensive tests when tests are disabled') endif endif @@ -2083,7 +2083,7 @@ subdir('src') subdir('tools') -if build_tests[0] +if tests_enabled[0] subdir('tests') else # Ensure that 'meson test' fails when tests are disabled, as opposed to @@ -2307,7 +2307,7 @@ endif misc_summary = { 'Warning Flags': supported_cc_flags, 'docs': gen_docs, - 'tests': build_tests, + 'tests': tests_enabled, 'DTrace': conf.has('WITH_DTRACE_PROBES'), 'firewalld': conf.has('WITH_FIREWALLD'), 'firewalld-zone': conf.has('WITH_FIREWALLD_ZONE'), diff --git a/meson_options.txt b/meson_options.txt index 7c428a9eb0..a0928102bf 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -10,7 +10,7 @@ option('git_werror', type: 'feature', value: 'auto', description: 'use -Werror i option('rpath', type: 'feature', value: 'auto', description: 'whether to include rpath information in installed binaries and libraries') option('docdir', type: 'string', value: '', description: 'documentation installation directory') option('docs', type: 'feature', value: 'auto', description: 'whether to generate documentation') -option('tests', type: 'feature', value: 'auto', description: 'whether to build tests') +option('tests', type: 'feature', value: 'auto', description: 'whether to build and run tests') # build dependencies options diff --git a/src/access/meson.build b/src/access/meson.build index 6ca953c932..fc5ba5b342 100644 --- a/src/access/meson.build +++ b/src/access/meson.build @@ -105,7 +105,7 @@ access_dep = declare_dependency( generated_sym_files += access_gen_sym -if build_tests[0] +if tests_enabled[0] test( 'check-aclperms', python3_prog, diff --git a/src/meson.build b/src/meson.build index e25b3e5980..5fc4d03b4a 100644 --- a/src/meson.build +++ b/src/meson.build @@ -946,7 +946,7 @@ meson.add_install_script( # Check driver files -if build_tests[0] +if tests_enabled[0] if host_machine.system() == 'linux' test( 'check-symfile', -- 2.41.0
On Wed, Oct 25, 2023 at 07:06:53PM +0200, Andrea Bolognani wrote:
Test pipeline: https://gitlab.com/abologna/libvirt/-/pipelines/1049326523
Changes from [v1]
* fix test suite on macOS and ensure it is run as part of the pipeline (with the previous version it would just always fail);
* disable a couple of tests (check-html, check-html-references) that I had missed the first time around.
[v1] https://listman.redhat.com/archives/libvir-list/2023-October/242491.html
Andrea Bolognani (8): tests: Fix some test cases on macOS ci: Disable optimizations on macOS meson: Do less when not building from git meson: Move all handling of test options together meson: Handle -Dtests=enabled with Clang meson: Make -Dexpensive_tests depend on -Dtests meson: Disable all tests when tests are disabled meson: Rename build_tests -> tests_enabled
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
build-aux/meson.build | 99 +++++---- ci/cirrus/build.yml | 2 +- docs/html/meson.build | 20 +- docs/meson.build | 24 ++- meson.build | 75 ++++--- meson_options.txt | 2 +- src/access/meson.build | 16 +- src/meson.build | 204 +++++++++--------- .../disk-source-fd.x86_64-latest.args | 10 +- .../disk-vhostvdpa.x86_64-latest.args | 2 +- tests/qemuxml2argvtest.c | 8 +- 11 files changed, 242 insertions(+), 220 deletions(-)
-- 2.41.0
participants (2)
-
Andrea Bolognani -
Martin Kletzander