[PATCH 00/35] vsh: Fix handling of commands and help - part 3 (virsh testing overhaul)

Part 3 was supposed to be the refactor of the command parser but since I wanted to add few tests I've noticed that there's a lot of old cruft and many tests are skipped. This series refactors virshtest and optimizes it to run multiple commands with one virsh instance. Doing this allows us to do more testing in the same run time allowing us to reduce the amount of "expensive" tests. Further down this removes all of the 'shell' test infra which was used for virsh. Peter Krempa (35): virshtest: Prepare for testing against output files virshtest: Allow to test failure of commands virshtest: Filter multiple occurences of string to drop in testFilterLine virshtest: Add support for testing commands read from input file and adapt alias tests virshtest: Adapt 'echo --split' tests to DO_TEST_SCRIPT virshtest: Drop some redundant 'echo' cases virshtest: Adapt some 'escaping' tests via DO_TEST_SCRIPT vrishtest: Prepare for simpler testing - echo tests virshtest: Test against output files ("echo" tests) virshtest: Drop string form of expected output data of "echo" tests virshtest: Replace list and nodeinfo tests by equivalent VIR_TEST_SCRIPT variant virshtest: Adapt tests for domain id lookup and state query to DO_TEST_SCRIPT virshtest: Adapt 'blkiotune' tests to DO_TEST_SCRIPT virshtest: Adapt 'iothread' tests to DO_TEST_SCRIPT virshtest: Drop support for testing against hardcoded strings virshtest: Adapt argument parsing tests from 'virsh-optparse' virshtest: Adapt 'snapshot-create-as' arg handling tests from 'virsh-optparse' virshtest: Adapt numeric option arg handling tests from 'virsh-optparse' virshtest: Adapt 'event' option arg handling tests from 'virsh-optparse' virshtest: Reimplement 'virsh-output' test qemuxmlconftest: Adapt XMLs from 'virsh-cpuset' and 'virsh-define-dev-segfault' cases virshtest: Adapt 'virsh-vcpupin' test virshtest: Adapt 'virsh-int-overflow' virshtest: Adapt 'virsh-schedinfo' virshtest: Adapt 'virsh-start' and 'virsh-undefine' tests as 'lifecycle' case virsh: Fix '--name' and '--parent' used together in '(snapshot|checkpoint)-list' command vsh: Allow non-interactive use of 'cd' command virshtest: Adapt 'virsh-snapshot' test virshtest: Adapt 'virsh-checkpoint' test virshtest: Adapt 'virsh-read-bufsiz' and 'virsh-read-non-seekable' virshtest: Adapt 'libvirtd-pool' tests: Re-implement '(virsh|virt-admin)-self-test' directly in meson tests: Reimplement 'libvirtd-fail' case directly in meson virshtest: Adapt virsh-uriprecedence test case tests: Drop 'test-lib.sh' build-aux/syntax-check.mk | 6 +- docs/manpages/virsh.rst | 17 +- src/meson.build | 5 + tests/libvirtd-fail | 16 - tests/libvirtd-pool | 38 - tests/meson.build | 39 +- .../console-compat-crash.x86_64-latest.args | 46 + .../console-compat-crash.x86_64-latest.xml | 65 ++ .../console-compat-crash.xml} | 36 +- .../cpuset-invalid.x86_64-latest.err | 1 + tests/qemuxmlconfdata/cpuset-invalid.xml | 12 + tests/qemuxmlconftest.c | 2 + tests/test-lib.sh | 280 ------ tests/virsh-checkpoint | 178 ---- tests/virsh-cpuset | 46 - tests/virsh-int-overflow | 18 - tests/virsh-optparse | 292 ------ tests/virsh-output | 29 - tests/virsh-output-commands | 94 -- tests/virsh-output.out | 496 ---------- tests/virsh-read-bufsiz | 49 - tests/virsh-read-non-seekable | 51 - tests/virsh-schedinfo | 40 - tests/virsh-self-test | 48 - tests/virsh-snapshot | 233 ----- tests/virsh-start | 41 - tests/virsh-undefine | 76 -- tests/virsh-uriprecedence | 97 -- tests/virsh-vcpupin | 100 -- tests/virshtest.c | 878 +++++++----------- tests/virshtestdata/argument-assignment.in | 15 + tests/virshtestdata/argument-assignment.out | 91 ++ tests/virshtestdata/attach-disk.in | 79 ++ tests/virshtestdata/attach-disk.out | 343 +++++++ tests/virshtestdata/blkiotune.in | 3 + tests/virshtestdata/blkiotune.out | 15 + tests/virshtestdata/checkpoint-c2.xml | 41 + tests/virshtestdata/checkpoint-c3.xml | 38 + tests/virshtestdata/checkpoint-redefine.out | 13 + tests/virshtestdata/checkpoint.in | 35 + tests/virshtestdata/checkpoint.out | 133 +++ tests/virshtestdata/dash-dash-argument-1.out | 1 + tests/virshtestdata/dash-dash-argument-2.out | 1 + tests/virshtestdata/dash-dash-argument-3.out | 1 + tests/virshtestdata/dash-dash-argument-4.out | 1 + tests/virshtestdata/dash-dash-argument-5.out | 1 + tests/virshtestdata/dash-dash-argument-6.out | 1 + tests/virshtestdata/domain-id-overflow.out | 3 + tests/virshtestdata/domain-id.in | 18 + tests/virshtestdata/domain-id.out | 73 ++ tests/virshtestdata/echo-alias-argv.out | 1 + tests/virshtestdata/echo-alias.in | 3 + tests/virshtestdata/echo-alias.out | 3 + tests/virshtestdata/echo-escaping-1.out | 2 + tests/virshtestdata/echo-escaping-2.out | 2 + tests/virshtestdata/echo-escaping-3.out | 2 + tests/virshtestdata/echo-escaping.in | 11 + tests/virshtestdata/echo-escaping.out | 11 + tests/virshtestdata/echo-quote-removal-1.out | 1 + tests/virshtestdata/echo-quote-removal-2.out | 1 + tests/virshtestdata/echo-quote-removal-3.out | 1 + tests/virshtestdata/echo-quote-removal-4.out | 1 + tests/virshtestdata/echo-quote-removal-5.out | 1 + tests/virshtestdata/echo-quote-removal-6.out | 1 + tests/virshtestdata/echo-quote-removal-7.out | 2 + tests/virshtestdata/echo-quote-removal-8.out | 1 + tests/virshtestdata/echo-split.in | 5 + tests/virshtestdata/echo-split.out | 24 + tests/virshtestdata/info-custom.in | 2 + tests/virshtestdata/info-custom.out | 15 + tests/virshtestdata/info-default.in | 2 + tests/virshtestdata/info-default.out | 13 + tests/virshtestdata/iothreads.in | 14 + tests/virshtestdata/iothreads.out | 51 + tests/virshtestdata/lifecycle.in | 23 + tests/virshtestdata/lifecycle.out | 106 +++ tests/virshtestdata/multiple-commands-1.out | 2 + tests/virshtestdata/multiple-commands-10.out | 1 + tests/virshtestdata/multiple-commands-11.out | 1 + tests/virshtestdata/multiple-commands-12.out | 1 + tests/virshtestdata/multiple-commands-2.out | 2 + tests/virshtestdata/multiple-commands-3.out | 2 + tests/virshtestdata/multiple-commands-4.out | 2 + tests/virshtestdata/multiple-commands-5.out | 3 + tests/virshtestdata/multiple-commands-6.out | 1 + tests/virshtestdata/multiple-commands-7.out | 2 + tests/virshtestdata/multiple-commands-8.out | 2 + tests/virshtestdata/multiple-commands-9.out | 1 + tests/virshtestdata/numeric-parsing-event.in | 26 + tests/virshtestdata/numeric-parsing-event.out | 10 + tests/virshtestdata/numeric-parsing.in | 43 + tests/virshtestdata/numeric-parsing.out | 12 + tests/virshtestdata/pool-define-as.out | 12 + tests/virshtestdata/read-big-pipe.out | 7 + .../schedinfo-invalid-argument.out | 5 + tests/virshtestdata/snapshot-create-args.in | 9 + tests/virshtestdata/snapshot-create-args.out | 82 ++ tests/virshtestdata/snapshot-redefine.out | 17 + tests/virshtestdata/snapshot-s2.xml | 43 + tests/virshtestdata/snapshot-s3.xml | 40 + tests/virshtestdata/snapshot.in | 54 ++ tests/virshtestdata/snapshot.out | 173 ++++ .../uriprecedence-LIBVIRT_DEFAULT_URI.out | 5 + ...riprecedence-VIRSH_DEFAULT_CONNECT_URI.out | 5 + tests/virshtestdata/uriprecedence-param.out | 5 + .../uriprecedence-xdg-config.out | 5 + .../bad/libvirt/libvirt.conf | 1 + .../good/libvirt/libvirt.conf | 1 + tests/virshtestdata/vcpupin.in | 23 + tests/virshtestdata/vcpupin.out | 26 + tests/virt-admin-self-test | 1 - tools/meson.build | 4 +- tools/virsh-checkpoint.c | 18 +- tools/virsh-snapshot.c | 18 +- tools/vsh.c | 5 - 115 files changed, 2352 insertions(+), 2852 deletions(-) delete mode 100755 tests/libvirtd-fail delete mode 100755 tests/libvirtd-pool create mode 100644 tests/qemuxmlconfdata/console-compat-crash.x86_64-latest.args create mode 100644 tests/qemuxmlconfdata/console-compat-crash.x86_64-latest.xml rename tests/{virsh-define-dev-segfault => qemuxmlconfdata/console-compat-crash.xml} (59%) mode change 100755 => 100644 create mode 100644 tests/qemuxmlconfdata/cpuset-invalid.x86_64-latest.err create mode 100644 tests/qemuxmlconfdata/cpuset-invalid.xml delete mode 100644 tests/test-lib.sh delete mode 100755 tests/virsh-checkpoint delete mode 100755 tests/virsh-cpuset delete mode 100755 tests/virsh-int-overflow delete mode 100755 tests/virsh-optparse delete mode 100755 tests/virsh-output delete mode 100755 tests/virsh-output-commands delete mode 100644 tests/virsh-output.out delete mode 100755 tests/virsh-read-bufsiz delete mode 100755 tests/virsh-read-non-seekable delete mode 100755 tests/virsh-schedinfo delete mode 100755 tests/virsh-self-test delete mode 100755 tests/virsh-snapshot delete mode 100755 tests/virsh-start delete mode 100755 tests/virsh-undefine delete mode 100755 tests/virsh-uriprecedence delete mode 100755 tests/virsh-vcpupin create mode 100644 tests/virshtestdata/argument-assignment.in create mode 100644 tests/virshtestdata/argument-assignment.out create mode 100755 tests/virshtestdata/attach-disk.in create mode 100644 tests/virshtestdata/attach-disk.out create mode 100644 tests/virshtestdata/blkiotune.in create mode 100644 tests/virshtestdata/blkiotune.out create mode 100644 tests/virshtestdata/checkpoint-c2.xml create mode 100644 tests/virshtestdata/checkpoint-c3.xml create mode 100644 tests/virshtestdata/checkpoint-redefine.out create mode 100755 tests/virshtestdata/checkpoint.in create mode 100644 tests/virshtestdata/checkpoint.out create mode 100644 tests/virshtestdata/dash-dash-argument-1.out create mode 100644 tests/virshtestdata/dash-dash-argument-2.out create mode 100644 tests/virshtestdata/dash-dash-argument-3.out create mode 100644 tests/virshtestdata/dash-dash-argument-4.out create mode 100644 tests/virshtestdata/dash-dash-argument-5.out create mode 100644 tests/virshtestdata/dash-dash-argument-6.out create mode 100644 tests/virshtestdata/domain-id-overflow.out create mode 100644 tests/virshtestdata/domain-id.in create mode 100644 tests/virshtestdata/domain-id.out create mode 100644 tests/virshtestdata/echo-alias-argv.out create mode 100644 tests/virshtestdata/echo-alias.in create mode 100644 tests/virshtestdata/echo-alias.out create mode 100644 tests/virshtestdata/echo-escaping-1.out create mode 100644 tests/virshtestdata/echo-escaping-2.out create mode 100644 tests/virshtestdata/echo-escaping-3.out create mode 100644 tests/virshtestdata/echo-escaping.in create mode 100644 tests/virshtestdata/echo-escaping.out create mode 100644 tests/virshtestdata/echo-quote-removal-1.out create mode 100644 tests/virshtestdata/echo-quote-removal-2.out create mode 100644 tests/virshtestdata/echo-quote-removal-3.out create mode 100644 tests/virshtestdata/echo-quote-removal-4.out create mode 100644 tests/virshtestdata/echo-quote-removal-5.out create mode 100644 tests/virshtestdata/echo-quote-removal-6.out create mode 100644 tests/virshtestdata/echo-quote-removal-7.out create mode 100644 tests/virshtestdata/echo-quote-removal-8.out create mode 100644 tests/virshtestdata/echo-split.in create mode 100644 tests/virshtestdata/echo-split.out create mode 100644 tests/virshtestdata/info-custom.in create mode 100644 tests/virshtestdata/info-custom.out create mode 100644 tests/virshtestdata/info-default.in create mode 100644 tests/virshtestdata/info-default.out create mode 100644 tests/virshtestdata/iothreads.in create mode 100644 tests/virshtestdata/iothreads.out create mode 100644 tests/virshtestdata/lifecycle.in create mode 100644 tests/virshtestdata/lifecycle.out create mode 100644 tests/virshtestdata/multiple-commands-1.out create mode 100644 tests/virshtestdata/multiple-commands-10.out create mode 100644 tests/virshtestdata/multiple-commands-11.out create mode 100644 tests/virshtestdata/multiple-commands-12.out create mode 100644 tests/virshtestdata/multiple-commands-2.out create mode 100644 tests/virshtestdata/multiple-commands-3.out create mode 100644 tests/virshtestdata/multiple-commands-4.out create mode 100644 tests/virshtestdata/multiple-commands-5.out create mode 100644 tests/virshtestdata/multiple-commands-6.out create mode 100644 tests/virshtestdata/multiple-commands-7.out create mode 100644 tests/virshtestdata/multiple-commands-8.out create mode 100644 tests/virshtestdata/multiple-commands-9.out create mode 100644 tests/virshtestdata/numeric-parsing-event.in create mode 100644 tests/virshtestdata/numeric-parsing-event.out create mode 100644 tests/virshtestdata/numeric-parsing.in create mode 100644 tests/virshtestdata/numeric-parsing.out create mode 100644 tests/virshtestdata/pool-define-as.out create mode 100644 tests/virshtestdata/read-big-pipe.out create mode 100644 tests/virshtestdata/schedinfo-invalid-argument.out create mode 100644 tests/virshtestdata/snapshot-create-args.in create mode 100644 tests/virshtestdata/snapshot-create-args.out create mode 100644 tests/virshtestdata/snapshot-redefine.out create mode 100644 tests/virshtestdata/snapshot-s2.xml create mode 100644 tests/virshtestdata/snapshot-s3.xml create mode 100755 tests/virshtestdata/snapshot.in create mode 100644 tests/virshtestdata/snapshot.out create mode 100644 tests/virshtestdata/uriprecedence-LIBVIRT_DEFAULT_URI.out create mode 100644 tests/virshtestdata/uriprecedence-VIRSH_DEFAULT_CONNECT_URI.out create mode 100644 tests/virshtestdata/uriprecedence-param.out create mode 100644 tests/virshtestdata/uriprecedence-xdg-config.out create mode 100644 tests/virshtestdata/uriprecedence-xdg/bad/libvirt/libvirt.conf create mode 100644 tests/virshtestdata/uriprecedence-xdg/good/libvirt/libvirt.conf create mode 100755 tests/virshtestdata/vcpupin.in create mode 100644 tests/virshtestdata/vcpupin.out delete mode 120000 tests/virt-admin-self-test -- 2.44.0