[libvirt] [PATCH v4 00/26] scripts: convert most perl scripts to python

This series is an effort to reduce the number of different languages we use by eliminating most use of perl in favour of python. This aligns with fact that the likely future build system we'll use (meson) is written in python, and that python is much more commonly used/understood by developers these days than perl. With this applied we use perl in a handful of places only: - src/rpc/gendispatch.pl - this is a horrendously large script and very hard to understand/follow. A straight syntax conversion to Python would still leave a hgue and hard to understand/follow script. It really needs a clean room rewrite from scratch, with better structure. - src/rpc/genprotocol.pl - fairly easy to convert, but might be obsolete depending on approach for rewriting gendispatch.pl, so ignored for now - tests/oomtrace.pl - will be purge by the patches that drop OOM handling anyway - tools/wireshark/util/genxdrstub.pl - a very large script, which I haven't got the courage to tackle yet. - cfg.mk/maint.mk - many syntax rules involve regexes which are fed to perl. Decision on what to do with syntax-check rules punted to another time. - build-aux/gitlog-to-changelog - build-aux/useless-if-before-free - Both pulled in from gnulib. Could be rewritten quite easily if desired, but given that we aren't maintainers of them right now, they're ignored as they don't really impact our developers. Note that the check-spacing.py script is significantly slower in Python than in Perl. After researching this it appears there is nothing that can be done. The Perl regex engine is simply much better optimized than the Python one. As previously discussed we need to loook at uncrustify or clang-format or some other tool to validate whitespace formatting. This is ongoing. We can either accept the slow down in the short term or keep the Perl version in the short term. In v4: - Moved all scripts into the scripts/ directory instead of having them scattered around source tree - Use re.search instead of re.match - Don't bother re-compiling regexes In v3: - All scripts comply with all flake8 style rules with exception of E129 visually indented line with same indent as next logical line In v2: - Pulled in patch to hacking file - Converted many more scripts - Forced UTF-8 character set to avoid ascii codec on py3 < 3.7 Daniel P. Berrangé (26): build: fix use of $(AUG_GENTEST) as a dependency build-aux: rewrite augeas test generator in Python build-aux: rewrite po file minimizer in Python build-aux: rewrite duplicate header checker in Python build-aux: rewrite whitespace checker in Python build-aux: rewrite mock inline checker in Python build-aux: rewrite header ifdef checker in Python src: rewrite ACL permissions checker in Python src: rewrite symfile sorting checker in Python src: rewrite symfile library checker in Python src: rewrite systemtap probe generator in Python src: rewrite systemtap function generator in Python src: rewrite driver name checker in Python src: rewrite driver impl checker in Python src: rewrite ACL rule checker in Python src: rewrite polkit ACL generator in Python src: rewrite remote protocol checker in Python tests: rewrite test argv line wrapper in Python tests: rewrite qemu capability grouper in Python tests: rewrite file access checker in Python docs: rewrite hvsupport.html page generator in python docs: rewrite polkit docs generator in Python docs: move apibuild.py to the scripts/ directory docs: move reformat-news.py to the scripts/ directory docs: move esx_vi_generator.py to the scripts/ directory docs: move hyperv_wmi_generator.py to the scripts/ directory .gitignore | 3 +- Makefile.am | 37 +- build-aux/augeas-gentest.pl | 60 --- build-aux/check-spacing.pl | 198 ------- build-aux/header-ifdef.pl | 182 ------- build-aux/minimize-po.pl | 37 -- build-aux/mock-noinline.pl | 75 --- build-aux/prohibit-duplicate-header.pl | 26 - cfg.mk | 32 +- docs/Makefile.am | 16 +- docs/genaclperms.pl | 125 ----- docs/hvsupport.pl | 458 ---------------- po/Makefile.am | 2 +- {docs => scripts}/apibuild.py | 0 scripts/augeas-gentest.py | 67 +++ scripts/check-aclperms.py | 75 +++ scripts/check-aclrules.py | 263 +++++++++ scripts/check-driverimpls.py | 102 ++++ scripts/check-drivername.py | 114 ++++ scripts/check-file-access.py | 123 +++++ scripts/check-remote-protocol.py | 136 +++++ scripts/check-spacing.py | 229 ++++++++ scripts/check-symfile.py | 80 +++ scripts/check-symsorting.py | 117 ++++ scripts/dtrace2systemtap.py | 143 +++++ {src/esx => scripts}/esx_vi_generator.py | 0 scripts/genaclperms.py | 123 +++++ scripts/genpolkit.py | 122 +++++ scripts/gensystemtap.py | 184 +++++++ scripts/group-qemu-caps.py | 123 +++++ scripts/header-ifdef.py | 231 ++++++++ scripts/hvsupport.py | 503 ++++++++++++++++++ .../hyperv_wmi_generator.py | 0 scripts/minimize-po.py | 58 ++ scripts/mock-noinline.py | 85 +++ scripts/prohibit-duplicate-header.py | 56 ++ {docs => scripts}/reformat-news.py | 0 scripts/test-wrap-argv.py | 170 ++++++ src/Makefile.am | 155 ++---- src/access/Makefile.inc.am | 6 +- src/access/genpolkit.pl | 119 ----- src/bhyve/Makefile.inc.am | 4 +- src/check-aclperms.pl | 73 --- src/check-aclrules.pl | 252 --------- src/check-driverimpls.pl | 80 --- src/check-drivername.pl | 83 --- src/check-symfile.pl | 70 --- src/check-symsorting.pl | 106 ---- src/dtrace2systemtap.pl | 130 ----- src/esx/Makefile.inc.am | 5 +- src/hyperv/Makefile.inc.am | 5 +- src/interface/Makefile.inc.am | 2 +- src/libxl/Makefile.inc.am | 4 +- src/locking/Makefile.inc.am | 6 +- src/logging/Makefile.inc.am | 2 +- src/lxc/Makefile.inc.am | 4 +- src/network/Makefile.inc.am | 2 +- src/node_device/Makefile.inc.am | 2 +- src/nwfilter/Makefile.inc.am | 2 +- src/qemu/Makefile.inc.am | 4 +- src/remote/Makefile.inc.am | 4 +- src/rpc/Makefile.inc.am | 1 - src/rpc/gensystemtap.pl | 193 ------- src/secret/Makefile.inc.am | 2 +- src/storage/Makefile.inc.am | 2 +- src/vbox/Makefile.inc.am | 2 +- src/vz/Makefile.inc.am | 2 +- tests/Makefile.am | 3 +- tests/check-file-access.pl | 126 ----- tests/file_access_whitelist.txt | 2 +- tests/group-qemu-caps.pl | 124 ----- tests/test-wrap-argv.pl | 174 ------ tests/testutils.c | 16 +- 73 files changed, 3243 insertions(+), 2879 deletions(-) delete mode 100755 build-aux/augeas-gentest.pl delete mode 100755 build-aux/check-spacing.pl delete mode 100644 build-aux/header-ifdef.pl delete mode 100755 build-aux/minimize-po.pl delete mode 100644 build-aux/mock-noinline.pl delete mode 100644 build-aux/prohibit-duplicate-header.pl delete mode 100755 docs/genaclperms.pl delete mode 100755 docs/hvsupport.pl rename {docs => scripts}/apibuild.py (100%) create mode 100755 scripts/augeas-gentest.py create mode 100755 scripts/check-aclperms.py create mode 100755 scripts/check-aclrules.py create mode 100755 scripts/check-driverimpls.py create mode 100644 scripts/check-drivername.py create mode 100755 scripts/check-file-access.py create mode 100644 scripts/check-remote-protocol.py create mode 100755 scripts/check-spacing.py create mode 100755 scripts/check-symfile.py create mode 100755 scripts/check-symsorting.py create mode 100755 scripts/dtrace2systemtap.py rename {src/esx => scripts}/esx_vi_generator.py (100%) create mode 100755 scripts/genaclperms.py create mode 100755 scripts/genpolkit.py create mode 100755 scripts/gensystemtap.py create mode 100755 scripts/group-qemu-caps.py create mode 100644 scripts/header-ifdef.py create mode 100755 scripts/hvsupport.py rename {src/hyperv => scripts}/hyperv_wmi_generator.py (100%) create mode 100755 scripts/minimize-po.py create mode 100644 scripts/mock-noinline.py create mode 100644 scripts/prohibit-duplicate-header.py rename {docs => scripts}/reformat-news.py (100%) create mode 100755 scripts/test-wrap-argv.py delete mode 100755 src/access/genpolkit.pl delete mode 100755 src/check-aclperms.pl delete mode 100755 src/check-aclrules.pl delete mode 100755 src/check-driverimpls.pl delete mode 100755 src/check-drivername.pl delete mode 100755 src/check-symfile.pl delete mode 100755 src/check-symsorting.pl delete mode 100755 src/dtrace2systemtap.pl delete mode 100755 src/rpc/gensystemtap.pl delete mode 100755 tests/check-file-access.pl delete mode 100755 tests/group-qemu-caps.pl delete mode 100755 tests/test-wrap-argv.pl -- 2.21.0