Re: [libvirt] [PATCH v5 02/23] build-aux: rewrite whitespace checker in Python
On 11/11/19 9:38 AM, Daniel P. Berrangé wrote:
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-spacing.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
I checked the speed difference mentioned in the cover letter.
Old:
time ./build-aux/check-spacing.pl `cat ~/src/libvirt/cfiles`
real 0m1.423s
New:
time ./scripts/check-spacing.py `cat cfiles`
real 0m5.607s
About around there. The script can easily be parallelized though:
diff --git a/scripts/check-spacing.py b/scripts/check-spacing.py
index 6b9f3ec1ba..f2ce376e80 100755
--- a/scripts/check-spacing.py
+++ b/scripts/check-spacing.py
@@ -222,8 +222,12 @@ def check_whitespace(filename):
ret = 0
-for filename in sys.argv[1:]:
- if check_whitespace(filename):
- ret = 1
+filenames = sys.argv[1:]
+import multiprocessing
+pool = multiprocessing.Pool()
+results = pool.map(check_whitespace, filenames)
+pool.close()
+pool.join()
+ret = int(any(results))
sys.exit(ret)
After that:
[:~/src/libvirt] (python3 *) $ time ./scripts/check-spacing.py `cat cfiles`
real 0m1.674s
user 0m10.506s
sys 0m0.041s
Which is pretty close to the perl version, but at the cost of more
cycles. Not sure how that will play with 'make' job control. I believe
the Pool defaults to number of logical host CPUs, which is 8 on my
T480s. Going above 4 doesn't seem to make much of a difference in my
testing for this case
- Cole