Re: [libvirt] [PATCH v5 02/23] build-aux: rewrite whitespace checker in Python

On 11/11/19 9:38 AM, Daniel P. Berrangé wrote: > As part of an goal to eliminate Perl from libvirt build tools, > rewrite the check-spacing.pl tool in Python. > > This was a straight conversion, manually going line-by-line to > change the syntax from Perl to Python. Thus the overall structure > of the file and approach is the same. > I checked the speed difference mentioned in the cover letter. Old: time ./build-aux/check-spacing.pl `cat ~/src/libvirt/cfiles` real 0m1.423s New: time ./scripts/check-spacing.py `cat cfiles` real 0m5.607s About around there. The script can easily be parallelized though: diff --git a/scripts/check-spacing.py b/scripts/check-spacing.py index 6b9f3ec1ba..f2ce376e80 100755 --- a/scripts/check-spacing.py +++ b/scripts/check-spacing.py @@ -222,8 +222,12 @@ def check_whitespace(filename): ret = 0 -for filename in sys.argv[1:]: - if check_whitespace(filename): - ret = 1 +filenames = sys.argv[1:] +import multiprocessing +pool = multiprocessing.Pool() +results = pool.map(check_whitespace, filenames) +pool.close() +pool.join() +ret = int(any(results)) sys.exit(ret) After that: [:~/src/libvirt] (python3 *) $ time ./scripts/check-spacing.py `cat cfiles` real 0m1.674s user 0m10.506s sys 0m0.041s Which is pretty close to the perl version, but at the cost of more cycles. Not sure how that will play with 'make' job control. I believe the Pool defaults to number of logical host CPUs, which is 8 on my T480s. Going above 4 doesn't seem to make much of a difference in my testing for this case