On Tue, Jul 28, 2020 at 15:08:11 +0100, Daniel Berrange wrote:
On Tue, Jul 28, 2020 at 04:04:39PM +0200, Pavel Hrdina wrote:
> On Tue, Jul 28, 2020 at 03:48:29PM +0200, Peter Krempa wrote:
> > On Thu, Jul 16, 2020 at 11:59:38 +0200, Pavel Hrdina wrote:
> > > Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com>
> > > ---
> > > build-aux/Makefile.in | 9 +++
> > > .../Makefile.nonreentrant | 0
> > > build-aux/meson.build | 30 +++++++++
> > > build-aux/syntax-check.mk | 62 +++++++++----------
> > > meson.build | 2 +
> > > 5 files changed, 72 insertions(+), 31 deletions(-)
> > > create mode 100644 build-aux/Makefile.in
> > > rename Makefile.nonreentrant => build-aux/Makefile.nonreentrant
(100%)
> > > create mode 100644 build-aux/meson.build
> >
> > [...]
> >
> > > +make_prog = find_program('make')
> > > +
> > > +# There is no way how to pass value to -j using run_target so let's
use
> > > +# it without any value to run all tests in parallel.
> > > +run_target(
> > > + 'syntax-check',
> > > + command: [
> > > + make_prog, '-C', meson.current_build_dir(), '-j',
'syntax-check',
> > > + ],
> >
> > While I do run syntax check with unlimited '-j'. I don't think
it's
> > entirely cool to impose that on everybody. Specifically overcommiting
> > the system is not cool. Since meson is automagically scaling can't we
> > use the meson-detected cpu number here?
>
> Unfortunately no, that was the first thing I was trying to figure out
> by going through meson code as well. It's not ideal I know.
>
> Other options are to not use -j at all which is no-go or we can add some
> code to detect the available number of CPUs. But again it would not
> reflect the fact if user runs 'ninja -j N'.
In libosinfo we put "syntax-check" as part of the unit tests, rather
than as a separate meson target. With that you don't need to pass
-j to syntax-check, because other unit tests are running in parallel
already, and chances are syntax-check will finish first even when
serialized.
Unfortunately it's not even close.
Serialized syntax-check:
real 0m22.139s
user 0m24.209s
sys 0m6.788s
test suite:
real 0m4.833s
user 0m12.408s
sys 0m3.918s
syntax-check with -j == ncpus: (24 thread box)
real 0m2.099s
user 0m28.558s
sys 0m7.739s
As said, I'm a big fan of -jncpus or -j. so I really want to keep it
especially given the data above, but on the other hand I don't want to
set the CI boxes on fire.