[petsc-dev] could we let "make -j N" work for PETSc and external packages?

[Fande Kong]

The default value determined by the configure sometimes makes the compiling process fail.  I have experienced a couple of times before. The solution is to use "MAKE_NP=1".



Fande,


________________________________
From: petsc-dev <petsc-dev-bounces at mcs.anl.gov> on behalf of Satish Balay <balay at mcs.anl.gov>
Sent: Tuesday, July 24, 2018 5:57:55 PM
To: Fande Kong
Cc: petsc-dev
Subject: Re: [petsc-dev] could we let "make -j N" work for PETSc and external packages?

Another alternative: use gmakefile directly - and ignore our
'makefile' wrapper.

[with makefile wrapper - we try to control a few things - and print
some extra debug info like the MAKEFLAGS].

But if you use 'gmakefile' directly - you would be using PETSc like
any other package - it supports 'V=1' for some debug info [which is
similar to info provided by most packages]


BTW: Any particular why MAKE_NP determined by configure [i.e the default value] is not useful?

Satish

On Tue, 24 Jul 2018, Fande Kong wrote:

> Thanks, Satish,
>
> I’m going to use MAKE_NP
>
> Fande
>
> Sent from my iPhone
>
> > On Jul 24, 2018, at 4:26 PM, Satish Balay <balay at mcs.anl.gov> wrote:
> >
> > I hate html e-mail which doesn't add reply markers - can't figure out easily what you saying :(
> >
> >>> However some older make version might not display this info properly. [it should use the user specified '-j' value though]
> >>
> >> This confused me, and now I understood.  But it is misleading. Could we do something at the PETSc side regardless the make version? At least display the user-specified value not the default value.  I thought my "-j" did not change anything.
> >
> > Thats a 'make' issue. you are specifying options to make. So we have no control on it [unless make tells us - and this mode is buggy on the OSX default make version].
> >
> > The only option we have is : not print anything. I think thats a bad option.
> >
> > You have alternatives:
> >
> > - use a newer make with --download-make
> >
> > - or use MAKE_NP
> >
> > Satish
> >
> >> On Tue, 24 Jul 2018, Fande Kong wrote:
> >>
> >>
> >>
> >>
> >> ________________________________
> >> From: Satish Balay <balay at mcs.anl.gov>
> >> Sent: Tuesday, July 24, 2018 4:06 PM
> >> To: Fande Kong
> >> Cc: Jed Brown; For users of the development version of PETSc
> >> Subject: Re: [petsc-dev] could we let "make -j N" work for PETSc and external packages?
> >>
> >> PETSc configure sets up a default value that always gets used.
> >>
> >>>>>>>>>
> >> balay at asterix /home/balay/tmp/petsc (maint=)
> >> $ make
> >> <snip>
> >> Using MAKEFLAGS: -j4 -l6.0 w -- PETSC_DIR=/home/balay/tmp/petsc PETSC_ARCH=arch-linux2-c-debug
> >> <<<
> >>
> >> You can always override this default by adding a -j option. [since the
> >> user specified -j value comes at the end - it overrides the one that
> >> is specified at the begining of make command]
> >>
> >>>>>>>>>>>>
> >> balay at asterix /home/balay/tmp/petsc (maint=)
> >> $ make -j20
> >> <snip>
> >> Using MAKEFLAGS: -j4 -l6.0 w -j20 --jobserver-auth=3,4 -- PETSC_DIR=/home/balay/tmp/petsc PETSC_ARCH=arch-linux2-c-debug
> >> <<<<<<<<<<
> >>
> >>
> >> However some older make version might not display this info properly. [it should use the user specified '-j' value though]
> >>
> >> This confused me, and now I understood.  But it is misleading. Could we do something at the PETSc side regardless the make version? At least display the user-specified value not the default value.  I thought my "-j" did not change anything.
> >>
> >> Fande,
> >>
> >>>>>>>>>
> >> [petsc at ipro petsc.maint ((879a78556f...))]$ make -j2
> >> Using MAKEFLAGS: -j7 -l12.0 w --jobserver-fds=3,4 -j -- PETSC_DIR=/Users/petsc/petsc.maint PETSC_ARCH=arch-osx-10.6
> >> <<<<<<<
> >>
> >> You can use the alternate mode - just modify the default '-j' value used MAKE_NP
> >>
> >>>>>>>>>>
> >> [petsc at ipro petsc.maint ((879a78556f...))]$ make MAKE_NP=2
> >> Using MAKEFLAGS: -j2 -l12.0 w -- PETSC_DIR=/Users/petsc/petsc.maint PETSC_ARCH=arch-osx-10.6 MAKE_NP=2
> >> <<<<<<<
> >>
> >> Or you can control this default value at configure time [this affects externalpackage builds]
> >>
> >> ./configure --with-make-np=2
> >>
> >> Satish
> >>
> >>
> >>> On Tue, 24 Jul 2018, Fande Kong wrote:
> >>>
> >>> Thanks, Jed and Satish,
> >>>
> >>>
> >>> But "make -j N" does not change anything when compiling petsc.
> >>>
> >>>
> >>>
> >>> If I do something like: "make -j whatever". Then I always get:
> >>>
> >>>
> >>> Using MAKEFLAGS: -j8 -l36.0 w
> >>>
> >>>
> >>> It looks like PETSc always uses "-j something" that we can not change using the command-line.
> >>>
> >>>
> >>> Is it a bug?
> >>>
> >>>
> >>>
> >>> Fande,
> >>>
> >>>
> >>>
> >>> ________________________________
> >>> From: Jed Brown <jed at jedbrown.org>
> >>> Sent: Tuesday, July 24, 2018 3:44 PM
> >>> To: Fande Kong; For users of the development version of PETSc
> >>> Subject: Re: [petsc-dev] could we let "make -j N" work for PETSc and external packages?
> >>>
> >>> Fande Kong <Fande.Kong at inl.gov> writes:
> >>>
> >>>> Hi Developers,
> >>>>
> >>>>
> >>>> Could we somehow let "make -j N" to take multiple threads for compiling PETSc. Do we use multiple threads when compiling the external packages such as hypre, parmetis etc.?
> >>>
> >>> Yes, see self.parallelMake=0 for the few packages that don't support parallel make.
> >>>
> >>
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20180725/d0dc254c/attachment.html>