[petsc-dev] could we let "make -j N" work for PETSc and external packages?

[Fande Kong]

Thanks, Satish,

I’m going to use MAKE_NP 

Fande 

Sent from my iPhone

> On Jul 24, 2018, at 4:26 PM, Satish Balay <balay at mcs.anl.gov> wrote:
> 
> I hate html e-mail which doesn't add reply markers - can't figure out easily what you saying :(
> 
>>> However some older make version might not display this info properly. [it should use the user specified '-j' value though]
>> 
>> This confused me, and now I understood.  But it is misleading. Could we do something at the PETSc side regardless the make version? At least display the user-specified value not the default value.  I thought my "-j" did not change anything.
> 
> Thats a 'make' issue. you are specifying options to make. So we have no control on it [unless make tells us - and this mode is buggy on the OSX default make version].
> 
> The only option we have is : not print anything. I think thats a bad option.
> 
> You have alternatives:
> 
> - use a newer make with --download-make
> 
> - or use MAKE_NP
> 
> Satish
> 
>> On Tue, 24 Jul 2018, Fande Kong wrote:
>> 
>> 
>> 
>> 
>> ________________________________
>> From: Satish Balay <balay at mcs.anl.gov>
>> Sent: Tuesday, July 24, 2018 4:06 PM
>> To: Fande Kong
>> Cc: Jed Brown; For users of the development version of PETSc
>> Subject: Re: [petsc-dev] could we let "make -j N" work for PETSc and external packages?
>> 
>> PETSc configure sets up a default value that always gets used.
>> 
>>>>>>>>> 
>> balay at asterix /home/balay/tmp/petsc (maint=)
>> $ make
>> <snip>
>> Using MAKEFLAGS: -j4 -l6.0 w -- PETSC_DIR=/home/balay/tmp/petsc PETSC_ARCH=arch-linux2-c-debug
>> <<<
>> 
>> You can always override this default by adding a -j option. [since the
>> user specified -j value comes at the end - it overrides the one that
>> is specified at the begining of make command]
>> 
>>>>>>>>>>>> 
>> balay at asterix /home/balay/tmp/petsc (maint=)
>> $ make -j20
>> <snip>
>> Using MAKEFLAGS: -j4 -l6.0 w -j20 --jobserver-auth=3,4 -- PETSC_DIR=/home/balay/tmp/petsc PETSC_ARCH=arch-linux2-c-debug
>> <<<<<<<<<<
>> 
>> 
>> However some older make version might not display this info properly. [it should use the user specified '-j' value though]
>> 
>> This confused me, and now I understood.  But it is misleading. Could we do something at the PETSc side regardless the make version? At least display the user-specified value not the default value.  I thought my "-j" did not change anything.
>> 
>> Fande,
>> 
>>>>>>>>> 
>> [petsc at ipro petsc.maint ((879a78556f...))]$ make -j2
>> Using MAKEFLAGS: -j7 -l12.0 w --jobserver-fds=3,4 -j -- PETSC_DIR=/Users/petsc/petsc.maint PETSC_ARCH=arch-osx-10.6
>> <<<<<<<
>> 
>> You can use the alternate mode - just modify the default '-j' value used MAKE_NP
>> 
>>>>>>>>>> 
>> [petsc at ipro petsc.maint ((879a78556f...))]$ make MAKE_NP=2
>> Using MAKEFLAGS: -j2 -l12.0 w -- PETSC_DIR=/Users/petsc/petsc.maint PETSC_ARCH=arch-osx-10.6 MAKE_NP=2
>> <<<<<<<
>> 
>> Or you can control this default value at configure time [this affects externalpackage builds]
>> 
>> ./configure --with-make-np=2
>> 
>> Satish
>> 
>> 
>>> On Tue, 24 Jul 2018, Fande Kong wrote:
>>> 
>>> Thanks, Jed and Satish,
>>> 
>>> 
>>> But "make -j N" does not change anything when compiling petsc.
>>> 
>>> 
>>> 
>>> If I do something like: "make -j whatever". Then I always get:
>>> 
>>> 
>>> Using MAKEFLAGS: -j8 -l36.0 w
>>> 
>>> 
>>> It looks like PETSc always uses "-j something" that we can not change using the command-line.
>>> 
>>> 
>>> Is it a bug?
>>> 
>>> 
>>> 
>>> Fande,
>>> 
>>> 
>>> 
>>> ________________________________
>>> From: Jed Brown <jed at jedbrown.org>
>>> Sent: Tuesday, July 24, 2018 3:44 PM
>>> To: Fande Kong; For users of the development version of PETSc
>>> Subject: Re: [petsc-dev] could we let "make -j N" work for PETSc and external packages?
>>> 
>>> Fande Kong <Fande.Kong at inl.gov> writes:
>>> 
>>>> Hi Developers,
>>>> 
>>>> 
>>>> Could we somehow let "make -j N" to take multiple threads for compiling PETSc. Do we use multiple threads when compiling the external packages such as hypre, parmetis etc.?
>>> 
>>> Yes, see self.parallelMake=0 for the few packages that don't support parallel make.
>>> 
>> 
>> 
>