[petsc-dev] make -j not supported? Why???

[Satish Balay]

Parallel build prints someting like:

>>>>>>>>
==========================================
Building PETSc using CMake with 5 build threads
==========================================
Re-run cmake file: Makefile older than: ../CMakeLists.txt
-- Configuring done
-- Generating done
-- Build files have been written to: /home/balay/spetsc/asterix64
Scanning dependencies of target petsc
[  0%] Building C object CMakeFiles/petsc.dir/src/sys/classes/draw/utils/lgc.c.o
[  1%] Building C object CMakeFiles/petsc.dir/src/mat/impls/mffd/mffd.c.o
[  1%] [  1%] Building C object CMakeFiles/petsc.dir/src/dm/impls/plex/ftn-auto/plexcreatef.c.o

<<<<<<<<<

etc..

Note: if you are building cuda stuff or have python < 2.5 - then you
are using legacy build [i.e sequential/recursive make] and see stuff
like:

>>>>>>
libfast in: /home/balay/spetsc/src
libfast in: /home/balay/spetsc/src/sys
libfast in: /home/balay/spetsc/src/sys/classes
libfast in: /home/balay/spetsc/src/sys/classes/viewer
libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls
libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/socket
libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/socket/ftn-custom
libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii
libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii/ftn-auto
libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii/ftn-custom
libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/binary
<<<<
etc..


Satish

On Mon, 25 Feb 2013, Nystrom, William D wrote:

> Is there a way to examine the petsc build logs and determine if I am really getting parallel builds?  I have
> a top level script that I use to the petsc configure script followed by make.  For a couple of our test bed
> systems that stay fairly unloaded, I use the following configure option: --with-make-np=16.  But I also
> invoke my script like this: /usr/bin/time -p makePetscDev_caddy >& makePetscDev_caddy.log.  And
> I'll get a time like the following:
> 
> real 2160.77
> user 1308.21
> sys 234.37
> 
> So it does not seem like I am getting much if any parallelism in my build.  I am also building with lots
> of external packages and have no idea if parallel builds are being done with them.
> 
> Thanks,
> 
> Dave
> 
> --
> Dave Nystrom
> LANL HPC-5
> Phone: 505-667-7913
> Email: wdn at lanl.gov
> Smail: Mail Stop B272
>        Group HPC-5
>        Los Alamos National Laboratory
>        Los Alamos, NM 87545
> 
> 
> ________________________________________
> From: petsc-dev-bounces at mcs.anl.gov [petsc-dev-bounces at mcs.anl.gov] on behalf of Barry Smith [bsmith at mcs.anl.gov]
> Sent: Monday, February 25, 2013 3:25 PM
> To: For users of the development version of PETSc
> Subject: Re: [petsc-dev] make -j not supported? Why???
> 
>   Tim,
> 
>    As usual, a too short error message without enough information. I've changed it so that it will now give you
> 
> Barrys-MacBook-Pro:petsc-dev barrysmith$ make -j 3 ccmake
> ********************* ERROR *************************
> PETSc compiles are automatically parallel, do not
> provide the -j option to make
> ******************************************************
> make: *** [chk_makej] Error 1
> Barrys-MacBook-Pro:petsc-dev barrysmith$ make -j 3 all-legacy
> ********************* ERROR *************************
> Parallel build with 'make -j' is not supported
> for PETSc legacy builds. Run without -j <np>
> or ./configure PETSc with --download-cmake
> to enable parallel builds with PETSc
> ******************************************************
> make[1]: *** [chk_makej] Error 1
> make: *** [all-legacy] Error 2
> Barrys-MacBook-Pro:petsc-dev barrysmith$
> 
> Explanations:
> When PETSc is using cmake makefiles from the top level makefiles we pass our own -j <np> down, make doesn't like having two levels of -j and at best prints a confusing warning message about the parallel build.
> 
> When using PETSc's legacy make system it uses a recursive make that is not able to be parallelized (yes it is bad and hence the cmake version is used by default now).
> 
> Notes:
> It would be nice if we could somehow remove the -j <np> at the top level so that it didn't generate the error message and just ran but make doesn't allow this type of thing.
> 
> It would be nice if we could eliminate the messages like
> 
> make[1]: *** [chk_makej] Error 1
> make: *** [all-legacy] Error 2
> 
> that make generates automatically but I cannot figure out how to do that.
> 
> So the current situation is a compromise where most users get automatically a parallel build and the users who knowingly provide a -j <np> option get slightly peeved. (My guess is 90+% of people who install PETSc don't know about the -j option).  At least now it will explain that it is doing a parallel build.
> 
> I'll check on the -k option next.
> 
>   Barry
> 
> On Feb 25, 2013, at 2:58 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
> 
> > This is to support different backends. If configure found CMake, then normal "make" will use multiple jobs, or you can do it explicitly with make -j3 $PETSC_ARCH.
> >
> > CMake would be terrible as a hard dependency (it's terrible enough as it is) but it's less bad than some alternatives.
> >
> >
> > On Mon, Feb 25, 2013 at 3:30 PM, Tim Tautges <tautges at mcs.anl.gov> wrote:
> > tautges at chronos:~/code/petsc-dev-moab$ make -j 8
> > ********************* ERROR *************************
> > Parallel build with 'make -j' is not supported
> > ******************************************************
> > make: *** [chk_makej] Error 1
> > tautges at chronos:~/code/petsc-dev-moab$
> >
> >
> > Why??? I can almost understand petsc's justification for its own autotools-looking python-based build system, but why its need to not support otherwise-standard make options?  Similarly, PETSC_MAKE_STOP_ON_ERROR instead of make -k?
> >
> > - tim
> >
> > --
> > ================================================================
> > "You will keep in perfect peace him whose mind is
> >   steadfast, because he trusts in you."               Isaiah 26:3
> >
> >              Tim Tautges            Argonne National Laboratory
> >          (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
> >  phone (gvoice): (608) 354-1459      1500 Engineering Dr.
> >             fax: (608) 263-4499      Madison, WI 53706
> >
> >
> 
>