[petsc-dev] make -j not supported? Why???

[Barry Smith]

  Is it still not possible to use cmake to build cuda stuff? That's crazy?

   Barry

On Feb 25, 2013, at 5:06 PM, Satish Balay <balay at mcs.anl.gov> wrote:

> Parallel build prints someting like:
> 
>>>>>>>>> 
> ==========================================
> Building PETSc using CMake with 5 build threads
> ==========================================
> Re-run cmake file: Makefile older than: ../CMakeLists.txt
> -- Configuring done
> -- Generating done
> -- Build files have been written to: /home/balay/spetsc/asterix64
> Scanning dependencies of target petsc
> [  0%] Building C object CMakeFiles/petsc.dir/src/sys/classes/draw/utils/lgc.c.o
> [  1%] Building C object CMakeFiles/petsc.dir/src/mat/impls/mffd/mffd.c.o
> [  1%] [  1%] Building C object CMakeFiles/petsc.dir/src/dm/impls/plex/ftn-auto/plexcreatef.c.o
> 
> <<<<<<<<<
> 
> etc..
> 
> Note: if you are building cuda stuff or have python < 2.5 - then you
> are using legacy build [i.e sequential/recursive make] and see stuff
> like:
> 
>>>>>>> 
> libfast in: /home/balay/spetsc/src
> libfast in: /home/balay/spetsc/src/sys
> libfast in: /home/balay/spetsc/src/sys/classes
> libfast in: /home/balay/spetsc/src/sys/classes/viewer
> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls
> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/socket
> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/socket/ftn-custom
> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii
> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii/ftn-auto
> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii/ftn-custom
> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/binary
> <<<<
> etc..
> 
> 
> Satish
> 
> On Mon, 25 Feb 2013, Nystrom, William D wrote:
> 
>> Is there a way to examine the petsc build logs and determine if I am really getting parallel builds?  I have
>> a top level script that I use to the petsc configure script followed by make.  For a couple of our test bed
>> systems that stay fairly unloaded, I use the following configure option: --with-make-np=16.  But I also
>> invoke my script like this: /usr/bin/time -p makePetscDev_caddy >& makePetscDev_caddy.log.  And
>> I'll get a time like the following:
>> 
>> real 2160.77
>> user 1308.21
>> sys 234.37
>> 
>> So it does not seem like I am getting much if any parallelism in my build.  I am also building with lots
>> of external packages and have no idea if parallel builds are being done with them.
>> 
>> Thanks,
>> 
>> Dave
>> 
>> --
>> Dave Nystrom
>> LANL HPC-5
>> Phone: 505-667-7913
>> Email: wdn at lanl.gov
>> Smail: Mail Stop B272
>>       Group HPC-5
>>       Los Alamos National Laboratory
>>       Los Alamos, NM 87545
>> 
>> 
>> ________________________________________
>> From: petsc-dev-bounces at mcs.anl.gov [petsc-dev-bounces at mcs.anl.gov] on behalf of Barry Smith [bsmith at mcs.anl.gov]
>> Sent: Monday, February 25, 2013 3:25 PM
>> To: For users of the development version of PETSc
>> Subject: Re: [petsc-dev] make -j not supported? Why???
>> 
>>  Tim,
>> 
>>   As usual, a too short error message without enough information. I've changed it so that it will now give you
>> 
>> Barrys-MacBook-Pro:petsc-dev barrysmith$ make -j 3 ccmake
>> ********************* ERROR *************************
>> PETSc compiles are automatically parallel, do not
>> provide the -j option to make
>> ******************************************************
>> make: *** [chk_makej] Error 1
>> Barrys-MacBook-Pro:petsc-dev barrysmith$ make -j 3 all-legacy
>> ********************* ERROR *************************
>> Parallel build with 'make -j' is not supported
>> for PETSc legacy builds. Run without -j <np>
>> or ./configure PETSc with --download-cmake
>> to enable parallel builds with PETSc
>> ******************************************************
>> make[1]: *** [chk_makej] Error 1
>> make: *** [all-legacy] Error 2
>> Barrys-MacBook-Pro:petsc-dev barrysmith$
>> 
>> Explanations:
>> When PETSc is using cmake makefiles from the top level makefiles we pass our own -j <np> down, make doesn't like having two levels of -j and at best prints a confusing warning message about the parallel build.
>> 
>> When using PETSc's legacy make system it uses a recursive make that is not able to be parallelized (yes it is bad and hence the cmake version is used by default now).
>> 
>> Notes:
>> It would be nice if we could somehow remove the -j <np> at the top level so that it didn't generate the error message and just ran but make doesn't allow this type of thing.
>> 
>> It would be nice if we could eliminate the messages like
>> 
>> make[1]: *** [chk_makej] Error 1
>> make: *** [all-legacy] Error 2
>> 
>> that make generates automatically but I cannot figure out how to do that.
>> 
>> So the current situation is a compromise where most users get automatically a parallel build and the users who knowingly provide a -j <np> option get slightly peeved. (My guess is 90+% of people who install PETSc don't know about the -j option).  At least now it will explain that it is doing a parallel build.
>> 
>> I'll check on the -k option next.
>> 
>>  Barry
>> 
>> On Feb 25, 2013, at 2:58 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
>> 
>>> This is to support different backends. If configure found CMake, then normal "make" will use multiple jobs, or you can do it explicitly with make -j3 $PETSC_ARCH.
>>> 
>>> CMake would be terrible as a hard dependency (it's terrible enough as it is) but it's less bad than some alternatives.
>>> 
>>> 
>>> On Mon, Feb 25, 2013 at 3:30 PM, Tim Tautges <tautges at mcs.anl.gov> wrote:
>>> tautges at chronos:~/code/petsc-dev-moab$ make -j 8
>>> ********************* ERROR *************************
>>> Parallel build with 'make -j' is not supported
>>> ******************************************************
>>> make: *** [chk_makej] Error 1
>>> tautges at chronos:~/code/petsc-dev-moab$
>>> 
>>> 
>>> Why??? I can almost understand petsc's justification for its own autotools-looking python-based build system, but why its need to not support otherwise-standard make options?  Similarly, PETSC_MAKE_STOP_ON_ERROR instead of make -k?
>>> 
>>> - tim
>>> 
>>> --
>>> ================================================================
>>> "You will keep in perfect peace him whose mind is
>>>  steadfast, because he trusts in you."               Isaiah 26:3
>>> 
>>>             Tim Tautges            Argonne National Laboratory
>>>         (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
>>> phone (gvoice): (608) 354-1459      1500 Engineering Dr.
>>>            fax: (608) 263-4499      Madison, WI 53706
>>> 
>>> 
>> 
>> 
>