[petsc-dev] make -j not supported? Why???

Barry Smith bsmith at mcs.anl.gov
Mon Feb 25 20:51:42 CST 2013 

On Feb 25, 2013, at 7:18 PM, Karl Rupp <rupp at mcs.anl.gov> wrote:

> Hi,
> 
>>   Is it still not possible to use cmake to build cuda stuff? That's crazy?
> 
> It is possible, but you cannot specify nvcc as CMAKE_CXX_COMPILER, but instead the script is doing some magic hackery replacing the C++-compiler with nvcc if the file extension is .cu.

  Karl,

   Ok, can you take a look at how Jed's cmake stuff is working and see if support for handling the .cu can be added?

    Thanks

   Barry

> 
> (I consider this to be a hack rather than a clean solution, but that's the status quo)
> 
> Best regards,
> Karli
> 
>> 
>> On Feb 25, 2013, at 5:06 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>> 
>>> Parallel build prints someting like:
>>> 
>>>>>>>>>>> 
>>> ==========================================
>>> Building PETSc using CMake with 5 build threads
>>> ==========================================
>>> Re-run cmake file: Makefile older than: ../CMakeLists.txt
>>> -- Configuring done
>>> -- Generating done
>>> -- Build files have been written to: /home/balay/spetsc/asterix64
>>> Scanning dependencies of target petsc
>>> [  0%] Building C object CMakeFiles/petsc.dir/src/sys/classes/draw/utils/lgc.c.o
>>> [  1%] Building C object CMakeFiles/petsc.dir/src/mat/impls/mffd/mffd.c.o
>>> [  1%] [  1%] Building C object CMakeFiles/petsc.dir/src/dm/impls/plex/ftn-auto/plexcreatef.c.o
>>> 
>>> <<<<<<<<<
>>> 
>>> etc..
>>> 
>>> Note: if you are building cuda stuff or have python < 2.5 - then you
>>> are using legacy build [i.e sequential/recursive make] and see stuff
>>> like:
>>> 
>>>>>>>>> 
>>> libfast in: /home/balay/spetsc/src
>>> libfast in: /home/balay/spetsc/src/sys
>>> libfast in: /home/balay/spetsc/src/sys/classes
>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer
>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls
>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/socket
>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/socket/ftn-custom
>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii
>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii/ftn-auto
>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii/ftn-custom
>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/binary
>>> <<<<
>>> etc..
>>> 
>>> 
>>> Satish
>>> 
>>> On Mon, 25 Feb 2013, Nystrom, William D wrote:
>>> 
>>>> Is there a way to examine the petsc build logs and determine if I am really getting parallel builds?  I have
>>>> a top level script that I use to the petsc configure script followed by make.  For a couple of our test bed
>>>> systems that stay fairly unloaded, I use the following configure option: --with-make-np=16.  But I also
>>>> invoke my script like this: /usr/bin/time -p makePetscDev_caddy >& makePetscDev_caddy.log.  And
>>>> I'll get a time like the following:
>>>> 
>>>> real 2160.77
>>>> user 1308.21
>>>> sys 234.37
>>>> 
>>>> So it does not seem like I am getting much if any parallelism in my build.  I am also building with lots
>>>> of external packages and have no idea if parallel builds are being done with them.
>>>> 
>>>> Thanks,
>>>> 
>>>> Dave
>>>> 
>>>> --
>>>> Dave Nystrom
>>>> LANL HPC-5
>>>> Phone: 505-667-7913
>>>> Email: wdn at lanl.gov
>>>> Smail: Mail Stop B272
>>>>       Group HPC-5
>>>>       Los Alamos National Laboratory
>>>>       Los Alamos, NM 87545
>>>> 
>>>> 
>>>> ________________________________________
>>>> From: petsc-dev-bounces at mcs.anl.gov [petsc-dev-bounces at mcs.anl.gov] on behalf of Barry Smith [bsmith at mcs.anl.gov]
>>>> Sent: Monday, February 25, 2013 3:25 PM
>>>> To: For users of the development version of PETSc
>>>> Subject: Re: [petsc-dev] make -j not supported? Why???
>>>> 
>>>>  Tim,
>>>> 
>>>>   As usual, a too short error message without enough information. I've changed it so that it will now give you
>>>> 
>>>> Barrys-MacBook-Pro:petsc-dev barrysmith$ make -j 3 ccmake
>>>> ********************* ERROR *************************
>>>> PETSc compiles are automatically parallel, do not
>>>> provide the -j option to make
>>>> ******************************************************
>>>> make: *** [chk_makej] Error 1
>>>> Barrys-MacBook-Pro:petsc-dev barrysmith$ make -j 3 all-legacy
>>>> ********************* ERROR *************************
>>>> Parallel build with 'make -j' is not supported
>>>> for PETSc legacy builds. Run without -j <np>
>>>> or ./configure PETSc with --download-cmake
>>>> to enable parallel builds with PETSc
>>>> ******************************************************
>>>> make[1]: *** [chk_makej] Error 1
>>>> make: *** [all-legacy] Error 2
>>>> Barrys-MacBook-Pro:petsc-dev barrysmith$
>>>> 
>>>> Explanations:
>>>> When PETSc is using cmake makefiles from the top level makefiles we pass our own -j <np> down, make doesn't like having two levels of -j and at best prints a confusing warning message about the parallel build.
>>>> 
>>>> When using PETSc's legacy make system it uses a recursive make that is not able to be parallelized (yes it is bad and hence the cmake version is used by default now).
>>>> 
>>>> Notes:
>>>> It would be nice if we could somehow remove the -j <np> at the top level so that it didn't generate the error message and just ran but make doesn't allow this type of thing.
>>>> 
>>>> It would be nice if we could eliminate the messages like
>>>> 
>>>> make[1]: *** [chk_makej] Error 1
>>>> make: *** [all-legacy] Error 2
>>>> 
>>>> that make generates automatically but I cannot figure out how to do that.
>>>> 
>>>> So the current situation is a compromise where most users get automatically a parallel build and the users who knowingly provide a -j <np> option get slightly peeved. (My guess is 90+% of people who install PETSc don't know about the -j option).  At least now it will explain that it is doing a parallel build.
>>>> 
>>>> I'll check on the -k option next.
>>>> 
>>>>  Barry
>>>> 
>>>> On Feb 25, 2013, at 2:58 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
>>>> 
>>>>> This is to support different backends. If configure found CMake, then normal "make" will use multiple jobs, or you can do it explicitly with make -j3 $PETSC_ARCH.
>>>>> 
>>>>> CMake would be terrible as a hard dependency (it's terrible enough as it is) but it's less bad than some alternatives.
>>>>> 
>>>>> 
>>>>> On Mon, Feb 25, 2013 at 3:30 PM, Tim Tautges <tautges at mcs.anl.gov> wrote:
>>>>> tautges at chronos:~/code/petsc-dev-moab$ make -j 8
>>>>> ********************* ERROR *************************
>>>>> Parallel build with 'make -j' is not supported
>>>>> ******************************************************
>>>>> make: *** [chk_makej] Error 1
>>>>> tautges at chronos:~/code/petsc-dev-moab$
>>>>> 
>>>>> 
>>>>> Why??? I can almost understand petsc's justification for its own autotools-looking python-based build system, but why its need to not support otherwise-standard make options?  Similarly, PETSC_MAKE_STOP_ON_ERROR instead of make -k?
>>>>> 
>>>>> - tim
>>>>> 
>>>>> --
>>>>> ================================================================
>>>>> "You will keep in perfect peace him whose mind is
>>>>>  steadfast, because he trusts in you."               Isaiah 26:3
>>>>> 
>>>>>             Tim Tautges            Argonne National Laboratory
>>>>>         (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
>>>>> phone (gvoice): (608) 354-1459      1500 Engineering Dr.
>>>>>            fax: (608) 263-4499      Madison, WI 53706
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>> 
>

More information about the petsc-dev mailing list