[petsc-dev] make -j not supported? Why???

Karl Rupp rupp at mcs.anl.gov
Tue Feb 26 07:31:54 CST 2013 

Hi,

yes, sure. The downside, however, is that it probably is not very much 
backwards compatible. I'll report...

Best regards,
Karli



On 02/25/2013 08:51 PM, Barry Smith wrote:
>
> On Feb 25, 2013, at 7:18 PM, Karl Rupp <rupp at mcs.anl.gov> wrote:
>
>> Hi,
>>
>>>    Is it still not possible to use cmake to build cuda stuff? That's crazy?
>>
>> It is possible, but you cannot specify nvcc as CMAKE_CXX_COMPILER, but instead the script is doing some magic hackery replacing the C++-compiler with nvcc if the file extension is .cu.
>
>    Karl,
>
>     Ok, can you take a look at how Jed's cmake stuff is working and see if support for handling the .cu can be added?
>
>      Thanks
>
>     Barry
>
>>
>> (I consider this to be a hack rather than a clean solution, but that's the status quo)
>>
>> Best regards,
>> Karli
>>
>>>
>>> On Feb 25, 2013, at 5:06 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>>>
>>>> Parallel build prints someting like:
>>>>
>>>>>>>>>>>>
>>>> ==========================================
>>>> Building PETSc using CMake with 5 build threads
>>>> ==========================================
>>>> Re-run cmake file: Makefile older than: ../CMakeLists.txt
>>>> -- Configuring done
>>>> -- Generating done
>>>> -- Build files have been written to: /home/balay/spetsc/asterix64
>>>> Scanning dependencies of target petsc
>>>> [  0%] Building C object CMakeFiles/petsc.dir/src/sys/classes/draw/utils/lgc.c.o
>>>> [  1%] Building C object CMakeFiles/petsc.dir/src/mat/impls/mffd/mffd.c.o
>>>> [  1%] [  1%] Building C object CMakeFiles/petsc.dir/src/dm/impls/plex/ftn-auto/plexcreatef.c.o
>>>>
>>>> <<<<<<<<<
>>>>
>>>> etc..
>>>>
>>>> Note: if you are building cuda stuff or have python < 2.5 - then you
>>>> are using legacy build [i.e sequential/recursive make] and see stuff
>>>> like:
>>>>
>>>>>>>>>>
>>>> libfast in: /home/balay/spetsc/src
>>>> libfast in: /home/balay/spetsc/src/sys
>>>> libfast in: /home/balay/spetsc/src/sys/classes
>>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer
>>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls
>>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/socket
>>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/socket/ftn-custom
>>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii
>>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii/ftn-auto
>>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/ascii/ftn-custom
>>>> libfast in: /home/balay/spetsc/src/sys/classes/viewer/impls/binary
>>>> <<<<
>>>> etc..
>>>>
>>>>
>>>> Satish
>>>>
>>>> On Mon, 25 Feb 2013, Nystrom, William D wrote:
>>>>
>>>>> Is there a way to examine the petsc build logs and determine if I am really getting parallel builds?  I have
>>>>> a top level script that I use to the petsc configure script followed by make.  For a couple of our test bed
>>>>> systems that stay fairly unloaded, I use the following configure option: --with-make-np=16.  But I also
>>>>> invoke my script like this: /usr/bin/time -p makePetscDev_caddy >& makePetscDev_caddy.log.  And
>>>>> I'll get a time like the following:
>>>>>
>>>>> real 2160.77
>>>>> user 1308.21
>>>>> sys 234.37
>>>>>
>>>>> So it does not seem like I am getting much if any parallelism in my build.  I am also building with lots
>>>>> of external packages and have no idea if parallel builds are being done with them.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Dave
>>>>>
>>>>> --
>>>>> Dave Nystrom
>>>>> LANL HPC-5
>>>>> Phone: 505-667-7913
>>>>> Email: wdn at lanl.gov
>>>>> Smail: Mail Stop B272
>>>>>        Group HPC-5
>>>>>        Los Alamos National Laboratory
>>>>>        Los Alamos, NM 87545
>>>>>
>>>>>
>>>>> ________________________________________
>>>>> From: petsc-dev-bounces at mcs.anl.gov [petsc-dev-bounces at mcs.anl.gov] on behalf of Barry Smith [bsmith at mcs.anl.gov]
>>>>> Sent: Monday, February 25, 2013 3:25 PM
>>>>> To: For users of the development version of PETSc
>>>>> Subject: Re: [petsc-dev] make -j not supported? Why???
>>>>>
>>>>>   Tim,
>>>>>
>>>>>    As usual, a too short error message without enough information. I've changed it so that it will now give you
>>>>>
>>>>> Barrys-MacBook-Pro:petsc-dev barrysmith$ make -j 3 ccmake
>>>>> ********************* ERROR *************************
>>>>> PETSc compiles are automatically parallel, do not
>>>>> provide the -j option to make
>>>>> ******************************************************
>>>>> make: *** [chk_makej] Error 1
>>>>> Barrys-MacBook-Pro:petsc-dev barrysmith$ make -j 3 all-legacy
>>>>> ********************* ERROR *************************
>>>>> Parallel build with 'make -j' is not supported
>>>>> for PETSc legacy builds. Run without -j <np>
>>>>> or ./configure PETSc with --download-cmake
>>>>> to enable parallel builds with PETSc
>>>>> ******************************************************
>>>>> make[1]: *** [chk_makej] Error 1
>>>>> make: *** [all-legacy] Error 2
>>>>> Barrys-MacBook-Pro:petsc-dev barrysmith$
>>>>>
>>>>> Explanations:
>>>>> When PETSc is using cmake makefiles from the top level makefiles we pass our own -j <np> down, make doesn't like having two levels of -j and at best prints a confusing warning message about the parallel build.
>>>>>
>>>>> When using PETSc's legacy make system it uses a recursive make that is not able to be parallelized (yes it is bad and hence the cmake version is used by default now).
>>>>>
>>>>> Notes:
>>>>> It would be nice if we could somehow remove the -j <np> at the top level so that it didn't generate the error message and just ran but make doesn't allow this type of thing.
>>>>>
>>>>> It would be nice if we could eliminate the messages like
>>>>>
>>>>> make[1]: *** [chk_makej] Error 1
>>>>> make: *** [all-legacy] Error 2
>>>>>
>>>>> that make generates automatically but I cannot figure out how to do that.
>>>>>
>>>>> So the current situation is a compromise where most users get automatically a parallel build and the users who knowingly provide a -j <np> option get slightly peeved. (My guess is 90+% of people who install PETSc don't know about the -j option).  At least now it will explain that it is doing a parallel build.
>>>>>
>>>>> I'll check on the -k option next.
>>>>>
>>>>>   Barry
>>>>>
>>>>> On Feb 25, 2013, at 2:58 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
>>>>>
>>>>>> This is to support different backends. If configure found CMake, then normal "make" will use multiple jobs, or you can do it explicitly with make -j3 $PETSC_ARCH.
>>>>>>
>>>>>> CMake would be terrible as a hard dependency (it's terrible enough as it is) but it's less bad than some alternatives.
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 25, 2013 at 3:30 PM, Tim Tautges <tautges at mcs.anl.gov> wrote:
>>>>>> tautges at chronos:~/code/petsc-dev-moab$ make -j 8
>>>>>> ********************* ERROR *************************
>>>>>> Parallel build with 'make -j' is not supported
>>>>>> ******************************************************
>>>>>> make: *** [chk_makej] Error 1
>>>>>> tautges at chronos:~/code/petsc-dev-moab$
>>>>>>
>>>>>>
>>>>>> Why??? I can almost understand petsc's justification for its own autotools-looking python-based build system, but why its need to not support otherwise-standard make options?  Similarly, PETSC_MAKE_STOP_ON_ERROR instead of make -k?
>>>>>>
>>>>>> - tim
>>>>>>
>>>>>> --
>>>>>> ================================================================
>>>>>> "You will keep in perfect peace him whose mind is
>>>>>>   steadfast, because he trusts in you."               Isaiah 26:3
>>>>>>
>>>>>>              Tim Tautges            Argonne National Laboratory
>>>>>>          (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
>>>>>> phone (gvoice): (608) 354-1459      1500 Engineering Dr.
>>>>>>             fax: (608) 263-4499      Madison, WI 53706
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

More information about the petsc-dev mailing list