[petsc-dev] Seeking OLCF users complaining about poor build times
Barry Smith
bsmith at mcs.anl.gov
Thu Feb 26 18:16:31 CST 2015
To make things simpler as Jed noted let's skip the --download-xxx completely and just do a basic configure and make on the systems all with master.
Barry
> On Feb 26, 2015, at 6:07 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
>
> One more thing. The tarballs for the external packages should be downloaded and then instead of --download-hypre you should use --download-hypre=/fullpathtohypre.tar.gz etc to eliminate the network time of downloading packages
>
> It may be if we do all this and get the numbers we will no longer have any right to bitch and moan about configure/make time on at least 2 of the systems.
>
> Barry
>
>> On Feb 26, 2015, at 5:59 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>
>>
>> Ok, based on what we have so far we need to completely rethink our process, our previous test results are pretty much useless Kill any tests you have outstanding at the moment.
>>
>> 1) Forget about maint branch!
>> 2) Forget about --download-superlu_dist since it does not have a parallel make
>>
>> So we want runs on master with --download-hypre --download-metis --download-parmetis --with-debugging=0 and good optimization flags and proper TMPDIRs, if appropriate, on all systems.
>>
>> Barry
>>
>>
>>> On Feb 26, 2015, at 5:47 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>>>
>>> And the following is for hopper at nersc. Note: the TMPDIR is on
>>> luster - so that might be contributing to long times - I'll have to
>>> redo some of this with a different TMPDIR
>>>
>>> Satish
>>>
>>> ----------------
>>> hopper.nersc MAKE_NP = 13
>>> ---------------
>>>
>>> balay at hopper03:~/petsc.clone> time ./reconfigure-arch-cray-xt6-pkgs-opt.py
>>> real 58m13.350s
>>> user 18m8.168s
>>> sys 28m3.945s
>>>
>>> balay at hopper03:~/petsc.clone> time make PETSC_DIR=/global/u2/b/balay/petsc.clone PETSC_ARCH=arch-cray-xt6-pkgs-opt all
>>>
>>> real 1m40.348s
>>> user 12m5.281s
>>> sys 1m38.422s
>>>
>>> balay at hopper03:~/petsc.clone> uptime
>>> 15:45pm up 8 days 0:55, 22 users, load average: 2.37, 3.14, 2.42
>>>
>>> balay at hopper03:~/petsc.clone> cat reconfigure-arch-cray-xt6-pkgs-opt.py
>>> #!/usr/bin/python
>>> if __name__ == '__main__':
>>> import sys
>>> import os
>>> sys.path.insert(0, os.path.abspath('config'))
>>> import configure
>>> configure_options = [
>>> '--COPTFLAGS=-fast -mp',
>>> '--CXXOPTFLAGS=-fast -mp',
>>> '--FOPTFLAGS=-fast -mp',
>>> '--download-hypre',
>>> '--download-metis',
>>> '--download-parmetis',
>>> '--download-superlu_dist',
>>> #'--with-blas-lapack-lib=-L/opt/acml/5.3.1/pgi64/lib -lacml',
>>> '--download-fblaslapack',
>>> '--with-cc=cc',
>>> '--with-clib-autodetect=0',
>>> '--with-cxx=CC',
>>> '--with-cxxlib-autodetect=0',
>>> '--with-debugging=0',
>>> '--with-fc=ftn',
>>> '--with-fortranlib-autodetect=0',
>>> '--with-shared-libraries=0',
>>> '--with-x=0',
>>> '--with-c2html=0',
>>> 'PETSC_ARCH=arch-cray-xt6-pkgs-opt',
>>> ]
>>> configure.petsc_configure(configure_options)
>>>
>>> On Thu, 26 Feb 2015, Satish Balay wrote:
>>>
>>>> Attaching logs.
>>>>
>>>> Note: 'uptime' is run a bit after the builds completed..
>>>>
>>>> Satish
>>>>
>>>> --------------
>>>> mira.alcf build: MAKE_NP = 32
>>>> --------------
>>>> [balay at miralac1 petsc.maint.35]$ time ./reconfigure-arch-time-test.py
>>>> <snip>
>>>> real 25m57.240s
>>>> user 17m33.944s
>>>> sys 6m41.065s
>>>>
>>>> [balay at miralac1 petsc.maint.35]$ time make PETSC_DIR=/gpfs/mira-home/balay/petsc.maint.35 PETSC_ARCH=arch-time-test all
>>>> <snip>
>>>> real 1m39.067s
>>>> user 22m56.248s
>>>> sys 7m42.658s
>>>>
>>>> [balay at miralac1 petsc.maint.35]$ uptime
>>>> 21:50:08 up 10 days, 2:14, 87 users, load average: 3.86, 5.02, 5.03
>>>>
>>>> --------------
>>>> Laptop build: MAKE_NP = 4
>>>> --------------
>>>> $ time ./configure --download-hypre --download-metis --download-parmetis --download-superlu_dist --download-fblaslapack --with-debugging=0 --with-c2html=0 PETSC_ARCH=arch-test COPTFLAGS="-O3 -fast" FOPTFLAGS="-O3 -fast" CXXOPTFLAGS="-O3 -fast"
>>>> <snip>
>>>> real 3m56.755s
>>>> user 2m47.458s
>>>> sys 0m57.206s
>>>>
>>>> $ time make PETSC_DIR=/home/balay/petsc PETSC_ARCH=arch-test all
>>>> <snip>
>>>> real 1m13.519s
>>>> user 3m40.708s
>>>> sys 0m47.002s
>>>>
>>>> $ uptime
>>>> 15:50:32 up 3 days, 5:20, 11 users, load average: 0.05, 0.30, 0.49
>>>>
>>>> --------------
>>>>
>>>> [balay at miralac1 petsc.maint.35]$ cat reconfigure-arch-time-test.py
>>>> #!/usr/bin/python
>>>> if __name__ == '__main__':
>>>> import sys
>>>> import os
>>>> sys.path.insert(0, os.path.abspath('config'))
>>>> import configure
>>>> configure_options = [
>>>> '--download-hypre=1',
>>>> '--download-metis=1',
>>>> '--download-parmetis=1',
>>>> '--download-superlu_dist=1',
>>>> #'--with-blas-lapack-lib=-L/soft/libraries/alcf/current/xl/LAPACK/lib -llapack -L/soft/libraries/alcf/current/xl/BLAS/lib -lblas',
>>>> '--download-fblaslapack=1',
>>>> '--with-c2html=0',
>>>> '--with-cc=mpixlc_r',
>>>> '--with-cxx=mpixlcxx_r',
>>>> '--with-debugging=0',
>>>> '--with-fc=mpixlf77_r -qnosave',
>>>> '--with-shared-libraries=0',
>>>> '--with-x=0',
>>>> '-COPTFLAGS= -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction',
>>>> '-CXXOPTFLAGS= -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction',
>>>> '-FOPTFLAGS= -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction',
>>>> 'PETSC_ARCH=arch-time-test',
>>>> ]
>>>> configure.petsc_configure(configure_options)
>>>>
>>>>
>>>>>>>> On Feb 26, 2015, at 8:13 AM, Nathan Collier <nathaniel.collier at gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Ok, so I built PETSc with metis, parmetis, superlu_dist, and hyper on
>>>>>>> Titan. The configure time is the second configure--when you run the
>>>>>>> reconfigure script that the batch submission generates for you.
>>>>>>>>
>>>>>>>> configure: 38m15.488s
>>>>>>>> make: 15m37.610s
>>>>>>>>
>>>>
>>> <configure_hopper.log.gz><make_hopper.log.gz>
>>
>
More information about the petsc-dev
mailing list