[petsc-dev] Experiments in test timing with new harness

[Smith, Barry F.]

  What branch? Master has only a small percentage of the examples in the new test harness.


> On Jan 5, 2018, at 10:11 PM, Scott Kruger <kruger at txcorp.com> wrote:
> 
> 
> 
> 
> On my laptop:
> 
> rm -rf $PETSC_ARCH/tests
> time make -f gmakefile.test test NO_RM=1
> 
> Results were this:
> 603.458u 191.569s 20:29.22 64.6%6030+0k 3297+2284io 224278pf+0w
> 
> Immediately redo (NO_RM=1 => no rebuilding of executables):
>      time make -f gmakefile.test test NO_RM=1
> Results were this:
> 280.566u 85.166s 15:54.69 38.3%Time0+0k 331+2653io 2135pf+0w
> 
> 
> (320 seconds shorter, or 53% shorter)
> 
> 
> Here are some tests with TIMEOUTS:
> TIMEOUT=10
> Removes (for me, see below):
> snes_tutorials-ex56_hypre sys_tutorials-ex3f_1
> 
> Shortens it to:
> 243.836u 91.350s 5:30.93 101.2%	0+0k 1295+1546io 7502pf+0w
> 
> 
> TIMEOUT=1
> Removes:
> snes_tutorials-ex13_2d_q3_0 snes_tutorials-ex13_3d_q3_0 snes_tutorials-ex56_0 sys_tutorials-ex3f sys_tutorials-ex3 snes_tutorials-ex13_3d_q2_0 ts_tutorials-ex18_p2p1_xyper_ref snes_tutorials-ex56_hypre ts_tests-ex4_2 ts_tutorials-ex26_2 ts_tests-ex4_4
> 
> Shortens it to:
> 214.718u 98.313s 5:02.38 103.5%	0+0k 597+1569io 7593pf+0w
> 
> (65 seconds shorter than 60 sec timeout; or ~23% shorter)
> 
> 
> The upshot is that improving test times means focusing
> on compile time.  While we cannot
> get rid of the compile time of the individual files,
> it is possible that having a single package executable,
> as Jed has advocated, could save a significant amount of
> time.   If we estimate that link time is 50% of
> the compile time, and that we can have a 50% savings by
> linking a single executable instead of multiple, then 25%
> time saving would occur with a single executable build.
> I have no idea if this is the right estimate, but it
> does motivate the single executable idea as the best
> method of improving the test timing, in addition to
> the disk space savings.
> 
> 
> 
> 
> Having tests take a "long time" is currently, for this
> configuration, not a big issue; however, it is important
> to note that while I have several packages, I certainly
> do not have exodus, for example.
> 
> 
> Here are my configuration options:
>  '--download-mpich=1',
>  '--with-fc=/opt/homebrew/bin/gfortran',
>  '--with-x=0',
>  '--with-cxx-dialect=C++11',
>  '--with-clanguage=C++',
>  '--with-debugging=0',
>  '--download-cmake=1',
>  '--download-hdf5',
>  '--download-netcdf',
>  '--download-hypre=1',
>  '--download-metis=1',
>  '--download-parmetis=1',
>  '--download-superlu_dist=1',
>  '--download-scalapack',
>  '--download-ptscotch',
>  '--download-mumps',
>  '--download-sowing=1',
>  '--with-shared-libraries=1',
>  '--DATAFILESPATH=/Users/kruger/petsc/datafiles'
> 
> 
> Scott
> 
> 
> -- 
> Tech-X Corporation               kruger at txcorp.com
> 5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
> Boulder, CO 80303                Fax:   (303) 448-7756