[petsc-dev] Experiments in test timing with new harness

[Scott Kruger]

On my laptop:

rm -rf $PETSC_ARCH/tests
time make -f gmakefile.test test NO_RM=1

Results were this:
603.458u 191.569s 20:29.22 64.6%6030+0k 3297+2284io 224278pf+0w

Immediately redo (NO_RM=1 => no rebuilding of executables):
       time make -f gmakefile.test test NO_RM=1
Results were this:
280.566u 85.166s 15:54.69 38.3%Time0+0k 331+2653io 2135pf+0w


(320 seconds shorter, or 53% shorter)


Here are some tests with TIMEOUTS:
TIMEOUT=10
Removes (for me, see below):
snes_tutorials-ex56_hypre sys_tutorials-ex3f_1

Shortens it to:
243.836u 91.350s 5:30.93 101.2%	0+0k 1295+1546io 7502pf+0w


TIMEOUT=1
Removes:
snes_tutorials-ex13_2d_q3_0 snes_tutorials-ex13_3d_q3_0 
snes_tutorials-ex56_0 sys_tutorials-ex3f sys_tutorials-ex3 
snes_tutorials-ex13_3d_q2_0 ts_tutorials-ex18_p2p1_xyper_ref 
snes_tutorials-ex56_hypre ts_tests-ex4_2 ts_tutorials-ex26_2 ts_tests-ex4_4

Shortens it to:
214.718u 98.313s 5:02.38 103.5%	0+0k 597+1569io 7593pf+0w

(65 seconds shorter than 60 sec timeout; or ~23% shorter)


The upshot is that improving test times means focusing
on compile time.  While we cannot
get rid of the compile time of the individual files,
it is possible that having a single package executable,
as Jed has advocated, could save a significant amount of
time.   If we estimate that link time is 50% of
the compile time, and that we can have a 50% savings by
linking a single executable instead of multiple, then 25%
time saving would occur with a single executable build.
I have no idea if this is the right estimate, but it
does motivate the single executable idea as the best
method of improving the test timing, in addition to
the disk space savings.




Having tests take a "long time" is currently, for this
configuration, not a big issue; however, it is important
to note that while I have several packages, I certainly
do not have exodus, for example.


Here are my configuration options:
   '--download-mpich=1',
   '--with-fc=/opt/homebrew/bin/gfortran',
   '--with-x=0',
   '--with-cxx-dialect=C++11',
   '--with-clanguage=C++',
   '--with-debugging=0',
   '--download-cmake=1',
   '--download-hdf5',
   '--download-netcdf',
   '--download-hypre=1',
   '--download-metis=1',
   '--download-parmetis=1',
   '--download-superlu_dist=1',
   '--download-scalapack',
   '--download-ptscotch',
   '--download-mumps',
   '--download-sowing=1',
   '--with-shared-libraries=1',
   '--DATAFILESPATH=/Users/kruger/petsc/datafiles'


Scott


-- 
Tech-X Corporation               kruger at txcorp.com
5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
Boulder, CO 80303                Fax:   (303) 448-7756