[petsc-dev] Experiments in test timing with new harness

[Scott Kruger]

This was essentially master.

Yes, these kinds of experiments will need to be redone as more tests get 
moved over.

Scott


On 1/5/18 11:11 PM, Smith, Barry F. wrote:
> 
>    What branch? Master has only a small percentage of the examples in the new test harness.
> 
> 
>> On Jan 5, 2018, at 10:11 PM, Scott Kruger <kruger at txcorp.com> wrote:
>>
>>
>>
>>
>> On my laptop:
>>
>> rm -rf $PETSC_ARCH/tests
>> time make -f gmakefile.test test NO_RM=1
>>
>> Results were this:
>> 603.458u 191.569s 20:29.22 64.6%6030+0k 3297+2284io 224278pf+0w
>>
>> Immediately redo (NO_RM=1 => no rebuilding of executables):
>>       time make -f gmakefile.test test NO_RM=1
>> Results were this:
>> 280.566u 85.166s 15:54.69 38.3%Time0+0k 331+2653io 2135pf+0w
>>
>>
>> (320 seconds shorter, or 53% shorter)
>>
>>
>> Here are some tests with TIMEOUTS:
>> TIMEOUT=10
>> Removes (for me, see below):
>> snes_tutorials-ex56_hypre sys_tutorials-ex3f_1
>>
>> Shortens it to:
>> 243.836u 91.350s 5:30.93 101.2%	0+0k 1295+1546io 7502pf+0w
>>
>>
>> TIMEOUT=1
>> Removes:
>> snes_tutorials-ex13_2d_q3_0 snes_tutorials-ex13_3d_q3_0 snes_tutorials-ex56_0 sys_tutorials-ex3f sys_tutorials-ex3 snes_tutorials-ex13_3d_q2_0 ts_tutorials-ex18_p2p1_xyper_ref snes_tutorials-ex56_hypre ts_tests-ex4_2 ts_tutorials-ex26_2 ts_tests-ex4_4
>>
>> Shortens it to:
>> 214.718u 98.313s 5:02.38 103.5%	0+0k 597+1569io 7593pf+0w
>>
>> (65 seconds shorter than 60 sec timeout; or ~23% shorter)
>>
>>
>> The upshot is that improving test times means focusing
>> on compile time.  While we cannot
>> get rid of the compile time of the individual files,
>> it is possible that having a single package executable,
>> as Jed has advocated, could save a significant amount of
>> time.   If we estimate that link time is 50% of
>> the compile time, and that we can have a 50% savings by
>> linking a single executable instead of multiple, then 25%
>> time saving would occur with a single executable build.
>> I have no idea if this is the right estimate, but it
>> does motivate the single executable idea as the best
>> method of improving the test timing, in addition to
>> the disk space savings.
>>
>>
>>
>>
>> Having tests take a "long time" is currently, for this
>> configuration, not a big issue; however, it is important
>> to note that while I have several packages, I certainly
>> do not have exodus, for example.
>>
>>
>> Here are my configuration options:
>>   '--download-mpich=1',
>>   '--with-fc=/opt/homebrew/bin/gfortran',
>>   '--with-x=0',
>>   '--with-cxx-dialect=C++11',
>>   '--with-clanguage=C++',
>>   '--with-debugging=0',
>>   '--download-cmake=1',
>>   '--download-hdf5',
>>   '--download-netcdf',
>>   '--download-hypre=1',
>>   '--download-metis=1',
>>   '--download-parmetis=1',
>>   '--download-superlu_dist=1',
>>   '--download-scalapack',
>>   '--download-ptscotch',
>>   '--download-mumps',
>>   '--download-sowing=1',
>>   '--with-shared-libraries=1',
>>   '--DATAFILESPATH=/Users/kruger/petsc/datafiles'
>>
>>
>> Scott
>>
>>
>> -- 
>> Tech-X Corporation               kruger at txcorp.com
>> 5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
>> Boulder, CO 80303                Fax:   (303) 448-7756
> 

-- 
Tech-X Corporation               kruger at txcorp.com
5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
Boulder, CO 80303                Fax:   (303) 448-7756