[petsc-dev] Test harness upgrade

[Barry Smith]

   See comments below in text, but I realized I should have brought something up about subtests and loops. Is there any assumption about the order that the subtests (entries) in loop are run? If so, why? Shouldn't each subtest be handed off to gnumake to decide when to run? i.e each subtest becomes its own script and each script its own rule for make to schedule?


> On Feb 10, 2017, at 5:41 PM, Scott Kruger <kruger at txcorp.com> wrote:
> 
> 
> 
> On 2/10/17 4:28 PM, Barry Smith wrote:
> 
>>> To illustrate the point here, I will use examples
>>> from ksp/ksp/examples/tutorials/ex10.c
>>> 
>>> Consider this test block:
>>> 
>>>  test:
>>>     suffix: 19
>>>     requires: datafilespath
>>>     args: -f0 ${DATAFILESPATH}/matrices/poisson1
>>>     args: -ksp_type cg -pc_type icc -pc_factor_levels 2
>>>     test:
>>>     test:
>> 
>>  Why is test listed twice here?
> 
> To invoke mpiexec twice.  The first invocation
> will have no additional args beyond what the parent
> has.  

    But why would I EVER want to run it again with the same arguments as the previous time? Run the same test with the same arguments?

> The second invocation adds an arg.
> 
>>> 
>>>  test:
>>>     requires: datafilespath
>>>     args: -f0 ${DATAFILESPATH}/matrices/medium
>>>     args: -ksp_type bicg
>>>     test:
>>>        suffix: 4
>>>        args: -pc_type lu
>>>     test:
>>>        suffix: 5
>>> 
>>> This will generate runex4.sh and runex5.sh that only
>>> differ in the '-pc_type lu' in the arguments and in
>>> the output file names.
>>> One can quickly see that they are very similar tests,
>>> and it's very compact in it's representation.
>> 
>>  So the indentation is what indicates it is a subtest versus a completely new test?
> 
> Indentation only: subtest (mpiexec/diff with info inherited from parent)
> Indentation+suffix: New test with info inherited from parent
> 
> 
> 
>>> Here is an example.  The key variable is
>>> separate_testvars which says that the loop
>>> variable should create separate tests.
>>> 
>>>  test:
>>>     suffix: 19
>>>     requires: datafilespath
>>>     args: -f0 ${DATAFILESPATH}/matrices/poisson1
>>>     args: -ksp_type cg -pc_type icc
>>>     args: -pc_factor_levels {{0 2 4}}
>>>     separate_testvars: pc_factor_levels
>> 
>>   Without the - ?
> 
> Yes.  Right now for loops have to be of the form:
>   -var {{values}}
> so it seemed less confusing to just specify var
> 
> 
>>>     test:
>>>        args:
>>>     test:
>>>        args: -mat_type seqsbaij
>>> 
>> 
>>   Instead of having the ugly separate_testvars argument could you somehow indicate it with the loop definition?
>> 
>>> args: -pc_factor_levels {{0 2 4}separate test output files}
>> 
>> and
>> 
>>> args: -pc_factor_levels {{0 2 4}shared test output file}
>> 
> 
> It's possible.
> 
> What about [[0 2 4]]  implying separate test output files?

   That works, the problem is that people have to remember what {{ means and and what [[ means. I would never be able to keep them straight, better to have a clear word that distinguishes them. I agree what I suggest is wordy, could be "shared/separate output" 


> 
> 
> 
> Scott
> 
> 
> -- 
> Tech-X Corporation               kruger at txcorp.com
> 5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
> Boulder, CO 80303                Fax:   (303) 448-7756