[petsc-dev] Test harness upgrade

[Scott Kruger]

OK.
Down with subtest -- Long live testset.
Pushed with updated documentation.

Scott


On 2/12/17 2:30 PM, Barry Smith wrote:
>
>> On Feb 12, 2017, at 12:34 PM, Scott Kruger <kruger at txcorp.com> wrote:
>>
>>>>>
>>>>> Why is test listed twice here?
>>>>
>>>> To invoke mpiexec twice.  The first invocation
>>>> will have no additional args beyond what the parent
>>>> has.
>>>
>>>    But why would I EVER want to run it again with the same arguments as the previous time? Run the same test with the same arguments?
>>
>> There is a confusion here.
>
>    Indeed.
>
>> When there are subtest blocks, the parent block does not generate any tests.
>>
>> Perhaps  a better specification would be to change the name of the top-level block since it is not generating a test:
>>
>>   container:
>>      suffix: 19
>>      requires: datafilespath
>>      args: -f0 ${DATAFILESPATH}/matrices/poisson1
>>      args: -ksp_type cg -pc_type icc
>>      args: -pc_factor_levels {{0 2 4}}
>>      separate_testvars: pc_factor_levels
>>      test:
>>      test:
>>         args: -mat_type seqsbaij
>>
>> Two mpiexec invocations, same output, but wanting to test minor permutations of arguments.
>>
>> Other examples in ex10 that are of the same ilk are
>> ILU, cg_singlereduction, mumps_cholesky, mumps_lu, pastix_lu, zeropivot
>>
>> It's more common though to have arguments change
>> the output requiring a new suffix:
>> {2,3},{4,5}, 7{a,b,c}, {13,14,15,17}, etc.
>>
>> Instead of container, perhaps "testset" would
>> be more descriptive.
>
>   I'm ok with testset
>>
>>
>>>>
>>>>>>    test:
>>>>>>       args:
>>>>>>    test:
>>>>>>       args: -mat_type seqsbaij
>>>>>>
>>>>>
>>>>>  Instead of having the ugly separate_testvars argument could you somehow indicate it with the loop definition?
>>>>>
>>>>>> args: -pc_factor_levels {{0 2 4}separate test output files}
>>>>>
>>>>> and
>>>>>
>>>>>> args: -pc_factor_levels {{0 2 4}shared test output file}
>>>>>
>>>>
>>>> It's possible.
>>>>
>>>> What about [[0 2 4]]  implying separate test output files?
>>>
>>>   That works, the problem is that people have to remember what {{ means and and what [[ means. I would never be able to keep them straight, better to have a clear word that distinguishes them. I agree what I suggest is wordy, could be "shared/separate output"
>>>
>>
>> So:
>> {{0 2 4}shared output}
>> and
>> {{0 2 4}separate output}
>>
>>
>> Would you accept {{0 2 4}} as being equivalent to {{0 2 4} shared ouput}
>
>   Yes
>
>> ?
>>
>> Scott
>>
>>
>>
>>>
>>>>
>>>>
>>>>
>>>> Scott
>>>>
>>>>
>>>> --
>>>> Tech-X Corporation               kruger at txcorp.com
>>>> 5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
>>>> Boulder, CO 80303                Fax:   (303) 448-7756
>>>
>>
>> --
>> Tech-X Corporation               kruger at txcorp.com
>> 5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
>> Boulder, CO 80303                Fax:   (303) 448-7756
>

-- 
Tech-X Corporation               kruger at txcorp.com
5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
Boulder, CO 80303                Fax:   (303) 448-7756