[petsc-dev] every test example runs in a new directory with new test harness

[Scott Kruger]

The basic idea of running multiple commands within a single shell
script was what I called a subtest (for lack of a better word).
So:


  test:
     suffix: restart
     requires: hdf5
     args: -run_type test -refinement_limit 0.0    -bc_type dirichlet 
-interpolate 1 -petscspace_order 1
     test:
         args: -dm_view hdf5:sol.h5 -vec_view hdf5:sol.h5::append
     test:
         args: -f sol.h5 -restart

The args in the subtest inherit from the parent test.  This seems
to be generally useful as a testing idiom in petsc tests as this
example nicely shows.

Each mpiexec would be tested separately and reported separately.
This would give you want you want, and should work as is.


Tobin pointed out that I broke the for loops and some of the subtest
functionality in some of the other feature implementations.  We
have come to consensus (right, Tobin?) on the
desired functionality and implementation.  A pull request
is planned this week.  It doesn't affect this directly, but
should have some minor improvements (like in the reporting).

Scott


On 2/6/17 7:10 AM, Matthew Knepley wrote:
> On Mon, Feb 6, 2017 at 1:05 AM, Jed Brown <jed at jedbrown.org
> <mailto:jed at jedbrown.org>> wrote:
>
>     Barry Smith <bsmith at mcs.anl.gov <mailto:bsmith at mcs.anl.gov>> writes:
>
>     >   test:
>     >     suffix: restart_0
>     >     requires: hdf5
>     >     args: -run_type test -refinement_limit 0.0    -bc_type dirichlet -interpolate 1 -petscspace_order 1 -dm_view hdf5:sol.h5 -vec_view hdf5:sol.h5::append
>     >
>     >   test:
>     >     suffix: restart_1
>     >     requires: hdf5
>     >     args: -run_type test -refinement_limit 0.0    -bc_type dirichlet -interpolate 1 -petscspace_order 1 -f sol.h5 -restart
>     >
>     > See a problem?
>     >
>     > Should the same run of the example view the files and then load them back in? versus trying to read in a data file from another run that may not even have been created before and even if it was, the file was definitely created  in a different directory?
>
>     So if write only is broken, do you want both to fail?  I think it's
>     better to read and write separately, with comparison using h5diff, since
>     that independently tests read vs write and establishes backward
>     compatibility, which you'd really like the test system to make you deal
>     with explicitly.
>
>
> I know the test is broken, but I did already mail the list about this
> and was waiting for an answer
> to be worked out.
>
> I agree with Satish that running two commands would be great. I could
> rewrite the example to
> both write and load it, but it would complicate it. Also, I am trying to
> get the pattern I expect the
> user to follow for checkpointing.
>
>    Matt
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener

-- 
Tech-X Corporation               kruger at txcorp.com
5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
Boulder, CO 80303                Fax:   (303) 448-7756