[petsc-dev] every test example runs in a new directory with new test harness

[Satish Balay]

I just tried it - and it doesn't appear to work..

Should the keyword for 'subtest' be different than 'test:'?

Satish

-----------------------------------------------------
balay at asterix /home/balay/petsc (next *=)
$ git diff|cat
diff --git a/src/snes/examples/tutorials/ex12.c b/src/snes/examples/tutorials/ex12.c
index 71166bf..c097018 100644
--- a/src/snes/examples/tutorials/ex12.c
+++ b/src/snes/examples/tutorials/ex12.c
@@ -1168,10 +1168,8 @@ int main(int argc, char **argv)
     suffix: restart_0
     requires: hdf5
     args: -run_type test -refinement_limit 0.0    -bc_type dirichlet -interpolate 1 -petscspace_order 1 -dm_view hdf5:sol.h5 -vec_view hdf5:sol.h5::append
-  test:
-    suffix: restart_1
-    requires: hdf5
-    args: -run_type test -refinement_limit 0.0    -bc_type dirichlet -interpolate 1 -petscspace_order 1 -f sol.h5 -restart
+    test:
+      args: -run_type test -refinement_limit 0.0    -bc_type dirichlet -interpolate 1 -petscspace_order 1 -f sol.h5 -restart
   # Periodicity
   test:
     suffix: periodic_0
balay at asterix /home/balay/petsc (next *=)
$ ./config/gmakegentest.py 
balay at asterix /home/balay/petsc (next *=)
$ cat arch-next/tests/src/snes/examples/tutorials/runex12_restart_0.sh 
#!/usr/bin/env bash
# This script was created by gmakegentest.py

# PATH for DLLs on windows
PATH="$PATH":"/home/balay/petsc/arch-next/lib"

mpiexec='mpiexec'
exec='../ex12'
testname='runex12_restart_0'
label='snes_tutorials-ex12_restart_0'
runfiles=''
wPETSC_DIR='/home/balay/petsc'
petsc_dir='/home/balay/petsc'
args='-run_type test -refinement_limit 0.0    -bc_type dirichlet -interpolate 1 -petscspace_order 1 -dm_view hdf5:sol.h5 -vec_view hdf5:sol.h5::append'

. "${petsc_dir}/config/petsc_harness.sh"

# The diff flags come from script arguments
diff_exe="${petsc_dir}/bin/petscdiff ${diff_flags}"


printf "ok ${label} # SKIP PETSC_HAVE_HDF5 requirement not met\n"
total=1; skip=1
petsc_testend "/home/balay/petsc/arch-next/tests" 
exit


petsc_testrun "${mpiexec} -n 1 ${exec} ${args}" ex12_restart_0.tmp ${testname}.err "${label}" ''

petsc_testrun "${diff_exe} /home/balay/petsc/src/snes/examples/tutorials/output/ex12_restart_0.out ex12_restart_0.tmp" diff-${testname}.out diff-${testname}.out diff-${label} ""

petsc_testend "/home/balay/petsc/arch-next/tests" 
balay at asterix /home/balay/petsc (next *=)
$ 


On Mon, 6 Feb 2017, Scott Kruger wrote:

> 
> The basic idea of running multiple commands within a single shell
> script was what I called a subtest (for lack of a better word).
> So:
> 
> 
>  test:
>     suffix: restart
>     requires: hdf5
>     args: -run_type test -refinement_limit 0.0    -bc_type dirichlet 
> -interpolate 1 -petscspace_order 1
>     test:
>         args: -dm_view hdf5:sol.h5 -vec_view hdf5:sol.h5::append
>     test:
>         args: -f sol.h5 -restart
> 
> The args in the subtest inherit from the parent test.  This seems
> to be generally useful as a testing idiom in petsc tests as this
> example nicely shows.
> 
> Each mpiexec would be tested separately and reported separately.
> This would give you want you want, and should work as is.
> 
> 
> Tobin pointed out that I broke the for loops and some of the subtest
> functionality in some of the other feature implementations.  We
> have come to consensus (right, Tobin?) on the
> desired functionality and implementation.  A pull request
> is planned this week.  It doesn't affect this directly, but
> should have some minor improvements (like in the reporting).
> 
> Scott
> 
> 
> On 2/6/17 7:10 AM, Matthew Knepley wrote:
> > On Mon, Feb 6, 2017 at 1:05 AM, Jed Brown <jed at jedbrown.org
> > <mailto:jed at jedbrown.org>> wrote:
> >
> >     Barry Smith <bsmith at mcs.anl.gov <mailto:bsmith at mcs.anl.gov>> writes:
> >
> > >   test:
> > >     suffix: restart_0
> > >     requires: hdf5
> > >     args: -run_type test -refinement_limit 0.0    -bc_type dirichlet
> > >     -interpolate 1 -petscspace_order 1 -dm_view hdf5:sol.h5 -vec_view
> > >     hdf5:sol.h5::append
> > >
> > >   test:
> > >     suffix: restart_1
> > >     requires: hdf5
> > >     args: -run_type test -refinement_limit 0.0    -bc_type dirichlet
> > >     -interpolate 1 -petscspace_order 1 -f sol.h5 -restart
> > >
> > > See a problem?
> > >
> > > Should the same run of the example view the files and then load them back
> > > in? versus trying to read in a data file from another run that may not
> > > even have been created before and even if it was, the file was definitely
> > > created  in a different directory?
> >
> >     So if write only is broken, do you want both to fail?  I think it's
> >     better to read and write separately, with comparison using h5diff, since
> >     that independently tests read vs write and establishes backward
> >     compatibility, which you'd really like the test system to make you deal
> >     with explicitly.
> >
> >
> > I know the test is broken, but I did already mail the list about this
> > and was waiting for an answer
> > to be worked out.
> >
> > I agree with Satish that running two commands would be great. I could
> > rewrite the example to
> > both write and load it, but it would complicate it. Also, I am trying to
> > get the pattern I expect the
> > user to follow for checkpointing.
> >
> >    Matt
> >
> > --
> > What most experimenters take for granted before they begin their
> > experiments is infinitely more interesting than any results to which
> > their experiments lead.
> > -- Norbert Wiener
> 
>