[petsc-dev] bullshit in Plex example under complex

[Jed Brown]

Barry Smith <bsmith at mcs.anl.gov> writes:
>   It seems to require a lot of boilerplate for each test. I sure as heck hope we don't have to write that by hand for each test.

Here is an example:

$ cat src/hpgmg/finite-element/test/t230-fmg-poisson2.sh 
#!/bin/sh

test_description='Test Q2 Poisson solve using FMG'

. ./hpgmg-sharness.sh

# Error norm is converging at fourth order (superconvergent at Lagrange nodes)
test_expect_stdout 'FE Poisson FMG solve fedegree=2 serial' 1 'hpgmg-fe fmg -op_type poisson2 -M 4,4,6 -smooth 4,3' '
F(4,3)  0: |e|_2/|u|_2 9.08e-03  |r|_2/|f|_2 3.35e-04
V(4,3)  1: |e|_2/|u|_2 9.17e-03  |r|_2/|f|_2 8.27e-07
V(4,3)  2: |e|_2/|u|_2 9.17e-03  |r|_2/|f|_2 5.54e-09
'

test_expect_stdout 'FE Poisson FMG solve fedegree=2 parallel' 4 'hpgmg-fe fmg -op_type poisson2 -M 4,4,6 -smooth 4,3 -p 1,2,2' '
F(4,3)  0: |e|_2/|u|_2 9.08e-03  |r|_2/|f|_2 3.35e-04
V(4,3)  1: |e|_2/|u|_2 9.17e-03  |r|_2/|f|_2 8.27e-07
V(4,3)  2: |e|_2/|u|_2 9.17e-03  |r|_2/|f|_2 5.54e-09
'

test_done


Since I have quite short output here, the test output is inline.

>   I'm not a fan of the "parse the PETSc makefiles approach"; I think we need to define a standard format for declaring tests that is very concise (only exactly what is needed; I have one try in a branch somewhere). Then we can have code that translates from the standard format to anything people want to try with testing scripts. I am even willing to manually convert makefile crap to this format when necessary 

My "parse the makefiles" was intended to automate the conversion to a
more concise format that a (hopefully sane) test system can use
directly.  I'm not wild about introducing extra translation phases
because it makes the testing system more difficult to understand and
exposes more possible failure modes.

FWIW, we could greatly simplify the makefiles by having each target
simply call a script that runs the actual program, collects stdout and
stderr and exit codes, and outputs the test result in a standard format
(e.g., TAP).  I.e., move the repetitive shell code we have now in each
target into a single script used for all tests.  This script could also
have an argument containing restrictions (NoComplex, etc.) so that it
can determine when to skip.  Moreover, it can sandbox the execution to a
temporary directory and clean up after it.  As I see it, this is
basically equivalent to moving the tests into a sharness-type system or
similar.

In my opinion, one crucial component in a revamped testing system is to
enable parallel execution with a maximum number of processes (perhaps
exceeded when a single job needs more than that, but definitely not
otherwise).  Sharness is really nice for tests that replicate a session
where the state is tested as various checkpoints along the way.  We have
a few of these (e.g., creating and reading files), but most of our tests
are independent, which maps to sharness as independent test scripts.
One file per test is actually the best case for merging, but can be
cumbersome otherwise.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20150829/ffbb7ca1/attachment.sig>