[petsc-dev] Introducing new "test harness" to PETSc

[Matthew Knepley]

On Fri, Jan 13, 2017 at 4:16 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
>    PETSc developers,
>
>      Scott Kruger has been implementing a new "test harness" for PETSc.
> Rather than having tests implemented as bash lines in the makefiles, the
> tests will be defined within the source code for each example with a simple
> format and the actual scripts that run the tests will be automatically
> generated. This offers three advantages over the current system
>
> 1) test definitions can be given in a very concise format that is
> independent of how the test is run
> 2) multiple approaches to actually running the tests can be implemented
> without requiring any changes to the test definitions for each example.
> 3) the tests can be compiled and run in parallel (eventually we hope to
> see much faster times for testing).
>
>    Here is an example of the format that defines tests
>
>      test:
>
>    test:
>       suffix: yaml
>       requires: yaml !complex
>       args: -options_file_yaml bag.yml -options_view
>       filter:  grep -v saws_port_auto_select
>       localrunfiles: bag.yml
>

How do I specify the number of processes to use for the test? Or where
would I look this up?

  Thanks,

     Matt


> The first is a test that takes no command line arguments and does no
> filtering of output. The second takes command line arguments, does some
> filtering of the output and this example also requires a data file
> associated with that example.  The expected output of tests continues to
> reside in the output directory with the previously naming convention of
> XXX_1.out XXX_2.out where XXX is the name of the example or XXX_suffix.out
> where suffix is defined in the test definition.
>
>   So far in the master branch we have only converted over two directories
> of examples sys/examples/tutorials and sys/examples/tests so you can look
> in those directories for other examples. Over the next few weeks we hope to
> convert over all the rest of the directories.
>
>   Special thanks to Scott, Satish, and Jed for all the work they have done
> to set up the new test harness.
>
>    Barry
>
>    Technical details: the new python script ./config/gmakegentest.py which
> is normally automatically run at configure walks through the examples
> directories looking for the formatted test definitions, for each test it
> generates in the directory $PETSC_ARCH/test/src/XXX/examples/tests or
> tutorials/ a shell script that can run that test. In addition
> config/gmakegentest.py  adds dependencies to the $PETSC_DIR/gmakefile for
> all the tests that it has found. The rule make alltests then executes all
> those dependencies.
>
>    We will be refining both the testing harness and the test definitions
> as we convert the rest of the examples over to the new approach.
>
>
>
>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20170117/866d026b/attachment.html>