[petsc-dev] plans for testing examples?

[Jed Brown]

On Mon, Sep 17, 2012 at 9:44 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:

> One reason I like everything in one file/place is that now lots of "test"
> examples are in the "tutorial" directories. To move them requires:
>
> 1)  hg move the example code
> 2) hg move several output/* files
> 3) edit the "tutorials" makefile to remove stuff
> 4) edit the "test" makefile to add the stuff
>
>   and plus since there will be name conflicts for source code names one
> will also need to change, for example, ex1.c to ex37.c in all these places.
>

I think the dumb numbers were a bad idea in the first place. There are way
too many examples that are ("this is a less maintained copy of some other
Bratu example"). How are we going to update inline output? We could write
an Emacs minor mode that updated it on file save, but to keep file save
fast enough, the output would have to be lying around. We'd also have to be
careful not to accidentally check in bad output. I hate the idea of having
to manually copy and paste into some piece of the output.


>   If everything was in one file I could just hg move that one file to the
> new location (changing the name of the file if needed). Sooo much easier.
> Why do you think no one has fixed the cluster fuck we have currently.
>

What about storing a SHA1 of the reference output in the source file (or
wherever the test was specified). Then have one big store of outputs for
everything, indexed by SHA1. The output store need not even be part of the
Hg checkout, it could be stored in a separate repository. When reference
output is updated, a new file is created in the archive and the SHA1 at the
test specification is updated. We could make a more convenient diff tool,
but this allows moving files around willy-nilly, including taking them
entirely outside of PETSc, without needing to futz with the test output
files.

If we did something like this, I'd be more comfortable specifying tests
with markup inside the source file (though I'd rather that not be the
_only_ way to specify tests).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20120917/77e22fd2/attachment.html>