[petsc-dev] Nightly tests quick summary page

[Jed Brown]

On Wed, Jan 23, 2013 at 9:08 PM, Matthew Knepley <knepley at gmail.com> wrote:

> I am always skeptical of big programs to overall a large piece of
> infrastructure that works fairly well.
>
> However, there is a really simple thing that we need which would make us
> much much much better
> at using our own tests. We need a better way to test numerical output. I
> am not sure what the right
> thing to do is, or I would have already done it.
>
> The current best solution is to print fewer digits, which judging from the
> HTML page is not sufficient.
> I think that current PETSc output is so stylized that parsing output is
> feasible, and would allow nice
> diffs with tolerances tailored to the type of output and individual test.
>

MOOSE puts all their test output into exodus files and uses exodiff. That
has the advantage of being structured enough that it can be diffed with
rtol and atol.

OTOH, we have a challenge that's mostly distinct from a discretization
package. We're not testing error in a discretization (which is unchanging
as long as the discretization doesn't change), we're testing the
intermediate, unconverged values, and comparing error using relative
tolerance (versus absolute tolerance, which would be better).

As we attempt to make our interfaces better for graphical front-ends and
automatic high-level controllers, I think we should try to use monitors
that provide structured output. This could be a JSON file with object
identification and convergence history or perhaps a sqlite database. I
suspect we could deal with most of our FP-sensitive testing with only a
handful of structured monitors. Providing this structured output is
providing an API so we should try to rapidly converge on an extensible data
model that can be relatively stable.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130123/8001b4d0/attachment.html>