[petsc-dev] Diff of numbers for new test framework

[Matthew Knepley]

On Thu, Feb 2, 2017 at 10:36 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
>
> > On Feb 2, 2017, at 10:24 PM, Matthew Knepley <knepley at gmail.com> wrote:
> >
> > How do we handle this?
>
>   $PETSC_DIR/bin/petscdiff is used, it considers any floating point number
> to equal any other floating point number :-)
>
>
> > My Python stuff parses it and compares numbers
> > using tolerances.
>
>    How do you provide the tolerances in your Python stuff?  Do you provide
> different tolerances for different examples? For different numbers in the
> same example? How is that information passed to the tester?
>
>     We can improve petscdiff, but we need specific suggestions on how do
> this. Specific difficulties include when the "correct" answer is 0 but
> "good enough answers may be negative or positive and what is good enough in
> that case?
>

I right now do the dumbest thing which is to have a fixed relative and
absolute tolerance for every comparison. This works MUCH better
than exact comparison and is not hard. I say we start with that.

  Matt


>    We could have a single "tolerance" for each test or to get complicated
> and have the "gold standard" output file could do stuff like   3.45+-.03
> and petscdiff could be smart enough to parse that and do the right thing,
> but is that needed?
>
>
>   Barry
>
>
> >
> >   Thanks,
> >
> >      Matt
> >
> > --
> > What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> > -- Norbert Wiener
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20170203/ae695956/attachment.html>