[petsc-dev] Diff of numbers for new test framework

[Scott Kruger]

On 2/3/17 6:25 AM, Matthew Knepley wrote:
> On Thu, Feb 2, 2017 at 10:36 PM, Barry Smith <bsmith at mcs.anl.gov
> <mailto:bsmith at mcs.anl.gov>> wrote:
>
>
>
>     > On Feb 2, 2017, at 10:24 PM, Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>> wrote:
>     >
>     > How do we handle this?
>
>       $PETSC_DIR/bin/petscdiff is used, it considers any floating point
>     number to equal any other floating point number :-)
>
>
>     > My Python stuff parses it and compares numbers
>     > using tolerances.
>
>        How do you provide the tolerances in your Python stuff?  Do you
>     provide different tolerances for different examples? For different
>     numbers in the same example? How is that information passed to the
>     tester?
>
>         We can improve petscdiff, but we need specific suggestions on
>     how do this. Specific difficulties include when the "correct" answer
>     is 0 but "good enough answers may be negative or positive and what
>     is good enough in that case?
>
>
> I right now do the dumbest thing which is to have a fixed relative and
> absolute tolerance for every comparison. This works MUCH better
> than exact comparison and is not hard. I say we start with that.


Matt:
If you add a petscdiff.py to bin that has the same arguments as
the current shell script plus atol and rtol, then I can hook up atol and
rtol keywords in the test harness.

Scott



>
>   Matt
>
>
>        We could have a single "tolerance" for each test or to get
>     complicated and have the "gold standard" output file could do stuff
>     like   3.45+-.03 and petscdiff could be smart enough to parse that
>     and do the right thing, but is that needed?
>
>
>       Barry
>
>
>     >
>     >   Thanks,
>     >
>     >      Matt
>     >
>     > --
>     > What most experimenters take for granted before they begin their
>     experiments is infinitely more interesting than any results to which
>     their experiments lead.
>     > -- Norbert Wiener
>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener

-- 
Tech-X Corporation               kruger at txcorp.com
5621 Arapahoe Ave, Suite A       Phone: (720) 974-1841
Boulder, CO 80303                Fax:   (303) 448-7756