[petsc-dev] Diff of numbers for new test framework

[Barry Smith]

> On Feb 2, 2017, at 10:24 PM, Matthew Knepley <knepley at gmail.com> wrote:
> 
> How do we handle this?

  $PETSC_DIR/bin/petscdiff is used, it considers any floating point number to equal any other floating point number :-) 


> My Python stuff parses it and compares numbers
> using tolerances.

   How do you provide the tolerances in your Python stuff?  Do you provide different tolerances for different examples? For different numbers in the same example? How is that information passed to the tester?

    We can improve petscdiff, but we need specific suggestions on how do this. Specific difficulties include when the "correct" answer is 0 but "good enough answers may be negative or positive and what is good enough in that case?

   We could have a single "tolerance" for each test or to get complicated and have the "gold standard" output file could do stuff like   3.45+-.03 and petscdiff could be smart enough to parse that and do the right thing, but is that needed?


  Barry


> 
>   Thanks,
> 
>      Matt
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener