[petsc-users] false negatives ontests

[Jed Brown]

Testing basically ignores numeric values (too many false positives), but when a test fails, it shows the whole diff.

Mark Adams <mfadams at lbl.gov> writes:

> I run this test and it passes. But I know it's wrong so I* change a 6 to a
> 7 in output/ex11_0.out *and I get this diff AND all the rest.
> Any idea what is going on?
> Thanks,
>
> 12:25 mark/feature-xgc-interface-rebase *> ~/Codes/petsc-master$ make -f
> ./gmakefile test globsearch="dm_impls_plex_tutorials-ex11_0" PETSC_DIR=$PWD
> Using MAKEFLAGS: PETSC_DIR=/Users/markadams/Codes/petsc-master
> globsearch=dm_impls_plex_tutorials-ex11_0
>         TEST
> arch-macosx-gnu-g/tests/counts/dm_impls_plex_tutorials-ex11_0.counts
>  ok dm_impls_plex_tutorials-ex11_0
> not ok diff-dm_impls_plex_tutorials-ex11_0 # Error code: 1
> #       55c55
> #       <               ***** FormRHSSource: have new_imp_rate=  4.975e+01
> dt=0.125 stepi=1 time=  1.000e-01
> #       ---
> #       >               ***** FormRHSSource: have new_imp_rate=  4.975e-04
> dt=0.125 stepi=1 time=  1.000e-01
> #       64,76c64,76
> #       <     0 SNES Function norm 3.285336127013e-04
> #       <     1 SNES Function norm 9.604648941118e-07
> #       <     2 SNES Function norm 1.964241915657e-08
> #       <     3 SNES Function norm 8.824980789615e-10
> #       <     4 SNES Function norm 4.556683618010e-11
> #       <     5 SNES Function norm 2.470586731358e-12
> #       <     6 SNES Function norm 1.385600544214e-13
>
> *#       <   Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE
> iterations 7*#       <               ***** FormRHSSource: have
> new_imp_rate=  1.981e+03 dt=0.125 stepi=1 time=  1.625e-01
> #       < [0] TSAdaptChoose_Basic(): Estimated scaled local truncation
> error 0.466012, accepting step of size 0.125
> #       <   2) species-0: charge density= -1.6018377279742e+01 z-momentum=
> -2.8181973830519e-19 energy=  1.2008542897309e+05
> #       <   2) species-1: charge density=  1.6024885661178e+01 z-momentum=
> -9.0961762196061e-18 energy=  1.2013832642627e+05
> #       <         2) Total: charge density=  6.5083814358857e-03, momentum=
> -9.3779959579113e-18, energy=  2.4022375539935e+05 (m_i[0]/m_e = 3670.94,
> 80 cells)
> #       ---
> #       >     0 SNES Function norm 3.285336109721e-04
> #       >     1 SNES Function norm 9.602918060455e-07
> #       >     2 SNES Function norm 1.961138067634e-08
> #       >     3 SNES Function norm 8.793960138447e-10
> #       >     4 SNES Function norm 4.534301360955e-11
> #       >     5 SNES Function norm 2.455937690832e-12
> #       >     6 SNES Function norm 1.404599098408e-13
>
> *#       >   Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE
> iterations 6*#       >               ***** FormRHSSource: have
> new_imp_rate=  5.192e-04 dt=0.125 stepi=1 time=  1.625e-01
> #       > [0] TSAdaptChoose_Basic(): Estimated scaled local truncation
> error 0.674842, accepting step of size 0.125
> #       >   2) species-0: charge density= -1.6018371940813e+01 z-momentum=
> -6.0282835882130e-19 energy=  1.2008542888021e+05
> #       >   2) species-1: charge density=  1.6024885680151e+01 z-momentum=
> -9.8338166614696e-18 energy=  1.2013832641421e+05
> #       >         2) Total: charge density=  6.5137393379366e-03, momentum=
> -1.0436645020291e-17, energy=  2.4022375529442e+05 (m_i[0]/m_e = 3670.94,
> 80 cells)
> #       78c78
> #       < 2 TS dt 0.15625 time 0.225
> #       ---
> #       > 2 TS dt 0.136947 time 0.225
>
> # -------------
> #   Summary
> # -------------
> # FAILED diff-dm_impls_plex_tutorials-ex11_0
> # success 1/2 tests (50.0%)
> # failed 1/2 tests (50.0%)
> # todo 0/2 tests (0.0%)
> # skip 0/2 tests (0.0%)
> #
> # Wall clock time for tests: 48 sec
> # Approximate CPU time (not incl. build time): 47.74 sec
> #
> # To rerun failed tests:
> #     /usr/bin/make -f gmakefile test test-fail=1
> #
> # Timing summary (actual test time / total CPU time):