[petsc-dev] Testing seems broken in master

[Matthew Knepley]

On Thu, Dec 31, 2020 at 1:48 PM Barry Smith <bsmith at petsc.dev> wrote:

>
>   So the programs output changes and should no longer match that in the
> output/* file yet the test harness does not error with a statement that the
> two outputs do not match?
>
>    I noticed the gmakegentest.py is not being run before it runs the test?
> Does this mean it is just running all the old stuff which does match fine?
>
>    Then either how petscdiff is called by the test harness has changed or
> petscdiff has changed and does not detect changes anymore
>
>    BTW: I always use -f ./gmakefile.test test not just the gmakefile
>
>    All the PETSc changes are trivial and can be seen with a simple diff,
> it is hard to believe they would cause this behavior but I guess they must.
>
>    You can go to PETSC_ARCH/tests/snes/tests and run the ex13 shell script
> directly.
>

It is the sed problem:

master *$:/PETSc3/petsc/petsc-dev$
/PETSc3/petsc/petsc-dev/lib/petsc/bin/petscdiff
/PETSc3/petsc/petsc-dev/src/snes/tests/output/ex13_bench.out ex13_bench.tmp

sed: 1: "s/\033[1;31m//g": unbalanced brackets ([])
sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([])
sed: 1: "s/\033[1;31m//g": unbalanced brackets ([])
sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([])

The error was getting eaten.

This is in current master. Is it fixed in a branch?

   Matt


>   Barry
>
>
> On Dec 31, 2020, at 12:38 PM, Matthew Knepley <knepley at gmail.com> wrote:
>
> I just pulled master, and simple alterations to tests do not produce a
> failure:
>
> master *$:/PETSc3/petsc/petsc-dev$ PETSC_ARCH=arch-master-debug make -f
> ./gmakefile test search="snes_tests-ex13_bench" TIMEOUT=5000
> EXTRA_OPTIONS="-dm_
> refine 0"
> Using MAKEFLAGS: EXTRA_OPTIONS=-dm_refine 0 TIMEOUT=5000
> search=snes_tests-ex13_bench
>         TEST arch-master-debug/tests/counts/snes_tests-ex13_bench.counts
>  ok snes_tests-ex13_bench
>  ok diff-snes_tests-ex13_bench
>
> I check that the runs produce different output when done manually.
>
> Scott and Barry, could this be related to changed to testing?
>
>   Thanks,
>
>      Matt
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20201231/8ba2259b/attachment-0001.html>