[petsc-dev] Testing seems broken in master

[Barry Smith]

  This is a different (3rd) problem. Funny it didn't bother anyone for two months. 

  Fix is in barry/2020-12-29/fix-petscdiff-bracket but the pipeline keeps failing ts_tutorials_advection-diffusion-reaction-ex3_2 fails on different machines with slightly different counts. I don't see how this change could cause that! But gets old results on my machine. Very frustrating.

  Barry


> On Dec 31, 2020, at 1:02 PM, Matthew Knepley <knepley at gmail.com> wrote:
> 
> On Thu, Dec 31, 2020 at 1:48 PM Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
> 
>   So the programs output changes and should no longer match that in the output/* file yet the test harness does not error with a statement that the two outputs do not match? 
> 
>    I noticed the gmakegentest.py is not being run before it runs the test? Does this mean it is just running all the old stuff which does match fine? 
> 
>    Then either how petscdiff is called by the test harness has changed or petscdiff has changed and does not detect changes anymore 
> 
>    BTW: I always use -f ./gmakefile.test test not just the gmakefile
> 
>    All the PETSc changes are trivial and can be seen with a simple diff, it is hard to believe they would cause this behavior but I guess they must.
> 
>    You can go to PETSC_ARCH/tests/snes/tests and run the ex13 shell script directly.
> 
> It is the sed problem:
> 
> master *$:/PETSc3/petsc/petsc-dev$ /PETSc3/petsc/petsc-dev/lib/petsc/bin/petscdiff   /PETSc3/petsc/petsc-dev/src/snes/tests/output/ex13_bench.out ex13_bench.tmp
> 
> sed: 1: "s/\033[1;31m//g": unbalanced brackets ([])
> sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([])
> sed: 1: "s/\033[1;31m//g": unbalanced brackets ([])
> sed: 1: "s/\033[0;39m\033[0;49m//g": unbalanced brackets ([])
> 
> The error was getting eaten.
> 
> This is in current master. Is it fixed in a branch? 
> 
>    Matt
>  
>   Barry
> 
> 
>> On Dec 31, 2020, at 12:38 PM, Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>> wrote:
>> 
>> I just pulled master, and simple alterations to tests do not produce a failure:
>> 
>> master *$:/PETSc3/petsc/petsc-dev$ PETSC_ARCH=arch-master-debug make -f ./gmakefile test search="snes_tests-ex13_bench" TIMEOUT=5000 EXTRA_OPTIONS="-dm_
>> refine 0"
>> Using MAKEFLAGS: EXTRA_OPTIONS=-dm_refine 0 TIMEOUT=5000 search=snes_tests-ex13_bench
>>         TEST arch-master-debug/tests/counts/snes_tests-ex13_bench.counts
>>  ok snes_tests-ex13_bench
>>  ok diff-snes_tests-ex13_bench
>> 
>> I check that the runs produce different output when done manually.
>> 
>> Scott and Barry, could this be related to changed to testing?
>> 
>>   Thanks,
>> 
>>      Matt
>> 
>> -- 
>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>> -- Norbert Wiener
>> 
>> https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
> 
> https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20201231/bd3a78b2/attachment-0001.html>