[petsc-dev] Regression tests

[Matthew Knepley]

On Thu, Jul 26, 2012 at 12:21 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

> Builder does some of this, but I think not all, and I don't think it's as
> convenient. The differences are partly syntactic. First, to add a test, you
> have to add lines to a single global file (currently 1500 lines). By
> creating the dictionary globally, we can't do consistency checks when the
> test is declared, instead you only do it later once you have lost line
> number information. You also can't build in the source directory, or
> anything similar, which makes it hard for a user to split an example out
> into their own toy project. Since you mess around with stdout, we are
> losing warning messages (including color) and lose some Emacs M-x compile
> convenience. (this can be fixed easily enough).
>

1) I am fine with changing where the options come from, meaning if we want
to also check for an import somewhere. Personally, I think
    its easier to maintain a central database than a distributed database.
I would only suggest a distributed database for scalability, which
    we do not need.

2) I take your point about users adding tests (extensibility), which is why
I would be fine with checking for an import

3) I have no idea what you mean about consistency checks

4) You can run builder on source anywhere. This is how I check the crazy
things people send us.


> Note that my suggestion does not _prevent_ using a global script, but it
> localizes the declaration and would make it possible to run locally.
>
> On Thu, Jul 26, 2012 at 11:37 AM, Matthew Knepley <knepley at gmail.com>wrote:
>
>> ./test.py build ex19
>>>
>>
> $ config/builder2.py buildExample src/snes/examples/tutorials/ex19.c
> Namespace(files=['src/snes/examples/tutorials/ex19.c'], func=<function
> buildExample at 0x1fa56e0>)
> Building ['/home/jed/petsc/src/snes/examples/tutorials/ex19.c']
> ERROR IN LINK ******************************
>
>                    /usr/bin/ld: cannot find -lpetsc
>
>                                      collect2: error: ld returned 1 exit
> status
>
> (maybe it doesn't work --with-single-library=0)
>

It does not. I don't see a reason to support this. If its does, it is
accidental.


> $ PETSC_ARCH=mpich-singlelib config/builder2.py buildExample
> src/snes/examples/tutorials/ex19.c
> Namespace(files=['src/snes/examples/tutorials/ex19.c'], func=<function
> buildExample at 0x25dd6e0>)
> Building ['/home/jed/petsc/src/snes/examples/tutorials/ex19.c']
>
> (hmm, where is the executable anyway)
>
> $ ls -ltr | tail -3
> lrwxrwxrwx 1 jed users       35 Jul 26 11:57 make.log.bkp ->
> /home/jed/petsc/mpich/conf/make.log
> lrwxrwxrwx 1 jed users       45 Jul 26 11:58 make.log ->
> /home/jed/petsc/mpich-singlelib/conf/make.log
> -rw-r--r-- 1 jed users  5271714 Jul 26 11:58 RDict.log
>
> (oh cool, it overwrote my library build with the make.log from this
> example)
>

It shouldn't. It should move make.log to make.log.bkp first. There must be
a bug.


> $ cat make.log
>   Loaded configure to cache: size 298004
> Building ['/home/jed/petsc/src/snes/examples/tutorials/ex19.c']
>   Compiling C files ['/home/jed/petsc/src/snes/examples/tutorials/ex19.c']
>         Pushing language C
> /opt/mpich2/bin/mpicc -c -I/home/jed/petsc/mpich-singlelib/include
> -I/home/jed/petsc/include -I/home/jed/petsc/mpich-singlelib/include
> -I/home/jed/usr/elemental-mpich/include -I/usr/include
> -I/opt/mpich2/include  -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing
> -Wno-unknown-pragmas -g3 -fno-inline -O0     -MMD -D__INSDIR__=
> /home/jed/petsc/src/snes/examples/tutorials/ex19.c
> sh: /opt/mpich2/bin/mpicc -c -I/home/jed/petsc/mpich-singlelib/include
> -I/home/jed/petsc/include -I/home/jed/petsc/mpich-singlelib/include
> -I/home/jed/usr/elemental-mpich/include -I/usr/include
> -I/opt/mpich2/include  -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing
> -Wno-unknown-pragmas -g3 -fno-inline -O0     -MMD -D__INSDIR__=
> /home/jed/petsc/src/snes/examples/tutorials/ex19.c
> Executing: /opt/mpich2/bin/mpicc -c
> -I/home/jed/petsc/mpich-singlelib/include -I/home/jed/petsc/include
> -I/home/jed/petsc/mpich-singlelib/include
> -I/home/jed/usr/elemental-mpich/include -I/usr/include
> -I/opt/mpich2/include  -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing
> -Wno-unknown-pragmas -g3 -fno-inline -O0     -MMD -D__INSDIR__=
> /home/jed/petsc/src/snes/examples/tutorials/ex19.c
> sh:
>         Popping language C
>         Moving ex19.o to
> /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19.o
>         Moving ex19.d to
> /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19.d
> Linking object ['/home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19.o']
> into /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19
>     Pushing language C
>                                 Pushing language C
>                                 Popping language C
>                                 Pushing language CUDA
>                                 Popping language CUDA
>                                 Pushing language Cxx
>                                 Popping language Cxx
>                                 Pushing language FC
>                                 Popping language FC
>         Pushing language C
>         Popping language C
> sh: /opt/mpich2/bin/mpicc  -o
> /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19    -fPIC -Wall
> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline
> -O0 /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19.o
> -L/home/jed/petsc/mpich-singlelib/lib -lpetsc
> -Wl,-rpath,/home/jed/petsc/mpich-singlelib/lib
> -L/home/jed/petsc/mpich-singlelib/lib -ltriangle -lX11 -lchaco -lparmetis
> -lmetis -Wl,-rpath,/home/jed/usr/elemental-mpich/lib
> -L/home/jed/usr/elemental-mpich/lib -lelemental -lplcg -lpmrrr -lamspub
> -lml -Wl,-rpath,/opt/mpich2/lib -L/opt/mpich2/lib
> -Wl,-rpath,/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1
> -L/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -lmpichcxx -lstdc++ -lpthread
> -lblacs -lHYPRE -lmpichcxx -lstdc++ -Wl,-rpath,/usr/lib -L/usr/lib
> -lcholmod -lamd -lcolamd -lcamd -lccolamd -lumfpack -lcholmod -lcamd
> -lccolamd -lcolamd -lamd -llapack -lblas -lm -L/usr/local/cuda/lib
> -Wl,-rpath,/opt/mpich2/lib -L/opt/mpich2/lib
> -Wl,-rpath,/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1
> -L/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -ldl
> -Wl,-rpath,/opt/mpich2/lib -lmpich -lopa -lmpl -lpthread -lrt -lgcc_s -ldl
> Executing: /opt/mpich2/bin/mpicc  -o
> /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19    -fPIC -Wall
> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline
> -O0 /home/jed/petsc/mpich-singlelib/lib/ex19-obj/ex19.o
> -L/home/jed/petsc/mpich-singlelib/lib -lpetsc
> -Wl,-rpath,/home/jed/petsc/mpich-singlelib/lib
> -L/home/jed/petsc/mpich-singlelib/lib -ltriangle -lX11 -lchaco -lparmetis
> -lmetis -Wl,-rpath,/home/jed/usr/elemental-mpich/lib
> -L/home/jed/usr/elemental-mpich/lib -lelemental -lplcg -lpmrrr -lamspub
> -lml -Wl,-rpath,/opt/mpich2/lib -L/opt/mpich2/lib
> -Wl,-rpath,/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1
> -L/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -lmpichcxx -lstdc++ -lpthread
> -lblacs -lHYPRE -lmpichcxx -lstdc++ -Wl,-rpath,/usr/lib -L/usr/lib
> -lcholmod -lamd -lcolamd -lcamd -lccolamd -lumfpack -lcholmod -lcamd
> -lccolamd -lcolamd -lamd -llapack -lblas -lm -L/usr/local/cuda/lib
> -Wl,-rpath,/opt/mpich2/lib -L/opt/mpich2/lib
> -Wl,-rpath,/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1
> -L/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.1 -ldl
> -Wl,-rpath,/opt/mpich2/lib -lmpich -lopa -lmpl -lpthread -lrt -lgcc_s -ldl
> sh:
>     Popping language C
>
> (the above output loses warnings)
>

Bug.


> $ mpich-singlelib/lib/ex19-obj/ex19 -happy-running
>
>
>>
>>> ./test.py check ex19_fas
>>>
>>
> $ PETSC_ARCH=mpich-singlelib config/builder2.py check --testnum 5
> src/snes/examples/tutorials/ex19.c
> Namespace(args=[], files=['src/snes/examples/tutorials/ex19.c'],
> func=<function check at 0xfe17d0>, retain=False, testnum=5)
> All tests pass
>
> (edit file, insert syntax error, re-run)
>
> [same output as above]
>
>
>>> ./test.py check ex19_\* # run all the ex19 tests
>>>
>>> ./test.py check -n 16 --if '(X || Fortran) && !Complex'
>>>
>>
> How?
>


Internally it knows all these predicates like num processes, Fortran and
complex. There is no command line syntax for it. I did
not think there was for make either. You can do that from the run
parameters at the op of builder.py.

   Matt


>
>>> The last line would mean to run all tests requiring X or Fortran, but
>>> not complex, using up to 16 MPI processes total.
>>>
>>> We could preserve "make runex19_fas" in the transition period, but I'd
>>> like to be able to do everything from test.py.
>>>
>>
>> builder already does all of this, and more stuff for code generation.
>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20120726/6f82bfc8/attachment.html>