Multilevel solver

[Matthew Knepley]

On Wed, Apr 23, 2008 at 9:07 AM,  <Amit.Itagi at seagate.com> wrote:
> Barry,
>
> This is what valgrind gives me. Any idea ? What is confusing me is that I
> get the crash after several GMRES iterations.

1) Always start with the simplest case, meaning serial

2) When you run valgrind in parallel, you need --trace-children=yes, since
    MPI usually spawns other processes

3) It is possible to corrupt memory so badly that valgrind crashes
like this, but it is hard.

   Matt

> [2]PETSC ERROR:
> ------------------------------------------------------------------------
> [3]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
> batch system) has told this process to end
> [3]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> ------------------------------------------------------------------------
> [3]PETSC ERROR: or see
> http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#Signal[3]PETSC
> ERROR: or try http://valgrind.org on linux or man libgmalloc on Apple to
> find memory corruption errors
> [2]PETSC ERROR: Caught signal number 1 Hang up: Some other process (or the
> batch system) has told this process to end
> [2]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [2]PETSC ERROR: or see
> http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#Signal[2]PETSC
> ERROR: or try http://valgrind.org on linux or man libgmalloc on Apple to
> find memory corruption errors
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
> batch system) has told this process to end
> [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [0]PETSC ERROR: or see
> http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#Signal[0]PETSC
> ERROR: or try http://valgrind.org on linux or man libgmalloc on Apple to
> find memory corruption errors
> [3]PETSC ERROR: likely location of problem given in stack below
> [3]PETSC ERROR: ---------------------  Stack Frames
> ------------------------------------
> ------------------------------------------------------------------------
> [1]PETSC ERROR: [2]PETSC ERROR: Caught signal number 15 Terminate: Somet
> process (or the batch system) has told this process to end
> likely location of problem given in stack below
> [1]PETSC ERROR: [2]PETSC ERROR: Try option -start_in_debugger or
> -on_error_attach_debugger
> ---------------------  Stack Frames ------------------------------------
> [1]PETSC ERROR: or see
> http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#Signal[1]PETSC
> ERROR: or try http://valgrind.org on linux or man libgmalloc on Apple to
> find memory corruption errors
> [0]PETSC ERROR: likely location of problem given in stack below
> [0]PETSC ERROR: ---------------------  Stack Frames
> ------------------------------------
> [1]PETSC ERROR: likely location of problem given in stack below
> [1]PETSC ERROR: ---------------------  Stack Frames
> ------------------------------------
> [3]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
> [3]PETSC ERROR:       INSTEAD the line number of the start of the function
> [3]PETSC ERROR:       is given.
> [2]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
> [3]PETSC ERROR: [2]PETSC ERROR:       INSTEAD the line number of the start
> of the function
> [2]PETSC ERROR:       is given.
> [3] PCApply_Shell line 122 src/ksp/pc/impls/shell/shellpc.c
> [3]PETSC ERROR: [3] PCApply line 346 src/ksp/pc/interface/precon.c
> [3]PETSC ERROR: [3] PCApplyBAorAB line 539 src/ksp/pc/interface/precon.c
> [3]PETSC ERROR: [3] GMREScycle line 133 src/ksp/ksp/impls/gmres/gmres.c
> [2]PETSC ERROR: [0]PETSC ERROR: Note: The EXACT line numbers in the stack
> are not available,
> [2] PCApply_Shell line 122 src/ksp/pc/impls/shell/shellpc.c
> [0]PETSC ERROR:       INSTEAD the line number of the start of the function
> [2]PETSC ERROR: [2] PCApply line 346 src/ksp/pc/interface/precon.c
> [0]PETSC ERROR:       is given.
> [2]PETSC ERROR: [2] PCApplyBAorAB line 539 src/ksp/pc/interface/precon.c
> [2]PETSC ERROR: [2] GMREScycle line 133 src/ksp/ksp/impls/gmres/gmres.c
> [1]PETSC ERROR: [0]PETSC ERROR: Note: The EXACT line numbers in the stack
> are not available,
> [0] PCApply_Shell line 122 src/ksp/pc/impls/shell/shellpc.c
> [1]PETSC ERROR:       INSTEAD the line number of the start of the function
> [0]PETSC ERROR: [0] PCApply line 346 src/ksp/pc/interface/precon.c
> [0]PETSC ERROR: [1]PETSC ERROR:       is given.
> [0] PCApplyBAorAB line 539 src/ksp/pc/interface/precon.c
> [0]PETSC ERROR: [0] GMREScycle line 133 src/ksp/ksp/impls/gmres/gmres.c
> [1]PETSC ERROR: [1] PCApply_Shell line 122 src/ksp/pc/impls/shell/shellpc.c
> [1]PETSC ERROR: [1] PCApply line 346 src/ksp/pc/interface/precon.c
> [1]PETSC ERROR: [3]PETSC ERROR: [1] PCApplyBAorAB line 539
> src/ksp/pc/interface/precon.c
> --------------------- Error Message ------------------------------------
> [1]PETSC ERROR: [1] GMREScycle line 133 src/ksp/ksp/impls/gmres/gmres.c
> [2]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [0]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [3]PETSC ERROR: Signal received!
> [3]PETSC ERROR:
> ------------------------------------------------------------------------
> [3]PETSC ERROR: Petsc Release Version 2.3.3, Patch 8, Fri Nov 16 17:03:40
> CST 2007 HG revision: 414581156e67e55c761739b0deb119f7590d0f4b
> [3]PETSC ERROR: See docs/changes/index.html for recent updates.
> [3]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>
> Thanks
>
> Rgds,
> Amit
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which
their experiments lead.
-- Norbert Wiener