[petsc-users] Problem with "likely location of problem given in stack below"?

Sat Apr 7 15:46:53 CDT 2012

  It is crashing in 2] VecView_VTK line 231 src/ksp/ksp/examples/tutorials/ex29.c

   This is an ASCII viewer and ASCII viewers cannot be used for large size problems.

    Barry

On Apr 7, 2012, at 3:41 PM, Alan Wei wrote:

> Dear All,
>     I hope you're having a nice weekend.
>     I got a problem with /ksp/ksp/example/tutorial. I was trying to execute this example with petsc-dev in a share  memory dell workstation with 8 cores. The parameter for DMDACreate2d is 500, 500, which means it generates a 1997*1997 2d grid. It is fine to run it with single process with either "./ex29" or "mpiexec -np 1 ./exe29". However, it came across a error when I tempted to run it with more than 1 process. The error message is like this, when I was trying to execute it with 4 processes. I was thinking about if the 'stack size' is the problem, so I enlarge it to 1000000000 with 'ulimit' command. However, the same error message keeps showing up.
> 
> [1]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
> [1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [1]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind[1]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
> [2]PETSC ERROR: ------------------------------------------------------------------------
> [2]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
> [2]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [2]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind[2]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
> [1]PETSC ERROR: likely location of problem given in stack below
> [1]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
> [2]PETSC ERROR: likely location of problem given in stack below
> [2]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
> [1]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
> [1]PETSC ERROR:       INSTEAD the line number of the start of the function
> [1]PETSC ERROR:       is given.
> [1]PETSC ERROR: [1] VecView_VTK line 231 src/ksp/ksp/examples/tutorials/ex29.c
> [1]PETSC ERROR: --------------------- Error Message ------------------------------------
> [1]PETSC ERROR: Signal received!
> [1]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: Petsc Development HG revision: 85e6055943e0711fccdec2d08caeba48971d3d55  HG Date: Fri Sep 23 14:46:14 2011 -0700
> [1]PETSC ERROR: See docs/changes/index.html for recent updates.
> [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [1]PETSC ERROR: See docs/index.html for manual pages.
> [1]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: ./ex29 on a arch-linu named l2118a-linux.soecs.ku.edu by zlwei Sat Apr  7 16:33:21 2012
> [1]PETSC ERROR: Libraries linked from /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib
> [1]PETSC ERROR: Configure run at Fri Sep 23 17:13:32 2011
> [1]PETSC ERROR: Configure options --download-f-blas-lapack=1 --download-mpich=1 --with-cc=gcc --with-fc=gfortran PETSC_ARCH=arch-linux2-c-debug
> [1]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: User provided function() line 0 in unknown directory unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1
> [cli_1]: aborting job:
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1
> [2]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
> [2]PETSC ERROR:       INSTEAD the line number of the start of the function
> [2]PETSC ERROR:       is given.
> [2]PETSC ERROR: [2] VecView_VTK line 231 src/ksp/ksp/examples/tutorials/ex29.c
> [2]PETSC ERROR: --------------------- Error Message ------------------------------------
> [2]PETSC ERROR: Signal received!
> [2]PETSC ERROR: ------------------------------------------------------------------------
> [2]PETSC ERROR: Petsc Development HG revision: 85e6055943e0711fccdec2d08caeba48971d3d55  HG Date: Fri Sep 23 14:46:14 2011 -0700
> [2]PETSC ERROR: See docs/changes/index.html for recent updates.
> [2]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [2]PETSC ERROR: See docs/index.html for manual pages.
> [2]PETSC ERROR: ------------------------------------------------------------------------
> [2]PETSC ERROR: ./ex29 on a arch-linu named l2118a-linux.soecs.ku.edu by zlwei Sat Apr  7 16:33:21 2012
> [2]PETSC ERROR: Libraries linked from /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib
> [2]PETSC ERROR: Configure run at Fri Sep 23 17:13:32 2011
> [2]PETSC ERROR: Configure options --download-f-blas-lapack=1 --download-mpich=1 --with-cc=gcc --with-fc=gfortran PETSC_ARCH=arch-linux2-c-debug
> [2]PETSC ERROR: ------------------------------------------------------------------------
> [2]PETSC ERROR: User provided function() line 0 in unknown directory unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 2
> [cli_2]: aborting job:
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 2
> [3]PETSC ERROR: ------------------------------------------------------------------------
> [3]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
> [3]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [3]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind[3]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
> [3]PETSC ERROR: likely location of problem given in stack below
> [3]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
> [3]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
> [3]PETSC ERROR:       INSTEAD the line number of the start of the function
> [3]PETSC ERROR:       is given.
> [3]PETSC ERROR: [3] VecView_VTK line 231 src/ksp/ksp/examples/tutorials/ex29.c
> [3]PETSC ERROR: --------------------- Error Message ------------------------------------
> [3]PETSC ERROR: Signal received!
> [3]PETSC ERROR: ------------------------------------------------------------------------
> [3]PETSC ERROR: Petsc Development HG revision: 85e6055943e0711fccdec2d08caeba48971d3d55  HG Date: Fri Sep 23 14:46:14 2011 -0700
> [3]PETSC ERROR: See docs/changes/index.html for recent updates.
> [3]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [3]PETSC ERROR: See docs/index.html for manual pages.
> [3]PETSC ERROR: ------------------------------------------------------------------------
> [3]PETSC ERROR: ./ex29 on a arch-linu named l2118a-linux.soecs.ku.edu by zlwei Sat Apr  7 16:33:21 2012
> [3]PETSC ERROR: Libraries linked from /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib
> [3]PETSC ERROR: Configure run at Fri Sep 23 17:13:32 2011
> [3]PETSC ERROR: Configure options --download-f-blas-lapack=1 --download-mpich=1 --with-cc=gcc --with-fc=gfortran PETSC_ARCH=arch-linux2-c-debug
> [3]PETSC ERROR: ------------------------------------------------------------------------
> [3]PETSC ERROR: User provided function() line 0 in unknown directory unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 3
> [cli_3]: aborting job:
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 3
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
> [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
> [0]PETSC ERROR: likely location of problem given in stack below
> [0]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
> [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
> [0]PETSC ERROR:       INSTEAD the line number of the start of the function
> [0]PETSC ERROR:       is given.
> [0]PETSC ERROR: [0] VecView_VTK line 231 src/ksp/ksp/examples/tutorials/ex29.c
> [0]PETSC ERROR: --------------------- Error Message ------------------------------------
> [0]PETSC ERROR: Signal received!
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Development HG revision: 85e6055943e0711fccdec2d08caeba48971d3d55  HG Date: Fri Sep 23 14:46:14 2011 -0700
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: ./ex29 on a arch-linu named l2118a-linux.soecs.ku.edu by zlwei Sat Apr  7 16:33:21 2012
> [0]PETSC ERROR: Libraries linked from /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib
> [0]PETSC ERROR: Configure run at Fri Sep 23 17:13:32 2011
> [0]PETSC ERROR: Configure options --download-f-blas-lapack=1 --download-mpich=1 --with-cc=gcc --with-fc=gfortran PETSC_ARCH=arch-linux2-c-debug
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
> [cli_0]: aborting job:
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
> 
> thanks in advance,
> Alan