[petsc-users] Caught signal number 15 Terminate

Sepideh Kavousi skavou1 at lsu.edu
Wed Aug 28 13:23:57 CDT 2019 

Dear Petsc users,
I have a code which is very strange that works for some cases and for some cases it gives an error. I run the following command:

mpiexec -np 20 valgrind --tool=memcheck -q --num-callers=20 --log-file=valgrind.log.%p  ./one.out -ts_monitor -snes_fd_color -ts_max_snes_failures -1  -ts_type bdf -ts_bdf_adapt -pc_type bjacobi -snes_linesearch_type l2 -snes_type ksponly -ksp_type gmres -ksp_gmres_restart 1001 -sub_pc_type ilu -sub_ksp_type preonly -malloc_dump

the error is as following and nothing is written in valgrid log files. Can someone please help me on how to solve the problem? the system size is 600x1200 and I have two degrees of freedom per grid point. Can this be due to the fact that I need to assign a larger number of processors?
How should I know if the problem is related to this or anything else?

Thanks,
Sepideh

.
.
.
.
.
.
[ 0]16 bytes PetscStrallocpy() line 197 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/utils/str.c
      [0]  PetscStrallocpy() line 197 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/utils/str.c
      [0]  PetscClassRegLogRegister() line 242 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/classlog.c
      [0]  PetscClassIdRegister() line 2042 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/plog.c
      [0]  PetscSysInitializePackage() line 45 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/classes/viewer/interface/dlregispetsc.c
      [0]  DMCreate() line 36 in /home/skavou1/Downloads/petsc-3.7.3/src/dm/interface/dm.c
      [0]  DMDACreate() line 460 in /home/skavou1/Downloads/petsc-3.7.3/src/dm/impls/da/dacreate.c
      [0]  DMDACreate2d() line 854 in /home/skavou1/Downloads/petsc-3.7.3/src/dm/impls/da/da2.c
[ 0]10032 bytes PetscSegBufferCreate() line 68 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/utils/segbuffer.c
[ 0]16 bytes PetscSegBufferCreate() line 67 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/utils/segbuffer.c
[ 0]16 bytes PetscOptionsHelpPrintedCreate() line 50 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/classes/viewer/interface/viewreg.c
[ 0]3200 bytes ClassPerfLogCreate() line 121 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/classlog.c
[ 0]16 bytes ClassPerfLogCreate() line 116 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/classlog.c
[ 0]16 bytes EventPerfLogCreate() line 92 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/eventlog.c
[ 0]16 bytes PetscStrallocpy() line 197 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/utils/str.c
[ 0]1600 bytes PetscClassRegLogCreate() line 36 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/classlog.c
[ 0]16 bytes PetscClassRegLogCreate() line 31 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/classlog.c
[ 0]16 bytes EventRegLogCreate() line 34 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/eventlog.c
[ 0]1280 bytes PetscStageLogCreate() line 635 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/stagelog.c
[ 0]512 bytes PetscIntStackCreate() line 177 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/stack.c
[ 0]16 bytes PetscIntStackCreate() line 172 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/stack.c
[ 0]48 bytes PetscStageLogCreate() line 628 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/logging/utils/stagelog.c
[ 0]10032 bytes PetscSegBufferCreate() line 68 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/utils/segbuffer.c
[ 0]16 bytes PetscSegBufferCreate() line 67 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/utils/segbuffer.c
[ 0]32 bytes PetscPushSignalHandler() line 307 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/error/signal.c
[0]PETSC ERROR: Memory requested 414030660
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.3, Jul, 24, 2016
[0]PETSC ERROR: ./one.out on a one named kratos by skavou1 Wed Aug 28 12:41:38 2019
[0]PETSC ERROR: Configure options PETSC_ARCH=one --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich --download-fblaslapack
[0]PETSC ERROR: #1 MatFDColoringSetUpBlocked_AIJ_Private() line 122 in /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: #2 PetscTrMallocDefault() line 188 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/memory/mtr.c
[0]PETSC ERROR: #3 MatFDColoringSetUpBlocked_AIJ_Private() line 122 in /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: #4 MatFDColoringSetUp_SeqXAIJ() line 277 in /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: #5 MatFDColoringSetUp() line 252 in /home/skavou1/Downloads/petsc-3.7.3/src/mat/matfd/fdmatrix.c
[0]PETSC ERROR: #6 SNESComputeJacobianDefaultColor() line 76 in /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snesj2.c
[0]PETSC ERROR: #7 SNESComputeJacobian() line 2312 in /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: #8 SNESSolve_KSPONLY() line 38 in /home/skavou1/Downloads/petsc-3.7.3/src/snes/impls/ksponly/ksponly.c
[0]PETSC ERROR: #9 SNESSolve() line 4005 in /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: #10 TS_SNESSolve() line 160 in /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: #11 TSBDF_Restart() line 181 in /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: #12 TSStep_BDF() line 216 in /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: #13 TSStep() line 3733 in /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: #14 TSSolve() line 3982 in /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: #15 main() line 267 in /data/skavou1/Ni-Nb/Fig-3/one.c
[0]PETSC ERROR: PETSc Option Table entries:
[0]PETSC ERROR: -ksp_gmres_restart 1001
[0]PETSC ERROR: -ksp_type gmres
[0]PETSC ERROR: -malloc_dump
[0]PETSC ERROR: -pc_type bjacobi
[0]PETSC ERROR: -snes_fd_color
[0]PETSC ERROR: -snes_linesearch_type l2
[0]PETSC ERROR: -snes_type ksponly
[0]PETSC ERROR: -sub_ksp_type preonly
[0]PETSC ERROR: -sub_pc_type ilu
[0]PETSC ERROR: -ts_bdf_adapt
[0]PETSC ERROR: -ts_max_snes_failures -1
[0]PETSC ERROR: -ts_monitor
[0]PETSC ERROR: -ts_type bdf
[0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov----------
application called MPI_Abort(MPI_COMM_WORLD, 55) - process 0
[unset]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 55) - process 0
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 15 Terminate: Some process (or the batch system) has told this process to end
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR:       INSTEAD the line number of the start of the function
[0]PETSC ERROR:       is given.
[0]PETSC ERROR: [0] MatFDColoringSetUp_SeqXAIJ line 184 /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: [0] MatFDColoringSetUp line 245 /home/skavou1/Downloads/petsc-3.7.3/src/mat/matfd/fdmatrix.c
[0]PETSC ERROR: [0] SNESComputeJacobianDefaultColor line 62 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snesj2.c
[0]PETSC ERROR: [0] SNES user Jacobian function line 2311 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESComputeJacobian line 2270 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESSolve_KSPONLY line 12 /home/skavou1/Downloads/petsc-3.7.3/src/snes/impls/ksponly/ksponly.c
[0]PETSC ERROR: [0] SNESSolve line 3958 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] TS_SNESSolve line 159 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSBDF_Restart line 174 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep_BDF line 204 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep line 3712 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: [0] TSSolve line 3933 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.3, Jul, 24, 2016
[0]PETSC ERROR: ./one.out on a one named kratos by skavou1 Wed Aug 28 12:41:38 2019
[0]PETSC ERROR: Configure options PETSC_ARCH=one --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich --download-fblaslapack
[0]PETSC ERROR: #1 User provided function() line 0 in  unknown file
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[unset]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 15 Terminate: Some process (or the batch system) has told this process to end
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR:       INSTEAD the line number of the start of the function
[0]PETSC ERROR:       is given.
[0]PETSC ERROR: [0] MatFDColoringSetUpBlocked_AIJ_Private line 67 /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: [0] MatFDColoringSetUp_SeqXAIJ line 184 /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: [0] MatFDColoringSetUp line 245 /home/skavou1/Downloads/petsc-3.7.3/src/mat/matfd/fdmatrix.c
[0]PETSC ERROR: [0] SNESComputeJacobianDefaultColor line 62 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snesj2.c
[0]PETSC ERROR: [0] SNES user Jacobian function line 2311 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESComputeJacobian line 2270 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESSolve_KSPONLY line 12 /home/skavou1/Downloads/petsc-3.7.3/src/snes/impls/ksponly/ksponly.c
[0]PETSC ERROR: [0] SNESSolve line 3958 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] TS_SNESSolve line 159 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSBDF_Restart line 174 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep_BDF line 204 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep line 3712 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: [0] TSSolve line 3933 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.3, Jul, 24, 2016
[0]PETSC ERROR: ./one.out on a one named kratos by skavou1 Wed Aug 28 12:41:38 2019
[0]PETSC ERROR: Configure options PETSC_ARCH=one --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich --download-fblaslapack
[0]PETSC ERROR: #1 User provided function() line 0 in  unknown file
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[unset]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 15 Terminate: Some process (or the batch system) has told this process to end
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR:       INSTEAD the line number of the start of the function
[0]PETSC ERROR:       is given.
[0]PETSC ERROR: [0] MatFDColoringSetUpBlocked_AIJ_Private line 67 /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: [0] MatFDColoringSetUp_SeqXAIJ line 184 /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: [0] MatFDColoringSetUp line 245 /home/skavou1/Downloads/petsc-3.7.3/src/mat/matfd/fdmatrix.c
[0]PETSC ERROR: [0] SNESComputeJacobianDefaultColor line 62 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snesj2.c
[0]PETSC ERROR: [0] SNES user Jacobian function line 2311 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESComputeJacobian line 2270 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESSolve_KSPONLY line 12 /home/skavou1/Downloads/petsc-3.7.3/src/snes/impls/ksponly/ksponly.c
[0]PETSC ERROR: [0] SNESSolve line 3958 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] TS_SNESSolve line 159 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSBDF_Restart line 174 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep_BDF line 204 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep line 3712 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: [0] TSSolve line 3933 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.3, Jul, 24, 2016
[0]PETSC ERROR: ./one.out on a one named kratos by skavou1 Wed Aug 28 12:41:38 2019
[0]PETSC ERROR: Configure options PETSC_ARCH=one --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich --download-fblaslapack
[0]PETSC ERROR: #1 User provided function() line 0 in  unknown file
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[unset]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 15 Terminate: Some process (or the batch system) has told this process to end
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR:       INSTEAD the line number of the start of the function
[0]PETSC ERROR:       is given.
[0]PETSC ERROR: [0] MatFDColoringSetUpBlocked_AIJ_Private line 67 /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: [0] MatFDColoringSetUp_SeqXAIJ line 184 /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: [0] MatFDColoringSetUp line 245 /home/skavou1/Downloads/petsc-3.7.3/src/mat/matfd/fdmatrix.c
[0]PETSC ERROR: [0] SNESComputeJacobianDefaultColor line 62 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snesj2.c
[0]PETSC ERROR: [0] SNES user Jacobian function line 2311 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESComputeJacobian line 2270 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESSolve_KSPONLY line 12 /home/skavou1/Downloads/petsc-3.7.3/src/snes/impls/ksponly/ksponly.c
[0]PETSC ERROR: [0] SNESSolve line 3958 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] TS_SNESSolve line 159 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSBDF_Restart line 174 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep_BDF line 204 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep line 3712 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: [0] TSSolve line 3933 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.3, Jul, 24, 2016
[0]PETSC ERROR: ./one.out on a one named kratos by skavou1 Wed Aug 28 12:41:38 2019
[0]PETSC ERROR: Configure options PETSC_ARCH=one --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich --download-fblaslapack
[0]PETSC ERROR: #1 User provided function() line 0 in  unknown file
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[unset]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 15 Terminate: Some process (or the batch system) has told this process to end
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR:       INSTEAD the line number of the start of the function
[0]PETSC ERROR:       is given.
[0]PETSC ERROR: [0] MatFDColoringSetUpBlocked_AIJ_Private line 67 /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: [0] MatFDColoringSetUp_SeqXAIJ line 184 /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: [0] MatFDColoringSetUp line 245 /home/skavou1/Downloads/petsc-3.7.3/src/mat/matfd/fdmatrix.c
[0]PETSC ERROR: [0] SNESComputeJacobianDefaultColor line 62 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snesj2.c
[0]PETSC ERROR: [0] SNES user Jacobian function line 2311 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESComputeJacobian line 2270 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESSolve_KSPONLY line 12 /home/skavou1/Downloads/petsc-3.7.3/src/snes/impls/ksponly/ksponly.c
[0]PETSC ERROR: [0] SNESSolve line 3958 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] TS_SNESSolve line 159 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSBDF_Restart line 174 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep_BDF line 204 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep line 3712 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: [0] TSSolve line 3933 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.3, Jul, 24, 2016
[0]PETSC ERROR: ./one.out on a one named kratos by skavou1 Wed Aug 28 12:41:38 2019
[0]PETSC ERROR: Configure options PETSC_ARCH=one --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich --download-fblaslapack
[0]PETSC ERROR: #1 User provided function() line 0 in  unknown file
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[unset]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 15 Terminate: Some process (or the batch system) has told this process to end
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR:       INSTEAD the line number of the start of the function
[0]PETSC ERROR:       is given.
[0]PETSC ERROR: [0] MatFDColoringSetUp_SeqXAIJ line 184 /home/skavou1/Downloads/petsc-3.7.3/src/mat/impls/aij/seq/fdaij.c
[0]PETSC ERROR: [0] MatFDColoringSetUp line 245 /home/skavou1/Downloads/petsc-3.7.3/src/mat/matfd/fdmatrix.c
[0]PETSC ERROR: [0] SNESComputeJacobianDefaultColor line 62 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snesj2.c
[0]PETSC ERROR: [0] SNES user Jacobian function line 2311 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESComputeJacobian line 2270 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] SNESSolve_KSPONLY line 12 /home/skavou1/Downloads/petsc-3.7.3/src/snes/impls/ksponly/ksponly.c
[0]PETSC ERROR: [0] SNESSolve line 3958 /home/skavou1/Downloads/petsc-3.7.3/src/snes/interface/snes.c
[0]PETSC ERROR: [0] TS_SNESSolve line 159 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSBDF_Restart line 174 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep_BDF line 204 /home/skavou1/Downloads/petsc-3.7.3/src/ts/impls/bdf/bdf.c
[0]PETSC ERROR: [0] TSStep line 3712 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: [0] TSSolve line 3933 /home/skavou1/Downloads/petsc-3.7.3/src/ts/interface/ts.c
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.3, Jul, 24, 2016
[0]PETSC ERROR: ./one.out on a one named kratos by skavou1 Wed Aug 28 12:41:38 2019
[0]PETSC ERROR: Configure options PETSC_ARCH=one --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich --download-fblaslapack
[0]PETSC ERROR: #1 User provided function() line 0 in  unknown file
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[unset]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 15 Terminate: Some process (or the batch system) has told this process to end
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: PetscMallocValidate: error detected at PetscSignalHandlerDefault() line 149 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/error/signal.c
[0]PETSC ERROR: Memory [id=0(414029056)] at address 0x2cfac680 already freed
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Memory corruption: http://www.mcs.anl.gov/petsc/documentation/installation.html#valgrind
[0]PETSC ERROR:
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.3, Jul, 24, 2016
[0]PETSC ERROR: ./one.out on a one named kratos by skavou1 Wed Aug 28 12:41:38 2019
[0]PETSC ERROR: Configure options PETSC_ARCH=one --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich --download-fblaslapack
[0]PETSC ERROR: #1 PetscMallocValidate() line 144 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/memory/mtr.c
[0]PETSC ERROR: #2 PetscSignalHandlerDefault() line 149 in /home/skavou1/Downloads/petsc-3.7.3/src/sys/error/signal.c
application called MPI_Abort(MPI_COMM_WORLD, 0) - process 0
[unset]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 0) - process 0
--------------------------------------------------------------------------
mpiexec noticed that process rank 15 with PID 0 on node kratos exited on signal 9 (Killed).
--------------------------------------------------------------------------
(base) skavou1 at kratos:/data/skavou1/Ni-Nb/Fig-3$

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190828/50cb94c2/attachment-0001.html>

More information about the petsc-users mailing list