[petsc-users] Error on process numbers greater than 2

Thu Aug 23 05:25:20 CDT 2012

Hi,
I am a new user and was running the example problems.

The given example in the package, ex2
(src/ksp/ksp/examples/tutorials/ex2.c), works fine with two processors
(mpiexec -n 2 ex2) but fails to work with any number greater than 2. For
example:
mpiexec -np 3 ex2, returns :

[1]PETSC ERROR: PetscTrFreeDefault() called from
PetscGatherMessageLengths() line 140 in src/sys/utils/mpimesg.c
[1]PETSC ERROR: Block [id=0(80)] at address 0x7f93799a1a20 is corrupted
(probably write past end of array)
[1]PETSC ERROR: Block allocated in PetscGatherMessageLengths() line 115 in
src/sys/utils/mpimesg.c
[1]PETSC ERROR: --------------------- Error Message
------------------------------------
[1]PETSC ERROR: Memory corruption!
[1]PETSC ERROR: Corrupted memory!
[1]PETSC ERROR:
------------------------------------------------------------------------
[1]PETSC ERROR: Petsc Release Version 3.3.0, Patch 2, Fri Jul 13 15:42:00
CDT 2012
[1]PETSC ERROR: See docs/changes/index.html for recent updates.
[1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
[1]PETSC ERROR: See docs/index.html for manual pages.
[1]PETSC ERROR:
------------------------------------------------------------------------
[1]PETSC ERROR: ex2 on a arch-darw named ball-9120.ivt.ntnu.no by bihs Thu
Aug 23 12:21:58 2012
[1]PETSC ERROR: Libraries linked from
/Users/bihs/petsc/arch-darwin-c-debug/lib
[1]PETSC ERROR: Configure run at Tue Aug 21 15:07:34 2012
[1]PETSC ERROR: Configure options --with-mpi-dir=/usr/local/include/openmpi
--with-fc=0 --download-f2cblaslapack
[1]PETSC ERROR:
------------------------------------------------------------------------
[1]PETSC ERROR: PetscTrFreeDefault() line 299 in src/sys/memory/mtr.c
[1]PETSC ERROR: PetscGatherMessageLengths() line 140 in
src/sys/utils/mpimesg.c
[1]PETSC ERROR: VecScatterCreate_PtoS() line 1664 in
src/vec/vec/utils/vpscat.c
[1]PETSC ERROR: VecScatterCreate() line 1331 in src/vec/vec/utils/vscat.c
[1]PETSC ERROR: MatSetUpMultiply_MPIAIJ() line 150 in
src/mat/impls/aij/mpi/mmaij.c
[1]PETSC ERROR: MatAssemblyEnd_MPIAIJ() line 680 in
src/mat/impls/aij/mpi/mpiaij.c
[1]PETSC ERROR: MatAssemblyEnd() line 4866 in src/mat/interface/matrix.c
[1]PETSC ERROR: main() line 107 in src/ksp/ksp/examples/tutorials/ex2.c
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
[0]PETSC ERROR:
------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
batch system) has told this process to end
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see
http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC ERROR:
or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory
corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: ---------------------  Stack Frames
------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR:       INSTEAD the line number of the start of the function
[0]PETSC ERROR:       is given.
[0]PETSC ERROR: [0] VecScatterCreate_PtoS line 1631
src/vec/vec/utils/vpscat.c
[0]PETSC ERROR: [0] VecScatterCreate line 844 src/vec/vec/utils/vscat.c
[0]PETSC ERROR: [0] MatSetUpMultiply_MPIAIJ line 26
src/mat/impls/aij/mpi/mmaij.c
[0]PETSC ERROR: [0] MatAssemblyEnd_MPIAIJ line 646
src/mat/impls/aij/mpi/mpiaij.c
[0]PETSC ERROR: [0] MatAssemblyEnd line 4857 src/mat/interface/matrix.c
[0]PETSC ERROR: --------------------- Error Message
------------------------------------
[0]PETSC ERROR: Signal received!
[0]PETSC ERROR:
------------------------------------------------------------------------
[0]PETSC ERROR: Petsc Release Version 3.3.0, Patch 2, Fri Jul 13 15:42:00
CDT 2012
[0]PETSC ERROR: See docs/changes/index.html for recent updates.
[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
[0]PETSC ERROR: See docs/index.html for manual pages.
[0]PETSC ERROR:
------------------------------------------------------------------------
[0]PETSC ERROR: ex2 on a arch-darw named ball-9120.ivt.ntnu.no by bihs Thu
Aug 23 12:21:58 2012
[0]PETSC ERROR: Libraries linked from
/Users/bihs/petsc/arch-darwin-c-debug/lib
[0]PETSC ERROR: Configure run at Tue Aug 21 15:07:34 2012
[0]PETSC ERROR: Configure options --with-mpi-dir=/usr/local/include/openmpi
--with-fc=0 --download-f2cblaslapack
[0]PETSC ERROR:
------------------------------------------------------------------------
[0]PETSC ERROR: User provided function() line 0 in unknown directory
unknown file
[2]PETSC ERROR:
------------------------------------------------------------------------
[2]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the
batch system) has told this process to end
[2]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[2]PETSC ERROR: or see
http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[2]PETSC ERROR:
or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory
corruption errors
[2]PETSC ERROR: likely location of problem given in stack below
[2]PETSC ERROR: ---------------------  Stack Frames
------------------------------------
[2]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[2]PETSC ERROR:       INSTEAD the line number of the start of the function
[2]PETSC ERROR:       is given.
[2]PETSC ERROR: [2] VecScatterCreate_PtoS line 1631
src/vec/vec/utils/vpscat.c
[2]PETSC ERROR: [2] VecScatterCreate line 844 src/vec/vec/utils/vscat.c
[2]PETSC ERROR: [2] MatSetUpMultiply_MPIAIJ line 26
src/mat/impls/aij/mpi/mmaij.c
[2]PETSC ERROR: [2] MatAssemblyEnd_MPIAIJ line 646
src/mat/impls/aij/mpi/mpiaij.c
[2]PETSC ERROR: [2] MatAssemblyEnd line 4857 src/mat/interface/matrix.c
[2]PETSC ERROR: --------------------- Error Message
------------------------------------
[2]PETSC ERROR: Signal received!
[2]PETSC ERROR:
------------------------------------------------------------------------
[2]PETSC ERROR: Petsc Release Version 3.3.0, Patch 2, Fri Jul 13 15:42:00
CDT 2012
[2]PETSC ERROR: See docs/changes/index.html for recent updates.
[2]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
[2]PETSC ERROR: See docs/index.html for manual pages.
[2]PETSC ERROR:
------------------------------------------------------------------------
[2]PETSC ERROR: ex2 on a arch-darw named ball-9120.ivt.ntnu.no by bihs Thu
Aug 23 12:21:58 2012
[2]PETSC ERROR: Libraries linked from
/Users/bihs/petsc/arch-darwin-c-debug/lib
[2]PETSC ERROR: Configure run at Tue Aug 21 15:07:34 2012
[2]PETSC ERROR: Configure options --with-mpi-dir=/usr/local/include/openmpi
--with-fc=0 --download-f2cblaslapack
[2]PETSC ERROR:
------------------------------------------------------------------------
[2]PETSC ERROR: User provided function() line 0 in unknown directory
unknown file

My machine has two 6 core Intel Xeon processors and have been running other
programs that use MPI to run 12 processes.
Could someone shed some light on why this is happening?
Thanks.

-- 
Arun Kamath
PhD Stipendiat
NTNU, Trondheim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120823/d8020192/attachment.html>