On Thu, Aug 23, 2012 at 5:25 AM, Arun Kamath <span dir="ltr"><<a href="mailto:arun.kamath.m@gmail.com" target="_blank">arun.kamath.m@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<div>I am a new user and was running the example problems.</div><div><br></div><div>The given example in the package, ex2 (src/ksp/ksp/examples/tutorials/ex2.c), works fine with two processors</div><div>(mpiexec -n 2 ex2) but fails to work with any number greater than 2. For example:</div>
<div>mpiexec -np 3 ex2, returns :</div><div><br></div><div><div>[1]PETSC ERROR: PetscTrFreeDefault() called from PetscGatherMessageLengths() line 140 in src/sys/utils/mpimesg.c</div><div>[1]PETSC ERROR: Block [id=0(80)] at address 0x7f93799a1a20 is corrupted (probably write past end of array)</div>
<div>[1]PETSC ERROR: Block allocated in PetscGatherMessageLengths() line 115 in src/sys/utils/mpimesg.c</div><div>[1]PETSC ERROR: --------------------- Error Message ------------------------------------</div><div>[1]PETSC ERROR: Memory corruption!</div>
<div>[1]PETSC ERROR: Corrupted memory!</div><div>[1]PETSC ERROR: ------------------------------------------------------------------------</div><div>[1]PETSC ERROR: Petsc Release Version 3.3.0, Patch 2, Fri Jul 13 15:42:00 CDT 2012 </div>
<div>[1]PETSC ERROR: See docs/changes/index.html for recent updates.</div><div>[1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div><div>[1]PETSC ERROR: See docs/index.html for manual pages.</div><div>
[1]PETSC ERROR: ------------------------------------------------------------------------</div><div>[1]PETSC ERROR: ex2 on a arch-darw named <a href="http://ball-9120.ivt.ntnu.no" target="_blank">ball-9120.ivt.ntnu.no</a> by bihs Thu Aug 23 12:21:58 2012</div>
<div>[1]PETSC ERROR: Libraries linked from /Users/bihs/petsc/arch-darwin-c-debug/lib</div><div>[1]PETSC ERROR: Configure run at Tue Aug 21 15:07:34 2012</div><div>[1]PETSC ERROR: Configure options --with-mpi-dir=/usr/local/include/openmpi --with-fc=0 --download-f2cblaslapack</div>
<div>[1]PETSC ERROR: ------------------------------------------------------------------------</div><div>[1]PETSC ERROR: PetscTrFreeDefault() line 299 in src/sys/memory/mtr.c</div><div>[1]PETSC ERROR: PetscGatherMessageLengths() line 140 in src/sys/utils/mpimesg.c</div>
<div>[1]PETSC ERROR: VecScatterCreate_PtoS() line 1664 in src/vec/vec/utils/vpscat.c</div><div>[1]PETSC ERROR: VecScatterCreate() line 1331 in src/vec/vec/utils/vscat.c</div><div>[1]PETSC ERROR: MatSetUpMultiply_MPIAIJ() line 150 in src/mat/impls/aij/mpi/mmaij.c</div>
<div>[1]PETSC ERROR: MatAssemblyEnd_MPIAIJ() line 680 in src/mat/impls/aij/mpi/mpiaij.c</div><div>[1]PETSC ERROR: MatAssemblyEnd() line 4866 in src/mat/interface/matrix.c</div><div>[1]PETSC ERROR: main() line 107 in src/ksp/ksp/examples/tutorials/ex2.c</div>
<div>--------------------------------------------------------------------------</div><div>MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD </div><div>with errorcode 1.</div><div><br></div><div>NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.</div>
<div>You may or may not see output from other processes, depending on</div><div>exactly when Open MPI kills them.</div><div>--------------------------------------------------------------------------</div><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[0]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the batch system) has told this process to end</div><div>[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger</div><div>[0]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC</a> ERROR: or try <a href="http://valgrind.org" target="_blank">http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors</div>
<div>[0]PETSC ERROR: likely location of problem given in stack below</div><div>[0]PETSC ERROR: --------------------- Stack Frames ------------------------------------</div><div>[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,</div>
<div>[0]PETSC ERROR: INSTEAD the line number of the start of the function</div><div>[0]PETSC ERROR: is given.</div><div>[0]PETSC ERROR: [0] VecScatterCreate_PtoS line 1631 src/vec/vec/utils/vpscat.c</div><div>
[0]PETSC ERROR: [0] VecScatterCreate line 844 src/vec/vec/utils/vscat.c</div><div>[0]PETSC ERROR: [0] MatSetUpMultiply_MPIAIJ line 26 src/mat/impls/aij/mpi/mmaij.c</div><div>[0]PETSC ERROR: [0] MatAssemblyEnd_MPIAIJ line 646 src/mat/impls/aij/mpi/mpiaij.c</div>
<div>[0]PETSC ERROR: [0] MatAssemblyEnd line 4857 src/mat/interface/matrix.c</div><div>[0]PETSC ERROR: --------------------- Error Message ------------------------------------</div><div>[0]PETSC ERROR: Signal received!</div>
<div>[0]PETSC ERROR: ------------------------------------------------------------------------</div><div>[0]PETSC ERROR: Petsc Release Version 3.3.0, Patch 2, Fri Jul 13 15:42:00 CDT 2012 </div><div>[0]PETSC ERROR: See docs/changes/index.html for recent updates.</div>
<div>[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div><div>[0]PETSC ERROR: See docs/index.html for manual pages.</div><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[0]PETSC ERROR: ex2 on a arch-darw named <a href="http://ball-9120.ivt.ntnu.no" target="_blank">ball-9120.ivt.ntnu.no</a> by bihs Thu Aug 23 12:21:58 2012</div><div>[0]PETSC ERROR: Libraries linked from /Users/bihs/petsc/arch-darwin-c-debug/lib</div>
<div>[0]PETSC ERROR: Configure run at Tue Aug 21 15:07:34 2012</div><div>[0]PETSC ERROR: Configure options --with-mpi-dir=/usr/local/include/openmpi --with-fc=0 --download-f2cblaslapack</div><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file</div><div>[2]PETSC ERROR: ------------------------------------------------------------------------</div><div>[2]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the batch system) has told this process to end</div>
<div>[2]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger</div><div>[2]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[2]PETSC" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[2]PETSC</a> ERROR: or try <a href="http://valgrind.org" target="_blank">http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors</div>
<div>[2]PETSC ERROR: likely location of problem given in stack below</div><div>[2]PETSC ERROR: --------------------- Stack Frames ------------------------------------</div><div>[2]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,</div>
<div>[2]PETSC ERROR: INSTEAD the line number of the start of the function</div><div>[2]PETSC ERROR: is given.</div><div>[2]PETSC ERROR: [2] VecScatterCreate_PtoS line 1631 src/vec/vec/utils/vpscat.c</div><div>
[2]PETSC ERROR: [2] VecScatterCreate line 844 src/vec/vec/utils/vscat.c</div><div>[2]PETSC ERROR: [2] MatSetUpMultiply_MPIAIJ line 26 src/mat/impls/aij/mpi/mmaij.c</div><div>[2]PETSC ERROR: [2] MatAssemblyEnd_MPIAIJ line 646 src/mat/impls/aij/mpi/mpiaij.c</div>
<div>[2]PETSC ERROR: [2] MatAssemblyEnd line 4857 src/mat/interface/matrix.c</div><div>[2]PETSC ERROR: --------------------- Error Message ------------------------------------</div><div>[2]PETSC ERROR: Signal received!</div>
<div>[2]PETSC ERROR: ------------------------------------------------------------------------</div><div>[2]PETSC ERROR: Petsc Release Version 3.3.0, Patch 2, Fri Jul 13 15:42:00 CDT 2012 </div><div>[2]PETSC ERROR: See docs/changes/index.html for recent updates.</div>
<div>[2]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div><div>[2]PETSC ERROR: See docs/index.html for manual pages.</div><div>[2]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[2]PETSC ERROR: ex2 on a arch-darw named <a href="http://ball-9120.ivt.ntnu.no" target="_blank">ball-9120.ivt.ntnu.no</a> by bihs Thu Aug 23 12:21:58 2012</div><div>[2]PETSC ERROR: Libraries linked from /Users/bihs/petsc/arch-darwin-c-debug/lib</div>
<div>[2]PETSC ERROR: Configure run at Tue Aug 21 15:07:34 2012</div><div>[2]PETSC ERROR: Configure options --with-mpi-dir=/usr/local/include/openmpi --with-fc=0 --download-f2cblaslapack</div><div>[2]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[2]PETSC ERROR: User provided function() line 0 in unknown directory unknown file</div></div><div><div><div><br></div><div>My machine has two 6 core Intel Xeon processors and have been running other programs that use MPI to run 12 processes.</div>
<div>Could someone shed some light on why this is happening?</div></div></div></blockquote><div><br></div><div>It looks like your MPI installation is bad. Try reconfiguring using --download-mpich</div><div><br></div><div>
Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div>Thanks.</div><span class="HOEnZb"><font color="#888888"><div><br></div>-- <br><div dir="ltr">
Arun Kamath <div>PhD Stipendiat</div><div>NTNU, Trondheim</div></div><br>
</font></span></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>