<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi,</p>
<p>I'm trying to run my MPI CFD code on Japan's K computer. My code
can run if I didn't make use of the PETSc DMDAVecGetArrayF90
subroutine. If it's called</p>
<p>call DMDAVecGetArrayF90(da_u,u_local,u_array,ierr)</p>
<p>I get the error below. I have no problem with my code on other
clusters using the new Intel compilers. I used to have problems
with DM when using the old Intel compilers. Now on the K computer,
I'm using Fujitsu's Fortran compiler. How can I troubleshoot?</p>
<p>Btw, I also tested on the ex13f90 example and it didn't work too.
The error is below.<br>
</p>
<p><br>
</p>
<p>My code error:<br>
</p>
<i><b> size_x,size_y,size_z 76x130x136</b></i><i><b><br>
</b></i><i><b> total grid size = 1343680</b></i><i><b><br>
</b></i><i><b> recommended cores (50k / core) = 26.87360000000000</b></i><i><b><br>
</b></i><i><b> 0</b></i><i><b><br>
</b></i><i><b> 1</b></i><i><b><br>
</b></i><i><b> 1</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: [1]PETSC ERROR:
------------------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Caught signal number 11 SEGV:
Segmentation Violation, probably memory access out of range</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Try option -start_in_debugger or
-on_error_attach_debugger</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: or see
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a></b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: or try <a class="moz-txt-link-freetext" href="http://valgrind.org">http://valgrind.org</a> on
GNU/linux and Apple Mac OS X to find memory corruption errors</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: likely location of problem given in
stack below</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: --------------------- Stack Frames
------------------------------------</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Note: The EXACT line numbers in the
stack are not available,</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: INSTEAD the line number of the
start of the function</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: is given.</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: [1] F90Array3dCreate line 244
/.global/volume2/home/hp150306/t00196/source/petsc-3.6.3/src/sys/f90-src/f90_cwrap.c</b></i><i><b><br>
</b></i><i><b> 1</b></i><i><b><br>
</b></i><i><b>------------------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: Caught signal number 11 SEGV:
Segmentation Violation, probably memory access out of range</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: Try option -start_in_debugger or
-on_error_attach_debugger</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: or see
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a></b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: or try <a class="moz-txt-link-freetext" href="http://valgrind.org">http://valgrind.org</a> on
GNU/linux and Apple Mac OS X to find memory corruption errors</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: likely location of problem given in
stack below</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: --------------------- Stack Frames
------------------------------------</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR:
------------------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: Caught signal number 11 SEGV:
Segmentation Violation, probably memory access out of range</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: Try option -start_in_debugger or
-on_error_attach_debugger</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: or see
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a></b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: or try <a class="moz-txt-link-freetext" href="http://valgrind.org">http://valgrind.org</a> on
GNU/linux and Apple Mac OS X to find memory corruption errors</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: likely location of problem given in
stack below</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: --------------------- Stack Frames
------------------------------------</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: Note: The EXACT line numbers in the
stack are not available,</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: INSTEAD the line number of the
start of the function</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: is given.</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: [0] F90Array3dCreate line 244
/.global/volume2/home/hp150306/t00196/source/petsc-3.6.3/src/sys/f90-src/f90_cwrap.c</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: --------------------- Error Message
----------------------------------------- 1</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR:
------------------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: Caught signal number 11 SEGV:
Segmentation Violation, probably memory access out of range</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: Try option -start_in_debugger or
-on_error_attach_debugger</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: or see
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a></b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: or try <a class="moz-txt-link-freetext" href="http://valgrind.org">http://valgrind.org</a> on
GNU/linux and Apple Mac OS X to find memory corruption errors</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: likely location of problem given in
stack below</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: --------------------- Stack Frames
------------------------------------</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: Note: The EXACT line numbers in the
stack are not available,</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: INSTEAD the line number of the
start of the function</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: is given.</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: [2] F90Array3dCreate line 244
/.global/volume2/home/hp150306/t00196/source/petsc-3.6.3/src/sys/f90-src/f90_cwrap.c</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: --------------------- Error Message
-----------------------------------------[3]PETSC ERROR: Note:
The EXACT line numbers in the stack are not available,</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: INSTEAD the line number of the
start of the function</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: is given.</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: [3] F90Array3dCreate line 244
/.global/volume2/home/hp150306/t00196/source/petsc-3.6.3/src/sys/f90-src/f90_cwrap.c</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: Signal received</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: See
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble
shooting.</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: Petsc Release Version 3.6.3, Dec,
03, 2015</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: ./a-debug.out on a petsc-3.6.3_debug
named b04-036 by Unknown Wed Jun 1 12:54:34 2016</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: Configure options --with-cc=mpifcc
--with-cxx=mpiFCC --with-fc=mpifrt --with-64-bit-pointers=1
--CC=mpifcc --CFLAGS="-Xg -O0" --CXX=mpiFCC --CXXFLAGS="-Xg -O0"
--FC=mpifrt --FFLAGS="-X9 -O0" --LD_SHARED= --LDDFLAGS=
--with-openmp=1 --with-mpiexec=mpiexec --known-endian=big
--with-shared----------------------</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: Signal received</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: See
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble
shooting.</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: Petsc Release Version 3.6.3, Dec,
03, 2015</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: ./a-debug.out on a petsc-3.6.3_debug
named b04-036 by Unknown Wed Jun 1 12:54:34 2016</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: Configure options --with-cc=mpifcc
--with-cxx=mpiFCC --with-fc=mpifrt --with-64-bit-pointers=1
--CC=mpifcc --CFLAGS="-Xg -O0" --CXX=mpiFCC --CXXFLAGS="-Xg -O0"
--FC=mpifrt --FFLAGS="-X9 -O0" --LD_SHARED= --LDDFLAGS=
--with-openmp=1 --with-mpiexec=mpiexec --known-endian=big
--with-shared-libraries=0 --with-blas-lapack-lib=-SSL2
--with-scalapack-lib=-SCALAPACK
--prefix=/home/hp150306/t00196/lib/petsc-3.6.3_debug
--with-fortran-interfaces=1 --with-debugging=1 --useThreads=0
--with-hypre=1
--with-hypre-dir=/home/hp150306/t00196/lib/hypre-2.10.0b-p4</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: #1 User provided function() line 0
in unknown file</b></i><i><b><br>
</b></i><i><b>--------------------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[m---------------------</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: Signal received</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: See
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble
shooting.</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: Petsc Release Version 3.6.3, Dec,
03, 2015</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: ./a-debug.out on a petsc-3.6.3_debug
named b04-036 by Unknown Wed Jun 1 12:54:34 2016</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: Configure options --with-cc=mpifcc
--with-cxx=mpiFCC --with-fc=mpifrt --with-64-bit-pointers=1
--CC=mpifcc --CFLAGS="-Xg -O0" --CXX=mpiFCC --CXXFLAGS="-Xg -O0"
--FC=mpifrt --FFLAGS="-X9 -O0" --LD_SHARED= --LDDFLAGS=
--with-openmp=1 --with-mpiexec=mpiexec --known-endian=big
--with-shared-libraries=0 --with-blas-lapack-lib=-SSL2
--with-scalapack-lib=-SCALAPACK
--prefix=/home/hp150306/t00196/lib/petsc-3.6.3_debug
--with-fortran-interfaces=1 --with-debugging=1 --useThreads=0
--with-hypre=1
--with-hypre-dir=/home/hp150306/t00196/lib/hypre-2.10.0b-p4</b></i><i><b><br>
</b></i><i><b>[2]PETSC ERROR: #1 User provided function() line 0
in unknown file</b></i><i><b><br>
</b></i><i><b>--------------------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[m[1]PETSC ERROR: --------------------- Error
Message
--------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Signal received</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: See
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble
shooting.</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Petsc Release Version 3.6.3, Dec,
03, 2015</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: ./a-debug.out on a petsc-3.6.3_debug
named b04-036 by Unknown Wed Jun 1 12:54:34 2016</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Configure options --with-cc=mpifcc
--with-cxx=mpiFCC --with-fc=mpifrt --with-64-bit-pointers=1
--CC=mpifcc --CFLAGS="-Xg -O0" --CXX=mpiFCC --CXXFLAGS="-Xg -O0"
--FC=mpifrt --FFLAGS="-X9 -O0" --LD_SHARED= --LDDFLAGS=
--with-openmp=1 --with-mpiexec=mpiexec --known-endian=big
--with-shared-libraries=0 --with-blas-lapack-lib=-SSL2
--with-scalapack-lib=-SCALAPACK
--prefix=/home/hp150306/t00196/lib/petsc-3.6.3_debug
--with-fortran-interfaces=1 --with-debugging=1 --useThreads=0
--with-hypre=1
--with-hypre-dir=/home/hp150306/t00196/lib/hypre-2.10.0b-p4</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: #1 User provided function() line 0
ilibraries=0 --with-blas-lapack-lib=-SSL2
--with-scalapack-lib=-SCALAPACK
--prefix=/home/hp150306/t00196/lib/petsc-3.6.3_debug
--with-fortran-interfaces=1 --with-debugging=1 --useThreads=0
--with-hypre=1
--with-hypre-dir=/home/hp150306/t00196/lib/hypre-2.10.0b-p4</b></i><i><b><br>
</b></i><i><b>[3]PETSC ERROR: #1 User provided function() line 0
in unknown file</b></i><i><b><br>
</b></i><i><b>--------------------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[mpi::mpi-api::mpi-abort]</b></i><i><b><br>
</b></i><i><b>MPI_ABORT was invoked on rank 3 in communicator
MPI_COMM_WORLD</b></i><i><b><br>
</b></i><i><b>with errorcode 59.</b></i><i><b><br>
</b></i><i><b><br>
</b></i><i><b>NOTE: invoking MPI_ABORT causes Open MPI to kill all
MPI processes.</b></i><i><b><br>
</b></i><i><b>You may or may not see output from other processes,
depending on</b></i><i><b><br>
</b></i><i><b>exactly when Open MPI kills them.</b></i><i><b><br>
</b></i><i><b>--------------------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[b04-036:28416]
/opt/FJSVtclang/GM-1.2.0-20/lib64/libmpi.so.0(orte_errmgr_base_error_abort+0x84)
[0xffffffff11360404]</b></i><i><b><br>
</b></i><i><b>[b04-036:28416]
/opt/FJSVtclang/GM-1.2.0-20/lib64/libmpi.so.0(ompi_mpi_abort+0x51c)
[0xffffffff1110391c]</b></i><i><b><br>
</b></i><i><b>[b04-036:28416]
/opt/FJSVtclang/GM-1.2.0-2pi::mpi-api::mpi-abort]</b></i><i><b><br>
</b></i><i><b>MPI_ABORT was invoked on rank 2 in communicator
MPI_COMM_WORLD</b></i><i><b><br>
</b></i><i><b>with errorcode 59.</b></i><br>
<br>
<p>ex13f90 error:</p>
<p><br>
</p>
<p><i><b>[t00196@b04-036 tutorials]$ mpiexec -np 2 ./ex13f90</b></i><i><b><br>
</b></i><i><b>jwe1050i-w The hardware barrier couldn't be used
and continues processing using the software barrier.</b></i><i><b><br>
</b></i><i><b>taken to (standard) corrective action, execution
continuing.</b></i><i><b><br>
</b></i><i><b>jwe1050i-w The hardware barrier couldn't be used
and continues processing using the software barrier.</b></i><i><b><br>
</b></i><i><b>taken to (standard) corrective action, execution
continuing.</b></i><i><b><br>
</b></i><i><b> Hi! We're solving van der Pol using 2
processes.</b></i><i><b><br>
</b></i><i><b><br>
</b></i><i><b> t x1 x2</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR:
------------------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Caught signal number 10 BUS: Bus
Error, possibly illegal memory access</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Try option -start_in_debugger or
-on_error_attach_debugger</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR:
------------------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: Caught signal number 10 BUS: Bus
Error, possibly illegal memory access</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: Try option -start_in_debugger or
-on_error_attach_debugger</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: or see
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a></b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: or try <a class="moz-txt-link-freetext" href="http://valgrind.org">http://valgrind.org</a> on
GNU/linux and Apple Mac OS X to find memory corruption errors</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: or see
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a></b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: or try <a class="moz-txt-link-freetext" href="http://valgrind.org">http://valgrind.org</a> on
GNU/linux and Apple Mac OS X to find memory corruption errors</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: likely location of problem given
in stack below</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: --------------------- Stack
Frames ------------------------------------</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Note: The EXACT line numbers in
the stack are not available,</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: INSTEAD the line number of
the start of the function</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: is given.</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: [1] F90Array4dCreate line 337
/.global/volume2/home/hp150306/t00196/source/petsc-3.6.3/src/sys/f90-src/f90_cwrap.c</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: likely location of problem given
in stack below</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: --------------------- Stack
Frames ------------------------------------</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: Note: The EXACT line numbers in
the stack are not available,</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: INSTEAD the line number of
the start of the function</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: is given.</b></i><i><b><br>
</b></i><i><b>[0]PETSC ERROR: [0] F90Array4dCreate line 337
/.global/volume2/home/hp150306/t00196/source/petsc-3.6.3/src/sys/f90-src/f90_cwrap.c</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: --------------------- Error
Message
--------------------------------------------------------------</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Signal received</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: See
<a class="moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for
trouble shooting.</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Petsc Release Version 3.6.3, Dec,
03, 2015</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: ./ex13f90 on a petsc-3.6.3_debug
named b04-036 by Unknown Wed Jun 1 13:04:34 2016</b></i><i><b><br>
</b></i><i><b>[1]PETSC ERROR: Configure options --with-cc=mpifcc
--with-cxx=mpiFCC --with-fc=mpifrt --with-64-bit-pointers=1
--CC=mpifcc --CFLAGS="-Xg -O0" --CXX=mpiFCC --CXXFLAGS="-Xg
-O0" --FC=mpifrt --FFLAGS="-X9 -O0" --LD_SHARED= --LDDFLAGS=
--with-openmp=1 --with-mpiexec=mpiexec --known-endian=big
--with-shared-libraries=0 --with-blas-lapack-lib=-SSL2
--with-scalapack-lib=-SCALAPACK
--prefix=/home/hp150306/t00196/lib/petsc-3.6.3_debug
--with-fortran-interfaces=1 --with-debugging=1 --useThreads=0
--with-hypre=1 --with-hyp</b></i><i><b><br>
</b></i></p>
<br>
<pre class="moz-signature" cols="72">--
Thank you
Yours sincerely,
TAY wee-beng</pre>
</body>
</html>