==10672== Memcheck, a memory error detector ==10672== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al. ==10672== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info ==10672== Command: mpirun -np 2 /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG ==10672== ==10672== Syscall param sched_setaffinity(mask) points to unaddressable byte(s) ==10672== at 0x37644E5159: syscall (in /lib64/libc-2.12.so) ==10672== by 0x54D74D8: opal_paffinity_linux_plpa_api_probe_init (in /usr/local/openmpi-1.4.5/lib/openmpi/mca_paffinity_linux.so) ==10672== by 0x54D7B7C: opal_paffinity_linux_plpa_init (in /usr/local/openmpi-1.4.5/lib/openmpi/mca_paffinity_linux.so) ==10672== by 0x54D8539: opal_paffinity_linux_plpa_have_topology_information (in /usr/local/openmpi-1.4.5/lib/openmpi/mca_paffinity_linux.so) ==10672== by 0x54D737F: linux_module_init (in /usr/local/openmpi-1.4.5/lib/openmpi/mca_paffinity_linux.so) ==10672== by 0x4E994DA: opal_paffinity_base_select (in /usr/local/openmpi-1.4.5/lib/libopen-pal.so.0.0.0) ==10672== by 0x4E6D464: opal_init (in /usr/local/openmpi-1.4.5/lib/libopen-pal.so.0.0.0) ==10672== by 0x4C1C7CC: orte_init (in /usr/local/openmpi-1.4.5/lib/libopen-rte.so.0.0.0) ==10672== by 0x402F0E: orterun (orterun.c:542) ==10672== by 0x402AF3: main (main.c:13) ==10672== Address 0x0 is not stack'd, malloc'd or (recently) free'd ==10672== [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Invalid argument! [0]PETSC ERROR: Scalar value must be same on all processes, argument # 3! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24 CDT 2013 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG on a linux named anakin by sgao Wed Mar 11 15:24:43 2015 [0]PETSC ERROR: Libraries linked from /tmp/PETSC33/petsc-3.3-p7/linux/lib [0]PETSC ERROR: Configure run at Wed Jan 15 12:04:54 2014 [0]PETSC ERROR: Configure options --with-mpi-dir=/usr/local.linux64/lib64/MPI-openmpi-1.4.5/ --with-shared-libraries=0 --COPTFLAGS=-g --FOPTFLAGS=-g --with-debugging=yes [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: VecMAXPY() line 1186 in src/vec/vec/interface/rvector.c [0]PETSC ERROR: KSPGMRESBuildSoln() line 345 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: KSPGMRESCycle() line 206 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: KSPSolve_GMRES() line 231 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: KSPSolve() line 446 in src/ksp/ksp/interface/itfunc.c [anakin:10675] *** Process received signal *** [anakin:10675] Signal: Aborted (6) [anakin:10675] Signal code: (-6) [anakin:10675] [ 0] /lib64/libpthread.so.0() [0x376480f710] [anakin:10675] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x3764432625] [anakin:10675] [ 2] /lib64/libc.so.6(abort+0x175) [0x3764433e05] [anakin:10675] [ 3] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(PetscTraceBackErrorHandler+0x627) [0x1313d67] [anakin:10675] [ 4] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(PetscError+0x38f) [0x130ce99] [anakin:10675] [ 5] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(VecMAXPY+0xdc2) [0x13f8184] [anakin:10675] [ 6] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG() [0x1581062] [anakin:10675] [ 7] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(KSPGMRESCycle+0x10bb) [0x157ea07] [anakin:10675] [ 8] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(KSPSolve_GMRES+0x42c) [0x157ef7c] [anakin:10675] [ 9] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(KSPSolve+0x1e1b) [0x1d1df0b] [anakin:10675] [10] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(kspsolve_+0x3b) [0x1571c05] [anakin:10675] [11] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(petsolv_+0x2680) [0xd55a0c] [anakin:10675] [12] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(flowsol_ng_+0x1dcd) [0x689615] [anakin:10675] [13] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(iterprc_+0x2d74) [0x732a58] [anakin:10675] [14] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(solnalg_+0xc54b) [0x7861d7] [anakin:10675] [15] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(flow_+0x84f2) [0x682992] [anakin:10675] [16] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(mpi_box_+0x73) [0x740027] [anakin:10675] [17] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(main+0x7c7) [0x9e5d07] [anakin:10675] [18] /lib64/libc.so.6(__libc_start_main+0xfd) [0x376441ed5d] [anakin:10675] [19] /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG(_ZNSt8ios_base4InitD1Ev+0x82) [0x4bbaea] [anakin:10675] *** End of error message *** -------------------------------------------------------------------------- mpirun noticed that process rank 1 with PID 10675 on node anakin exited on signal 6 (Aborted). -------------------------------------------------------------------------- [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the batch system) has told this process to end [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors [0]PETSC ERROR: likely location of problem given in stack below [0]PETSC ERROR: --------------------- Stack Frames ------------------------------------ [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, [0]PETSC ERROR: INSTEAD the line number of the start of the function [0]PETSC ERROR: is given. [0]PETSC ERROR: [0] PetscCommDuplicate line 144 src/sys/objects/tagm.c [0]PETSC ERROR: [0] PetscHeaderCreate_Private line 30 src/sys/objects/inherit.c [0]PETSC ERROR: [0] PetscViewerCreate line 33 src/sys/viewer/interface/viewreg.c [0]PETSC ERROR: [0] PetscViewerBinaryOpen line 699 src/sys/viewer/impls/binary/binv.c [0]PETSC ERROR: [0] VecMAXPY line 1174 src/vec/vec/interface/rvector.c [0]PETSC ERROR: [0] KSPGMRESBuildSoln line 316 src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: [0] KSPGMRESCycle line 126 src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: [0] KSPSolve_GMRES line 220 src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: [0] KSPSolve line 351 src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Signal received! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24 CDT 2013 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: /home/cfd/sgao/mycodes/fensap_new_edge_coefficient/fensapng-mf-newmuscl-overledg_org/bin/fensapMPI_LINUX64_DEBUG on a linux named anakin by sgao Wed Mar 11 15:24:43 2015 [0]PETSC ERROR: Libraries linked from /tmp/PETSC33/petsc-3.3-p7/linux/lib [0]PETSC ERROR: Configure run at Wed Jan 15 12:04:54 2014 [0]PETSC ERROR: Configure options --with-mpi-dir=/usr/local.linux64/lib64/MPI-openmpi-1.4.5/ --with-shared-libraries=0 --COPTFLAGS=-g --FOPTFLAGS=-g --with-debugging=yes [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file ==10672== ==10672== HEAP SUMMARY: ==10672== in use at exit: 245,020 bytes in 1,426 blocks ==10672== total heap usage: 10,456 allocs, 9,030 frees, 14,768,577 bytes allocated ==10672== ==10672== LEAK SUMMARY: ==10672== definitely lost: 14,645 bytes in 49 blocks ==10672== indirectly lost: 1,520 bytes in 30 blocks ==10672== possibly lost: 0 bytes in 0 blocks ==10672== still reachable: 228,855 bytes in 1,347 blocks ==10672== suppressed: 0 bytes in 0 blocks ==10672== Rerun with --leak-check=full to see details of leaked memory ==10672== ==10672== For counts of detected and suppressed errors, rerun with: -v ==10672== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 6 from 6)