[mpich-discuss] Error in MPI_Init

Thu Feb 3 11:34:21 CST 2011

As I debugged my program as follows, when the error comes out, it looks using MPICH2.
It is using "PMPI_Init" from my MPICH2 library, "/home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib".
Is it still the problem from MPICH1?
Can you give me any advice for it?
Thanks.

---------------------------------------------------------------
(gdb) run
[Thread debugging using libthread_db enabled]
warning: the debug information found in "/soft/apps/packages/libgfortran/libgfortran.so.1.0.0" does not match "/soft/apps/packages/libgfortran/libgfortran.so.1" (CRC mismatch).

Internal Error: invalid error code f59ea0 (Ring Index out of range) in MPIR_Init_thread:394

Program received signal SIGSEGV, Segmentation fault.
0x00007fffed59035e in _IO_vfprintf_internal (s=0x7fffffffc600, format=<value optimized out>, ap=0x7fffffffcdf0) at vfprintf.c:1614
	in vfprintf.c
(gdb) bt
#0  0x00007fffed59035e in _IO_vfprintf_internal (s=0x7fffffffc600, format=<value optimized out>, ap=0x7fffffffcdf0) at vfprintf.c:1614
#1  0x00007fffed5918e0 in buffered_vfprintf (s=0x7fffed8c4860, format=0x7ffff58d64af "] %s Aborting program %s\n", args=0x28) at vfprintf.c:2254
#2  0x00007fffed58c5ae in _IO_vfprintf_internal (s=0x7fffed8c4860, format=0x7ffff58d64ac "[%d] %s Aborting program %s\n", ap=0x7fffffffcdf0) at vfprintf.c:1306
#3  0x00007fffed644aab in ___fprintf_chk (fp=0x7fffed8c4860, flag=1, format=0x2 <Address 0x2 out of bounds>) at fprintf_chk.c:37
#4  0x00007ffff5861494 in MPID_Abort () from /usr/lib/libhdf5.so.6
#5  0x00007fffee55a8c2 in handleFatalError () from /home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib/libmpich.so.1.2
#6  0x00007fffee55aacf in MPIR_Err_return_comm () from /home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib/libmpich.so.1.2
#7  0x00007fffee5752b0 in PMPI_Init () from /home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib/libmpich.so.1.2
#8  0x0000000000425c2c in main (argc=1, argv=0x7fffffffe5d8) at ParallelMesh.cpp:24
(gdb) 

-----------------------------
Hong-Jun Kim
Post-doc researcher
MCS, Argonne National Laboratory
9700 S. Cass Ave. B240/R2147
Argonne, IL 60439
630-252-4791
hongjun at mcs.anl.gov
-----------------------------

----- Original Message -----
> From: "Pavan Balaji" <balaji at mcs.anl.gov>
> To: mpich-discuss at mcs.anl.gov
> Cc: "Hong-Jun Kim" <hongjun at mcs.anl.gov>
> Sent: Wednesday, February 2, 2011 3:42:42 PM
> Subject: Re: [mpich-discuss] Error in MPI_Init
> The error is coming from MPICH-1, not MPICH-2. Looks like something
> went
> wrong in your build linking to MPICH2, instead of MPICH-1.
> 
> -- Pavan
> 
> On 02/02/2011 03:37 PM, Hong-Jun Kim wrote:
> > Hello,
> >
> > When I compile my program with "mpich2-1.2.1" and execute it, I have
> > the following error from "MPI_Init" function.
> >
> > -----------------------------------------------------
> > thrash:~/source/MeshKit/algs/test> ./ParallelMesh_test
> > Internal Error: invalid error code 2087ea0 (Ring Index out of range)
> > in MPIR_Init_thread:394
> > p0_28347: p4_error: interrupt SIGSEGV: 11
> > -----------------------------------------------------
> >
> > The error is coming both for serial and parallel executions.
> > The executable is compiled with the following options without any
> > problem.
> > I also attached a configure log file for my program, then please
> > give me any advices to solve it.
> > Thank you in advance.
> >
> > ----------------------------------------------------------
> > thrash:~/source/MeshKit/algs/test> make ParallelMesh_test
> > /homes/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/bin/mpicxx
> > -DHAVE_CONFIG_H -I. -I../.. -DHAVE_VSNPRINTF -D_FILE_OFFSET_BITS=64
> > -DHAVE_IGEOM -I/homes/hongjun/lib/CGM/include -DHAVE_IMESH
> > -I/homes/hongjun/lib/MOAB-new/include -DHAVE_IREL
> > -I/homes/hongjun/lib/Lasso/include -DHAVE_MOAB -DHAVE_CGM -DUSE_MPI
> > -DSRCDIR=. -I../.. -I../.. -I../../utils -I../../algs -I../../camal
> > -DVALGRIND -pipe -g -MT ParallelMesh.o -MD -MP -MF
> > .deps/ParallelMesh.Tpo -c -o ParallelMesh.o ParallelMesh.cpp
> > mv -f .deps/ParallelMesh.Tpo .deps/ParallelMesh.Po
> > /bin/sh ../../libtool --tag=CXX --mode=link
> > /homes/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/bin/mpicxx -pipe -g
> > -L/homes/hongjun/lib/CGM/lib
> > -L/homes/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib
> > -L/home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib
> > -L/home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib
> > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3
> > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3
> > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib -L/lib/../lib
> > -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../..
> > -L/usr/lib/x86_64-linux-gnu
> > -L/homes/fathom/3rdparty/netcdf/netcdf-4.0.1/parallel/gcc/lib
> > -L/homes/fathom/3rdparty/zlib/zlib-1.2.4/gcc/lib
> > -L/homes/fathom/3rdparty/szip/szip-2.1/gcc/lib
> > -L/homes/fathom/3rdparty/hdf5/hdf5-1.8.3/parallel/gcc/lib
> > -L/homes/hongjun/lib/CGM/lib
> > -L/homes/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib -o
> > ParallelMesh_test ParallelMesh.o ../libMKalgs.la
> > ../../camal/libMKcamal.la -lCMLEdgeMesher -lCMLTet -lCMLSweep
> > -lCMLDelTri -lCMLTri -lCML
> SurfMapper -lCMLPave -lCMLSizer -L/homes/hongjun/lib/Lasso/lib -liRel
> -lm -L/homes/fathom/3rdparty/netcdf/netcdf-4.0.1/parallel/gcc/lib
> -L/homes/fathom/3rdparty/zlib/zlib-1.2.4/gcc/lib
> -L/homes/fathom/3rdparty/szip/szip-2.1/gcc/lib
> -L/homes/fathom/3rdparty/hdf5/hdf5-1.8.3/parallel/gcc/lib
> -L/homes/hongjun/lib/CGM/lib
> -L/homes/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib
> -L/homes/hongjun/lib/MOAB-new/lib -L/homes/hongjun/lib/MOAB-new/lib
> -liMesh -lMOAB -lnetcdf -lhdf5_hl -lhdf5 -lcurl -lhdf5 -lz -lm
> -L/home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib
> -L/home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib
> -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3
> -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3
> -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib -L/lib/../lib
> -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../..
> -L/usr/lib/x86_64-linux-gnu -lmpichcxx -lmpich -lopa -lpthread -lrt
> -lstdc++ -L/homes/hongjun/lib/CGM/lib
> -L/homes/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib -lcg
> m -lTKSTL -lTKSTEP -lTKSTEP209 -lTKSTEPAttr -lTKSTEPBase -lTKXSBase
> -lTKIGES -lTKXSBase -lTKLCAF -lTKBinL -lTKHLR -lTKOffset -lTKShHealing
> -lTKFillet -lTKFeat -lTKBool -lTKBO -lTKPrim -lTKMesh -lTKTopAlgo
> -lTKGeomAlgo -lTKBRep -lTKGeomBase -lTKG3d -lTKG2d -lTKMath -lTKernel
> -ldl -L/homes/hongjun/lib/CGM/lib
> -L/homes/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib
> -L/home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib
> -L/home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib
> -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3
> -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3
> -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib -L/lib/../lib
> -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../..
> -L/usr/lib/x86_64-linux-gnu -liGeom -lcgm -lTKSTL -lTKSTEP -lTKSTEP209
> -lTKSTEPAttr -lTKSTEPBase -lTKXSBase -lTKIGES -lTKXSBase -lTKLCAF
> -lTKBinL -lTKHLR -lTKOffset -lTKShHealing -lTKFillet -lTKFeat -lTKBool
> -lTKBO -lTKPrim -lTKMesh -lTKTopAlgo -lTKGeomAlgo -lTKBRep
> -lTKGeomBase -lTKG3d -lTKG2d -lTKMath -lTKe
> rnel -ldl -lmpichcxx -lmpich -lopa -lpthread -lrt -lstdc++ -lm -lgcc_s
> -lc -lgcc_s -lm
> > libtool: link:
> > /homes/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/bin/mpicxx -pipe -g
> > -o ParallelMesh_test ParallelMesh.o -L/homes/hongjun/lib/CGM/lib
> > -L/homes/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib
> > -L/home/fathom/3rdparty/mpich2/mpich2-1.2.1/gcc/lib
> > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3
> > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib -L/lib/../lib
> > -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../..
> > -L/usr/lib/x86_64-linux-gnu
> > -L/homes/fathom/3rdparty/netcdf/netcdf-4.0.1/parallel/gcc/lib
> > -L/homes/fathom/3rdparty/zlib/zlib-1.2.4/gcc/lib
> > -L/homes/fathom/3rdparty/szip/szip-2.1/gcc/lib
> > -L/homes/fathom/3rdparty/hdf5/hdf5-1.8.3/parallel/gcc/lib
> > ../.libs/libMKalgs.a -L/homes/hongjun/lib/Lasso/lib
> > -L/home/fathom/3rdparty/hdf5/hdf5-1.8.3/parallel/gcc/lib
> > -L/home/fathom/3rdparty/zlib/zlib-1.2.4/gcc/lib
> > -L/home/fathom/3rdparty/szip/szip-2.1/gcc/lib
> > -L/homes/hongjun/lib/MOAB-new/lib -L/usr/lib/mpich/lib/
> > ../../camal/.libs/libMKcamal.a -L/homes/hongjun/lib/camal.5.
> 1.0-Linux64/lib/Linux64 -lCMLEdgeMesher -lCMLTet -lCMLSweep
> -lCMLDelTri -lCMLTri -lCMLSurfMapper -lCMLPave -lCMLSizer
> /homes/hongjun/lib/Lasso/lib/libiRel.a
> /homes/hongjun/lib/MOAB-new/lib/libiMesh.a
> /homes/hongjun/lib/MOAB-new/lib/libMOAB.a
> /home/fathom/3rdparty/netcdf/netcdf-4.0.1/parallel/gcc/lib/libnetcdf.so
> -lnetcdf /usr/lib/libhdf5_hl.so /usr/lib/libcurl.so
> /usr/lib/libhdf5.so -lz /homes/hongjun/lib/CGM/lib/libiGeom.a
> /homes/hongjun/lib/CGM/lib/libcgm.a
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKSTL.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKSTEP.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKSTEP209.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKSTEPAttr.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKSTEPBase.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKIGES.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKXSBase.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKBinL.so /h
> ome/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKLCAF.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKCDF.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKHLR.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKOffset.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKShHealing.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKFillet.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKFeat.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKBool.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKBO.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKPrim.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKMesh.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKTopAlgo.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKGeomAlgo.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKBRep.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKGeomBase.so
> /home/fathom/3rd
> party/occ/OpenCASCADE6.3.0/gcc/lib/libTKG3d.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKG2d.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKMath.so
> /home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib/libTKernel.so -ldl
> -lmpichcxx -lmpich -lopa -lpthread -lrt -lstdc++ -lc -lgcc_s -lm
> -Wl,-rpath
> -Wl,/home/fathom/3rdparty/netcdf/netcdf-4.0.1/parallel/gcc/lib
> -Wl,-rpath -Wl,/home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib
> -Wl,-rpath
> -Wl,/home/fathom/3rdparty/netcdf/netcdf-4.0.1/parallel/gcc/lib
> -Wl,-rpath -Wl,/home/fathom/3rdparty/occ/OpenCASCADE6.3.0/gcc/lib
> > /homes/hongjun/lib/CGM/lib/libiGeom.a(iGeom_CGMA.o): In function
> > `iGeom_load_cub_geometry':
> > /homes/hongjun/source/CGM/itaps/iGeom_CGMA.cc:6265: warning: the use
> > of `tmpnam' is dangerous, better use `mkstemp'
> >
> > -----------------------------
> > Hong-Jun Kim
> > Post-doc researcher
> > MCS, Argonne National Laboratory
> > 9700 S. Cass Ave. B240/R2147
> > Argonne, IL 60439
> > 630-252-4791
> > hongjun at mcs.anl.gov
> > -----------------------------
> >
> >
> >
> >
> > _______________________________________________
> > mpich-discuss mailing list
> > mpich-discuss at mcs.anl.gov
> > https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
> 
> --
> Pavan Balaji
> http://www.mcs.anl.gov/~balaji