pnetcdf nfmpi_put_vara_real_all problem

[Wong, David]

Hi Wei-keng,

     It failed at the following step:

         err = nfmpi_create(MPI_COMM_WORLD, filename, NF_CLOBBER,
     +                       MPI_INFO_NULL, ncid)

with the following error message (from one of the processor):

Rank 1 [Thu Sep 19 04:27:50 2013] [c5-0c1s4n3] Fatal error in MPI_Comm_test_inter: Invalid communicator, error stack:
MPI_Comm_test_inter(110): MPI_Comm_test_inter(comm=0x84000002, flag=0x7fffffff8f90) failed
MPI_Comm_test_inter(83).: Invalid communicator
Rank 2 [Thu Sep 19 04:27:50 2013] [c5-0c1s4n3] Fatal error in MPI_Comm_test_inter: Invalid communicator, error stack:
MPI_Comm_test_inter(110): MPI_Comm_test_inter(comm=0x84000002, flag=0x7fffffff8f90) failed
MPI_Comm_test_inter(83).: Invalid communicator
Rank 3 [Thu Sep 19 04:27:50 2013] [c5-0c1s4n3] Fatal error in MPI_Comm_test_inter: Invalid communicator, error stack:
MPI_Comm_test_inter(110): MPI_Comm_test_inter(comm=0x84000002, flag=0x7fffffff8f90) failed
MPI_Comm_test_inter(83).: Invalid communicator
Rank 0 [Thu Sep 19 04:27:50 2013] [c5-0c1s4n3] Fatal error in MPI_Comm_test_inter: Invalid communicator, error stack:
MPI_Comm_test_inter(110): MPI_Comm_test_inter(comm=0x84000004, flag=0x7fffffff8f90) failed
MPI_Comm_test_inter(83).: Invalid communicator
forrtl: error (76): Abort trap signal
Image              PC                Routine            Line        Source
libc.so.6          00002AAAAB92CB35  Unknown               Unknown  Unknown
libc.so.6          00002AAAAB92E111  Unknown               Unknown  Unknown
epa.x              0000000000435E42  Unknown               Unknown  Unknown
epa.x              000000000042B160  Unknown               Unknown  Unknown
epa.x              000000000042B30D  Unknown               Unknown  Unknown
libmpich_intel.so  00002AAAAC248B83  Unknown               Unknown  Unknown
epa.x              0000000000523C3E  Unknown               Unknown  Unknown
epa.x              000000000051E508  Unknown               Unknown  Unknown
epa.x              000000000051DBC5  Unknown               Unknown  Unknown
epa.x              00000000004245EF  MAIN__                     33  epa.F
epa.x              000000000042454C  Unknown               Unknown  Unknown
libc.so.6          00002AAAAB918C16  Unknown               Unknown  Unknown
epa.x              000000000042444D  Unknown               Unknown  Unknown

    Regarding to Rob's suggestion, I am using aprun or mpirun to launch the executable so I don't know how to invoke valgrind at this time.

Cheers,
David

--
David C. Wong Ph.D.
Atmospheric Modeling and Analysis Division
National Exposure Research Laboratory
US Environmental Protection Agency
Mail Drop E243-03
109 T. W. Alexander Dr.
Research Triangle Park, NC 27711
919-541-3400 919-541-1379 (fax)

________________________________________
From: Wei-keng Liao <wkliao at ece.northwestern.edu>
Sent: Wednesday, September 18, 2013 7:13 PM
To: Wong, David
Cc: parallel-netcdf at mcs.anl.gov
Subject: Re: pnetcdf nfmpi_put_vara_real_all problem

Hi, David,

Could you please try the attached program and let us know if it
generates the same error? It is written based on the information
you provided.