[MPICH] debug flag

[Wei-keng Liao]

Rob,

I can see from adio/common/malloc.c that if malloc(), calloc(), or 
realloc() fails, an error message "Out of memory in file ..." will be 
printed and MPI_Abort(MPI_COMM_WORLD, 1) called. But, I did not see the 
"Out of memory" in my error message.

If the MPICH2 is not configured with --enable-g=dbg, will
   FPRINTF(stderr, "Out of memory in file %s, line %d\n", fname, lineno);
be seen?

I have sent email to the system and ask what options were used for that 
build of MPICH2-1.0.2.

If it was built without option --enable-g=dbg, what debug flags I can use 
to get as much error info from MPICH as possible?

Wei-keng



On Fri, 25 May 2007, Robert Latham wrote:

> On Fri, May 25, 2007 at 03:56:16PM -0500, Wei-keng Liao wrote:
>>
>> I have an MPI I/O application that runs fine up to 1000 processes, but
>> failed when using 4000 processes. Parts of error message are
>>     ...
>>     Stack Trace:  ------------------------------
>>     #0  0x00000000002d46fe in ADIOI_Calc_my_req()
>>     #1  0x00000000002d2280 in ADIOI_GEN_WriteStridedColl()
>>     #2  0x00000000002a397c in MPIOI_File_write_all()
>>     #3  0x00000000002a3a4a in PMPI_File_write_all()
>>     #4  0x00000000002913a8 in pmpi_file_write_all_()
>>     could not find symbol for addr 0x73696e6966204f49
>>     aborting job:
>>     application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1456
>>     ...
>>
>> My question is what debug flags should I use for compiling and running in
>> order to help find what exact location in function ADIOI_Calc_my_req()
>> causes this error?
>
> Hi Wei-keng
>
> If you build MPICH2 with --enable-g=dbg, then all of MPI will be built
> with debugging symbols.   Be sure to 'make clean' first: the ROMIO
> objects might not rebuild otherwise.
>
> I wonder what caused the abort?  maybe ADIOI_Malloc failed to allocate
> memory?  Well, a stack trace with debugging symbols should be
> interesting.
>
> ==rob
>
> -- 
> Rob Latham
> Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
> Argonne National Lab, IL USA                 B29D F333 664A 4280 315B
>