[mpich-discuss] Problem using Total View with mpiexec.hydra

[Pavan Balaji]

Hi Anatoly,

We have seen this problem before.  It has something to do with the 
interaction between tcsh and gdb, but we haven't quite narrowed it down yet.

But in your particular case, you might be able to workaround it since 
totalview is a parallel debugger.  You can use it to manage all 
processes, instead of having a separate instance for each process, so 
there will not be any fd forwarding for each process.

See 
http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager#Debugging

  -- Pavan

On 05/10/2012 12:23 AM, Anatoly G wrote:
> Dear MPICH2,
> I have following short endless program.
> 1) I perform MPI_Init
> 2) MPI_Recv without any send. Program is stalled (as expected).
>
> I"m using  MPICH2
> I execute program on Linux Red Hat 64 bit Server  release 6.0 (Santiago).
> I use /bin/tcsh.
>
> I want to debug only first process (rank 0) of program with
> TotalView (TotalView 8.9.0-0)
>
> My execute line is
> /space/local/mpich2/bin/mpiexec.hydra -genvall -disable-auto-cleanup -f
> machines.txt -launcher=rsh -n 1 /usr/bin/tv mpi_endless : -n 1 mpi_endless
>
> Total View is starting.
> I start step-by-step debug, but Total View fails on any of 3 MPI_Init
> lines (I tried each one of them - 3 different runs):
> *)  // MPI_Init_thread( 0, 0, MPI_THREAD_MULTIPLE, &provided );
> *)  // MPI_Init_thread( 0, 0, MPI_THREAD_SINGLE, &provided );
> *)   MPI_Init(&argc, &argv);
>
> *_The error message that I see in shell window:_*
> system msg for write_line failure : Bad file descriptor
> [cli_0]: Unable to write to PMI_fd
> [cli_0]: write_line error; fd=6 buf=:cmd=get_appnum
> :
> system msg for write_line failure : Bad file descriptor
> Fatal error in MPI_Init: Other MPI error, error stack:
> MPIR_Init_thread(411):
> MPID_Init(138).......: channel initialization failed
> MPID_Init(423).......: PMI_Get_appnum returned -1
>
>
>
> Can you please advice me?
>
>
> Anatoly.
>
>
>
> _______________________________________________
> mpich-discuss mailing list     mpich-discuss at mcs.anl.gov
> To manage subscription options or unsubscribe:
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji