[mpich-discuss] MPI error

Wed Apr 18 14:34:45 CDT 2012

Hi there,

Can someone help me on the following issue? The MPI program we are using 
works fine with single PC of all OS (from XP to Win7, 32 and 64). However, 
when we run it on a cluster, for some cases, we got the following error. 
We are using MPICH2-1.2.1-win-ia32. 

Thanks you.

Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)

job aborted:
rank: node: exit code[: error message]
0: 10.0.0.52: 157: process 0 exited without calling finalize
1: 10.0.0.52: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
2: 10.0.0.52: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
3: 10.0.0.52: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
4: 10.0.0.52: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
5: 10.0.0.52: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
6: 10.0.0.52: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
7: 10.0.0.52: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
8: 10.0.0.61: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
9: 10.0.0.61: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
10: 10.0.0.61: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
11: 10.0.0.61: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
12: 10.0.0.61: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
13: 10.0.0.61: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
14: 10.0.0.61: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
15: 10.0.0.61: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
16: 10.0.0.59: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
17: 10.0.0.59: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
18: 10.0.0.59: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
19: 10.0.0.59: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
20: 10.0.0.59: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
21: 10.0.0.59: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
22: 10.0.0.59: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
23: 10.0.0.59: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
24: 10.0.0.57: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
25: 10.0.0.57: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
26: 10.0.0.57: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
27: 10.0.0.57: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
28: 10.0.0.57: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
29: 10.0.0.57: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
30: 10.0.0.57: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
31: 10.0.0.57: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
32: 10.0.0.60: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
33: 10.0.0.60: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
34: 10.0.0.60: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
35: 10.0.0.60: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
36: 10.0.0.60: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
37: 10.0.0.60: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
38: 10.0.0.60: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
39: 10.0.0.60: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
40: 10.0.0.64: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
41: 10.0.0.64: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
42: 10.0.0.64: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
43: 10.0.0.64: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
44: 10.0.0.64: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
45: 10.0.0.64: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
46: 10.0.0.64: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
47: 10.0.0.64: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
48: 10.0.0.63: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
49: 10.0.0.63: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
50: 10.0.0.63: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
51: 10.0.0.63: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
52: 10.0.0.63: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
53: 10.0.0.63: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
54: 10.0.0.63: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
55: 10.0.0.63: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
56: 10.0.0.62: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
57: 10.0.0.62: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
58: 10.0.0.62: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(187).............................: MPI_Recv(buf=0012F93C, 
count=1, MPI_INTEGER, src=0, tag=4638, MPI_COMM_WORLD, status=00000001) 
failed
MPIDI_CH3i_Progress_wait(215).............: an error occurred while 
handling an event returned by MPIDU_Sock_Wait()
MPIDI_CH3I_Progress_handle_sock_event(420): 
MPIDU_Sock_wait(2606).....................: The specified network name is 
no longer available. (errno 64)
59: 10.0.0.62: 1: Fatal error in MPI_Recv: Other MPI error, error stack:
_________________________________________________________________________

The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential and/or privileged
material.  Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited.   If you
Received this in error, please contact the sender and delete the material 
from any computer.
_________________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20120418/a37b7c6c/attachment-0001.htm>