[mpich-discuss] thread MPI calls

[chong tan]

D,
thanks for the info.  I will check with the engineer implementinig this thread comm
feature.    I will keep you posted.

A side note.  We also try a different strategy by calling IRecv in the master process
right after it has recieved and applied all data from 'slave process', like this, in non-threaded
mode :

     master :                                       slave :
TOP :                                             TOP : 
     MPI_Wait_all                             MPI_Send (to master)
     for all slave                                 MPI_Recv( from master )
         MPI_Send
     appy-data                                    apply-data                       
     for all slave
        MPI_Irecv()
     do works                                     do works
     repeat TOP                                   repeat TOP


compared to brute forced blocked Recv,  this Irecv and wait-all can cause performance
loss of 5%.   


tan




________________________________
From: Darius Buntinas <buntinas at mcs.anl.gov>
To: mpich-discuss at mcs.anl.gov
Sent: Friday, July 24, 2009 8:37:39 AM
Subject: Re: [mpich-discuss] thread MPI calls


Hi Tan,

Did your program call MPI_Init_thread, or MPI_Init?  If you called
MPI_Init_thread, what thread level did you request?

Based on what thread level you selected, MPICH will enable internal
thread safety.  Without looking at the code, I believe that (at least
for the unix implementation) the internal thread safety mechanism is
only enabled with the thread multiple level, so the performance of
using, say, thread funneled should be the same as using thread single
(or just callin MPI_Init rather than MPI_Init_thread).

-d

On 07/23/2009 05:40 PM, chong tan wrote:
> box X86 box running Linux.  MPICH2 1.1 configured with
>  
>  --disable-f77 --disable-f90 --with-device=ch3:nemesis  --enable-fast
>  
> machine : 16 cores, load of mem.  running  with mpiexec -n 4
> The master process (MPICH rank 0) has 2 thread, main and recv, the
> locks are constructed using pthread.  The master's
> recv function wait for the recv thread to be done with all recieving,
> then process the data, and reenable the recv thread,
> like this:
>  
>              main                                                recv thread
>  
>              recvFunc() {                                  wait for 'run' 
>              wait for read-done                          for n processes
>              apply data                                            
> call MPI_IRecv
>              signal run                                        MPI_waitall
>              }                                                    
> signal read-done
>        
> recvFunc is repeatedly called during the life of my applications.  (In a
> few tests I have, it is called 100+ billion times)
>  
> compared to the same application with no thread and using IRecv or Recv,
> the threaded version can run 20+%
> slower.  From the process monitoring, it looks like the MPI Irecv calls
> maybe thread. This performance degration
> is inline with 2.0.6 configured using --enable-threads=funneled.  The
> application ran for many hours, so 20% is
> significant.
>  
> Questions :
> -  is MPICH2 1.1 self-sensing ?  that is, does it know it can decide if
> it has to do thread-multiple ?
> -  IRecv and MPI wait all, is that also threaded ?
> -  anyone experimented this before ?
>  
> thanks
> tan
>  
> ------------------------------------------------------------------------
> *From:* Rajeev Thakur <thakur at mcs.anl.gov>
> *To:* mpich-discuss at mcs.anl.gov
> *Sent:* Friday, June 26, 2009 12:12:18 PM
> *Subject:* Re: [mpich-discuss] does 1.1 support real threading of MPI call ?
> 
> Yes, it supports MPI_THREAD_MULTIPLE as defined by the MPI standard.
>  
> Rajeev
> 
>    ------------------------------------------------------------------------
>    *From:* mpich-discuss-bounces at mcs.anl.gov
>    [mailto:mpich-discuss-bounces at mcs.anl.gov] *On Behalf Of *chong tan
>    *Sent:* Friday, June 26, 2009 2:04 PM
>    *To:* mpich-discuss at mcs.anl.gov
>    *Subject:* [mpich-discuss] does 1.1 support real threading of MPI call ?
> 
>    Does anyone know if the 1.1 release supports real threaded MPI
>    calls.  That is,
>    a process, say 1, may contain n threads, and each calling
>    MPI_Send/Recv to
>    other processes, but still think they are all process 1 ?
>      
>    I am not looking at the funneled solution, that does not help me.
>      
>    thanks
>    tan
> 
>      
> 
> 
> 



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20090724/b3e0f0e5/attachment.htm>