[mpich-discuss] Message Protocols

[Marcelo Tomim]

Hello Rajeev,

I am using the techniques suggested in
Kielmann, T., Bal, H. E., & Verstoep, K. (2000). Fast measurement of 
LogP parameters for message passing platforms. of Lecture Notes in 
Computer Science, 1800, 1176-1183.
 
Below is the code excerpt  for measuring the sending overhead:

              for (k = 0; k < M; k++){
                if (rank == 0){
                    // This is the measuring process
                    start = MPI_Wtime();
                    MPI_Send(buffer, size, MPI_BYTE, 1, k, world);
                    stop_os = MPI_Wtime();
                    time_os += stop_os - start; // Take the time for a 
single send
                             time_os /= M; //! Take the time for a 
single send
                }
                else{
                    MPI_Recv(buffer, size, MPI_BYTE, 0, k, world, &status);
                }
       }

Now the receiving overhead timing:

             for (k = 0; k < M; k++){
                if (rank == 0){
                    // This is the measuring process
                    MPI_Send(buffer, 0, MPI_BYTE, 1, k, world);
                    wait_delta(delta_t);
                    start_or = MPI_Wtime();                   
                    MPI_Recv(buffer, size, MPI_BYTE, 1, k, world, &status);
                    stop_or = MPI_Wtime();
                    time_or += stop_or - start_or;
                }
                else{
                    MPI_Recv(buffer, 0, MPI_BYTE, 0, k, world, &status);
                    MPI_Send(buffer, size, MPI_BYTE, 0, k, world);
                }
       }
       
While the sending overhead timing is quite intuitive, the receiving 
overhead demands a little more explanation. The main difference here is 
the presence of the controlled waiting function wait_delta. In this 
case, making delta_t big enough so it is greater than the round trip 
time for messages of size m will guarantee that there won't be any idle 
time in the MPI_Recv and the time measured is the true receiving 
overhead. The round trip time is measured before this routine is called.

I hope it makes sense.

Regards,
Marcelo Tomim.

Rajeev Thakur wrote:
> How are you measuring the overhead for receiving versus sending?
>
> Rajeev 
>
>   
>> -----Original Message-----
>> From: mpich-discuss-bounces at mcs.anl.gov 
>> [mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of Marcelo Tomim
>> Sent: Wednesday, April 29, 2009 7:31 PM
>> To: mpich-discuss at mcs.anl.gov
>> Subject: [mpich-discuss] Message Protocols
>>
>> Hello MPI users,
>>
>> I have the results of some point-to-point benchmarks on 
>> MPICH2 library 
>> and I have been having a hard time to interpret them.
>> The test is based on paired Send and Recv routines, similar 
>> to ping-pong 
>> tests. One specific detail I have been wondering about is the 
>> fact that 
>> for large messages (larger than 128kB, i.e., rendezvous 
>> protocol), the 
>> overhead for sending is much smaller that the overhead for 
>> receiving. I 
>> was expecting them to be close to one another, based on the fact that 
>> the receiver has to acknowledge to the sender that the 
>> message has been 
>> successfully received. There must be a flaw in this 
>> reasoning, but I don 
>> not where it is.
>> Could any one help me with that or indicate me a document which 
>> explains, in detail, all the message protocols (short, eager and 
>> rendezvous) employed in the MPICH2 implementation?
>>
>> Thank you all very much for your attention.
>>
>> Regards,
>> Marcelo Tomim.
>>
>>     
>
>