<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Hi All, </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">I implemented an algorithm with MPI
and tried to measure its execution time. </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">The Execution time varies too much. I measured each parts of the algorithm separately and found the problem. In my code there is a send command
after a receive command in a loop. This receive causes sometimes 2 seconds latency.<span style> It does not appear every iteration but irregularly. </span>Can somebody help me about this? </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0.0001pt"><span style lang="EN-US">To show this problem I wrote the
following code end executed it several times. Sometimes I am getting 0.2 seconds
latency. Can somebody please help me? <br></span></p><p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">I cant use MPI_Sendrecv because there are some computations between send and receive in my real code. <br>
</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">I am executing it on a linux cluster.
Here is the example code and its output. </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">#include <stdio.h></span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">#include <mpi.h></span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">int main (argc, argv)</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>int
argc;</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>char
*argv[];</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">{</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>int
rank, size;</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Init
(&argc, &argv);<span style> </span>/*
starts MPI */</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Comm_rank
(MPI_COMM_WORLD, &rank);<span style> </span>/*
get current process id */</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Comm_size
(MPI_COMM_WORLD, &size);<span style> </span>/*
get number of processes */</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Status
status; </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span></span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>double
timer; </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>int
buffer[1000]; </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>int
iter; </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>for
(iter =0; iter<30; iter++) </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>{</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>if
(rank==0){</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Send(buffer,
1000, MPI_INT, 1, 123, MPI_COMM_WORLD );</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Barrier(MPI_COMM_WORLD);
//Barrier </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>timer
= MPI_Wtime(); //Execution time is <span style> </span>started</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Recv(buffer,
1000, MPI_INT, 1, 234, MPI_COMM_WORLD, &status);</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>timer
= MPI_Wtime()-timer;<span style> </span>//Execution time is
<span style> </span>stopped </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>printf("Execution
time is %f iteration is %d \n", timer, iter); </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span><span style> </span>}</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span></span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>if
(rank==1) {</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Recv(buffer,
1000, MPI_INT, 0, 123, MPI_COMM_WORLD, &status);</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Barrier(MPI_COMM_WORLD);
</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Send(buffer,
1000, MPI_INT, 0, 234, MPI_COMM_WORLD ); </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>}</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>}<span style> </span></span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>MPI_Finalize();</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"><span style> </span>return
0;</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">}</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">root@rck00:~> mpirun -np 2
/shared/sendrecv/./a.out </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000412 iteration is 0 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000175 iteration is 1 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><b style><span style="color:red" lang="EN-US">Execution time is <span style> </span>0.250426 iteration is 2 </span></b></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.038946 iteration is 3 </span></p>
<p class="MsoNormal" style="margin-bottom:0.0001pt;color:rgb(255,0,0)"><b><span style lang="EN-US">Execution time is <span style> </span>0.258253 iteration is 4 </span></b></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.038870 iteration is 5 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.008447 iteration is 6 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.008509 iteration is 7 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000234 iteration is 8 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000245 iteration is 9 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000141 iteration is 10 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.001203 iteration is 11 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.002454 iteration is 12 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000123 iteration is 13 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.033437 iteration is 14 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000118 iteration is 15 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000128 iteration is 16 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000145 iteration is 17 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.001204 iteration is 18 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.006472 iteration is 19 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000119 iteration is 20 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000127 iteration is 21 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.033959 iteration is 22 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000118 iteration is 23 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000131 iteration is 24 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.043315 iteration is 25 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000111 iteration is 26 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000130 iteration is 27 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><b style><span style="color:red" lang="EN-US">Execution time is <span style> </span>0.253596 iteration is 28 </span></b></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.079988 iteration is 29</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">root@rck00:~> mpirun -np 2
/shared/sendrecv/./a.out </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000423 iteration is 0 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.005989 iteration is 1 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000100 iteration is 2 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000093 iteration is 3 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000092 iteration is 4 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><b style><span style="color:red" lang="EN-US">Execution time is <span style> </span>0.258258 iteration is 5 </span></b></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.040296 iteration is 6 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.048177 iteration is 7 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000223 iteration is 8 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000223 iteration is 9 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000145 iteration is 10 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.001117 iteration is 11 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.001528 iteration is 12 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000136 iteration is 13 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000141 iteration is 14 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000148 iteration is 15 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000150 iteration is 16 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000159 iteration is 17 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000153 iteration is 18 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000142 iteration is 19 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.005847 iteration is 20 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000146 iteration is 21 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000111 iteration is 22 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><b style><span style="color:red" lang="EN-US">Execution time is <span style> </span>0.253392 iteration is 23 </span></b></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><b style><span style="color:red" lang="EN-US">Execution time is <span style> </span>0.328551 iteration is 24 </span></b></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.038838 iteration is 25 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000207 iteration is 26 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000221 iteration is 27 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.033603 iteration is 28 </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Execution time is <span style> </span>0.000225 iteration is 29</span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Best Regards, </span></p>
<p class="MsoNormal" style="margin-bottom:0cm;margin-bottom:.0001pt"><span style lang="EN-US">Ümit</span></p>