[MPICH] slowdown in replica exchange dynamics calculation

[baxa at uchicago.edu]

Hi Rajeev,

Yes.  Both the single node computation and the 10 node
computations are do-loops calling the same subroutine.

As for input and output, both programs initially read in a
coordinate file and various parameter files.  Every certain
number of timesteps (e.g. 500 steps), the coordinates are
written to an archive file.  The simulation time wasn't
affected by decreasing the number of time frames saved to the
archive file.

Regards,

Michael

---- Original message ----
>Date: Tue, 19 Feb 2008 15:06:56 -0600
>From: "Rajeev Thakur" <thakur at mcs.anl.gov>  
>Subject: RE: [MPICH] slowdown in replica exchange dynamics
calculation  
>To: <baxa at uchicago.edu>, <mpich-discuss at mcs.anl.gov>
>
>> I am running replica exchange combined with langevin dynamics
>> using a single fortran program.  For example, on a set of 10
>> nodes, a langevin dynamics trajectory is calculated on each
>> node for 10,000 steps.  After all the nodes have completed
>> their number of steps, the nodes communicate with one another
>> via MPI and determine whether to swap temperatures.  Once
>> temperatures have been swapped, the pattern repeats.
>> 
>> It turns out though, that the dynamics calculation portion of
>> the above program takes longer than it would if I were to just
>> run a langevin dynamics calculation on a single node for 10000
>> steps.  For 10 nodes, it is approximately 5-10X slower.  While
>> no MPI commands are being executed during this time, the
>> duration of the dynamics portion appears to depend on the
>> number of nodes.  For this reason, it was suspected that this
>> slowdown may be due to a suboptimal distribution of tasks,
>> unnecessary barriers, or IO waits.
>
>Is the computation in the single-node case identical to the
computation on
>10 nodes. If not, you can't really compare.
>
>If there was no MPI in this phase, there should be no
barriers either, and
>even suboptimal distribution of tasks should not cause a
slowdown if it is
>all local computation. Is there any file I/O?
>
>Rajeev
>
>
>