Here is a graph of the time for an allgather as a function of message size. We would like to scale up to 98K processors but as you can see the time for all gather dramatically increases at 49K processors. At 98K the time increases by around a factor of 4 for the few datapoints that I have tested (they are not in the graph).<br>
<br>Thanks,<br>Justin<br><br><br><br><div class="gmail_quote">On Mon, Nov 2, 2009 at 5:08 PM, Rajeev Thakur <span dir="ltr"><<a href="mailto:thakur@mcs.anl.gov">thakur@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2" face="Arial">You can see the algorithms used for different messages sizes
in MPICH2 here: <a href="https://svn.mcs.anl.gov/repos/mpi/mpich2/trunk/src/mpi/coll/allgatherv.c" target="_blank">https://svn.mcs.anl.gov/repos/mpi/mpich2/trunk/src/mpi/coll/allgatherv.c</a>,
but Cray could be using a modified version. What is the total message size being
gathered and how many processes are you running on?</font></span></div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2" face="Arial"></font></span> </div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2" face="Arial">Rajeev</font></span></div>
<div dir="ltr" align="left"><span></span> </div><br>
<blockquote style="border-left: 2px solid rgb(0, 0, 255); padding-left: 5px; margin-left: 5px; margin-right: 0px;" dir="ltr">
<div dir="ltr" align="left" lang="en-us">
<hr>
<font size="2" face="Tahoma"><b>From:</b> <a href="mailto:mpich-discuss-bounces@mcs.anl.gov" target="_blank">mpich-discuss-bounces@mcs.anl.gov</a>
[mailto:<a href="mailto:mpich-discuss-bounces@mcs.anl.gov" target="_blank">mpich-discuss-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Justin
Luitjens<br><b>Sent:</b> Monday, November 02, 2009 5:59 PM<br><b>To:</b>
<a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br><b>Subject:</b> [mpich-discuss] MPI_Allgatherv
complexity<br></font><br></div><div><div></div><div class="h5">
<div></div>Hi,<br><br>I'm wondering what is the parallel complexity of
MPI_Allgatherv. I would suspect in the worst case it would be something
like N log P but from the timings i'm collecting this doesn't seem to be the
case. I'm running tests on Kraken (<a href="http://www.nics.tennessee.edu/computing-resources/kraken" target="_blank">http://www.nics.tennessee.edu/computing-resources/kraken</a>
<div>) and it looks like it might be something like P
log P.<br><br>Thanks,</div></div></div></blockquote></div>
<br>_______________________________________________<br>
mpich-discuss mailing list<br>
<a href="mailto:mpich-discuss@mcs.anl.gov">mpich-discuss@mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
<br></blockquote></div><br>