<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:times new roman, new york, times, serif;font-size:12pt"><DIV><BR></DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">thanks darius,</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"> </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">When I did the comparison (or benchmarking), I have 2 identical source trees. Everything</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">were recompiled group up and compiled/linked accordinglyto the version of MPICH2 </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">to be used.</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"> </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">I have many tests, this is the only one showing this behavior, and is predictably repeatable.</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">most of my tests are showing comaptible performance and many do better</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">with 1.1.</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"> </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">The 'weirdest' thing is the ~1 minute span where there is no activity on the box at all, zipo </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">activity except 'top', with machine load at around 0.12. I don't know how to explain this</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">'behavior', and I am extremely curious if anyone can explain this.</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"> </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">I can't repeat this on AMD boxes as I don't have one that has only 32G of memory. I can't</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">repeat this on Niagara box as thread multiple won't build.</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"> </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">I will try to rebuild 1.1 without thread-multiple. Will keep you posted.</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"> </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">Meanwhile, if anyone has any speculations on this, please bring them up.</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"> </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">thanks</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">tan</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"> </DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=2 face=Tahoma>
<HR SIZE=1>
<B><SPAN style="FONT-WEIGHT: bold">From:</SPAN></B> Darius Buntinas <buntinas@mcs.anl.gov><BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> mpich-discuss@mcs.anl.gov<BR><B><SPAN style="FONT-WEIGHT: bold">Sent:</SPAN></B> Monday, July 13, 2009 8:30:19 AM<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> Re: [mpich-discuss] version 1.1 strange behavior : all processes become idle for extensive period<BR></FONT><BR>Tan,<BR><BR>Did you just re-link the applications, or did you recompile them?<BR>Version 1.1 is most likely not binary compatible with 1.0.6, so you<BR>really need to recompile the application.<BR><BR>Next, don't use the --enable-threads=multiple flag when configuring<BR>mpich2. By default, mpich2 supports all thread levels and will select<BR>the thread level at run time (depending on the parameters passed to<BR>MPI_Init_thread). By allowing the thread level to be selected<BR>automatically at run time, you'll avoid
the overhead of thread safety<BR>when it's not needed, allowing your non-threaded applications to run faster.<BR><BR>Let us know if either of these fixes the problem, especially if just<BR>removing the --enable-threads option fixes this.<BR><BR>Thanks,<BR>-d<BR><BR>On 07/10/2009 06:19 PM, chong tan wrote:<BR>> I am seeing this funny situation which I did not see on 1.0.6 and<BR>> 1.0.8. Some background:<BR>> <BR>> machine : INTEL 4Xcore 2<BR>> <BR>> running mpiexec -n 4<BR>> <BR>> machine has 32G of mem. <BR>> <BR>> when my application runs, almost all memory are used. However, there<BR>> is no swapping.<BR>> I have exclusive use of the machine, so contention is not an issue.<BR>> <BR>> issue #1 : processes take extra long to be initialized, compared to 1.0.6<BR>> issue #2 : during the run, at time all of them will become idle at the<BR>> same time, for
almost a<BR>> minute. We never observed this with 1.0.6<BR>> <BR>> <BR>> The codes are the same, only linked with different versions of MPICH2.<BR>> <BR>> MPICH2 was built with --enable-threads=multiple for 1.1. without for<BR>> 1.0.6 or 1.0.8<BR>> <BR>> MPI calls are all in the main application thread. I used only 4 MPI<BR>> functions :<BR>> init(), Send(), Recv() and Barrier(). <BR>> <BR>> <BR>> <BR>> any suggestion ?<BR>> <BR>> thanks<BR>> tan<BR>> <BR>> <BR>> <BR>> <BR>> <BR>> <BR></DIV></div><br>
</body></html>