<div class="gmail_quote">On Fri, May 11, 2012 at 4:03 PM, Dave Goodell <span dir="ltr"><<a href="mailto:goodell@mcs.anl.gov" target="_blank">goodell@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On May 11, 2012, at 1:49 PM CDT, Dave Goodell wrote:<br>
<br>
> On May 11, 2012, at 1:17 PM CDT, Jed Brown wrote:<br>
><br>
>> Can you make this fast, like ~100 clocks for the common case?<br>
><br>
> Can you send us a small benchmark that you would like us to optimize? Presumably you already have one since you've measured the overhead.<br></div></blockquote><div><br></div><div>I benchmarked an example using the PETSc ThreadComm. For a micro-benchmark that doesn't use pthreads, I would just call MPI_Attr_get() in a loop. I can prepare that if you like.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
><br>
> If the current performance is really ~1000 instructions I suspect we can easily knock it down by a large factor. I'm not sure whether 100 is realistic or not until I play around in there some more.<br>
<br>
</div>Also, please send me the configure options that you are using for your MPICH2 under petsc. The absolute performance is going to vary substantially depending on whether error checking is enabled and your choice of a few other options.<br>
</blockquote><div><br></div><div>I tested these two configurations, what do you suggest I use if I want that function to be fast?</div><div><br></div><div><div>MPICH2 Version: 1.4</div><div>MPICH2 Release date: Thu Jun 16 16:41:08 CDT 2011</div>
<div>MPICH2 Device: ch3:nemesis</div><div>MPICH2 configure: --prefix=/opt/mpich2 --enable-shared --enable-sharedlibs=gcc --enable-error-checking=runtime --enable-error-messages=all --enable-timer-type=clock_gettime --with-python=python2</div>
<div>MPICH2 CC: gcc -march=native -O1 -pipe -O2</div><div>MPICH2 CXX: c++ -march=native -O1 -pipe -O2</div><div>MPICH2 F77: gfortran -O2</div><div>MPICH2 FC: gfortran -O2</div></div><div><br></div>
<div><div>MPICH2 Version: <span class="Apple-tab-span" style="white-space:pre"> </span>1.5b1</div><div>MPICH2 Release date:<span class="Apple-tab-span" style="white-space:pre"> </span>unreleased development copy</div><div>
MPICH2 Device: <span class="Apple-tab-span" style="white-space:pre"> </span>ch3:nemesis</div><div>MPICH2 configure: <span class="Apple-tab-span" style="white-space:pre"> </span>--prefix=/homes/jedbrown/usr/mpich-intel --enable-shared --enable-error-checking=runtime --enable-error-messages=all --enable-timer-type=clock_gettime CC=icc CXX=icpc --enable-fc=0 --enable-f77=0 FC= F77=</div>
<div>MPICH2 CC: <span class="Apple-tab-span" style="white-space:pre"> </span>icc -O2</div><div>MPICH2 CXX: <span class="Apple-tab-span" style="white-space:pre"> </span>icpc -O2</div><div>MPICH2 F77: <span class="Apple-tab-span" style="white-space:pre"> </span>gfortran </div>
<div>MPICH2 FC: <span class="Apple-tab-span" style="white-space:pre"> </span>gfortran</div></div></div>