If I add the N , the peak performance do not go up.<br><br>I use the netpipe(<a href="http://www.scl.ameslab.gov/netpipe/" target="_blank">http://www.scl.ameslab.gov/netpipe/</a>) and IMB-MPI to test the communication between two core in one cpu.<br>
<br>with no diskless platform.<br><br>I run mpirun -np 2 ./NPmpi and
mpirun -np 2 ./IMB-mpi in a node (two prcesses run on the two core),
the bandwidth is high and latency is small<br>
<br>but with diskless.<br><br>I run mpirun -np 2 ./NPmpi and mpirun -np
2 ./IMB-mpi in a node (two prcesses run on the two core), the bandwidth
is very low and latency is large.<br><br><br><div class="gmail_quote">2009/12/6 Gustavo Correa <span dir="ltr"><<a href="mailto:gus@ldeo.columbia.edu">gus@ldeo.columbia.edu</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi Liaoyin<br>
<br>
Your problem size N=1000 is too small.<br>
The order of P,Q also matters.<br>
See my previous message for suggestions.<br><font color="#888888">
<br>
Gus Correa</font><div><div></div><div class="h5"><br>
<br>
On Dec 5, 2009, at 8:06 AM, liaoyin wrote:<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
thank you .<br>
my mpi is mpich2 1.2.<br>
<br>
My cluster is a diskless platform that is load kernel and mount filesystem from the server.<br>
Single node has one cpu with four core.(the theoretical peak performance in single core is 3.2 Gflops )<br>
<br>
I am runing the HPL ( A Portable Implementation of the High-Performance Linpack Benchmark for Distributed-Memory Computers) to do linpcke test.<br>
<br>
I use the Atlas to as a blas lib that is a single thread version.<br>
<br>
case 1, I run the mpirun -np 1 ./xhlp the Gflops is 0.4 ( I found that one core to run with the top command HPL.dat is that n=1000,ps*qs=1*1)<br>
<br>
case 2, I run the mpirun -np 2 ./xhlp the Gflops is 0.04 (I found that one core to run with the top command HPL.dat is that n=1000,ps*qs=1*2)<br>
<br>
I use the netpipe(<a href="http://www.scl.ameslab.gov/netpipe/" target="_blank">http://www.scl.ameslab.gov/netpipe/</a>) and IMB-MPI to test the communication between two core in one cpu.<br>
<br>
<br>
with no diskless platform.<br>
<br>
I run mpirun -np 2 ./NPmpi and mpirun -np 2 ./IMB-mpi in a node (two prcesses run on the two core), the bandwidth is high and latency is small<br>
<br>
but with diskless.<br>
<br>
I run mpirun -np 2 ./NPmpi and mpirun -np 2 ./IMB-mpi in a node (two prcesses run on the two core), the bandwidth is very low and latency is large.<br>
<br>
why?<br>
<br>
<br>
2009/12/5 Gus Correa <<a href="mailto:gus@ldeo.columbia.edu" target="_blank">gus@ldeo.columbia.edu</a>><br>
Hi Liaoyin<br>
<br>
Besides Dave Goodell's questions,<br>
I wonder if this may be more of an HPL problem<br>
than of an MPICH2 problem.<br>
<br>
Here are some questions/suggestions about HPL:<br>
<br>
***<br>
<br>
1) If you just want to check if MPICH2 is working,<br>
then don't use HPL. Use something simpler.<br>
The cpi.c and hellow.c programs in the MPICH2 "examples" directory<br>
will tell you if MPICH2 is working properly, and are way much<br>
simpler than HPL to setup and run.<br>
<br>
***<br>
<br>
2) Note, there may be a typo on the mpirun command line<br>
on your message, it should be "xhpl" not "xhlp" as you wrote,<br>
unless you changed the executable name.<br>
<br>
***<br>
<br>
3) Are you sure you are running HPL on 2 cores?<br>
<br>
If you use Linux, you can submit your HPL job,<br>
then use "top" (and type "1") to see how many<br>
cores are actually running xhpl.<br>
<br>
Do you really see 2 cores in action on "top"?<br>
<br>
***<br>
<br>
4) What are the contents of your HPL.dat parameter file,<br>
when you try to run on 2 cores?<br>
<br>
Is is the same that you use for the one core run or is it different?<br>
<br>
This may not be news to you, but here it goes just in case:<br>
<br>
To run HPL on a given number of cores on your machine,<br>
the product of the values or Ps and Qs in HPL.dat has to be<br>
equal to the N value on your "mpirun -np N"<br>
For example, if you want to use 2 cores (mpirun -np 2),<br>
you could use this on your HPL.dat file:<br>
<br>
1 2 Ps<br>
2 1 Qs<br>
<br>
Or to use four cores:<br>
<br>
1 2 4 Ps<br>
4 2 1 Qs<br>
<br>
(Tip: You *must* use a *single blank space* field separator in HPL.dat.)<br>
<br>
***<br>
<br>
5) Very small problem sizes don't scale well with<br>
the number of processors.<br>
<br>
To see speedups when more cores are added,<br>
you need to choose a relatively large value for Ns in HPL.dat.<br>
However, Ns is constrained by how much memory (RAM) your computer has.<br>
A reasonable upper bound to Ns is sqrt(0.8*memory_in_bytes/8).<br>
<br>
See item 5) below.<br>
<br>
***<br>
<br>
6) Finally, if you haven't read it, the HPL TUNING file is a<br>
*must read* for anybody who wants to run the HPL benchmark:<br>
<br>
<a href="http://www.netlib.org/benchmark/hpl/tuning.html" target="_blank">http://www.netlib.org/benchmark/hpl/tuning.html</a><br>
<br>
***<br>
<br>
Good luck!<br>
<br>
I hope this helps,<br>
Gus Correa<br>
---------------------------------------------------------------------<br>
Gustavo Correa<br>
Lamont-Doherty Earth Observatory - Columbia University<br>
Palisades, NY, 10964-8000 - USA<br>
---------------------------------------------------------------------<br>
<br>
<br>
<br>
<br>
Dave Goodell wrote:<br>
What MPI implementation are you using? If you are using MPICH2, please make sure that you are using the latest version (1.2.1).<br>
<br>
-Dave<br>
<br>
On Dec 4, 2009, at 3:02 AM, liaoyin wrote:<br>
<br>
I am runing the HPL ( A Portable Implementation of the High-Performance Linpack Benchmark for Distributed-Memory Computers) to do linpcke test.<br>
<br>
I use the Atlas to as blas lib.<br>
<br>
My machine has one cpu with four core.<br>
<br>
I run the mpirun -np 1 ./xhlp the Gflops is 0.4 (one core to run)<br>
<br>
but I run the mpirun -np 2 ./xhlp the Gflops is 0.04 (two core to run)<br>
<br>
why is the two core is slower.<br>
<br>
_______________________________________________<br>
mpich-discuss mailing list<br>
<a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
<br>
_______________________________________________<br>
mpich-discuss mailing list<br>
<a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
<br>
_______________________________________________<br>
mpich-discuss mailing list<br>
<a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
<br>
_______________________________________________<br>
mpich-discuss mailing list<br>
<a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
</blockquote>
<br>
_______________________________________________<br>
mpich-discuss mailing list<br>
<a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
</div></div></blockquote></div><br>