Hello Prof. Gustavo,<br><br>Thank you a lot for your patience on answering my message. I confess you that during the process I thought about give up, but then I watched a documentary about an university here in Brazil that is going to construct the biggest cluster system in South America, connecting about 5 universities bases. So I thought, 'if they are connecting so many computers that is impossible that I can't make just 9 work'. And then I I went on and now it is finished and I'm in vacations (reason by which I didn't answer you before, I want to apologize for this). Your words gave me new strenght to go on, reading you learn a lot but like in the universities, sometimes you need someone saying you where to go in the next step, and your orientations have cleared a lot of this long way. Really thank you.<br>
Now my professor is using the cluster, after some weeks trying to make it work. soon I'll have some results for discuss with you and all the list.<br>I hope your work is going on ok and I wish all the best!<br><br>Ari<br>
<br><div class="gmail_quote">2008/7/3 Gus Correa <<a href="mailto:gus@ldeo.columbia.edu" target="_blank">gus@ldeo.columbia.edu</a>>:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hello Ariovaldo and list<br>
<br>
Ariovaldo, don't shy off.<br>
Your questions are reasonable.<br>
Beginners should be encouraged, not scoffed,<br>
regardless of whether they are students or Nobel laureates.<br>
<br>
I've always got a lot of knowledgeable and generous advice<br>
from the MPICH developers and other list subscribers,<br>
even when I asked naive questions.<br>
<br>
Having said that, here are a few suggestions.<br>
Take them with a grain of salt, because<br>
I am an MPICH user, not an expert.<br>
<br>
1) How are you launching your mpd daemon?<br>
<br>
The processes are distributed across the machines in different ways,<br>
depending on whether you specify or not the number of cpus on each machine,<br>
when you start the mpd daemon.<br>
For details, please read section 5.1.7 in the MPICH Installer's Guide (available in PDF format<br>
on the MPICH site).<br>
<br>
In your case, to group processes tightly on each machine, I would say<br>
you could use -ncpus=8, since you have four physical slots per machine,<br>
and each one is dual-core processor, right?<br>
I am guessing a bit here, though.<br>
I didn't understand your description of your machine configuration,<br>
but since you know it well, you can figure out the right number, which may not be 8.<br>
Anyway, if your OS is Linux,<br>
you can get the exact number cores available on each machine by looking at /proc/cpuinfo<br>
("more /proc/cpuinfo").<br>
<br>
However, beware that speedup in multi-core CPUs is not really linear, due to memory contention<br>
and bus "congestion".<br>
There are people who simply leave one or more cores out of the game because of this.<br>
<br>
2) For tighter control of where the processes run you can use the "-configfile" feature of mpiexec.<br>
See the manual page of mpiexec (man mpiexec) for details.<br>
You may need to point to the exact man page path if you have many versions of mpiexec in your computer.<br>
<br>
3) The cpi.c program may not be a great speedup test.<br>
I guess the program was meant just to test if MPI is working, not as a performance benchmark.<br>
<br>
If you look at the code you'll see that the number of iterations<br>
in the program's single loop is10000, and the computations are very modest.<br>
With such a small number of iterations, if you divide the task by, say, 10 processors,<br>
each one does only 1000 simple computations, which is very fas compared to<br>
other tasks in the beginning and end of the code.<br>
<br>
The execution time is likely to be dominated by non-parallel tasks (initialization, etc),<br>
which don't scale with the number of processors.<br>
This is an aspect of "Amdahl's Law":<br>
<a href="http://en.wikipedia.org/wiki/Amdahl%27s_law" target="_blank">http://en.wikipedia.org/wiki/Amdahl's_law</a><br>
<br>
Actually, as you increase the number of processors,<br>
the communication overhead during initialization and finalization<br>
may worsen the execution time when the computations are so modest as they are in cpi.c.<br>
<br>
I guess the cpi.c code also has an unnecessary call to mpi_broadcast, which may be a leftover<br>
from a previous version, and which may add to the overhead.<br>
But this only the MPICH developers can tell.<br>
<br>
There are synthetic benchmarks out there.<br>
However, since you want to run the molecular dynamics code,<br>
maybe you could setup a simple speedup experiment with it.<br>
This may take less effort, and will help you use and understand the code you are interested in.<br>
<br>
I hope this helps.<br>
<br>
Gus Correa<br>
<br>
-- <br>
---------------------------------------------------------------------<br>
Gustavo J. Ponce Correa, PhD - Email: <a href="mailto:gus@ldeo.columbia.edu" target="_blank">gus@ldeo.columbia.edu</a><br>
Lamont-Doherty Earth Observatory - Columbia University<br>
P.O. Box 1000 [61 Route 9W] - Palisades, NY, 10964-8000 - USA<br>
Oceanography Bldg., Rm. 103-D, ph. (845) 365-8911, fax (845) 365-8736<br>
---------------------------------------------------------------------<div><br>
<br>
<br>
Ariovaldo de Souza Junior wrote:<br>
<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div>
Hi Chong,<br>
<br>
Yes, I'm a student. In the truth who will use mpi is my teacher, who will run NAMD, for molecular dynamics. I was challenged about set it up this cluster and I had no knowledge about Linux before start it. Once it running now, I think that maybe I could go a bit more far. Theoretically my work is over, the machines are working well. But yet I would like to know how to extract the maximum performance of these computers, that have already proof that they are good ones, once we had utilized them already for run Gaussian 03 molecular calculations.<br>
<br>
I wanted to know a bit more because I love computers and now I was introduced to this universe of clusters, I wanted to know a bit more. Just it. I'm reading already the tips you gave me, even it is a bit complicated to extract the information I want from there. thanks a lot for your attention.<br>
<br>
Ari.<br>
<br></div>
2008/7/2 chong tan <<a href="mailto:chong_guan_tan@yahoo.com" target="_blank">chong_guan_tan@yahoo.com</a> <mailto:<a href="mailto:chong_guan_tan@yahoo.com" target="_blank">chong_guan_tan@yahoo.com</a>>>:<div>
<br>
<br>
<br>
Ari,<br>
<br>
Are you a student ? Anyway, I like to point you to the answer of<br>
your problem:<br>
<br>
mpiexec -help<br>
<br>
<br>
or look at your mpich2 packge, under www/www1, there is a<br>
mpiexec.html<br>
<br>
<br>
it is easier to give your the answer, but getting you to look for<br>
the answer is better.<br>
<br>
<br>
<br>
stan<br>
<br>
<br>
--- On *Wed, 7/2/08, Ariovaldo de Souza Junior<br></div>
/<<a href="mailto:ariovaldojunior@gmail.com" target="_blank">ariovaldojunior@gmail.com</a> <mailto:<a href="mailto:ariovaldojunior@gmail.com" target="_blank">ariovaldojunior@gmail.com</a>>>/*<div>
<br>
wrote:<br>
<br>
From: Ariovaldo de Souza Junior <<a href="mailto:ariovaldojunior@gmail.com" target="_blank">ariovaldojunior@gmail.com</a><br></div>
<mailto:<a href="mailto:ariovaldojunior@gmail.com" target="_blank">ariovaldojunior@gmail.com</a>>><div><br>
Subject: [mpich-discuss] core 2 quad and other multiple core<br>
processors<br></div>
To: <a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a> <mailto:<a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a>><div><div></div>
<div><br>
Date: Wednesday, July 2, 2008, 1:15 PM<br>
<br>
<br>
Hello everybody!<br>
<br>
I'm really a newbie on clustering, so I have some, let's say,<br>
stupid questions. When I'm starting a job like this "mpiexec<br>
-l -n 6 ./cpi" in my small cluster of (until now) 6 core 2<br>
quad machines, I'm sending 1 process to each node, right?<br>
Assuming that I'm correct, each process will utilize only 1<br>
core of each node? and how to make 1 process run utilizing the<br>
whole processing capacity of the processor, the 4 cores? is<br>
there a way to do this? or I'll always utilize just one<br>
processor for each process? if I change this submission to<br>
"mpiexec -l -n 24 ./cpi" then the same process will run 24<br>
times, 4 times per node (maybe simultaneously) and one process<br>
per core, right?<br>
<br>
I'm asking all this because I think it is a bit strange to see<br>
the processing time increasing each time I put one more<br>
process to run, once in my mind it should be the contrary.<br>
I'll give some examples:<br>
<br>
mpiexec -n 1 ./cpi<br>
wall clock time = 0.000579<br>
<br>
mpiexec -n 2 ./cpi<br>
wall clock time = 0.002442<br>
<br>
mpiexec -n 3 ./cpi<br>
wall clock time = 0.004568<br>
<br>
mpiexec -n 4 ./cpi<br>
wall clock time = 0.005150<br>
<br>
mpiexec -n 5 ./cpi<br>
wall clock time = 0.008923<br>
<br>
mpiexec -n 6 ./cpi<br>
wall clock time = 0.009309<br>
<br>
mpiexec -n 12 ./cpi<br>
wall clock time = 0.019445<br>
<br>
mpiexec -n 18 ./cpi<br>
wall clock time = 0.032204<br>
<br>
mpiexec -n 24 ./cpi<br>
wall clock time = 0.045413<br>
<br>
mpiexec -n 48 ./cpi<br>
wall clock time = 0.089815<br>
<br>
mpiexec -n 96 ./cpi<br>
wall clock time = 0.218894<br>
<br>
mpiexec -n 192 ./cpi<br>
wall clock time = 0.492870<br>
<br>
So, as you all can see is that as more processes I add, more<br>
time it takes, what makes me think that mpi is performing this<br>
test 192 times in the end and due to this the time increased.<br>
Is that correct that mpi performed the same test 192? Or did<br>
it divide the process into 192 pieces, calculated and then<br>
gathered the results and mounted the output again? I really<br>
would like to understand this relationship processor # x<br>
process # x .<br>
<br>
I have the feeling that my questions are a bit "poor" and<br>
really from a newbie, but the answer will help me on utilizing<br>
other programs that will need mpi to run.<br>
<br>
Thanks to all!<br>
<br>
Ari - UFAM - Brazil<br>
<br>
<br>
<br>
</div></div></blockquote>
<br>
</blockquote></div><br>