<table cellspacing="0" cellpadding="0" border="0" ><tr><td valign="top" style="font: inherit;"><DIV><BR>the test were for checking out the resource capacity of you machine, not for testing MPICH2 nor your parallel algorithm. If the machine can handle 4 of you jobs effectively,</DIV>
<DIV>then you can point your finger to MPICH2.</DIV>
<DIV> </DIV>
<DIV>tan</DIV>
<DIV><BR>--- On <B>Tue, 1/13/09, Gustavo Miranda Teixeira <I><magusbr@gmail.com></I></B> wrote:<BR></DIV>
<BLOCKQUOTE style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: rgb(16,16,255) 2px solid">From: Gustavo Miranda Teixeira <magusbr@gmail.com><BR>Subject: Re: [mpich-discuss] MPICH v1.2.7p1 and SMP clusters<BR>To: chong_guan_tan@yahoo.com, mpich-discuss@mcs.anl.gov<BR>Date: Tuesday, January 13, 2009, 4:09 PM<BR><BR>
<DIV id=yiv330830250>I've had the issue both in a SuSe Linux 9 and a Ubuntu Sever 8.04 both using Kernel 2.6. But I don't get what you want with your experiment, since you ask me to run non-MPI applications (I don't know which ones) when the problem is intrinsically linked with MPI. Also the problem is occurring in MPICH v1.2.7p1.<BR><BR>
<DIV class=gmail_quote>On Tue, Jan 13, 2009 at 7:50 PM, chong tan <SPAN dir=ltr><<A href="mailto:chong_guan_tan@yahoo.com" target=_blank rel=nofollow>chong_guan_tan@yahoo.com</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<TABLE cellSpacing=0 cellPadding=0 border=0>
<TBODY>
<TR>
<TD vAlign=top>
<DIV><BR>Since I don't know how parallalism is explored in your application, I have to assume that inter-process communication is not a crtitical factor (based on your run result).</DIV>
<DIV> </DIV>
<DIV>Back to HW. 1 CPU with 2 cores !== 2CPU with 1 core in terms of performance. </DIV>
<DIV>With 2 boxes, your aggreated memory bandwidth is larger. Cache bandwidth can also be better on 2 boxes. Therefore you maybe get better throughput frrom 2 boxes of 2X2 by running 1 job per CPU. The older version of Linux (I assume you have this) may not be able to distribute job evenly per physical CPU, the later ones do a reasonably good job on this, hhowever.</DIV>
<DIV> </DIV>
<DIV>As a way to convince yourselves, would you mind try the following experiment :</DIV>
<DIV>1. use taskset to run 2 copy of your non-MPI application, first on the same CPU</DIV>
<DIV>2. repeat the last step, but on different physical CPUs</DIV>
<DIV> </DIV>
<DIV>if you want to see how the total available resource is affecting perforamnce, do :</DIV>
<DIV>3. run 4 copies of your non MPI applicatino on one box.</DIV>
<DIV> </DIV>
<DIV>hopefully, the perforamnce you derived can tell you something. Otherwise, MPICH2 has have some serious bug.</DIV>
<DIV> </DIV>
<DIV>tan</DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV>--- On <B>Tue, 1/13/09, Marcus Vinicius Brandão Soares <I><<A href="mailto:mvbsoares@gmail.com" target=_blank rel=nofollow>mvbsoares@gmail.com</A>></I></B> wrote:<BR></DIV>
<BLOCKQUOTE style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: rgb(16,16,255) 2px solid">From: Marcus Vinicius Brandão Soares <<A href="mailto:mvbsoares@gmail.com" target=_blank rel=nofollow>mvbsoares@gmail.com</A>>
<DIV class=Ih2E3d><BR>Subject: Re: [mpich-discuss] MPICH v1.2.7p1 and SMP clusters<BR>To: <A href="mailto:mpich-discuss@mcs.anl.gov" target=_blank rel=nofollow>mpich-discuss@mcs.anl.gov</A><BR></DIV>Date: Tuesday, January 13, 2009, 2:04 PM
<DIV>
<DIV></DIV>
<DIV class=Wj3C7c><BR><BR>
<DIV>Hello Gustavo,<BR><BR>My point is: depending on the structure of the connections (the edges of the graph) that are used interconnecting the processors and the type of processsing (cpu intensive or communication intensive) and, even, the level of paralelism (much sequential to divide-and-conquer) it may be easy to explain the differences you made reference to. <BR><BR>Do you agree ?<BR><BR>
<DIV class=gmail_quote>Best regards,<BR><BR><BR><BR>2009/1/13 Gustavo Miranda Teixeira <SPAN dir=ltr><<A href="mailto:magusbr@gmail.com" target=_blank rel=nofollow>magusbr@gmail.com</A>></SPAN><BR>
<BLOCKQUOTE class=gmail_quote style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">Hello Marcus,<BR><BR>I can't see what your point is. Do you want to know in which core is the processes allocated? Like if it's on the same processor or different ones?
<DIV>
<DIV></DIV>
<DIV><BR><BR><BR>
<DIV class=gmail_quote>On Tue, Jan 13, 2009 at 3:06 PM, Marcus Vinicius Brandão Soares <SPAN dir=ltr><<A href="mailto:mvbsoares@gmail.com" target=_blank rel=nofollow>mvbsoares@gmail.com</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">Hello Gustavo and all,<BR><BR>You described that you are using two machines with a dual processor in each one. If I can model it in a simple graph, we have two vertices and two unidirectional edges.<BR><BR>Each machine has a dual processor, each one with dual core, so there are 8 processor. But lets think again in the graph model: now we have two vertices, each one with two more vertices; these last two vertices have two more vertices too, and so this is the end.<BR><BR>Do you know the structure of the communication lines of the core processors ? <BR><BR>
<DIV class=gmail_quote>2009/1/13 Gustavo Miranda Teixeira <SPAN dir=ltr><<A href="mailto:magusbr@gmail.com" target=_blank rel=nofollow>magusbr@gmail.com</A>></SPAN>
<DIV>
<DIV></DIV>
<DIV><BR>
<BLOCKQUOTE class=gmail_quote style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">Hello everyone!<BR><BR>I've been experiencing some issues when using MPICH v1.2.7p1 and a SMP cluster and thought maybe some one can help me here.<BR><BR>I have a small cluster with two dual processor machines with gigabit ethernet communication. Each processor is a dual core which sums up to 8 cores of processors. When I run an application spreading 4 processes in both the machines (like distributing 2 processes in one machine and 2 processes in another) I get a significantly better performance than when I run the same application using 4 processes in only one machine. Isn`t it a bit curious? I know some people who also noticed that, but no one can explain me why this happens. Googling it didn't helped either. I originally thought it was a problem from my kind of application (a heart simulator which using PETSc to solve some
differential equations) but some simple experimentations showed a simple MPI_Send inside a huge loop causes the same issue. Measuring cache hits and misses showed it`s not a memory contention problem. I also know that a in-node communication in MPICH uses the loopback interface, but as far as I know a message that uses loopback interface simply takes a shortcut to the input queue instead of being sent to the device, so there is no reason for the message to take longer to get to the other processes. So, I have no idea why it`s taking longer to use MPICH in the same machine. Does anyone else have noticed that too? Is there some logical explanation for this to happen?<BR><BR>Thanks,<BR><FONT color=#888888>Gustavo Miranda Teixeira<BR></FONT></BLOCKQUOTE></DIV></DIV></DIV><BR><BR clear=all><BR>-- <BR>Marcus Vinicius<BR>--<BR>"Havendo suficientes colaboradores,<BR>Qualquer problema é passível de solução"<BR>Eric S. Raymond<BR>A Catedral e o
Bazar<BR><BR>"O passado é apenas um recurso para o presente"<BR>Clave de Clau<BR><BR>"Ninguém é tão pobre que não possa dar um abraço; e <BR>Ninguém é tão rico que não necessite de um abraço.<BR>Anônimo<BR></BLOCKQUOTE></DIV><BR></DIV></DIV></BLOCKQUOTE></DIV><BR><BR clear=all><BR>-- <BR>Marcus Vinicius<BR>--<BR>"Havendo suficientes colaboradores,<BR>Qualquer problema é passível de solução"<BR>Eric S. Raymond<BR>A Catedral e o Bazar<BR><BR>"O passado é apenas um recurso para o presente"<BR>Clave de Clau<BR><BR>"Ninguém é tão pobre que não possa dar um abraço; e <BR>Ninguém é tão rico que não necessite de um abraço.<BR>Anônimo<BR></DIV></DIV></DIV></BLOCKQUOTE></TD></TR></TBODY></TABLE><BR></BLOCKQUOTE></DIV><BR></DIV></BLOCKQUOTE></td></tr></table><br>