[mpich-discuss] MPICH2-1.0.8 performance issues on Opteron Cluster

[Dmitry V Golovashkin]

We have similar experiences with nemesis in a prior mpich2 release.
(scalapack-ish applications on multicore linux cluster).
The resultant times were remarkably slower. The nemesis channel was an
experimental feature back then, I attributed slower performance to a
possible misconfiguration.
Is it possible to submit a new ticket (for non-ANL folks)?



On Mon, 2009-01-05 at 09:00 -0500, James S Perrin wrote:
> Hi,
> 	I thought I'd just mention that I too have found that our software 
> performs poorly with nemesis compared to ssm on our multi-core machines. 
> I've tried it on both a 2xDual core AMD x64 and 2xQuad core Xeon x64 
> machines. It's roughly 30% slower. I've not been able to do any analysis 
> as yet as to where the nemesis version is loosing out?
> 
> 	The software performs mainly point-to-point communication in a master 
> and slaves model. As the software is interactive the slaves call 
> MPI_Iprobe while waiting for commands. Having compiled against the ssm 
> version would have no effect, would it?
> 
> Regards
> James
> 
> Sarat Sreepathi wrote:
> > Hello,
> > 
> > We got a new 10-node Opteron cluster in our research group. Each node 
> > has two quad core Opterons. I installed MPICH2-1.0.8 with Pathscale(3.2) 
> > compilers and three device configurations (nemesis,ssm,sock). I built 
> > and tested using the Linpack(HPL) benchmark with ACML 4.2 BLAS library 
> > for the three different device configurations.
> > 
> > I observed some unexpected results as the 'nemesis' configuration gave 
> > the worst performance. For the same problem parameters, the 'sock' 
> > version was faster and the 'ssm' version hangs. For further analysis, I 
> > obtained screenshots from the Ganglia monitoring tool for the three 
> > different runs. As you can see from the attached screenshots, the 
> > 'nemesis' version is consuming more 'system cpu' according to Ganglia. 
> > The 'ssm' version fares slightly better but it hangs towards the end.
> > 
> > I may be missing something trivial here but can anyone account for this 
> > discrepancy? Isn't 'nemesis' device or 'ssm' device recommended for this 
> > cluster configuration? Your help is greatly appreciated.
>