[mpich-discuss] MPICH2-1.0.8 performance issues on Opteron Cluster

[James S Perrin]

Hi,
	I thought I'd just mention that I too have found that our software 
performs poorly with nemesis compared to ssm on our multi-core machines. 
I've tried it on both a 2xDual core AMD x64 and 2xQuad core Xeon x64 
machines. It's roughly 30% slower. I've not been able to do any analysis 
as yet as to where the nemesis version is loosing out?

	The software performs mainly point-to-point communication in a master 
and slaves model. As the software is interactive the slaves call 
MPI_Iprobe while waiting for commands. Having compiled against the ssm 
version would have no effect, would it?

Regards
James

Sarat Sreepathi wrote:
> Hello,
> 
> We got a new 10-node Opteron cluster in our research group. Each node 
> has two quad core Opterons. I installed MPICH2-1.0.8 with Pathscale(3.2) 
> compilers and three device configurations (nemesis,ssm,sock). I built 
> and tested using the Linpack(HPL) benchmark with ACML 4.2 BLAS library 
> for the three different device configurations.
> 
> I observed some unexpected results as the 'nemesis' configuration gave 
> the worst performance. For the same problem parameters, the 'sock' 
> version was faster and the 'ssm' version hangs. For further analysis, I 
> obtained screenshots from the Ganglia monitoring tool for the three 
> different runs. As you can see from the attached screenshots, the 
> 'nemesis' version is consuming more 'system cpu' according to Ganglia. 
> The 'ssm' version fares slightly better but it hangs towards the end.
> 
> I may be missing something trivial here but can anyone account for this 
> discrepancy? Isn't 'nemesis' device or 'ssm' device recommended for this 
> cluster configuration? Your help is greatly appreciated.

-- 
------------------------------------------------------------------------
   James S. Perrin
   Visualization

   Research Computing Services
   Devonshire House, University Precinct
   The University of Manchester
   Oxford Road, Manchester, M13 9PL

   t: +44 (0) 161 275 6945
   e: james.perrin at manchester.ac.uk
   w: www.manchester.ac.uk/researchcomputing
------------------------------------------------------------------------
  "The test of intellect is the refusal to belabour the obvious"
  - Alfred Bester
------------------------------------------------------------------------