Fwd: Re: [MPICH] heterogeneous cluster

[Syed Irfan]

Hello all,

Thx Phil for the info :)

What are the gotchas on getting it to work under risc processors? If I 
want to get mpich2 working on them.

Any pointers on which code area I need to dig in :)

Hmm btw bluegene/L seems to use powerpc cores for its computation nodes 
running mpi1x/mpi2x/mpichx? any one can confirm?

Cheers,
Irfan

Philip Sydney Lavers wrote:
 >  my experience is with small clusters (4-6 nodes), including
 > one dual processor machine on each. At home the cluster is 32
 > bit x86 machines. At campus the cluster is 64 bit machines.
 > With pvm I mixed 64 and 32 bit - no problems, but I would not
 > bother now that I have installed MPICH2.
 >
 > At one stage both clusters included Solaris10 x86, Mandrake
 > Linux 10.1, and FreeBSD 5.3. No problems - simply download the
 > source into each node and build on that node. I have tried
 > both SunStudio10 and gcc on the Solaris machine.
 >
 > The endianess is handled in each separate build by configure
 > and make. Thereafter, after starting the ring with mpd -h foo
 > -p xyz on each machine everything works fine. my programmes
 > are built from the same code on each node, BUT I make sure
 > that all data communicated is only by packages of MPI types or
 > my user defined derivatives of such.
 >
 > e.g.
 >
 > ...
 > using namespace MPI;
 > ...
 > ...
 > MPI::COMM_WORLD.Bcast(vortslocbuff,2*numvorts ,DOUBLE,0);   ...
 > ...
 >
 > and so on;
 >
 > MPI guarantees that MPI::DOUBLE will be correct for each
 > machine, regardless of its endianes, size of double etc etc.
 >
 > The reason I no longer mix 32 and 64 bit nodes is speed.
 > There is no point in having slow nodes, and my 32 bit machines
 >  are somewhat slower than the 64 bit platforms.
 > For the same reason my campus cluster is now only FreeBSD. For
 > reasons I dont understand, any given machine runs my
 > simulations more than three times faster under FreeBSD than
 > under LINUX (gcc i.e. mpicxx on both, CFLAGS=-g -pg
 > $(INCLUDES) -ffast-math -O3 -march=athlon64 on both).
 >
 > At home I continue to develop and test on the mixed cluster.
 > Here the Solaris10 is on a dual processor 350 MHz Pentium. The
 >   Linux and FreeBSD are on athlon 32 bit machines. The Solaris
 > machine is painfully slow, but I love it because of the
 > general excellence of Solaris10 AND because I bought the
 > machine for $70 at a computer fair and the Solaris is free.
 >
 > Hope this helps
 >
 > regards, Phil Lavers