[mpich-discuss] HP-XC 3000 cluster issues

[Dave Goodell]

If you want to run MPICH2 on a cluster using SLURM, don't use the  
mpiexec/mpirun included with MPICH2.  Do not use the HP-MPI mpiexec/ 
mpirun that is installed on your system.  Use srun instead.  If some  
piece of software you want to run requires that the job run script is  
named "mpiexec"/"mpirun", create a script that will do the right thing  
(calling srun or bsub+srun) and make sure it's in the right location  
in the PATH.  Anything else is almost certainly not what you want to do.

Another option is to simply use the HP-MPI if you are just trying to  
run normal MPI programs.  Your cluster vendor should have provided  
ample documentation on how to do this and it will likely provide  
better network performance than a default installation of MPICH2 will,  
since your vendor is able to tune it to your particular network.

-Dave

On Mar 5, 2009, at 1:02 PM, Gauri Kulkarni wrote:

> In the installation that we have tried here, there definitely is a  
> folder named bin/ that contains mpirun and mpiexec which point to  
> mpiexec.py. I have modified PATH so that when I issue "which mpirun"  
> I get the mpirun in mpich2 installation. So if I use mpirun from  
> command line instead of bsub, it calls the mpirun from mpich2  
> directory and runs correctly on one node (with 8 procs).
>
> Gauri.
> ---------
>
>
> On Thu, Mar 5, 2009 at 8:34 PM, Dave Goodell <goodell at mcs.anl.gov>  
> wrote:
> On Mar 5, 2009, at 8:23 AM, Gauri Kulkarni wrote:
>
> Of course, using the same command - only with mpirun -srun ./ 
> helloworld.mpich2 - still gives me the error that no mpd is running  
> on the node on which the job was assigned.
>
> Does this mean, I do not need to use mpirun when MPICH2 is  
> configured with SLURM? What about the softwares that make specific  
> call to mpirun or mpiexec?
>
> I believe that the mpirun command is for the HP MPI packaged with  
> your cluster and will not work with MPICH2.  You'll have to figure  
> out how to handle each usage of mpiexec or mpirun on a case-by-case  
> basis.  One strategy for fixing this is to create a shell script  
> named "mpirun" or "mpiexec" that simply invokes the appropriate bsub/ 
> srun commands and then figure out how to set the PATH correctly for  
> those applications so that your custom mpirun is called instead of  
> the system one.
>
> -Dave
>
>