[mpich-discuss] MPICH2 and OpenPBS/Torque using MPICH2-Hydra

[balaji at mcs.anl.gov]

Hi Simon,

The process management would be very similar. The only difference will be that instead of using pbsmon daemons, we'd use ssh daemons for the initial launch.

I expect the ranks to be allocated in the order that is specified by the PBS environment. You can, of course, change this to whatever you want using the hostfile option (see http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager). All regular options of Hydra should work in the same manner as now.

Let me poke around with the code today; I'll get back with more details.

 -- Pavan

----- "Simon Hammond" <simon.hammond at gmail.com> wrote:

> This would definitely be helpful.
> 
> I'm guessing these processes (under SSH) wouldn't be managed in the
> same way that PBS would say an OpenMPI job? Getting rid of the -n
> would be a good start. Can I ask how node placement and rank
> allocation etc would work?
> 
> 
> 
> 
> S.
> 
> 
> 2009/9/23  <balaji at mcs.anl.gov>:
> >
> > PBS support is not added in Hydra yet. See:
> https://trac.mcs.anl.gov/projects/mpich2/ticket/443
> >
> > However, so far we were considering PBS support as a bootstrap
> server (which has a lot more functionality than just providing the
> number of nodes). But adding PBS as a resource management kernel (RMK)
> is possible too and should be simpler as well. Doing this will allow
> you to skip the "-n" option in PBS environments, but you'll internally
> still be using ssh to start the processes. If this is helpful for you,
> we can consider adding it for the next release.
> >
> >  -- Pavan
> >
> > ----- "Si Hammond" <simon.hammond at gmail.com> wrote:
> >
> >> Hi everyone,
> >>
> >> We're trying to get MPICH2 to work under PBS so that users don't
> need
> >>
> >> to specify the number of processors etc. We have built the install
> >> with the MPICH2-Hydra device and this seems to work (in that jobs
> will
> >>
> >> run) but users still have to specify the number of MPI ranks using
> >> -n.
> >>
> >> I found the -rmk flag in some documentation using Google but "-rmk
> >> pbs" doesn't seem to work either (the example uses "-rmk lsf")
> >>
> >> How does a user like myself go about making a PBS-enabled MPICH2
> build
> >>
> >> to the job placement etc is all handled under the covers? Is this
> >> possible? OpenMPI seems to have this covered but we'd like to
> install
> >>
> >> MPICH2 as well since one of our codes only runs with MPICH2.
> >>
> >> Thanks for your help.
> >>
> >>
> >>
> >>
> ---------------------------------------------------------------------------------------
> >> Si Hammond
> >>
> >> Performance Modelling, Analysis and Optimisation Laboratory
> >> High Performance Systems Group
> >> Department of Computer Science
> >> University of Warwick, CV4 7AL, UK
> >>
> ----------------------------------------------------------------------------------------
> >