[mpich-discuss] Hydra: LSF resource querying in exclusive mode

[Yauheni Zelenko]

In second case I explicitely asked for more then 2 CPUs. -n is not only way to request CPUs.

Eugene.
________________________________________
From: Pavan Balaji [balaji at mcs.anl.gov]
Sent: Thursday, November 11, 2010 10:36 AM
To: Yauheni Zelenko
Cc: mpich-discuss at mcs.anl.gov
Subject: Re: [mpich-discuss] Hydra: LSF resource querying in exclusive mode

On 11/11/2010 12:15 PM, Yauheni Zelenko wrote:
> I tried to submit sample jobs on LSF 7.0.5.
>
> For bsub -x -n 2 LSB_MCPU_HOSTS contains 2.
>
> For bsub -x -R "CPUS>=2" LSB_MCPU_HOSTS contains 1.
>
> In both cases physical host may contains more CPUs which still may be
> used by applications.

We talked a bit about this here, and we think that this behavior is
correct. In the first case when the user specifies "-n 2", the user is
explicitly asking for two MPI processes. In this case, mpiexec should
not launch more than 2 MPI processes. In the case where the user does
not specify a "-n" parameter at all, LSF is defaulting to "-n 1", which
also seems to be correct.

The reasoning is that, we don't know if the user wants to use MPI
processes on all cores, or needed exclusive access to the node in order
to use all its memory (rather than all its cores), or wants to use
threads on the remaining cores. So, whenever the user specifies "-n",
using that value for the number of MPI processes, and defaulting to
whatever LSF wants to use when the user doesn't specify "-n" seems fine
to us.

  -- Pavan

--
Pavan Balaji
http://www.mcs.anl.gov/~balaji