[mpich-discuss] Hydra: LSF resource querying in exclusive mode

[Pavan Balaji]

On 11/11/2010 12:15 PM, Yauheni Zelenko wrote:
> I tried to submit sample jobs on LSF 7.0.5.
>
> For bsub -x -n 2 LSB_MCPU_HOSTS contains 2.
>
> For bsub -x -R "CPUS>=2" LSB_MCPU_HOSTS contains 1.
>
> In both cases physical host may contains more CPUs which still may be
> used by applications.

We talked a bit about this here, and we think that this behavior is 
correct. In the first case when the user specifies "-n 2", the user is 
explicitly asking for two MPI processes. In this case, mpiexec should 
not launch more than 2 MPI processes. In the case where the user does 
not specify a "-n" parameter at all, LSF is defaulting to "-n 1", which 
also seems to be correct.

The reasoning is that, we don't know if the user wants to use MPI 
processes on all cores, or needed exclusive access to the node in order 
to use all its memory (rather than all its cores), or wants to use 
threads on the remaining cores. So, whenever the user specifies "-n", 
using that value for the number of MPI processes, and defaulting to 
whatever LSF wants to use when the user doesn't specify "-n" seems fine 
to us.

  -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji