[mpich-discuss] Sorting nodes by CPUs with -ppn (1.4 and r9834)

[Yauheni Zelenko]

Information about CPUs assignment is passed in environment in case of rsh/ssh bootstrap or extracted form LSF/SGE environment. So it's available at some point of application life.

Probably you suggestion with new communicator will solve our problem, but I think will be good idea to improve Hydra. In this case it behavior be consistent and other application could benefit from it.

Eugene.
________________________________________
From: Pavan Balaji [balaji at mcs.anl.gov]
Sent: Tuesday, May 08, 2012 3:25 PM
To: mpich-discuss at mcs.anl.gov
Cc: Yauheni Zelenko
Subject: Re: [mpich-discuss] Sorting nodes by CPUs with -ppn (1.4 and r9834)

The -ppn setting was always to specify how many cores you want Hydra to
consider on each node (we can call it -cpn for cores per node, if it
makes it clearer).  This is needed, if you have more processes than the
number of nodes either at the start, or through dynamic processes, when
we might have to wrap back to first node.

Also, if you are creating threads based on the number of cores
available, don't you already know the number of cores on each node?  If
yes, why not just create a new communicator with the ranks reordered?

  -- Pavan

On 05/08/2012 05:09 PM, Yauheni Zelenko wrote:
> Hi!
>
> Hydra assumes number of processes per node as number of cores (in 1.4 and r9834) when -ppn specified.
>
> In this case -order-nodes behaves inconsistently with -ppn (order from command line) and without (nodes are sorted).
>
> Our application could use mixed MPI/multi-threads mode. In this case with use -ppn 1, but still sorting nodes by number of CPUs make sense for us to run master process (rank 0) on host with most CPUs.
>
> This is especially useful in farm environment like LSF/SGE/etc when CPUs on hosts may be allocated randomly.
>
> Eugene.
> _______________________________________________
> mpich-discuss mailing list     mpich-discuss at mcs.anl.gov
> To manage subscription options or unsubscribe:
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss

--
Pavan Balaji
http://www.mcs.anl.gov/~balaji