[Swift-devel] [Swift-user] swift on ranger

[Michael Wilde]

The Ranger user guide:

http://www.tacc.utexas.edu/user-services/user-guides/ranger-user-guide

Has a section (below) on using < 16 cores per node. (Im not sure if "15" below is a typo; I think 16 was intended).

- Mike

Using fewer than 16 cores per node

When you want to use less than 16 MPI tasks per node, the choice of tasks per node is limited to the set of numbers {1, 2, 4, 8, 12, and 15}.

When the number of tasks you need is equal to "Number of Tasks per Node x Number of Nodes", then use the following command:

#$ -pe <TpN>way <NoN x 16>

where <TpN>is a number in the set {1, 2, 4, 8, 12, 15}.

If the Total number of Tasks that you need is less than "Number of Tasks per Node x Number of Nodes", then set the MY_NSLOTS environment variable to the Total number of Tasks <TnoT>needed. In a job script, use the following -pe option and environment variable statement:

#$ -pe <TpN>way <NoN x 16>
export MY_NSLOTS=<TnoT> # For Bourne shells

or

setenv MY_NSLOTS <TnoT> # For C shells

where <TpN>is a number in the set {1, 2, 4, 8, 12, 15}. For example, using a Bourne shell:

#$ -pe 8way 64 # Use 8 Tasks per Node, 4 Nodes requested
export MY_NSLOTS=31 # 31 tasks are launched

----- Original Message -----
> From: "Mihael Hategan" <hategan at mcs.anl.gov>
> To: "Sarah Kenny" <skenny at uci.edu>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Swift User" <swift-user at ci.uchicago.edu>
> Sent: Wednesday, December 21, 2011 5:53:53 PM
> Subject: Re: [Swift-user] [Swift-devel] swift on ranger
> On Wed, 2011-12-21 at 14:50 -0800, Sarah Kenny wrote:
> > yr right ketan, if i change it to: <profile namespace="globus"
> > key="jobsPerNode">16</profile> the warning message goes away.
> > however,
> > there are times i don't want to run 16 jobs per node...e.g. bcs a
> > single job needs all the available memory so even though the node
> > has
> > 16 processors i can't actually use them all. so perhaps this is just
> > a
> > scheduling issue with ranger/sge in that they don't want you to
> > submit
> > a job that's going to leave processors idle? that seems a bit
> > restrictive though...
> 
> Hmmm. I think that one should be able to request 16 cores and run only
> two jobs at a time on them. Have you tried pe=2way?
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory