[Swift-devel] coasters on UC teraport with head job running on a worker node

[Ben Clifford]

I hacked around with coasters here to see about getting the head job 
running on a cluster worker node rather than on the cluster head node.

This works on teraport through PBS. The below patch contains the changes I 
made to make that happen.

http://www.ci.uchicago.edu/~benc/tmp/coaster-head-elsewhere

There are three changes I made:

    i) submit to pbs jobmanager instead of to fork jobmanager
   ii) start coaster workers with IP address of the head-worker node 
       instead of the address of the cluster head node
  iii) hack the environment to point to teraport's CA directory (in the 
       environment that I get there, there is no automatically findable 
       CA directory, and an ENV profile appeared to not work).

In situations where the cluster nodes have outbound network connectivity, 
this seems like a nice thing to do, and I want to make this a configurable 
option to go into the SVN.

I think:

i) above should probably be an extension to the existing three-field 
coaster jobmanager string, ii) should be a configuration option to go 
along-side the coasterInternalIP setting and iii) should be a more general 
ability to set the environment for a coaster worker.

Here is the site.xml that I used with this patch:

<config>

  <pool handle="teraport" >
    <gridftp  url="gsiftp://tp-osg.ci.uchicago.edu" /> 
    <execution provider="coaster" url="tp-osg.ci.uchicago.edu" 
jobManager="gt2:gt2:pbs" />
    <workdirectory >/home/benc/swifttest</workdirectory>
  </pool>

</config>

--