[Swift-devel] Re: Persistent coasters

[Allan Espinosa]

Since PBS on PADS doesn't care if we requested for  multiple nodes or
multiple jobs, i just send multiple jobs of

worker.pl <coaster-service-url> ... ...

2011/1/13 Michael Wilde <wilde at mcs.anl.gov>:
> You should configure the tools for this mode of operation on PADS (and any PBS system):
>
> - run the commands on a login node (but should work on any PADS node that you are ssh'ed into)
>
> - use qsub to obtain nodes
>  -- mode 1: 1 N-node M-core job
>  -- mode 2: N 1-core jobs
>
> Do mode 1 first:
>
> Job script (the script you use as an arg to qsub) should use a foreach loop to start one worker.pl on each node of the job. You can adapt the code below from Swift R start-swift:
>
> make-pbs-submit-file()
> {
>  if [ $queue != default ]; then
>    queueDirective="#PBS -q $queue"
>  else
>    queueDirective=""
>  fi
> cat >pbs.sub <<END
> #PBS -S /bin/sh
> #PBS -N SwiftR-workers
> #PBS -m n
> #PBS -l nodes=$nodes
> #PBS -l walltime=$time
> #PBS -o $HOME
> #PBS -e $HOME
> $queueDirective
> WORKER_LOGGING_ENABLED=true # FIXME: parameterize; fix w PBS -v
> #cd / && /usr/bin/perl $SWIFTBIN/worker.pl $CONTACT SwiftR-workers $HOME/.globus/coasters $IDLETIMEOUT
> HOST=\$(echo $CONTACT | sed -e 's,^http://,,' -e 's/:.*//')
> PORT=\$(echo $CONTACT | sed -e 's,^.*:,,')
> echo '***' PBS_NODEFILE file: \$PBS_NODEFILE CONTACT:$CONTACT
> cat \$PBS_NODEFILE
> echo '***' unique nodes are:
> sort < \$PBS_NODEFILE|uniq
> for h in \$(sort < \$PBS_NODEFILE|uniq); do
>  ssh \$h "echo Swift R startup running on host; hostname; cd /; /usr/bin/perl $SWIFTBIN/worker.pl $CONTACT SwiftR-\$h $HOME/.globus/\
> coasters $IDLETIMEOUT" &
> done
> wait
> END
> }
>
> then:
>
>  make-${server}-submit-file
>  qsub pbs.sub >$pbsjobidfile
>
>
> Mike
>
> ----- Original Message -----
>> How should I proceed in testing the persistent coasters scripts on
>> PADS? Should I use workers-ssh from the login node to pads? Should I
>> copy the format of workers-cobalt and modify it to use qsub parameters
>> that work with pbs?
>>
>> David

-- 
Allan M. Espinosa <http://amespinosa.wordpress.com>
PhD student, Computer Science
University of Chicago <http://people.cs.uchicago.edu/~aespinosa>