[Swift-devel] manual coasters

[Michael Wilde]

My understanding is that with the pool entry below, Swift will start the coaster service by submitting a PBS job. Then the swift command will print the service service URL (host:port ?) on stderr, and you manually start workers, passing that host:port on the command line to connect back to the coaster service.

>> Then, when swift starts the coaster service, it will print the URL of
>> that on stderr.

> >> worker.pl <ServiceURL> <blockid> <logdir>

- Mike

----- "Allan Espinosa" <aespinosa at cs.uchicago.edu> wrote:

> So for the pool entry below, where is the serviceURL?  the submit
> host
> will issue a pbs request for a service host?
> 
> 
> Thanks,
> -Allan
> 
> 2010/7/1  <wilde at mcs.anl.gov>:
> > Very cool - thanks, Mihael!
> >
> > For the sites entry, do we still use the current format to indicate
> where the server should start?
> > Eg:
> >
> >  <pool handle="coasterpool01">
> >    <execution provider="coaster" url="none" jobManager="pbs"/>
> >    <profile namespace="globus">
> key="workerManager">passive</profile>
> >    <profile namespace="globus" key="queue">fast</profile>
> >    <profile namespace="karajan" key="initialScore">10000</profile>
> >    <profile namespace="karajan" key="jobThrottle">.07</profile>
> >    <gridftp  url="local://localhost" />
> >    <workdirectory >/home/wilde/swiftwork</workdirectory>
> >  </pool>
> >
> > Is the full range of provider options available to start the server
> in passive mode?
> >
> > Will throttling settings be honored?
> >
> > Can we start multiple coaster servers in different places?
> >
> >
> > - Mike
> >
> >
> > ----- "Mihael Hategan" <hategan at mcs.anl.gov> wrote:
> >
> >> Manual coasters are in trunk. I did some limited testing on
> >> localhost.
> >>
> >> The basic idea is that you say <profile namespace="globus"
> >> key="workerManager">passive</profile> in sites.xml. Other than
> that
> >> you
> >> may want to set workersPerNode, but the other options are useless.
> >>
> >> Then, when swift starts the coaster service, it will print the URL
> of
> >> that on stderr.
> >>
> >> You carefully dig for worker.pl and then launch it in whatever way
> >> you
> >> like:
> >>
> >> worker.pl <ServiceURL> <blockid> <logdir>
> >>
> >> The blockid can be whatever you want, but it can be used to group
> >> workers in the traditional blocks. The logdir is where you want
> the
> >> worker logs to go. They are all mandatory.
> >>
> >> When workers connect to the service, the service should start
> >> shipping
> >> jobs to them. When the service is shut down, it will also try to
> shut
> >> down the workers (they are useless anyway at that point), but it
> >> cannot
> >> control the LRM jobs, so it may fail to do so (or rather said, it
> is
> >> more likely to fail to do so).
> >>
> >> Mihael
> >>

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory