[Swift-devel] Re: scheduling

[Mihael Hategan]

On Wed, 2010-03-31 at 17:30 +0000, Ben Clifford wrote:
> > > - the Swift execution wrapper _swiftwrap and/or the post-execution
> > > logic and cache management logic in Swift which can influence whether
> > > and how long a file stays in the site shared/ cache
> > 
> > Assuming that there is a shared cache. The trend we're seeing with EAGER
> > is that there is no shared directory any more, which was used solely as
> > a caching mechanism due to the fact that the client did the staging.
> 
> That is the situation in other places where people might want to use Swift 
> too, although not ones that have funding to be developed - one is the 
> european model of a grid-central resource broker that carries everything 
> along in a job submission, and the other is a condor pool running on 
> workstations with no shared filesystem. (both of those have been 
> prototyped with Swift in the past).
> 

Right. This is also the traditional GRAM model (GRAM does staging). We
avoided that because of the issues with WS-GRAM. However, now that
WS-GRAM is going to be slowly retired and because coasters support
client-to-worker-node staging, it is probably time to move away from
that choice.

It does, however, introduce some additional complications when it comes
to managing resources. As opposed to client-directed staging, where we
could say "initiate no more than 4 concurrent transfers", it's hard to
get a bunch of distributed entities to agree on when to transfer or not.