[Swift-user] Re: Provider staging vs coaster data provider

[Mihael Hategan]

On Fri, 2010-11-05 at 15:28 -0500, Michael Wilde wrote:
> Mihael, you may have explained this to me already, but can you clarify:
> 
> In addition to provider staging using the coaster provider, it seems
> you can say <filesystem provider=coaster> (or <gridftp>) which uses
> the coaster channel and agent to move data?

Yes.

> 
> One difference is that with provider staging, *all* file transfer for
> all sites is done via that method, correct?

And without provider staging all file transfer for all sites is done
without provider staging (i.e. the converse is also true). This is a
consequence of vdl-int getting too messy to have both in the same run,
but it's not a theoretical impossibility.

> 
> If a coaster filesystem provider is available, then the user can use
> that on selected sites, while other sites can use any other provider,
> correct?

That is correct.

> 
> Provider staging with the coaster provider is done via the coaster
> data channel and all data goes directly to a job directory typically
> placed on the worker node local filesystem, right?

Also correct.

> 
> Now, with coaster data provider staging, does that same restriction
> apply, or can the coaster data provider (assuming it really exists)
> place data on any path accessible to the worker? I.e., one could use a
> standard shared workdirectory if one was so inclined, although that
> would be counter-productive.

The coaster data provider works like any other data provider and its
usage is consistent with swift's traditional way of working. In other
words, files are copied to a shared directory, cached there, and the
shared directory must be accessible to the worker node.

Though which one of the methods is "restrictive" I don't know.

Mihael