[Swift-user] Re: mixing gram execution and local filesystem providers

[Ben Clifford]

On Sun, 1 Feb 2009, Allan Espinosa wrote:

> i tried running the job normally via gridftp and still got the same thing:

Did you intend to post a log on this line?

> with coasters:
> log: http://www.ci.uchicago.edu/~aespinosa/swift/blast-20090201-0047-hav48kr0.log

That is showing errors in file put.

wrt you previous message in this thread, the local filesystem provider is 
for when the Swift submit side has access to the same filesystem as the 
worker nodes. In your case, that seems to be untrue.

The local filesystem provider works by copying things around from the 
Swift command line client basically as if you had typed 'cp'. If you can't 
cp files into place from the same place that you are running Swift, then 
you can't use the local filesystem provider.

If you have files elsewhere, you can map them using a URI. For example, 
something like this:

  file myfile <"gsiftp://tp-login.ci.uchicago.edu/etc/group">;

(or likewise for other mappers).

Then if you use the gridftp provider, third party transfer will be used, 
so the data won't actually leave the cluster.

Exciting feature creep for the coaster filesystem provider might be 
remote site-local filesystem staging, to handle your case.

--