[Swift-user] Re: mixing gram execution and local filesystem providers
benc at hawaga.org.uk
Sun Feb 1 04:56:00 CST 2009
On Sun, 1 Feb 2009, Allan Espinosa wrote:
> i tried running the job normally via gridftp and still got the same thing:
Did you intend to post a log on this line?
> with coasters:
> log: http://www.ci.uchicago.edu/~aespinosa/swift/blast-20090201-0047-hav48kr0.log
That is showing errors in file put.
wrt you previous message in this thread, the local filesystem provider is
for when the Swift submit side has access to the same filesystem as the
worker nodes. In your case, that seems to be untrue.
The local filesystem provider works by copying things around from the
Swift command line client basically as if you had typed 'cp'. If you can't
cp files into place from the same place that you are running Swift, then
you can't use the local filesystem provider.
If you have files elsewhere, you can map them using a URI. For example,
something like this:
file myfile <"gsiftp://tp-login.ci.uchicago.edu/etc/group">;
(or likewise for other mappers).
Then if you use the gridftp provider, third party transfer will be used,
so the data won't actually leave the cluster.
Exciting feature creep for the coaster filesystem provider might be
remote site-local filesystem staging, to handle your case.
More information about the Swift-user