[Swift-devel] Re: Problems with file and sfs stagingMethods

[Mihael Hategan]

I committed a few changes to deal with the problem. Cog r2890 contains
them.

Justin, it's possible that I may have introduced some incompatibilities
with pinned:

Essentially all staging paths need to now be of one of the following
forms:
1. <protocol>://<host>/<path>
2. <protocol>:<path>
3. <path>

Also, direct providers (i.e. "sfs") need to have absolute paths for
client files because the CWD of the client is unknown to the worker.

So please test and tell me if it works.

Mihael

On Sat, 2010-09-25 at 12:03 -0700, Mihael Hategan wrote:
> I can reproduce the problem with my existing configuration. I'm trying
> to figure out how that differs from what I saw before.
> 
> Mihael
> 
> On Thu, 2010-09-23 at 17:54 -0600, wilde at mcs.anl.gov wrote:
> > Hi Mihael,
> > 
> > I've reproduced these problems I mentioned a week ago with provider staging. At the moment, only "proxy" mode works for me.
> > 
> > On the CI net in /home/wilde/swift/lab/:
> > 
> > All tests were done using run.sh which sets all the config files. Each test was a single cat of data.txt to outdir/f.0001.out
> > 
> > Each run dir contains a coasters/ dir with the coaster worker log.
> > 
> > run01: this run worked, with stagingMethod proxy
> > 
> > run02: stagingMethod file, files.  The remote side is opening a bad file name (same I I showed you a week ago) with the file:// left in the *middle* of the path name. What gets returned as data to the worker is the Java exception and traceback text.
> > 
> > run03: stagingMethod sfs. What happens here is (a) the sfs:// gets stripped off, but localhost// remains at the front of the source file name.  When I strip that off, the next problem is that the destination directory has not been created. When I create that, the last problem is that the worker is referencing the relative input pathname "data.txt" from a different directory and without the necessary directory prefix.  I explored this in dirs run04-run07 with mods to worker.pl that you can see in the coaster logs, which have been placed in each run dir below coasters/ using a symlink created in run.sh
> > 
> > Can you fix these two stagingMethods?  Or, if your tests are working, please point me to your config files, script, and swift revision.
> > 
> > My swift is in: ~/swift/rev/trunk/bin/swift
> >  which is a symlink to the dist/ in my ~/swift/src/trunk
> > 
> > Thanks,
> > 
> > Mike
> > 
> > 
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel