[Swift-devel] Re: Problems with file and sfs stagingMethods

[Mihael Hategan]

On Mon, 2010-09-27 at 15:00 -0500, Justin M Wozniak wrote:
> Yeah, that's incompatible.  Do you think it would help if I switched the 
> URL prefix from "pinned:" to "pinned-" ?

No. I think that we should fix whatever codepath is involved. Can you
let me know how to reproduce the problem?

Mihael

> 
> On Sat, 25 Sep 2010, Mihael Hategan wrote:
> 
> > I committed a few changes to deal with the problem. Cog r2890 contains
> > them.
> >
> > Justin, it's possible that I may have introduced some incompatibilities
> > with pinned:
> >
> > Essentially all staging paths need to now be of one of the following
> > forms:
> > 1. <protocol>://<host>/<path>
> > 2. <protocol>:<path>
> > 3. <path>
> >
> > Also, direct providers (i.e. "sfs") need to have absolute paths for
> > client files because the CWD of the client is unknown to the worker.
> >
> > So please test and tell me if it works.
> >
> > Mihael
> >
> > On Sat, 2010-09-25 at 12:03 -0700, Mihael Hategan wrote:
> >> I can reproduce the problem with my existing configuration. I'm trying
> >> to figure out how that differs from what I saw before.
> >>
> >> Mihael
> >>
> >> On Thu, 2010-09-23 at 17:54 -0600, wilde at mcs.anl.gov wrote:
> >>> Hi Mihael,
> >>>
> >>> I've reproduced these problems I mentioned a week ago with provider staging. At the moment, only "proxy" mode works for me.
> >>>
> >>> On the CI net in /home/wilde/swift/lab/:
> >>>
> >>> All tests were done using run.sh which sets all the config files. Each test was a single cat of data.txt to outdir/f.0001.out
> >>>
> >>> Each run dir contains a coasters/ dir with the coaster worker log.
> >>>
> >>> run01: this run worked, with stagingMethod proxy
> >>>
> >>> run02: stagingMethod file, files.  The remote side is opening a bad file name (same I I showed you a week ago) with the file:// left in the *middle* of the path name. What gets returned as data to the worker is the Java exception and traceback text.
> >>>
> >>> run03: stagingMethod sfs. What happens here is (a) the sfs:// gets stripped off, but localhost// remains at the front of the source file name.  When I strip that off, the next problem is that the destination directory has not been created. When I create that, the last problem is that the worker is referencing the relative input pathname "data.txt" from a different directory and without the necessary directory prefix.  I explored this in dirs run04-run07 with mods to worker.pl that you can see in the coaster logs, which have been placed in each run dir below coasters/ using a symlink created in run.sh
> >>>
> >>> Can you fix these two stagingMethods?  Or, if your tests are working, please point me to your config files, script, and swift revision.
> >>>
> >>> My swift is in: ~/swift/rev/trunk/bin/swift
> >>>  which is a symlink to the dist/ in my ~/swift/src/trunk
> >>>
> >>> Thanks,
> >>>
> >>> Mike
> >>>
> >>>
> >>
> >>
> >> _______________________________________________
> >> Swift-devel mailing list
> >> Swift-devel at ci.uchicago.edu
> >> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> >
> >
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> >
>