[Swift-devel] Re: Problems with file and sfs stagingMethods

[Justin M Wozniak]

Yeah, that's incompatible.  Do you think it would help if I switched the 
URL prefix from "pinned:" to "pinned-" ?

On Sat, 25 Sep 2010, Mihael Hategan wrote:

> I committed a few changes to deal with the problem. Cog r2890 contains
> them.
>
> Justin, it's possible that I may have introduced some incompatibilities
> with pinned:
>
> Essentially all staging paths need to now be of one of the following
> forms:
> 1. <protocol>://<host>/<path>
> 2. <protocol>:<path>
> 3. <path>
>
> Also, direct providers (i.e. "sfs") need to have absolute paths for
> client files because the CWD of the client is unknown to the worker.
>
> So please test and tell me if it works.
>
> Mihael
>
> On Sat, 2010-09-25 at 12:03 -0700, Mihael Hategan wrote:
>> I can reproduce the problem with my existing configuration. I'm trying
>> to figure out how that differs from what I saw before.
>>
>> Mihael
>>
>> On Thu, 2010-09-23 at 17:54 -0600, wilde at mcs.anl.gov wrote:
>>> Hi Mihael,
>>>
>>> I've reproduced these problems I mentioned a week ago with provider staging. At the moment, only "proxy" mode works for me.
>>>
>>> On the CI net in /home/wilde/swift/lab/:
>>>
>>> All tests were done using run.sh which sets all the config files. Each test was a single cat of data.txt to outdir/f.0001.out
>>>
>>> Each run dir contains a coasters/ dir with the coaster worker log.
>>>
>>> run01: this run worked, with stagingMethod proxy
>>>
>>> run02: stagingMethod file, files.  The remote side is opening a bad file name (same I I showed you a week ago) with the file:// left in the *middle* of the path name. What gets returned as data to the worker is the Java exception and traceback text.
>>>
>>> run03: stagingMethod sfs. What happens here is (a) the sfs:// gets stripped off, but localhost// remains at the front of the source file name.  When I strip that off, the next problem is that the destination directory has not been created. When I create that, the last problem is that the worker is referencing the relative input pathname "data.txt" from a different directory and without the necessary directory prefix.  I explored this in dirs run04-run07 with mods to worker.pl that you can see in the coaster logs, which have been placed in each run dir below coasters/ using a symlink created in run.sh
>>>
>>> Can you fix these two stagingMethods?  Or, if your tests are working, please point me to your config files, script, and swift revision.
>>>
>>> My swift is in: ~/swift/rev/trunk/bin/swift
>>>  which is a symlink to the dist/ in my ~/swift/src/trunk
>>>
>>> Thanks,
>>>
>>> Mike
>>>
>>>
>>
>>
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>

-- 
Justin M Wozniak