[Swift-devel] Problems with file and sfs stagingMethods

[wilde at mcs.anl.gov]

Hi Mihael,

I've reproduced these problems I mentioned a week ago with provider staging. At the moment, only "proxy" mode works for me.

On the CI net in /home/wilde/swift/lab/:

All tests were done using run.sh which sets all the config files. Each test was a single cat of data.txt to outdir/f.0001.out

Each run dir contains a coasters/ dir with the coaster worker log.

run01: this run worked, with stagingMethod proxy

run02: stagingMethod file, files.  The remote side is opening a bad file name (same I I showed you a week ago) with the file:// left in the *middle* of the path name. What gets returned as data to the worker is the Java exception and traceback text.

run03: stagingMethod sfs. What happens here is (a) the sfs:// gets stripped off, but localhost// remains at the front of the source file name.  When I strip that off, the next problem is that the destination directory has not been created. When I create that, the last problem is that the worker is referencing the relative input pathname "data.txt" from a different directory and without the necessary directory prefix.  I explored this in dirs run04-run07 with mods to worker.pl that you can see in the coaster logs, which have been placed in each run dir below coasters/ using a symlink created in run.sh

Can you fix these two stagingMethods?  Or, if your tests are working, please point me to your config files, script, and swift revision.

My swift is in: ~/swift/rev/trunk/bin/swift
 which is a symlink to the dist/ in my ~/swift/src/trunk

Thanks,

Mike