<div dir="ltr">Hi Yadu,<div><br></div><div>I have tested the new posted tutorial on s3fs and it works. I can run my wordCount on s3fs now. </div><div><br></div><div>But there's a little problem.</div><div>I put all script and input file in /s3/wordCount-s3fs, and set the work directory to /s3/wordCount-s3fs. Then I found that I can't set relative path of files in my swift script, I have to set absolute path for all files.</div><div><br></div><div>*If I use absolute path to all input and output files, everything works fine.</div><div><br></div><div>If I did:</div><div><br></div><div><p style="margin:0px;font-size:11px;font-family:Menlo">file infile[] <filesys_mapper;pattern="input/split-*", location=".">;</p><p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo">It will have a exception:</p><p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo">----------------------------------------</p><p style="margin:0px;font-size:11px;font-family:Menlo">Execution failed:</p><p style="margin:0px;font-size:11px;font-family:Menlo">Exception in python:</p><p style="margin:0px;font-size:11px;font-family:Menlo"> Arguments: [/s3/wordCount-s3fs/./wordCount.py, input/split-0006]</p><p style="margin:0px;font-size:11px;font-family:Menlo"> Host: cloud-static</p><p style="margin:0px;font-size:11px;font-family:Menlo"> Directory: wordCount-run001/jobs/g/python-g9k2d6zl</p><p style="margin:0px;font-size:11px;font-family:Menlo"> exception @ swift-int-staging.k, line: 167</p><p style="margin:0px;font-size:11px;font-family:Menlo">Caused by: Application /usr/bin/python failed with an exit code of 1</p><p style="margin:0px;font-size:11px;font-family:Menlo;min-height:13px"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo">------- Application STDERR --------</p><p style="margin:0px;font-size:11px;font-family:Menlo">wordcount error: file name "./input/split-0006" not exist.</p><p style="margin:0px;font-size:11px;font-family:Menlo">Traceback (most recent call last):</p><p style="margin:0px;font-size:11px;font-family:Menlo"> File "/s3/wordCount-s3fs/./wordCount.py", line 12, in <module></p><p style="margin:0px;font-size:11px;font-family:Menlo"> f=open(fileName, 'r')</p><p style="margin:0px;font-size:11px;font-family:Menlo">
</p><p style="margin:0px;font-size:11px;font-family:Menlo">IOError: [Errno 2] No such file or directory: './input/split-0006'</p><div><p style="margin:0px;font-size:11px;font-family:Menlo">-----------------------------------</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> exception @ swift-int-staging.k, line: 163</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Caused by: Block task failed: Connection to worker lost</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">java.io.EOFException</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> at org.globus.cog.coaster.channels.AbstractStreamCoasterChannel.readFromChannel(AbstractStreamCoasterChannel.java:253)</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> at org.globus.cog.coaster.channels.AbstractStreamCoasterChannel.stepNIO(AbstractStreamCoasterChannel.java:186)</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> at org.globus.cog.coaster.channels.NIOMultiplexer.loop(NIOMultiplexer.java:116)</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> at org.globus.cog.coaster.channels.NIOMultiplexer.run(NIOMultiplexer.java:75)</p>
<p style="margin:0px;font-size:11px;font-family:Menlo;min-height:13px"><br></p>
<p style="margin:0px;font-size:11px;font-family:Menlo;min-height:13px"><br></p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> k:assign @ swift.k, line: 171</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Caused by: Exception in python:</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> Arguments: [/s3/wordCount-s3fs/./wordCount.py, input/split-0006]</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> Host: cloud-static</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> Directory: wordCount-run001/jobs/g/python-g9k2d6zl</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> exception @ swift-int-staging.k, line: 167</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Caused by: Application /usr/bin/python failed with an exit code of 1</p>
<p style="margin:0px;font-size:11px;font-family:Menlo;min-height:13px"><br></p>
<p style="margin:0px;font-size:11px;font-family:Menlo">------- Application STDERR --------</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">wordcount error: file name "./input/split-0006" not exist.</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Traceback (most recent call last):</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> File "/s3/wordCount-s3fs/./wordCount.py", line 12, in <module></p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> f=open(fileName, 'r')</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">IOError: [Errno 2] No such file or directory: './input/split-0006'</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">-----------------------------------</p></div><p style="margin:0px;font-size:11px;font-family:Menlo;min-height:13px"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo"> exception @ swift-int-staging.k, line: 163</p><p style="margin:0px;font-size:11px;font-family:Menlo">Caused by: Block task failed: Connection to worker lost</p><p style="margin:0px;font-size:11px;font-family:Menlo">java.io.EOFException</p><p style="margin:0px;font-size:11px;font-family:Menlo"> at org.globus.cog.coaster.channels.AbstractStreamCoasterChannel.readFromChannel(AbstractStreamCoasterChannel.java:253)</p><p style="margin:0px;font-size:11px;font-family:Menlo"> at org.globus.cog.coaster.channels.AbstractStreamCoasterChannel.stepNIO(AbstractStreamCoasterChannel.java:186)</p><p style="margin:0px;font-size:11px;font-family:Menlo"> at org.globus.cog.coaster.channels.NIOMultiplexer.loop(NIOMultiplexer.java:116)</p><p style="margin:0px;font-size:11px;font-family:Menlo">
</p><p style="margin:0px;font-size:11px;font-family:Menlo"> at org.globus.cog.coaster.channels.NIOMultiplexer.run(NIOMultiplexer.java:75)</p><p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo">------------------------------------------------</p><p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo">Which basically says ./input/split-000 not exist. I'm sure /s3/wordCount-s3fs/input/split-000 do exist. And I don't want to enter absolute path to all files if possible.</p><p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo">Any idea about how to deal with it?</p><p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo">Thanks,</p><p style="margin:0px;font-size:11px;font-family:Menlo">Jiada Tu</p><p style="margin:0px;font-size:11px;font-family:Menlo"><br></p></div></div>