[Swift-devel] Provider staging is failing
Mihael Hategan
hategan at mcs.anl.gov
Mon Aug 30 20:18:56 CDT 2010
It's getting some error from the coaster service. I wonder why it isn't
being printed. But the coaster/swift log will probably have it.
Mihael
On Mon, 2010-08-30 at 19:02 -0600, wilde at mcs.anl.gov wrote:
> OK, I see now that it is honoring the workdirectory tag. (I thought that was not used with provider staging, but seems that it is).
>
> WHen mkdir was failing I was getting an error code 524; now Im getting an error code 520 - seems to be failing now in the actual transfer of swiftwrap.
>
> worker log is pasted below.
>
> - Mike
>
> com$ cat worker-0830-560709-000000.log
> 1283216169.574 INFO - 0830-560709-000000 Logging started: Mon Aug 30 19:56:09 2010
> 1283216169.576 INFO - Running on node communicado.ci.uchicago.edu
> 1283216169.576 DEBUG - uri=http://128.135.125.17:50001
> 1283216169.576 DEBUG - scheme=http
> 1283216169.576 DEBUG - host=128.135.125.17
> 1283216169.576 DEBUG - port=50001
> 1283216169.576 DEBUG - blockid=0830-560709-000000
> 1283216169.576 INFO - Connecting (0)...
> 1283216169.576 DEBUG - Trying 128.135.125.17:50001...
> 1283216169.578 INFO - Connected
> 1283216169.578 DEBUG - Replies: {}
> 1283216169.578 DEBUG - OUT: len=8, tag=0, flags=0
> 1283216169.578 DEBUG - OUT: len=18, tag=0, flags=0
> 1283216169.578 DEBUG - OUT: len=0, tag=0, flags=2
> 1283216169.578 DEBUG - done sending frags for 0
> 1283216169.623 DEBUG - Fin flag set
> 1283216169.624 INFO 000000 Registration successful. ID=000000
> 1283216169.624 DEBUG 000000 New request (1)
> 1283216169.624 DEBUG 000000 Fin flag set
> 1283216169.624 DEBUG 000000 Processing request
> 1283216169.625 DEBUG 000000 Cmd is SUBMITJOB
> 1283216169.625 INFO 000000 1283216169479 Job info received (tag=1)
> 1283216169.625 DEBUG 000000 1283216169479 Job check ok (dir: /home/wilde/swiftwork/catsn-20100830-1956-z61dqk05-d-cat-duz412yj)
> 1283216169.625 INFO 000000 1283216169479 Sending submit reply (tag=1)
> 1283216169.625 DEBUG 000000 OUT: len=2, tag=1, flags=3
> 1283216169.625 DEBUG 000000 done sending frags for 1
> 1283216169.625 INFO 000000 1283216169479 Submit reply sent (tag=1)
> 1283216169.625 DEBUG 000000 Replies: {}
> 1283216169.625 DEBUG 000000 OUT: len=9, tag=1, flags=0
> 1283216169.625 DEBUG 000000 OUT: len=13, tag=1, flags=0
> 1283216169.625 DEBUG 000000 OUT: len=2, tag=1, flags=0
> 1283216169.625 DEBUG 000000 OUT: len=1, tag=1, flags=0
> 1283216169.625 DEBUG 000000 OUT: len=15, tag=1, flags=2
> 1283216169.626 DEBUG 000000 done sending frags for 1
> 1283216169.626 DEBUG 000000 1283216169479 Staging in file://localhost//home/wilde/swift/rev/trunk/bin/../libexec/_swiftwrap.staging
> 1283216169.626 DEBUG 000000 1283216169479 src: file://localhost//home/wilde/swift/rev/trunk/bin/../libexec/_swiftwrap.staging, protocol: file, path: localhost//home/wilde/swift/rev/trunk/bin/../libexec/_swiftwrap.staging
> 1283216169.627 DEBUG 000000 Opening /home/wilde/swiftwork/catsn-20100830-1956-z61dqk05-d-cat-duz412yj/_swiftwrap.staging in cwd /
> ...
> 1283216169.628 DEBUG 000000 1283216169479 Opened /home/wilde/swiftwork/catsn-20100830-1956-z61dqk05-d-cat-duz412yj/_swiftwrap.staging
> 1283216169.628 DEBUG 000000 Replies: {1 = ARRAY(0x93ce8f0)}
> 1283216169.628 DEBUG 000000 OUT: len=3, tag=2, flags=0
> 1283216169.628 DEBUG 000000 OUT: len=78, tag=2, flags=0
> 1283216169.628 DEBUG 000000 OUT: len=84, tag=2, flags=2
> 1283216169.628 DEBUG 000000 done sending frags for 2
> 1283216169.647 DEBUG 000000 Fin flag set
> 1283216169.653 DEBUG 000000 1283216169479 getFileCBDataIn jobid: 1283216169479, state: 0, tag: 2, err: 4, fin: 0
> 1283216169.653 DEBUG 000000 Replies: {2 = ARRAY(0x93ce980)}
> 1283216169.653 DEBUG 000000 OUT: len=9, tag=3, flags=0
> 1283216169.653 DEBUG 000000 OUT: len=13, tag=3, flags=0
> 1283216169.653 DEBUG 000000 OUT: len=1, tag=3, flags=0
> 1283216169.653 DEBUG 000000 OUT: len=3, tag=3, flags=0
> 1283216169.653 DEBUG 000000 OUT: len=239, tag=3, flags=2
> 1283216169.653 DEBUG 000000 done sending frags for 3
> 1283216169.653 DEBUG 000000 Fin flag set
> 1283216169.653 DEBUG 000000 1283216169479 getFileCBDataIn jobid: 1283216169479, state: 0, tag: 2, err: 4, fin: 2
> 1283216169.653 DEBUG 000000 Replies: {3 = ARRAY(0x93ce8d0)}
> 1283216169.653 DEBUG 000000 OUT: len=9, tag=4, flags=0
> 1283216169.653 DEBUG 000000 OUT: len=13, tag=4, flags=0
> 1283216169.653 DEBUG 000000 OUT: len=1, tag=4, flags=0
> 1283216169.653 DEBUG 000000 OUT: len=3, tag=4, flags=0
> 1283216169.653 DEBUG 000000 OUT: len=2223, tag=4, flags=2
> 1283216169.653 DEBUG 000000 done sending frags for 4
> 1283216169.698 DEBUG 000000 Fin flag set
> 1283216169.698 DEBUG 000000 Fin flag set
> 1283216169.902 DEBUG 000000 New request (2)
> 1283216169.902 DEBUG 000000 Fin flag set
> 1283216169.902 DEBUG 000000 Processing request
> 1283216169.902 DEBUG 000000 Cmd is SHUTDOWN
> 1283216169.902 DEBUG 000000 Shutdown command received
> 1283216169.902 DEBUG 000000 OUT: len=2, tag=2, flags=3
> 1283216169.902 DEBUG 000000 done sending frags for 2
> com$
>
> ----- wilde at mcs.anl.gov wrote:
>
> > Mihael, Justin,
> >
> > Im trying to use provider-staging for the first time. It seems to be
> > starting the worker in /, and hence staging in fails right away (on
> > _swiftwrap).
> >
> > Where is the worker supposed to start when using provider staging?
> >
> > Ive tried to set the jobdir to /tmp using the <scratch> tag but that
> > doesnt seem to be honored.
> >
> > Ive tried a few different sites configurations; Im running from
> > bridled to communicado using ssh. My most recent is:
> >
> > <pool handle="localhost">
> >
> > <execution provider="coaster" url="communicado.ci.uchicago.edu"
> > jobmanager="ssh:local"/>
> > <!-- <profile namespace="globus"
> > key="workerManager">passive</profile> -->
> >
> > <profile namespace="globus" key="workersPerNode">8</profile>
> > <profile key="jobThrottle" namespace="karajan">.07</profile>
> > <profile namespace="karajan" key="initialScore">10000</profile>
> >
> > <profile namespace="swift" key="stagingMethod">file</profile>
> >
> > <filesystem provider="local" url="none" />
> > <!--
> > <workdirectory>/home/wilde/swiftwork</workdirectory>
> > -->
> > <scratch>/tmp/wilde/scratch</scratch>
> > </pool>
> >
> >
> > - Mike
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>
More information about the Swift-devel
mailing list