[Swift-user] MODIS demo

[Heather Stoller]

Hello Swift Group, coming back to MODIS:

I followed Jonathan's advice to set retries to 0 and looked at the wrapper
section in the files in the *.d directory as per David's advice and now get
an error right away, as follows:
hstoller at ubuntu:~/modis$ ./demo.local urban 4
runid=modis-2012.0509.1527-urban-4-10
Swift 0.93 swift-r5483 cog-r3339

RunID: 20120509-1527-bsxjpz4g
 (input): found 0 files
Progress:  time: Wed, 09 May 2012 15:27:53 -0700
Progress:  time: Wed, 09 May 2012 15:27:54 -0700  Submitting:7  Submitted:1
Progress:  time: Wed, 09 May 2012 15:27:55 -0700  Active:7  Checking
status:1
Progress:  time: Wed, 09 May 2012 15:27:56 -0700  Stage in:1  Checking
status:3  Stage out:1  Finished successfully:4
Execution failed:
        Progress:  time: Wed, 09 May 2012 15:27:57 -0700  Checking
status:1  Failed:4  Finished successfully:4
File not found:
/home/hstoller/swiftwork/modis-20120509-1527-bsxjpz4g/shared/landuse/h15v05.color.png
Progress:  time: Wed, 09 May 2012 15:28:00 -0700  Initializing:1  Failed:4
Finished successfully:5

I think my question is, where should that shared/landuse directory be
getting the missing file?

Thank you for your help!

Sincerely,
Heather



On Fri, Apr 13, 2012 at 10:17 PM, Jonathan Monette <jonmon at mcs.anl.gov>wrote:

> I echo David's suggestion but would like to add another. It looks like
> soft error handling is being used which may not be the best approach when
> exploring Swift. In the config file you should set the retry count to 0 and
> set lazy.errors=false. This will cause Swift to fail as soon as the first
> error is encountered and will provide an error message. This is useful for
> when you are exploring Swift behavior.
>
> On Apr 13, 2012, at 23:17, David Kelly <davidk at ci.uchicago.edu> wrote:
>
> > Heather,
> >
> > You might want to check the path names in the tc.local file. Since about
> half the tasks fail, I'm guessing either colormodis or getlanduse is
> pointing to the wrong place. You can verify this by looking at the
> directory called modis-2012<timestamp>.d. In there is a list of files that
> end with -info. Look at the "Wrapper" section of these files and you should
> find more information about what is causing the failures.
> >
> > David
> >
> > ----- Original Message -----
> >> From: "Heather Stoller" <heather.stoller at gmail.com>
> >> To: swift-user at ci.uchicago.edu
> >> Sent: Thursday, April 5, 2012 9:37:33 AM
> >> Subject: [Swift-user] MODIS demo
> >> Hello,
> >>
> >> I'm a UC student working with Mike Wilde doing some Swift stuff - at
> >> present, I'm trying to run the demo to see what can be seen. I get:
> >>
> >> ^Cheather at ubuntu:~/modis$ ./demo.local urban 10
> >> runid=modis-2012.0405.0704-urban-10-10
> >> Swift 0.93 swift-r5483 cog-r3339
> >>
> >> RunID: 20120405-0704-drh1g0ob
> >> (input): found 0 files
> >> Progress: time: Thu, 05 Apr 2012 07:04:48 -0700
> >> Progress: time: Thu, 05 Apr 2012 07:04:49 -0700 Stage in:19
> >> Submitting:1
> >> Progress: time: Thu, 05 Apr 2012 07:04:50 -0700 Stage in:13
> >> Submitting:1 Active:6
> >> Progress: time: Thu, 05 Apr 2012 07:04:53 -0700 Stage in:10
> >> Submitting:2 Submitted:2 Active:6
> >> Progress: time: Thu, 05 Apr 2012 07:04:54 -0700 Stage in:6
> >> Submitting:1 Submitted:2 Active:9 Checking status:2
> >> Progress: time: Thu, 05 Apr 2012 07:04:55 -0700 Stage in:2
> >> Submitting:2 Submitted:1 Active:9 Checking status:3 Stage out:3
> >> Progress: time: Thu, 05 Apr 2012 07:04:57 -0700 Submitting:1
> >> Submitted:2 Active:9 Checking status:1 Stage out:7
> >> Progress: time: Thu, 05 Apr 2012 07:04:58 -0700 Active:3 Checking
> >> status:2 Stage out:7 Finished successfully:7 Failed but can retry:1
> >> Progress: time: Thu, 05 Apr 2012 07:04:59 -0700 Stage in:2
> >> Submitting:2 Submitted:1 Active:3 Stage out:1 Finished successfully:10
> >> Failed but can retry:2
> >> Progress: time: Thu, 05 Apr 2012 07:05:00 -0700 Active:10 Stage out:1
> >> Finished successfully:10
> >> Progress: time: Thu, 05 Apr 2012 07:05:01 -0700 Active:9 Checking
> >> status:2 Finished successfully:11
> >> Progress: time: Thu, 05 Apr 2012 07:05:02 -0700 Submitting:1 Active:3
> >> Checking status:3 Stage out:2 Finished successfully:11 Failed but can
> >> retry:2
> >> Progress: time: Thu, 05 Apr 2012 07:05:03 -0700 Stage in:2 Active:6
> >> Stage out:1 Finished successfully:11 Failed but can retry:2
> >> Execution failed:
> >> Progress: time: Thu, 05 Apr 2012 07:05:04 -0700 Active:7 Checking
> >> status:2 Stage out:1 Failed:1 Finished successfully:11
> >> Progress: time: Thu, 05 Apr 2012 07:05:05 -0700 Active:3 Checking
> >> status:1 Stage out:3 Failed:4 Finished successfully:11
> >> Progress: time: Thu, 05 Apr 2012 07:05:06 -0700 Failed:11 Finished
> >> successfully:11
> >>
> >> lots of "failed but can retry". Does this look right?
> >>
> >> --
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20120509/d88770d6/attachment.html>