[Swift-user] MODIS demo
Heather Stoller
heather.stoller at gmail.com
Wed May 9 18:46:24 CDT 2012
Hi David,
Here are the contents of my tc.local file. Jonathan, I have duly checked
that lazy.errors = false. Thank you both!
# site transformation path obsolete fields for
compatibility
localhost echo /bin/echo null null null
localhost cat /bin/cat null null null
localhost ls /bin/ls null null null
localhost grep /bin/grep null null null
localhost sort /bin/sort null null null
localhost paste /bin/paste null null null
localhost pwd /bin/pwd null null null
# For cluster usage
#pbs getlanduse
/home/hstoller/swift/demo/modis/bin/getlanduse.sh null
null null
#pbs analyzelanduse
/home/hstoller/swift/demo/modis/bin/analyzelanduse.sh null null null
#pbs colormodis
/home/hstoller/swift/demo/modis/bin/colormodis.sh null
null null
#pbs assemble
/home/hstoller/swift/demo/modis/bin/assemble.sh null
null null
# For localhost testing
localhost getlanduse /home/hstoller/modis/bin/getlanduse.sh
null null null
localhost analyzelanduse
/home/hstoller/modis/bin/analyzelanduse2.sh null null null
localhost colormodis /home/hstoller/modis/bin/colormodis.sh
null null null
localhost assemble
/home/hstoller/modis/bin/assemble2.sh null null null
localhost markmap
/home/hstoller/modis/bin/markmap.sh null null null
On Wed, May 9, 2012 at 3:56 PM, David Kelly <davidk at ci.uchicago.edu> wrote:
> Heather,
>
> Can you please paste the contents of your tc.local file?
>
> Thanks,
> David
>
> ----- Original Message -----
> > From: "Heather Stoller" <heather.stoller at gmail.com>
> > To: swift-user at ci.uchicago.edu
> > Cc: "David Kelly" <davidk at ci.uchicago.edu>, "Jonathan Monette" <
> jonmon at mcs.anl.gov>
> > Sent: Wednesday, May 9, 2012 5:30:40 PM
> > Subject: Re: [Swift-user] MODIS demo
> > Hello Swift Group, coming back to MODIS:
> >
> > I followed Jonathan's advice to set retries to 0 and looked at the
> > wrapper section in the files in the *.d directory as per David's
> > advice and now get an error right away, as follows:
> > hstoller at ubuntu:~/modis$ ./demo.local urban 4
> > runid=modis-2012.0509.1527-urban-4-10
> > Swift 0.93 swift-r5483 cog-r3339
> >
> > RunID: 20120509-1527-bsxjpz4g
> > (input): found 0 files
> > Progress: time: Wed, 09 May 2012 15:27:53 -0700
> > Progress: time: Wed, 09 May 2012 15:27:54 -0700 Submitting:7
> > Submitted:1
> > Progress: time: Wed, 09 May 2012 15:27:55 -0700 Active:7 Checking
> > status:1
> > Progress: time: Wed, 09 May 2012 15:27:56 -0700 Stage in:1 Checking
> > status:3 Stage out:1 Finished successfully:4
> > Execution failed:
> > Progress: time: Wed, 09 May 2012 15:27:57 -0700 Checking status:1
> > Failed:4 Finished successfully:4
> > File not found:
> >
> /home/hstoller/swiftwork/modis-20120509-1527-bsxjpz4g/shared/landuse/h15v05.color.png
> > Progress: time: Wed, 09 May 2012 15:28:00 -0700 Initializing:1
> > Failed:4 Finished successfully:5
> >
> > I think my question is, where should that shared/landuse directory be
> > getting the missing file?
> >
> > Thank you for your help!
> >
> > Sincerely,
> > Heather
> >
> >
> >
> >
> > On Fri, Apr 13, 2012 at 10:17 PM, Jonathan Monette <
> > jonmon at mcs.anl.gov > wrote:
> >
> >
> > I echo David's suggestion but would like to add another. It looks like
> > soft error handling is being used which may not be the best approach
> > when exploring Swift. In the config file you should set the retry
> > count to 0 and set lazy.errors=false. This will cause Swift to fail as
> > soon as the first error is encountered and will provide an error
> > message. This is useful for when you are exploring Swift behavior.
> >
> >
> >
> > On Apr 13, 2012, at 23:17, David Kelly < davidk at ci.uchicago.edu >
> > wrote:
> >
> > > Heather,
> > >
> > > You might want to check the path names in the tc.local file. Since
> > > about half the tasks fail, I'm guessing either colormodis or
> > > getlanduse is pointing to the wrong place. You can verify this by
> > > looking at the directory called modis-2012<timestamp>.d. In there is
> > > a list of files that end with -info. Look at the "Wrapper" section
> > > of these files and you should find more information about what is
> > > causing the failures.
> > >
> > > David
> > >
> > > ----- Original Message -----
> > >> From: "Heather Stoller" < heather.stoller at gmail.com >
> > >> To: swift-user at ci.uchicago.edu
> > >> Sent: Thursday, April 5, 2012 9:37:33 AM
> > >> Subject: [Swift-user] MODIS demo
> > >> Hello,
> > >>
> > >> I'm a UC student working with Mike Wilde doing some Swift stuff -
> > >> at
> > >> present, I'm trying to run the demo to see what can be seen. I get:
> > >>
> > >> ^Cheather at ubuntu:~/modis$ ./demo.local urban 10
> > >> runid=modis-2012.0405.0704-urban-10-10
> > >> Swift 0.93 swift-r5483 cog-r3339
> > >>
> > >> RunID: 20120405-0704-drh1g0ob
> > >> (input): found 0 files
> > >> Progress: time: Thu, 05 Apr 2012 07:04:48 -0700
> > >> Progress: time: Thu, 05 Apr 2012 07:04:49 -0700 Stage in:19
> > >> Submitting:1
> > >> Progress: time: Thu, 05 Apr 2012 07:04:50 -0700 Stage in:13
> > >> Submitting:1 Active:6
> > >> Progress: time: Thu, 05 Apr 2012 07:04:53 -0700 Stage in:10
> > >> Submitting:2 Submitted:2 Active:6
> > >> Progress: time: Thu, 05 Apr 2012 07:04:54 -0700 Stage in:6
> > >> Submitting:1 Submitted:2 Active:9 Checking status:2
> > >> Progress: time: Thu, 05 Apr 2012 07:04:55 -0700 Stage in:2
> > >> Submitting:2 Submitted:1 Active:9 Checking status:3 Stage out:3
> > >> Progress: time: Thu, 05 Apr 2012 07:04:57 -0700 Submitting:1
> > >> Submitted:2 Active:9 Checking status:1 Stage out:7
> > >> Progress: time: Thu, 05 Apr 2012 07:04:58 -0700 Active:3 Checking
> > >> status:2 Stage out:7 Finished successfully:7 Failed but can retry:1
> > >> Progress: time: Thu, 05 Apr 2012 07:04:59 -0700 Stage in:2
> > >> Submitting:2 Submitted:1 Active:3 Stage out:1 Finished
> > >> successfully:10
> > >> Failed but can retry:2
> > >> Progress: time: Thu, 05 Apr 2012 07:05:00 -0700 Active:10 Stage
> > >> out:1
> > >> Finished successfully:10
> > >> Progress: time: Thu, 05 Apr 2012 07:05:01 -0700 Active:9 Checking
> > >> status:2 Finished successfully:11
> > >> Progress: time: Thu, 05 Apr 2012 07:05:02 -0700 Submitting:1
> > >> Active:3
> > >> Checking status:3 Stage out:2 Finished successfully:11 Failed but
> > >> can
> > >> retry:2
> > >> Progress: time: Thu, 05 Apr 2012 07:05:03 -0700 Stage in:2 Active:6
> > >> Stage out:1 Finished successfully:11 Failed but can retry:2
> > >> Execution failed:
> > >> Progress: time: Thu, 05 Apr 2012 07:05:04 -0700 Active:7 Checking
> > >> status:2 Stage out:1 Failed:1 Finished successfully:11
> > >> Progress: time: Thu, 05 Apr 2012 07:05:05 -0700 Active:3 Checking
> > >> status:1 Stage out:3 Failed:4 Finished successfully:11
> > >> Progress: time: Thu, 05 Apr 2012 07:05:06 -0700 Failed:11 Finished
> > >> successfully:11
> > >>
> > >> lots of "failed but can retry". Does this look right?
> > >>
> > >> --
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20120509/8ef9e582/attachment.html>
More information about the Swift-user
mailing list