Hi David,<br><br>Here are the contents of my tc.local file. Jonathan, I have duly checked that lazy.errors = false. Thank you both!<br><br># site transformation path obsolete fields for compatibility<br>
<br>localhost echo /bin/echo null null null<br>localhost cat /bin/cat null null null<br>localhost ls /bin/ls null null null<br>localhost grep /bin/grep null null null<br>
localhost sort /bin/sort null null null<br>localhost paste /bin/paste null null null<br>localhost pwd /bin/pwd null null null<br><br># For cluster usage<br>
<br>#pbs getlanduse /home/hstoller/swift/demo/modis/bin/getlanduse.sh null null null<br>#pbs analyzelanduse /home/hstoller/swift/demo/modis/bin/analyzelanduse.sh null null null<br>#pbs colormodis /home/hstoller/swift/demo/modis/bin/colormodis.sh null null null<br>
#pbs assemble /home/hstoller/swift/demo/modis/bin/assemble.sh null null null<br><br># For localhost testing<br><br>localhost getlanduse /home/hstoller/modis/bin/getlanduse.sh null null null<br>
localhost analyzelanduse /home/hstoller/modis/bin/analyzelanduse2.sh null null null<br>localhost colormodis /home/hstoller/modis/bin/colormodis.sh null null null<br>localhost assemble /home/hstoller/modis/bin/assemble2.sh null null null<br>
localhost markmap /home/hstoller/modis/bin/markmap.sh null null null<br><br><br><div class="gmail_quote">On Wed, May 9, 2012 at 3:56 PM, David Kelly <span dir="ltr"><<a href="mailto:davidk@ci.uchicago.edu" target="_blank">davidk@ci.uchicago.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Heather,<br>
<br>
Can you please paste the contents of your tc.local file?<br>
<br>
Thanks,<br>
<div class="im HOEnZb">David<br>
<br>
----- Original Message -----<br>
> From: "Heather Stoller" <<a href="mailto:heather.stoller@gmail.com">heather.stoller@gmail.com</a>><br>
> To: <a href="mailto:swift-user@ci.uchicago.edu">swift-user@ci.uchicago.edu</a><br>
</div><div class="HOEnZb"><div class="h5">> Cc: "David Kelly" <<a href="mailto:davidk@ci.uchicago.edu">davidk@ci.uchicago.edu</a>>, "Jonathan Monette" <<a href="mailto:jonmon@mcs.anl.gov">jonmon@mcs.anl.gov</a>><br>
> Sent: Wednesday, May 9, 2012 5:30:40 PM<br>
> Subject: Re: [Swift-user] MODIS demo<br>
> Hello Swift Group, coming back to MODIS:<br>
><br>
> I followed Jonathan's advice to set retries to 0 and looked at the<br>
> wrapper section in the files in the *.d directory as per David's<br>
> advice and now get an error right away, as follows:<br>
> hstoller@ubuntu:~/modis$ ./demo.local urban 4<br>
> runid=modis-2012.0509.1527-urban-4-10<br>
> Swift 0.93 swift-r5483 cog-r3339<br>
><br>
> RunID: 20120509-1527-bsxjpz4g<br>
> (input): found 0 files<br>
> Progress: time: Wed, 09 May 2012 15:27:53 -0700<br>
> Progress: time: Wed, 09 May 2012 15:27:54 -0700 Submitting:7<br>
> Submitted:1<br>
> Progress: time: Wed, 09 May 2012 15:27:55 -0700 Active:7 Checking<br>
> status:1<br>
> Progress: time: Wed, 09 May 2012 15:27:56 -0700 Stage in:1 Checking<br>
> status:3 Stage out:1 Finished successfully:4<br>
> Execution failed:<br>
> Progress: time: Wed, 09 May 2012 15:27:57 -0700 Checking status:1<br>
> Failed:4 Finished successfully:4<br>
> File not found:<br>
> /home/hstoller/swiftwork/modis-20120509-1527-bsxjpz4g/shared/landuse/h15v05.color.png<br>
> Progress: time: Wed, 09 May 2012 15:28:00 -0700 Initializing:1<br>
> Failed:4 Finished successfully:5<br>
><br>
> I think my question is, where should that shared/landuse directory be<br>
> getting the missing file?<br>
><br>
> Thank you for your help!<br>
><br>
> Sincerely,<br>
> Heather<br>
><br>
><br>
><br>
><br>
> On Fri, Apr 13, 2012 at 10:17 PM, Jonathan Monette <<br>
> <a href="mailto:jonmon@mcs.anl.gov">jonmon@mcs.anl.gov</a> > wrote:<br>
><br>
><br>
> I echo David's suggestion but would like to add another. It looks like<br>
> soft error handling is being used which may not be the best approach<br>
> when exploring Swift. In the config file you should set the retry<br>
> count to 0 and set lazy.errors=false. This will cause Swift to fail as<br>
> soon as the first error is encountered and will provide an error<br>
> message. This is useful for when you are exploring Swift behavior.<br>
><br>
><br>
><br>
> On Apr 13, 2012, at 23:17, David Kelly < <a href="mailto:davidk@ci.uchicago.edu">davidk@ci.uchicago.edu</a> ><br>
> wrote:<br>
><br>
> > Heather,<br>
> ><br>
> > You might want to check the path names in the tc.local file. Since<br>
> > about half the tasks fail, I'm guessing either colormodis or<br>
> > getlanduse is pointing to the wrong place. You can verify this by<br>
> > looking at the directory called modis-2012<timestamp>.d. In there is<br>
> > a list of files that end with -info. Look at the "Wrapper" section<br>
> > of these files and you should find more information about what is<br>
> > causing the failures.<br>
> ><br>
> > David<br>
> ><br>
> > ----- Original Message -----<br>
> >> From: "Heather Stoller" < <a href="mailto:heather.stoller@gmail.com">heather.stoller@gmail.com</a> ><br>
> >> To: <a href="mailto:swift-user@ci.uchicago.edu">swift-user@ci.uchicago.edu</a><br>
> >> Sent: Thursday, April 5, 2012 9:37:33 AM<br>
> >> Subject: [Swift-user] MODIS demo<br>
> >> Hello,<br>
> >><br>
> >> I'm a UC student working with Mike Wilde doing some Swift stuff -<br>
> >> at<br>
> >> present, I'm trying to run the demo to see what can be seen. I get:<br>
> >><br>
> >> ^Cheather@ubuntu:~/modis$ ./demo.local urban 10<br>
> >> runid=modis-2012.0405.0704-urban-10-10<br>
> >> Swift 0.93 swift-r5483 cog-r3339<br>
> >><br>
> >> RunID: 20120405-0704-drh1g0ob<br>
> >> (input): found 0 files<br>
> >> Progress: time: Thu, 05 Apr 2012 07:04:48 -0700<br>
> >> Progress: time: Thu, 05 Apr 2012 07:04:49 -0700 Stage in:19<br>
> >> Submitting:1<br>
> >> Progress: time: Thu, 05 Apr 2012 07:04:50 -0700 Stage in:13<br>
> >> Submitting:1 Active:6<br>
> >> Progress: time: Thu, 05 Apr 2012 07:04:53 -0700 Stage in:10<br>
> >> Submitting:2 Submitted:2 Active:6<br>
> >> Progress: time: Thu, 05 Apr 2012 07:04:54 -0700 Stage in:6<br>
> >> Submitting:1 Submitted:2 Active:9 Checking status:2<br>
> >> Progress: time: Thu, 05 Apr 2012 07:04:55 -0700 Stage in:2<br>
> >> Submitting:2 Submitted:1 Active:9 Checking status:3 Stage out:3<br>
> >> Progress: time: Thu, 05 Apr 2012 07:04:57 -0700 Submitting:1<br>
> >> Submitted:2 Active:9 Checking status:1 Stage out:7<br>
> >> Progress: time: Thu, 05 Apr 2012 07:04:58 -0700 Active:3 Checking<br>
> >> status:2 Stage out:7 Finished successfully:7 Failed but can retry:1<br>
> >> Progress: time: Thu, 05 Apr 2012 07:04:59 -0700 Stage in:2<br>
> >> Submitting:2 Submitted:1 Active:3 Stage out:1 Finished<br>
> >> successfully:10<br>
> >> Failed but can retry:2<br>
> >> Progress: time: Thu, 05 Apr 2012 07:05:00 -0700 Active:10 Stage<br>
> >> out:1<br>
> >> Finished successfully:10<br>
> >> Progress: time: Thu, 05 Apr 2012 07:05:01 -0700 Active:9 Checking<br>
> >> status:2 Finished successfully:11<br>
> >> Progress: time: Thu, 05 Apr 2012 07:05:02 -0700 Submitting:1<br>
> >> Active:3<br>
> >> Checking status:3 Stage out:2 Finished successfully:11 Failed but<br>
> >> can<br>
> >> retry:2<br>
> >> Progress: time: Thu, 05 Apr 2012 07:05:03 -0700 Stage in:2 Active:6<br>
> >> Stage out:1 Finished successfully:11 Failed but can retry:2<br>
> >> Execution failed:<br>
> >> Progress: time: Thu, 05 Apr 2012 07:05:04 -0700 Active:7 Checking<br>
> >> status:2 Stage out:1 Failed:1 Finished successfully:11<br>
> >> Progress: time: Thu, 05 Apr 2012 07:05:05 -0700 Active:3 Checking<br>
> >> status:1 Stage out:3 Failed:4 Finished successfully:11<br>
> >> Progress: time: Thu, 05 Apr 2012 07:05:06 -0700 Failed:11 Finished<br>
> >> successfully:11<br>
> >><br>
> >> lots of "failed but can retry". Does this look right?<br>
> >><br>
> >> --<br>
</div></div></blockquote></div><br><br>