[Swift-user] MODIS demo

Wed May 9 18:02:48 CDT 2012

Also, to get a better error message(I think) set lazy.errors=false in you configuration file. 

On May 9, 2012, at 17:56, David Kelly <davidk at ci.uchicago.edu> wrote:

> Heather,
> 
> Can you please paste the contents of your tc.local file?
> 
> Thanks,
> David
> 
> ----- Original Message -----
>> From: "Heather Stoller" <heather.stoller at gmail.com>
>> To: swift-user at ci.uchicago.edu
>> Cc: "David Kelly" <davidk at ci.uchicago.edu>, "Jonathan Monette" <jonmon at mcs.anl.gov>
>> Sent: Wednesday, May 9, 2012 5:30:40 PM
>> Subject: Re: [Swift-user] MODIS demo
>> Hello Swift Group, coming back to MODIS:
>> 
>> I followed Jonathan's advice to set retries to 0 and looked at the
>> wrapper section in the files in the *.d directory as per David's
>> advice and now get an error right away, as follows:
>> hstoller at ubuntu:~/modis$ ./demo.local urban 4
>> runid=modis-2012.0509.1527-urban-4-10
>> Swift 0.93 swift-r5483 cog-r3339
>> 
>> RunID: 20120509-1527-bsxjpz4g
>> (input): found 0 files
>> Progress: time: Wed, 09 May 2012 15:27:53 -0700
>> Progress: time: Wed, 09 May 2012 15:27:54 -0700 Submitting:7
>> Submitted:1
>> Progress: time: Wed, 09 May 2012 15:27:55 -0700 Active:7 Checking
>> status:1
>> Progress: time: Wed, 09 May 2012 15:27:56 -0700 Stage in:1 Checking
>> status:3 Stage out:1 Finished successfully:4
>> Execution failed:
>> Progress: time: Wed, 09 May 2012 15:27:57 -0700 Checking status:1
>> Failed:4 Finished successfully:4
>> File not found:
>> /home/hstoller/swiftwork/modis-20120509-1527-bsxjpz4g/shared/landuse/h15v05.color.png
>> Progress: time: Wed, 09 May 2012 15:28:00 -0700 Initializing:1
>> Failed:4 Finished successfully:5
>> 
>> I think my question is, where should that shared/landuse directory be
>> getting the missing file?
>> 
>> Thank you for your help!
>> 
>> Sincerely,
>> Heather
>> 
>> 
>> 
>> 
>> On Fri, Apr 13, 2012 at 10:17 PM, Jonathan Monette <
>> jonmon at mcs.anl.gov > wrote:
>> 
>> 
>> I echo David's suggestion but would like to add another. It looks like
>> soft error handling is being used which may not be the best approach
>> when exploring Swift. In the config file you should set the retry
>> count to 0 and set lazy.errors=false. This will cause Swift to fail as
>> soon as the first error is encountered and will provide an error
>> message. This is useful for when you are exploring Swift behavior.
>> 
>> 
>> 
>> On Apr 13, 2012, at 23:17, David Kelly < davidk at ci.uchicago.edu >
>> wrote:
>> 
>>> Heather,
>>> 
>>> You might want to check the path names in the tc.local file. Since
>>> about half the tasks fail, I'm guessing either colormodis or
>>> getlanduse is pointing to the wrong place. You can verify this by
>>> looking at the directory called modis-2012<timestamp>.d. In there is
>>> a list of files that end with -info. Look at the "Wrapper" section
>>> of these files and you should find more information about what is
>>> causing the failures.
>>> 
>>> David
>>> 
>>> ----- Original Message -----
>>>> From: "Heather Stoller" < heather.stoller at gmail.com >
>>>> To: swift-user at ci.uchicago.edu
>>>> Sent: Thursday, April 5, 2012 9:37:33 AM
>>>> Subject: [Swift-user] MODIS demo
>>>> Hello,
>>>> 
>>>> I'm a UC student working with Mike Wilde doing some Swift stuff -
>>>> at
>>>> present, I'm trying to run the demo to see what can be seen. I get:
>>>> 
>>>> ^Cheather at ubuntu:~/modis$ ./demo.local urban 10
>>>> runid=modis-2012.0405.0704-urban-10-10
>>>> Swift 0.93 swift-r5483 cog-r3339
>>>> 
>>>> RunID: 20120405-0704-drh1g0ob
>>>> (input): found 0 files
>>>> Progress: time: Thu, 05 Apr 2012 07:04:48 -0700
>>>> Progress: time: Thu, 05 Apr 2012 07:04:49 -0700 Stage in:19
>>>> Submitting:1
>>>> Progress: time: Thu, 05 Apr 2012 07:04:50 -0700 Stage in:13
>>>> Submitting:1 Active:6
>>>> Progress: time: Thu, 05 Apr 2012 07:04:53 -0700 Stage in:10
>>>> Submitting:2 Submitted:2 Active:6
>>>> Progress: time: Thu, 05 Apr 2012 07:04:54 -0700 Stage in:6
>>>> Submitting:1 Submitted:2 Active:9 Checking status:2
>>>> Progress: time: Thu, 05 Apr 2012 07:04:55 -0700 Stage in:2
>>>> Submitting:2 Submitted:1 Active:9 Checking status:3 Stage out:3
>>>> Progress: time: Thu, 05 Apr 2012 07:04:57 -0700 Submitting:1
>>>> Submitted:2 Active:9 Checking status:1 Stage out:7
>>>> Progress: time: Thu, 05 Apr 2012 07:04:58 -0700 Active:3 Checking
>>>> status:2 Stage out:7 Finished successfully:7 Failed but can retry:1
>>>> Progress: time: Thu, 05 Apr 2012 07:04:59 -0700 Stage in:2
>>>> Submitting:2 Submitted:1 Active:3 Stage out:1 Finished
>>>> successfully:10
>>>> Failed but can retry:2
>>>> Progress: time: Thu, 05 Apr 2012 07:05:00 -0700 Active:10 Stage
>>>> out:1
>>>> Finished successfully:10
>>>> Progress: time: Thu, 05 Apr 2012 07:05:01 -0700 Active:9 Checking
>>>> status:2 Finished successfully:11
>>>> Progress: time: Thu, 05 Apr 2012 07:05:02 -0700 Submitting:1
>>>> Active:3
>>>> Checking status:3 Stage out:2 Finished successfully:11 Failed but
>>>> can
>>>> retry:2
>>>> Progress: time: Thu, 05 Apr 2012 07:05:03 -0700 Stage in:2 Active:6
>>>> Stage out:1 Finished successfully:11 Failed but can retry:2
>>>> Execution failed:
>>>> Progress: time: Thu, 05 Apr 2012 07:05:04 -0700 Active:7 Checking
>>>> status:2 Stage out:1 Failed:1 Finished successfully:11
>>>> Progress: time: Thu, 05 Apr 2012 07:05:05 -0700 Active:3 Checking
>>>> status:1 Stage out:3 Failed:4 Finished successfully:11
>>>> Progress: time: Thu, 05 Apr 2012 07:05:06 -0700 Failed:11 Finished
>>>> successfully:11
>>>> 
>>>> lots of "failed but can retry". Does this look right?
>>>> 
>>>> --