[Swift-user] Question about programs, wrappers and exceptions

[Lorenzo Pesce]

Thanks a lot for your reply.

On Jan 22, 2013, at 4:04 PM, Michael Wilde wrote:

> 
>> From: "Lorenzo Pesce" <lpesce at uchicago.edu>
>> 
>> My current issue has to deal with returning return values from
>> programs, contained into wrappers and to a degree how to
>> transparently manage exceptions and return values within swift.
> 
> Programs (i.e., app() functions) can only return files.
> 
> If you want to return values (eg a set of scalars) from an app() back into the calling Swift script, your app should place them into a file, and you can then use readData() or readData2() to read these values into Swift scalars, arrays, or structures.
> 
> The best way to do this is to define a wrapping function written in Swift to do the readData() and make it transparent to higher-level callers.

I see, so if I need to perform a step that produces an unknown number of files with unknown patters (they depend upon the content of the file and we don't know what they are till we are done processing it) , I put that in a file and then have
http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html#_readdata
Then I can use the output to create a mapper and send out the new files for the following step.

I assume that swift can take into account the implied dependencies created by the readdata step. Right?

If so, it makes perfect sense, thanks.

> Regarding errors and exceptions: Swift does not have an exception facility.  An app() is considered by the Swift runtime to have failed when it returns a non-zero exit code. When it does so, Swift considers the app() call to have failed and will then apply the retry and recovery logic specific by the properties execution.retries and lazy.errors.  Swift will also consider the app to have failed if the files that it expects the app to create (per its parameters and their mappings) do not exist.

That is good enough, I can hack the wrappers to make them return the exit code I want depending upon how it ran.
I can trap the signals inside of the apps as you told me previously and that should take care of it. I think.

> 
>> I would also to understand in more details about the mechanics of
>> coasters. It is not entirely clear to me how sometimes they seem to
>> relinquish control of nodes even when not all the calculations are
>> completed. It seems like a new batch of jobs is submitted as a new
>> calculation even when it is not strictly necessary (or doesn't seem
>> to be).
> 
> A coaster worker will exit when no ready app() jobs submitted by the Swift script will fit into the remaining wall time of the worker.  Could that be happening in the cases of this you have seen?

That is what I thought, but the coasters were scheduled to run for 10 days and they were exiting a lot before that time (and being resubmitted). We noticed because some other submissions slipped in before the new ones.
I haven't gone carefully over those logs, so it is possible that I am missing something here.

Thanks again,

Lorenzo

> 
> - Mike