[Swift-user] Re: [falkon-user] 1) disable retry mechanism and 2) continue on failure?

[Michael Wilde]

Ben, thanks - these patches sound great. Can the use of /tmp be 
controlled by a property, ideally on a per-application basis in tc.data, 
and these changes committed to svn?

Seems like wrapper-tmp-log-locally could be done for all apps as the 
default, and only turned off for certain debugging scenarios.

Can you do application caching as well, in a general manner?

We'll measure over the next few days and report back.

- Mike


On 3/31/08 2:34 AM, Ben Clifford wrote:
> On Mon, 31 Mar 2008, Ben Clifford wrote:
> 
>> This temporary directory handling is pretty ugly - it should be a couple 
>> lines change to wrapper.sh to get similar functionality using the existing 
>> swift temporary direcotry handling - change the path to /tmp and use cp 
>> instead of ln -s. That way you can take advantage of Swift's existing 
>> unique job IDs and error handling too.
> 
> Attached are three patches that will apply against svn r1775:
> 
> The first puts temporary directories in /tmp rather than on shared fs.
> http://www.ci.uchicago.edu/~benc/tmp/wrapper-tmp-dirs-on-tmp
> 
> The second copies the application file to the worker in each job execution 
> (though doesn't do any worker-node caching of such between jobs)
> http://www.ci.uchicago.edu/~benc/tmp/wrapper-tmp-dirs-mv-executable
> 
> The third creates the worker node log on /tmp and copies it at the end.
> http://www.ci.uchicago.edu/~benc/tmp/wrapper-tmp-log-locally
> 
> The three modify all wrapper.sh and should be applied in the above order.
> 
> With the first two patches, the timestamps in the usual info logs will 
> provide information about how long the copies take, in the same way that 
> they usually indicate times for other execution stages.
>