[Swift-user] Does SWIFT_JOBDIR_PATH work for output?

[Michael Wilde]

On 3/16/09 4:07 AM, Ben Clifford wrote:
> On Sun, 15 Mar 2009, Michael Wilde wrote:
> 
>> Does the env var SWIFT_JOBDIR_PATH set the jobdir for each job in a script to
>> this value, and use it for output as well as input?  Or is it only used for
>> input?
> 
> It sets the root under which the per-job working directories exist, that 
> is otherwise the run shared directory with /jobs/ on the end

OK, thats exactly what I as hoping it did.

>> But as my jobs run, I can see that they are writing their log data, line by
>> line, to the output directories of the shared workdirectory. Thats the
>> overhead I was hoping to avoid with SWIFT_JOBDIR_PATH.
> 
> You mean the wrapper logs are being written to the wrapper log directory 
> info/ ?

No, this is an application log file - just one of the app's outputs. I 
could see these ".log" files being created in the workdir as soon as the 
script started, and was able to tail -f these files to see the app's 
progress.

So if this feature works as above, then something is wrong. Possibly I 
didnt specify it correctly, or its broken. I'll need to investigate.


> That will happen. Zhao has hacked things in the past to not store the log. 
> Another thing that can be done is to create the log in the job directory 
> and copy it at the end. However, in certain failure modes you then won't 
> get any log data at all.
> 
> I can implement an option for that.
>