[Swift-devel] Re: Question of wrapper.sh

[Ben Clifford]

On Fri, 7 Mar 2008, Ioan Raicu wrote:


> > I notice you're using quite an old version of swift 
> iraicu at login1.surveyor:/home/zzhang/cog/modules/vdsk> svn info

> Revision: 1673

Zhao's log reported this:

zhaozhang at viper:~/vdsk-0.3/examples/vdsk> swift first.swift                     
Swift v0.3 r1319 (modified locally)   

which is nowhere near the version that you report - something in the 1600s 
is a reasonable number, but that's not what the log output was. So please 
clarify which version you actually have poor behaviour for. If it is in 
the r1600 range, then I'm interested to fix up - but in the r1600s there 
should be absolutely no cross-node shared log files at all.

> Every job still has a scratch space sandbox, which results in a mkdir, 
> symbolic linking, and finally a cleanup remove dir.  I think this is the 
> dir he is referring to.  BTW, if there would be an easy way to eliminate 
> this entire mkdir part of the wrapper script without breaking anything 
> in Swift, it would be nice.  The apps we are dealing with don't need the 
> sandboxing, as we know all input files, and all output files, and we'll 
> never have input as *.fits that might be ambiguous if we don't sandbox.

If the jobs touch only those files, I think you can probably eliminate the 
mkdir, the ln to copy files in and the cp to copy files back out, and run 
the job in the shared directory directly. However, as each node will then 
be trying to do stuff to that shared directly, my initial thoughts would 
be that it wouldn't really change much (perhaps better, perhaps worse).

--