[Swift-devel] Re: Question of wrapper.sh

[Ioan Raicu]

Ben Clifford wrote:
> On Fri, 7 Mar 2008, Ioan Raicu wrote:
>
>
>   
>>> I notice you're using quite an old version of swift 
>>>       
>> iraicu at login1.surveyor:/home/zzhang/cog/modules/vdsk> svn info
>>     
>
>   
>> Revision: 1673
>>     
>
> Zhao's log reported this:
>
> zhaozhang at viper:~/vdsk-0.3/examples/vdsk> swift first.swift                     
> Swift v0.3 r1319 (modified locally)   
>   
Notice that this Swift is from viper, a machine in RI.  He might be 
playing with this to get used to learning about Swift, but that is not 
being used on the BG/P in any way, as everything we run on the BG/P 
needs to run from the login nodes.  Zhao, can you confirm that the runs 
you are making with Swift on the BG/P are indeed using a recent 
version?  Can you also do an update of Swift on the BG/P to make sure we 
have the latest?
> which is nowhere near the version that you report - something in the 1600s 
> is a reasonable number, but that's not what the log output was. So please 
> clarify which version you actually have poor behaviour for. If it is in 
> the r1600 range, then I'm interested to fix up - but in the r1600s there 
> should be absolutely no cross-node shared log files at all.
>
>   
>> Every job still has a scratch space sandbox, which results in a mkdir, 
>> symbolic linking, and finally a cleanup remove dir.  I think this is the 
>> dir he is referring to.  BTW, if there would be an easy way to eliminate 
>> this entire mkdir part of the wrapper script without breaking anything 
>> in Swift, it would be nice.  The apps we are dealing with don't need the 
>> sandboxing, as we know all input files, and all output files, and we'll 
>> never have input as *.fits that might be ambiguous if we don't sandbox.
>>     
>
> If the jobs touch only those files, I think you can probably eliminate the 
> mkdir, the ln to copy files in and the cp to copy files back out, and run 
> the job in the shared directory directly. However, as each node will then 
> be trying to do stuff to that shared directly, my initial thoughts would 
> be that it wouldn't really change much (perhaps better, perhaps worse).
>   
That is what I thought.  We'll try that and see what we get!  We'll keep 
you posted.

Ioan

-- 
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
http://dev.globus.org/wiki/Incubator/Falkon
http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
===================================================
===================================================


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20080307/61e6c1de/attachment.html>