[Swift-devel] a couple of file staging management ideas in the next 3 months

[Ioan Raicu]

Ben and I are meeting on Thursday to hopefully flesh out some of these 
concrete things. We'll send out a follow-up message then.

Ioan

Mihael Hategan wrote:
> I'm a bit fuzzy on the kind of generic changes that would facilitate the
> move to Falkon. I'd rather like to see the concrete things that are
> needed.
>
> On Tue, 2008-12-02 at 19:55 +0000, Ben Clifford wrote:
>   
>> Here are a couple of ideas that might be useful to implement over the next 
>> three months.
>>
>> This is intended to be read in context of two previous notes documenting 
>> what I think is the abstract model that SwiftScript provides, and the 
>> present Swift implementation of that.
>>
>> These notes refer to changes that could reasonably be made to the 
>> production Swift in the next ~3 months, to both facilitate the 
>> experimental side of things (i.e. Falkon) and to broaden the range of 
>> sites which Swift could be used on in production (i.e. gLite).
>>
>> Specifically, they are not intended discuss approaches to architecting 
>> Falkon, other than to discuss how approaches in Falkon might be coherently 
>> interfaced to Swift.
>>
>> Two distinct approaches (which are complimentary and could both 
>> implemented) seem feasible to implement in the next two months and 
>> interesting at the Swift layer (in as much as they facilitate ongoing 
>> work):
>>
>>  * change wrapper.sh to access the shared filesystem in various
>>    selectable/configurable/pluggable ways; keep everything else the same.
>>
>>    So, this changes the requirement that wrapper.sh have posix access to
>>    the site cache into a requirement that there is command-line scriptable
>>    access to the site cache.
>>
>>    This would facilitate gLite's model of no posix access to the site
>>    storage system without radically changing architecture. Some OSG users 
>>    have suggested that this is desirable on OSG.
>>
>>  * Fiddle with the abstraction at the execute2 layer
>>
>>    execute2 is a routine in the Swift library that selects a site,
>>    and then causes stagein to site shared directory, execution on the
>>    remote site, and stageout from that site to occur.
>>
>>    This layer could be adjusted so the sequence: stagein/execute/stageout
>>    is pluggable. By default the exist implementation would be used; but it
>>    would allow other code to be plugged in that could replace both the
>>    existing client site stagein/execute/stageout code and the remote
>>    wrapper.sh code. This would facilitate running in environments where
>>    the same underlying layer wants to do both data management and
>>    execution management (such as falkon)
>>
>>    In the falkon case, what is now provider-deef would plug in at this
>>    level and offload all staging responsibility from core Swift.
>>
>>     
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>
>   

-- 
===================================================
Ioan Raicu
Ph.D. Candidate
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
http://dev.globus.org/wiki/Incubator/Falkon
http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
===================================================
===================================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20081202/a5b22e0f/attachment.html>