[Swift-devel] Re: moldyn intermediate files vs gridftp URLs

[Mihael Hategan]

On Mon, 2007-09-24 at 10:49 -0500, Ioan Raicu wrote:
> Are URLs supported by the current Swift?  That is what I am doing with
> the data caching in Falkon, but to the compute node (to local disk).
> If Swift were to do this to the compute site (i.e. to GPFS), then you
> also need a way to keep track of what files are at what site, so then
> other sites can then retrieve the files directly from peer sites,
> rather than going back to the submit host (what would currently happen
> on a cross site run).  Is this mapping handled in Swift at the moment?
> And if so, does it use RLS, or something similar, or an in-house
> solution?  I just haven't heard of this functionality within the
> context of Swift so far, and am curious what is there and what is not.

Nothing is there but the ability to map to arbitrary URLs. Which means
files can go somewhere else than the local machine, but it is explicit
(i.e. specified in the script).

> 
> Ioan
> 
> Mihael Hategan wrote: 
> > On Mon, 2007-09-24 at 12:15 +0000, Ben Clifford wrote:
> >   
> > > In the MolDyn app, the swift runtime is 'customised' (cough) so that it 
> > > doesn't stage certain intermediate files from the single execution site 
> > > where those files are created and consumed to the submit host.
> > > 
> > > If those intermediate files were mapped to remote files on the execution 
> > > site (eg as gridftp URLs), what would happen?
> > > 
> > > Would swift try to stage back those files to their submit-side locations 
> > > which would actually be on the compute site?
> > >     
> > 
> > Yes, I think so.
> > 
> >   
> > > Would that help this workflow?
> > > 
> > >     
> > 
> > It might.
> > 
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > 
> >   
> 
> -- 
> ============================================
> Ioan Raicu
> Ph.D. Student
> ============================================
> Distributed Systems Laboratory
> Computer Science Department
> University of Chicago
> 1100 E. 58th Street, Ryerson Hall
> Chicago, IL 60637
> ============================================
> Email: iraicu at cs.uchicago.edu
> Web:   http://www.cs.uchicago.edu/~iraicu
>        http://dsl.cs.uchicago.edu/
> ============================================
> ============================================