[Swift-devel] wrapper staging

[Michael Wilde]

I'd like to move this item higher up on our priority list, hopefully after /faster gets to a stable resting point.

This is another area where we should consider the needs of Swift/T and Swift/K together.

- Mike

----- Original Message -----
> From: "Michael Wilde" <wilde at mcs.anl.gov>
> To: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Thursday, February 14, 2013 1:35:04 PM
> Subject: Re: [Swift-devel] wrapper staging
> 
> Hi All,
> 
> I'd like to resume the discussion of this feature, in particular how
> it might offer an effective way to a very user-friendly direct IO
> mechanism. Which experience is indicating that we urgently need.
> 
> We need to do a case analysis, including:
> 
> - workdir/jobdir on shared fs
> - jobdir on local fs
> - job dir below swift cwd (something we dont do but makes sense to
> consider)
> 
> How input is handled
> How output is handled
> Who creates necessary dirs
> 
> Also need to consider how various kinds of pathnames and URIs are
> treated:
> 
> - /fully/qualified
> - relative
> - relative/structured
> - various URIs: gsiftp:// etc
> 
> Mike
> 
> ----- Original Message -----
> > From: "Mihael Hategan" <hategan at mcs.anl.gov>
> > To: "Swift Devel" <swift-devel at ci.uchicago.edu>
> > Sent: Saturday, May 26, 2012 6:23:32 PM
> > Subject: [Swift-devel] wrapper staging
> > 
> > There has been some discussion in the past where we agreed that it
> > might
> > be a good idea to try a staging mechanism in which the wrapper (or
> > some
> > other entity on the worker node) does the staging.
> > 
> > I added a skeleton for that in trunk. The relevant files are
> > vdl-int-wrapper-staging.k
> > and _swiftwrap.wrapperstaging.
> > 
> > There is a basic implementation that recognizes "file://" URLs and
> > does
> > a cp for stage-ins and outs.
> > 
> > Mihael
> > 
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> > 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>