[Swift-devel] wrapper staging

[Michael Wilde]

Hi All,

I'd like to resume the discussion of this feature, in particular how it might offer an effective way to a very user-friendly direct IO mechanism. Which experience is indicating that we urgently need.

We need to do a case analysis, including:

- workdir/jobdir on shared fs
- jobdir on local fs
- job dir below swift cwd (something we dont do but makes sense to consider)

How input is handled
How output is handled
Who creates necessary dirs

Also need to consider how various kinds of pathnames and URIs are treated:

- /fully/qualified
- relative
- relative/structured
- various URIs: gsiftp:// etc

Mike

----- Original Message -----
> From: "Mihael Hategan" <hategan at mcs.anl.gov>
> To: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Saturday, May 26, 2012 6:23:32 PM
> Subject: [Swift-devel] wrapper staging
> 
> There has been some discussion in the past where we agreed that it
> might
> be a good idea to try a staging mechanism in which the wrapper (or
> some
> other entity on the worker node) does the staging.
> 
> I added a skeleton for that in trunk. The relevant files are
> vdl-int-wrapper-staging.k
> and _swiftwrap.wrapperstaging.
> 
> There is a basic implementation that recognizes "file://" URLs and
> does
> a cp for stage-ins and outs.
> 
> Mihael
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>