[Swift-devel] Swift staging method names

[Mihael Hategan]

Right. I don't think the names will be able to fully convey what they
do, and we'll need documentation for that, but I think we can do better
at making the names more representative of what they... represent.

So far we have the following things:
- swift does staging in swift-int using some filesystem provider
- swift tells the provider that it needs files staged:
  - from the client side
  - from the head node
  - from some directory on the compute node
 (as an aside, I'm not sure these should necessarily be schemes in the
URL, but we could instead have file://localhost/foo.txt,
file://#headnode/foo.txt, file://#computenode/foo.txt).
- a special version of _swiftwrap does the staging
- swift passes direct paths to the app as much as possible and tells the
provider that it needs some files staged for the rest
- *MAYBE* swift passes direct paths to the app as much as possible and
(does staging of|tells _swiftwrap to stage) the rest

Mihael

On Mon, 2014-08-11 at 16:09 -0500, Tim Armstrong wrote:
> I agree, I don't think the names give a good idea of what they're about.  I
> also feel like it's difficult to explain some of these without diagrams.
> 
> - Tim
> 
> 
> On Mon, Aug 11, 2014 at 3:34 PM, Yadu Nand <yadudoc1729 at gmail.com> wrote:
> 
> > Hi Everyone,
> >
> > In the current conventions for staging, we are calling what used to be
> > proxy staging method as local, which I think is counter-intuitive, when
> > local staging is actually staging to remote sites.
> >
> > Staging method "swift" does not give any idea about what it actually does
> > either.
> >
> > I'm pasting the staging methods table from the documentation below.
> >
> > Staging MethodDescription
> >
> > swift
> >
> > This method instructs Swift to use a filesystem provider to direct all
> > necessary staging operations from the Swift client-side to the cluster head
> > node. If this method is used, the workDirectory must point to a head node
> > path that is on a shared file system accessible by the compute nodes.
> >
> > wrapper
> >
> > File staging is done by the Swift application wrapper
> >
> > local
> >
> > Used to indicate that files should be staged in/out from/to the site on
> > which Swift is running. In the case of Swift Coasters, the system proxies
> > the tranfers between client side and compute nodes through the Coaster
> > Service.
> >
> > service-local
> >
> > This method instructs the execution mechanism provider to stage input and
> > output files from the remote site where the execution service is located.
> > For example, if a Coaster Service is started on the login node of a
> > cluster, the Coaster Service will perform the staging from a file system on
> > the login node to the compute node and back.
> >
> > shared-fs
> >
> > This method is used by Coasters to implement a simple staging mechanism in
> > which files are accessed using a shared filesystem that is accessible by
> > compute nodes
> >
> >
> > ​Thanks,​
> > Yadu Nand B
> >
> >
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >
> >
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel