[Swift-devel] remote file/directory stuff (bug 22)

[Yong Zhao]

I like this stated fact:

"Meaning that rather than Swift specifying the remote name for s, a and f,
instead the app block specifies where those three files are."

It pushes application logic further info the app block, which is where it
is supposed to be, and at the workflow level it is much cleaner.

Do you have a nice example about input layout?

Yong.

On Fri, 2 Mar 2007, Ben Clifford wrote:

>
>
> On Fri, 2 Mar 2007, Yong Zhao wrote:
>
> > Can you elaborate on this issue a little bit so that we can make a
> > unanimous decision:
> >
> > 1. what was the problem exactly
>
> Some programs that we run in swift do not use the traditional VDS-like API
> of being told on the commandline the names of the files that they must
> input and output to. Instead, they make up some of the names themselves.
>
> For example, one of Nika's programs has the syntax:
>
>  ./program inputfilename
>
> and places its outputs in inputfile.stuff, inputfile.abc, inputfile.foo
>
> > 2. what are you proposing
>
> To extend the syntax of the app {} block to permit specification of the
> above interface, with a syntax something like:
>
> (stuffoutfile s, abcoutfile a, foooutfile f) myproc(inputfile i)
>   app {
>     program @i;
>     s < @strcat(@inputfile,".stuff")
>     a < @strcat(@inputfile,".abc")
>     f < @strcat(@inputfile,".foo")
>   }
>  }
>
> Meaning that rather than Swift specifying the remote name for s, a and f,
> instead the app block specifies where those three files are.
>
> These will be staged back into the submit-side location defined in the
> existing mappers.
>
> > 3. to what extent does the proposal solve the problem
>
> It should solve Nika's immediate problem, I think.
>
> > 4. what is the implication to the mapping interface
>
> A longer term perspective is that this is the beginning of longer work to
> implement fuller execute-side mappers (which have also been called
> application mappers in some threads).
>
> So it is mapping, but on the execute side. It fits in in a fairly
> straightforward way with mapping on the submit side, which is what we have
> now.
>
> Submit side mapping maps between submit-side data and SwiftScript
> variables/structures, so that the user can arrange his submit-side data in
> a way that he wants (rather than swift compelling it to be in a particular
> format)
>
> Execute side mapping maps between SwiftScript variables and execute side
> data, so that data can be laid out on the execute side in the way that the
> program wants it (rather than swift compelling it to be in a particular
> format)
>
> With the present implementation, this amounts to being able to specify
> different paths and filenames on the submit and execute side for each data
> file.
>
> In the longer term, it might also be useful in defining things like how to
> map data on a submit-side database to some format on the execute side for
> processing. If we have only submit side mappers, then we can map data
> between a submit side database and SwiftScript structures, but not map
> between those structures and the execute side...
>
> --
>