[Swift-devel] Swift Question/Request

[Tiberiu Stef-Praun]

On 4/3/07, Mihael Hategan <hategan at mcs.anl.gov> wrote:
> How exactly does seq make Swift know how many outputs processStageOne()
> has?
>
> On Tue, 2007-04-03 at 10:25 -0500, Tiberiu Stef-Praun wrote:
> > Sometimes i find myself in the position of wanting to do one step of
> > the swift script only after the previous phase has finished. The case
> > is the following (not really swift script):
> >
> > seq{
> > file stageOneOutputs[] = processStageOne();
> > file combine = combine(stageOneOutputs);
> > }
> >
> > In my case the problem seems to be the fact that swift cannot dream up
> > how many stageOneOutput files will exist (unless we extend the
> > language -probably the input mapper which I use in this case- to
> > support this feature), so it could decide to start the combine stage
> > prematurely.
>
> I seriously doubt that. It may not start it at all, but prematurely, no.
>
> > Supposedly one solution is to create a dummy output file at the end of
> > stage one, and pass it tot stage two.
> >
> > Any suggestions ?
>
> This kind of problem can only be addressed in two ways:
> 1. You know how many files you have when you write your program (and use
> a fixed array mapper).

What I had in mind was something along these lines:
I know how many output files to expect, but It might be ugly
specifying their names, so I was thinking of adding a regexp limit to
the filesys mapper: In my case I know I am expecting 36 output files,
so don't start the second stage untill the filesys mapper can produce
those many files.
But for now I will probably go with the fixed array mapper.


> 2. Swift can figure out how many files are out there. This is something
> that, by virtue of the number of times we've seen this cropping and the
> fact that there is no hack that can solve it, should be on our list of
> things (if not already there).
>
> >
> > I'm thinking that having an example in the user guide would help others as well.
> > Maybe we should have a "Patterns in Swift" document.
> >
> > Thanks
> > Tibi
> >
>
>


-- 
Tiberiu (Tibi) Stef-Praun, PhD
Research Staff, Computation Institute
5640 S. Ellis Ave, #405
University of Chicago
http://www-unix.mcs.anl.gov/~tiberius/