[Swift-devel] Multiple output files

[Mihael Hategan]

Sorry, I'm not sure I'm 100% with you on this.

My opinion is that this problem is NOT a language/mapper issue. It is an
issue of implementation: how do you get the information about files to a
place where it can be used.

So I believe that whether we add a new mapper or make it work with
existing mappers, we still need to fix that other more complex problem.
This is the reason why I believe we shouldn't add anything to the
language.

Anyway, give me a couple of days. I cleaned up the stagein/stageout code
a bit yesterday. It was messy for no good reason that I could see.
Closing the remote loop on the dynamic mappers is next.

Mihael


On Sat, 2014-03-22 at 11:31 -0500, Michael Wilde wrote:
> This is now filed as enhancement bug 1225 and assigned to you, Mihael.  
> The description has been revised as you suggested, to propose that the 
> feature be first done using simple_mapper rather than a new mapper.
> 
> - Mike
> 
> On 3/21/14, 8:39 AM, Michael Wilde wrote:
> > Mihael, All,
> >
> > I'd like to propose a Swift/K feature to provide a reasonable solution 
> > to this very common need for an app to return a dynamically determined 
> > set of files.
> >
> > file dynarry[ ] <runtime; prefix="myoutput.", suffix=".out", 
> > indexes="int">;
> >
> > dynarry = myApp(myArgs);
> >
> > The "runtime" mapper should initially have the same arguments and 
> > semantics (roughly) as simple_mapper, except for two new arguments:
> >
> >  "indexes" which determines how the matched file names will be indexed 
> > in the returned array
> > "int" | "string" | "sequential"
> > sequential: return the matched files as consecutive integer indices 
> > starting with 0
> > int: expect the filename component between prefix and suffix to be 
> > convertible to an integer, and use that as the index
> > eg myfile.012.out and myfile.204.out will return an array with the 
> > mapped files at indices 12 and 204.
> > string: similar to int but return a string-indexed associative array.
> > "sequential" is simplest and should be the default.
> >
> > "paths" which determines if the match names will be absolute or 
> > relative to the job dir
> > paths="relative" | "absolute"
> > (may not be needed if this can be determined uniquely based on the 
> > location argument.
> >
> > swiftwrap will allow array variables mapped in this manner to have any 
> > number of files, including zero. I.e. "runtime-mapped" files should 
> > not be listed in the expected output list for an app invocation. Its 
> > up to the users app to ensure that some files match the pattern. An 
> > additional arg could set e.g. minfiles and/or maxfiles, in which case 
> > the wrapper code needs to validate the count of files matched and 
> > returned, but not their exact names.
> >
> > We can call this mapper "experimental" until we validate its usability 
> > and suitability as a permanent feature. But as we hope to revise the 
> > entire mapper family and semantics, in a sense all mappers are subject 
> > to change.
> >
> > Mihael, is the definition sound, and how long would it take you to 
> > develop it?
> >
> > Thanks,
> >
> > - Mike
> >
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel