[Swift-devel] Multiple output files

[Mihael Hategan]

Fortunately this would not be something that the users would see.

But if they did, a*b -> c*d seems pretty intuitive to me. Maybe even
more so than 
a(.*)b -> c\1d. It's the implementation of this that is more difficult.

Mihael

On Wed, 2014-03-26 at 13:42 -0500, Michael Wilde wrote:
> Excellent!
> 
> The path-difference challenge sounds like the same things we faced in 
> CDM, revisited.
> 
> I suspect we can find ways to smooth this out in a way that works 
> naturally for users.
> 
> - Mike
> 
> 
> On 3/26/14, 1:24 PM, Mihael Hategan wrote:
> > Almost there.
> >
> > It works for non-provider staging.
> > I'm working on provider staging now. The difficulty is in that providers
> > must now support glob-pattern staging with a twist. The twist is that
> > local and remote path names are different. For example, without glob
> > patterns, a stageout could look like this:
> >
> > __root__/dir/a.txt -> /dir/a.txt
> >
> > With globs, something like this is possible:
> >
> > __root__/dir/a_????_b_????.txt -> /dir/a_????_b_????.txt
> >
> > In this case the code needs to recursively glob things and substitute
> > each glob group in the destination for the respective matching glob in
> > the source.
> >
> > Luckily there are only two providers that support staging at this point:
> > local and coasters. Unfortunately this has to be implemented twice: once
> > in Java for local, and once in Perl for coasters.
> >
> > Mihael
> >
> > On Mon, 2014-03-24 at 08:46 -0700, Mihael Hategan wrote:
> >> An update on this.
> >>
> >> I'm still working on it, but here is the basic idea:
> >>
> >> - non-static mappers now support a method that, given a data type,
> >> returns a list of glob patterns that can be used to search for files
> >> that could be mapped by that mapper. The list (as opposed to one glob
> >> pattern) is necessary because there might be cases when you have:
> >>      type s { file a; file[] b};
> >>      s[] x;
> >>      Then s[] could match either s_????.a or s_????.????
> >> - this (possibly empty) list gets sent to _swiftwrap
> >> - after the job is done, _swiftwrap creates a list of files matching
> >> those patterns
> >> - swift-int copies that list back and the files in it and uses the list
> >> to populate data in a fashion similar to what is done for input
> >> variables
> >>
> >> This is without provider staging.
> >>
> >> For provider staging, providers that support staging need to be modified
> >> to support staging out of files using glob patterns. There might be some
> >> complications there due to the local vs. remote path naming conventions.
> >>
> >> Mihael
> >>
> >> On Sat, 2014-03-22 at 14:08 -0500, Michael Wilde wrote:
> >>> On 3/22/14, 1:38 PM, Mihael Hategan wrote:
> >>>> My opinion is that this problem is NOT a language/mapper issue. It is an
> >>>> issue of implementation: how do you get the information about files to a
> >>>> place where it can be used.
> >>>>
> >>>> So I believe that whether we add a new mapper or make it work with
> >>>> existing mappers, we still need to fix that other more complex problem.
> >>>> This is the reason why I believe we shouldn't add anything to the
> >>>> language.
> >>> Mihael and I discussed this in a chat just now, and I think we are in
> >>> fact *in* sync.
> >>> So he's going to push forward on this.
> >>>
> >>> - Mike
> >>
> >> _______________________________________________
> >> Swift-devel mailing list
> >> Swift-devel at ci.uchicago.edu
> >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >
>