[Swift-devel] Re: [Swift-user] assigning file variables

[Ben Clifford]

On Thu, 26 Feb 2009, Michael Wilde wrote:

> Another seemingly-simple solution is a generalization of simple_mapper that
> allows a more powerful sprintf-like expression to form the file name.

I think the an interesting approach is to look at having a mapper call an 
arbitrary Swift procedure that returns a string.

> I wonder if we could actually move *all* our mappers to "ext" implementations,
> and implement them with shell, perl, awk, etc scripts?
> This would seem to make testing new ideas and enhancements pretty easy (and in
> fact more user extensible), and would have virtually no performance impact on
> most workflows.

I think the ext interface isn't sufficiently expressive for that at the 
moment.

The whole mapper API feels rather messy to me at the moment, and if we're 
doing development there, putting more serious consideration into what it 
should look like seems worthwhile.

> and would have virtually no performance impact on 
> most workflows.                                                               

Do you have numbers to back that up?
                                
> > > - ext-mapper cant pre-map a dynamic output structure with any dimensions
> > > whose
> > > size cant be passed to the mapper (I think?)
> > 
> > yes.
> 
> Can this be lifted, as above?

I think not easily. But see above paragraph about API design.

> > However, in the case of multidimensional arrays, this only happens when the
> > entire top level array has no more assignments at all, not as each subarray
> > happens to become finished.
> 
> OK, so in my case, effectively that restriction remains (although I appreciate
> the explanation below). Note that I'm not complaining about that restriction
> in this example. In my case, moving the inner loop into a separate procedure
> made the code read a bit nicer, in fact. But it led to bumping into the other
> restrictions mentioned.

I think its an undesirable restriction. However...

> > Static analysis of arrays (and even runtime analysis to discover when no
> > more assignments may happen to a particular piece) is extremely hard because
> > you're allowed to construct your own indicies, and you're allowed to use
> > them in a way that isn't single assignment; I think they're a fairly poor
> > structure to have in SwiftScript the way its going.
> 
> By "theyre a fairly poor structure" do you mean user-specified array indices?
> I fear that removing them will take us too  deep into the
> imperative/functional debate, but perhaps we need to keep that discussion
> going.

Yes, I mean user-specified array indices.

> We could discuss if such things could be added as experiments without (yet)
> removing their imperative equivalents. I think that the question of the
> attractiveness of the functional model to distributed and parallel programming
> is a promising research topic. But its not at the top of my priority list for
> the group, which is  usability/productivity, platform support, performance,

I think that its important from a user-interface perspective, not 
particularly from a research perspective.

This style of piecewise assignment to arrays plays merry hell with trying 
to do data-dependent ordering in a way that I think is not easily 
resolvable; and anyone trying to do anything at all interesting with 
arrays gets hit by strange things happening - "I know i've assigned 
everything but somehow the next stage isn't running".

Syntactically this stuff doesn't have to look too different from what it 
looks like now, and we don't have to use particularly scary words like map 
or haskell (although I will point out the doublethink inherent in "we 
don't want functional' vs. 'google map/reduce is god')

> I was wondering about that difference - I thought it was inconsistent usage in
> various documents/tutorials. So we should clarify that terminology in the user
> guide. But better to erase the differnce - all callable things, I feel, should
> have the same name - function or procedure, and they are either built-in, or
> user (or eventually library) defined.

> Whats the semantic difference between the two today?  One distinction I see is
> that built-in things like trace() can take varying arg types, but trace has no
> @ and thus looks more like a user-defined procedure syntactically.

@strcat takes varargs too.

Those differences are ever more insignificant and with time will disappear 
entirely, I think. At the moment, its the return semantics that make them 
different.

Historically, @functions returned in-memory values, and procedures 
operated on files; with @functions being intended for constructing 
parameters in mapping parameters, and procedures being the equivalent of a 
VDL1 procedure invocation.

That distinction has blurred greatly over time.

> I dont feel that we have yet adequately described the model, neither for a CS
> paper *nor* for the programmer.  I think that a good start is to write a data
> model description (in the user guide, in a detailed "skip this on first
> reading" section, that specifies the data model in
> language-reference-specification fashion).

right. I'll see about writing more, as I'm in a writing mood this month ;)

--