[Swift-devel] [Bug 76] disable intermediate stageout of data

[Yong Zhao]

My original thinking about value/filename was that we don't distinguish
those at the logical level, essentially they could all just be values.
Then when we need to call mapper functions (getFilename, for instance), we
interprete the values differently. So in the case of getFilename, we can
interprete the value either as
1) the filename itself, returning the value directly
2) writing the value into a file, and returning an automatically generated
filename.
3) some other possibilities, e.g. a directory of files.

The current DSHandle interface does allow nested trees, so a mapper
could return a dshandle tree as the implementation currently stands.

Yong.


On Thu, 5 Jul 2007, Ben Clifford wrote:

>
> so I was thinking the other day while poking through code.
>
> 'data' in SwiftScript terms is mostly represented by DSHandle objects.
>
> such objects (which can have one of several implementing classes, and
> potentially more in future) have a number of properties, such as:
>
>  .  value - what the 'value' is, for adding to other values, using @strcat
> on, performing array/member access using [] and .
>
>  . submit-side location - what is extracted with @filename and used when
> that 'data' is passed to an application rather than being operated on by
> submit-side functions.
>
> Neither of these are compulsory (and I think in practice at the moment it
> works out that you either have a filename or a value and never
> meaningfully both).
>
> So a different model of mapping (which might work better when we want data
> that doesn't necessarily exist as discrete files or as in-memory values -
> the two examples that I've seen talked about are 'data from an sql
> database' and 'constants in a csv file') might be that mappers generate
> DSHandle trees (specifically a mapper generates a DSHandle, which might
> have descendants). Those DSHandles might have values, might have
> filenames, might have other attributes, might have ongoing annotation
> (which could include keeping track of where within-this-run copies have
> been made).
>
> --
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>