[Swift-devel] RE: [Swift-user] Swift 0.3 released

[Ben Clifford]

On Thu, 4 Oct 2007, Allen, M. David wrote:

> I'm not sure I understand...
> 
> Why would the CSV mapper actually attempt to fetch a URL that's just a
> string inside of a CSV file?  There's no code that instructs it to do
> that.

That's what mappers do. They tell swift where data files are, so that it 
then can move those data files around and put them where your job will 
run.

The 'code that instructs it to do that' is the mapper syntax with <angle 
brackets like this> - that syntax says "this mapper will tell you where to 
find the filenames for this variable".

The CSV mapper is a way to say "I want to have an array of structures of 
files in SwiftScript, and here are the filenames that you should use for 
the elements of that array of structures."

> Additionally, I can't find any of those directories that would be 
> created if this is what it was doing.

You shouldn't see them after the workflow has run as they should tidied 
up.

Try running   ls -l @url   instead of echo @url

and see what you get.

> If the CSV file contained other string literals with arbitrary data are 
> you saying that swift would attempt to treat those literals as if they 
> were files or URLs and access them too?

yes, it should be.

I'm slightly surprised you don't get problems with the first column.

> Why would wanting just the URL as a text literal be a problem?  If I
> used a CSV mapper to refer to only filenames, still it would be passing
> those filenames as string literals, right?

> If the CSV mapper can only give the location of files, and not string
> literals, how does this program even echo a portion of the URL?  If
> only given the location of the file, wouldn't you expect it to echo the
> path to blogs.csv?

The filenames that get passed are filenames for the application program to 
use on the execute side of things. Swift is insistent that it will place 
the input files in the correct place and then tell you where it has put 
them.

When you pass in http://my.url.com/foo/bar as an input file, Swift will 
place that file in your job's run directory as foo/bar and then tell your 
app where it had put it.

(conversely, it might decide to put it in a file called XXX; in which case 
your application would receive XXX as an input string; and your app should 
find that the local file XXX contained the content retrieved from 
http://my.url.com)

> In fact, now I have to admit I'm completely confused as to what the 
> point of the CSV mapper is.

Not what you want it for. Anything to do with mappers is for instructing 
swift on which files your application wants moved around for it 
automatically.

> Was there a bugfix that went in that broke previous behavior?  Earlier
> versions worked just like I expected them to, and 0.3 now shows this
> problem.  It seems intuitive to me that the csv mapper would allow you
> to access string literals inside of a CSV file.  What was the bug that
> was fixed?

Its an incorrect intuition, probably caused by the fact that we don't 
document mappers well enough / in the right way.

The bug was that URLs didn't work properly for what they were intended.

--