[Swift-devel] RE: [Swift-user] Swift 0.3 released

[Allen, M. David]

I'm not sure I understand...

Why would the CSV mapper actually attempt to fetch a URL that's just a
string inside of a CSV file?  There's no code that instructs it to do
that.  Additionally, I can't find any of those directories that would
be created if this is what it was doing.  If the CSV file contained
other string literals with arbitrary data are you saying that swift
would attempt to treat those literals as if they were files or URLs and
access them too?

Why would wanting just the URL as a text literal be a problem?  If I
used a CSV mapper to refer to only filenames, still it would be passing
those filenames as string literals, right?

If the CSV mapper can only give the location of files, and not string
literals, how does this program even echo a portion of the URL?  If
only given the location of the file, wouldn't you expect it to echo the
path to blogs.csv?

In fact, now I have to admit I'm completely confused as to what the
point of the CSV mapper is.

Was there a bugfix that went in that broke previous behavior?  Earlier
versions worked just like I expected them to, and 0.3 now shows this
problem.  It seems intuitive to me that the csv mapper would allow you
to access string literals inside of a CSV file.  What was the bug that
was fixed?

-- David

-----Original Message-----
From: Ben Clifford [mailto:benc at hawaga.org.uk] 
Sent: Thursday, October 04, 2007 11:14 AM
To: Allen, M. David
Cc: swift-user at ci.uchicago.edu; swift-devel at ci.uchicago.edu
Subject: RE: [Swift-user] Swift 0.3 released


What is most likely happening is that Swift is retrieving that URL and 
placing it into your working directory in a file called 
'bardir/index.rdf'; then passing the path 'bardir/index.rdf' to your
app 
(or rather to echo), essentially saying "I was told to give you a data 
file and I have done that and placed it at bardir/index.rdf".

If you want the URL itself, that's a problem; if you only want to
retrieve 
the content of the URL, that should perhaps be less of a problem.

The CSV mapper (in fact any mapper) can only give the location of data 
files to the swift runtime engine, not pass string literals - though
bugs 
have previously made that work.

Reading literals from CSV is a feature a bunch of people want, though,
so 
I think we should look at implementing something.

On Thu, 4 Oct 2007, Allen, M. David wrote:

> I just dropped 0.3 into place, and tried to re-run a test workflow
that
> I have.  
> 
> For some reason, 0.3 seems to be causing problems with CSV files.
> Attached is a simple swiftscript, the CSV file it requires as input,
> and the output files generated as part of the run.
> 
> The CSV file is simple:
> 
> name,feedURL
> The Foo Blog,http://foo.com/somedir/index.rdf
> The Bar Blog,http://bar.com/bardir/index.rdf
> (...)
> 
> All the swiftscript does is take the feedURL, and then print each one
> out into a different file.  As far as I can tell though, splitting
this
> file by a comma delimiter is causing problems.  Instead of getting
the
> full URL as an argument to "echo", it is only passing
> "somedir/index.rdf", "bardir/index.rdf", and so on.
> 
> One other thing -- my echo statement looks like this:
> 
>    echo @b.feedURL stdout=@filename(headlines);
> 
> >From what I've read, the '@' in front of b.feedURL shouldn't be
> required.  If this isn't present though, the output for every line of
> the csv is the string literal 'true' instead of even a portion of the
> URL.
> 
> Any idea what might be going on here?
> 
> Apologies if I'm making some kind of silly mistake, but I can't find
> any reference to this issue in the CHANGES file, and this has worked
> just fine with previous releases.
> 
> Thanks,
> 
> -- David
>  
> 
> -----Original Message-----
> From: swift-user-bounces at ci.uchicago.edu
> [mailto:swift-user-bounces at ci.uchicago.edu] On Behalf Of Ben Clifford
> Sent: Thursday, October 04, 2007 7:20 AM
> To: swift-user at ci.uchicago.edu; swift-devel at ci.uchicago.edu
> Subject: [Swift-user] Swift 0.3 released
> 
> 
> Swift 0.3 is now available for download from the swift downloads
page,
> http://www.ci.uchicago.edu/swift/downloads/
> 
> Swift 0.3 is a development release intended to distribute new 
> functionality and fixes that have gone into our codebase since v0.2
was
> 
> released in July.
> 
> There are many changes, detailed in the CHANGES.txt file inside the 
> release.
> 
> Some significant changes:
> 
>   * mappers can now map files in remote locations in addition to the
> local 
>     disk (for example, accessed through gridftp or dcache)
> 
>   * PBS direct job submission, for running Swift directly on a PBS 
>     cluster avoiding GRAM.
> 
>   * Changes to logging formats to make mechanical analysis easier.
> 
>   * sequential iteration language construct (for example, for running

>     simulations with each step being a separate job)
> 
> Swift homepage: http://www.ci.uchicago.edu/swift/
> 
> Please download and enjoy, and do not hesitate to post mail to either
> the 
> swift-devel or swift-user list with questions, comments and
complaints.
> 
>