[Swift-devel] readData
Michael Wilde
wilde at mcs.anl.gov
Wed Oct 10 10:34:09 CDT 2007
OK, what seems to be happening is that a simple script using readdata
fails occasionally. Ie, some kind of race.
Heres the test script and data file:
-- script t30.swift:
type circle {
int x;
int y;
}
circle ca;
ca = readdata("d1");
print(ca.x," ",ca.y);
-- and file d1:
x y
1 10
2 20
$
Running this 6 times, I got 4 successes and 2 failures (of which 5 runs
are shown here with a few extraneous intevening commands removed):
...
RunID: 20071010-1024-1h41p37d
1 10
$ swift -sites.file ./sites.xml -tc.file ./tc.data t30.swift
Swift v0.3-dev r1339
RunID: 20071010-1024-kiimrf7f
1 10
$ swift -sites.file ./sites.xml -tc.file ./tc.data t30.swift
Swift v0.3-dev r1339
RunID: 20071010-1024-49v2bnzb
1 10
$ swift -sites.file ./sites.xml -tc.file ./tc.data t30.swift
Swift v0.3-dev r1339
RunID: 20071010-1025-m0qmf0hc
1 10
$ swift -sites.file ./sites.xml -tc.file ./tc.data t30.swift
Swift v0.3-dev r1339
RunID: 20071010-1025-cicpxii8
org.griphyn.vdl.mapping.DataNode with no value at dataset=ca path=.x
org.griphyn.vdl.mapping.DataNode with no value at dataset=ca path=.y
$
On 10/10/07 10:12 AM, Michael Wilde wrote:
> Thats not what I encountered when I tested (which surprised me).
> I will restest and see what confused me (or my code).
>
> - Mike
>
> On 10/10/07 9:31 AM, Mihael Hategan wrote:
>> On Wed, 2007-10-10 at 09:00 -0500, Michael Wilde wrote:
>>> Mihael, all - readdata() works great, and I think gives Andrew
>>> exactly what he asked for.
>>>
>>> I updated the example parameter-sweep loop to use readdata to grab
>>> the multi-column input file.
>>>
>>> One note: as far as I can tell, you use the conventions the data
>>> columns must be exactly 16 characters wide, space separated. Is that
>>> correct? (I assume we'll generalize this time permits).
>>
>> No. They must be horizontal-whitespace separated. The 16 characters wide
>> restriction does not exist. The following is valid:
>> a b c d
>> 1 2 3 4
>> 5 6 7 8
>> 9 10 11 12
>>
>>> Here's the new example, Andrew.
>>>
>>> - Mike
>>>
>>> type file;
>>>
>>> // Simulate encapsulating an app's parameters as a struct
>>>
>>> type params {
>>> int x;
>>> int y;
>>> float r;
>>> boolean b;
>>> string infilename;
>>> string outfilename;
>>> };
>>>
>>> // Simulate an app
>>>
>>> myapp(params p, file infile, file outfile )
>>> {
>>> app {
>>> db "pecho:" p.x p.y p.r p.b p.infilename p.outfilename
>>> @infile @outfile ;
>>> }
>>> }
>>>
>>> // Loop over the parameter array, calling app in parallel
>>>
>>> doall(params plist[])
>>> {
>>> foreach pval,j in plist {
>>>
>>> // convert filename string to mapped file reference
>>> file infile <single_file_mapper;file=pval.infilename>;
>>> file outfile <single_file_mapper;file=pval.outfilename>;
>>>
>>> // Call the application
>>> myapp(pval,infile, outfile);
>>> }
>>> }
>>>
>>> // Main
>>>
>>> params plist[];
>>> plist = readdata("parameters");
>>> doall(plist);
>>>
>>> // Data File "parameters" follows. Data files listed in it must exist.
>>> // each line is greater than 80 bytes and is only wrapped here by email
>>> // (actual files attached)
>>>
>>> x y r b infilename
>>> outfilename
>>> 1 2 1.234 1 inf001.data
>>> outf001.data
>>> 3 4 5.678 0 inf002.data
>>> outf002.data
>>>
>>>
>>>
>>>
>>>
>>> On 10/10/07 5:23 AM, Ben Clifford wrote:
>>>> Mihael added the below language construct to the language the other
>>>> day.
>>>>
>>>> This might be useful where the csv_mapper was being used before to
>>>> read in non-file data.
>>>>
>>>> Its in the SVN.
>>>>
>>>>
>>>> Mihael Hategan wrote:
>>>>
>>>> There's a new function: readData. It's not an @function, so don't
>>>> use it
>>>> that way because it won't work (it needs to know what variable it
>>>> assigns to, so that it knows how to interpret the contents of the
>>>> file).
>>>> It can read primitive things, arrays of primitive things, structs and
>>>> arrays of structs.
>>>> It can either take a file or a string as a parameter, although I
>>>> recommend the former since it can deal with data dependencies.
>>>>
>>>> For example usage, see tests/language-behaviour/readData.swift.
>>>>
>>>> Here's a short preview:
>>>> type circle {
>>>> int x;
>>>> int y;
>>>> float r;
>>>> string name;
>>>> }
>>>>
>>>> circle ca[];
>>>>
>>>> ca = readData("readData.circleArray.in");
>>>>
>>>> readData.circleArray.in:
>>>> x y r name
>>>> 1 1 5 CircleOne
>>>> 2 2 7 CircleTwo
>>>>
>>>> It doesn't deal with spaces in strings in the CSV format for now, but
>>>> it's a start.
>>>>
>>>> Mihael
>>>>
>>> plain text document attachment (t5g.swift)
>>> type file;
>>>
>>> // Simulate encapsulating an app's parameters as a struct
>>>
>>> type params {
>>> int x;
>>> int y;
>>> float r;
>>> boolean b;
>>> string infilename;
>>> string outfilename;
>>> };
>>>
>>> // Simulate an app
>>>
>>> myapp(params p, file infile, file outfile )
>>> { app { db "pecho:" p.x p.y p.r p.b p.infilename
>>> p.outfilename @infile @outfile ;
>>> } }
>>> // Loop over the parameter array, calling app in parallel
>>>
>>> doall(params plist[])
>>> {
>>> foreach pval,j in plist {
>>>
>>> // convert filename string to mapped file reference
>>> file infile <single_file_mapper;file=pval.infilename>;
>>> file outfile <single_file_mapper;file=pval.outfilename>;
>>>
>>> // Call the application
>>> myapp(pval,infile, outfile);
>>> }
>>> }
>>>
>>> // Main
>>>
>>> params plist[];
>>> plist = readdata("parameters");
>>> doall(plist);
>>> plain text document attachment (parameters)
>>> x y r b
>>> infilename outfilename
>>> 1 2 1.234 1
>>> inf001.data outf001.data
>>> 3 4 5.678 0
>>> inf002.data outf002.data
>>> _______________________________________________
>>> Swift-devel mailing list
>>> Swift-devel at ci.uchicago.edu
>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>>
>>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>
>
More information about the Swift-devel
mailing list