[Swift-user] Output Files, ReadData and Order of Execution

[Carolyn Phillips]

Sure

There are a lot of extra stuff running around in the script, fyi

# Types 
type file;
type unlabeleddata;
type labeleddata;
type errorlog;

# Structured Types
type pointfile {
unlabeleddata points;
errorlog error;
}

type simulationfile {
file output;
}

# Apps
app (file o) cat (file i)
{
  cat @i stdout=@o;
}

app (file o) cat2 (file i)
{
  systeminfo stdout=@o;
}

app (pointfile o) generatepoints (file c, labeleddata f,  string mode, int Npoints)
{
  matlab_callgeneratepoints @c @f mode Npoints @o.points @o.error;
}

#app (simulationfile o) runSimulation(string p)
#{
#  launchjob p @o.output;
#}

#Files (using single file mapper)
file config <"designspace.config">;
labeleddata labeledpoints <"emptypoints.dat">;

type pointlog;

# Loop
iterate passes {

    # Generate Parameters
    pointfile np <simple_mapper;prefix="mypoints.",suffix=".dat">;
    np = generatepoints(config,labeledpoints, "uniform", 50);

    int checkforerror = readData(np.error);
    tracef("%s: %i\n", "Generate Parameters Error Value", checkforerror); 
    
    # Issue Jobs
    #simulationfile simfiles[] <simple_mapper;location="/scratch/midway/phillicl/SwiftJobs/",prefix=@strcat("output.",passes,"."),suffix=".job">;
    if(checkforerror==0) {
        unlabeleddata pl = np.points;
    	string parameters[] =readData(pl);
    	foreach p,pindex in parameters {
      		tracef("Launch Job for Parameters: %s\n", p);
                #simfiles[pindex] = runSimulation(p);
    	}
     }

    # Analyze Jobs

    # Generate Prediction



    # creates an array of datafiles named swifttest.<passes>.out to write to
    file out[]<simple_mapper; location=".", prefix=@strcat("swifttest.",passes,"."),suffix=".out">;

    # creates a default of 10 files
    foreach j in [1:@toInt(@arg("n","10"))] {
      file data<"data.txt">;
      out[j] = cat2(data);
    }

    # try writing the iteration to a log file
    file passlog <"passes.log">;
    passlog = writeData(passes);

    # try reading from another log file
    int readpasses = readData(passlog);
    
    # Write to the Output Log 
    tracef("%s: %i\n", "Iteration :", passes);
    tracef("%s: %i\n", "Iteration Read :", readpasses);

#} until (readpasses == 2);  # Determine if Done
} until (passes == 1);  # Determine if Done


On Sep 1, 2012, at 1:57 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:

> Can you post the entire script?
> 
> On Sat, 2012-09-01 at 12:29 -0500, Carolyn Phillips wrote:
>> Yes, I tried that
>> 
>>        unlabeleddata pl = np.points;
>>    	string parameters[] =readData(pl);
>> 
>> 
>> and I got
>> 
>> Execution failed:
>> 	mypoints..dat (No such file or directory)
>> 
>> On Aug 31, 2012, at 8:27 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
>> 
>>> On Fri, 2012-08-31 at 20:11 -0500, Carolyn Phillips wrote:
>>>> How would this line work for what I have below?
>>>> 
>>>>>> string parameters[] =readData(np.points);
>>>> 
>>> 
>>> unlabeleddata tmp = np.points;
>>> string parameters[] = readData(tmp);
>>> 
>>>> 
>>>> 
>>>> 
>>>> On Aug 31, 2012, at 7:49 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
>>>> 
>>>>> Another bug.
>>>>> 
>>>>> I committed a fix. In the mean time, the solution is:
>>>>> 
>>>>> 
>>>>> errorlog fe = np.errorlog;
>>>>> 
>>>>> int error = readData(fe);
>>>>> 
>>>>> On Fri, 2012-08-31 at 19:29 -0500, Carolyn Phillips wrote:
>>>>>> Hi Mihael,
>>>>>> 
>>>>>> the reason I added the "@" was because
>>>>>> 
>>>>>> now this (similar) line
>>>>>> 
>>>>>>  if(checkforerror==0) {
>>>>>>      string parameters[] =readData(np.points);
>>>>>>     }
>>>>>> 
>>>>>> gives me this:
>>>>>> 
>>>>>> Execution failed:
>>>>>> 	mypoints..dat (No such file or directory)
>>>>>> 
>>>>>> as in now its not getting the name of the file correct
>>>>>> 
>>>>>> On Aug 31, 2012, at 7:17 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
>>>>>> 
>>>>>>> @np.error means the file name of np.error which is known statically. So
>>>>>>> readData(@np.error) can run as soon as the script starts.
>>>>>>> 
>>>>>>> You probably want to say readData(np.error).
>>>>>>> 
>>>>>>> Mihael
>>>>>>> 
>>>>>>> 
>>>>>>> On Fri, 2012-08-31 at 18:55 -0500, Carolyn Phillips wrote:
>>>>>>>> So I execute an atomic procedure to generate a datafile, and then next
>>>>>>>> I want to do something with that data file.  However, my program is
>>>>>>>> trying to do something with the datafile before it has been written
>>>>>>>> to.  So something with order of execution is not working.  I think the
>>>>>>>> problem is that the name of my file exists, but the file itself does
>>>>>>>> not yet, but execution proceeds anyway!
>>>>>>>> 
>>>>>>>> Here are my lines
>>>>>>>> 
>>>>>>>> type pointfile {
>>>>>>>> unlabeleddata points;
>>>>>>>> errorlog error;
>>>>>>>> }
>>>>>>>> 
>>>>>>>> # Generate Parameters
>>>>>>>> pointfile np <simple_mapper;prefix="mypoints.",suffix=".dat">; 
>>>>>>>> np = generatepoints(config,labeledpoints, "uniform", 50);
>>>>>>>> 
>>>>>>>> int checkforerror = readData(@np.error);
>>>>>>>> 
>>>>>>>> This gives an error :
>>>>>>>> mypoints.error.dat (No such file or directory)
>>>>>>>> 
>>>>>>>> If I comment out the last line.. all the files show up in the directory.  (e.g. mypoints.points.dat and mypoints.error.dat) )  and if forget to remove the .dat files from a prior run, it also runs fine!
>>>>>>>> 
>>>>>>>> How do you fix a problem like that?
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> Swift-user mailing list
>>>>>>>> Swift-user at ci.uchicago.edu
>>>>>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>> 
> 
>