[Swift-user] Output Files, ReadData and Order of Execution

[Mihael Hategan]

The error comes from int checkforerror = readData(np.error);

You have to use the workaround for both.

On Sat, 2012-09-01 at 15:23 -0500, Carolyn Phillips wrote:
> Sure
> 
> There are a lot of extra stuff running around in the script, fyi
> 
> # Types 
> type file;
> type unlabeleddata;
> type labeleddata;
> type errorlog;
> 
> # Structured Types
> type pointfile {
> unlabeleddata points;
> errorlog error;
> }
> 
> type simulationfile {
> file output;
> }
> 
> # Apps
> app (file o) cat (file i)
> {
>   cat @i stdout=@o;
> }
> 
> app (file o) cat2 (file i)
> {
>   systeminfo stdout=@o;
> }
> 
> app (pointfile o) generatepoints (file c, labeleddata f,  string mode, int Npoints)
> {
>   matlab_callgeneratepoints @c @f mode Npoints @o.points @o.error;
> }
> 
> #app (simulationfile o) runSimulation(string p)
> #{
> #  launchjob p @o.output;
> #}
> 
> #Files (using single file mapper)
> file config <"designspace.config">;
> labeleddata labeledpoints <"emptypoints.dat">;
> 
> type pointlog;
> 
> # Loop
> iterate passes {
> 
>     # Generate Parameters
>     pointfile np <simple_mapper;prefix="mypoints.",suffix=".dat">;
>     np = generatepoints(config,labeledpoints, "uniform", 50);
> 
>     int checkforerror = readData(np.error);
>     tracef("%s: %i\n", "Generate Parameters Error Value", checkforerror); 
>     
>     # Issue Jobs
>     #simulationfile simfiles[] <simple_mapper;location="/scratch/midway/phillicl/SwiftJobs/",prefix=@strcat("output.",passes,"."),suffix=".job">;
>     if(checkforerror==0) {
>         unlabeleddata pl = np.points;
>     	string parameters[] =readData(pl);
>     	foreach p,pindex in parameters {
>       		tracef("Launch Job for Parameters: %s\n", p);
>                 #simfiles[pindex] = runSimulation(p);
>     	}
>      }
> 
>     # Analyze Jobs
> 
>     # Generate Prediction
> 
> 
> 
>     # creates an array of datafiles named swifttest.<passes>.out to write to
>     file out[]<simple_mapper; location=".", prefix=@strcat("swifttest.",passes,"."),suffix=".out">;
> 
>     # creates a default of 10 files
>     foreach j in [1:@toInt(@arg("n","10"))] {
>       file data<"data.txt">;
>       out[j] = cat2(data);
>     }
> 
>     # try writing the iteration to a log file
>     file passlog <"passes.log">;
>     passlog = writeData(passes);
> 
>     # try reading from another log file
>     int readpasses = readData(passlog);
>     
>     # Write to the Output Log 
>     tracef("%s: %i\n", "Iteration :", passes);
>     tracef("%s: %i\n", "Iteration Read :", readpasses);
> 
> #} until (readpasses == 2);  # Determine if Done
> } until (passes == 1);  # Determine if Done
> 
> 
> On Sep 1, 2012, at 1:57 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
> 
> > Can you post the entire script?
> > 
> > On Sat, 2012-09-01 at 12:29 -0500, Carolyn Phillips wrote:
> >> Yes, I tried that
> >> 
> >>        unlabeleddata pl = np.points;
> >>    	string parameters[] =readData(pl);
> >> 
> >> 
> >> and I got
> >> 
> >> Execution failed:
> >> 	mypoints..dat (No such file or directory)
> >> 
> >> On Aug 31, 2012, at 8:27 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
> >> 
> >>> On Fri, 2012-08-31 at 20:11 -0500, Carolyn Phillips wrote:
> >>>> How would this line work for what I have below?
> >>>> 
> >>>>>> string parameters[] =readData(np.points);
> >>>> 
> >>> 
> >>> unlabeleddata tmp = np.points;
> >>> string parameters[] = readData(tmp);
> >>> 
> >>>> 
> >>>> 
> >>>> 
> >>>> On Aug 31, 2012, at 7:49 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
> >>>> 
> >>>>> Another bug.
> >>>>> 
> >>>>> I committed a fix. In the mean time, the solution is:
> >>>>> 
> >>>>> 
> >>>>> errorlog fe = np.errorlog;
> >>>>> 
> >>>>> int error = readData(fe);
> >>>>> 
> >>>>> On Fri, 2012-08-31 at 19:29 -0500, Carolyn Phillips wrote:
> >>>>>> Hi Mihael,
> >>>>>> 
> >>>>>> the reason I added the "@" was because
> >>>>>> 
> >>>>>> now this (similar) line
> >>>>>> 
> >>>>>>  if(checkforerror==0) {
> >>>>>>      string parameters[] =readData(np.points);
> >>>>>>     }
> >>>>>> 
> >>>>>> gives me this:
> >>>>>> 
> >>>>>> Execution failed:
> >>>>>> 	mypoints..dat (No such file or directory)
> >>>>>> 
> >>>>>> as in now its not getting the name of the file correct
> >>>>>> 
> >>>>>> On Aug 31, 2012, at 7:17 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
> >>>>>> 
> >>>>>>> @np.error means the file name of np.error which is known statically. So
> >>>>>>> readData(@np.error) can run as soon as the script starts.
> >>>>>>> 
> >>>>>>> You probably want to say readData(np.error).
> >>>>>>> 
> >>>>>>> Mihael
> >>>>>>> 
> >>>>>>> 
> >>>>>>> On Fri, 2012-08-31 at 18:55 -0500, Carolyn Phillips wrote:
> >>>>>>>> So I execute an atomic procedure to generate a datafile, and then next
> >>>>>>>> I want to do something with that data file.  However, my program is
> >>>>>>>> trying to do something with the datafile before it has been written
> >>>>>>>> to.  So something with order of execution is not working.  I think the
> >>>>>>>> problem is that the name of my file exists, but the file itself does
> >>>>>>>> not yet, but execution proceeds anyway!
> >>>>>>>> 
> >>>>>>>> Here are my lines
> >>>>>>>> 
> >>>>>>>> type pointfile {
> >>>>>>>> unlabeleddata points;
> >>>>>>>> errorlog error;
> >>>>>>>> }
> >>>>>>>> 
> >>>>>>>> # Generate Parameters
> >>>>>>>> pointfile np <simple_mapper;prefix="mypoints.",suffix=".dat">; 
> >>>>>>>> np = generatepoints(config,labeledpoints, "uniform", 50);
> >>>>>>>> 
> >>>>>>>> int checkforerror = readData(@np.error);
> >>>>>>>> 
> >>>>>>>> This gives an error :
> >>>>>>>> mypoints.error.dat (No such file or directory)
> >>>>>>>> 
> >>>>>>>> If I comment out the last line.. all the files show up in the directory.  (e.g. mypoints.points.dat and mypoints.error.dat) )  and if forget to remove the .dat files from a prior run, it also runs fine!
> >>>>>>>> 
> >>>>>>>> How do you fix a problem like that?
> >>>>>>>> 
> >>>>>>>> _______________________________________________
> >>>>>>>> Swift-user mailing list
> >>>>>>>> Swift-user at ci.uchicago.edu
> >>>>>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> >>>>>>> 
> >>>>>>> 
> >>>>>> 
> >>>>> 
> >>>>> 
> >>>> 
> >>> 
> >>> 
> >> 
> > 
> > 
>