[Swift-user] Resuming jobs when script has changed

[Bronevetsky, Greg]

Thanks, Mihael! I've modified my original code based on your suggestions (new code below) but I'm getting the following error:
Could not start execution
        Procedure exists is not defined.

I'm using Swift 0.94.1, which I believe should have the exists function.

Greg Bronevetsky
Lawrence Livermore National Lab
(925) 424-5756
bronevetsky at llnl.gov
http://greg.bronevetsky.com


type file;

app (file outF) writeFile(string message) {
  echo message stdout=@filename(outF);
}

app (file outF) copyFile(file inF) {
  cp @filename(inF) @filename(outF);
}

file data;
file dataOld <single_file_mapper; file="dataF">;
if(!exists(dataOld)) 
{ (data) = writeData("hello"); }
else 
{ data = dataOld; }

file copy;
file copyOld <single_file_mapper; file="copyF">;
if(!exists(copyOld))
{ (copy) = copyFile(data); }
else 
{ copy = copyOld; }

-----Original Message-----
From: Mihael Hategan [mailto:hategan at mcs.anl.gov] 
Sent: Monday, May 19, 2014 3:06 PM
To: Bronevetsky, Greg
Cc: swift-user at ci.uchicago.edu
Subject: Re: [Swift-user] Resuming jobs when script has changed

On Mon, 2014-05-19 at 21:47 +0000, Bronevetsky, Greg wrote:
> I'm using Swift 0.94 so I'll be happy to switch to the built-in exists 
> function. However, I don't think resolves the extra copying issue. The 
> reason why I had to go the way I did is that I had to make sure that 
> regardless of whether a file exists or not, the Swift variable that 
> represents it needs to be the output of some task, otherwise Swift 
> can't track dependencies on it. The only way I could think of to 
> achieve this while, making sure that the file associated with the 
> Swift-visible variable had the right contents, was to copy the data 
> from the original to the Swift-visible file. However Swift then copies 
> the file back to the original since I gave them the same names.

Ok, so you still need the ability to derive things from the existing files that might be different from what was derived before.

I suggest the following:

file newOrOld; // without a mapper

file old <"/old/work/dir/xyz">;

if (exists(old) {
  newOrOld = old; // will not do a copy, but alias newOrOld to old } else {
  file new <"work/dir/xyz">;
  new = makeNew(...);
  newOrOld = new; // will also not do a copy, but the result will be in work/dir/xzy }

// now newOrOld is either /old/work/dir/xyz or work/dir/xyz

Mihael