[Swift-user] Resuming jobs when script has changed

[Bronevetsky, Greg]

Thanks Mihael. I just tried using your operations in my test code (below). It seems to detect whether the files do/don't exist and seems to try to generate the files but they never appear in my directory. During a fresh run in an empty directory I get the following output:

Swift 0.94.1 swift-r7114 cog-r3803

RunID: 20140519-2230-4lyyh8me
Progress:  time: Mon, 19 May 2014 22:30:03 -0700
Progress:  time: Mon, 19 May 2014 22:30:04 -0700  Stage in:1  Submitting:1
Progress:  time: Mon, 19 May 2014 22:30:09 -0700  Active:1  Checking status:1
path=g/g15/bronevet/apps/swift-0.94.1/examples/test/work/dataF, exists=false
Generating file://localhost/_concurrent/data-5fbfabc7-8f5d-4240-82b1-4c7ecfdc33aa-path=g/g15/bronevet/apps/swift-0.94.1/examples/test/work/copyF, exists=false
Generating file://localhost/_concurrent/copy-44d394f6-53f3-4f5b-9ec9-fa2c19f5fcdf- from file://localhost/_concurrent/data-5fbfabc7-8f5d-4240-82b1-4c7ecfdc33aa-Final status: Mon, 19 May 2014 22:30:09 -0700  Finished successfully:3

Greg Bronevetsky
Lawrence Livermore National Lab
(925) 424-5756
bronevetsky at llnl.gov
http://greg.bronevetsky.com

import "common";
type file;

string WORK_PATH="/g/g15/bronevet/apps/swift-0.94.1/examples/test/work";

app (file outF) writeFile(string message) {
  echo message stdout=@filename(outF);
}

app (file outF) copyFile(file inF) {
  cp @filename(inF) @filename(outF);
}

file data;
file dataOld <single_file_mapper; file="dataF">;
if(!exists(@strcat(WORK_PATH, "/dataF"))) { 
  tracef("Generating %M\n", data);
  (data) = writeData("hello"); 
} else 
{ data = dataOld; }

file copy; 
file copyOld <single_file_mapper; file="copyF">;
if(!exists(@strcat(WORK_PATH, "/copyF"))) { 
  tracef("Generating %M from %M\n", copy, data);
  (copy) = copyFile(data);
} else 
{ copy = copyOld; }


-----Original Message-----
From: Mihael Hategan [mailto:hategan at mcs.anl.gov] 
Sent: Monday, May 19, 2014 4:25 PM
To: Bronevetsky, Greg
Cc: swift-user at ci.uchicago.edu
Subject: Re: [Swift-user] Resuming jobs when script has changed

Here's the swift part. It will only work with absolute paths. It can probably be modified to only work with relative paths.

type file;

app (file outf) existsApp(string path) {
  existsApp path stdout=@filename(outf); }

(boolean result) exists(string path) {
  result = readData(existsApp(path));
}

file there <"/bin/bash">;
file notthere <"/bin/notthere">;

tracef("Y %b\n", exists(@filename(there))); tracef("N %b\n", exists(@filename(notthere)));

And existsApp:
#!/bin/bash

# add leading / since @filename removes it # this will not work with swift > 0.94 if [ -f "/$1" ]; then
        echo "true"
else
        echo "false"
fi



On Mon, 2014-05-19 at 15:28 -0700, Mihael Hategan wrote:
> On Mon, 2014-05-19 at 22:21 +0000, Bronevetsky, Greg wrote:
> > Thanks, Mihael! I've modified my original code based on your suggestions (new code below) but I'm getting the following error:
> > Could not start execution
> >         Procedure exists is not defined.
> 
> Sorry. That's where your implementation of exists for absolute path 
> names goes. I'll send you the relevant code shortly. I'm working on 
> the array splitting issue now.
> 
> Mihael
> 
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user