<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">The external mapper mechanism looks to be appropriate. To make sure I understand you, is the idea that I’d replace my current mappers which are<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><single_file_mapper; file=@strcat("a”,”b”,”c”)>;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">with an external mapper<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> < ext; exec="script.sh", arg=@strcat("a”,”b”,”c”)>;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Then script.sh would return the string @strcat("a”,”b”,”c”) if the file with this path does not exist and an empty file it does.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D">Greg Bronevetsky</span><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D">Lawrence Livermore National Lab</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D">(925) 424-5756</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D"><a href="mailto:bronevetsky@llnl.gov">bronevetsky@llnl.gov</a></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D"><a href="http://greg.bronevetsky.com">http://greg.bronevetsky.com</a></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> swift-user-bounces@ci.uchicago.edu [mailto:swift-user-bounces@ci.uchicago.edu]
<b>On Behalf Of </b>Michael Wilde<br>
<b>Sent:</b> Friday, May 16, 2014 10:53 AM<br>
<b>To:</b> swift-user@ci.uchicago.edu<br>
<b>Subject:</b> Re: [Swift-user] Resuming jobs when script has changed<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Greg,<br>
<br>
The Swift resume mechanism can't do this, as its driven by internal variables name within a Swift run, not by external file existence.<br>
<br>
The best way to do what you describe below is to write an external input file mapper (e.g. a simple shell or py script) that returns only the files that still need to be produced or processed. (This is the "ext" mapper:
<a href="http://swift-lang.org/guides/release-0.94/userguide/userguide.html#_external_mapper">
http://swift-lang.org/guides/release-0.94/userguide/userguide.html#_external_mapper</a> )<br>
<br>
You can also call a local app to determine what files need to be produced, then use readData() to read those file names into an array, and array_mapper to map an array of remaining work to do.<br>
<br>
Would one of these approaches meet your needs?<br>
<br>
- Mike<br>
<br>
<o:p></o:p></p>
<div>
<p class="MsoNormal">On 5/16/14, 12:35 PM, Bronevetsky, Greg wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<pre>I have Swift scripts that scan some portion of a design space and after I have scanned a sub-space of all the possibilities I modify the script to target a different, overlapping portion of the design space. However, it seems that when I do this Swift ignores the fact that I've already computed many of the tasks in the new run when I performed the prior run, and re-executes them redundantly. Is there a way for me to avoid such redundant executions? Can the -resume flag be used in this case?<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Greg Bronevetsky<o:p></o:p></pre>
<pre>Lawrence Livermore National Lab<o:p></o:p></pre>
<pre>(925) 424-5756<o:p></o:p></pre>
<pre><a href="mailto:bronevetsky@llnl.gov">bronevetsky@llnl.gov</a><a href="mailto:bronevetsky@llnl.gov"><mailto:bronevetsky@llnl.gov></a><o:p></o:p></pre>
<pre><a href="http://greg.bronevetsky.com">http://greg.bronevetsky.com</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>Swift-user mailing list<o:p></o:p></pre>
<pre><a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><o:p></o:p></pre>
<pre><a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<pre>-- <o:p></o:p></pre>
<pre>Michael Wilde<o:p></o:p></pre>
<pre>Mathematics and Computer Science Computation Institute<o:p></o:p></pre>
<pre>Argonne National Laboratory The University of Chicago<o:p></o:p></pre>
</div>
</body>
</html>