<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 6, 2015 at 1:13 PM, Michael Wilde <span dir="ltr"><<a href="mailto:wilde@anl.gov" target="_blank">wilde@anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span class="">
<br>
<div>On 1/6/15 1:04 PM, Ketan Maheshwari
wrote:<br>
</div>
</span><span class=""><blockquote type="cite">
<div dir="ltr">Hi Mike,
<div><br>
</div>
<div>The app is creating the file but the way the app is
invoked, the filename does not appear in the command-line. It
creates the file at the top level, so just afile.txt. </div>
<div><br>
</div>
<div>However, since there are many calls to the app, to avoid
this file being overwritten, I need to put this file into a
separate directory which is why I am using directory outdirN. </div>
<div><br>
</div>
<div>I was thinking if it is possible for Swift runtime to find
from the app definition that the file _appout is expected
output similar to stdout and move the file to the outdirN
(again similar to stdout).</div>
</div>
</blockquote></span>
I think what you're asking for is the ability to declare for each
app that its temporary "sandbox" working dir gets saved below the
current working dir in which you're running the swift command. And,
further, to be able to name that directory from the source script.<span class=""><br>
<br>
That *might* be a reasonable feature, but will need more
discussion. I suggest a bugzilla ticket to capture this as a
proposed enhancement. It seems however that the current way of doing
this, explicitly, is simple and sufficient for now.<br></span></div></blockquote><div><br></div><div>I think it is simpler. Consider the following app:</div><div><br></div><div>(file _appout) app someapp (string _appin){</div><div><br></div><div> appcmd _appin;</div><div>}</div><div><br></div><div>file appout <"appout.txt">;</div><div><br></div><div>(appout) = someapp ("hello");</div><div> </div><div><br></div><div>Swift will bring the "appout.txt" from workdir to the current dir if it is produced by app (even without being specified in commandline). </div><div><br></div><div>So, I am facing a special case of this pattern where the appout file is mapped into a directory instead of at the toplevel:</div><div><br></div><div>file appout <single_file_mapper; file=strcat("outdir", i, "/appout.txt")>;</div><div><br></div><div>Currently, this pattern gets honored for stdout/stderr files, ie. Swift creates the outdirN and puts the stdout/stderr files into it. </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><span class="">
<br>
- Mike<br>
</span><blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>Thanks,</div>
<div>Ketan</div>
</div><div><div class="h5">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jan 6, 2015 at 12:53 PM,
Michael Wilde <span dir="ltr"><<a href="mailto:wilde@anl.gov" target="_blank">wilde@anl.gov</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> Is your app
touch_app( ) correctly creating output files of the form
outdirN/afile.txt?<br>
<br>
From the error message, I suspect that it is not.<br>
<br>
Your app is declared as:<br>
<br>
app (file _stdout, file _stderr, file _appout)
touch_app(string _instr){<br>
t _instr stdout=@_stdout stderr=@_stderr;<br>
}<br>
<br>
<br>
You need to pass filename(_appout) to the app, via its
command line, so that it knows the correct output filename
to create. Then you need to ensure that the app does
indeed create that file.<br>
<br>
- Mike
<div>
<div><br>
<br>
<div>On 1/6/15 11:23 AM, Ketan Maheshwari wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Trying trunk for this pattern.
<div><br>
</div>
<div>A toy application invoked over a foreach loop
that creates an output file, an stdout and an
stderr files. </div>
<div><br>
</div>
<div>The files are mapped into an output directory
named with the loop index as suffix so that the
files do not get overwritten:</div>
<div><br>
</div>
<div>
<div>foreach i in [0:9]{</div>
<div><br>
</div>
<div> file out<single_file_mapper;
file=strcat("outdir", i, "/std.out")>;</div>
<div> file err<single_file_mapper;
file=strcat("outdir", i, "/std.err")>;</div>
<div> file appout<single_file_mapper;
file=strcat("outdir", i, "/afile.txt")>;</div>
<div><br>
</div>
<div> (out, err, appout) = touch_app("Hello");</div>
<div>}</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>The stdout and stderr files correctly ends up
in their respective directories but the app
generated file does not. </div>
<div><br>
</div>
<div>I see following error message:</div>
<div><br>
</div>
<div>
<div>Execution failed:</div>
<div>Exception in t:</div>
<div> Arguments: [Hello]</div>
<div> Host: edison1</div>
<div> Directory:
touchafile-run001/jobs/t/t-ffv33r2m</div>
<div><span style="white-space:pre-wrap"> </span>exception
@ swift-int-staging.k, line: 165</div>
<div>Caused by: The following output files were
not created by the application:
outdir4/afile.txt</div>
</div>
<div><br>
</div>
<div>Any suggestions for fixing this?</div>
<div><br>
</div>
<div>Attached is the test directory with sources
and executable with rundir.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Ketan</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Dec 22, 2014 at
5:14 PM, Mihael Hategan <span dir="ltr"><<a href="mailto:hategan@mcs.anl.gov" target="_blank">hategan@mcs.anl.gov</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>Hi,<br>
<br>
I don't think 0.95 supports dynamic arrays
output from apps. You will<br>
need trunk/0.96 for that.<br>
<br>
Mihael<br>
<br>
On Mon, 2014-12-22 at 15:37 -0600, Ketan
Maheshwari wrote:<br>
</span>
<div>
<div>> Hi Mihael,<br>
><br>
> This is with Swift 0.95.<br>
><br>
> Thanks,<br>
> Ketan<br>
><br>
> On Sun, Dec 21, 2014 at 2:40 PM,
Hategan-Marandiuc, Philip M. <<br>
> <a href="mailto:hategan@mcs.anl.gov" target="_blank">hategan@mcs.anl.gov</a>>
wrote:<br>
><br>
> > Hi Ketan,<br>
> ><br>
> > Sorry for the delay. Is this
trunk or 0.95?<br>
> ><br>
> > Mihael<br>
> ><br>
> > On Wed, 2014-12-17 at 14:26
-0600, Ketan Maheshwari wrote:<br>
> > > Hi,<br>
> > ><br>
> > > I am dealing with a
workflow pattern where an app expects
multiple output<br>
> > > files with a pattern.<br>
> > ><br>
> > > The app signature is:<br>
> > ><br>
> > > app (file[] _wrfout, file
_out, file _err) wrf_app (file _wrf_in,
file[]<br>
> > > _tbl, file[] _ozone, ...)<br>
> > > {<br>
> > > wrf stdout=@_out
stderr=@_err;<br>
> > > }<br>
> > ><br>
> > > The _wrfout files are the
app result files which follows a pattern:<br>
> > wrfout_*<br>
> > ><br>
> > > So, I am invoking the
application in a foreach loop as:<br>
> > ><br>
> > > foreach i in [0:2]{<br>
> > > file[]
wrfout<simple_mapper;
location=strcat("outdir",i),<br>
> > > pattern="wrfout_*">;<br>
> > > file
wrfstdout<single_file_mapper;
file=strcat("outdir", i,<br>
> > "/std.out")>;<br>
> > > file
wrfstderr<single_file_mapper;
file=strcat("outdir", i,<br>
> > "/std.err")>;<br>
> > ><br>
> > > (wrfout, wrfstdout,
wrfstderr) = wrf_app (wrfin, tbl, ozone,
tr, data,<br>
> > > gribmap, namelist,
co2_trans, input_sounding);<br>
> > > }<br>
> > ><br>
> > > The script hangs at runtime
with the following messages:<br>
> > ><br>
> > > No events in 1s.<br>
> > > Finding dependency loops...<br>
> > ><br>
> > > Waiting threads:<br>
> > > Thread: R-6-0-4, waiting on
wrfout (declared on line 50)<br>
> > > swift:stageOut, wf.edison,
line 134<br>
> > > swift:execute, wf.edison,
line 123<br>
> > > wrf_app, wf.edison, line
242<br>
> > ><br>
> > > Thread: R-6-2-4, waiting on
wrfout (declared on line 50)<br>
> > > swift:stageOut, wf.edison,
line 134<br>
> > > swift:execute, wf.edison,
line 123<br>
> > > wrf_app, wf.edison, line
242<br>
> > ><br>
> > > Thread: R-6-1-4, waiting on
wrfout (declared on line 50)<br>
> > > swift:stageOut, wf.edison,
line 134<br>
> > > swift:execute, wf.edison,
line 123<br>
> > > wrf_app, wf.edison, line
242<br>
> > ><br>
> > > Any suggestions?<br>
> > ><br>
> > > Thanks,<br>
> > > Ketan<br>
> > >
_______________________________________________<br>
> > > Swift-user mailing list<br>
> > > <a href="mailto:Swift-user@ci.uchicago.edu" target="_blank">Swift-user@ci.uchicago.edu</a><br>
> > > <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
> ><br>
> ><br>
> ><br>
<br>
<br>
_______________________________________________<br>
Swift-user mailing list<br>
<a href="mailto:Swift-user@ci.uchicago.edu" target="_blank">Swift-user@ci.uchicago.edu</a><br>
<a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Swift-user mailing list
<a href="mailto:Swift-user@ci.uchicago.edu" target="_blank">Swift-user@ci.uchicago.edu</a>
<a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a></pre>
</blockquote>
<br>
</div>
</div>
<span><font color="#888888">
<pre cols="72">--
Michael Wilde
Mathematics and Computer Science Computation Institute
Argonne National Laboratory The University of Chicago
</pre>
</font></span></div>
<br>
_______________________________________________<br>
Swift-user mailing list<br>
<a href="mailto:Swift-user@ci.uchicago.edu" target="_blank">Swift-user@ci.uchicago.edu</a><br>
<a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
</blockquote>
</div>
<br>
</div>
</div></div></blockquote><div><div class="h5">
<br>
<pre cols="72">--
Michael Wilde
Mathematics and Computer Science Computation Institute
Argonne National Laboratory The University of Chicago
</pre>
</div></div></div>
<br>_______________________________________________<br>
Swift-user mailing list<br>
<a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><br>
<a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br></blockquote></div><br></div></div>