[Swift-user] output file array
Michael Wilde
wilde at anl.gov
Tue Jan 6 13:13:37 CST 2015
On 1/6/15 1:04 PM, Ketan Maheshwari wrote:
> Hi Mike,
>
> The app is creating the file but the way the app is invoked, the
> filename does not appear in the command-line. It creates the file at
> the top level, so just afile.txt.
>
> However, since there are many calls to the app, to avoid this file
> being overwritten, I need to put this file into a separate directory
> which is why I am using directory outdirN.
>
> I was thinking if it is possible for Swift runtime to find from the
> app definition that the file _appout is expected output similar to
> stdout and move the file to the outdirN (again similar to stdout).
I think what you're asking for is the ability to declare for each app
that its temporary "sandbox" working dir gets saved below the current
working dir in which you're running the swift command. And, further, to
be able to name that directory from the source script.
That *might* be a reasonable feature, but will need more discussion. I
suggest a bugzilla ticket to capture this as a proposed enhancement. It
seems however that the current way of doing this, explicitly, is simple
and sufficient for now.
- Mike
>
> Thanks,
> Ketan
>
> On Tue, Jan 6, 2015 at 12:53 PM, Michael Wilde <wilde at anl.gov
> <mailto:wilde at anl.gov>> wrote:
>
> Is your app touch_app( ) correctly creating output files of the
> form outdirN/afile.txt?
>
> From the error message, I suspect that it is not.
>
> Your app is declared as:
>
> app (file _stdout, file _stderr, file _appout) touch_app(string
> _instr){
> t _instr stdout=@_stdout stderr=@_stderr;
> }
>
>
> You need to pass filename(_appout) to the app, via its command
> line, so that it knows the correct output filename to create.
> Then you need to ensure that the app does indeed create that file.
>
> - Mike
>
>
> On 1/6/15 11:23 AM, Ketan Maheshwari wrote:
>> Trying trunk for this pattern.
>>
>> A toy application invoked over a foreach loop that creates an
>> output file, an stdout and an stderr files.
>>
>> The files are mapped into an output directory named with the loop
>> index as suffix so that the files do not get overwritten:
>>
>> foreach i in [0:9]{
>>
>> file out<single_file_mapper; file=strcat("outdir", i, "/std.out")>;
>> file err<single_file_mapper; file=strcat("outdir", i, "/std.err")>;
>> file appout<single_file_mapper; file=strcat("outdir", i,
>> "/afile.txt")>;
>>
>> (out, err, appout) = touch_app("Hello");
>> }
>>
>>
>> The stdout and stderr files correctly ends up in their respective
>> directories but the app generated file does not.
>>
>> I see following error message:
>>
>> Execution failed:
>> Exception in t:
>> Arguments: [Hello]
>> Host: edison1
>> Directory: touchafile-run001/jobs/t/t-ffv33r2m
>> exception @ swift-int-staging.k, line: 165
>> Caused by: The following output files were not created by the
>> application: outdir4/afile.txt
>>
>> Any suggestions for fixing this?
>>
>> Attached is the test directory with sources and executable with
>> rundir.
>>
>> Thanks,
>> Ketan
>>
>>
>> On Mon, Dec 22, 2014 at 5:14 PM, Mihael Hategan
>> <hategan at mcs.anl.gov <mailto:hategan at mcs.anl.gov>> wrote:
>>
>> Hi,
>>
>> I don't think 0.95 supports dynamic arrays output from apps.
>> You will
>> need trunk/0.96 for that.
>>
>> Mihael
>>
>> On Mon, 2014-12-22 at 15:37 -0600, Ketan Maheshwari wrote:
>> > Hi Mihael,
>> >
>> > This is with Swift 0.95.
>> >
>> > Thanks,
>> > Ketan
>> >
>> > On Sun, Dec 21, 2014 at 2:40 PM, Hategan-Marandiuc, Philip M. <
>> > hategan at mcs.anl.gov <mailto:hategan at mcs.anl.gov>> wrote:
>> >
>> > > Hi Ketan,
>> > >
>> > > Sorry for the delay. Is this trunk or 0.95?
>> > >
>> > > Mihael
>> > >
>> > > On Wed, 2014-12-17 at 14:26 -0600, Ketan Maheshwari wrote:
>> > > > Hi,
>> > > >
>> > > > I am dealing with a workflow pattern where an app
>> expects multiple output
>> > > > files with a pattern.
>> > > >
>> > > > The app signature is:
>> > > >
>> > > > app (file[] _wrfout, file _out, file _err) wrf_app
>> (file _wrf_in, file[]
>> > > > _tbl, file[] _ozone, ...)
>> > > > {
>> > > > wrf stdout=@_out stderr=@_err;
>> > > > }
>> > > >
>> > > > The _wrfout files are the app result files which
>> follows a pattern:
>> > > wrfout_*
>> > > >
>> > > > So, I am invoking the application in a foreach loop as:
>> > > >
>> > > > foreach i in [0:2]{
>> > > > file[] wrfout<simple_mapper; location=strcat("outdir",i),
>> > > > pattern="wrfout_*">;
>> > > > file wrfstdout<single_file_mapper;
>> file=strcat("outdir", i,
>> > > "/std.out")>;
>> > > > file wrfstderr<single_file_mapper;
>> file=strcat("outdir", i,
>> > > "/std.err")>;
>> > > >
>> > > > (wrfout, wrfstdout, wrfstderr) = wrf_app (wrfin,
>> tbl, ozone, tr, data,
>> > > > gribmap, namelist, co2_trans, input_sounding);
>> > > > }
>> > > >
>> > > > The script hangs at runtime with the following messages:
>> > > >
>> > > > No events in 1s.
>> > > > Finding dependency loops...
>> > > >
>> > > > Waiting threads:
>> > > > Thread: R-6-0-4, waiting on wrfout (declared on line 50)
>> > > > swift:stageOut, wf.edison, line 134
>> > > > swift:execute, wf.edison, line 123
>> > > > wrf_app, wf.edison, line 242
>> > > >
>> > > > Thread: R-6-2-4, waiting on wrfout (declared on line 50)
>> > > > swift:stageOut, wf.edison, line 134
>> > > > swift:execute, wf.edison, line 123
>> > > > wrf_app, wf.edison, line 242
>> > > >
>> > > > Thread: R-6-1-4, waiting on wrfout (declared on line 50)
>> > > > swift:stageOut, wf.edison, line 134
>> > > > swift:execute, wf.edison, line 123
>> > > > wrf_app, wf.edison, line 242
>> > > >
>> > > > Any suggestions?
>> > > >
>> > > > Thanks,
>> > > > Ketan
>> > > > _______________________________________________
>> > > > Swift-user mailing list
>> > > > Swift-user at ci.uchicago.edu
>> <mailto:Swift-user at ci.uchicago.edu>
>> > > >
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>> > >
>> > >
>> > >
>>
>>
>> _______________________________________________
>> Swift-user mailing list
>> Swift-user at ci.uchicago.edu <mailto:Swift-user at ci.uchicago.edu>
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>>
>>
>>
>>
>> _______________________________________________
>> Swift-user mailing list
>> Swift-user at ci.uchicago.edu <mailto:Swift-user at ci.uchicago.edu>
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>
> --
> Michael Wilde
> Mathematics and Computer Science Computation Institute
> Argonne National Laboratory The University of Chicago
>
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu <mailto:Swift-user at ci.uchicago.edu>
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>
>
--
Michael Wilde
Mathematics and Computer Science Computation Institute
Argonne National Laboratory The University of Chicago
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20150106/0e14041d/attachment.html>
More information about the Swift-user
mailing list