[Swift-user] output file array

[Ketan Maheshwari]

On Fri, Jan 9, 2015 at 12:37 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:

> Hi Ketan,
>
> On Fri, 2015-01-09 at 11:17 -0600, Ketan Maheshwari wrote:
> [...]
> > >
> > > Not automagically! You pass the stdout/stderr file names to the app
> > > command line using stdout= and stderr=.
> > >
> >
> > True. However, the scheme is not quite consistent due to the following
> > behavior:
> >
> > -- When specifying stdout and stderr in command line, I do not specify
> them
> > in directories. Yet, if I map them in non-existent directories, Swift
> will
> > create those directories and put the stdout/stderr files in them. This
> does
> > not happen with other files even if they are specified in command line.
>
> I do not think that is true. Swift creates directory structures for all
> input and output files, whether they are specified on the command line
> or not. If this isn't working, then we are talking about a bug, but the
> code was designed to do this from the start. I did a quick check and it
> seems to work fine for both 0.95 and trunk.
>

May be it is a bug. See the attached tarball which have example cases of
things that work and does not. touchafile1.swift illustrates this above
point.

>
> >
> > -- If a file is specified as output file in app definition but does not
> > appear in the command line, Swift will still bring it in as an output.
> > Again, this will happen only if the file is mapped at top level and not
> in
> > any directory.
>
> I do not believe that to be true either. Swift stages in all files that
> are parameters to and app and stages out all files that are returns from
> an app. There is no code to treat files differently based on whether
> they are in sub-directories or not. Again, if this isn't working on a
> specific version of the code, we may be looking at a bug. I checked
> this, too, and it works on 0.95 and trunk with the simple configurations
> I tried.
>

touchafile2.swift illustrates this point.


>
> >
> > >
> > > So in all cases swift honors what you tell it to honor. The question is
> > > what is it that you really want here:
> > > - is it to automatically create multiple versions of files when an app
> > > specifies the same output file and there are multiple invocations?
> > >
> >
> > Yes. The app specifies it alright but the executable does not specify it
> on
> > command line.
>
> I am not sure how that answers the question above. What I meant was
> this:
>
> foreach i in [0:10] {
>   file outf <"out.txt">;
>   outf = echo(i);
> }
>
> where echo(i) is an app that returns a file.
>
> The question was whether you are suggesting that the code above should
> be valid code. It is not currently because it violates the local
> consistency rule. In other words, writing the equivalent of this in a
> shell script will result in out.txt being overwritten, and that results
> in nondeterminism based on exactly how the echo() invocations are
> ordered.
>

I understand this is not a valid code because successive echo invocations
will overwrite the out.txt. This is why I want it to be like this:

foreach i in [0:9]{
  file outf <single_file_mapper; file=strcat("outdir", i, "/out.txt")>;
  outf = echo(i);
}


>
> > > - or is it for swift to ignore the directory in the remote app output
> > > when staging out files (but not locally)?
>
> Mihae
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20150109/1120ee12/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: touchafile.tgz
Type: application/x-gzip
Size: 1066 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20150109/1120ee12/attachment.bin>