<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Jonathan, I'd need to test this, but my suspicion is that specifying
location only changes the directory that the mapper should look in.<br>
<br>
But you also need to give it a prefix and/or a suffix - else it
won't know what you want it to map *within* that drectory.<br>
<br>
My guess is that you had a mapper with "prefix=" that worked, and
you just changed "prefix" to "location"?<br>
<br>
Also - please see Mihael's post to swift-devel in the past hour. He
has committed the enhancement to dynamically map an output array of
files on the compute node, as you requested and we discussed. I
think you'll have to build a swift from trunk source to try this,
but thats only an svn co and a few commands (see Download at
swift-lang.org).<br>
<br>
-- <br>
Mike<br>
<br>
<div class="moz-cite-prefix">On 3/27/14, 5:02 PM, Jonathan Ozik
wrote:<br>
</div>
<blockquote
cite="mid:B38ADDB4-5F5B-4D81-948C-68665AED9247@uchicago.edu"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
Mike,
<div><br>
</div>
<div>I've been making a good amount of progress and I'm able to
successfully run a number of concurrent instances locally.</div>
<div><br>
</div>
<div>I had a question about the filesys_mapper, and how it might
(or might not) be able to handle nested folders. If I point it
to a "location" like this:</div>
<div>file data[] <filesys_mapper;location="data">;</div>
<div>I get a file not found error when it is used as an input
(presumably because that is when the mapper gets resolved) if
there is a nested folder within that location.</div>
<div>data/</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>temp.txt</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>insideData/</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>temp1.txt</div>
<div><br>
</div>
<div>Is this expected behavior? If so, is there a workaround to
gather all files under a folder as input?</div>
<div><br>
</div>
<div>Jonathan</div>
<div><br>
<div>
<div>On Mar 22, 2014, at 11:31 AM, Michael Wilde <<a
moz-do-not-send="true" href="mailto:wilde@anl.gov">wilde@anl.gov</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div text="#000000" bgcolor="#FFFFFF"> This is now filed as
enhancement bug 1225 and assigned to you, Mihael. The
description has been revised as you suggested, to propose
that the feature be first done using simple_mapper rather
than a new mapper.<br>
<br>
- Mike<br>
<br>
<span style="font-family: Helvetica, Arial, Geneva;
font-size: 17px; font-style: normal; font-variant:
normal; font-weight: bold; letter-spacing: normal;
line-height: normal; text-align: start; text-indent:
0px; text-transform: none; white-space: normal;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(208, 208, 208); display: inline
!important; float: none; "><span
class="Apple-converted-space"></span></span>
<div class="moz-cite-prefix">On 3/21/14, 8:39 AM, Michael
Wilde wrote:<br>
</div>
<blockquote cite="mid:532C410C.5080309@anl.gov"
type="cite">
<div class="moz-text-flowed" style="font-family:
-moz-fixed; font-size: 12px;" lang="x-western">Mihael,
All, <br>
<br>
I'd like to propose a Swift/K feature to provide a
reasonable solution to this very common need for an
app to return a dynamically determined set of files. <br>
<br>
file dynarry[ ] <runtime; prefix="myoutput.",
suffix=".out", indexes="int">; <br>
<br>
dynarry = myApp(myArgs); <br>
<br>
The "runtime" mapper should initially have the same
arguments and semantics (roughly) as simple_mapper,
except for two new arguments: <br>
<br>
"indexes" which determines how the matched file names
will be indexed in the returned array <br>
"int" | "string" | "sequential" <br>
sequential: return the matched files as consecutive
integer indices starting with 0 <br>
int: expect the filename component between prefix and
suffix to be convertible to an integer, and use that
as the index <br>
eg myfile.012.out and myfile.204.out will return an
array with the mapped files at indices 12 and 204. <br>
string: similar to int but return a string-indexed
associative array. <br>
"sequential" is simplest and should be the default. <br>
<br>
"paths" which determines if the match names will be
absolute or relative to the job dir <br>
paths="relative" | "absolute" <br>
(may not be needed if this can be determined uniquely
based on the location argument. <br>
<br>
swiftwrap will allow array variables mapped in this
manner to have any number of files, including zero.
I.e. "runtime-mapped" files should not be listed in
the expected output list for an app invocation. Its up
to the users app to ensure that some files match the
pattern. An additional arg could set e.g. minfiles
and/or maxfiles, in which case the wrapper code needs
to validate the count of files matched and returned,
but not their exact names. <br>
<br>
We can call this mapper "experimental" until we
validate its usability and suitability as a permanent
feature. But as we hope to revise the entire mapper
family and semantics, in a sense all mappers are
subject to change. <br>
<br>
Mihael, is the definition sound, and how long would it
take you to develop it? <br>
<br>
Thanks, <br>
<br>
- Mike <br>
<br>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Michael Wilde
Mathematics and Computer Science Computation Institute
Argonne National Laboratory The University of Chicago
</pre>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</body>
</html>