<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Thanks David (especially for pointing me to the newer user guide).<div>Mihael, would the "swift/0.95-RC5" version contain the FilesysMapper that accepts the extended glob patterns?</div><div><br></div><div>Jonathan</div><div><br><div><div>On Apr 2, 2014, at 10:17 AM, David Kelly <<a href="mailto:davidkelly@uchicago.edu">davidkelly@uchicago.edu</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"><div dir="ltr">Hi Jonathan,<div><br></div><div>A fairly recent version of 0.95 is available on Midway in the module "swift/0.95-RC5". It should backwards compatible with 0.94. You may see some warnings about deprecated use of @ in front of functions, and you will see logs going into a run directory named run001, run002, etc instead of going to your current working directory. There's more information about this and the other new (and optional) configuration changes at <a href="http://swiftlang.org/guides/trunk/userguide/userguide.html#_configuration">http://swiftlang.org/guides/trunk/userguide/userguide.html#_configuration</a>. Please let me know if you have any issues.</div>
<div><br>Thanks,</div><div>David</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 2, 2014 at 9:51 AM, Jonathan Ozik <span dir="ltr"><<a href="mailto:jozik@uchicago.edu" target="_blank">jozik@uchicago.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex; position: static; z-index: auto; ">Yadu, David, Mihael,<br>
<br>
Thanks for your responses.<br>
I'm thinking of using the ext mapper for now. Would the trunk/0.95 be available on Midway?<br>
<span class="HOEnZb"><font color="#888888"><br>
Jonathan<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On Apr 1, 2014, at 8:50 PM, Mihael Hategan <<a href="mailto:hategan@mcs.anl.gov">hategan@mcs.anl.gov</a>> wrote:<br>
<br>
> At least in trunk/0.95, FilesysMapper accepts extended glob patterns, so<br>
> you should be able to say:<br>
><br>
> file[] f <FilesysMapper; location=".", pattern="**/*.*">;<br>
><br>
> Mihael<br>
><br>
> On Tue, 2014-04-01 at 19:49 -0500, David Kelly wrote:<br>
>> I created a ticket about this on Monday because I was running into similar<br>
>> issues in my scripts. An ext mapper worked for me, but I think this is a<br>
>> common pattern we can make easier in future releases.<br>
>> <a href="https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1229" target="_blank">https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1229</a>.<br>
>><br>
>><br>
>> On Tue, Apr 1, 2014 at 7:09 PM, Yadu Nand <<a href="mailto:yadudoc1729@gmail.com">yadudoc1729@gmail.com</a>> wrote:<br>
>><br>
>>> Hi Jonathan,<br>
>>><br>
>>> What I'd do in this case is map every file under the directory you want to<br>
>>> send to the compute nodes, into an array and pass<br>
>>> that along to your apps. You can do this mapping using either ext mapper<br>
>>> or array mappers.<br>
>>><br>
>>> I have the following dir structure in my folders:<br>
>>><br>
>>> ./dirs/foo_a/foo_a.txt<br>
>>> ./dirs/foo_a/foo_b.txt<br>
>>> ./dirs/foo_b/bar_1<br>
>>> ./dirs/foo_b/bar_2<br>
>>><br>
>>> Here's my ext mapper (mapper.sh) :<br>
>>> #!/bin/bash<br>
>>> find ./dirs -type f | awk '{printf("[%d] %s\n", NR, $0)}'<br>
>>><br>
>>> If you are using ext mappers, you would need a script which generates<br>
>>> output in the form [<index>] <filename><br>
>>> Here I use find to just output files and awk to get the right format.<br>
>>><br>
>>> The swift mapping would be like this:<br>
>>> file array[] <ext ; exec="mapper.sh" >;<br>
>>><br>
>>> You could also use array mappers to read all files you need from a file<br>
>>> containing the filenames. I filled filenames.txt with<br>
>>> the names of all files in the folders.<br>
>>><br>
>>> string[] names = readData("filenames.txt");<br>
>>> file dirmap[] <array_mapper; files=names>;<br>
>>><br>
>>> I've got both cases as examples tarballed here if you'd like to take a<br>
>>> look : <a href="http://swift.rcc.uchicago.edu:8042/directory_mapping.tar" target="_blank">http://swift.rcc.uchicago.edu:8042/directory_mapping.tar</a><br>
>>><br>
>>> Thanks,<br>
>>> Yadu<br>
>>><br>
>>> On Tue, Apr 1, 2014 at 8:56 AM, Jonathan Ozik <<a href="mailto:jozik@uchicago.edu">jozik@uchicago.edu</a>> wrote:<br>
>>><br>
>>>> Hello all,<br>
>>>><br>
>>>> Is there a simple way to specify "all files including files in subfolders<br>
>>>> within a folder" as a file mapper? It looks like the filesys_mapper does<br>
>>>> get everything within a folder, but has problems if there's a folder in<br>
>>>> there.<br>
>>>><br>
>>>> Jonathan<br>
>>>><br>
>>>> _______________________________________________<br>
>>>> Swift-user mailing list<br>
>>>> <a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><br>
>>>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
>>>><br>
>>><br>
>>><br>
>>><br>
>>> --<br>
>>> Yadu Nand B<br>
>>><br>
>>><br>
>>> _______________________________________________<br>
>>> Swift-user mailing list<br>
>>> <a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><br>
>>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
>>><br>
>> _______________________________________________<br>
>> Swift-user mailing list<br>
>> <a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><br>
>> <a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br>
><br>
><br>
<br>
</div></div></blockquote></div><br></div>
</blockquote></div><br></div></body></html>