[Swift-user] Unix find as external mapper

[Michael Wilde]

Neil,

While your suggested syntax sounds reasonable, thats not how the ext mapper currently works.  You do indeed need to create a small wrapper script for such situations. The exec parameter is only used to specify the pathname of the mapper script (absolute or relative to the working dir in which you are running the swift command).

I'll file an enhancement ticket to record your suggestion, which in hindsight seems obvious, and which I like :)

Thanks,

- Mike

----- Original Message -----
> From: "Neil Best" <nbest at ci.uchicago.edu>
> To: swift-user at ci.uchicago.edu
> Sent: Wednesday, December 5, 2012 4:17:22 PM
> Subject: [Swift-user] Unix find as external mapper
> I tried to use find as an external mapper in a way that seemed fairly
> natural to me:
> 
> file monthly[] <ext; exec="find data/nc -mindepth 1 -type d -printf
> '[%f] %p.nc\n'\|.txt'">;
> 
> When I run this find by itself at the command line the output looks
> like this:
> 
> $ find data/nc -mindepth 1 -type d -printf '[%f] %p.nc\n' | head
> [197901] data/nc/197901.nc
> [197903] data/nc/197903.nc
> [197904] data/nc/197904.nc
> [197902] data/nc/197902.nc
> [197905] data/nc/197905.nc
> [197906] data/nc/197906.nc
> [197907] data/nc/197907.nc
> [197908] data/nc/197908.nc
> [197909] data/nc/197909.nc
> [197910] data/nc/197910.nc
> 
> The find actually looks at directories. These .nc files don't exist
> yet, but each one will be an aggregation of the .nc files within the
> folder of the same name using this command:
> 
> cdo mergetime data/nc/197901/*.single.nc data/nc/197901.nc
> 
> I thought I could then do a nested foreach over years and months to
> execute the invocations.
> 
> I presume that the external mapper is treating the exec argument as a
> command name and not parsing the arguments. Using the other
> parameters in the ext construct would force me to write a wrapper
> script for the find invocation since all of the arguments must have
> names apparently.
> 
> Do I have this right? Maybe there is a more straightforward approach.
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory