[Swift-user] Custom/External Mappers

[Robin Weiss]

Interesting.  I wrote the custom mapper script expecting swift to call it
with a whitespace (not equal sign) between the parameter switch and the
value that goes with it (I.e.: ./clusMapper_name.sh -name name_goes_here).
 This is how the example in the swift users guide for 0.93
(http://www.ci.uchicago.edu/swift/guides/release-0.93/userguide/userguide.h
tml#_external_mapper) does it.  The example from the users guide will also
chokes if you put an equal sign instead of whitespace between the param
name switch and the value.

This issue popped up after we updated to the latest trunk version from
0.93.  Did the way swift calls external mappers change (I.e. Equal sign
separated instead of whitespace between param name and value)?

Robin


-- 
Robin M. Weiss
Research Programmer
Research Computing Center
The University of Chicago
6030 S. Ellis Ave., Suite 289C
Chicago, IL 60637
robinweiss at uchicago.edu <mailto:labello at uchicago.edu>
773.702.9030






On 9/14/12 1:14 PM, "Michael Wilde" <wilde at mcs.anl.gov> wrote:

>Robin, I looked into this problem a bit. It seems that your external
>mapper scripts are not returning anything on standard output (and an
>error message on stderr) for the arguments that you are passing in the
>failing cases.
>
>Here's what my tests show:
>
>$ ./clusMapper_name.sh
>[0] out/.0.out
>[1] out/.1.out
>[2] out/.2.out
>[3] out/.3.out
>$
>
>$ ./clusMapper_name.sh -name=name_here
>./clusMapper_name.sh: bad mapper args
>$
>
>$ ./clusMapper_loop.sh -nfiles=3
>./clusMapper_loop.sh: bad mapper args
>$ 
>
>I dont see exactly why Swift is hanging in the cases where the mappers
>return nothing. I would have thought a nicer behavior would be to (a)
>exit complaining that the external mapper didnt return with exit code 0,
>or (b) proceed as if the arrays were not mapped, and use the default
>(concurrent) mapper.
>
>I suspect whats happening is that Swift is not closing the array when the
>mapper fails, or some similar synchronization error.  We'll need to look
>into this. I've filed this as bugzilla bug # 829.
>
>- Mike
>
>----- Original Message -----
>> From: "Robin Weiss" <robinweiss at uchicago.edu>
>> To: swift-user at ci.uchicago.edu
>> Sent: Monday, September 10, 2012 4:56:45 PM
>> Subject: [Swift-user] Custom/External Mappers
>> Howdy swifters,
>> 
>> 
>> I am having some issues getting external mappers to work. In
>> particular, swift appears to hang when you use a custom mapper that
>> takes in one or more command line arguments.
>> 
>> 
>> Attached is a tar ball with a basic example of the issue. You can
>> uncomment each line in the mapper.swift script to see the behavior. I
>> have included 4 versions of my external mapper, two work and two do
>> not (see mapper.swift). Also included are the config, sites (using
>> localhost), and tc files I'm using.
>> 
>> 
>> I noticed this issue after moving from version 0.93 to trunk (Swift
>> trunk swift-r5917 cog-r3463) when some scripts I had been using
>> started to hang.
>> 
>> 
>> Thanks in advance,
>> Robin
>> 
>> 
>> 
>> 
>> --
>> 
>> Robin M. Weiss
>> Research Programmer
>> Research Computing Center
>> The University of Chicago
>> 6030 S. Ellis Ave., Suite 289C
>> Chicago, IL 60637
>> robinweiss at uchicago.edu
>> 773.702.9030
>> _______________________________________________
>> Swift-user mailing list
>> Swift-user at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>
>-- 
>Michael Wilde
>Computation Institute, University of Chicago
>Mathematics and Computer Science Division
>Argonne National Laboratory
>