[Swift-user] Custom/External Mappers

[Michael Wilde]

Robin, I looked into this problem a bit. It seems that your external mapper scripts are not returning anything on standard output (and an error message on stderr) for the arguments that you are passing in the failing cases.

Here's what my tests show:

$ ./clusMapper_name.sh          
[0] out/.0.out
[1] out/.1.out
[2] out/.2.out
[3] out/.3.out
$

$ ./clusMapper_name.sh -name=name_here
./clusMapper_name.sh: bad mapper args
$

$ ./clusMapper_loop.sh -nfiles=3
./clusMapper_loop.sh: bad mapper args
$ 

I dont see exactly why Swift is hanging in the cases where the mappers return nothing. I would have thought a nicer behavior would be to (a) exit complaining that the external mapper didnt return with exit code 0, or (b) proceed as if the arrays were not mapped, and use the default (concurrent) mapper.

I suspect whats happening is that Swift is not closing the array when the mapper fails, or some similar synchronization error.  We'll need to look into this. I've filed this as bugzilla bug # 829.

- Mike

----- Original Message -----
> From: "Robin Weiss" <robinweiss at uchicago.edu>
> To: swift-user at ci.uchicago.edu
> Sent: Monday, September 10, 2012 4:56:45 PM
> Subject: [Swift-user] Custom/External Mappers
> Howdy swifters,
> 
> 
> I am having some issues getting external mappers to work. In
> particular, swift appears to hang when you use a custom mapper that
> takes in one or more command line arguments.
> 
> 
> Attached is a tar ball with a basic example of the issue. You can
> uncomment each line in the mapper.swift script to see the behavior. I
> have included 4 versions of my external mapper, two work and two do
> not (see mapper.swift). Also included are the config, sites (using
> localhost), and tc files I'm using.
> 
> 
> I noticed this issue after moving from version 0.93 to trunk (Swift
> trunk swift-r5917 cog-r3463) when some scripts I had been using
> started to hang.
> 
> 
> Thanks in advance,
> Robin
> 
> 
> 
> 
> --
> 
> Robin M. Weiss
> Research Programmer
> Research Computing Center
> The University of Chicago
> 6030 S. Ellis Ave., Suite 289C
> Chicago, IL 60637
> robinweiss at uchicago.edu
> 773.702.9030
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory