[Swift-user] Custom/External Mappers

[Michael Wilde]

Ooops, my mistake, sorry.

You are correct, the ext mapper args are passed as "-argname value". The user guide is correct and that did not likely change in trunk.

So the problem is elsewhere.  I will investigate further.

- Mike


----- Original Message -----
> From: "Robin Weiss" <robinweiss at uchicago.edu>
> To: "Michael Wilde" <wilde at mcs.anl.gov>, swift-user at ci.uchicago.edu
> Sent: Friday, September 14, 2012 1:26:39 PM
> Subject: Re: [Swift-user] Custom/External Mappers
> Interesting. I wrote the custom mapper script expecting swift to call
> it
> with a whitespace (not equal sign) between the parameter switch and
> the
> value that goes with it (I.e.: ./clusMapper_name.sh -name
> name_goes_here).
> This is how the example in the swift users guide for 0.93
> (http://www.ci.uchicago.edu/swift/guides/release-0.93/userguide/userguide.h
> tml#_external_mapper) does it. The example from the users guide will
> also
> chokes if you put an equal sign instead of whitespace between the
> param
> name switch and the value.
> 
> This issue popped up after we updated to the latest trunk version from
> 0.93. Did the way swift calls external mappers change (I.e. Equal sign
> separated instead of whitespace between param name and value)?
> 
> Robin
> 
> 
> --
> Robin M. Weiss
> Research Programmer
> Research Computing Center
> The University of Chicago
> 6030 S. Ellis Ave., Suite 289C
> Chicago, IL 60637
> robinweiss at uchicago.edu <mailto:labello at uchicago.edu>
> 773.702.9030
> 
> 
> 
> 
> 
> 
> On 9/14/12 1:14 PM, "Michael Wilde" <wilde at mcs.anl.gov> wrote:
> 
> >Robin, I looked into this problem a bit. It seems that your external
> >mapper scripts are not returning anything on standard output (and an
> >error message on stderr) for the arguments that you are passing in
> >the
> >failing cases.
> >
> >Here's what my tests show:
> >
> >$ ./clusMapper_name.sh
> >[0] out/.0.out
> >[1] out/.1.out
> >[2] out/.2.out
> >[3] out/.3.out
> >$
> >
> >$ ./clusMapper_name.sh -name=name_here
> >./clusMapper_name.sh: bad mapper args
> >$
> >
> >$ ./clusMapper_loop.sh -nfiles=3
> >./clusMapper_loop.sh: bad mapper args
> >$
> >
> >I dont see exactly why Swift is hanging in the cases where the
> >mappers
> >return nothing. I would have thought a nicer behavior would be to (a)
> >exit complaining that the external mapper didnt return with exit code
> >0,
> >or (b) proceed as if the arrays were not mapped, and use the default
> >(concurrent) mapper.
> >
> >I suspect whats happening is that Swift is not closing the array when
> >the
> >mapper fails, or some similar synchronization error. We'll need to
> >look
> >into this. I've filed this as bugzilla bug # 829.
> >
> >- Mike
> >
> >----- Original Message -----
> >> From: "Robin Weiss" <robinweiss at uchicago.edu>
> >> To: swift-user at ci.uchicago.edu
> >> Sent: Monday, September 10, 2012 4:56:45 PM
> >> Subject: [Swift-user] Custom/External Mappers
> >> Howdy swifters,
> >>
> >>
> >> I am having some issues getting external mappers to work. In
> >> particular, swift appears to hang when you use a custom mapper that
> >> takes in one or more command line arguments.
> >>
> >>
> >> Attached is a tar ball with a basic example of the issue. You can
> >> uncomment each line in the mapper.swift script to see the behavior.
> >> I
> >> have included 4 versions of my external mapper, two work and two do
> >> not (see mapper.swift). Also included are the config, sites (using
> >> localhost), and tc files I'm using.
> >>
> >>
> >> I noticed this issue after moving from version 0.93 to trunk (Swift
> >> trunk swift-r5917 cog-r3463) when some scripts I had been using
> >> started to hang.
> >>
> >>
> >> Thanks in advance,
> >> Robin
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Robin M. Weiss
> >> Research Programmer
> >> Research Computing Center
> >> The University of Chicago
> >> 6030 S. Ellis Ave., Suite 289C
> >> Chicago, IL 60637
> >> robinweiss at uchicago.edu
> >> 773.702.9030
> >> _______________________________________________
> >> Swift-user mailing list
> >> Swift-user at ci.uchicago.edu
> >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> >
> >--
> >Michael Wilde
> >Computation Institute, University of Chicago
> >Mathematics and Computer Science Division
> >Argonne National Laboratory
> >

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory