[Swift-devel] [Bug 345] New: Print a clear error message when the argument list of an app is too long

[bugzilla-daemon at mcs.anl.gov]

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=345

           Summary: Print a clear error message when the argument list of
                    an app is too long
           Product: Swift
           Version: 0.92
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: SwiftScript language
        AssignedTo: skenny at uchicago.edu
        ReportedBy: wilde at mcs.anl.gov


This problem is illustrated in the following email thread:


No.  It was the same.  As I was digging deeper into the problem I started
remembering that all this seemed familiar.  I checked my properties file and
accidentally deleted the wrapper.parameter.mode line.  I wanted to confirm this
was the problem before I reported back and it just took awhile before the
workflow got to that section.

That error makes sense once I remembered what was wrong.  This might be another
place where a better error message(not just the one linux reported back) would
have probably helped shed light on what is going on, imho.

On Tue, Apr 12, 2011 at 6:10 AM, Michael Wilde <wilde at mcs.anl.gov> wrote:
Jon, the error is "Argument list too long" (E2BIG) - most likely because you
are trying to form an argument list to an app that is longer than the Linux
limit (which is host-specific and settable during Linux configuration, *I
think*).

As I was writing this response, I realized you already dealt with this issue in
a thread on swift-user in Oct 2010 (subject "Argument list too long"). Is your
current problem some new manifestation of the prior argument passing problem?

I also realized that I dont understand the interaction between property
"wrapper.parameter.mode" and the need to use writeData() to form argument
lists, and that this needs to be explained in a Cookbook -> User Guide section.

So before I go deeper into this, can you explain if the current problem is
different than the prior one?

Thanks,

- Mike

----


This page has some useful info on the limit:
 http://www.in-ulm.de/~mascheck/various/argmax/

On communicado the limit seems to be some percentage of the stack size, which
defaults to 10M. The article above says that Linux 2.26 and above use .25 *
stack size.

Where are you running, and what is the likely command line length of your
largest app invocations? (Keep in mind that pathnames may be longer than you
expect and that Swift is running _swiftwrap which also has specific arg passing
conventions).

This is often an issue when you call a summarization app, and try to pass it
all the files processed in some large prior foreach loop.

Ben added some support for passing much longer argument lists in a file via the
function writeData, and there is also this property:

# Controls how swift will supply parameters to the remote wrapper script.
# 'args' mode will pass parameters on the command line
# 'files' mode will pass parameters through an additional input file
#
# valid values: args, files
# Default: files
#
# wrapper.parameter.mode=args




----- Original Message -----
> Does anyone know what is this error trying to tell me?
>
>
> Caused by: Cannot run program "/bin/bash" (in directory
> "/gpfs/pads/swift/jonmon/Swift/work/localhost/rectified-20110411-1923-pqygyay7"):
> java.io.IOException: error=7, Argument list too long
> Caused by: java.io.IOException: Cannot run program "/bin/bash" (in
> directory
> "/gpfs/pads/swift/jonmon/Swift/work/localhost/rectified-20110411-1923-pqygyay7"):
> java.io.IOException: error=7, Argument list too long
> --
> Any intelligent fool can make things bigger and more complex... It
> takes a touch of genius - and a lot of courage to move in the opposite
> direction.
> - Albert Einstein
>
>
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel

--
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory




-- 
Any intelligent fool can make things bigger and more complex... It takes a
touch of genius - and a lot of courage to move in the opposite direction.
- Albert Einstein

-- 
Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the reporter.