[Swift-devel] foreach.max.threads question

[David Kelly]

Previously it was not explicitly set, so I am assuming it would have been 1024. As a test I tried setting it to 520 (the maximum number of available cores), but that did not seem to help.

----- Original Message -----
> From: "Mihael Hategan" <hategan at mcs.anl.gov>
> To: "David Kelly" <davidk at ci.uchicago.edu>
> Cc: "swift-devel Devel" <swift-devel at ci.uchicago.edu>
> Sent: Tuesday, October 16, 2012 1:44:47 PM
> Subject: Re: [Swift-devel] foreach.max.threads question
> What was foreach.max.threads set to before?
> 
> On Tue, 2012-10-16 at 11:04 -0500, David Kelly wrote:
> > Hello,
> >
> > I have noticed that since the foreach.max.threads changes, the DSSAT
> > script is now running out of memory. I have the heap size set to 4
> > gigabytes. There are 120K items in gridLists.
> > The main foreach loop of the script looks like this:
> >
> > foreach g,i in gridLists {
> >    file tar_output <single_file_mapper; file=@strcat("output/",
> >    gridLists[i], "output.tar.gz")>;
> >    file part_output <single_file_mapper; file=@strcat("parts/",
> >    gridLists[i], ".part")>;
> >
> >    file in1[] <filesys_mapper; location=@strcat(@arg("scenarios"),
> >    "/", gridLists[i]), pattern="*">; // Scenario files
> >    file in2[] <filesys_mapper; location=@strcat(@arg("weather"),
> >    "/", gridLists[i]), pattern="*">; // Weather files
> >    file in3[] <filesys_mapper; location=@arg("refdata"),
> >    pattern="*">; // Common data
> >    file in4[] <filesys_mapper; location=@arg("bindata"),
> >    pattern="*.EXE">; // Binaries
> >    file in5[] <filesys_mapper; location=@arg("bindata"),
> >    pattern="*.pl">; // Perl scripts
> >    file wrapper <single_file_mapper; file="RunDSSAT.sh">; //
> >    RunDSSAT wrapper
> >
> >    (tar_output, part_output) = RunDSSAT(xfile, in1, in2, in3, in4,
> >    in5, wrapper);
> > }
> >
> > Is there any way to throttle foreach again, or any other workarounds
> > I could use to avoid this?
> >
> > Thanks,
> > David
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel