[Swift-devel] foreach.max.threads question

[Michael Wilde]

Mihael, I though that when we discussed this last FRiday, the plan was to add the app() throttle but also leave the foreach throttle in place, just in case we needed it.  Was there a reason that you needed to remove the foreach throttle to do the app throttle?

- Mike


----- Original Message -----
> From: "Mihael Hategan" <hategan at mcs.anl.gov>
> To: "David Kelly" <davidk at ci.uchicago.edu>
> Cc: "swift-devel Devel" <swift-devel at ci.uchicago.edu>
> Sent: Tuesday, October 16, 2012 1:44:47 PM
> Subject: Re: [Swift-devel] foreach.max.threads question
> What was foreach.max.threads set to before?
> 
> On Tue, 2012-10-16 at 11:04 -0500, David Kelly wrote:
> > Hello,
> >
> > I have noticed that since the foreach.max.threads changes, the DSSAT
> > script is now running out of memory. I have the heap size set to 4
> > gigabytes. There are 120K items in gridLists.
> > The main foreach loop of the script looks like this:
> >
> > foreach g,i in gridLists {
> >    file tar_output <single_file_mapper; file=@strcat("output/",
> >    gridLists[i], "output.tar.gz")>;
> >    file part_output <single_file_mapper; file=@strcat("parts/",
> >    gridLists[i], ".part")>;
> >
> >    file in1[] <filesys_mapper; location=@strcat(@arg("scenarios"),
> >    "/", gridLists[i]), pattern="*">; // Scenario files
> >    file in2[] <filesys_mapper; location=@strcat(@arg("weather"),
> >    "/", gridLists[i]), pattern="*">; // Weather files
> >    file in3[] <filesys_mapper; location=@arg("refdata"),
> >    pattern="*">; // Common data
> >    file in4[] <filesys_mapper; location=@arg("bindata"),
> >    pattern="*.EXE">; // Binaries
> >    file in5[] <filesys_mapper; location=@arg("bindata"),
> >    pattern="*.pl">; // Perl scripts
> >    file wrapper <single_file_mapper; file="RunDSSAT.sh">; //
> >    RunDSSAT wrapper
> >
> >    (tar_output, part_output) = RunDSSAT(xfile, in1, in2, in3, in4,
> >    in5, wrapper);
> > }
> >
> > Is there any way to throttle foreach again, or any other workarounds
> > I could use to avoid this?
> >
> > Thanks,
> > David
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory