[Swift-user] Swift running errors

[Mihael Hategan]

On Wed, 2008-02-20 at 12:32 -0600, Ioan Raicu wrote:
> I doubt its a bug in the JVM, its probably the application not closing
> all the file/stream handles.

Hard to miss an opportunity to argue, isn't it?

Well, my debugger disagrees with you. And I'll trust the debugger on
this one.

>    For example, I ran into this problem when I was using:
> > Process child = Runtime.getRuntime().exec(command);
> but not consuming and closing the stdout and stderr streams which get
> created automatically when the child process gets created.  Simply
> setting child to null after the process exited did not close the
> streams.

Excellent. You've identified a similar problem. Eventually files get
closed when you set things to null, but only when the garbage collector
gets to those objects and finalizes them.

>   This quickly lead to too many open file handles after a few hundred
> exec() calls.  I had to make a wrapper class of the Process class that
> took care of consuming and closing the output streams by default, and
> my problems went away.
> 
> Just another idea of where to look for potential problems in the code.
> 
> Ioan
> 
> Mihael Hategan wrote: 
> > > > 2. When runing a workflow which involves 1000nodes, I 
> > > > encounter the following errors very frequently, but not all 
> > > > the time:
> > > >       
> > > [..]
> > >     
> > > > java.io.IOException: Too many open files
> > > >       
> > > What machine are you running on? Have you changed any configuration 
> > > parameters? A basic 1000 job workflow should run fine with the default 
> > > settings.
> > > 
> > > On the machine you are running on, type   ulimit -a   and paste that here.
> > > 
> > >     
> > 
> > There is some odd bug in the JVM. Some, and I don't remember exactly
> > which, classes related to time/date open time zone resource files
> > repeatedly and they forget to close them. If this is done fast enough
> > (faster than the garbage collector reaches those files which get closed
> > when finalized), you get this error.
> > 
> > It might be something else though.
> > 
> > _______________________________________________
> > Swift-user mailing list
> > Swift-user at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-user
> > 
> >   
> 
> -- 
> ==================================================
> Ioan Raicu
> Ph.D. Candidate
> ==================================================
> Distributed Systems Laboratory
> Computer Science Department
> University of Chicago
> 1100 E. 58th Street, Ryerson Hall
> Chicago, IL 60637
> ==================================================
> Email: iraicu at cs.uchicago.edu
> Web:   http://www.cs.uchicago.edu/~iraicu
> http://dev.globus.org/wiki/Incubator/Falkon
> http://www.ci.uchicago.edu/wiki/bin/view/VDS/DslCS
> ==================================================
> ==================================================
>