[Swift-devel] jobchart

[Mihael Hategan]

On Sun, 2007-03-11 at 09:57 -0500, Ian Foster wrote:
> Mihael:
> 
> Is it easy to do a version of this that shows what each specific CPU
> is doing?

This is all running locally. One CPU for Swift and /bin/sleep. It's not
really representative of "real" workflow runs. I was only trying to test
"jobchart". It's intriguing nonetheless :)

>  That would be helpful in terms of understanding bottlenecks. I guess
> it is difficult when a separate GRAM submission is performed per task.
> It would be easy when using DeeF.
> 
> Ian.
> 
> Mihael Hategan wrote: 
> > I updated it a bit.
> > Two interesting ones (warning: ~700kB files):
> > http://www-unix.mcs.anl.gov/~hategan/helloworld-i4lb1xpvedgs0.log.png
> > 
> > and 
> > 
> > http://www-unix.mcs.anl.gov/~hategan/helloworld-okdn8oj4qg411.log.png
> > 
> > The first one has the gradual throttling disabled. The second one has it
> > set to a low value.
> > Granted, this is running /bin/sleep 2 (ignore the fact that the label
> > says "echo"), but the fact that lack of throttling can cause resource
> > saturation and slightly worse performance is interesting.
> > I still have to figure out what, besides checking the exit code file,
> > causes the long delays after the job is done. I'm guessing it's some CPU
> > intensive stuff that doesn't parallelize very well on my laptop.
> > 
> > Mihael
> > 
> > On Sat, 2007-03-10 at 22:09 -0600, Mike Wilde wrote:
> >   
> > > That is beautiful!  Nice work, Mihael.
> > > 
> > > - Mike
> > > 
> > > Mihael Hategan wrote, On 3/10/2007 7:44 PM:
> > >     
> > > > There's a new tool in bin.
> > > > 
> > > > It's a spin off Jens' "show-id" tool.
> > > > After careful analysis of show-id, it became apparent that a lot of the
> > > > difficulty was in gathering and organizing the data, rather than in
> > > > generating the plots. This one's written in python and lacks the command
> > > > line options to control sizes, but includes the logic in Jens' tool that
> > > > automatically scale things.
> > > > 
> > > > It does not show individual stage-ins and stage-outs. I'll have to think
> > > > of a way to represent those on the plot without making it messy.
> > > > It needs the logs to contain debugging info from individual tasks:
> > > > log4j.logger.org.globus.cog.abstraction.impl.common.task.TaskImpl=DEBUG
> > > > 
> > > > I've updated this in SVN, but if you want to run it on older builds, you
> > > > need the above in log4j.properties.
> > > > 
> > > > I attached a sample output.
> > > > 
> > > > Mihael
> > > > 
> > > > 
> > > > ------------------------------------------------------------------------
> > > > 
> > > > _______________________________________________
> > > > Swift-devel mailing list
> > > > Swift-devel at ci.uchicago.edu
> > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > >       
> > 
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > 
> >   
> 
> -- 
> 
>    Ian Foster, Director, Computation Institute
> Argonne National Laboratory & University of Chicago
> Argonne: MCS/221, 9700 S. Cass Ave, Argonne, IL 60439
> Chicago: Rm 405, 5640 S. Ellis Ave, Chicago, IL 60637
> Tel: +1 630 252 4619.  Web: www.ci.uchicago.edu.
>       Globus Alliance: www.globus.org.