[Swift-devel] jobchart

[Mihael Hategan]

On Sat, 2007-03-10 at 23:35 -0600, Yong Zhao wrote:
> In Swift there is dir creation, exit code checking, cleanup etc. but that
> is still a lot of extra time (25s vs 3s), kinda weird

Cleanup happens only at the end of the whole workflow. And it's a
handful of jobs. In this case (one site) it's only one rm -rf.

And the scheduler is also not the culprit. If you look at the second
graph, the one with throttling, and if you look at the scheduler job
submit line (black) in bulk submits, you see that it pretty much a
vertical line, which means that the scheduler (having only one thread)
processes the jobs almost instantaneously.

However, it's not the 25s vs. 3s that is concerning, because it's hard
to isolate problems when there are many things happening, but the 5 vs.
2.08 seconds.

> 
> Yong.
> 
> On Sat, 10 Mar 2007, Mihael Hategan wrote:
> 
> > On Sat, 2007-03-10 at 23:16 -0600, Mihael Hategan wrote:
> > > On Sat, 2007-03-10 at 23:14 -0600, Mihael Hategan wrote:
> > > > On Sat, 2007-03-10 at 23:02 -0600, Mihael Hategan wrote:
> > > > > On Sat, 2007-03-10 at 23:02 -0600, Yong Zhao wrote:
> > > > > > I see, so this does not involve stage-in and outs. Why did postprocessing
> > > > > > take that much time in the other two graphs, where in this case it is
> > > > > > minimal?
> > > > >
> > > > > I think that's a cpu that does too many things at once. But I welcome
> > > > > other explanations.
> > > >
> > > > Actually I'm kinda confused. Running bash with 300 parallel "/bin/sleep
> > > > 2" takes 2.4 seconds and doing it with plain Karajan takes 5.
> > >
> > > Also, running 300 parallel wait=2000 in karajan takes 2.08 seconds. So
> > > the bulk must be in the implementation of execute() and/or the local
> > > provider.
> >
> > Note: not counting the jvm startup:
> > print(time(parallelfor(i, range(1, 300), wait(delay=2000)))
> >
> > >
> > > >
> > > > Some thinking needs to happen there.
> > > >
> > > > >
> > > > > >
> > > > > > Yong.
> > > > > >
> > > > > > On Sat, 10 Mar 2007, Mihael Hategan wrote:
> > > > > >
> > > > > > > Preprocessing is the time between "Running job..." log message in
> > > > > > > vdl-int.k and when the task status gets changed to "Submitted".
> > > > > > >
> > > > > > > Postprocessing is between when the task status is changed to "Completed"
> > > > > > > and the "Completed job" log message in vdl-int.k, and includes the check
> > > > > > > for the exit code file.
> > > > > > >
> > > > > > > Mihael
> > > > > > >
> > > > > > > On Sat, 2007-03-10 at 22:51 -0600, Yong Zhao wrote:
> > > > > > > > nice graph, what's preprocessing and postprocessing involved in this case?
> > > > > > > >
> > > > > > > > Yong.
> > > > > > > >
> > > > > > > > On Sat, 10 Mar 2007, Mihael Hategan wrote:
> > > > > > > >
> > > > > > > > > There's a new tool in bin.
> > > > > > > > >
> > > > > > > > > It's a spin off Jens' "show-id" tool.
> > > > > > > > > After careful analysis of show-id, it became apparent that a lot of the
> > > > > > > > > difficulty was in gathering and organizing the data, rather than in
> > > > > > > > > generating the plots. This one's written in python and lacks the command
> > > > > > > > > line options to control sizes, but includes the logic in Jens' tool that
> > > > > > > > > automatically scale things.
> > > > > > > > >
> > > > > > > > > It does not show individual stage-ins and stage-outs. I'll have to think
> > > > > > > > > of a way to represent those on the plot without making it messy.
> > > > > > > > > It needs the logs to contain debugging info from individual tasks:
> > > > > > > > > log4j.logger.org.globus.cog.abstraction.impl.common.task.TaskImpl=DEBUG
> > > > > > > > >
> > > > > > > > > I've updated this in SVN, but if you want to run it on older builds, you
> > > > > > > > > need the above in log4j.properties.
> > > > > > > > >
> > > > > > > > > I attached a sample output.
> > > > > > > > >
> > > > > > > > > Mihael
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > >
> > >
> > > _______________________________________________
> > > Swift-devel mailing list
> > > Swift-devel at ci.uchicago.edu
> > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > >
> >
> >
>