[Swift-devel] Swift and BGP plots

[Ben Clifford]

you can use swift-plot-log ot do that

swift-plot-log my.log karatasks.transitions

or something like that.

don't reimplement all of that yourself.

On Tue, 27 Oct 2009, Ioan Raicu wrote:

> OK, so this looks to be a bit more complex then. Can you suggest a way to
> extract the Active and Complete timestamps (along with the task ID)? It seems
> that if I grep for JOB_SUBMISSION, the status Active is not found on the same
> line, so I can't filter according to those two keywords. Also, the Completed
> keyword also comes up with many other hits. Any ideas on how to get the info I
> need (with cat, grep, etc) without writing some custom script/program that
> actually understands in depth the logging of the Swift log?
> 
> Thanks,
> Ioan
> 
> Mihael Hategan wrote:
> > On Tue, 2009-10-27 at 14:17 -0500, Ioan Raicu wrote:
> >   
> > > OK, I converted the logs, and here is what I got.
> > > 
> > > The thing that bugs me is that things ran faster (per task) at the
> > > larger scale (a bit counter intuitive).
> > >     
> > 
> > Nope. Same workload but more processors. 
> >   
> > >  Also, the maximum number of concurrent tasks I found was 20K, in both
> > > experiments. This doesn't seem right, as the experiments should have
> > > 4K or 6K active tasks at any time.
> > >     
> > 
> > Right. You should see around 4k or 6k max active tasks.
> > 
> >   
> > >  Could I be looking at the wrong entries from the Swift logs?
> > > 
> > > I did this to get the data I needed from the Swift logs:
> > > cat dc-4000.log | grep "JOB_START" > dc-4000-start-end.txt
> > > cat dc-4000.log | grep "JOB_END" >> dc-4000-start-end.txt
> > > 
> > > Is this correct? Or should I be looking for other job events?
> > >     
> > 
> > You should probably be looking for "Task(type=JOB_SUBMISSION...) setting
> > status to Active" and "Task(type=JOB_SUBMISSION...) setting status to
> > Completed". Or something like that.
> > 
> > JOB_START and JOB_END are statuses for the Swift execute2 processes.
> > 
> > 
> >   
> 
>