[Swift-devel] finding the execution sites from swift logs

[Ketan Maheshwari]

In addition to JOB_START, vdl:execute2 is associated with many events:

THREAD_ASSOCIATION
APPLICATION_EXCEPTION
STAGING_OUT
..

Ketan

On Thu, Jun 30, 2011 at 5:39 PM, Allan Espinosa
<aespinosa at cs.uchicago.edu>wrote:

> I think JOB_START are the vdl:execute2 ones if i remember correctly.
>
> 2011/6/30 Ketan Maheshwari <ketancmaheshwari at gmail.com>:
> > Thanks Allan.
> >
> >  In some log files, I do not see any execute2 lines. However, I do see
> > JOB_START lines ending with host=<hostname>. I confirmed that the logs
> > indeed belong to successful complete runs. While in other logs
> corresponding
> > to other successful runs, I do see the execute2 lines.
> >
> > All the runs were carried out using the same version of Swift. Am I
> missing
> > something here?
> >
> > Ketan
> >
> >
> > On Thu, Jun 30, 2011 at 11:45 AM, Allan Espinosa <
> aespinosa at cs.uchicago.edu>
> > wrote:
> >>
> >> Hi Ketan,
> >>
> >> What I do is match the jobnames with the sites by matching execute2
> >> log information.  I have a set of R and ruby scripts in
> >> ~aespinosa/Documents/swift that analyzes the number of transfers per
> >> site.  You can modify them to look for job execution.
> >>
> >> If you look at one of the makefile targets in libexec/log-processing,
> >> you can find files with the name 'color'.  these targets to swift plot
> >> logs colors the plot per site.
> >>
> >> To just explore the general statistics, I obtain the execute2.event
> >> file from the log using
> >>
> >> $swift-plot-log logfile execute2.event
> >>
> >> And then just use R to analyze the *.event file.
> >>
> >> 2011/6/30 Ketan Maheshwari <ketancmaheshwari at gmail.com>:
> >> > Hello,
> >> >
> >> > Does anyone knows from swift log, how to find how many jobs executed
> on
> >> > a
> >> > given site when there is a mix of localhost and osg sites?
> >> >
> >> > I have logs of many runs each with about 3400 app tasks ran on
> localhost
> >> > +
> >> > osg sites.
> >> >
> >> > Trying to look into log and find that there are multiple messages
> >> > corresponding to staging in-out, run and other events (change in
> score,
> >> > thread associations, etc.).
> >> >
> >> > What should I be looking to identify each job uniquely?
> >> >
> >> > Thanks for any help on this.
> >> >
> >> > Regards,
> >> > --
> >> > Ketan
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > Swift-devel mailing list
> >> > Swift-devel at ci.uchicago.edu
> >> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Allan M. Espinosa <http://amespinosa.wordpress.com>
> >> PhD student, Computer Science
> >> University of Chicago <http://people.cs.uchicago.edu/~aespinosa>
> >
> >
> >
> > --
> > Ketan
> >
> >
> >
>
>
>
> --
> Allan M. Espinosa <http://amespinosa.wordpress.com>
> PhD student, Computer Science
> University of Chicago <http://people.cs.uchicago.edu/~aespinosa>
>



-- 
Ketan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110630/168f2751/attachment.html>