[Swift-devel] finding the execution sites from swift logs

[Allan Espinosa]

Make sure you have log4j.logger.swift=DEBUG when you run jobs.

-Allan

2011/6/30 Ketan Maheshwari <ketancmaheshwari at gmail.com>:
> Thanks Allan.
>
>  In some log files, I do not see any execute2 lines. However, I do see
> JOB_START lines ending with host=<hostname>. I confirmed that the logs
> indeed belong to successful complete runs. While in other logs corresponding
> to other successful runs, I do see the execute2 lines.
>
> All the runs were carried out using the same version of Swift. Am I missing
> something here?
>
> Ketan
>
>
> On Thu, Jun 30, 2011 at 11:45 AM, Allan Espinosa <aespinosa at cs.uchicago.edu>
> wrote:
>>
>> Hi Ketan,
>>
>> What I do is match the jobnames with the sites by matching execute2
>> log information.  I have a set of R and ruby scripts in
>> ~aespinosa/Documents/swift that analyzes the number of transfers per
>> site.  You can modify them to look for job execution.
>>
>> If you look at one of the makefile targets in libexec/log-processing,
>> you can find files with the name 'color'.  these targets to swift plot
>> logs colors the plot per site.
>>
>> To just explore the general statistics, I obtain the execute2.event
>> file from the log using
>>
>> $swift-plot-log logfile execute2.event
>>
>> And then just use R to analyze the *.event file.
>>
>> 2011/6/30 Ketan Maheshwari <ketancmaheshwari at gmail.com>:
>> > Hello,
>> >
>> > Does anyone knows from swift log, how to find how many jobs executed on
>> > a
>> > given site when there is a mix of localhost and osg sites?
>> >
>> > I have logs of many runs each with about 3400 app tasks ran on localhost
>> > +
>> > osg sites.
>> >
>> > Trying to look into log and find that there are multiple messages
>> > corresponding to staging in-out, run and other events (change in score,
>> > thread associations, etc.).
>> >
>> > What should I be looking to identify each job uniquely?
>> >
>> > Thanks for any help on this.
>> >
>> > Regards,
>> > --
>> > Ketan
>> >
>> >
>> >
>> > _______________________________________________
>> > Swift-devel mailing list
>> > Swift-devel at ci.uchicago.edu
>> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>> >
>> >
>>
>>
>>
>> --
>> Allan M. Espinosa <http://amespinosa.wordpress.com>
>> PhD student, Computer Science
>> University of Chicago <http://people.cs.uchicago.edu/~aespinosa>
>
>
>
> --
> Ketan
>
>
>



-- 
Allan M. Espinosa <http://amespinosa.wordpress.com>
PhD student, Computer Science
University of Chicago <http://people.cs.uchicago.edu/~aespinosa>