[Swift-devel] Re: Swift log analysis tools

[Jonathan Monette]

On Jun 17, 2011, at 2:04 PM, Michael Wilde wrote:

> 
> In discussing the SCEC workflow with Ketan, it became clear that a tool to extract useful debugging information from the Swift run .log file would be very useful in diagnosing failures and understanding better how the run is behaving.
> 
> Ketan is about to make a few log filter scripts to tease out and format this info.
> 
> Does anyone have any similar tools that wouldbe useful for this purpose?

The only script I have is one that counts the number of times and an app has been completed.  I think it may also check to see how many times certain files have been staged in/out successfully but not sure on that one.  It is just  a one line script of a common grep line that I do.

> 
> Do the new (or old) log plotting tools have and preprocessing scripts that can be used for this?

I am not sure how well the old tool work and currently there is not a lot of new tools.

> 
> Should we common scripts to reformat the log for both human analysis and erformance plotting?

I think this would be very helpful for both debuging and plotting purposes.  I normally just eyeball the log file and see if anything funky is happening but for large workflows that is very difficult.  Scripts that pull out useful information would be helpful.  But the logging levels that they need turned on should be well documented in a place that is easily found.

> 
> - Mike