[Swift-devel] Notes for log processing

[Jonathan Monette]

In Mike's example….how was the *96.data files made?  Did you mean maybe *.norm?

On Sep 22, 2011, at 11:05 AM, Justin M Wozniak wrote:

> 
> An asciidoc-rendered version of the plotter README is up at:
> 
> http://www.mcs.anl.gov/~wozniak/plotter-readme.html
> 
> On Thu, 22 Sep 2011, Michael Wilde wrote:
> 
>> I wanted to do some basic plots to better understand the performance and bottlenecks of a Swift script. I started with this doc on log processing:
>> 
>> http://www.ci.uchicago.edu/swift/wwwdev/guides/trunk/userguide/userguide.html#_log_processing
>> 
>> Here's what I had to do to generate a load plot of my Swift run. Hopefully this will help in documenting how to use the new Java plotting configs now, as well as packaging them for easy use with higher level scripts.
>> 
>> - Mike
>> 
>> 
>> # Point to the log processing tools
>> 
>> lp=/homes/wilde/swift/src/0.93/cog/modules/swift/dist/swift-svn/libexec/log-processing
>> 
>> # fix normalize-log.pl: FH -> START
>> 
>> # find the start time of the log in decimal unixtime
>> 
>> $ $lp/normalize-log.pl stime amwg_stats-20110922-0541-ii4a5z96.iso >amwg_stats-20110922-0541-ii4a5z96.norm
>> 
>> # convert ISO time to UNIX time in decimal seconds
>> 
>> $ $lp/iso-to-secs <amwg_stats-20110922-0541-ii4a5z96.log >amwg_stats-20110922-0541-ii4a5z96.iso
>> 
>> # Normalize the log to start at time 0.0
>> 
>> $ $lp/normalize-log.pl stime amwg_stats-20110922-0541-ii4a5z96.iso >amwg_stats-20110922-0541-ii4a5z96.norm
>> 
>> This gives, eg:
>> 
>> 0 DEBUG Loader arguments: [-config, cf.properties, ...
>> 0.00199985504150391 DEBUG Loader Max heap: 257294336
>> 0.0199999809265137 DEBUG textfiles BEGIN CDM FILE:
>> 0.0199999809265137 DEBUG textfiles END CDM FILE:
>> 0.622999906539917 DEBUG textfiles BEGIN SWIFTSCRIPT:
>> 0.623999834060669 DEBUG textfiles END SWIFTSCRIPT:
>> 
>> # Install the new plotter tools
>> 
>> $ svn co https://svn.ci.uchicago.edu/svn/vdl2/usertools/plotter
>> U   plotter
>> Checked out revision 5151.
>> fusion$ cd plotter
>> fusion$ ant
>> Buildfile: build.xml
>> 
>> compile:
>>   [javac] Compiling 7 source files to /fusion/gpfs/home/wilde/amwg/run01/plotter/build
>> 
>> jar:
>>     [jar] Building jar: /fusion/gpfs/home/wilde/amwg/run01/plotter/lib/plotter.jar
>> 
>> BUILD SUCCESSFUL
>> Total time: 2 seconds
>> 
>> 
>> # Edit load.fg (or change your plot files to match it)
>> 
>> # cp $lp/load.cfg . # then edit to contain:
>> 
>> xlabel = time
>> ylabel = load
>> 
>> shape.amwg_stats-20110922-0541-ii4a5z96.load.data = none
>> label.amwg_stats-20110922-0541-ii4a5z96.load.data = load
>> 
>> # (was eg: label.load.data = load)
>> 
>> # Generate a Load plot:
>> 
>> $ ./plotter/swift_plotter.zsh -s ./load.cfg load.eps *96.load.data
>> 
> 
> -- 
> Justin M Wozniak
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel