[Swift-commit] r5000 - in branches/release-0.93: docs docs/log-processing libexec/log-processing

Wed Aug 24 15:29:12 CDT 2011

Author: davidk
Date: 2011-08-24 15:29:12 -0500 (Wed, 24 Aug 2011)
New Revision: 5000

Added:
   branches/release-0.93/docs/log-processing/
   branches/release-0.93/docs/log-processing/log-processing.txt
   branches/release-0.93/libexec/log-processing/README.txt
Removed:
   branches/release-0.93/libexec/log-processing/README.txt
Log:
Add log processing docs to the nightly build process


Added: branches/release-0.93/docs/log-processing/log-processing.txt
===================================================================

--- branches/release-0.93/docs/log-processing/log-processing.txt	                        (rev 0)
+++ branches/release-0.93/docs/log-processing/log-processing.txt	2011-08-24 20:29:12 UTC (rev 5000)
@@ -0,0 +1,104 @@
+
+Log Processing
+==============
+
+To properly generate log plots, you must enable VDL/Karajan logging.
+TODO:How?
+
+
+You should check the scripts that you intend to use to determine
+what log lines they require and ensure that you are generating
+those lines via log4j.properties
+
+Make sure log4.properties contains:
+--------------------------------------
+log4j.logger.swift=DEBUG
+log4j.logger.org.globus.cog.abstraction.coaster.service.job.manager.Cpu=DEBUG
+log4j.logger.org.globus.cog.abstraction.coaster.service.job.manager.Block=DEBUG
+--------------------------------------
+TODO: Does it work for coasters-based runs only?
+
+Normalize event times in the log to the run start time
+------------------------------------------------------
+
+* Generate the log, assuming the log is titled +swift-run.log+
+
+------------------------------------------
+./normalize-log.pl file.contains.start.time swift-run.log > swift-run.norm
+------------------------------------------
+TODO:In what format does the start time be in 'file.contains.start.time'
+
+
+Make a basic load plot from Coasters Cpu log lines
+--------------------------------------------------
+
+. Normalize the log.
+. Build up a load data file:
++
+------------------------------------------
+./cpu-job-load.pl < swift-run.norm > load.data
+------------------------------------------
+. Plot with the JFreeChart-based plotter in usertools/plotter:
++
+------------------------------------------
+swift_plotter.zsh -s load.cfg load.eps load.data
+------------------------------------------
+Note: Th load.cfg is available from swift/libexec/log-processing/
+
+
+Make a basic job completion plot from Coasters Cpu log lines
+------------------------------------------------------------
+
+. Normalize the log.
+
+. Build up a completed data file:
++
+------------------------------------------
+./cpu-job-completed.pl < swift-run.norm > completed.data
+------------------------------------------
+
+. Plot with the JFreeChart-based plotter in usertools/plotter:
++
+------------------------------------------
+swift_plotter.zsh -s completed.cfg completed.eps completed.data
+------------------------------------------
+
+Make a basic Block allocation plot from Coasters Block log lines
+----------------------------------------------------------------
+
+. Normalize the log.
+
+. Build up a block allocation data file:
++
+------------------------------------------
+./block-level.pl < swift-run.norm > blocks.data
+------------------------------------------
+
+. Plot with the JFreeChart-based plotter in usertools/plotter:
++
+------------------------------------------
+swift_plotter.zsh -s blocks.{cfg,eps,data}
+------------------------------------------
+
+Make a job runtime distribution plot from Coasters Cpu log lines
+----------------------------------------------------------------
+
+. Normalize the log.
+
+. Build up a job runtime file:
++
+------------------------------------------
+./extract-times.pl < swift-run.norm > times.data
+------------------------------------------
+
+. Put the job runtimes into 1-second buckets:
++
+------------------------------------------
+./ buckets.pl 1 times.data > buckets.data
+------------------------------------------
+
+. Plot with the JFreeChart-based plotter in usertools/plotter:
++
+------------------------------------------
+swift_plotter.zsh -s buckets.cfg buckets.eps buckets.data
+------------------------------------------

Deleted: branches/release-0.93/libexec/log-processing/README.txt
===================================================================
--- branches/release-0.93/libexec/log-processing/README.txt	2011-08-24 16:16:39 UTC (rev 4999)
+++ branches/release-0.93/libexec/log-processing/README.txt	2011-08-24 20:29:12 UTC (rev 5000)
@@ -1,104 +0,0 @@
-
-Log Processing
-==============
-
-To properly generate log plots, you must enable VDL/Karajan logging.
-TODO:How?
-
-
-You should check the scripts that you intend to use to determine
-what log lines they require and ensure that you are generating
-those lines via log4j.properties
-
-Make sure log4.properties contains:
---------------------------------------
-log4j.logger.swift=DEBUG
-log4j.logger.org.globus.cog.abstraction.coaster.service.job.manager.Cpu=DEBUG
-log4j.logger.org.globus.cog.abstraction.coaster.service.job.manager.Block=DEBUG
---------------------------------------
-TODO: Does it work for coasters-based runs only?
-
-Normalize event times in the log to the run start time
-------------------------------------------------------
-
-* Generate the log, assuming the log is titled +swift-run.log+
-
-------------------------------------------
-./normalize-log.pl file.contains.start.time swift-run.log > swift-run.norm
-------------------------------------------
-TODO:In what format does the start time be in 'file.contains.start.time'
-
-
-Make a basic load plot from Coasters Cpu log lines
---------------------------------------------------
-
-. Normalize the log.
-. Build up a load data file:
-+
-------------------------------------------
-./cpu-job-load.pl < swift-run.norm > load.data
-------------------------------------------
-. Plot with the JFreeChart-based plotter in usertools/plotter:
-+
-------------------------------------------
-swift_plotter.zsh -s load.cfg load.eps load.data
-------------------------------------------
-Note: Th load.cfg is available from swift/libexec/log-processing/
-
-
-Make a basic job completion plot from Coasters Cpu log lines
-------------------------------------------------------------
-
-. Normalize the log.
-
-. Build up a completed data file:
-+
-------------------------------------------
-./cpu-job-completed.pl < swift-run.norm > completed.data
-------------------------------------------
-
-. Plot with the JFreeChart-based plotter in usertools/plotter:
-+
-------------------------------------------
-swift_plotter.zsh -s completed.cfg completed.eps completed.data
-------------------------------------------
-
-Make a basic Block allocation plot from Coasters Block log lines
-----------------------------------------------------------------
-
-. Normalize the log.
-
-. Build up a block allocation data file:
-+
-------------------------------------------
-./block-level.pl < swift-run.norm > blocks.data
-------------------------------------------
-
-. Plot with the JFreeChart-based plotter in usertools/plotter:
-+
-------------------------------------------
-swift_plotter.zsh -s blocks.{cfg,eps,data}
-------------------------------------------
-
-Make a job runtime distribution plot from Coasters Cpu log lines
-----------------------------------------------------------------
-
-. Normalize the log.
-
-. Build up a job runtime file:
-+
-------------------------------------------
-./extract-times.pl < swift-run.norm > times.data
-------------------------------------------
-
-. Put the job runtimes into 1-second buckets:
-+
-------------------------------------------
-./ buckets.pl 1 times.data > buckets.data
-------------------------------------------
-
-. Plot with the JFreeChart-based plotter in usertools/plotter:
-+
-------------------------------------------
-swift_plotter.zsh -s buckets.cfg buckets.eps buckets.data
-------------------------------------------

Added: branches/release-0.93/libexec/log-processing/README.txt
===================================================================
--- branches/release-0.93/libexec/log-processing/README.txt	                        (rev 0)
+++ branches/release-0.93/libexec/log-processing/README.txt	2011-08-24 20:29:12 UTC (rev 5000)
@@ -0,0 +1 @@
+link ../../docs/log-processing/log-processing.txt
\ No newline at end of file


Property changes on: branches/release-0.93/libexec/log-processing/README.txt
___________________________________________________________________
Added: svn:special
   + *


