[Swift-devel] Request: doStage(In|Out) plots colored per site

[Allan Espinosa]

The line "log(LOG:DEBUG, "THREAD_ASSOCIATION jobid={jobid} thread={#thread}
host={rhost}" replicationGroup={replicationGroup}")  in vdl-int.k is inside an
execute2 element so the log-to-dostagein-transitions script ignores this.  

After poking around, I'm going to test the modification below:

diff --git a/libexec/log-processing/log-to-dostagein-transitions b/libexec/log-p
index 16d0547..7e42bf6 100755
--- a/libexec/log-processing/log-to-dostagein-transitions
+++ b/libexec/log-processing/log-to-dostagein-transitions
@@ -9,10 +9,11 @@
 
 # 1193169216.993 DEBUG vdl:dostagein FILE_STAGE_IN_START file=file://localhost/
 
-grep ' vdl:dostagein ' | iso-to-secs | \
+perl -wn -e '/vdl:dostagein|vdl:execute2/ and print' | iso-to-secs | \
 grep -E '^[^ ]+ +[^ ]+ +vdl:dostagein ' | \
 sed 's/\([^ ]*\) [^ ]* *vdl:dostagein FILE_STAGE_IN_.*$//' | \
-sed 's/\([^ ]*\) INFO  vdl:dostagein \([^ ]*\) jobid=\([^ ]*\) -.*$/\1 \3 \2/' 
+sed 's/^\(.*\) DEBUG vdl:execute2 THREAD_ASSOCIATION jobid=\([^ ]*\) thread=\([
+sed 's/\([^ ]*\) INFO  vdl:dostagein \([^ ]*\) jobid=\([^ ]*\) host=\([^ ]*\) -
 grep -E '^[^$]'
 
 exit 0

Take note that I have yet to test this :)

-Allan

On Tue, May 11, 2010 at 06:47:37AM +0000, Ben Clifford wrote:
> 
> > I was wondering if there are other solutions to getting site transfer stats
> > aside from this proposed hack like querying provenance information or doing
> > site query per jobid to existing *.event logs so I can reused my old runs.
> 
> I think the jobid in the dostage lines imply the site. They should 
> correspond with the job id in this line:
> 
>                                 log(LOG:DEBUG, "THREAD_ASSOCIATION 
> jobid={jobid} thread={#thread} host={rhost} 
> replicationGroup={replicationGroup}")
> 
> in vdl-int.k
> 
> Something does per-site coloured graphs already but I don't remember what. 
> Whatever that does, you can probably adapt to this.
> 
> -- 
> 
>