[Swift-devel] Request: doStage(In|Out) plots colored per site

[Allan Espinosa]

Ok that didn't work.  

But i think i got it this time:

diff --git a/libexec/log-processing/log-to-dostagein-transitions b/libexec/log-p
index 16d0547..b82be5f 100755
--- a/libexec/log-processing/log-to-dostagein-transitions
+++ b/libexec/log-processing/log-to-dostagein-transitions
@@ -9,8 +9,9 @@
 
 # 1193169216.993 DEBUG vdl:dostagein FILE_STAGE_IN_START file=file://localhost/
 
-grep ' vdl:dostagein ' | iso-to-secs | \
-grep -E '^[^ ]+ +[^ ]+ +vdl:dostagein ' | \
+grep -E ' (vdl:dostagein|vdl:execute2) ' | iso-to-secs | \
+grep -E '^[^ ]+ +[^ ]+ +(vdl:dostagein |vdl:execute2 THREAD_ASSOCIATION)' | \
+sed 's/^\(.*\) DEBUG vdl:execute2 THREAD_ASSOCIATION jobid=\([^ ]*\) thread=\([
 sed 's/\([^ ]*\) [^ ]* *vdl:dostagein FILE_STAGE_IN_.*$//' | \
 sed 's/\([^ ]*\) INFO  vdl:dostagein \([^ ]*\) jobid=\([^ ]*\) -.*$/\1 \3 \2/' 
 grep -E '^[^$]'

sample output:

$ tail dostagein.event
1269611776.560 0.0420000553131104 surfeis_rspectra-j8pe0mpj END 0-10-7-13-1-4-1 FIREFLY y7pe0mpj
1269611811.671 0.0469999313354492 surfeis_rspectra-k8pe0mpj END 0-10-7-13-2-4-1 FIREFLY 08pe0mpj
1269611811.676 0.0789999961853027 surfeis_rspectra-l8pe0mpj END 0-10-5-13-1-4-1 FIREFLY 28pe0mpj
1269611823.316 0.065000057220459 surfeis_rspectra-m8pe0mpj END 0-10-9-13-2-4-1 FIREFLY 48pe0mpj
1269611867.418 0.0410001277923584 surfeis_rspectra-n8pe0mpj END 0-10-5-13-2-4-1 FIREFLY 98pe0mpj
1269611871.688 0.0490000247955322 surfeis_rspectra-o8pe0mpj END 0-10-9-13-1-4-1 FIREFLY b8pe0mpj
1269611871.689 0.0870001316070557 surfeis_rspectra-p8pe0mpj END 0-10-2-13-1-4-1 FIREFLY f8pe0mpj
1269611871.696 0.122999906539917 surfeis_rspectra-q8pe0mpj END 0-10-2-13-2-4-1 FIREFLY h8pe0mpj
1269611031.646 0.064000129699707 surfeis_rspectra-v7pe0mpj END 0-10-6-13-2-4-1 FIREFLY 67pe0mpj
1269611038.059 0.0799999237060547 surfeis_rspectra-z7pe0mpj END 0-10-6-13-1-4-1 FIREFLY 87pe0mpj

Site info at the second to the last column. yay! :)

-Allan


On Tue, May 11, 2010 at 08:00:00PM -0500, Allan Espinosa wrote:
> The line "log(LOG:DEBUG, "THREAD_ASSOCIATION jobid={jobid} thread={#thread}
> host={rhost}" replicationGroup={replicationGroup}")  in vdl-int.k is inside an
> execute2 element so the log-to-dostagein-transitions script ignores this.  
> 
> After poking around, I'm going to test the modification below:
> 
> diff --git a/libexec/log-processing/log-to-dostagein-transitions b/libexec/log-p
> index 16d0547..7e42bf6 100755
> --- a/libexec/log-processing/log-to-dostagein-transitions
> +++ b/libexec/log-processing/log-to-dostagein-transitions
> @@ -9,10 +9,11 @@
>  
>  # 1193169216.993 DEBUG vdl:dostagein FILE_STAGE_IN_START file=file://localhost/
>  
> -grep ' vdl:dostagein ' | iso-to-secs | \
> +perl -wn -e '/vdl:dostagein|vdl:execute2/ and print' | iso-to-secs | \
>  grep -E '^[^ ]+ +[^ ]+ +vdl:dostagein ' | \
>  sed 's/\([^ ]*\) [^ ]* *vdl:dostagein FILE_STAGE_IN_.*$//' | \
> -sed 's/\([^ ]*\) INFO  vdl:dostagein \([^ ]*\) jobid=\([^ ]*\) -.*$/\1 \3 \2/' 
> +sed 's/^\(.*\) DEBUG vdl:execute2 THREAD_ASSOCIATION jobid=\([^ ]*\) thread=\([
> +sed 's/\([^ ]*\) INFO  vdl:dostagein \([^ ]*\) jobid=\([^ ]*\) host=\([^ ]*\) -
>  grep -E '^[^$]'