[Swift-devel] swift-plot-log parsing errors

[Allan Espinosa]

Hi,

I get these parsing errors:

$swift-plot-log ~aespinosa/workflows/cybershake/postproc-3sites_TEST.log
...
...
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
(standard_in) 1: parse error
kickstarts-to-event > kickstart.event
execution-summaries > execution-counts.txt
table-jobs-sites > jobs-sites.html
per-site-execute2-durations > site-duration.txt
cat execute2.transitions | swap-and-sort |last-transition-line  > execute2.last
cat execute2.last | sed 's/^\([^ ]*\) \([^ ]*\) \([^ ]*\)\(.*\)/\3/' | sort | uniq -c > execute2.lastsummary
cat execute.transitions | swap-and-sort |last-transition-line  > execute.last 
cat execute.last | sed 's/^\([^ ]*\) \([^ ]*\) \([^ ]*\)\(.*\)/\3/' | sort | uniq -c > execute.lastsummary
cat execute2.event  | cut -f 5 -d ' ' | sort | uniq -c | sort | sed 's/^ *\(.*\) .*$/\1/' | uniq -c > jobs.retrycount.summary
cat execute.event | cut -f 5 -d ' ' | sort  | uniq -c > trname-summary
m4 -I/home/aespinosa/swift/current/bin/../libexec/log-processing/ /home/aespinosa/swift/current/bin/../libexec/log-processing//index.html.template > index.html
m4 -I/home/aespinosa/swift/current/bin/../libexec/log-processing/ /home/aespinosa/swift/current/bin/../libexec/log-processing//execute2.html.template > execute2.html

m4 -I/home/aespinosa/swift/current/bin/../libexec/log-processing/ /home/aespino
sa/swift/current/bin/../libexec/log-processing//execute.html.template > execute
.html
m4 -I/home/aespinosa/swift/current/bin/../libexec/log-processing/ /home/aespino
sa/swift/current/bin/../libexec/log-processing//karajan.html.template > karajan
.html
m4 -I/home/aespinosa/swift/current/bin/../libexec/log-processing/ /home/aespino
sa/swift/current/bin/../libexec/log-processing//info.html.template > info.html
m4 -I/home/aespinosa/swift/current/bin/../libexec/log-processing/ /home/aespino
sa/swift/current/bin/../libexec/log-processing//assorted.html.template > assort
ed.html
m4 -I/home/aespinosa/swift/current/bin/../libexec/log-processing/ /home/aespino
sa/swift/current/bin/../libexec/log-processing//kickstart.html.template > kicks
tart.html
/home/aespinosa/swift/current/bin/../libexec/log-processing//kickstart.html.tem
plate:23: m4: Cannot open kickstart.stats: No such file or directory
m4 -I/home/aespinosa/swift/current/bin/../libexec/log-processing/ /home/aespino
sa/swift/current/bin/../libexec/log-processing//falkon.html.template > falkon.h
tml
m4 -I/home/aespinosa/swift/current/bin/../libexec/log-processing/ /home/aespinosa/swift/current/bin/../libexec/log-processing//scheduler.html.template > scheduler.html
m4 -I/home/aespinosa/swift/current/bin/../libexec/log-processing/ /home/aespinosa/swift/current/bin/../libexec/log-processing//coasters.html.template > coasters.html
rm karatasks.FILE_OPERATION.sorted-start.event karatasks.JOB_SUBMISSION.Active.event karatasks.JOB_SUBMISSION.Queue.sorted-start.event karatasks.last dostageout.sorted-start.event karatasks.JOB_SUBMISSION.seenstates karatasks.FILE_TRANSFER.sorted-start.event execute.last karatasks.JOB_SUBMISSION.Queue.eip initshareddir.event karatasks.JOB_SUBMISSION.Active.sorted-start.event execute.seenstates karatasks.FILE_OPERATION.seenstates karatasks.JOB_SUBMISSION.eip karatasks.JOB_SUBMISSION.Queue.sorted-by-duration karatasks.FILE_OPERATION.eip createdirset.event karatasks.JOB_SUBMISSION.event dostageout.sorted-by-duration karatasks.FILE_TRANSFER.seenstates karatasks.FILE_OPERATION.event karatasks.JOB_SUBMISSION.Queue.event execute2.seenstates karatasks.FILE_TRANSFER.eip dostagein.sorted-by-duration karatasks.JOB_SUBMISSION.Active.sorted-by-duration karatasks.JOB_SUBMISSION.Active.eip karatasks.FILE_TRANSFER.event execute2.last execute.sorted-start.event karatasks.JOB_SUBMISSION.sorted-start.event
cp: cannot stat `clean': No such file or directory
cp: cannot stat `webpage.kara': No such file or directory
cp: cannot stat `webpage': No such file or directory

I was thinking it was because of some swift.properties that we don't typically
use.:

$ diff /home/aespinosa/swift/current/etc/swift.properties  swift.properties 
44c44
< lazy.errors=false
---
> lazy.errors=true
99c99
< clustering.enabled=false
---
> clustering.enabled=true
114c114
< clustering.min.time=60
---
> clustering.min.time=360
152c152
< wrapperlog.always.transfer=false
---
> wrapperlog.always.transfer=true
241c241
< sitedir.keep=false
---
> sitedir.keep=true
247c247
< execution.retries=2
---
> execution.retries=0
263c263
< replication.min.queue.time=60
---
> replication.min.queue.time=180
322c322
< foreach.max.threads=16384
---
> foreach.max.threads=1024
352c352
< use.provider.staging=false
\ No newline at end of file
---
> use.provider.staging=false

I enabled job replicas as well before but disabled them back again so I can
isolate what's really going wrong.

-Allan