[Swift-devel] CDM tests on OSG

[Allan Espinosa]

Hi,

Here's the result of testing CDM features on two OSG sites.

the policy file
rule .*TEST_f[x|y]_644.sgt
DIRECT /osg/storage/data/engage/scec/SgtFiles/TEST
rule .*TEST_f[x|y]_644.sgt
DIRECT /panfs/panasas/CMS/data/engage/scec/SgtFiles/TEST
rule .*/[0-9]+/[0-9]+/.*.txt.variation.*
DIRECT /osg/storage/data/engage/scec/RuptureVariations
rule .*/[0-9]+/[0-9]+/.*.txt.variation.*
DIRECT /panfs/panasas/CMS/data/engage/scec/RuptureVariations
rule .* DEFAULT


sample wrapper log on host HARVARD:
Progress  2010-05-04 15:55:55.081637000-0400  LOG_START

_____________________________________________________________________________

        Wrapper
_____________________________________________________________________________

Progress  2010-05-04 15:55:55.086896000-0400
SOURCE_CDM_LIB /osg/storage/data/engage/swift/postproc-tw
Job directory mode is: link on shared filesystem
DIR=jobs/d/extract-d5i5khrj
EXEC=/osg/storage/app/engage/JBSim3d/bin/jbsim3d
STDIN=
STDOUT=stdout.txt
STDERR=stderr.txt
DIRS=TEST|158/8|panfs/panasas/CMS/data/engage/swift/158/8
INF=TEST/TEST_fy_644.sgt|TEST/TEST_fx_644.sgt|
158/8/158_8.txt.variation-s0001-h0000
OUTF=panfs/panasas/CMS/data/engage/swift/158/8/TEST_158_8_subfy.sgt|
panfs/panasas/CMS/data/engage/swift
KICKSTART=
CDM_FILE=shared/fs.data
ARGS=stat=TEST extract_sgt=1 slon=-118.286 slat=34.0192
rupmodfile=158/8/158_8.txt.variation-s0001-h000
ARGC=9
Progress  2010-05-04 15:55:55.089785000-0400  CREATE_JOBDIR
Created job directory: jobs/d/extract-d5i5khrj
Progress  2010-05-04 15:55:55.092773000-0400  CREATE_INPUTDIR
Created output directory: jobs/d/extract-d5i5khrj/TEST
Created output directory: jobs/d/extract-d5i5khrj/158/8
Created output directory:
jobs/d/extract-d5i5khrj/panfs/panasas/CMS/data/engage/swift/158/8
Progress  2010-05-04 15:55:55.100016000-0400  LINK_INPUTS
CDM_POLICY: TEST/TEST_fy_644.sgt ->
DIRECT /panfs/panasas/CMS/data/engage/scec/SgtFiles
CDM_ACTION: jobs/d/extract-d5i5khrj INPUT TEST/TEST_fy_644.sgt
DIRECT /panfs/panasas/CMS/data/engage/sc
CDM[DIRECT]: Linking
to /panfs/panasas/CMS/data/engage/scec/SgtFiles/TEST/TEST_fy_644.sgt via
jobs/d/ex
CDM[DIRECT]: /panfs/panasas/CMS/data/engage/scec/SgtFiles/TEST/TEST_fy_644.sgt does not exist!

_____________________________________________________________________________


It cannot find the *.sgt files because it followed the first rule that
matched.  

wrapper log on FIREFLY:
Progress  2010-05-04 14:55:48.953957000-0500  LOG_START

_____________________________________________________________________________

        Wrapper
_____________________________________________________________________________

Progress  2010-05-04 14:55:49.219389000-0500
SOURCE_CDM_LIB /panfs/panasas/CMS/data/engage/swift/postp
Job directory mode is: link on shared filesystem
DIR=jobs/5/extract-55i5khrj
EXEC=/panfs/panasas/CMS/app/engage/JBSim3d/bin/jbsim3d
STDIN=
STDOUT=stdout.txt
STDERR=stderr.txt
DIRS=TEST|158/5|panfs/panasas/CMS/data/engage/swift/158/5
INF=TEST/TEST_fy_644.sgt|TEST/TEST_fx_644.sgt|
158/5/158_5.txt.variation-s0001-h0000
OUTF=panfs/panasas/CMS/data/engage/swift/158/5/TEST_158_5_subfy.sgt|
panfs/panasas/CMS/data/engage/swift
KICKSTART=
CDM_FILE=shared/fs.data
ARGS=stat=TEST extract_sgt=1 slon=-118.286 slat=34.0192
rupmodfile=158/5/158_5.txt.variation-s0001-h000
ARGC=9
Progress  2010-05-04 14:55:49.543697000-0500  CREATE_JOBDIR
Created job directory: jobs/5/extract-55i5khrj
Progress  2010-05-04 14:55:51.058719000-0500  CREATE_INPUTDIR
Created output directory: jobs/5/extract-55i5khrj/TEST
Created output directory: jobs/5/extract-55i5khrj/158/5
Created output directory:
jobs/5/extract-55i5khrj/panfs/panasas/CMS/data/engage/swift/158/5
Progress  2010-05-04 14:55:51.141540000-0500  LINK_INPUTS
CDM_POLICY: TEST/TEST_fy_644.sgt ->
DIRECT /panfs/panasas/CMS/data/engage/scec/SgtFiles
CDM_ACTION: jobs/5/extract-55i5khrj INPUT TEST/TEST_fy_644.sgt
DIRECT /panfs/panasas/CMS/data/engage/sc
CDM[DIRECT]: Linking
to /panfs/panasas/CMS/data/engage/scec/SgtFiles/TEST/TEST_fy_644.sgt via
jobs/5/ex
CDM_POLICY: TEST/TEST_fx_644.sgt ->
DIRECT /panfs/panasas/CMS/data/engage/scec/SgtFiles
CDM_ACTION: jobs/5/extract-55i5khrj INPUT TEST/TEST_fx_644.sgt
DIRECT /panfs/panasas/CMS/data/engage/sc
CDM[DIRECT]: Linking
to /panfs/panasas/CMS/data/engage/scec/SgtFiles/TEST/TEST_fx_644.sgt via
jobs/5/ex
Could not locate input file: 158/5/158_5.txt.variation-s0001-h0000


In this job, I have two input files.  since it already found the first
one, it did not go to the other input files which also has a DIRECT
policy.