[Swift-devel] CDM tests on OSG

[Allan Espinosa]

I was initially thinking of having the policy description inside the
<pool> handle.  But having pool dependent entries on fs.data should work
as well.

Thanks Justin!
-Allan

On Mar, 2010-05-04 at 15:52 -0500, Justin M Wozniak wrote:
> Ok, I think the big picture here is that you want to match on not just the 
> file name but also the job location.  That's something I have not totally 
> thought through but now's a great opportunity to do that.
> 
> What if the pattern match syntax also took into account the pool handle 
> from sites.xml?  Would that do the job?
> 
> On Tue, 4 May 2010, Allan Espinosa wrote:
> 
> > Oh, i placed the wrong policy on my *variation* files. I fixed them all
> > jobs in FIREFLY executed successfully.  However, the problem at the
> > HARVARD site still remain.
> >
> > -Allan
> >
> > On Mar, 2010-05-04 at 15:05 -0500, Allan Espinosa wrote:
> >> Hi,
> >>
> >> Here's the result of testing CDM features on two OSG sites.
> >>
> >> the policy file
> >> rule .*TEST_f[x|y]_644.sgt
> >> DIRECT /osg/storage/data/engage/scec/SgtFiles/TEST
> >> rule .*TEST_f[x|y]_644.sgt
> >> DIRECT /panfs/panasas/CMS/data/engage/scec/SgtFiles/TEST
> >> rule .*/[0-9]+/[0-9]+/.*.txt.variation.*
> >> DIRECT /osg/storage/data/engage/scec/RuptureVariations
> >> rule .*/[0-9]+/[0-9]+/.*.txt.variation.*
> >> DIRECT /panfs/panasas/CMS/data/engage/scec/RuptureVariations
> >> rule .* DEFAULT
> >>
> >>
> >> sample wrapper log on host HARVARD:
> >> Progress  2010-05-04 15:55:55.081637000-0400  LOG_START
> >>
> >> _____________________________________________________________________________
> >>
> >>         Wrapper
> >> _____________________________________________________________________________
> >>
> >> Progress  2010-05-04 15:55:55.086896000-0400
> >> SOURCE_CDM_LIB /osg/storage/data/engage/swift/postproc-tw
> >> Job directory mode is: link on shared filesystem
> >> DIR=jobs/d/extract-d5i5khrj
> >> EXEC=/osg/storage/app/engage/JBSim3d/bin/jbsim3d
> >> STDIN=
> >> STDOUT=stdout.txt
> >> STDERR=stderr.txt
> >> DIRS=TEST|158/8|panfs/panasas/CMS/data/engage/swift/158/8
> >> INF=TEST/TEST_fy_644.sgt|TEST/TEST_fx_644.sgt|
> >> 158/8/158_8.txt.variation-s0001-h0000
> >> OUTF=panfs/panasas/CMS/data/engage/swift/158/8/TEST_158_8_subfy.sgt|
> >> panfs/panasas/CMS/data/engage/swift
> >> KICKSTART=
> >> CDM_FILE=shared/fs.data
> >> ARGS=stat=TEST extract_sgt=1 slon=-118.286 slat=34.0192
> >> rupmodfile=158/8/158_8.txt.variation-s0001-h000
> >> ARGC=9
> >> Progress  2010-05-04 15:55:55.089785000-0400  CREATE_JOBDIR
> >> Created job directory: jobs/d/extract-d5i5khrj
> >> Progress  2010-05-04 15:55:55.092773000-0400  CREATE_INPUTDIR
> >> Created output directory: jobs/d/extract-d5i5khrj/TEST
> >> Created output directory: jobs/d/extract-d5i5khrj/158/8
> >> Created output directory:
> >> jobs/d/extract-d5i5khrj/panfs/panasas/CMS/data/engage/swift/158/8
> >> Progress  2010-05-04 15:55:55.100016000-0400  LINK_INPUTS
> >> CDM_POLICY: TEST/TEST_fy_644.sgt ->
> >> DIRECT /panfs/panasas/CMS/data/engage/scec/SgtFiles
> >> CDM_ACTION: jobs/d/extract-d5i5khrj INPUT TEST/TEST_fy_644.sgt
> >> DIRECT /panfs/panasas/CMS/data/engage/sc
> >> CDM[DIRECT]: Linking
> >> to /panfs/panasas/CMS/data/engage/scec/SgtFiles/TEST/TEST_fy_644.sgt via
> >> jobs/d/ex
> >> CDM[DIRECT]: /panfs/panasas/CMS/data/engage/scec/SgtFiles/TEST/TEST_fy_644.sgt does not exist!
> >>
> >> _____________________________________________________________________________