[Swift-devel] SWIFT_JOBDIR_PATH env not passed on remote sites

[Ben Clifford]

Swift+falkon+envvars has come up on this list before.

The test suite almost catches this - it has a test for environment 
variables, but that is not regularly run against non-local providers.

I should probably fix that.

On Fri, 8 May 2009, Michael Wilde wrote:

> Yes, under provider-deef and Falkon the entire environment is lost. We think
> that the C worker agent is not receiving or setting the environment.
> 
> - Mike
> 
> On 5/8/09 4:37 PM, Allan Espinosa wrote:
> > It works with gt2.  Is this a pbs problem?
> > 
> > Other provider that I am testing that it does not work is on
> > provider-deef.  For my purposes, I would like to have it work under
> > provider-deef and provider-pbs
> > 
> > Thanks,
> > -Allan
> > 
> > 2009/5/8 Mihael Hategan <hategan at mcs.anl.gov>:
> > > Does it also happen if you use GT2 instead of PBS?
> > > 
> > > On Fri, 2009-05-08 at 15:46 -0500, Allan Espinosa wrote:
> > > > Hi,
> > > > 
> > > > SWIFT_JOBDIR_PATH only works in local executions.  it looks like the
> > > > environment variable is not being passed for remote job submissions:
> > > > 
> > > > sites.xml:
> > > > <config>
> > > >   <pool handle="TERAPORT" sysinfo="INTEL32::LINUX">
> > > >     <execution provider="pbs" url="none" />
> > > >     <profile namespace="globus" key="queue">fast</profile>
> > > >     <gridftp url="local://localhost" />
> > > >     <workdirectory
> > > > >/home/aespinosa/workflows/jobdirpath/sitedir</workdirectory>
> > > >     <profile namespace="env"
> > > > key="SWIFT_JOBDIR_PATH">/home/aespinosa/workflows/localnode</profile>
> > > >   </pool>
> > > > </config>
> > > > 
> > > > resulting wrapper log:
> > > > Progress  2009-05-08 15:33:31.258501000-0500  LOG_START
> > > > 
> > > > _____________________________________________________________________________
> > > > 
> > > >         Wrapper
> > > > _____________________________________________________________________________
> > > > 
> > > > Job directory mode is: link on shared filesystem
> > > > DIR=jobs/v/cat-vbmpqiaj
> > > > EXEC=/bin/cat
> > > > STDIN=
> > > > STDOUT=061-cattwo.out
> > > > STDERR=stderr.txt
> > > > DIRS=
> > > > INF=061-cattwo.1.in|061-cattwo.2.in
> > > > OUTF=061-cattwo.out
> > > > KICKSTART=
> > > > ARGS=061-cattwo.1.in 061-cattwo.2.in
> > > > ARGC=2
> > > > Progress  2009-05-08 15:33:31.277785000-0500  CREATE_JOBDIR
> > > > Created job directory: jobs/v/cat-vbmpqiaj
> > > > Progress  2009-05-08 15:33:31.289956000-0500  CREATE_INPUTDIR
> > > > Progress  2009-05-08 15:33:31.294467000-0500  LINK_INPUTS
> > > > Linked input:
> > > > /home/aespinosa/workflows/jobdirpath/sitedir/061-cattwo-20090508-1533-x71gz29g/shared/061-cattwo.1.in
> > > > to jobs/v/cat-vbmpqiaj/061-cattwo.1.in
> > > > Linked input:
> > > > /home/aespinosa/workflows/jobdirpath/sitedir/061-cattwo-20090508-1533-x71gz29g/shared/061-cattwo.2.in
> > > > to jobs/v/cat-vbmpqiaj/061-cattwo.2.in
> > > > Progress  2009-05-08 15:33:31.327217000-0500  EXECUTE
> > > > Moving back to workflow directory
> > > > /home/aespinosa/workflows/jobdirpath/sitedir/061-cattwo-20090508-1533-x71gz29g
> > > > Progress  2009-05-08 15:33:31.338850000-0500  EXECUTE_DONE
> > > > Job ran successfully
> > > > Progress  2009-05-08 15:33:31.343910000-0500  COPYING_OUTPUTS
> > > > Progress  2009-05-08 15:33:31.373251000-0500  RM_JOBDIR
> > > > Progress  2009-05-08 15:33:31.383733000-0500  TOUCH_SUCCESS
> > > > Progress  2009-05-08 15:33:31.398652000-0500  END
> > > > 
> > > > the directory "localnode" is also not created.  Test peformed using
> > > > 061-cattwo.  my testcase script is in ~aespinosa/workflows/jobdirpath
> > > > 
> > > > [aespinosa at tp-login1 jobdirpath]$ ./runtest.sh
> > > > Swift svn swift-r2911 cog-r2394
> > > > 
> > > > RunID: remoterun
> > > > Progress:  uninitialized:1
> > > > Progress:  Active:1
> > > > Final status:  Finished successfully:1
> > > > Remote test FAIL
> > > > Swift svn swift-r2911 cog-r2394
> > > > 
> > > > RunID: localrun
> > > > Progress:
> > > > Final status:  Finished successfully:1
> > > > Local test PASS
> > > > 
> > > > -Allan
> > > > 
> > > > 
> > > 
> > > 
> > 
> > 
> > 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> 
>