[Swift-devel] SWIFT_JOBDIR_PATH env not passed on remote sites

[Michael Wilde]

On 5/9/09 4:22 AM, Ben Clifford wrote:
> Swift+falkon+envvars has come up on this list before.
> 
> The test suite almost catches this - it has a test for environment 
> variables, but that is not regularly run against non-local providers.
> 
> I should probably fix that.
> 
> On Fri, 8 May 2009, Michael Wilde wrote:
> 
>> Yes, under provider-deef and Falkon the entire environment is lost. We think
>> that the C worker agent is not receiving or setting the environment.

And by the way, this should be easy to fix; Allan, Zhao or I should do that.

- Mike

>> - Mike
>>
>> On 5/8/09 4:37 PM, Allan Espinosa wrote:
>>> It works with gt2.  Is this a pbs problem?
>>>
>>> Other provider that I am testing that it does not work is on
>>> provider-deef.  For my purposes, I would like to have it work under
>>> provider-deef and provider-pbs
>>>
>>> Thanks,
>>> -Allan
>>>
>>> 2009/5/8 Mihael Hategan <hategan at mcs.anl.gov>:
>>>> Does it also happen if you use GT2 instead of PBS?
>>>>
>>>> On Fri, 2009-05-08 at 15:46 -0500, Allan Espinosa wrote:
>>>>> Hi,
>>>>>
>>>>> SWIFT_JOBDIR_PATH only works in local executions.  it looks like the
>>>>> environment variable is not being passed for remote job submissions:
>>>>>
>>>>> sites.xml:
>>>>> <config>
>>>>>   <pool handle="TERAPORT" sysinfo="INTEL32::LINUX">
>>>>>     <execution provider="pbs" url="none" />
>>>>>     <profile namespace="globus" key="queue">fast</profile>
>>>>>     <gridftp url="local://localhost" />
>>>>>     <workdirectory
>>>>>> /home/aespinosa/workflows/jobdirpath/sitedir</workdirectory>
>>>>>     <profile namespace="env"
>>>>> key="SWIFT_JOBDIR_PATH">/home/aespinosa/workflows/localnode</profile>
>>>>>   </pool>
>>>>> </config>
>>>>>
>>>>> resulting wrapper log:
>>>>> Progress  2009-05-08 15:33:31.258501000-0500  LOG_START
>>>>>
>>>>> _____________________________________________________________________________
>>>>>
>>>>>         Wrapper
>>>>> _____________________________________________________________________________
>>>>>
>>>>> Job directory mode is: link on shared filesystem
>>>>> DIR=jobs/v/cat-vbmpqiaj
>>>>> EXEC=/bin/cat
>>>>> STDIN=
>>>>> STDOUT=061-cattwo.out
>>>>> STDERR=stderr.txt
>>>>> DIRS=
>>>>> INF=061-cattwo.1.in|061-cattwo.2.in
>>>>> OUTF=061-cattwo.out
>>>>> KICKSTART=
>>>>> ARGS=061-cattwo.1.in 061-cattwo.2.in
>>>>> ARGC=2
>>>>> Progress  2009-05-08 15:33:31.277785000-0500  CREATE_JOBDIR
>>>>> Created job directory: jobs/v/cat-vbmpqiaj
>>>>> Progress  2009-05-08 15:33:31.289956000-0500  CREATE_INPUTDIR
>>>>> Progress  2009-05-08 15:33:31.294467000-0500  LINK_INPUTS
>>>>> Linked input:
>>>>> /home/aespinosa/workflows/jobdirpath/sitedir/061-cattwo-20090508-1533-x71gz29g/shared/061-cattwo.1.in
>>>>> to jobs/v/cat-vbmpqiaj/061-cattwo.1.in
>>>>> Linked input:
>>>>> /home/aespinosa/workflows/jobdirpath/sitedir/061-cattwo-20090508-1533-x71gz29g/shared/061-cattwo.2.in
>>>>> to jobs/v/cat-vbmpqiaj/061-cattwo.2.in
>>>>> Progress  2009-05-08 15:33:31.327217000-0500  EXECUTE
>>>>> Moving back to workflow directory
>>>>> /home/aespinosa/workflows/jobdirpath/sitedir/061-cattwo-20090508-1533-x71gz29g
>>>>> Progress  2009-05-08 15:33:31.338850000-0500  EXECUTE_DONE
>>>>> Job ran successfully
>>>>> Progress  2009-05-08 15:33:31.343910000-0500  COPYING_OUTPUTS
>>>>> Progress  2009-05-08 15:33:31.373251000-0500  RM_JOBDIR
>>>>> Progress  2009-05-08 15:33:31.383733000-0500  TOUCH_SUCCESS
>>>>> Progress  2009-05-08 15:33:31.398652000-0500  END
>>>>>
>>>>> the directory "localnode" is also not created.  Test peformed using
>>>>> 061-cattwo.  my testcase script is in ~aespinosa/workflows/jobdirpath
>>>>>
>>>>> [aespinosa at tp-login1 jobdirpath]$ ./runtest.sh
>>>>> Swift svn swift-r2911 cog-r2394
>>>>>
>>>>> RunID: remoterun
>>>>> Progress:  uninitialized:1
>>>>> Progress:  Active:1
>>>>> Final status:  Finished successfully:1
>>>>> Remote test FAIL
>>>>> Swift svn swift-r2911 cog-r2394
>>>>>
>>>>> RunID: localrun
>>>>> Progress:
>>>>> Final status:  Finished successfully:1
>>>>> Local test PASS
>>>>>
>>>>> -Allan
>>>>>
>>>>>
>>>>
>>>
>>>
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>>
>>