[Swift-user] Question about swiftR temporary and work directories

[Lorenzo Pesce]

I was on paternity leave and lost a bit track of things ;-)

Let me get back to you soon. The last  decision was that swiftR was not capable of handling this type of problem and that we were most likely going to use Swift or Bash + R.


On Jun 24, 2012, at 9:53 PM, Michael Wilde wrote:

> Lorenzo, I see now that Tim answered this. Is SwiftR now working for you for this problem?
> 
> - Mike
> 
> 
> ----- Original Message -----
>> From: "Michael Wilde" <wilde at mcs.anl.gov>
>> To: "Lorenzo Pesce" <lpesce at uchicago.edu>
>> Cc: swift-user at ci.uchicago.edu
>> Sent: Sunday, June 24, 2012 9:47:10 PM
>> Subject: Re: [Swift-user] Question about swiftR temporary and work directories
>> The use of /tmp might be set in the R code of the SwiftR package; will
>> need to check and get back to you.
>> 
>> - Mike
>> 
>> ----- Original Message -----
>>> From: "Lorenzo Pesce" <lpesce at uchicago.edu>
>>> To: swift-user at ci.uchicago.edu
>>> Sent: Wednesday, June 13, 2012 10:51:32 AM
>>> Subject: [Swift-user] Question about swiftR temporary and work
>>> directories
>>> We are using swift R. Since we are working on a machine where the
>>> compute nodes see only the /lustre file system, we need to be able
>>> to
>>> set all the temporary directories there.
>>> We know how to do that with swift, but swiftR doesn't seem to work
>>> that way.
>>> 
>>> This is the command:
>>> 
>>>> library(Swift)
>>>> swiftInit(server="custom", tc.file="ROC.tc", cf.file="ROC.cf",
>>>> sites.file="ROCSites.xml")
>>> Working in /tmp/lpesce/SwiftR/swift.ZKHU
>>> Running in /tmp/lpesce/SwiftR/swift.ZKHU (linked to
>>> /tmp/lpesce/SwiftR/swift.custom)
>>> Started worker manager with pid 10153
>>> 
>>> I would like both of those to point to /lustre/beagle/`whoami`/<some
>>> path>
>>> where I can decide where to send them or at least that the place is
>>> under my /lustre folder.
>>> 
>>> Any ideas?
>>> 
>>> We set these variables before running R:
>>> module load swift
>>> export SWIFT_HEAP_MAX=1024M
>>> export USER_HOME=/lustre/beagle/GCNet
>>> export SWIFT_HOME=/soft/swift/0.93/
>>> 
>>> # set temporary directory references for R, currdir is
>>> /lustre/beagle/`whoami`
>>> export TMP=$currdir/tmp
>>> export TMPDIR=$TMP
>>> export TEMP=$TMP
>>> mkdir -p $TMP
>>> 
>>> 
>>> This is the sites file:
>>> less ROCSites.xml
>>> <config>
>>> <pool handle="pbs">
>>> <execution provider="coaster" jobmanager="local:pbs"/>
>>> <profile namespace="globus" key="project">CI-CCR000070</profile>
>>> 
>>> 
>>> <profile namespace="globus"
>>> key="providerAttributes">pbs.aprun;pbs.mpp;depth=24</profile>
>>> 
>>> <profile namespace="globus" key="jobsPerNode">24</profile>
>>> <profile namespace="globus" key="maxTime">1800</profile>
>>> <profile namespace="globus" key="maxwalltime">0:2:00</profile>
>>> <profile namespace="globus" key="lowOverallocation">100</profile>
>>> <profile namespace="globus" key="highOverallocation">100</profile>
>>> 
>>> <profile namespace="globus" key="slots">1</profile>
>>> <profile namespace="globus" key="nodeGranularity">1</profile>
>>> <profile namespace="globus" key="maxNodes">1</profile>
>>> 
>>> <profile namespace="karajan" key="jobThrottle">.63</profile>
>>> <profile namespace="karajan" key="initialScore">10000</profile>
>>> 
>>> <filesystem provider="local"/>
>>> <workdirectory>/lustre/beagle/lpesce/ROCSwift/swift.workdir</workdirectory>
>>> </pool>
>>> </config>
>>> 
>>> 
>>> _______________________________________________
>>> Swift-user mailing list
>>> Swift-user at ci.uchicago.edu
>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>> 
>> --
>> Michael Wilde
>> Computation Institute, University of Chicago
>> Mathematics and Computer Science Division
>> Argonne National Laboratory
>> 
>> _______________________________________________
>> Swift-user mailing list
>> Swift-user at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> 
> -- 
> Michael Wilde
> Computation Institute, University of Chicago
> Mathematics and Computer Science Division
> Argonne National Laboratory
>