[Swift-user] Question about swiftR temporary and work directories

[Tim Armstrong]

There is an environment variable called SWIFTR_TMP that can control this.

Though... are you sure you want to do this?  It should work fine if it's
just pointing to /tmp/.  It isn't necessary that this temporary directory
be visible to nodes in the cluster, since it is just used for temporary
storage for the local processes. Many small files are created in the
directory, so I have no idea how it will work with lustre.

- Tim

On Wed, Jun 13, 2012 at 10:51 AM, Lorenzo Pesce <lpesce at uchicago.edu> wrote:

> We are using swift R. Since we are working on a machine where the compute
> nodes see only the /lustre file system, we need to be able to set all the
> temporary directories there.
> We know how to do that with swift, but swiftR doesn't seem to work that
> way.
>
> This is the command:
>
> > library(Swift)
> > swiftInit(server="custom", tc.file="ROC.tc", cf.file="ROC.cf",
> sites.file="ROCSites.xml")
> Working in /tmp/lpesce/SwiftR/swift.ZKHU
> Running in /tmp/lpesce/SwiftR/swift.ZKHU (linked to
> /tmp/lpesce/SwiftR/swift.custom)
> Started worker manager with pid 10153
>
> I would like both of those to point to /lustre/beagle/`whoami`/<some path>
> where I can decide where to send them or at least that the place is under
> my /lustre folder.
>
> Any ideas?
>
> We set these variables before running R:
> module load swift
> export SWIFT_HEAP_MAX=1024M
> export USER_HOME=/lustre/beagle/GCNet
> export SWIFT_HOME=/soft/swift/0.93/
>
> # set temporary directory references for R, currdir is
> /lustre/beagle/`whoami`
> export TMP=$currdir/tmp
> export TMPDIR=$TMP
> export TEMP=$TMP
> mkdir -p $TMP
>
>
> This is the sites file:
>  less ROCSites.xml
> <config>
>  <pool handle="pbs">
>    <execution provider="coaster" jobmanager="local:pbs"/>
>    <profile namespace="globus" key="project">CI-CCR000070</profile>
>
>
>    <profile namespace="globus"
> key="providerAttributes">pbs.aprun;pbs.mpp;depth=24</profile>
>
>    <profile namespace="globus" key="jobsPerNode">24</profile>
>    <profile namespace="globus" key="maxTime">1800</profile>
>    <profile namespace="globus" key="maxwalltime">0:2:00</profile>
>    <profile namespace="globus" key="lowOverallocation">100</profile>
>    <profile namespace="globus" key="highOverallocation">100</profile>
>
>    <profile namespace="globus" key="slots">1</profile>
>    <profile namespace="globus" key="nodeGranularity">1</profile>
>    <profile namespace="globus" key="maxNodes">1</profile>
>
>    <profile namespace="karajan" key="jobThrottle">.63</profile>
>    <profile namespace="karajan" key="initialScore">10000</profile>
>
>    <filesystem provider="local"/>
>
>  <workdirectory>/lustre/beagle/lpesce/ROCSwift/swift.workdir</workdirectory>
>  </pool>
> </config>
>
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20120613/355e65f4/attachment.html>