[Swift-commit] r8231 - www/Swift-T

Wed Aug 27 15:11:14 CDT 2014

Author: wozniak
Date: 2014-08-27 15:11:13 -0500 (Wed, 27 Aug 2014)
New Revision: 8231

Modified:
   www/Swift-T/turbine-sites.html
Log:
Document Titan


Modified: www/Swift-T/turbine-sites.html
===================================================================

--- www/Swift-T/turbine-sites.html	2014-08-26 03:46:38 UTC (rev 8230)
+++ www/Swift-T/turbine-sites.html	2014-08-27 20:11:13 UTC (rev 8231)
@@ -746,8 +746,8 @@
 <h1>Turbine Sites Guide</h1>
 <span id="author">Justin M. Wozniak</span><br />
 <span id="email"><code><<a href="mailto:wozniak at mcs.anl.gov">wozniak at mcs.anl.gov</a>></code></span><br />
-<span id="revnumber">version 0.5.0,</span>
-<span id="revdate">April 2014</span>
+<span id="revnumber">version 0.6.1,</span>
+<span id="revdate">August 2014</span>
 <div id="toc">
   <div id="toctitle">Table of Contents</div>
   <noscript><p><b>JavaScript must be enabled in your browser to display the table of contents.</b></p></noscript>
@@ -1661,11 +1661,129 @@
 <h2 id="_cray">Cray</h2>
 <div class="sectionbody">
 <div class="sect2">
+<h3 id="_titan">Titan</h3>
+<div class="paragraph"><p>Titan is a Cray XK7 at the Oak Ridge Leadership Computing Facility.</p></div>
+<div class="sect3">
+<h4 id="_public_installation_7">Public installation</h4>
+<div class="paragraph"><p><strong>As of:</strong> 8/27/2014</p></div>
+<div class="ulist"><ul>
+<li>
+<p>
+Swift/T trunk - 8/27/2013
+</p>
+</li>
+<li>
+<p>
+Turbine: <code>/lustre/atlas2/mat049/proj-shared/sfw/compute/turbine/scripts/submit/cray/turbine-cray-run.zsh</code>
+</p>
+</li>
+<li>
+<p>
+STC: <code>/lustre/atlas2/mat049/proj-shared/sfw/compute/stc/bin/stc</code>
+</p>
+</li>
+<li>
+<p>
+SWIG: <code>/lustre/atlas2/mat049/proj-shared/sfw/swig-3.0.2</code>
+</p>
+</li>
+<li>
+<p>
+Tcl:  <code>/lustre/atlas2/mat049/proj-shared/sfw/tcl-8.6.2</code>
+</p>
+</li>
+</ul></div>
+</div>
+<div class="sect3">
+<h4 id="_submitting_jobs">Submitting jobs</h4>
+<div class="paragraph"><p>Titan requires that user output goes to a Lustre file system.  Set a
+soft link like this so that Turbine output goes to Lustre:</p></div>
+<div class="listingblock">
+<div class="content">
+<pre><code>mkdir /lustre/atlas/scratch/YOUR_USERNAME/turbine-output
+cd ~
+ln -s /lustre/atlas/scratch/YOUR_USERNAME/turbine-output</code></pre>
+</div></div>
+<div class="paragraph"><p>Or, you may set <code>TURBINE_OUTPUT</code> <a href="#variables">manually</a>.</p></div>
+<div class="paragraph"><p>Titan requires the submit script to specify job size using different
+directives to other Cray systems.  It does not support the <code>#PBS -l ppn:</code>
+directive. The correct directive is:</p></div>
+<div class="listingblock">
+<div class="content">
+<pre><code>#PBS -l nodes=32</code></pre>
+</div></div>
+<div class="paragraph"><p><code>PPN</code> is handled by setting the <code>-n</code> argument.</p></div>
+<div class="paragraph"><p>The <code>turbine-aprun-run.zsh</code> script supports Titan.  You can invoke
+it as follows (for a single node/32 processes per node):</p></div>
+<div class="listingblock">
+<div class="content">
+<pre><code>export PROJECT=MAT049
+export QUEUE=debug
+export TITAN=true
+turbine-aprun-run.zsh -n 32 helloworld.tcl</code></pre>
+</div></div>
+<div class="paragraph"><p>These environment variables may be placed in your <code>-s</code>
+<a href="#options">settings file</a>.</p></div>
+</div>
+<div class="sect3">
+<h4 id="_build_procedure_7">Build procedure</h4>
+<div class="paragraph"><p><strong>As of:</strong> 8/27/2014</p></div>
+<div class="paragraph"><p>Cray systems do not use <code>mpicc</code>.  We set <code>CC=gcc</code> and use compiler
+flags to configure the MPI library.</p></div>
+<div class="ulist"><ul>
+<li>
+<p>
+Configure ADLB with:
+</p>
+<div class="listingblock">
+<div class="content">
+<pre><code>./configure --prefix=/path/to/lb --with-c-utils=/path/to/c-utils
+CC=gcc
+CFLAGS=-I/opt/cray/mpt/default/gni/mpich2-gnu/47/include
+LDFLAGS="-L/opt/cray/mpt/default/gni/mpich2-gnu/47/lib -lmpich"
+--enable-mpi-2</code></pre>
+</div></div>
+</li>
+<li>
+<p>
+In the Turbine configure step, replace the <code>--with-mpi</code> option with:
+</p>
+<div class="listingblock">
+<div class="content">
+<pre><code>--enable-custom-mpi --with-mpi=/opt/cray/mpt/default/gni/mpich2-gnu/47</code></pre>
+</div></div>
+</li>
+</ul></div>
+</div>
+<div class="sect3">
+<h4 id="_submitting_jobs_2">Submitting jobs</h4>
+<div class="paragraph"><p>Titan requires the submit script to specify job size using different
+directives to other Cray systems.  It does not support the <code>#PBS -l ppn:</code>
+directive. The correct directive is:</p></div>
+<div class="listingblock">
+<div class="content">
+<pre><code>#PBS -l nodes=32</code></pre>
+</div></div>
+<div class="paragraph"><p><code>PPN</code> is handled by setting the <code>-n</code> argument.</p></div>
+<div class="paragraph"><p>The <code>turbine-aprun-run.zsh</code> script supports Titan.  You can invoke
+it as follows (for a single node/32 processes per node):</p></div>
+<div class="listingblock">
+<div class="content">
+<pre><code>export QUEUE=normal
+export TITAN=true
+export PPN=32
+turbine-aprun-run.zsh -n 32 helloworld.tcl</code></pre>
+</div></div>
+<div class="paragraph"><p>These environment variables may be placed in your
+<a href="#settings_file">settings file</a>.</p></div>
+</div>
+</div>
+<div class="sect2">
 <h3 id="_blue_waters">Blue Waters</h3>
 <div class="paragraph"><p>Blue Waters is a Cray XE6/XK7 at the University of Illinois at
 Urbana-Champaign.</p></div>
 <div class="sect3">
-<h4 id="_build_procedure_7">Build procedure</h4>
+<h4 id="_build_procedure_8">Build procedure</h4>
 <div class="paragraph"><p><strong>As of:</strong>  11/05/2013</p></div>
 <div class="paragraph"><p>Cray systems do not use <code>mpicc</code>.  We set <code>CC=gcc</code> and use compiler
 flags to configure the MPI library.</p></div>
@@ -1715,7 +1833,7 @@
 </ul></div>
 </div>
 <div class="sect3">
-<h4 id="_submitting_jobs">Submitting jobs</h4>
+<h4 id="_submitting_jobs_3">Submitting jobs</h4>
 <div class="paragraph"><p>Submitting jobs on Blue Waters is largely the same with with other Cray
 systems. One difference is that the size of the job is specified using a
 different notation.</p></div>
@@ -1883,7 +2001,7 @@
 NFS (including home directories).  Thus, you must install Turbine and
 its libraries in <code>/lustre</code>.  Also, your data must be in <code>/lustre</code>.</p></div>
 <div class="sect3">
-<h4 id="_public_installation_7">Public installation</h4>
+<h4 id="_public_installation_8">Public installation</h4>
 <div class="sect4">
 <h5 id="_login_nodes">Login nodes</h5>
 <div class="paragraph"><p>This installation is for use on the <a href="#login_node">login node</a>.</p></div>
@@ -1945,7 +2063,7 @@
 </div>
 </div>
 <div class="sect3">
-<h4 id="_build_procedure_8">Build procedure</h4>
+<h4 id="_build_procedure_9">Build procedure</h4>
 <div class="paragraph"><p>Cray systems do not use <code>mpicc</code>.  We set <code>CC=gcc</code> and use compiler
 flags to configure the MPI library.</p></div>
 <div class="ulist"><ul>
@@ -2004,7 +2122,7 @@
 <h3 id="_raven">Raven</h3>
 <div class="paragraph"><p>Raven is a Cray XE6/XK7 at Cray.</p></div>
 <div class="sect3">
-<h4 id="_build_procedure_9">Build procedure</h4>
+<h4 id="_build_procedure_10">Build procedure</h4>
 <div class="ulist"><ul>
 <li>
 <p>
@@ -2141,12 +2259,31 @@
 </div>
 </div>
 </div>
+<div class="sect1">
+<h2 id="_mac_os_x">Mac OS X</h2>
+<div class="sectionbody">
+<div class="paragraph"><p>Swift/T is regularly tested on the Mac.  You may use Swift/T as on any
+other single system.</p></div>
+<div class="ulist"><ul>
+<li>
+<p>
+SWIG: You may use SWIG from source or the MacPorts <code>swig-tcl</code> package
+</p>
+</li>
+<li>
+<p>
+MPI: You may use any MPI implementation
+</p>
+</li>
+</ul></div>
 </div>
+</div>
+</div>
 <div id="footnotes"><hr /></div>
 <div id="footer">
 <div id="footer-text">
-Version 0.5.0<br />
-Last updated 2014-07-21 14:36:09 CDT
+Version 0.6.1<br />
+Last updated 2014-08-27 13:43:30 CDT
 </div>
 </div>
 </body>


