[Swift-devel] MPI jobs with plain PBS provider

[David Kelly]

Hello,

I am trying to run an MPI job with the plain PBS provider (no coasters) on Fusion. I'm following the instructions in the user guide, section 19.3.1 (http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html#_launching_mpi_jobs_from_swift). It works well with one node. How can I start only one worker, but request multiple nodes? I need something like this in the submit script:

#PBS -l nodes=2:ppn=8

>From looking at the provider, it looks like the value of nodes is determined by the internal 'count' variable, which I believe is always set to 1 for non-coaster jobs. I can control the ppn with the ppn attribute, but as far as I can tell, there is no nodes attribute. Does it make sense to add one for MPI jobs like this?

Here are the files I am testing with:

sites.xml
---------
<config>
<pool handle="fusion">
  <execution jobmanager="local:pbs" provider="pbs" url="none"/>
  <filesystem provider="local" url="none" />
  <profile namespace="globus" key="maxtime">60</profile>
  <profile namespace="globus" key="jobsPerNode">1</profile>
  <profile namespace="globus" key="ppn">8</profile>
  <profile namespace="karajan" key="jobThrottle">5.99</profile>
  <profile namespace="karajan" key="initialScore">10000</profile>
  <workdirectory>/homes/davidk/swiftwork</workdirectory>
</pool>
</config>
---------


testmpi.swift
-------------
type file;

app (file r) mpi ()
{
  mpi;
}

file results <"results.tar">;
results = mpi();
-------------


mpi.sh
------
#!/bin/bash

mpiexec -machinefile $PBS_NODEFILE /home/davidk/testmpi/testmpi
------

David