[Swift-user] Swift+MPI+LSF

[Andriy Fedorov]

Hi,

I am trying to use Swift to run an MPI job via LSF scheduler (TG
Lonestar, http://www.tacc.utexas.edu/services/userguides/lonestar/).
Previously, I had problems running stuff like this with PBS (see
http://mail.ci.uchicago.edu/pipermail/swift-user/2008-July/000443.html).

Right now I am using the solution suggested by Ben (submit single node
job, and run a shell wrapper to launch mpirun, ). This doesn't seem to
work with LSF. I specify "GLOBUS::jobType=single,host_xcount=10", and
have my shell wrapper run

#!/bin/bash
ibrun /home/teragrid/tg457149/meshreg/trunk/build-mpicc/bin/blockMatchingMPI $*

but I get one node allocated.

According to Lonestar manual, the number of nodes is specified in the
script like this (note: CPUs are specified in #BSUB, not as an
argument to ibrun):

#!/bin/tcsh 			
				  # first line specifies shell
#BSUB -J jobname 		  #name the job "jobname"
#BSUB -o out.o%J   		  #output->   out.o&ltjobID>
#BSUB -e err.o%J   		  #error -> error.o&ltjobID>
#BSUB -n 4 -W 1:30                #4 CPU cores and 1hr+30min
#BSUB -q normal                   #Use normal queue.
set echo                          #Echo all commands.
cd $LS_SUBCWD                     #cd to directory of submission
ibrun ./a.out                     #use ibrun for "pam -g 1 mvapich_wrapper"
                                  #CPUs are specified above in -n option.

Is this a known issue? Has anyone run into something like this?

--
Andrey Fedorov

Center for Real-Time Computing
College of William and Mary
http://www.cs.wm.edu/~fedorov