[Swift-devel] MPI jobs with plain PBS provider

[David Kelly]

Mike,

Thank you - the test MPI application is now working well using multiple nodes on Fusion.

David

----- Original Message -----
> From: "Michael Wilde" <wilde at mcs.anl.gov>
> To: "David Kelly" <davidk at ci.uchicago.edu>
> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>
> Sent: Wednesday, June 6, 2012 9:26:09 AM
> Subject: Re: [Swift-devel] MPI jobs with plain PBS provider
> David, adding jobType and count attributes as follows should do what
> you want:
> 
> <profile namespace="globus" key="count">2</profile>
> <profile namespace="globus" key="jobType">single</profile>
> 
> This was added to trunk on May 22, r3412:
> ---
> Accept jobType value "single" in sites.xml to force a multi-node
> non-coaster non-JETS PBS job to run a single application.
> ---
> 
> Test on plain PBS (PADS), not yet tested on Cray PBS.
> 
> - Mike
> 
> 
> ----- Original Message -----
> > From: "David Kelly" <davidk at ci.uchicago.edu>
> > To: "Swift Devel" <swift-devel at ci.uchicago.edu>
> > Sent: Tuesday, June 5, 2012 9:19:49 PM
> > Subject: [Swift-devel] MPI jobs with plain PBS provider
> > Hello,
> >
> > I am trying to run an MPI job with the plain PBS provider (no
> > coasters) on Fusion. I'm following the instructions in the user
> > guide,
> > section 19.3.1
> > (http://www.ci.uchicago.edu/swift/guides/trunk/userguide/userguide.html#_launching_mpi_jobs_from_swift).
> > It works well with one node. How can I start only one worker, but
> > request multiple nodes? I need something like this in the submit
> > script:
> >
> > #PBS -l nodes=2:ppn=8
> >
> > From looking at the provider, it looks like the value of nodes is
> > determined by the internal 'count' variable, which I believe is
> > always
> > set to 1 for non-coaster jobs. I can control the ppn with the ppn
> > attribute, but as far as I can tell, there is no nodes attribute.
> > Does
> > it make sense to add one for MPI jobs like this?
> >
> > Here are the files I am testing with:
> >
> > sites.xml
> > ---------
> > <config>
> > <pool handle="fusion">
> > <execution jobmanager="local:pbs" provider="pbs" url="none"/>
> > <filesystem provider="local" url="none" />
> > <profile namespace="globus" key="maxtime">60</profile>
> > <profile namespace="globus" key="jobsPerNode">1</profile>
> > <profile namespace="globus" key="ppn">8</profile>
> > <profile namespace="karajan" key="jobThrottle">5.99</profile>
> > <profile namespace="karajan" key="initialScore">10000</profile>
> > <workdirectory>/homes/davidk/swiftwork</workdirectory>
> > </pool>
> > </config>
> > ---------
> >
> >
> > testmpi.swift
> > -------------
> > type file;
> >
> > app (file r) mpi ()
> > {
> > mpi;
> > }
> >
> > file results <"results.tar">;
> > results = mpi();
> > -------------
> >
> >
> > mpi.sh
> > ------
> > #!/bin/bash
> >
> > mpiexec -machinefile $PBS_NODEFILE /home/davidk/testmpi/testmpi
> > ------
> >
> > David
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> --
> Michael Wilde
> Computation Institute, University of Chicago
> Mathematics and Computer Science Division
> Argonne National Laboratory