[Swift-user] Swift/MPI & OpenFoam

[Yadu Nand Babuji]

Hi Matthew,

I am in the middle of updating our MPI documentation, and I think it 
would be best to
address this question through a generalized example. Here's what I have 
in mind :

You have hosts, node001...008, that you have password-less ssh access to.

1. Set hosts to node001...node008, and hostcount to N.

2. A script similar to start-coaster-service accesses the above 
variables and creates
     partitions with hostcount number of hosts. The script then starts 
the swift worker
     on the first node in each partition of hosts, and sets a node list 
environment variable.
     Eg. you have node001 ... node008 and hostcount = 4
     Each worker(of 2) gets a portion of the nodes, the worker on 
node001 gets
     NODE_LIST=[node001 node002 node003 node4]

3. Tasks submitted to the worker can use mpirun --host <NODE_LIST> to start
     mpi tasks with hostcount number of nodes.

4. Run mpi tasks to check if the ranks are on the right nodes.

Does this sound reasonable ?

Thanks,
Yadu


On 02/25/2015 10:12 AM, Matthew Shaxted wrote:
>
> Hi all,
>
> A few questions about Swift’s interaction with MPI – specifically 
> related to OpenFoam CFD:
>
> To date, we are running OpenFoam CFD jobs on a single machine at the 
> moment, but are looking to scale up computation by way of OpenFoam’s 
> MPI support.
>
> We discussed a strategy of enabling Lustre or a similar file system 
> across our internal resources, connecting our worker nodes to the head 
> node via an MPI listener, and submitting/running the jobs via 
> OpenFoam’s MPI run command.
>
> Does Swift support a connection via MPI?
>
> Perhaps Swift can help manage setting up the in the same way a 
> worker.pl is setup for coaster-service?
>
> Thanks for any insights.
>
> Matthew
>
> MATTHEW SHAXTED
>
> SKIDMORE, OWINGS & MERRILL LLP
> 224 SOUTH MICHIGAN AVENUE
> CHICAGO, IL 60604
> T  (312) 360-4368
> MATTHEW.SHAXTED at SOM.COM <mailto:MATTHEW.SHAXTED at SOM.COM>
>
> cid:image001.png at 01CF9071.6FB46030 <http://www.som.com/>
>
> The information contained in this communication may be confidential, 
> is intended only for the use of the recipient(s) named above, and may 
> be legally privileged. If the reader of this message is not the 
> intended recipient, you are hereby notified that any dissemination, 
> distribution, or copying of this communication, or any of its 
> contents, is strictly prohibited and may be unlawful. If you have 
> received this communication in error, please return it to the 
> sen­­­der immediately and delete the original message and any copy of 
> it from your computer system. If you have any questions concerning 
> this message, please contact the sender.
>
> http://intranet.som.com/common/admin/file.cfm?f=%2Fresources%2Fcontent%2F5%2F0%2F4%2F4%2F6%2F4%2F0%2F3%2Fdocuments%2Fimagea560bf%2Egif%406e10073b%2E30854c37
>
>
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20150225/118e8a04/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 5311 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20150225/118e8a04/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 566 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20150225/118e8a04/attachment.gif>