[Swift-devel] Falkon and Coaster support for MPI

[Wei Jiang]

Here I am ready to provide any information you want for the sample CHARMM run on BGP or KBT, and the perl script fe.pl, which generates all the PBS scripts/jobs. If necessary, we can meet in ANL or campus some day this week. 

Wei
----- Original Message -----
From: "Michael Wilde" <wilde at mcs.anl.gov>
To: "Benoit Roux" <roux at uchicago.edu>
Cc: "Ben Clifford" <benc at hawaga.org.uk>, "swift-devel" <swift-devel at ci.uchicago.edu>, "Kamil Iskra" <iskra at mcs.anl.gov>, "Kaz Yoshii" <kazutomo at mcs.anl.gov>, "Wei Jiang" <wjiang at mcs.anl.gov>, "Ray Loy" <rloy at mcs.anl.gov>
Sent: Sunday, June 29, 2008 4:30:58 PM GMT -06:00 US/Canada Central
Subject: Re: [Swift-devel] Falkon and Coaster support for MPI

 > *Note that a non-MPI CHARMM job on BG would be not super fast, and MPI
 > capability would be desirable.*

Right, Benoit. This thread is about initiating some of the work from the 
Swift/Falkon side needed for the MPI version of CHARMM. It needs to join 
together with the Zeptos/MPI effort by Kaz and Kamil, but will also be 
of use in running CHARMM and other apps under Swift in MPI mode on other 
clusters.

- Mike

On 6/29/08 4:17 PM, Benoit Roux wrote:
> 
> On Jun 29, 2008, at 4:07 PM, Michael Wilde wrote:
> 
>> Ben,
>>
>> On 6/29/08 2:19 PM, Ben Clifford wrote:
>>> On Sun, 29 Jun 2008, Michael Wilde wrote:
>>>> The app that accelerated this discussion is CHARMM for molecular 
>>>> dynamics. CHARMM on BGP with the parameters needed in the use case 
>>>> in question has a long runtime on 1 CPU (24-36 hours), and seems to 
>>>> peak in performance using MPI at 32 CPUS. Runs >6 hours are also not 
>>>> runnable today on the Argonne BGP without a reservation. So the most 
>>>> effective way to use the BGP for some specific CHARMM runs needed by 
>>>> Benoit Roux's group is to run large numbers of multi-hour 32-rank 
>>>> MPI jobs.
>>> OK.
>>> This seems like an interesting application.
>>> I can spend some time working on this in the next few weeks, as long 
>>> as you provide me with the following:
>>>  i) someone who knows how the charmm applications work (eg from 
>>> Benoit's group)
>>
>> Wei Jiang from Benoit's group spent several hours with me Friday 
>> explaining how they run CHARMM.
>>
>> The approach uses scripts that Yuqing Deng, who recently left the 
>> group, developed.  These are on the Argonne "KBT" cluster. You should 
>> get on the Argonne accounts web and request access to that cluster, 
>> kbt.mcs.anl.gov.
>>
>> One there, you can run perl scripts that generate either shell scripts 
>> or pbs jobs to do a CHARMM run. It was fairly complex, and I need to 
>> transcribe some notes on this.
>>
>> Benoit was going to try to locate slides and/or docs from a talk that 
>> Yuqing gave before he left, on how to run his tools.
>>
>> *Ray Loy of the ALCF team is working on compiling a non-MPI CHARMM 
>> runnable under ZeptoOS on the BGP, and he also wanted to try HTC mode.*
> 
> *Note that a non-MPI CHARMM job on BG would be not super fast, and MPI 
> capability would be desirable.*
> 
>>
>> Nika's latest workflows running CHARMM are also a good reference point.
>>
>> Wei can provide info on how to do a sample run from Yuquing's scripts.
>>
>>>  ii) someone who knows how MPI works on the BG/P (maybe Kamil?)
>>
>> MPI on BGP under ZeptoOS is still several weeks away from 
>> friendly-user testing capability. Kamil and Kaz (and Pete Beckman) are 
>> the Argonne ZeptoOS team. Kaz is working on ZeptoOS MPI and just 
>> reached hello-world about a week ago.
>>
>> - Mike
>