[Swift-devel] Falkon and Coaster support for MPI

[Mihael Hategan]

On Mon, 2008-06-30 at 03:43 -0500, Ian Foster wrote:

> 5) The question has been raised of how to implement (2). One proposal  
> is to adapt coaster to support MPI jobs. I'm a bit concerned that this  
> could be expensive: we already have Falkon running well on BG/P, and  
> given our other commitments to support NSF user communities, putting  
> scarce resources into replicating that work may not be optimal.
> 

As far as I understand from what Ioan says, Falkon doesn't support MPI
jobs. I think the requirement came from Benoit's group realization that
running applications on BG/P without MPI is rather slow. Not that I
haven't said that.

In any event, it seems like such support is necessary for achieving
reasonable performance on BG/P. So it pretty much boils down to whether
we want to reasonably support BG/P or not.

In terms of the scheduling, what we must keep in mind is that
coasters/glideins/falkons can be implemented in such a way as to ensure
performance is never worse than without them. So if a 256 node job is
submitted, requesting 256 nodes from the queuing system and making sure
that no other job will get them before the 256 node job, while also
submitting the job earlier if a sufficient number of nodes becomes
available is never going to be worse than not having the mechanism. This
also ensures that starvation does not happen unless the underlying
system would have starved the job anyway.