[petsc-users] PGMRES and IBCGS on Kraken

[John Mousel]

Thanks Jed.


On Fri, Jan 3, 2014 at 4:14 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

> John Mousel <john.mousel at gmail.com> writes:
>
> > I would like to investigate using  pipelined GMRES and IBCGS on Kraken
> but
> > it seems from the FAQ page that Cray MPT-5.6 is required for the
> pipelined
> > methods to be effective. Checking the module avail, I see that they only
> > have up to MPT-5.3.6. Is there any benefit at all to these methods if I
> > can't configure with MPT-5.6?
>
> IBCGS does not use pipelining so MPT version is irrelevant.  For
> pipelined GMRES/CG/CR, you need MPT-5.6 or later to use the asynchronous
> interface (MPI_Iallreduce).  Without that, the pipelined methods will do
> fewer reductions, but they will be synchronous instead of asynchronous,
> thus likely not paying off.
>
> Unfortunately, even MPT-5.6 does not managed to make significant
> asynchronous progress.  So when you use MPI_Iallreduce, almost nothing
> happens until you call MPI_Wait.  The Cray MPI team says they think this
> is a software/implementation problem rather than a hardware problem, and
> they are hopeful that they'll be able to release a new MPT that does a
> good job of making asynchronous progress.
>
> This probably means that PGMRES won't provide large benefits until
> Cray's MPI team writes some code and you switch to a machine that uses
> that new code.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140103/79234b72/attachment-0001.html>