[petsc-users] PGMRES and IBCGS on Kraken

[Jed Brown]

John Mousel <john.mousel at gmail.com> writes:

> I would like to investigate using  pipelined GMRES and IBCGS on Kraken but
> it seems from the FAQ page that Cray MPT-5.6 is required for the pipelined
> methods to be effective. Checking the module avail, I see that they only
> have up to MPT-5.3.6. Is there any benefit at all to these methods if I
> can't configure with MPT-5.6?

IBCGS does not use pipelining so MPT version is irrelevant.  For
pipelined GMRES/CG/CR, you need MPT-5.6 or later to use the asynchronous
interface (MPI_Iallreduce).  Without that, the pipelined methods will do
fewer reductions, but they will be synchronous instead of asynchronous,
thus likely not paying off.

Unfortunately, even MPT-5.6 does not managed to make significant
asynchronous progress.  So when you use MPI_Iallreduce, almost nothing
happens until you call MPI_Wait.  The Cray MPI team says they think this
is a software/implementation problem rather than a hardware problem, and
they are hopeful that they'll be able to release a new MPT that does a
good job of making asynchronous progress.

This probably means that PGMRES won't provide large benefits until
Cray's MPI team writes some code and you switch to a machine that uses
that new code.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140103/d7dab58a/attachment.pgp>