[petsc-users] Add a matrix to MatNest twice

[Jed Brown]

On Tue, Feb 21, 2012 at 15:37, Barry Smith <bsmith at mcs.anl.gov> wrote:

> Why not just have MatAssemblyBegin_Nest() call the inner
> MatAssemblyBegin/End() together and stop the charade that there is any
> overlap of communication and computations etc anyway?


(I pushed this.)

So there is a very real latency issue in matrix assembly that comes from
the reduction to determine how many receives are necessary. Due to MPI
limitations, that code (PetscGatherNumberOfMessages() and
PetscGatherMessageLengths()) is synchronizing, but MPI-3 will offer
non-blocking collectives that we could use for those operations. Now the
two entrance points (MatAssemblyBegin() and MatAssemblyEnd()) are not
sufficient to make progress on this task of assembly without also having an
internal request system (where either a comm thread or callbacks from other
library functions poked the progress along).

There are also signs that sometime soon it will be common to have a comm
thread that manages packing, in which case communication could actually
start happening concurrently with computation.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120221/b47f7e41/attachment.htm>