[MPICH] MPI_REDUCE with MPI_IN_PLACE fails with memory error

[Rajeev Thakur]

> And there is really no way to force MPI to use a user-supplied buffer?
> It will always allocate its own buffer?

Well, there is no way to pass a user-supplied buffer. On non-root nodes,
what is passed in the recvbuf argument cannot be trusted. To do a reduction
like a = a + b, you need to store the incoming b and the new result a
somewhere. So, non-root nodes need to allocate *two* buffers. The root node
needs to allocate only one buffer as it can use its recvbuf.

As Ashley said, an implementation can do a pipelined operation in smaller
chunks, but we are not doing that yet (we probably should), so you will need
to do it in your code for now.
 
Rajeev