[MPICH] MPI_REDUCE with MPI_IN_PLACE fails with memory error

[Anthony Chan]

We have no access to intel 9.x compilers yet, can't directly verify the
reported problem.  However you may be using MPI_IN_PLACE incorrectly
in non-root process.

On Tue, 13 Mar 2007, Martin Kleinschmidt wrote:

> The corresponding lines of code are:
>
> #################
> #ifdef PARALLEL
>          if (myid .eq. 0) then
>             call MPI_Reduce(MPI_IN_PLACE, vecf2(1),
>      $           n*nneue,
>      $           MPI_double_precision, MPI_SUM, 0,
>      $           MPI_Comm_World, MPIerr)
>          else
>             call MPI_Reduce(vecf2(1),MPI_IN_PLACE,
>      $           n*nneue,
>      $           MPI_double_precision, MPI_SUM, 0,
>      $           MPI_Comm_World, MPIerr)
>          endif
> #endif

Try calling MPI_Reduce with MPI_IN_PLACE in send buffer in all ranks, i.e.

              call MPI_Reduce(MPI_IN_PLACE, vecf2(1),
       $           n*nneue,
       $           MPI_DOUBLE_PRECISION, MPI_SUM, 0,
       $           MPI_COMM_WORLD, MPIerr)


A.Chan

> #################
> with n*nneue = 76160987, and 76160987*8 = 609287896, about 600 MB
>
> The point is: I thought, I could avoid the need for allocationg
> additional memory by using MPI_IN_PLACE, which obviously does not work.
>
> - do I use MPI_IN_PLACE in the right way?
> - why does MPI_IN_PLACE need additional memory?
> - is it possible to rewrite this code in a way that eliminates the need
>   for allocating additional memory? This part of the code is not
>   time-critical - it is executed once every few hours.
>
> (I'm using mpich2-1.0.5p2, Intel fortran compiler 9.1.040, Intel C
> compiler 9.1.045 for compiling both mpich and my code)
>
>    ...martin
>
>