[MPICH] MPI_REDUCE with MPI_IN_PLACE fails with memory error

[Rajeev Thakur]

In my opinion, there is no problem with the use of MPI_IN_PLACE on non-root
nodes in the example below, because the recvbuf argument is not significant
on non-root nodes. You can pass NULL or any garbage or even MPI_IN_PLACE
there.

Rajeev
 

> -----Original Message-----
> From: owner-mpich-discuss at mcs.anl.gov 
> [mailto:owner-mpich-discuss at mcs.anl.gov] On Behalf Of Anthony Chan
> Sent: Tuesday, March 13, 2007 12:15 PM
> To: Rusty Lusk
> Cc: Martin Kleinschmidt; mpich-discuss at mcs.anl.gov
> Subject: Re: [MPICH] MPI_REDUCE with MPI_IN_PLACE fails with 
> memory error
> 
> 
> The error checking profiling library should be able to detect 
> the problem.
> 
> A.Chan
> 
> On Tue, 13 Mar 2007, Rusty Lusk wrote:
> 
> > I believe our error-checking profiling library detects this error,
> > doesn't it?
> >
> > On Mar 13, 2007, at 11:08 AM, Anthony Chan wrote:
> >
> > >
> > > We have no access to intel 9.x compilers yet, can't 
> directly verify
> > > the
> > > reported problem.  However you may be using MPI_IN_PLACE 
> incorrectly
> > > in non-root process.
> > >
> > > On Tue, 13 Mar 2007, Martin Kleinschmidt wrote:
> > >
> > >> The corresponding lines of code are:
> > >>
> > >> #################
> > >> #ifdef PARALLEL
> > >>          if (myid .eq. 0) then
> > >>             call MPI_Reduce(MPI_IN_PLACE, vecf2(1),
> > >>      $           n*nneue,
> > >>      $           MPI_double_precision, MPI_SUM, 0,
> > >>      $           MPI_Comm_World, MPIerr)
> > >>          else
> > >>             call MPI_Reduce(vecf2(1),MPI_IN_PLACE,
> > >>      $           n*nneue,
> > >>      $           MPI_double_precision, MPI_SUM, 0,
> > >>      $           MPI_Comm_World, MPIerr)
> > >>          endif
> > >> #endif
> > >
> > > Try calling MPI_Reduce with MPI_IN_PLACE in send buffer in all
> > > ranks, i.e.
> > >
> > >               call MPI_Reduce(MPI_IN_PLACE, vecf2(1),
> > >        $           n*nneue,
> > >        $           MPI_DOUBLE_PRECISION, MPI_SUM, 0,
> > >        $           MPI_COMM_WORLD, MPIerr)
> > >
> > >
> > > A.Chan
> > >
> > >> #################
> > >> with n*nneue = 76160987, and 76160987*8 = 609287896, about 600 MB
> > >>
> > >> The point is: I thought, I could avoid the need for allocationg
> > >> additional memory by using MPI_IN_PLACE, which obviously does not
> > >> work.
> > >>
> > >> - do I use MPI_IN_PLACE in the right way?
> > >> - why does MPI_IN_PLACE need additional memory?
> > >> - is it possible to rewrite this code in a way that 
> eliminates the
> > >> need
> > >>   for allocating additional memory? This part of the code is not
> > >>   time-critical - it is executed once every few hours.
> > >>
> > >> (I'm using mpich2-1.0.5p2, Intel fortran compiler 
> 9.1.040, Intel C
> > >> compiler 9.1.045 for compiling both mpich and my code)
> > >>
> > >>    ...martin
> > >>
> > >>
> > >
> >
> >
> 
>