[mpich-discuss] MPI_Alltoallv with long* for displacements instead of int*

[Dave Goodell]

Have you actually encountered a concrete need for this functionality yet?  Or is this just a theoretical problem for the moment?  Also, for alltoallv in particular, recall that displacements are specified in units of type_size(recvtype), not bytes.  So for most use cases this will at least push the maximum displacement limit to 8GiB or beyond.  MPI_Alltoallw does have a stricter 2GiB limit because it deals in byte displacements.

Various proposals to deal with the implicit 32-bit limitation present in many routines of the MPI API have been brought to the MPI Forum.  But most of the comprehensive fixes have been rejected for various practical or aesthetic reasons.  My understanding is that these two tickets represent the current consensus proposals:

https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/265
https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/269

However, both have just recently been brought to the full Forum and there is a possibility that  one or neither will become part of the next MPI standard.  I personally think that #269 currently has some serious problems.

Ticket #265 makes it theoretically possible to work around the "int" displacement issue, although it will be very cumbersome to do so, IMO.

-Dave

On Mar 24, 2011, at 3:57 PM CDT, Raghu Reddy wrote:

> Thank you Dave!
> 
> I'm sure it is obvious why I was asking for this, but just to be documented in the thread I should have explicitly mentioned that the displacements I was dealing with are larger than the value that can be held in an int.
> 
> Machines with more memory than 2 GB/core are fairly common, and if one were to use 1 MPI rank per node, then it is very much possible to run into these limits.
> 
> Should this be recommend to be considered for the next standard?
> 
> Thanks again for the prompt response!
> 
> --rr
> Please Note: Email transcribed by speech recognition; May have unintended
> words in some cases, especially when I fail to do a good job of
> proofreading!
> 
> --On Thursday, March 24, 2011 3:36 PM -0500 Dave Goodell <goodell at mcs.anl.gov> wrote:
> 
>> On Mar 24, 2011, at 3:25 PM CDT, Raghu Reddy wrote:
>> 
>>> Currently the type signature for this routine from the Synopsis is:
>>> 
>>> # include "mpi.h"
>>> int MPI_Alltoallv (
>>>      void *sendbuf,
>>>      int *sendcnts,
>>>      int *sdispls,
>>>      MPI_Datatype sendtype,
>>>      void *recvbuf,
>>>      int *recvcnts,
>>>      int *rdispls,
>>>      MPI_Datatype recvtype,
>>>      MPI_Comm comm )
>>> 
>>> Is there a version of this routine with long* instead of int* for
>>> displacements?
>> 
>> No, such a routine is not specified by the MPI-2.2 standard, and MPICH2
>> contains no non-standard extension that would accomplish the same thing.
>> 
>> The MPI Forum is currently considering possible fixes for general problem
>> for the upcoming MPI-3 standard.  However, I am not sure that this
>> particular routine will be addressed in any way.
>> 
>> -Dave
>> 
>> _______________________________________________
>> mpich-discuss mailing list
>> mpich-discuss at mcs.anl.gov
>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
> 
>