[mpich-discuss] Fortran integer support 4/8-bytes

[Martin Siegert]

On Fri, Jun 08, 2012 at 07:13:08AM -0500, Jed Brown wrote:
> 
>    On Fri, Jun 8, 2012 at 4:30 AM, Reuti <[1]reuti at staff.uni-marburg.de>
>    wrote:
> 
>    Aha, I thought the supported data types were meant.
>    For a Fortran integer 64 interface I had to compile a special version
>    for an application called Dirac:
>    [2]http://wiki.chem.vu.nl/dirac/index.php/How_to_build_MPI_libraries_fo
>    r_64-bit_integers
> 
>    But this only changes the size of Fortran integer. It doesn't change C
>    int and in practice, the Fortran bindings call C bindings that use C
>    int, so larger values would be truncated. So you can get 8-byte
>    integers in Fortran if you're determined, but it doesn't circumvent the
>    2^31 limit.
> 
>    "Do these permit values greater than what is representable in a 4-byte
>    integer or does the lower level implementation in C preclude the use
>    of arguments greater than 2^31?"
> 
> References
> 
>    1. mailto:reuti at staff.uni-marburg.de
>    2. http://wiki.chem.vu.nl/dirac/index.php/How_to_build_MPI_libraries_for_64-bit_integers

I have compiled dirac with openmpi, but I am assuming that mpich is
similar. When you configure openmpi and specify a Fortran compiler
that uses 8-bit integers (usually by specifying flags like
-fdefault-integer-8 (gfortran) or -i8 (ifort)) the configure script
determines the mapping of Fortran integers (MPI_INTEGER) to C integers
(MPI_INT). Thus, those datatypes will get mapped correctly:
integer -> long (on x86_64), e.g.,
call MPI_Send(buf, count, MPI_INTEGER, dest, tag, comm, mpierr)
should work correctly send an array buf corresponding to
long *buf in C.
However, the big problem that the dirac authors completely ignore
is that all "count" variables as in

MPI_Send(*buf, count, datatype, dest, tag, comm)

are int in C, i.e., 32-bit. No mapping takes place there. I.e., when
you use

call MPI_Send(buf, count, datatype, dest, tag, comm, mpierr)

with "count" being a 64-bit integer in Fortran you should check first
whether count is smaller than 2^31. Dirac does not do that and this
has caused me tremenduous grief. There are a bunch of MPI_Allreduce
statements that I needed to wrap, because otherwise they would cause
integer overflows. There is a potential of this happening all over
the code.

I am not aware of the MPI-2 standard even addressing the problem of
the size of the count variables, e.g., the definitions

int MPI_Send(void* buf, int count, MPI_Datatype datatype, int dest,
             int tag, MPI_Comm comm)

MPI_SEND(BUF, COUNT, DATATYPE, DEST, TAG, COMM, IERROR)
<type> BUF(*)
INTEGER COUNT, DATATYPE, DEST, TAG, COMM, IERROR

are in principle contradictory since "int count" and "INTEGER COUNT"
may not be of the same size. If that would really be allowed then
all MPI distributions would have to check the value of the Fortran count
variable first and then call the corresponding C routine repeatedly, if
necessary. As far as I know this is not done.
I am still hoping that MPI-3 will solve the problem and somehow make
it possible to have "count" variables larger than 2^31 ...

Cheers,
Martin

-- 
Martin Siegert
WestGrid/ComputeCanada
Simon Fraser University
Burnaby, British Columbia
Canada