[mpich-discuss] MPI I/O using large integers in file type

[Rajeev Thakur]

Nick,
        With a 32-bit OS, you will have trouble using indexed types for files >4G, unless the indexing follows a regular pattern that is repeated. You may want to run on a machine with a 64-bit OS. You can use hindexed; just multiply the index by the size of the elementary type.

Rajeev

On Sep 7, 2011, at 5:38 PM, Nick Stokes wrote:

> Dear Rajeev, 
> 
> On Wed, Sep 7, 2011 at 3:27 PM, Nick Stokes <randomaccessiterator at gmail.com> wrote:
> 
> 
> On Wed, Sep 7, 2011 at 3:22 PM, Rajeev Thakur <thakur at mcs.anl.gov> wrote:
> MPI_Type_create_hindexed does.
> 
> 
> 
> Moreover, on my 32-bit system  MPI_Aint is typedef'ed to int (at least in my installation of MPICH2 - 1.3.3 on Windows 32-bit).  So hindexed type seems not to solve the large file problem in general.  If it were possible to create an indexed type with MPI_Offset displacements, that would have helped. 
> 
> Maybe I am on the wrong track here. Does my question make sense?  Perhaps a more explicit and perhaps more relate-able example would be as follows: 
> 
> Imagine a simulation where there are a total of 10^9  degrees of freedom, and double precision is used.  The computation result is to be output to a single file, which would add up to 8 GBs.  MPI is utilized to implement parallelization via domain partitioning, and each partition holds a certain chunk of data.   These chunks correspond to non-overlapping regions with gaps in the final output.   How would you do a collective READ/WRITE in such a case without integer overflow?
> 
> thanks many
> 
> _______________________________________________
> mpich-discuss mailing list     mpich-discuss at mcs.anl.gov
> To manage subscription options or unsubscribe:
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss