[mpich-discuss] MPI-IO issue

[Grismer, Matthew J Civ USAF AFMC AFRL/RBAT]

Thanks for the info, I was not aware of the extent of the derived type.  I
used MPI_Type_create_resized to change the lower bound and extent of the
derived types on each processor to cover the entire array, not just the
sub-blocks I wanted.  Now x reads correctly on each processor, but then y
and z are still wrong.  After the MPI_File_read_all command for x, I used
the MPI_File_get_position command to see where each processors file pointer
was.  Only on processor 0 has the file pointer moved to the correct location
in the file, on all the other processors it is different and far short of
the correct location.  I'm defining the derived types and extents the same
way on all the processors, and even check the extents after I set them to
verify they are correct.  Any other thoughts on what I am missing?  Thanks.

Matt

-----Original Message-----
From: mpich-discuss-bounces at mcs.anl.gov
[mailto:mpich-discuss-bounces at mcs.anl.gov] On Behalf Of Wei-keng Liao
Sent: Wednesday, February 08, 2012 7:33 PM
To: mpich-discuss at mcs.anl.gov
Subject: Re: [mpich-discuss] MPI-IO issue

Hi,

I am guessing the problem is the datatype you created.
Please check the type extent to see if it equals to the
size of entire array, using MPI_Type_extent(). If not,
the file pointer after the first read will not move to
the beginning of the second array you expected, which
leads to reading data from a wrong file location.

Also, if each process reads a block subarray, I suggest
to use MPI_Type_create_subarray to create the new datatype.


Wei-keng


On Feb 8, 2012, at 3:46 PM, Grismer, Matthew J Civ USAF AFMC AFRL/RBAT
wrote:

> I am attempting to use MPI-IO to read block, structured data from a
> file.  Three three-dimensional coordinate arrays are stored in the file,
> one after another with Fortran ordering and in binary (not Fortran
> unformatted):
> 
> header info
> (((x(i,j,k),i=1,ie),j=1,je),k=1,ke)
> (((y(i,j,k),i=1,ie),j=1,je),k=1,ke)
> (((z(i,j,k),i=1,ie),j=1,je),k=1,ke)
> 
> Each processor reads a block subset of the arrays, and I've defined a
> derived type using MPI_TYPE_INDEXED for the blocks that go to each
> processor.  MPI_TYPE_INDEXED takes as arguments the number of blocks,
> block size, and block displacement;  I created (and committed) the
> datatype with the number of blocks/displacements 3 times (i.e.
> subsetie*subsetje*subsetke*3) what I need for one variable above on a
> given processor.  Then I used the following to read the file across the
> processors:
> 
> count = subsetie*subsetje*subsetke
> call MPI_FILE_SET_VIEW ( fh, skipheader, mpi_real8, newdatatype,
> "native", mpi_info_null, ierror )
> call MPI_FILE_READ_ALL ( fh, x, count, mpi_real8, ierror )
> call MPI_FILE_READ_ALL ( fh, y, count, mpi_real8, ierror ) 
> call MPI_FILE_READ_ALL ( fh, z, count, mpi_real8, ierror ) 
> 
> The result from this is x is read correctly, y and z are not.  However,
> if I define one variable xyz(subsetie,subsetje,subsetke,3) and read all
> the data in one call:
> 
> count = subsetie*subsetje*subsetke*3
> call MPI_FILE_SET_VIEW ( fh, skipheader, mpi_real8, newdatatype,
> "native", mpi_info_null, ierror )
> call MPI_FILE_READ_ALL ( fh, xyz, count, mpi_real8, ierror )
> 
> everything is read correctly, which also verifies my derived type is
> correct.  Alternatively I can reset the view (with correct
> displacements) after each READ_ALL and read into the individual
> variables.
> 
> Is this the expected behavior?  If so I do not understand how to make
> successive collective reads from a file without resetting the view
> before every read, which will take a toll on performance when I continue
> to read many additional variables from the file.
> 
> Matt
> _______________________________________________
> mpich-discuss mailing list     mpich-discuss at mcs.anl.gov
> To manage subscription options or unsubscribe:
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss

_______________________________________________
mpich-discuss mailing list     mpich-discuss at mcs.anl.gov
To manage subscription options or unsubscribe:
https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5688 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20120210/f3a2350c/attachment.bin>