[MPICH] Using MPI_File_write_all()

[Christina Patrick]

Hi All,


I am trying to write a test program using MPI_File_write_all(). I have
defined a file view using MPI derived datatypes. When I write to the
file using MPI_File_write_at_all(), I get correct results. However,
using the same file view, when I write to the file using
MPI_File_write_all(), I get really weird results. My access pattern is
as below. The file/array consists of 16 rows and 4 columns and the
size of the collective buffer is 4 elements (1 row) (all for
illustrative purposes only, later I will scale the sizes). First 4
rows are written by P0, next 4 rows are written by P1 and so on.

Filesize = 256 bytes and Element Type = MPI_INT
------------------------------------------
|                                         |    P0
------------------------------------------
|                                         |    P1
------------------------------------------
|                                         |    P2
------------------------------------------
|                                         |    P3
------------------------------------------

The first write to the file using MPI_File_write_all() is at the
correct offset. However, all writes after the first write using
MPI_File_write_all() skip an offset of FILESIZE bytes from its
previous write and then writes to the file.





The program is as follows:

#define N                                   4
#define ROWS                           ((N)*4)
#define COLS                             (N)
#define DATATYPE                     MPI_INT
#define SIZEOF_DATATYPE        4
#define COLL_BUFSIZE              ((N)*(SIZEOF_DATATYPE))
#define ELTS_IN_COLL_BUF      ((COLL_BUFSIZE)/(SIZEOF_DATATYPE))
#define FILESIZE                       ((ROWS)*(COLS)*(SIZEOF_DATATYPE))
#define NUM_OF_ITERATIONS   ((FILESIZE)/(COLL_BUFSIZE))
#define FILENAME                     "pvfs2:/pvfs2-fs/test"

int main(int argc, char **argv) {
.....

  // array_of_blocklength                                       //
array_of_displacements                     // array_of_types
  blen[0] = 1;
indices[0] = 0;                                      oldtypes[0] =
MPI_LB;
  blen[1] = FILESIZE/nprocs/SIZEOF_DATATYPE; indices[1] =
mynod*(FILESIZE/nprocs);  oldtypes[1] = DATATYPE;
  blen[2] = 1;
indices[2] = nprocs*(FILESIZE/nprocs);  oldtypes[2] = MPI_UB;

  error_code = MPI_Type_struct(3, blen, indices, oldtypes, &newtype);
  MPI_ERR_CHECK(error_code);

  error_code = MPI_Type_commit(&newtype);
  MPI_ERR_CHECK(error_code);

  error_code = MPI_File_set_view(fhC, 0, DATATYPE, newtype, "native",
MPI_INFO_NULL);
  MPI_ERR_CHECK(error_code);

  iterations = (FILESIZE)/(COLL_BUFSIZE)/nprocs;

  for(i = 0, offset = 0, count = (ROWS/nprocs)*mynod*COLS; i <
iterations; i++, offset += COLS) {
    for (j = 0; j < ELTS_IN_COLL_BUF; j++) {
      buffer[j] = count++;
      fprintf(stdout, "buffer[%d] = %d\n", j, buffer[j]);
    }
#if 0 <============================
    error_code = MPI_File_write_at_all(fhC, offset, buffer,
ELTS_IN_COLL_BUF, DATATYPE, &status);
#else <============================
    error_code = MPI_File_write_all(fhC, buffer, ELTS_IN_COLL_BUF,
DATATYPE, &status);
#endif <============================
    MPI_ERR_CHECK(error_code);
  }

  error_code = MPI_File_close(&fhC);
  MPI_ERR_CHECK(error_code);
.....
}



When I use a seek() in with a write_all(), I get the correct results
(illustrated as below).

<============================
error_code = MPI_File_seek(fhC, offset, MPI_SEEK_SET);
error_code = MPI_File_write_all(fhC, buffer, ELTS_IN_COLL_BUF,
DATATYPE, &status);
<============================

Is MPI_File_write_all() always supposed to be used with a
MPI_File_seek() or am I doing something wrong.
Could you'll please help me so that I can understand
MPI_File_write_all() better. If you'll could also point me to some
good documentation, that would help too.

Thanks,
Christina.