[mpich-discuss] ROMIO individual file pointer

[Wei-keng Liao]

In ROMIO romio/adio/common/ad_set_view.c, line 60 states the individual 
file point, fd->fp_ind points to the first byte to be accessed. I can see 
fd->fp_ind is set to the right value in this file.

However, in romio/adio/common/ad_read_coll.c, function 
ADIOI_Calc_my_off_len() line 443, fd->fp_ind is set to the value of 
variable "off". Now, the problem is how "off" is calculated. From the 
codes between lines 428 and 438, "off" is moved up to the next flat_file 
segment, index j. Since flat_file may contains an empty-length element 
(either first or last) whose blocklens[] is equal to zero, when user 
buffer size "bufsize" is filled, "off" will not moved to the first byte to 
be accessed in the next collective I/O.

This problem appears when I defined a non-contiguous file view and did two 
collective I/O consecutively, each requesting data size equal to one whole 
file view. Then at the beginning of the second collective I/O, fd->fp_ind 
of all processes are having the same value, pointing to the beginning of 
the second file view, instead of individual starting offset.

The attached codes demonstrate this problem. If a printf statement for 
fd->fp_ind is inserted at the beginning of ADIOI_GEN_WriteStridedColl() in 
file ad_write_coll.c, the standard outputs are 
  0: First collective write -----------------------
  0: fd->fpind = 0
  1: fd->fpind = 5
  2: fd->fpind = 50
  3: fd->fpind = 55
  0: Second collective write -----------------------
  0: fd->fpind = 100
  1: fd->fpind = 100     <-- not right
  2: fd->fpind = 100     <-- not right
  3: fd->fpind = 100     <-- not right

The correct results for the second collective write should be:
  0: Second collective write -----------------------
  0: fd->fpind = 100
  1: fd->fpind = 105
  2: fd->fpind = 150
  3: fd->fpind = 155


My fix to this probelm is to insert the following codes in between
lines 435 and 436 of file ad_read_coll.c.

    while (flat_file->blocklens[j]==0) {
        j++;
        if (j == flat_file->count) {
            j = 0;
            n_filetypes++;
        }
    }


Wei-keng
-------------- next part --------------
#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>

/*----< main() >------------------------------------------------------------*/
int main(int argc, char **argv) {
    int          rank, np;
    char         buf[25];
    int          array_of_sizes[2], array_of_subsizes[2], array_of_starts[2];
    MPI_File     fh;
    MPI_Datatype ftype;
    MPI_Status   status;

    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &np);
    if (argc != 2) {
        if (!rank)fprintf(stderr,"Usage: %s filename\n",argv[0]);
	MPI_Finalize(); return 1;
    }
    if (np != 4) {
        if (!rank)fprintf(stderr,"Require 4 MPI processes ... exiting\n");
	MPI_Finalize(); return 1;
    }

    /*---- Set up MPI file view ---------------------------------------------*/
    array_of_sizes[0]    =    array_of_sizes[1] = 10;
    array_of_subsizes[0] = array_of_subsizes[1] = 5;

    if (rank == 0 || rank == 1) array_of_starts[0] = 0;
    if (rank == 2 || rank == 3) array_of_starts[0] = 5;
    if (rank == 0 || rank == 2) array_of_starts[1] = 0;
    if (rank == 1 || rank == 3) array_of_starts[1] = 5;

    MPI_Type_create_subarray(2, array_of_sizes, array_of_subsizes,
                             array_of_starts, MPI_ORDER_C, MPI_CHAR, &ftype);
    MPI_Type_commit(&ftype);

    /* open the file --------------------------------------------------------*/
    MPI_File_open(MPI_COMM_WORLD, argv[1], MPI_MODE_CREATE | MPI_MODE_WRONLY,
                  MPI_INFO_NULL, &fh);

    MPI_File_set_view(fh, 0, MPI_BYTE, ftype, "native", MPI_INFO_NULL);

    if (!rank) printf("First collective write -----------------------\n");
    MPI_File_write_all(fh, buf, 25, MPI_CHAR, &status);
    fflush(stdout);

    if (!rank) printf("Second collective write -----------------------\n");
    MPI_File_write_all(fh, buf, 25, MPI_CHAR, &status);

    MPI_File_close(&fh);

    MPI_Finalize();
    return 0;
}