[petsc-users] A problem with MPI Derived Data Type and 'calloc'

[Zhenglun (Alan) Wei]

Dear All,
      I hope you're having a nice day.
      I came across a problem while using MPI derived data type. 
Although it does not related to PETSc, I hope you guys can give me some 
suggestions or comments.
      The reason I want to do this is that, in my PETSc code, I want to 
send a set of data. It is fairly easy to do that if data is contiguous; 
while it would be comparatively hard if it is not. I searched online and 
found the MPI derived data type may help me. What I learned are 
MPI_TYPE_CONTIGUOUS, MPI_TYPE_VECTOR  and MPI_TYPE_STRUCT. I did a test 
program, which basically combined several example programs in 
"http://computing.llnl.gov/tutorials/mpi/#Derived_Data_Types" with all 
these three MPI derived data types. It works fine. (Please see my 
attachment for AlanFloatStruct.c). Briefly saying, it generates a 4 by 4 
matrix 'a[i][j]', where 'a' is a structure with members of 'u' and 'v':

a[i][j].u is
    01   02   03   04      // 2 digits just for E-mail format. Actually, 
you may find that data type for member '.u' is 'double'
    05   06   07   08
    09   10   11   12
    13   14   15   16
a[i][j].v = 10*a[i][j].u  // I will neglect .v later, because I want to 
focus on .u member

      As I defined, 'j' in 'a[i][j]' is the inner loop. For example, 
a[0][0] = 1,  a[0][1] = 2, a[0][2] = 3 ... So, they are contiguous. 
Then, 'i' is the outer loop, a[0][0] = 1, a[1][0] = 5 ..., which are not 
contiguous.
      My purpose is let rank = 0 to send the 4 contiguous data of 
a[n][m] (m = 0~3) to n'th rank and store them to 'b[n][p]' in the n'th 
rank. Then, I want to send the 4 non-contiguous data of a[m][n] (m = 
0~3) to n'th rank and store them to 'c[p][n]' in the n'th rank. For 
example, if 'b' and 'c' are 4 by 4 matrix and initialized by '-5', after 
communication with rank = 0, I expect in rank = 1

b[i][j].u
    -5   -5   -5   -5
    05   06   07   08
    -5   -5   -5   -5
    -5   -5   -5   -5
c[i][j].v
    -5   02   -5   -5
    -5   06   -5   -5
    -5   10   -5   -5
    -5   14   -5   -5

     Everything is fine in AlanFloatStruct.c. *However, when I was 
trying to use 'calloc' to allocate memory to 'a[][]' rather than 
directly define 'a[][]' as a structure matrix, the problem comes.* (see 
AlanTEST1.c). 'b' matrix is ok, yet, it keeps giving me problems for 'c' 
that
c[i][j].u
    -5   02   -5   -5
    -5   00   -5   -5
    -5   00   -5   -5
    -5   00   -5   -5

     I guess it would be some problem for byte counting or padding of 
MPI data in 'calloc'. I refer to 'Notes' part in 
'www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Type_struct/html' 
and made AlanTEST2.c. The problem is still there.
     I wonder what should I for this? Or, does PETS have any alternative 
way to fix this problem. BTW, if you want to run my code, you need 
change the name of .c file to AlanTEST.c. My script file 'AlanRun' is 
written based on AlanTEST.c.

thank you so much in advance,
Alan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120412/a4b7493e/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MPI_TYPE_STRUCT.zip
Type: application/x-zip-compressed
Size: 3845 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120412/a4b7493e/attachment.bin>