DAcreate2d process layout order

[Sean Dettrick]

Barry Smith wrote:

>
> Bill, does the MPI standard dictate this decomposition or
> could different implementations do it the opposite way?
> Then we'd have to make the DA logic a bit more complicated.

I don't have a copy of the standard, but to quote page 255 of "MPI, the 
complete reference" by Snir et al:
 "Row-major numbering is always used for the processes in a Cartesian 
structure".
Their diagram in figure 6.1 matches my code output for coords couplets 
(i,j):

  0      1       2       3
(0,0)  (0,1)   (0,2)   (0,3)

  4      5       6       7
(1,0)  (1,1)   (1,2)   (1,3)

  8      9      10      11
(2,0)  (2,1)   (2,2)   (2,3)


By the way I agree with you, I *should* be able to swap the x and y 
myself.  Just haven't had much luck yet in that regard.

Sean

>
>
> On Thu, 18 May 2006, Sean Dettrick wrote:
>
>> Barry Smith wrote:
>>
>>> On Thu, 18 May 2006, Sean Dettrick wrote:
>>>
>>>> Hi Barry,
>>>> the order is determined by MPI_Cart_create.
>>>
>>>
>>>
>>>    Do you mean that MPI_Cart_create() orders across the 2nd (y-axis)
>>> fastest and then the first (x-axis)? Hmmm, maybe we should change the
>>> DA? Changing it once and for all (not supporting both) is probably
>>> not a big deal and shouldn't break much (I hope).
>>
>>
>> Hi Barry,
>>
>> it depends, what do you call x and what do you call y?
>> MPI_Cart_coords returns a vector, coords - I tend to say x is 
>> coords[0], y is coords[1] and z is coords[2]. For what it's worth, 
>> there's a short code appended to this email, which produces:
>>
>> rank = 0 has Cartesian coords = { 0, 0 }
>> rank = 1 has Cartesian coords = { 0, 1 }
>> rank = 2 has Cartesian coords = { 1, 0 }
>> rank = 3 has Cartesian coords = { 1, 1 }
>> rank = 0 has DA range x=[0,50) and y=[0,50)
>> rank = 1 has DA range x=[50,100) and y=[0,50)
>> rank = 2 has DA range x=[0,50) and y=[50,100)
>> rank = 3 has DA range x=[50,100) and y=[50,100)
>>
>>>>>    I don't completely understand what goes wrong. Is it because YOUR
>>>>> application orders the processors related to geometry in the 
>>>>> following way?
>>>>>
>>>>>     ^ y direction
>>>>>     |
>>>>>        2   5  8
>>>>>        1   4  7
>>>>>        0   3  6
>>>>>
>>>>>                     -> x direction
>>>>>
>>>>> Or is this something inherent in MPI_Cart_create?
>>>>
>>>>
>>
>> For my interpretation of x and y, MPI_Cart_create produces the above 
>> layout. But if I said x=coords[1] and y=coords[0], then it would 
>> match the one below.
>>
>>>>>
>>>>> PETSc does it so
>>>>>
>>>>>     ^ y direction
>>>>>     |
>>>>>        6   7  8
>>>>>        3   4  5
>>>>>        0   1  2
>>>>>
>>>>>                     -> x direction
>>>>>
>>>>>
>>>>
>>
>> Code and makefile attached ... hopefully within the message size limit.
>> Just make cartcommtest.
>>
>> Sean
>>
>>
>>
>>
>
>