DAcreate2d process layout order

[Sean Dettrick]

Hi Barry,
the order is determined by MPI_Cart_create.  I've tried to swap x and y 
in the DA to make the problem go away but haven't succeeded yet.  Not 
enough brain cells I suspect ... still thinking about it.
Thanks for the response,
Sean

Barry Smith wrote:

>
>
>   Sean,
>
>    I don't completely understand what goes wrong. Is it because YOUR
> application orders the processors related to geometry in the following 
> way?
>
>     ^ y direction
>     |
>        2   5  8
>        1   4  7
>        0   3  6
>
>                     -> x direction
>
> Or is this something inherent in MPI_Cart_create?
>
> PETSc does it so
>
>     ^ y direction
>     |
>        6   7  8
>        3   4  5
>        0   1  2
>
>                     -> x direction
>
>   If you want to "force" the PETSc DA to match the first case you 
> could just
> "lie" to PETSc and treat the x direction as the y direction and the x 
> as the y.
> Thus you would flip the i and j indices. You could do the same trick in
> 3d.
>
> There is no easy way to change the DA to do the ordering as the first
> case above or support both approaches (possible? yes, but ugly
> duplicate code to handle the two cases).
>
> I've cc:ed to Bill since he would know more about the details of 
> MPI_cart_create().
>
>    Barry
>
> On Mon, 15 May 2006, Sean Dettrick wrote:
>
>> Hi,
>>
>> I'm trying to use DACreate2d and KSP in my existing MPI application.  
>> I already have a Cartesian communicator established, and I set 
>> PETSC_COMM_WORLD equal to it and then call PetscInitialize.
>>
>> This works fine on a prime number of CPUs, because there is only one 
>> possible ordered MPI layout in one dimension.  But with a non-prime 
>> number there are two possible ordered layouts and it just happens 
>> that my 2D CPU layout (determined by MPI_Cart_create) is the 
>> transpose of the PETSc 2D CPU layout. Is there a way to organize the 
>> DA layout more explicitly than with DACreate2d?   Or to tell PETSc to 
>> transpose its CPU order?  I also wonder about the 3D case.
>>
>> thanks
>> Sean
>>
>>
>
>