DAcreate2d process layout order

[Barry Smith]

   Sean,

    I don't completely understand what goes wrong. Is it because YOUR
application orders the processors related to geometry in the following way?

     ^ y direction
     |
        2   5  8
        1   4  7
        0   3  6

                     -> x direction

Or is this something inherent in MPI_Cart_create?

PETSc does it so

     ^ y direction
     |
        6   7  8
        3   4  5
        0   1  2

                     -> x direction

   If you want to "force" the PETSc DA to match the first case you could just
"lie" to PETSc and treat the x direction as the y direction and the x as the y.
Thus you would flip the i and j indices. You could do the same trick in
3d.

There is no easy way to change the DA to do the ordering as the first
case above or support both approaches (possible? yes, but ugly
duplicate code to handle the two cases).

I've cc:ed to Bill since he would know more about the details of MPI_cart_create().

    Barry

On Mon, 15 May 2006, Sean Dettrick wrote:

> Hi,
>
> I'm trying to use DACreate2d and KSP in my existing MPI application.  I 
> already have a Cartesian communicator established, and I set PETSC_COMM_WORLD 
> equal to it and then call PetscInitialize.
>
> This works fine on a prime number of CPUs, because there is only one possible 
> ordered MPI layout in one dimension.  But with a non-prime number there are 
> two possible ordered layouts and it just happens that my 2D CPU layout 
> (determined by MPI_Cart_create) is the transpose of the PETSc 2D CPU layout. 
> Is there a way to organize the DA layout more explicitly than with 
> DACreate2d?   Or to tell PETSc to transpose its CPU order?  I also wonder 
> about the 3D case.
>
> thanks
> Sean
>
>