[petsc-users] Mapping between application ordering and Petsc ordering

[Mani Chandra]

Hi,

I'm having trouble interfacing petsc to an application which I think is
related to the ordering of the nodes. Here's what I'm trying to do:

The application uses a structured grid with a global array having
dimensions N1 x N2, which is then decomposed into a local array with
dimensions NX1 x NX2.

I create a Petsc DMDA using

    DMDACreate2d(MPI_COMM_WORLD,
                 DM_BOUNDARY_PERIODIC, DM_BOUNDARY_PERIODIC,
                 DMDA_STENCIL_BOX,
                 N1, N2,
                 N1/NX1, N2/NX2,
                 1, nghost, PETSC_NULL, PETSC_NULL,
                 &dmda);

and then use this to create a vec:

  DMCreateGlobalVector(dmda, &vec);

Now I copy the local contents of the application array to the petsc array
using the following:

Let i, j be the application indices and iPetsc and jPetsc be petsc's
indices, then:

DMDAGetCorners(dmda, &iStart, &jStart, &kStart,
                                         &iSize, &jSize, &kSize
                              );


double **arrayPetsc;
DMDAVecGetArray(dmda, vec, &arrayPetsc);

for (int j=0, jPetsc=jStart; j<NX2, jPetsc<jStart+jSize; j++, jPetsc++)
{
  for (int i=0, iPetsc=iStart; i<NX1, iPetsc<iStart+iSize; i++, iPetsc++)
  {
     arrayPetsc[jPetsc][iPetsc] = arrayApplication[j][i];
  }
}

DMDAVecRestoreArray(dmda, vec, &arrayPetsc);

Now if I VecView(vec, viewer) and look at the data that petsc has, it looks
right when run with 1 proc, but if I use 4 procs it's all messed up (see
attached plots).

I should probably be using the AO object but its not clear how. Could you
help me out?

Thanks,
Mani
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150808/c045b080/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1_proc.png
Type: image/png
Size: 31474 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150808/c045b080/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 4_proc.png
Type: image/png
Size: 33689 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150808/c045b080/attachment-0003.png>