[petsc-users] Mapping between application ordering and Petsc ordering

[Barry Smith]

> On Aug 8, 2015, at 3:08 PM, Mani Chandra <mc0710 at gmail.com> wrote:
> 
> Tried flipping the indices, I get a seg fault.

  You would have to be careful in exactly what you flip.  Note that the meaning of N1 and N2 etc would also be reversed between your code and the PETSc DMDA code.

  I would create a tiny DMDA and put entires like 1 2 3 4 ... into the array so you can track where the values go

  Barry

> 
> On Sat, Aug 8, 2015 at 3:03 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
> > On Aug 8, 2015, at 2:45 PM, Mani Chandra <mc0710 at gmail.com> wrote:
> >
> > Thanks. Any suggestions for a fix?
> 
>   Just flip the meaning of the x indices and the y indices in the PETSc parts of the code?
> 
>   Also run with a very different N1 and  N2 (instead of equal size) to better test the code coupling.
> 
>   Barry
> 
> 
> >
> > Reorder the indices in arrayApplication?
> >
> > On Sat, Aug 8, 2015 at 2:19 PM, Matthew Knepley <knepley at gmail.com> wrote:
> > On Sat, Aug 8, 2015 at 1:52 PM, Mani Chandra <mc0710 at gmail.com> wrote:
> > Hi,
> >
> > I'm having trouble interfacing petsc to an application which I think is related to the ordering of the nodes. Here's what I'm trying to do:
> >
> > The application uses a structured grid with a global array having dimensions N1 x N2, which is then decomposed into a local array with dimensions NX1 x NX2.
> >
> > I create a Petsc DMDA using
> >
> >     DMDACreate2d(MPI_COMM_WORLD,
> >                  DM_BOUNDARY_PERIODIC, DM_BOUNDARY_PERIODIC,
> >                  DMDA_STENCIL_BOX,
> >                  N1, N2,
> >                  N1/NX1, N2/NX2,
> >                  1, nghost, PETSC_NULL, PETSC_NULL,
> >                  &dmda);
> >
> > and then use this to create a vec:
> >
> >   DMCreateGlobalVector(dmda, &vec);
> >
> > Now I copy the local contents of the application array to the petsc array using the following:
> >
> > Let i, j be the application indices and iPetsc and jPetsc be petsc's indices, then:
> >
> > DMDAGetCorners(dmda, &iStart, &jStart, &kStart,
> >                                          &iSize, &jSize, &kSize
> >                               );
> >
> >
> > double **arrayPetsc;
> > DMDAVecGetArray(dmda, vec, &arrayPetsc);
> >
> > for (int j=0, jPetsc=jStart; j<NX2, jPetsc<jStart+jSize; j++, jPetsc++)
> > {
> >   for (int i=0, iPetsc=iStart; i<NX1, iPetsc<iStart+iSize; i++, iPetsc++)
> >   {
> >      arrayPetsc[jPetsc][iPetsc] = arrayApplication[j][i];
> >   }
> > }
> >
> > DMDAVecRestoreArray(dmda, vec, &arrayPetsc);
> >
> > Now if I VecView(vec, viewer) and look at the data that petsc has, it looks right when run with 1 proc, but if I use 4 procs it's all messed up (see attached plots).
> >
> > I should probably be using the AO object but its not clear how. Could you help me out?
> >
> > It looks like you have the global order of processes reversed, meaning you have
> >
> >   1   3
> >
> >   0   2
> >
> > and it should be
> >
> >   2  3
> >
> >   0  1
> >
> >   Thanks,
> >
> >       Matt
> >
> > Thanks,
> > Mani
> > --
> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> > -- Norbert Wiener
> >
> 
>