[petsc-users] VecScatter from slice of global vector to sequential vector + AO errors

[Matthew Knepley]

On Thu, Oct 3, 2013 at 2:07 PM, Randall Mackie <rlmackie862 at gmail.com>wrote:

> I am trying to create a VecScatter that will scatter all the values from a
> horizontal slice of a 3D DMDA global vector (with dof=3) to a sequential
> vector on every processor. So far I have been unsuccessful, most likely
> because I don't completely understand how to get the appropriate IS to do
> this. I believe that you need to create a sequential IS on every processor
> that has the PETSc global number for the appropriate values I want to
> scatter. What I most recently tried was the following code which takes the
> values at level k:
>
>   DM          :: da3
>   Vec         :: daSlice, daVec
>   PetscInt    :: i, j, k, ierr, nis, istart, istep
>   IS          :: from, to
>   AO          :: ntog
>   VecScatter  :: vscat
>   PetscInt, allocatable :: indices(:)
>
>   ! Get the AO context for this DM
>   call DMDAGetAO(da3,ntog,ierr)
>
>   ! Create indices for scatter from global vector
>   allocate (indices(3*nx*ny))
>   istart=(k-1)*nx*ny*3
>   nis=nx*ny*3
>   do i=1,nis
>     indices(i)=istart+i-1
>   end do
>
>   ! Convert the indices from natural ordering to PETSc ordering
>   call AOApplicationToPetsc(ntog,nis,indices,ierr)
>
>   call
> ISCreateGeneral(PETSC_COMM_SELF,nis,indices,PETSC_COPY_VALUES,from,ierr)
>
>   ! Get the index set that specifies elements to scatter to sequential
> vector
>   istep=1
>   istart=0
>   nis=3*nx*ny
>   call ISCreateStride(PETSC_COMM_SELF,nis,istart,istep,to,ierr)
>
>   ! Create the Vector Scatter to get the values to every processor
>   call VecScatterCreate(daVec,from,daSlice,to,vscat,ierr)
>
>   ! Release memory
>   deallocate (indices)
>   call AODestroy(ntog,ierr)
>   call ISDestroy(from,ierr)
>   call ISDestroy(to,ierr)
>
>
> This code is throwing errors as follows
>
> [2]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [2]PETSC ERROR: Invalid argument!
> [2]PETSC ERROR: Wrong type of object: Parameter # 1!
> [2]PETSC ERROR:
> ------------------------------------------------------------------------
> [2]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013
> [2]PETSC ERROR: See docs/changes/index.html for recent updates.
> [2]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [2]PETSC ERROR: See docs/index.html for manual pages.
> [2]PETSC ERROR:
> ------------------------------------------------------------------------
> [2]PETSC ERROR: Libraries linked from
> /home/rmackie/PETSc/petsc-3.4.2/linux-gfortran-debug/lib
> [2]PETSC ERROR: Configure run at Thu Sep 19 15:08:47 2013
> [2]PETSC ERROR: Configure options --with-scalar-type=complex
> --with-debugging=1 --with-fortran=1 --download-mpich=1
> [2]PETSC ERROR:
> ------------------------------------------------------------------------
> [2]PETSC ERROR: AOApplicationToPetsc() line 261 in
> /home/rmackie/PETSc/petsc-3.4.2/src/vec/is/ao/interface/ao.c
> [0]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [0]PETSC ERROR: Invalid argument!
> [0]PETSC ERROR: Wrong type of object: Parameter # 1!
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [1]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [1]PETSC ERROR: Invalid argument!
> [1]PETSC ERROR: Wrong type of object: Parameter # 1!
> [1]PETSC ERROR:
> ------------------------------------------------------------------------
> [1]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013
> [1]PETSC ERROR: See docs/changes/index.html for recent updates.
> [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [1]PETSC ERROR: See docs/index.html for manual pages.
> [1]PETSC ERROR:
> ------------------------------------------------------------------------
> [1]PETSC ERROR: Libraries linked from
> /home/rmackie/PETSc/petsc-3.4.2/linux-gfortran-debug/lib
> [1]PETSC ERROR: Configure run at Thu Sep 19 15:08:47 2013
> [1]PETSC ERROR: Configure options --with-scalar-type=complex
> --with-debugging=1 --with-fortran=1 --download-mpich=1
> [1]PETSC ERROR:
> ------------------------------------------------------------------------
> [1]PETSC ERROR: AOApplicationToPetsc() line 261 in
> /home/rmackie/PETSc/petsc-3.4.2/src/vec/is/ao/interface/ao.c
> [0]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR: -----------------------------------
>
> etc.
>
>
> First, I don't see where my error is that is causing these problems, but
> second, is there a better way to create this vector scatter without having
> to use an AO? I suspect there is,  but so far I have been unable to figure
> out how to create the correct IS.
>

I think this is the best way to do what you want. The error says that
DMDAGetAO() does not actually give you an AO.
Can you see what you do get in the debugger? I am wondering if you have
overwrite in one of your index arrays.

  Matt


> Thanks very much,
>
> Randy
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20131003/7e2e246d/attachment.html>