[petsc-users] VecScatter from slice of global vector to sequential vector + AO errors

[Randall Mackie]

On Oct 3, 2013, at 12:24 PM, Matthew Knepley <knepley at gmail.com> wrote:

> On Thu, Oct 3, 2013 at 2:07 PM, Randall Mackie <rlmackie862 at gmail.com> wrote:
> I am trying to create a VecScatter that will scatter all the values from a horizontal slice of a 3D DMDA global vector (with dof=3) to a sequential vector on every processor. So far I have been unsuccessful, most likely because I don't completely understand how to get the appropriate IS to do this. I believe that you need to create a sequential IS on every processor that has the PETSc global number for the appropriate values I want to scatter. What I most recently tried was the following code which takes the values at level k:
> 
>   DM          :: da3
>   Vec         :: daSlice, daVec
>   PetscInt    :: i, j, k, ierr, nis, istart, istep
>   IS          :: from, to
>   AO          :: ntog
>   VecScatter  :: vscat
>   PetscInt, allocatable :: indices(:)
> 
>   ! Get the AO context for this DM
>   call DMDAGetAO(da3,ntog,ierr)
> 
>   ! Create indices for scatter from global vector
>   allocate (indices(3*nx*ny))
>   istart=(k-1)*nx*ny*3
>   nis=nx*ny*3
>   do i=1,nis
>     indices(i)=istart+i-1
>   end do
> 
>   ! Convert the indices from natural ordering to PETSc ordering
>   call AOApplicationToPetsc(ntog,nis,indices,ierr)
> 
>   call ISCreateGeneral(PETSC_COMM_SELF,nis,indices,PETSC_COPY_VALUES,from,ierr)
> 
>   ! Get the index set that specifies elements to scatter to sequential vector
>   istep=1
>   istart=0
>   nis=3*nx*ny
>   call ISCreateStride(PETSC_COMM_SELF,nis,istart,istep,to,ierr)
> 
>   ! Create the Vector Scatter to get the values to every processor
>   call VecScatterCreate(daVec,from,daSlice,to,vscat,ierr)
> 
>   ! Release memory
>   deallocate (indices)
>   call AODestroy(ntog,ierr)
>   call ISDestroy(from,ierr)
>   call ISDestroy(to,ierr)
> 
> 
> This code is throwing errors as follows
> 
> [2]PETSC ERROR: --------------------- Error Message ------------------------------------
> [2]PETSC ERROR: Invalid argument!
> [2]PETSC ERROR: Wrong type of object: Parameter # 1!
> [2]PETSC ERROR: ------------------------------------------------------------------------
> [2]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 
> [2]PETSC ERROR: See docs/changes/index.html for recent updates.
> [2]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [2]PETSC ERROR: See docs/index.html for manual pages.
> [2]PETSC ERROR: ------------------------------------------------------------------------
> [2]PETSC ERROR: Libraries linked from /home/rmackie/PETSc/petsc-3.4.2/linux-gfortran-debug/lib
> [2]PETSC ERROR: Configure run at Thu Sep 19 15:08:47 2013
> [2]PETSC ERROR: Configure options --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=1
> [2]PETSC ERROR: ------------------------------------------------------------------------
> [2]PETSC ERROR: AOApplicationToPetsc() line 261 in /home/rmackie/PETSc/petsc-3.4.2/src/vec/is/ao/interface/ao.c
> [0]PETSC ERROR: --------------------- Error Message ------------------------------------
> [0]PETSC ERROR: Invalid argument!
> [0]PETSC ERROR: Wrong type of object: Parameter # 1!
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: --------------------- Error Message ------------------------------------
> [1]PETSC ERROR: Invalid argument!
> [1]PETSC ERROR: Wrong type of object: Parameter # 1!
> [1]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 
> [1]PETSC ERROR: See docs/changes/index.html for recent updates.
> [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [1]PETSC ERROR: See docs/index.html for manual pages.
> [1]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: Libraries linked from /home/rmackie/PETSc/petsc-3.4.2/linux-gfortran-debug/lib
> [1]PETSC ERROR: Configure run at Thu Sep 19 15:08:47 2013
> [1]PETSC ERROR: Configure options --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=1
> [1]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: AOApplicationToPetsc() line 261 in /home/rmackie/PETSc/petsc-3.4.2/src/vec/is/ao/interface/ao.c
> [0]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR: -----------------------------------
> 
> etc.
> 
> 
> First, I don't see where my error is that is causing these problems, but second, is there a better way to create this vector scatter without having to use an AO? I suspect there is,  but so far I have been unable to figure out how to create the correct IS.
> 
> I think this is the best way to do what you want. The error says that DMDAGetAO() does not actually give you an AO.
> Can you see what you do get in the debugger? I am wondering if you have overwrite in one of your index arrays.
> 
>   Matt
>  

The error came from my call to AODestroy. Apparently, there is no need to destroy an AO that has been obtained by a call to DMDAGetAO.
Thanks for your help.

Randy


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20131003/1e89c424/attachment.html>