<div dir="ltr">On Thu, Oct 3, 2013 at 2:07 PM, Randall Mackie <span dir="ltr"><<a href="mailto:rlmackie862@gmail.com" target="_blank">rlmackie862@gmail.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I am trying to create a VecScatter that will scatter all the values from a horizontal slice of a 3D DMDA global vector (with dof=3) to a sequential vector on every processor. So far I have been unsuccessful, most likely because I don't completely understand how to get the appropriate IS to do this. I believe that you need to create a sequential IS on every processor that has the PETSc global number for the appropriate values I want to scatter. What I most recently tried was the following code which takes the values at level k:<div>
<br></div><div><div> DM :: da3</div><div> Vec :: daSlice, daVec</div><div> PetscInt :: i, j, k, ierr, nis, istart, istep</div><div> IS :: from, to</div><div> AO :: ntog</div><div>
VecScatter :: vscat</div><div> PetscInt, allocatable :: indices(:)</div><div><br></div><div> ! Get the AO context for this DM</div><div> call DMDAGetAO(da3,ntog,ierr)</div><div><br></div><div> ! Create indices for scatter from global vector</div>
<div> allocate (indices(3*nx*ny))</div><div> istart=(k-1)*nx*ny*3</div><div> nis=nx*ny*3</div><div> do i=1,nis</div><div> indices(i)=istart+i-1</div><div> end do</div><div><br></div><div> ! Convert the indices from natural ordering to PETSc ordering</div>
<div> call AOApplicationToPetsc(ntog,nis,indices,ierr)</div><div><br></div><div> call ISCreateGeneral(PETSC_COMM_SELF,nis,indices,PETSC_COPY_VALUES,from,ierr)</div><div><br></div><div> ! Get the index set that specifies elements to scatter to sequential vector</div>
<div> istep=1</div><div> istart=0</div><div> nis=3*nx*ny</div>
<div> call ISCreateStride(PETSC_COMM_SELF,nis,istart,istep,to,ierr)</div><div><br></div><div> ! Create the Vector Scatter to get the values to every processor</div><div> call VecScatterCreate(daVec,from,daSlice,to,vscat,ierr)</div>
<div><br></div><div> ! Release memory</div><div> deallocate (indices)</div><div> call AODestroy(ntog,ierr)</div><div> call ISDestroy(from,ierr)</div><div> call ISDestroy(to,ierr)</div></div><div><br></div><div><br></div>
<div>This code is throwing errors as follows</div><div><br></div><div><div>[2]PETSC ERROR: --------------------- Error Message ------------------------------------</div><div>[2]PETSC ERROR: Invalid argument!</div><div>[2]PETSC ERROR: Wrong type of object: Parameter # 1!</div>
<div>[2]PETSC ERROR: ------------------------------------------------------------------------</div><div>[2]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 </div><div>[2]PETSC ERROR: See docs/changes/index.html for recent updates.</div>
<div>[2]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div><div>[2]PETSC ERROR: See docs/index.html for manual pages.</div><div>[2]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[2]PETSC ERROR: Libraries linked from /home/rmackie/PETSc/petsc-3.4.2/linux-gfortran-debug/lib</div>
<div>[2]PETSC ERROR: Configure run at Thu Sep 19 15:08:47 2013</div><div>[2]PETSC ERROR: Configure options --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=1</div><div>[2]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[2]PETSC ERROR: AOApplicationToPetsc() line 261 in /home/rmackie/PETSc/petsc-3.4.2/src/vec/is/ao/interface/ao.c</div><div>[0]PETSC ERROR: --------------------- Error Message ------------------------------------</div>
<div>[0]PETSC ERROR: Invalid argument!</div><div>[0]PETSC ERROR: Wrong type of object: Parameter # 1!</div><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div><div>[1]PETSC ERROR: --------------------- Error Message ------------------------------------</div>
<div>[1]PETSC ERROR: Invalid argument!</div><div>[1]PETSC ERROR: Wrong type of object: Parameter # 1!</div><div>[1]PETSC ERROR: ------------------------------------------------------------------------</div><div>[1]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 </div>
<div>[1]PETSC ERROR: See docs/changes/index.html for recent updates.</div><div>[1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div><div>[1]PETSC ERROR: See docs/index.html for manual pages.</div><div>
[1]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[1]PETSC ERROR: Libraries linked from /home/rmackie/PETSc/petsc-3.4.2/linux-gfortran-debug/lib</div><div>[1]PETSC ERROR: Configure run at Thu Sep 19 15:08:47 2013</div><div>[1]PETSC ERROR: Configure options --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=1</div>
<div>[1]PETSC ERROR: ------------------------------------------------------------------------</div><div>[1]PETSC ERROR: AOApplicationToPetsc() line 261 in /home/rmackie/PETSc/petsc-3.4.2/src/vec/is/ao/interface/ao.c</div>
<div>[0]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 </div><div>[0]PETSC ERROR: See docs/changes/index.html for recent updates.</div><div>[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div>
<div>[0]PETSC ERROR: See docs/index.html for manual pages.</div><div>[0]PETSC ERROR: -----------------------------------</div></div><div><br></div><div>etc.</div><div><br></div><div><br></div><div>First, I don't see where my error is that is causing these problems, but second, is there a better way to create this vector scatter without having to use an AO? I suspect there is, but so far I have been unable to figure out how to create the correct IS.</div>
</div></blockquote><div><br></div><div>I think this is the best way to do what you want. The error says that DMDAGetAO() does not actually give you an AO.</div><div>Can you see what you do get in the debugger? I am wondering if you have overwrite in one of your index arrays.</div>
<div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Thanks very much,</div><div><br></div><div>Randy</div><div>
<br>
</div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener
</div></div>