[petsc-users] MatSetValues dropping non-local entries

[Matthew Knepley]

On Wed, Aug 24, 2016 at 5:01 PM, Randall Mackie <rlmackie862 at gmail.com>
wrote:

> I already create my own matrix with the appropriate size and layout. The
> problem seems to be the local to global mapping from
> DMGetLocalToGlobalMapping, which I suspect does not allow for these
> non-local entries outside the stencil width.
>
> How is one suppose to determine the local to global mapping without a call
> to this?
>
> @Matthew: I had tried MatSetValuesStencil with the same result, and in
> fact the web page says this: The columns and rows in the stencil passed
> in MUST be contained within the ghost region of the given process as set
> with DMDACreateXXX() or MatSetStencil
> <http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetStencil.html#MatSetStencil>
> ().
>

You are right. In the deep past, we determined this directly using the
dimensions. Now we use the map.

You will have to calculate the global indices you want by hand in the PETSc
ordering, which means
knowing what process you want owns your (i,j,k). This is of course tedious,
which is why we prefer to
use the maps, but its not possible to store every global index in the local
map.

   Matt


> Randy
>
> On Aug 24, 2016, at 2:52 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
>
> On Aug 24, 2016, at 4:45 PM, Randall Mackie <rlmackie862 at gmail.com> wrote:
>
> Well, I only need this particular matrix to multiply a vector (ordering
> based on the DMDA grid), so I don’t need to do any ghost communication
> (like residual calculations). I just need to be able to set a few non-local
> entries. Is there no way to do that without increasing the stencil width of
> the DMDA?
>
>
>   Create your own matrix of the appropriate size and layout to match the
> DMDA vector and then put your values in it with MatSetValues(); don't use
> the matrix from DMCreateMatrix()
>
>  Barry
>
>
> Randy
>
>
> On Aug 24, 2016, at 2:39 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
>
> On Aug 24, 2016, at 4:27 PM, Randall Mackie <rlmackie862 at gmail.com> wrote:
>
> I’ve run into a situation where MatSetValues seems to be dropping
> non-local entries. Most of the entries that are set are local, but a few
> are possibly non-local, and are only maximum a few grid points off the
> local part of the grid.
>
> Specifically, I get the local to global mapping, and the indices like so:
>
> call DMGetLocalToGlobalMapping(da,ltogm,ierr)
> call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr)
>
> then set the row using ltog(idltog + row +1) etc
>
> When run on 1 process, everything worked fine, but for > 1 I was not
> getting the right result (I know what the right answer should be for a
> simple case).
>
> I found that when I increased the stencil width on the DA (in the call to
> DACreate3d) that was used in the DMGetLocalToGlobalMapping to be large
> enough that the non-local points would be in the ghost region, everything
> was fine even for > 1 process.
>
>
> So, in conclusion, it seems like if I use a local to global mapping from
> DMGetLocalToGlobalMapping, then MatSetValues will drop any non-local
> entries that are not included in the ghost region.
>
>
> Is this the correct behavior and if so, is there another way to set these
> non-local values so they don’t get dropped?
>
>
> Yes, this is the expected behavior. Note also that DMDA only allocates
> space in the matrix for these locations and if it did stick in your "extra"
> locations it would be very very slow because it would have to reallocate
> the matrix data structures.
>
> Why would you want to put in matrix entries that are not represented in
> the ghosting? The whole point of the ghosting is to indicate what values
> need to be communicated so you putting additional values in that do not fit
> the ghosting does not match the paradigm.
>
> Barry
>
>
>
> Thanks,
>
> Randy
>
>
>
>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160824/bde44be9/attachment.html>