[petsc-users] MatSetValues dropping non-local entries

[Randall Mackie]

I already create my own matrix with the appropriate size and layout. The problem seems to be the local to global mapping from DMGetLocalToGlobalMapping, which I suspect does not allow for these non-local entries outside the stencil width.

How is one suppose to determine the local to global mapping without a call to this?

@Matthew: I had tried MatSetValuesStencil with the same result, and in fact the web page says this: The columns and rows in the stencil passed in MUST be contained within the ghost region of the given process as set with DMDACreateXXX() or MatSetStencil <http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetStencil.html#MatSetStencil>(). 

Randy

> On Aug 24, 2016, at 2:52 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
> 
>> On Aug 24, 2016, at 4:45 PM, Randall Mackie <rlmackie862 at gmail.com> wrote:
>> 
>> Well, I only need this particular matrix to multiply a vector (ordering based on the DMDA grid), so I don’t need to do any ghost communication (like residual calculations). I just need to be able to set a few non-local entries. Is there no way to do that without increasing the stencil width of the DMDA?
> 
>   Create your own matrix of the appropriate size and layout to match the DMDA vector and then put your values in it with MatSetValues(); don't use the matrix from DMCreateMatrix()
> 
>  Barry
> 
>> 
>> Randy
>> 
>> 
>>> On Aug 24, 2016, at 2:39 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>> 
>>> 
>>>> On Aug 24, 2016, at 4:27 PM, Randall Mackie <rlmackie862 at gmail.com> wrote:
>>>> 
>>>> I’ve run into a situation where MatSetValues seems to be dropping non-local entries. Most of the entries that are set are local, but a few are possibly non-local, and are only maximum a few grid points off the local part of the grid.
>>>> 
>>>> Specifically, I get the local to global mapping, and the indices like so:
>>>> 
>>>> call DMGetLocalToGlobalMapping(da,ltogm,ierr)
>>>> call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr)
>>>> 
>>>> then set the row using ltog(idltog + row +1) etc
>>>> 
>>>> When run on 1 process, everything worked fine, but for > 1 I was not getting the right result (I know what the right answer should be for a simple case).
>>>> 
>>>> I found that when I increased the stencil width on the DA (in the call to DACreate3d) that was used in the DMGetLocalToGlobalMapping to be large enough that the non-local points would be in the ghost region, everything was fine even for > 1 process.
>>>> 
>>>> 
>>>> So, in conclusion, it seems like if I use a local to global mapping from DMGetLocalToGlobalMapping, then MatSetValues will drop any non-local entries that are not included in the ghost region.
>>>> 
>>>> 
>>>> Is this the correct behavior and if so, is there another way to set these non-local values so they don’t get dropped?
>>> 
>>> Yes, this is the expected behavior. Note also that DMDA only allocates space in the matrix for these locations and if it did stick in your "extra" locations it would be very very slow because it would have to reallocate the matrix data structures. 
>>> 
>>> Why would you want to put in matrix entries that are not represented in the ghosting? The whole point of the ghosting is to indicate what values need to be communicated so you putting additional values in that do not fit the ghosting does not match the paradigm.
>>> 
>>> Barry
>>> 
>>>> 
>>>> 
>>>> Thanks, 
>>>> 
>>>> Randy
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160824/1e0f12b5/attachment-0001.html>