[petsc-users] Understanding DMDAGetElements()

[Jed Brown]

Constantine Khroulev <c.khroulev at gmail.com> writes:

> Hello PETSc developers,  I recently discovered 
> DMDAGetElements()[1] and got excited about using it... but then I 
> realized that I don't understand what it does well enough to use 
> it.   So, here is a question. Consider the four by four grid 
> pictured below, split across 2 processes. There are 16 nodes 
> (marked with MPI ranks that own them) and 9 elements. 
> #+BEGIN_SRC artist 
>   1---------1---------1---------1 |         |         |         | 
>   |    6    |    7    |    8    | |         |         |         | 
>   1---------1---------1---------1 |         |         |         | 
>   |    3    |    4    |    5    | |         |         |         | 
>   0---------0---------0---------0 |         |         |         | 
>   |    0    |    1    |    2    | |         |         |         | 
>   0---------0---------0---------0 
> #+END_SRC  I expect DMDAGetElements(...) to return 6 elements (0, 
> 1, 2, 3, 4, 5) on rank 0, because those are the elements I would 
> need to integrate over to add up contributions to nodes owned by 
> this rank. Similarly, on rank 1 I also expect to get 6 elements 
> (3, 4, 5, 6, 7, 8).   Instead I see 3 elements on rank 0 and 6 
> elements on rank 1.

The function is returning a non-overlapping partition of the elements.
If you integrate over those elements and use ADD_VALUES, you'll get the
correct result.

It would be more subtle if you were to get the closure of the owned
vertices, both at the insert stage and because some of those vertices
aren't in the local vector, so you wouldn't have the state required to
evaluate nonlinearities.

If you want different behavior, you should either do it in your code or
add a new variant to PETSc.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160202/c2793fd4/attachment.pgp>