[petsc-users] Implementing periodicity using DMPlex

Thu Jun 18 12:20:34 CDT 2020

Thank you for the help. I followed your suggestion and removed the
"DMPlexConstructGhostCells" function and now I am using a gmsh generated
mesh with periodicity in both directions, and distributing the mesh with 1
level of overlap. I can see that the periodic faces now have two
neighbouring cells on the opposite side of the domain which I can use to
get the periodic cell values using "DMGlobalToLocal" without the need to
construct ghost cells. There are two main issues that I am facing now,
regarding geometric information needed for implementing periodic bc.

One, I may need the centroid of periodic counterpart of a cell which is
possible to get from a "ghost" cell created on the periodic face, and
containing the solution data from periodic counterpart. But now, I would
need to shift the coordinates of the neighbouring cell accordingly. If you
see the sample output given below along with the screenshot of the sample
2D mesh that I have used (please find attached), while reconstructing the
left and right solution values on face 32 for a second order scheme, I need
the centroid of a ghost cell on left side of the face which is at (x, y) =
(-1.25, 8.75). I wanted to know if there is some way to get these shifted
"ghost" coordinates directly from the geometric information obtained using
"PetscFV" class. Kindly let me know if you have some suggestions regarding
this.

Sample Output:

face: 32, centroid = 0.000000, 8.750000, normal = 2.500000, -0.000000
support[0] = 0, cent = 1.250000, 8.750000, area = 6.250000
support[1] = 3, cent = 8.750000, 8.750000, area = 6.250000

face: 33, centroid = 1.250000, 7.500000, normal = -0.000000, -2.500000
support[0] = 0, cent = 1.250000, 8.750000, area = 6.250000
support[1] = 4, cent = 1.250000, 6.250000, area = 6.250000

Second, the face normals of the periodic faces obtained using the face
geometry data from "DMPlexGetDataFVM" function seem to be in the wrong
direction i.e., from the right cell "support[1]" to the left cell
"support[0]", while the other faces have the opposite direction. For face:
32, left cell is 0 and the right cell is 3, but, the normal points the
opposite way (cell 3 would be on the left side of cell 0 on periodic
boundary), while, for face: 33 which is an interior face, the normal
direction is from left to right cell.

Also, I am getting incorrect normal values for some of the faces (including
some interior faces). Here is a sample output of the same:

face: 59, centroid = 3.750000, 2.500000, normal = 0.000000, -7.500000
support[0] = 11, cent = 8.750000, 3.750000, area = 6.250000
support[1] = 15, cent = 8.750000, 1.250000, area = 6.250000

this face has the correct direction (from 11 to 15), but the value is wrong
(all the faces are 2.5 units). This problem does not occur when I ignore
periodicity by setting -dm_plex_gmsh_periodic to 0.

These issues occur regardless of the number of processes. I am attaching
the test code that I am using along with the gmsh file for your reference.
Please suggest to me, what might be the issue here.

Regards,
Shashwat

On Mon, Jun 15, 2020 at 2:54 PM Stefano Zampini <stefano.zampini at gmail.com>
wrote:

> It is enough if you use DMPlexDistribute with 1 level of overlap, set the
> local section for your dofs,  and call DMGlobalToLocal . The local vector
> will contain data for what you call “ghost” cells. I call them “not-owned’
> cells.
>
> On Jun 15, 2020, at 12:09 PM, Shashwat Tiwari <shaswat121994 at gmail.com>
> wrote:
>
> The way I'm trying to implement periodic bc is, when I loop over the
> boundary faces, say, on the left boundary of the domain to compute flux and
> residual, I need solution values from the two cells neighbouring the face,
> i.e. the left cell and the right cell (the face normal pointing from the
> left cell to the right cell and the right cell being a boundary ghost cell
> for boundary faces). For the boundary to be periodic, I need the value that
> I get from the right cell (boundary ghost cell) to be the solution value at
> its periodic counterpart, i.e. the solution value at left cell of the face
> on right boundary of the domain in this case. My question is how do I
> update the value at a boundary ghost cell with the value at the real cell
> which is its periodic counterpart from the other side of the domain. Is
> there some kind of mapping of the boundary ghost cells to their
> corresponding real cells which I can use to update the solution values at
> ghost cells?
>
> Regards,
> Shashwat
>
> On Sun, Jun 14, 2020 at 5:11 AM Matthew Knepley <knepley at gmail.com> wrote:
>
>> On Fri, Jun 12, 2020 at 3:19 PM Shashwat Tiwari <shaswat121994 at gmail.com>
>> wrote:
>>
>>> Hi,
>>> I am writing a first order 2D solver for unstructured grids with
>>> periodic boundaries using DMPlex. After generating the mesh, I use
>>> "DMSetPeriodicity" function to set periodicity in both directions. After
>>> which I partition the mesh (DMPlexDistribute), construct ghost cells
>>> (DMPlexConstructGhostCells),
>>>
>>
>> These ghost cells are for FVM boundary conditions. If you want cells to
>> be shared across parallel partitions, then you want to give overlap=1
>> to DMPlexDIstribute(). Is that what you want?
>>
>>   Thanks,
>>
>>      Matt
>>
>>
>>> create a section, and set some initial values in the global vector. Then
>>> I use "VecGhostUpdateBegin" to start updating the boundary ghost cell
>>> values, but, I get the following error in case I use multiple processors:
>>>
>>> [0]PETSC ERROR: Invalid argument
>>> [0]PETSC ERROR: Vector is not ghosted
>>> [0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html
>>> for trouble shooting.
>>>
>>> if I run with a single process, there is no error but the values remain
>>> empty (zero) and are not updated. Kindly let me know, if I am missing some
>>> crucial step before I can update the ghost values in order to implement the
>>> periodic bc, or if there is any other approach to achieve it. I am
>>> attaching a small code to demonstrate the issue for your reference.
>>>
>>> Regards,
>>> Shashwat
>>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> <http://www.cse.buffalo.edu/~knepley/>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200618/4493926c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mesh.png
Type: image/png
Size: 18621 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200618/4493926c/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.c
Type: text/x-csrc
Size: 5358 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200618/4493926c/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ug_periodic.geo
Type: application/octet-stream
Size: 518 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200618/4493926c/attachment-0001.obj>