<div dir="ltr">Dear all,<div><br></div><div>I am playing around with creating DMPlex from a periodic Gmsh mesh (still in the same finite volume code that solves the Euler equations) because I want to run some classical periodic test cases from the literature (like convecting a vortex and so on).</div><div>I got from the mailing list and the examples that I gotta use -dm_plex_gmsh_periodic for the reader and the DM creator to do their job properly and read the $Periodic section from the Gmsh mesh. That works fine, I can write the DM in HDF5 format afterwards and the periodicity is clearly there (i can see in Paraview that opposite sides are "attached" to each other).</div><div>Now, in the context of the finite volume solver, I wrote my one right-hand side term computing routine, and I rely heavily on routines such as DMPlexGetFaceFields. Those routines, in turn rely on the ghost cells and ... I am having an issue with DMPlexConstructGhostCells. When I use -dm_plex_gmsh_periodic, DMPlexCreateFromFile works fine, DMSetBasicAdjacency(dm, true, false) works fine, DMPlexDistribute works fine, but when I get to DMPlexConstructGhostCells, I get</div><div><br></div><div> "<span style="font-variant-ligatures:no-common-ligatures;background-color:rgb(252,238,207);color:rgb(0,0,0);font-family:Menlo;font-size:11px">DM has boundary face 2048 with 2 support cells"</span></div>
<div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div><div>I understand the DM begin periodic, even the "boundary" faces have two neighbors because of the topological folding, so ... yeah, the ghost cells cannot really exist. But then how can I proceed ? Because I need that "ghost" label for all the routines I call from the RHS computing routine ...</div><div><br></div><div>I attach to this email the periodic Gmsh mesh I am playing with, and the routine from my code where I create the DM and so on:</div><div><br></div><div><div style="color:rgb(212,212,212);background-color:rgb(30,30,30);font-family:Menlo,Monaco,"Courier New",monospace;font-size:12px;line-height:18px;white-space:pre"><div> <span style="color:rgb(86,156,214)">subroutine</span> <span style="color:rgb(220,220,170)">initmesh</span></div><br><div> PetscErrorCode :: ierr</div><div> DM :: dmDist, dmGhost</div><div> <span style="color:rgb(86,156,214)">integer</span> :: overlap</div><div> PetscViewer :: vtkViewer</div><br><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">PetscLogEventBegin</span>(timer_initmesh, ierr); CHKERRA(ierr)</div><br><div> <span style="color:rgb(106,153,85)">! Number of neighbours taken into account in MP communications(1 - Order 1; 2 - Order 2)</span></div><div> overlap = <span style="color:rgb(181,206,168)">1</span></div><br><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">PetscPrintf</span>(PETSC_COMM_WORLD, <span style="color:rgb(206,145,120)">"Initializing mesh...\n"</span>, ierr) ; CHKERRA(ierr)</div><br><div> <span style="color:rgb(106,153,85)">! Force DMPlex to use gmsh marker</span></div><div> <span style="color:rgb(106,153,85)">! call PetscOptionsSetValue(PETSC_NULL_OPTIONS, "-dm_plex_gmsh_use_marker", "true", ierr); CHKERRA(ierr)</span></div><br><div> <span style="color:rgb(106,153,85)">! Read mesh from file name 'meshname'</span></div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">DMPlexCreateFromFile</span>(PETSC_COMM_WORLD, meshname, PETSC_TRUE, dm, ierr); CHKERRA(ierr)</div><br><div> <span style="color:rgb(106,153,85)">! Distribute on processors</span></div><div> <span style="color:rgb(106,153,85)">! Start with connectivity</span></div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">DMSetBasicAdjacency</span>(dm, PETSC_TRUE, PETSC_FALSE, ierr) ; CHKERRA(ierr)</div><br><div> <span style="color:rgb(106,153,85)">! Distribute on processors</span></div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">DMPlexDistribute</span>(dm, overlap, PETSC_NULL_SF, dmDist, ierr) ; CHKERRA(ierr)</div><br><div> <span style="color:rgb(106,153,85)">! Security check</span></div><div> <span style="color:rgb(197,134,192)">if</span> (dmDist <span style="color:rgb(86,156,214)">/=</span> PETSC_NULL_DM) <span style="color:rgb(197,134,192)">then</span></div><div> <span style="color:rgb(106,153,85)">! Destroy previous dm</span></div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">DMDestroy</span>(dm, ierr) ; CHKERRA(ierr)</div><div> <span style="color:rgb(106,153,85)">! Replace with dmDist</span></div><div> dm = dmDist</div><div> <span style="color:rgb(197,134,192)">end if</span></div><br><div> <span style="color:rgb(106,153,85)">! Finalize setup of the object</span></div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">DMSetFromOptions</span>(dm, ierr) ; CHKERRA(ierr)</div><br><div> <span style="color:rgb(106,153,85)">! Boundary condition with ghost cells</span></div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">DMPlexConstructGhostCells</span>(dm, PETSC_NULL_CHARACTER, PETSC_NULL_INTEGER, dmGhost, ierr); CHKERRA(ierr)</div><br><div> <span style="color:rgb(106,153,85)">! Security check</span></div><div> <span style="color:rgb(197,134,192)">if</span> (dmGhost <span style="color:rgb(86,156,214)">/=</span> PETSC_NULL_DM) <span style="color:rgb(197,134,192)">then</span></div><div> <span style="color:rgb(106,153,85)">! Destroy previous dm</span></div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">DMDestroy</span>(dm, ierr) ; CHKERRA(ierr)</div><div> <span style="color:rgb(106,153,85)">! Replace with dmGhost</span></div><div> dm = dmGhost</div><div> <span style="color:rgb(197,134,192)">end if</span></div><br><div> <span style="color:rgb(197,134,192)">if</span> (debug) <span style="color:rgb(197,134,192)">then</span></div><div> <span style="color:rgb(106,153,85)">! Show in terminal</span></div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">PetscPrintf</span>(PETSC_COMM_WORLD, <span style="color:rgb(206,145,120)">":: [DEBUG] Visualizing DM in console ::\n"</span>, ierr); CHKERRA(ierr)</div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">DMView</span>(dm, PETSC_VIEWER_STDOUT_WORLD, ierr) ; CHKERRA(ierr)</div><div> <span style="color:rgb(106,153,85)">! VTK viewer</span></div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">PetscViewerCreate</span>(PETSC_COMM_WORLD, vtkViewer, ierr) ; CHKERRA(ierr)</div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">PetscViewerSetType</span>(vtkViewer, PETSCVIEWERHDF5, ierr) ; CHKERRA(ierr)</div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">PetscViewerFileSetMode</span>(vtkViewer, FILE_MODE_WRITE, ierr) ; CHKERRA(ierr)</div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">PetscViewerFileSetName</span>(vtkViewer, <span style="color:rgb(206,145,120)">"debug_initmesh.h5"</span>, ierr) ; CHKERRA(ierr)</div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">DMView</span>(dm, vtkViewer, ierr) ; CHKERRA(ierr)</div><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">PetscViewerDestroy</span>(vtkViewer, ierr) ; CHKERRA(ierr)</div><div> <span style="color:rgb(197,134,192)">end if</span></div><br><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">PetscPrintf</span>(PETSC_COMM_WORLD, <span style="color:rgb(206,145,120)">"Done !\n"</span>, ierr) ; CHKERRA(ierr)</div><br><div> <span style="color:rgb(197,134,192)">call</span> <span style="color:rgb(220,220,170)">PetscLogEventEnd</span>(timer_initmesh, ierr); CHKERRA(ierr)</div><br><div> <span style="color:rgb(86,156,214)">end subroutine</span> <span style="color:rgb(220,220,170)">initmesh</span></div></div></div><div><br></div><div>Thank you very much for your help !</div><div><br></div><div>Best regards, </div><div><br></div><div>Thibault Bridel-Bertomeu</div></div></div></div></div></div></div></div></div></div>