[petsc-dev] DMPlex overlap redistribution

[Matthew Knepley]

On Wed, Mar 6, 2024 at 5:44 PM Adrian Croucher <a.croucher at auckland.ac.nz>
wrote:

> hi Matt,
> On 7/03/24 2:55 am, Matthew Knepley wrote:
>
> Let's do this. Can you send me a dual porosity mesh, and I will figure out
> why the stratification
> algorithm does not work? I think you can just output the Plex as binary or
> HDF5, but you could
> also give me code that makes one.
>
> OK, I've attached an example dual porosity / MINC DM, output before the
> attempted redistribution.
>
>  The original mesh is a vertical column of 11 hexahedral cells. Six of the
> cells (2 - 7) have two levels of interior MINC cells added (i.e. an
> interior cell, and then another interior cell inside that). So there are a
> total of 11 + 2*6 = 23 cells. The DM was distributed over two processes.
>
> For each extra interior face I also create one interior edge and one
> interior vertex. I give the interior vertices arbitrary (0,0,0) locations.
> I assign all the extra interior DM points (cells, vertices, faces, edges)
> cell type DM_POLYTOPE_INTERIOR_GHOST.
>
> Okay, here are my first steps

1. I fixed the problem with repeated stratification

knepley/fix-da-periodicity *$:/PETSc3/petsc/petsc-dev$ git diff
diff --git a/src/dm/impls/plex/plex.c b/src/dm/impls/plex/plex.c
index 0ffc372ea08..9d27d308538 100644
--- a/src/dm/impls/plex/plex.c
+++ b/src/dm/impls/plex/plex.c
@@ -4439,6 +4439,7 @@ PetscErrorCode DMPlexStratify(DM dm)
   PetscCall(PetscLogEventBegin(DMPLEX_Stratify, dm, 0, 0, 0));

   // Create depth label
+  PetscCall(DMRemoveLabel(dm, "depth", NULL));
   PetscCall(DMCreateLabel(dm, "depth"));

   PetscCall(DMPlexGetDepthLabel(dm, &label));

I put it in an MR: https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/merge_requests/7347__;!!G_uCfscf7eWS!akE1sOGj7q57_P3AdMmPLRvCeEmgzsdIGxognMBQWmrcCwBpjPGdMYH9Z9FEokIWKyOyOXJH7cBRBon3iXCi$ 

2. I wrote a small program to load the mesh, and then restratify it. It is
attached.

3. I ran stratification on this mesh, but I did not get an error:

master *:~/Downloads/tmp/Adrian$ ./minc -dm_plex_filename minc_dm.h5
-dm_view

DM Object: 1 MPI process

  type: plex

DM_0x84000000_0 in 3 dimensions:

  Number of 0-cells per rank: 60 (12)

  Number of 1-cells per rank: 104 (12)

  Number of 2-cells per rank: 68 (12)

  Number of 3-cells per rank: 23 (12)

Labels:

  depth: 4 strata with value/size (0 (60), 1 (104), 2 (68), 3 (23))

  Cell Sets: 1 strata with value/size (0 (11))

  boundary: 1 strata with value/size (1 (46))

  celltype: 5 strata with value/size (0 (48), 1 (92), 13 (48), 4 (56), 7
(11))

  ghost: 1 strata with value/size (1 (46))

  minc_level: 3 strata with value/size (0 (6), 1 (6), 2 (6))

  minc_rocktype_zone: 1 strata with value/size (1 (6))

  minc_zone: 1 strata with value/size (1 (6))

  open_boundary: 1 strata with value/size (1 (1))

  rock_type: 2 strata with value/size (1 (5), 3 (6))

  source: 2 strata with value/size (0 (1), 1 (1))

  vtk: 1 strata with value/size (1 (11))

  zone_minc: 1 strata with value/size (1 (6))

DM Object: DM_0x84000000_0 1 MPI process

  type: plex

DM_0x84000000_0 in 3 dimensions:

  Number of 0-cells per rank: 60 (12)

  Number of 1-cells per rank: 104 (12)

  Number of 2-cells per rank: 68 (12)

  Number of 3-cells per rank: 23 (12)

Labels:

  Cell Sets: 1 strata with value/size (0 (11))

  boundary: 1 strata with value/size (1 (46))

  celltype: 5 strata with value/size (0 (48), 1 (92), 13 (48), 4 (56), 7
(11))

  ghost: 1 strata with value/size (1 (46))

  minc_level: 3 strata with value/size (0 (6), 1 (6), 2 (6))

  minc_rocktype_zone: 1 strata with value/size (1 (6))

  minc_zone: 1 strata with value/size (1 (6))

  open_boundary: 1 strata with value/size (1 (1))

  rock_type: 2 strata with value/size (1 (5), 3 (6))

  source: 2 strata with value/size (0 (1), 1 (1))

  vtk: 1 strata with value/size (1 (11))

  zone_minc: 1 strata with value/size (1 (6))

  depth: 4 strata with value/size (0 (60), 1 (104), 2 (68), 3 (23))

> - Adrian
>
> --
> Dr Adrian Croucher
> Senior Research Fellow
> Department of Engineering Science
> Waipapa Taumata Rau / University of Auckland, New Zealand
> email: a.croucher at auckland.ac.nz
> tel: +64 (0)9 923 4611
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!akE1sOGj7q57_P3AdMmPLRvCeEmgzsdIGxognMBQWmrcCwBpjPGdMYH9Z9FEokIWKyOyOXJH7cBRBlTTBnv2$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!akE1sOGj7q57_P3AdMmPLRvCeEmgzsdIGxognMBQWmrcCwBpjPGdMYH9Z9FEokIWKyOyOXJH7cBRBjxEQXbi$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20240308/d077ae34/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: minc.c
Type: application/octet-stream
Size: 452 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20240308/d077ae34/attachment-0001.obj>