[petsc-users] Question about DMPLEX/P4EST with different Sections

Berend van Wachem berend.vanwachem at ovgu.de
Mon Mar 23 08:08:41 CDT 2020


Dear Matt,

It seems I judged to early. The solution works when the mesh is refined, 
and for multiple times refinement, but does not work for re-coarsening.

I have created a working example where I show this, please find it attached.

Starting from line 190, the forest is refined. A call to the routine 
"AdaptVector" then does a similar refinement  for another vector created 
on a cloned DM with a different section .

Starting from line 208, the forest is coarsened. The code crashes in 
transferring the vector from the finer dmForest to the coarser one.

If I write "DM_ADAPT_REFINE" instead of "DM_ADAPT_COARSEN" on line 211, 
everything works fine. So there seems to be an issue when coarsening.

The error I get is:

[0]PETSC ERROR: 
------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, 
probably memory access out of range
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see 
https://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
X to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: ---------------------  Stack Frames 
------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR:       INSTEAD the line number of the start of the function
[0]PETSC ERROR:       is given.
[0]PETSC ERROR: [0] DMPlexTransferVecTree_Inject line 4330 
/usr/local/petsc-3.12.4/src/dm/impls/plex/plextree.c
[0]PETSC ERROR: [0] DMPlexTransferVecTree line 4505 
/usr/local/petsc-3.12.4/src/dm/impls/plex/plextree.c
[0]PETSC ERROR: [0] DMForestTransferVec_p8est line 4829 
/usr/local/petsc-3.12.4/src/dm/impls/forest/p4est/pforest.c
[0]PETSC ERROR: [0] DMForestTransferVec line 997 
/usr/local/petsc-3.12.4/src/dm/impls/forest/forest.c
[0]PETSC ERROR: --------------------- Error Message 
--------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html 
for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.12.4, Feb, 04, 2020
[0]PETSC ERROR: ./workingexample on a linux-gcc-dev named 
multiflow.multiflow.org by berend Mon Mar 23 14:05:17 2020
[0]PETSC ERROR: Configure options --download-metis=yes 
--download-parmetis=yes --download-hdf5=yes --download-p4est=yes 
--with-zlib-include=/usr/include --with-zlib-lib=/usr/lib64/libz.a 
--download-triangle=yes --download-ctetgen=yes
[0]PETSC ERROR: #1 User provided function() line 0 in  unknown file
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0


Do you have any ideas?

Thanks, I really appreciate it.

Best wishes, Berend.




On 2020-03-19 23:39, Matthew Knepley wrote:
> Okay this runs for me.
> 
>    Thanks,
> 
>      Matt
> 
> On Thu, Mar 19, 2020 at 6:07 PM Matthew Knepley <knepley at gmail.com 
> <mailto:knepley at gmail.com>> wrote:
> 
>     On Fri, Mar 13, 2020 at 9:45 AM Berend van Wachem
>     <berend.vanwachem at ovgu.de <mailto:berend.vanwachem at ovgu.de>> wrote:
> 
>         Dear Matt,
> 
>         Thanks for your response. My understanding of the DM and DMClone
>         is the
>         same - and I have tested this with a DMPLEX DM without problems.
> 
>         However, for some reason, I cannot change/set the section of a
>         P4EST dm.
>         In the attached example code, I get an error in line 140, where
>         I try to
>         create a new section from the cloned P4EST DM. Is it not
>         possible to
>         create/set a section on a P4EST DM? Or maybe I am doing
>         something else
>         wrong? Do you suggest a workaround?
> 
> 
>     Hi Berend,
> 
>     Sorry I am behind. The problem on line 140 is that you call a DMPlex
>     function (DMPlexCreateSection)
>     with a DMForest object. That is illegal. You can, however, call that
>     function using the Plex you get from
>     a Forest using DMConvert(DMForestClone, DMPLEX, &plexClone). I will
>     get your code running as soon
>     as I can, but after you create the Section, attaching it should be fine.
> 
>        Thanks,
> 
>           Matt
> 
>         Many thanks, Berend.
> 
> 
>         On 2020-03-13 00:19, Matthew Knepley wrote:
>          > On Thu, Mar 12, 2020 at 7:40 AM Berend van Wachem
>          > <berend.vanwachem at ovgu.de <mailto:berend.vanwachem at ovgu.de>
>         <mailto:berend.vanwachem at ovgu.de
>         <mailto:berend.vanwachem at ovgu.de>>> wrote:
>          >
>          >     Dear All,
>          >
>          >     I have started to use DMPLEX with P4EST for a
>         computational fluid
>          >     dynamics application. I am solving a coupled system of 4
>         discretised
>          >     equations (for 3 velocity components and one pressure) on
>         a mesh.
>          >     However, next to these 4 variables, I also have a few
>         single field
>          >     variables (such as density and viscosity) defined over
>         the mesh,
>          >     which I
>          >     don't solve for (they should not be part of the matrix
>         with unknowns).
>          >     Most of these variables are at the cell centers, but in a
>         few cases, it
>          >     want to define them at cell faces.
>          >
>          >     With just DMPLEX, I solve this by:
>          >
>          >     DMPlexCreateMesh, so I get an initial DM
>          >     DMPlexCreateSection, indicating the need for 4 variables
>          >     DMSetLocalSection
>          >     DMCreateGlobalVector (and Matrix), so I get an Unknown
>         vector, a RHS
>          >     vector, and a matrix for the 4 variables.
>          >
>          >     To get a vector for a single variable at the cell center
>         or the cell
>          >     face, I clone the original DM, I define a new Section on
>         it, and then
>          >     create the vector from that which I need (e.g. for density,
>          >     viscosity or
>          >     a velocity at the cell face).
>          >
>          >     Then I loop over the mesh, and with MatSetValuesLocal, I
>         set the
>          >     coefficients. After that, I solve the system for multiple
>         timesteps
>          >     (sequential solves) and get the solution vector with the
>         4 variables
>          >     after each solve.
>          >
>          >     So-far, this works fine with DMPLEX. However, now I want
>         to use P4EST,
>          >     and I have difficulty defining a variable vector other
>         than the
>          >     original 4.
>          >
>          >     I have changed the code structure:
>          >
>          >     DMPlexCreateMesh, so I get an initial DM
>          >     DMPlexCreateSection, indicating the need for 4 variables
>          >     DMSetLocalSection
>          >     DMForestSetBaseDM(DM, DMForest) to create a DMForest
>          >     DMCreateGlobalVector (and Matrix), so I get a Unknown
>         vector, a RHS
>          >     vector, and a matrix for the 4 variables
>          >
>          >     then I perform multiple time-steps,
>          >         DMForestTemplate(DMForest ->  DMForestPost)
>          >         Adapt DMForestPost
>          >         DMCreateGlovalVector(DMForestPost, RefinedUnknownVector)
>          >         DMForestTransferVec(UnknownVector , RefinedUnknownVector)
>          >         DMForestPost -> DMForest
>          >     and then DMConvert(DMForest,DMPLEX,DM)
>          >     and I can solve the system as usual. That also seems to work.
>          >
>          >     But my conceptual question: how can I convert the other
>         variable
>          >     vectors
>          >     (obtained with a different section on the same DM) such
>         as density and
>          >     viscosity and faceVelocity within this framework?
>          >
>          >
>          > Here is my current thinking about DMs. A DM is a function space
>          > overlaying a topology. Much to my dismay, we
>          > do not have a topology object, so it hides inside DM.
>         DMClone() creates
>          > a shallow copy of the topology. We use
>          > this to have any number of data layouts through PetscSection,
>         laying
>          > over the same underlying topology.
>          >
>          > So for each layout you have, make a separate clone. Then
>         things like
>          > TransferVec() will respond to the layout in
>          > that clone. Certainly it works this way in Plex. I admit to
>         not having
>          > tried this for TransferVec(), but let me know if
>          > you have any problems.
>          >
>          > BTW, I usually use a dm for the solution, which I give to the
>         solver,
>          > say SNESSetDM(snes, dm), and then clone
>          > it as dmAux which has the layout for all the auxiliary fields
>         that are
>          > not involved in the solve. The Plex examples
>          > all use this form.
>          >
>          >    Thanks,
>          >
>          >       Matt
>          >
>          >     The DMForest has the same Section as the original DM and
>         will thus have
>          >     the space for exactly 4 variables per cell. I tried
>         pushing another
>          >     section on the DMForest and DMForestPost, but that does
>         not seem to
>          >     work. Please find attached a working example with code to
>         do this,
>          >     but I
>          >     get the error:
>          >
>          >     PETSC ERROR: PetscSectionGetChart() line 513 in
>          >   
>           /usr/local/petsc-3.12.4/src/vec/is/section/interface/section.c
>         Wrong
>          >     type of object: Parameter # 1
>          >
>          >     So, I is there a way to DMForestTransferVec my other
>         vectors from one
>          >     DMForest to DMForestPost. How can I do this?
>          >
>          >     Many thanks for your help!
>          >
>          >     Best wishes, Berend.
>          >
>          >
>          >
>          > --
>          > What most experimenters take for granted before they begin their
>          > experiments is infinitely more interesting than any results
>         to which
>          > their experiments lead.
>          > -- Norbert Wiener
>          >
>          > https://www.cse.buffalo.edu/~knepley/
>         <http://www.cse.buffalo.edu/~knepley/>
> 
> 
> 
>     -- 
>     What most experimenters take for granted before they begin their
>     experiments is infinitely more interesting than any results to which
>     their experiments lead.
>     -- Norbert Wiener
> 
>     https://www.cse.buffalo.edu/~knepley/
>     <http://www.cse.buffalo.edu/~knepley/>
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their 
> experiments is infinitely more interesting than any results to which 
> their experiments lead.
> -- Norbert Wiener
> 
> https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dmplexp4est-2.c
Type: text/x-csrc
Size: 9128 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200323/886f9542/attachment-0001.bin>


More information about the petsc-users mailing list