[petsc-users] Question about DMPLEX/P4EST with different Sections
Berend van Wachem
berend.vanwachem at ovgu.de
Mon Mar 23 08:08:41 CDT 2020
Dear Matt,
It seems I judged to early. The solution works when the mesh is refined,
and for multiple times refinement, but does not work for re-coarsening.
I have created a working example where I show this, please find it attached.
Starting from line 190, the forest is refined. A call to the routine
"AdaptVector" then does a similar refinement for another vector created
on a cloned DM with a different section .
Starting from line 208, the forest is coarsened. The code crashes in
transferring the vector from the finer dmForest to the coarser one.
If I write "DM_ADAPT_REFINE" instead of "DM_ADAPT_COARSEN" on line 211,
everything works fine. So there seems to be an issue when coarsening.
The error I get is:
[0]PETSC ERROR:
------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
probably memory access out of range
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see
https://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS
X to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: --------------------- Stack Frames
------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR: INSTEAD the line number of the start of the function
[0]PETSC ERROR: is given.
[0]PETSC ERROR: [0] DMPlexTransferVecTree_Inject line 4330
/usr/local/petsc-3.12.4/src/dm/impls/plex/plextree.c
[0]PETSC ERROR: [0] DMPlexTransferVecTree line 4505
/usr/local/petsc-3.12.4/src/dm/impls/plex/plextree.c
[0]PETSC ERROR: [0] DMForestTransferVec_p8est line 4829
/usr/local/petsc-3.12.4/src/dm/impls/forest/p4est/pforest.c
[0]PETSC ERROR: [0] DMForestTransferVec line 997
/usr/local/petsc-3.12.4/src/dm/impls/forest/forest.c
[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html
for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.12.4, Feb, 04, 2020
[0]PETSC ERROR: ./workingexample on a linux-gcc-dev named
multiflow.multiflow.org by berend Mon Mar 23 14:05:17 2020
[0]PETSC ERROR: Configure options --download-metis=yes
--download-parmetis=yes --download-hdf5=yes --download-p4est=yes
--with-zlib-include=/usr/include --with-zlib-lib=/usr/lib64/libz.a
--download-triangle=yes --download-ctetgen=yes
[0]PETSC ERROR: #1 User provided function() line 0 in unknown file
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
Do you have any ideas?
Thanks, I really appreciate it.
Best wishes, Berend.
On 2020-03-19 23:39, Matthew Knepley wrote:
> Okay this runs for me.
>
> Thanks,
>
> Matt
>
> On Thu, Mar 19, 2020 at 6:07 PM Matthew Knepley <knepley at gmail.com
> <mailto:knepley at gmail.com>> wrote:
>
> On Fri, Mar 13, 2020 at 9:45 AM Berend van Wachem
> <berend.vanwachem at ovgu.de <mailto:berend.vanwachem at ovgu.de>> wrote:
>
> Dear Matt,
>
> Thanks for your response. My understanding of the DM and DMClone
> is the
> same - and I have tested this with a DMPLEX DM without problems.
>
> However, for some reason, I cannot change/set the section of a
> P4EST dm.
> In the attached example code, I get an error in line 140, where
> I try to
> create a new section from the cloned P4EST DM. Is it not
> possible to
> create/set a section on a P4EST DM? Or maybe I am doing
> something else
> wrong? Do you suggest a workaround?
>
>
> Hi Berend,
>
> Sorry I am behind. The problem on line 140 is that you call a DMPlex
> function (DMPlexCreateSection)
> with a DMForest object. That is illegal. You can, however, call that
> function using the Plex you get from
> a Forest using DMConvert(DMForestClone, DMPLEX, &plexClone). I will
> get your code running as soon
> as I can, but after you create the Section, attaching it should be fine.
>
> Thanks,
>
> Matt
>
> Many thanks, Berend.
>
>
> On 2020-03-13 00:19, Matthew Knepley wrote:
> > On Thu, Mar 12, 2020 at 7:40 AM Berend van Wachem
> > <berend.vanwachem at ovgu.de <mailto:berend.vanwachem at ovgu.de>
> <mailto:berend.vanwachem at ovgu.de
> <mailto:berend.vanwachem at ovgu.de>>> wrote:
> >
> > Dear All,
> >
> > I have started to use DMPLEX with P4EST for a
> computational fluid
> > dynamics application. I am solving a coupled system of 4
> discretised
> > equations (for 3 velocity components and one pressure) on
> a mesh.
> > However, next to these 4 variables, I also have a few
> single field
> > variables (such as density and viscosity) defined over
> the mesh,
> > which I
> > don't solve for (they should not be part of the matrix
> with unknowns).
> > Most of these variables are at the cell centers, but in a
> few cases, it
> > want to define them at cell faces.
> >
> > With just DMPLEX, I solve this by:
> >
> > DMPlexCreateMesh, so I get an initial DM
> > DMPlexCreateSection, indicating the need for 4 variables
> > DMSetLocalSection
> > DMCreateGlobalVector (and Matrix), so I get an Unknown
> vector, a RHS
> > vector, and a matrix for the 4 variables.
> >
> > To get a vector for a single variable at the cell center
> or the cell
> > face, I clone the original DM, I define a new Section on
> it, and then
> > create the vector from that which I need (e.g. for density,
> > viscosity or
> > a velocity at the cell face).
> >
> > Then I loop over the mesh, and with MatSetValuesLocal, I
> set the
> > coefficients. After that, I solve the system for multiple
> timesteps
> > (sequential solves) and get the solution vector with the
> 4 variables
> > after each solve.
> >
> > So-far, this works fine with DMPLEX. However, now I want
> to use P4EST,
> > and I have difficulty defining a variable vector other
> than the
> > original 4.
> >
> > I have changed the code structure:
> >
> > DMPlexCreateMesh, so I get an initial DM
> > DMPlexCreateSection, indicating the need for 4 variables
> > DMSetLocalSection
> > DMForestSetBaseDM(DM, DMForest) to create a DMForest
> > DMCreateGlobalVector (and Matrix), so I get a Unknown
> vector, a RHS
> > vector, and a matrix for the 4 variables
> >
> > then I perform multiple time-steps,
> > DMForestTemplate(DMForest -> DMForestPost)
> > Adapt DMForestPost
> > DMCreateGlovalVector(DMForestPost, RefinedUnknownVector)
> > DMForestTransferVec(UnknownVector , RefinedUnknownVector)
> > DMForestPost -> DMForest
> > and then DMConvert(DMForest,DMPLEX,DM)
> > and I can solve the system as usual. That also seems to work.
> >
> > But my conceptual question: how can I convert the other
> variable
> > vectors
> > (obtained with a different section on the same DM) such
> as density and
> > viscosity and faceVelocity within this framework?
> >
> >
> > Here is my current thinking about DMs. A DM is a function space
> > overlaying a topology. Much to my dismay, we
> > do not have a topology object, so it hides inside DM.
> DMClone() creates
> > a shallow copy of the topology. We use
> > this to have any number of data layouts through PetscSection,
> laying
> > over the same underlying topology.
> >
> > So for each layout you have, make a separate clone. Then
> things like
> > TransferVec() will respond to the layout in
> > that clone. Certainly it works this way in Plex. I admit to
> not having
> > tried this for TransferVec(), but let me know if
> > you have any problems.
> >
> > BTW, I usually use a dm for the solution, which I give to the
> solver,
> > say SNESSetDM(snes, dm), and then clone
> > it as dmAux which has the layout for all the auxiliary fields
> that are
> > not involved in the solve. The Plex examples
> > all use this form.
> >
> > Thanks,
> >
> > Matt
> >
> > The DMForest has the same Section as the original DM and
> will thus have
> > the space for exactly 4 variables per cell. I tried
> pushing another
> > section on the DMForest and DMForestPost, but that does
> not seem to
> > work. Please find attached a working example with code to
> do this,
> > but I
> > get the error:
> >
> > PETSC ERROR: PetscSectionGetChart() line 513 in
> >
> /usr/local/petsc-3.12.4/src/vec/is/section/interface/section.c
> Wrong
> > type of object: Parameter # 1
> >
> > So, I is there a way to DMForestTransferVec my other
> vectors from one
> > DMForest to DMForestPost. How can I do this?
> >
> > Many thanks for your help!
> >
> > Best wishes, Berend.
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> > experiments is infinitely more interesting than any results
> to which
> > their experiments lead.
> > -- Norbert Wiener
> >
> > https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dmplexp4est-2.c
Type: text/x-csrc
Size: 9128 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200323/886f9542/attachment-0001.bin>
More information about the petsc-users
mailing list