[petsc-users] specifying vertex coordinates using DMPlexCreateFromCellListParallel

Thu Jun 29 08:59:48 CDT 2017

Matthew,

Not sure if I understood completely, I have the ordering according to the CGNS standard for all cells in the mesh, or I can change it if needed, but I don’t understand what you mean by reporting all faces as ordered vertices? Do you mean I just define the face connectivity?Could you explain a bit more?

Thank you
-Hassan

From: Matthew Knepley [mailto:knepley at gmail.com]
Sent: Tuesday, June 27, 2017 1:54 PM
To: Hassan Raiesi <Hassan.Raiesi at aero.bombardier.com>
Cc: petsc-users at mcs.anl.gov
Subject: Re: [petsc-users] specifying vertex coordinates using DMPlexCreateFromCellListParallel

On Tue, Jun 27, 2017 at 11:30 AM, Hassan Raiesi <Hassan.Raiesi at aero.bombardier.com<mailto:Hassan.Raiesi at aero.bombardier.com>> wrote:
I meant the interpolation,
DMPlex supports those element shapes, however, there are two problems, one, “DMPlexBuildFromCellList_Parallel” does not take mixed elements (numCorners must be constant for all elements in current implementation), that was easy to fix,
I already extended DMPlexBuildFromCellList_Parallel_Private to take elements with different shapes,  then I realized the interpolation does not work when the mesh has elements other than tets and hex.

Okay, here is what is needed. You need to

  a) prescribe an order for the vertices in a prism/pyramid (all input cells must have this order)

  b) report all faces as sets of ordered vertices
    - You have to order them matching the Plex vertex order for the lower dimensional shape
    - They should be oriented as to have outward facing normal

Regarding the interpolation, I also noticed that the memory requirement is huge if I load the whole mesh on one core and interpolate (I cannot interpolate a mesh with 9M tets on a machine with 128GB of memory, it ran out of memory, I’ll try to run with petsc memory logs and send).

I am not sure what the upper limit is, but in a 3D mesh you could have many times the number of cells in faces and edges. Note that
you would need --with-64-bit-indices to go beyond 4GB.

is there anyways to interpolate the mesh after  DMPlexdistribute? The code crashes if I move DMPlexInterpolate to after calling DMPlexdistribute., I guess what is needed is a DMPlexInterpolate on already distributed meshes.

This should work. Maybe try a small example? If that crashes, just send it.

  Thanks,

    Matt

Thank you
-Hassan

From: Matthew Knepley [mailto:knepley at gmail.com<mailto:knepley at gmail.com>]
Sent: Tuesday, June 27, 2017 11:53 AM
To: Hassan Raiesi <Hassan.Raiesi at aero.bombardier.com<mailto:Hassan.Raiesi at aero.bombardier.com>>
Cc: petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] specifying vertex coordinates using DMPlexCreateFromCellListParallel

On Tue, Jun 27, 2017 at 10:08 AM, Hassan Raiesi <Hassan.Raiesi at aero.bombardier.com<mailto:Hassan.Raiesi at aero.bombardier.com>> wrote:
Great, It’s clear now ☺,
One more question, any plan to support other element shapes (prism and pyramid) in 3D?,  DMPlexGetRawFaces_Internal only supports tets and hexs in 3D, can prisms and pyramids be used as degenerate hexahedrons?

It depends on what you mean "support". Right now, we can represent these shapes in Plex. However, if you
want mesh interpolation to work, then yes you need to extend GetRawFaces() to understand that shape. If
you want them read out of a file format, other than Gmsh, we would likely have to extend that as well. These
are straightforward once I understand what exactly you want to do.

  Thanks,

    Matt

Thank you
-Hassan

From: Matthew Knepley [mailto:knepley at gmail.com<mailto:knepley at gmail.com>]
Sent: Tuesday, June 27, 2017 10:17 AM
To: Hassan Raiesi <Hassan.Raiesi at aero.bombardier.com<mailto:Hassan.Raiesi at aero.bombardier.com>>
Cc: petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] specifying vertex coordinates using DMPlexCreateFromCellListParallel

On Tue, Jun 27, 2017 at 9:12 AM, Hassan Raiesi <Hassan.Raiesi at aero.bombardier.com<mailto:Hassan.Raiesi at aero.bombardier.com>> wrote:
Thanks for your reply,

Is there any example where each rank owns more than 1 element, i.e for the simple mesh here(attached png file), how should I pack and pass the coordinates of the vertices owned by rank0, rank1

Rank0:numcells = 2; num nodes=4, cells=[154 245] , nodes=[1 5 2 4]
nodes = 1245
vertex coords: in what node order ?
[coords_n1 coords_n2 coords_n4 coords_n5] or
[coords_n2 coords_n4 coords_n1 coords_n5] or …..?

rank1: numcells = 2; num nodes=2, cells=[532 635], nodes=[6 3]
vertexcoords [how to pack the nodes coords here?]
should it be  [x6y6 x3y3] or [x3y3 x6y6]? In what order?

I think there is a misunderstanding here.

There is NO connection between the cell order and the vertex order. Each process gets a contiguous
set of cells (in the global numbering) and a contiguous set of vertices (in the global numbering). These
two are NOT related. We then move the vertices to the correct processes. In this way, we can load
completely in parallel, without requiring any setup in the mesh file.

If you are worried, you can always arrange the order of vertices to "match" the order of cells.

  Thanks,

    Matt

thanks

From: Matthew Knepley [mailto:knepley at gmail.com<mailto:knepley at gmail.com>]
Sent: Sunday, June 25, 2017 1:04 PM
To: Hassan Raiesi <Hassan.Raiesi at aero.bombardier.com<mailto:Hassan.Raiesi at aero.bombardier.com>>
Cc: petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] specifying vertex coordinates using DMPlexCreateFromCellListParallel

On Tue, Apr 11, 2017 at 9:21 AM, Hassan Raiesi <Hassan.Raiesi at aero.bombardier.com<mailto:Hassan.Raiesi at aero.bombardier.com>> wrote:
Hello,

I’m trying to use DMPlexCreateFromCellListParallel to create a DM from an already partitioned mesh,
It requires an array of numVertices*spaceDim numbers, but how should one order the coordinates of the vertices?

Global order. Here is the idea. You must read the file in chunks so that each proc can read its own chunk in parallel
without talking to anyone else.

we only pass the global vertex numbers using ‘const int cells[]’ to define the cell-connectivity, so passing the vertex coordinates in local ordering wouldn’t make sense?

Yes.

If it needs to be in global ordering, should I sort the global index of the node numbers owned by each rank (as they wont be continuous).

Nope.

  Thanks,

    Matt

Thank you

Hassan Raiesi,
Bombardier Aerospace
www.bombardier.com<http://www.bombardier.com>

--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

http://www.caam.rice.edu/~mk51/

--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

http://www.caam.rice.edu/~mk51/

--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

http://www.caam.rice.edu/~mk51/

--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

http://www.caam.rice.edu/~mk51/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170629/2865e4e0/attachment-0001.html>