[MOAB-dev] Performance issues for finite-volume computations
lorenzo alessio botti
lorenzoalessiobotti at gmail.com
Wed Jun 19 12:51:47 CDT 2019
You can also store adjacencies as tags using the specific type for moab
entities. That's what I usually do to save computation type at the expense
of storing integers.
Bests
Lorenxo
On Wed, Jun 19, 2019, 17:32 Vijay S. Mahadevan via moab-dev <
moab-dev at mcs.anl.gov> wrote:
> Dear Jamond,
>
> Good to hear that you are considering MOAB as a mesh database for your
> application. Please see my answers inline. Do let me know if something
> is unclear.
>
> > So we consider replacing our legacy data/structure by moab. We already
> did this replacement in a dedicated branch of our software, and it works
> (it succeed to pass our test base). But the cpu costs of adjacencies
> queries increased hugely... For example, for a finite-volume fluid
> computation (the cells "often" query their adjacent faces, and the faces
> "often" query their adjacent cells), with our legacy structure, the
> adjacency queries cost about nothing, and with moab cost about 50% of the
> total computation!
>
> Can you provide us a little more detail on the type of meshes you use
> in the applications ? I assume 3-D, with mixed hex/tets ? Also, do you
> create most meshes in memory or are the initial meshes loaded from a
> file.
>
> > Maybe we just misuse moab? In the moab documentation, it is written that
> "Non-vertex entity to entity adjacencies are never stored, unless
> explicitly requested by the application.". Because memory consumption is
> not our primary concern, we tried that, assuming that the request for
> storing adjacencies is done with the "add_adjacencies" function (in the end
> of this message a very simple sample code representative of what we do with
> moab is provided). But the cost of the adjacency queries is still very
> high...
>
> MOAB takes a lazy approach to computing the cell adjacencies. What
> that means is that if the adjacency information for face to cell
> relation is unavailable on first query, it computes it and caches it
> in memory so that the next call to get the adjacency is much faster.
> Motivation is to get a balance between memory requirement and cost for
> adjacency query. Typically, if you create a mesh in memory, the user
> also sets the adjacencies as well so that it can be queried later. So
> the workflow would be to call add_adjacencies BEFORE get_adjacencies
> here. However, if you loaded a mesh from file, you can query the mesh
> with get_adjacencies in addition to "create_if_missing" set to true,
> in case the mesh file itself does not store explicit adjacency
> information.
>
> > - for each edge represented by its two vertices, loop over some existing
> edges represented by their EntityHandle to find which one matches the
> current edge (a large portion of the computation time is spent in
> "moab::AEntityFactory::entities_equivalent(unsigned long, unsigned long
> const*, int, moab::EntityType)" )
>
> MOAB adjacency datastructure is optimized for vertex to element type
> queries since these are explicitly stored and available by default. And
> this is
> why the implementation to get face->element adjacency information goes
> through that intermediate step to decipher the relation. The AHF
> datastructure
> does not have this indirection and may provide much faster queries
> depending
> on the use-case.
>
> > Does moab actually store the adjacencies with "add_adjacencies"? For
> example, by following in a debugger what moab does on the sample program
> provided when querying the edges adjacent to a quadrangular cell, it seems
> that whereas the adjacencies being stored, moab:
>
> Yes. If the user explicitly calls add_adjacencies, this information is
> stored internally so that it can be queried later.
>
> > We see in the code that there is a "create_adjacency_option" that is not
> yet supported. Could this option help once implemented?
>
> Can you point me to this piece of the implementation ?
>
> > We also checked the AHF feature. At this time, this feature cannot apply
> in our applications because it is not compatible with modified and
> mixed-type meshes. Is it planned for AHF to become compatible with modified
> and mixed-type meshes?
>
> I am also cc'ing Navamita Ray who implemented AHF [1] in MOAB.
> Navamita, can you talk more about the AHF support with the adjacency
> cost comparison with native MOAB implementation ? Also, if you can
> provide some thoughts about extending AHF to support mixed meshes and
> polytopes, that would be good as well.
>
> Vijay
>
> [1] https://imr.sandia.gov/papers/imr22/IMR22_25_Dyedov.pdf
>
>
> On Wed, Jun 19, 2019 at 11:08 AM JAMOND Olivier via moab-dev
> <moab-dev at mcs.anl.gov> wrote:
> >
> > Hello,
> >
> >
> >
> > I am working at the french CEA "French Alternative Energies and Atomic
> Energy Commission" on the development of a new
> finite-elements/finite-volume software for mechanical applications.
> >
> >
> >
> > Until now, we had an internal data base for storing and querying
> adjencencies/connectivities between mesh entities. In this legacy data
> structure, every adjacencies/connectivities are computed and stored at the
> beginning of the computation, and then queries during the actual
> computation steps just "read" in this data structure. Our problem is that
> this legacy data structure is way too "static": for example, it is very
> difficult to add/remove cells (for AMR, or load balancing for example)
> during the computation steps.
> >
> >
> >
> > So we consider replacing our legacy data/structure by moab. We already
> did this replacement in a dedicated branch of our software, and it works
> (it succeed to pass our test base). But the cpu costs of adjacencies
> queries increased hugely... For example, for a finite-volume fluid
> computation (the cells "often" query their adjacent faces, and the faces
> "often" query their adjacent cells), with our legacy structure, the
> adjacency queries cost about nothing, and with moab cost about 50% of the
> total computation!
> >
> >
> >
> > Maybe we just misuse moab? In the moab documentation, it is written that
> "Non-vertex entity to entity adjacencies are never stored, unless
> explicitly requested by the application.". Because memory consumption is
> not our primary concern, we tried that, assuming that the request for
> storing adjacencies is done with the "add_adjacencies" function (in the end
> of this message a very simple sample code representative of what we do with
> moab is provided). But the cost of the adjacency queries is still very
> high...
> >
> >
> >
> > Does moab actually store the adjacencies with "add_adjacencies"? For
> example, by following in a debugger what moab does on the sample program
> provided when querying the edges adjacent to a quadrangular cell, it seems
> that whereas the adjacencies being stored, moab:
> >
> > - get the nodes of each edges of the quadrangle
> >
> > - for each edge represented by its two vertices, loop over some existing
> edges represented by their EntityHandle to find which one matches the
> current edge (a large portion of the computation time is spent in
> "moab::AEntityFactory::entities_equivalent(unsigned long, unsigned long
> const*, int, moab::EntityType)" )
> >
> >
> >
> > We see in the code that there is a "create_adjacency_option" that is not
> yet supported. Could this option help once implemented? We also checked the
> AHF feature. At this time, this feature cannot apply in our applications
> because it is not compatible with modified and mixed-type meshes. Is it
> planned for AHF to become compatible with modified and mixed-type meshes?
> >
> >
> >
> > Is there something that we miss, or a more efficient way to use moab?
> >
> >
> >
> > Many thanks,
> >
> > _____________________________
> >
> > Olivier JAMOND
> >
> > Laboratoire d'études de dynamique
> >
> > CEA Saclay, DEN/DANS/DM2S/SEMT/DYN
> >
> > F-91191 GIF SUR YVETTE, FRANCE
> >
> > Tél. : 01.69.08.44.90
> >
> >
> >
> >
> >
> > * ------------- Sample code --------------- *
> >
> >
> >
> > #include <iostream>
> > #include <sstream>
> > #include <vector>
> >
> > #include "moab/Core.hpp"
> >
> > //
> > // 3 ----- 5 ----- 5
> > // | | |
> > // | | |
> > // | | |
> > // 0 ----- 1 ----- 4
> > //
> >
> > int main()
> > {
> >
> > std::vector<moab::EntityHandle> adj;
> >
> > // > init moab
> > moab::Core mb;
> > mb.set_dimension(2);
> > // <
> >
> > // > create the vertices (foo coordinates)
> > double coords[3]={0.,0.,0.};
> >
> > std::vector<moab::EntityHandle> nodes;
> > nodes.resize(6);
> > mb.create_vertex(coords, nodes[0]);
> > mb.create_vertex(coords, nodes[1]);
> > mb.create_vertex(coords, nodes[2]);
> > mb.create_vertex(coords, nodes[3]);
> > mb.create_vertex(coords, nodes[4]);
> > mb.create_vertex(coords, nodes[5]);
> > // <
> >
> > // > create the 2 quadrangles
> > moab::EntityHandle q0;
> > std::vector<moab::EntityHandle> conn0;
> > conn0.resize(4);
> > conn0[0]=nodes[0];
> > conn0[1]=nodes[1];
> > conn0[2]=nodes[2];
> > conn0[3]=nodes[3];
> > mb.create_element(moab::MBQUAD, conn0.data(), 4, q0);
> >
> > moab::EntityHandle q1;
> > std::vector<moab::EntityHandle> conn1;
> > conn1.resize(4);
> > conn1[0]=nodes[1];
> > conn1[1]=nodes[4];
> > conn1[2]=nodes[5];
> > conn1[3]=nodes[2];
> > mb.create_element(moab::MBQUAD, conn1.data(), 4, q1);
> > // <
> >
> > // > compute and store cell<->face adjacencies
> > adj.clear();
> > mb.get_adjacencies (&q0, 1, 1, true, adj);
> > mb.add_adjacencies(q0, adj.data(), adj.size(), true);
> > adj.clear();
> > mb.get_adjacencies (&q1, 1, 1, true, adj);
> > mb.add_adjacencies(q1, adj.data(), adj.size(), true);
> > // <
> >
> > // > example of queries
> > // > get the faces of the first cell
> > adj.clear();
> > mb.get_adjacencies (&q1, 1, 1, false, adj);
> > std::cout << "## face of the first cell -------------------#\n";
> > mb.list_entities(adj.data(), adj.size());
> > // <
> >
> > // > get the face shared by the two cells
> > adj.clear();
> > moab::EntityHandle cells[2]={q0,q1};
> > mb.get_adjacencies(cells, 2, 1, false, adj);
> > std::cout << "## face shared by the two cells -------------#\n";
> > mb.list_entities(adj.data(), adj.size());
> > // <
> >
> > // > get the two cells from the middle face
> > moab::EntityHandle face=adj[0];
> > adj.clear();
> > mb.get_adjacencies(&face, 1, 2, false, adj);
> > std::cout << "## cells adjacent to the middle face --------#\n";
> > mb.list_entities(adj.data(), adj.size());
> > // <
> > // <
> >
> > return 0;
> > }
> >
> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/moab-dev/attachments/20190619/f7f55410/attachment-0001.html>
More information about the moab-dev
mailing list