[MOAB-dev] periodic boundaries
Vijay S. Mahadevan
vijay.m at gmail.com
Tue Apr 12 12:04:02 CDT 2016
Alexander, first I apologize about the long delay. This was in my
draft and slipped my mind. If you have found a solution in the mean
time, do let us know your approach. There is another question in the
list that may be relevant to this discussion.
>> Are you proposing to use a structured grid with triangles and tets here ?
>
> Would be an option to reproduce benchmarks on regular meshes with triangles and tets. Fully unstructured is better though, since I could also check that the meshing (numerics following from that) + MPI don’t hurt convergence.
Ok, in that case, I don't see a value add in spending time on
structured triangles/tets.
> I guess I am almost there. Have the entity handles of the periodic faces and the entity handles of the elements on both sides. Unfortunately I am stuck at convincing MOAB that the faces now have two neighbors. Here’s what I am trying:
Like I said before, the adjacency routines in MOAB are not equipped to
handle the query at the periodic boundary. Your tag based method will
have to use your custom tag value to relate entities on either side of
the periodic boundary. I would think the adjacency based method should
work out of the box and unsure why it doesn't. We may need to add few
other things in parallel (especially in computation of the skin), but
the serial runs should not fail.
Iulian, do we have a consistency check to verify if the adjacency
actually makes sense ? For example, if a face had 3 adjacent elements
for whatever reason, it may be a valid interpretation in AMR.
Similarly, for a boundary surface with only one adjacent element, if
we add another to represent periodicity, that should work for Alex's
case, no ?
Vijay
On Thu, Mar 3, 2016 at 12:28 AM, Alexander Nikolas Breuer
<anbreuer at ucsd.edu> wrote:
> Vijay, thanks for your input!
>
>> Are you proposing to use a structured grid with triangles and tets here ?
>
> Would be an option to reproduce benchmarks on regular meshes with triangles and tets. Fully unstructured is better though, since I could also check that the meshing (numerics following from that) + MPI don’t hurt convergence.
>
>> I'll have to discuss with the team to see if we can support this
>> natively in parallel.
>
> Ok, great!
>
>> Go ahead with the Tag-based route and let me
>> know if that suffices for your use-cases
>
> I guess I am almost there. Have the entity handles of the periodic faces and the entity handles of the elements on both sides. Unfortunately I am stuck at convincing MOAB that the faces now have two neighbors. Here’s what I am trying:
>
>> LOG(INFO) << "before";
>> LOG(INFO) << "the two faces: " << l_faHandle << " " << l_faHandleP << " the two triangles: " << l_element[0] << " " << l_elementP[0];
>>
>> moab::Range l_before;
>> l_error = m_core.get_adjacencies( &l_faHandle,
>> 1,
>> m_dim,
>> true,
>> l_before,
>> moab::Interface::UNION );
>> checkError(l_error);
>>
>> moab::Range l_beforeP;
>> l_error = m_core.get_adjacencies( &l_faHandleP,
>> 1,
>> m_dim,
>> true,
>> l_beforeP,
>> moab::Interface::UNION );
>> checkError(l_error);
>>
>> LOG(INFO) << l_before.size() << " " << l_before[0];
>> LOG(INFO) << l_beforeP.size() << " " << l_beforeP[0];
>>
>> l_error = m_core.add_adjacencies( l_faHandle, l_elementP, false );
>> checkError( l_error );
>>
>> l_error = m_core.add_adjacencies( l_faHandleP, l_element, false );
>> checkError( l_error );
>>
>> moab::Range l_after;
>> l_error = m_core.get_adjacencies( &l_faHandle,
>> 1,
>> m_dim,
>> true,
>> l_after,
>> moab::Interface::UNION );
>> checkError(l_error);
>>
>> moab::Range l_afterP;
>> l_error = m_core.get_adjacencies( &l_faHandleP,
>> 1,
>> m_dim,
>> true,
>> l_afterP,
>> moab::Interface::UNION );
>> checkError(l_error);
>>
>> LOG(INFO) << "after";
>> LOG(INFO) << l_after.size() << " " << l_after[0];
>> LOG(INFO) << l_afterP.size() << " " << l_afterP[0];
>>
>> assert(false);
>
> and that’s the resulting output:
>
>> 2016-03-02 22:10:03,226 INFO [default] initialized MOAB:
>> 2016-03-02 22:10:03,226 INFO [default] MOAB API version 1.01
>> 2016-03-02 22:10:03,226 INFO [default] MOAB 4.9.0
>
> [...]
>> 2016-03-02 22:10:03,228 INFO [default] before
>> 2016-03-02 22:10:03,228 INFO [default] the two faces: 1152921504606846987 1152921504606847016 the two triangles: 2305843009213693998 2305843009213694003
>> 2016-03-02 22:10:03,228 INFO [default] 1 2305843009213693998
>> 2016-03-02 22:10:03,228 INFO [default] 1 2305843009213694003
>> 2016-03-02 22:10:03,228 INFO [default] after
>> 2016-03-02 22:10:03,228 INFO [default] 1 2305843009213693998
>> 2016-03-02 22:10:03,228 INFO [default] 1 2305843009213694003
>
> Am I missing something obvious here? Should I build my own lookup?
>
> Alex
>
> --
> Alexander Breuer
> University of California, San Diego
> San Diego Supercomputer Center
> 9500 Gilman Drive MC 0505
> La Jolla, California 92093-0505, USA
> http://hpgeoc.sdsc.edu
> +1 (858) 822-5692
>
>
>
>> On Mar 2, 2016, at 4:25 PM, Vijay S. Mahadevan <vijay.m at gmail.com> wrote:
>>
>>> For the setup: I am finally aiming at Tets, but triangles are good for testing.
>>
>> Understood.
>>
>>> I also found some documentation on structured meshes using hexes in MOAB (incl. periodic boundaries), could this be extended to triangles and tets (keeping the structure alive)?
>>
>> Yes that's for a pure structured mesh (ScdInterface) which doesn't
>> store any connectivity or adjacency information explicitly. You can
>> use it for stencil computations and apply periodic B.C but it is not
>> straightforward to extend.
>>
>> Are you proposing to use a structured grid with triangles and tets here ?
>>
>>> I am using periodic boundaries mainly for convergence benchmarks: Serial is most important, I wouldn’t say no to a MPI-version though.
>>
>> As I said, the unstructured mesh doesn't care what you impose as B.C.
>> You can define sidesets as you need and create sets internally in MOAB
>> to store the faces on the periodic boundary. Once you have that, you
>> can internally have a map for adjacency or face-neighbors.
>>
>> Supporting periodicity also means that we need to do a validation when
>> we load a mesh. If the topology doesn't match, the whole setup will
>> fail. Of course the user needs to guarantee this if you want sensible
>> behavior.
>>
>>> I need the face-neighbors of my elements: Going through surfaces is therefore the most natural choice. Could also go through vertices and find elements appearing twice (triangles) or three times (tets). Whatever is simple.
>>
>> I would go through surfaces. For FV/DG, that's natural.
>>
>>> Tags are a good idea for a workaround: I’ll tell gmsh to forward the periodic boundaries as a tag and then match via the tangential dimensions for a start.
>>
>> Yes. This is the simplest way to do it.
>>
>> I'll have to discuss with the team to see if we can support this
>> natively in parallel. Go ahead with the Tag-based route and let me
>> know if that suffices for your use-cases. I'll keep you updated as we
>> continue our discussions.
>>
>> VIjay
>>
>> On Wed, Mar 2, 2016 at 1:37 PM, Alexander Nikolas Breuer
>> <anbreuer at ucsd.edu> wrote:
>>>> Alex, I don't think we have the necessary semantics to express
>>>> periodic boundaries natively on an unstructured mesh in MOAB. Partly
>>>> because, no one asked for it.
>>>
>>> Ok, this explains why I couldn’t find any docs on periodic boundaries ;)
>>>
>>> For the setup: I am finally aiming at Tets, but triangles are good for testing. It is FV/DG -> Information is shared between face-neighbors (tets) or edge-neighbors (triangles), no dependencies on the vertices. The periodic boundaries are for testing and convergence benchmarks.
>>>
>>> I also found some documentation on structured meshes using hexes in MOAB (incl. periodic boundaries), could this be extended to triangles and tets (keeping the structure alive)? One would “just” have to split the hexes. This way I could use structured meshes with triangles and tets for MPI, however handle them unstructured in my simulations internally. For the serial runs I would go for workaround to get true unstructured meshes in.
>>>
>>>> 1) Serial or parallel runs ?
>>>> a) Serial is relatively much simpler and you can get away with MOAB
>>>> implementation as it stands.
>>>> b) Parallel complicates this since the elements on either side of a
>>>> periodic boundary will be on different partitions and we need to
>>>> modify shared entity resolution to let it know which process to send
>>>> messages.
>>>
>>> I am using periodic boundaries mainly for convergence benchmarks: Serial is most important, I wouldn’t say no to a MPI-version though.
>>>
>>>> 2) Boundary description: nodes vs surfaces (edges)
>>>> a) In serial, it could be as simple as matching nodes with the same
>>>> tag value or say Y-coordinate during your setup phase and using that
>>>> information later during computation
>>>> b) In parallel, again, you need to rely on resolve_shared_ents to work
>>>> correctly so that things work out of the box.
>>>
>>> I need the face-neighbors of my elements: Going through surfaces is therefore the most natural choice. Could also go through vertices and find elements appearing twice (triangles) or three times (tets). Whatever is simple.
>>> Tags are a good idea for a workaround: I’ll tell gmsh to forward the periodic boundaries as a tag and then match via the tangential dimensions for a start.
>>>
>>>> 3) Do you need adjacency queries ?
>>>
>>> During runtime of my simulation? No, after the setup through MOAB all the information goes through an interface into internal structures (pointers, offsets, etc.).
>>>
>>>> 4) In parallel, do you expect ghost layers on the periodic boundary ?
>>>
>>> Yes, this is the smarter choice. Otherwise I would have to convince the partitioner not to cut along the boundaries.
>>>
>>> ***
>>>
>>> Something unrelated: Are there thread safe parts in MOAB? E.g. could I use OMP to iterate over my entities and then ask the core for information on those (read only)?
>>>
>>> Alex
>>>
>>> --
>>> Alexander Breuer
>>> University of California, San Diego
>>> San Diego Supercomputer Center
>>> 9500 Gilman Drive MC 0505
>>> La Jolla, California 92093-0505, USA
>>> http://hpgeoc.sdsc.edu
>>> +1 (858) 822-5692
>>>
>>>
>>>
>>>> On Mar 2, 2016, at 9:55 AM, Vijay S. Mahadevan <vijay.m at gmail.com> wrote:
>>>>
>>>> Alex, I don't think we have the necessary semantics to express
>>>> periodic boundaries natively on an unstructured mesh in MOAB. Partly
>>>> because, no one asked for it.
>>>>
>>>> So couple of questions for you.
>>>>
>>>> 1) Serial or parallel runs ?
>>>> a) Serial is relatively much simpler and you can get away with MOAB
>>>> implementation as it stands.
>>>> b) Parallel complicates this since the elements on either side of a
>>>> periodic boundary will be on different partitions and we need to
>>>> modify shared entity resolution to let it know which process to send
>>>> messages.
>>>>
>>>> 2) Boundary description: nodes vs surfaces (edges)
>>>> a) In serial, it could be as simple as matching nodes with the same
>>>> tag value or say Y-coordinate during your setup phase and using that
>>>> information later during computation
>>>> b) In parallel, again, you need to rely on resolve_shared_ents to work
>>>> correctly so that things work out of the box.
>>>>
>>>> 3) Do you need adjacency queries ?
>>>>
>>>> 4) In parallel, do you expect ghost layers on the periodic boundary ?
>>>>
>>>> Depending on what your use case is, this could be done in a variety of
>>>> ways. Let us know.
>>>>
>>>> Vijay
>>>>
>>>> On Wed, Mar 2, 2016 at 11:29 AM, Alexander Nikolas Breuer
>>>> <anbreuer at ucsd.edu> wrote:
>>>>> Writing about nodes:
>>>>>
>>>>> I am searching the neighbors by going from triangles over edges to
>>>>> neighboring triangles again. Let me know if it's worth trying to go through
>>>>> the vertices for the boundary.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Alex
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Mar 2, 2016, at 9:20 AM, Alexander Nikolas Breuer <anbreuer at ucsd.edu>
>>>>> wrote:
>>>>>
>>>>> Uh, ok. Here's a link:
>>>>> https://www.dropbox.com/s/f41mmkl47zxrm0a/gmsh.msh?dl=0
>>>>>
>>>>> The bnd condition is described in terms of nodes:
>>>>>
>>>>> $Periodic
>>>>> 3
>>>>> 0 2 1
>>>>> 1
>>>>> 2 1
>>>>> 0 3 4
>>>>> 1
>>>>> 3 4
>>>>> 1 2 4
>>>>> 9
>>>>> 14 40
>>>>> 15 39
>>>>> 16 38
>>>>> 17 37
>>>>> 18 36
>>>>> 19 35
>>>>> 20 34
>>>>> 21 33
>>>>> 22 32
>>>>> $EndPeriodic
>>>>>
>>>>> Alex
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Mar 2, 2016, at 6:19 AM, Grindeanu, Iulian R. <iulian at mcs.anl.gov> wrote:
>>>>>
>>>>> attachment was scrubbed, can you send it again ? maybe as a zip file?
>>>>> somehow, it does not like msh suffix
>>>>> Thanks,
>>>>> Iulian
>>>>> ________________________________
>>>>> From: moab-dev-bounces at mcs.anl.gov [moab-dev-bounces at mcs.anl.gov] on behalf
>>>>> of Alexander Nikolas Breuer [anbreuer at ucsd.edu]
>>>>> Sent: Tuesday, March 01, 2016 11:20 PM
>>>>> To: moab-dev at mcs.anl.gov
>>>>> Subject: [MOAB-dev] periodic boundaries
>>>>>
>>>>> Dear MOAB-team,
>>>>>
>>>>> I am trying to read a triangular mesh with periodic boundaries created by
>>>>> gmsh into MOAB (attached). However I don’t see this when going through
>>>>> MOAB’s adjacencies. Is there a way to make the mesh reader aware of periodic
>>>>> boundaries?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Alex
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Alexander Breuer
>>>>> University of California, San Diego
>>>>> San Diego Supercomputer Center
>>>>> 9500 Gilman Drive MC 0505
>>>>> La Jolla, California 92093-0505, USA
>>>>> http://hpgeoc.sdsc.edu
>>>>> +1 (858) 822-5692
>>>>>
>>>>>
>>>>>
>>>
>
More information about the moab-dev
mailing list