[MOAB-dev] Polyhedra elements in parallell

Fri May 1 13:52:26 CDT 2015

Thanks for confirming James. Glad we can be of help.

The branch should get reviewed and merged over the weekend.

Best,
Vijay

On Fri, May 1, 2015 at 1:46 PM, WARNER, JAMES E. (LARC-D309)
<james.e.warner at nasa.gov> wrote:
> Vijay & Iulian,
>
> I pulled the code and the bug seems to be fixed. Thanks a lot for the
> quick response - I appreciate that. Have a great weekend!
>
> Best,
> Jim
>
>
>
> --
> James E. Warner, PhD
> Research Computer Scientist
> NASA Langley Research Center
> Durability, Damage Tolerance, & Reliability Branch
> Building 1205; Rm 212
>
>
>
>
> On 5/1/15 10:12 AM, "Vijay S. Mahadevan" <vijay.m at gmail.com> wrote:
>
>>James,
>>
>>There is a PR that Iulian created to fix this issue.
>>
>>https://bitbucket.org/fathomteam/moab/pull-request/106/ghosting-polyhedra-
>>elements/
>>
>>Can you fetch from the repo and test against the branch:
>>iulian07/ghost_polyhedra to see if the bug has been fixed ? If you can
>>confirm, we will review and merge the PR.
>>
>>Vijay
>>
>>On Tue, Apr 28, 2015 at 4:04 PM, Grindeanu, Iulian R.
>><iulian at mcs.anl.gov> wrote:
>>> Hi Jim,
>>>
>>> I am able to reproduce the error when I use ghosting
>>>
>>> mpiexec -np 2 mbconvert -O PARALLEL=READ_PART -O
>>> PARTITION=PARALLEL_PARTITION -O PARALLEL_RESOLVE_SHARED_ENTS -o
>>> PARALLEL=WRITE_PART -O PARALLEL_GHOSTS=3.0.1 twoPolyhedra.p2.h5m jj.h5
>>>
>>> It is a bug, thank you for reporting it. It is related to polyhedra
>>> elements, most likely.
>>>
>>> I am looking at it,
>>> thanks,
>>> Iulian
>>>
>>>
>>>
>>>
>>> ________________________________
>>> From: WARNER, JAMES E. (LARC-D309) [james.e.warner at nasa.gov]
>>> Sent: Tuesday, April 28, 2015 3:26 PM
>>>
>>> To: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
>>> Subject: Re: [MOAB-dev] Polyhedra elements in parallell
>>>
>>> Hi Iulian,
>>>
>>> Thanks for the quick responseŠ
>>>
>>> I am able to run that 'mbconvert' command in parallel as well without
>>>error.
>>>
>>> Where I am running into an issue is when I try to read the partitioned
>>>mesh
>>> file (twoPolyhedra.p2.h5m) into a finite element code using the
>>> DMMoabLoadFromFile(Š) function on two processors. I get the following
>>>error
>>> after calling DMMoabLoadFromFile() :
>>>
>>> "
>>> ParallelComm.cpp:2928: moab::ErrorCode
>>> moab::ParallelComm::get_local_handles(moab::EntityHandle*, int, const
>>> std::vector<long unsigned int>&): Assertion
>>>`ID_FROM_HANDLE(from_vec[i]) <
>>> (int) new_ents.size()' failed.
>>> "
>>>
>>> However, reading the unpartitioned mesh (twoPolyhedra.p1.h5m) on one
>>> processor works fine.
>>>
>>> -Jim
>>>
>>>
>>> From: <Grindeanu>, "Iulian R." <iulian at mcs.anl.gov>
>>> Date: Tuesday, April 28, 2015 2:58 PM
>>> To: "WARNER, JAMES E. (LARC-D309)" <james.e.warner at nasa.gov>,
>>> "moab-dev at mcs.anl.gov" <moab-dev at mcs.anl.gov>
>>> Subject: RE: [MOAB-dev] Polyhedra elements in parallell
>>>
>>> Hi Jim,
>>> I was able to "mbconvert" in parallel your file, with a command like
>>> mpiexec -np 2 mbconvert -O PARALLEL=READ_PART -O
>>> PARTITION=PARALLEL_PARTITION -O PARALLEL_RESOLVE_SHARED_ENTS -o
>>> PARALLEL=WRITE_PART twoPolyhedra.p2.h5m jj.h5m
>>>
>>> mbpart created correctly the partitions, (each with one element).
>>>
>>> What options do you use when you load the file in parallel?
>>>
>>> You can add also an option like -O DEBUG_IO=2, it will output more debug
>>> info during reading
>>>
>>>  So what options do you pass when loading the file?
>>>
>>> Thanks,
>>> Iulian
>>>
>>> ________________________________
>>> From: WARNER, JAMES E. (LARC-D309) [james.e.warner at nasa.gov]
>>> Sent: Tuesday, April 28, 2015 12:47 PM
>>> To: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
>>> Subject: Re: [MOAB-dev] Polyhedra elements in parallell
>>>
>>> Hi Iulian,
>>>
>>> I fixed the bug I mentioned previously but still get the same MOAB error
>>> when reading the mesh with two partitions. Attached here are the
>>>original
>>> (.p1.) and partitioned (.p2.) meshes. Let me know if you have any
>>> suggestions when you get a chance to take a look. Thanks a lot!
>>>
>>> Best,
>>> Jim
>>>
>>>
>>> From: <WARNER>, "WARNER, JAMES E. (LARC-D309)" <james.e.warner at nasa.gov>
>>> Date: Monday, April 27, 2015 5:00 PM
>>> To: "Grindeanu, Iulian R." <iulian at mcs.anl.gov>, "moab-dev at mcs.anl.gov"
>>> <moab-dev at mcs.anl.gov>
>>> Subject: Re: [MOAB-dev] Polyhedra elements in parallell
>>>
>>> Hi Iulian,
>>>
>>> The face that the two polyhedron elements have in common was given a
>>> different ID in each element's connectivityŠ so that is probably the
>>>issue.
>>> I'm going to look into fixing that and I'll let you know if it's
>>>resolved
>>> afterwards. Sorry about that!
>>>
>>> -Jim
>>>
>>>
>>> From: <Grindeanu>, "Iulian R." <iulian at mcs.anl.gov>
>>> Date: Monday, April 27, 2015 4:42 PM
>>> To: "WARNER, JAMES E. (LARC-D309)" <james.e.warner at nasa.gov>,
>>> "moab-dev at mcs.anl.gov" <moab-dev at mcs.anl.gov>
>>> Subject: RE: [MOAB-dev] Polyhedra elements in parallell
>>>
>>>
>>> my guess is that mbpart (with default options) distributed all faces to
>>> different processors, so maybe meshes are not "complete" in the
>>>partition; I
>>> think that mbpart will need to change for polyhedral meshes.
>>> As Rajeev said, can you send your mesh files (before and after
>>>partitioning)
>>> ?
>>> Thanks,
>>> Iulian
>>> ________________________________
>>> From:moab-dev-bounces at mcs.anl.gov [moab-dev-bounces at mcs.anl.gov] on
>>>behalf
>>> of WARNER, JAMES E. (LARC-D309) [james.e.warner at nasa.gov]
>>> Sent: Monday, April 27, 2015 3:12 PM
>>> To: moab-dev at mcs.anl.gov
>>> Subject: [MOAB-dev] Polyhedra elements in parallell
>>>
>>> Hello,
>>>
>>> I generated a MOAB finite element mesh with polyhedra elements that
>>>works
>>> fine on one processor. However, when I use mbpart to generate two
>>>partitions
>>> and run my finite element code in parallel, I get the following error
>>>when
>>> calling DMMoabLoadFromFile() :
>>>
>>> "
>>> ParallelComm.cpp:2928: moab::ErrorCode
>>> moab::ParallelComm::get_local_handles(moab::EntityHandle*, int, const
>>> std::vector<long unsigned int>&): Assertion
>>>`ID_FROM_HANDLE(from_vec[i]) <
>>> (int) new_ents.size()' failed.
>>> "
>>>
>>> Ultimately, I'd like to use MOAB's polyhedra elements to implement
>>>arbitrary
>>> user-defined finite elements in the code I work on, but as a simple
>>>initial
>>> test I am trying to create a mesh with just two 3D bricks using
>>>MBPOLYHEDRON
>>> rather than MBHEX. I first generate 2D quad elements for the six facets
>>>of
>>> each 3D element and use an array of these facets as the connectivity for
>>> each polyhedron element, according to the documentation. I don't get any
>>> warning or error messages when generating the input mesh file or when
>>> calling mbpart to partition the mesh, only when trying to read it into
>>>the
>>> finite element code in parallel. And again, it appears to be working
>>> correctly in serial.
>>>
>>> It's possible I am overlooking something in my code for generating the
>>>mesh,
>>> so if you can provide any info about the error above and what may be
>>>causing
>>> it I'd appreciate it!
>>>
>>> Thanks a lot,
>>> -Jim
>>>
>>>
>>>
>>>
>