[MOAB-dev] Polyhedra elements in parallell

Fri May 1 13:46:11 CDT 2015

Vijay & Iulian,

I pulled the code and the bug seems to be fixed. Thanks a lot for the
quick response - I appreciate that. Have a great weekend!

Best,
Jim

-- 
James E. Warner, PhD
Research Computer Scientist
NASA Langley Research Center
Durability, Damage Tolerance, & Reliability Branch
Building 1205; Rm 212

On 5/1/15 10:12 AM, "Vijay S. Mahadevan" <vijay.m at gmail.com> wrote:

>James,
>
>There is a PR that Iulian created to fix this issue.
>
>https://bitbucket.org/fathomteam/moab/pull-request/106/ghosting-polyhedra-
>elements/
>
>Can you fetch from the repo and test against the branch:
>iulian07/ghost_polyhedra to see if the bug has been fixed ? If you can
>confirm, we will review and merge the PR.
>
>Vijay
>
>On Tue, Apr 28, 2015 at 4:04 PM, Grindeanu, Iulian R.
><iulian at mcs.anl.gov> wrote:
>> Hi Jim,
>>
>> I am able to reproduce the error when I use ghosting
>>
>> mpiexec -np 2 mbconvert -O PARALLEL=READ_PART -O
>> PARTITION=PARALLEL_PARTITION -O PARALLEL_RESOLVE_SHARED_ENTS -o
>> PARALLEL=WRITE_PART -O PARALLEL_GHOSTS=3.0.1 twoPolyhedra.p2.h5m jj.h5
>>
>> It is a bug, thank you for reporting it. It is related to polyhedra
>> elements, most likely.
>>
>> I am looking at it,
>> thanks,
>> Iulian
>>
>>
>>
>>
>> ________________________________
>> From: WARNER, JAMES E. (LARC-D309) [james.e.warner at nasa.gov]
>> Sent: Tuesday, April 28, 2015 3:26 PM
>>
>> To: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
>> Subject: Re: [MOAB-dev] Polyhedra elements in parallell
>>
>> Hi Iulian,
>>
>> Thanks for the quick responseŠ
>>
>> I am able to run that 'mbconvert' command in parallel as well without
>>error.
>>
>> Where I am running into an issue is when I try to read the partitioned
>>mesh
>> file (twoPolyhedra.p2.h5m) into a finite element code using the
>> DMMoabLoadFromFile(Š) function on two processors. I get the following
>>error
>> after calling DMMoabLoadFromFile() :
>>
>> "
>> ParallelComm.cpp:2928: moab::ErrorCode
>> moab::ParallelComm::get_local_handles(moab::EntityHandle*, int, const
>> std::vector<long unsigned int>&): Assertion
>>`ID_FROM_HANDLE(from_vec[i]) <
>> (int) new_ents.size()' failed.
>> "
>>
>> However, reading the unpartitioned mesh (twoPolyhedra.p1.h5m) on one
>> processor works fine.
>>
>> -Jim
>>
>>
>> From: <Grindeanu>, "Iulian R." <iulian at mcs.anl.gov>
>> Date: Tuesday, April 28, 2015 2:58 PM
>> To: "WARNER, JAMES E. (LARC-D309)" <james.e.warner at nasa.gov>,
>> "moab-dev at mcs.anl.gov" <moab-dev at mcs.anl.gov>
>> Subject: RE: [MOAB-dev] Polyhedra elements in parallell
>>
>> Hi Jim,
>> I was able to "mbconvert" in parallel your file, with a command like
>> mpiexec -np 2 mbconvert -O PARALLEL=READ_PART -O
>> PARTITION=PARALLEL_PARTITION -O PARALLEL_RESOLVE_SHARED_ENTS -o
>> PARALLEL=WRITE_PART twoPolyhedra.p2.h5m jj.h5m
>>
>> mbpart created correctly the partitions, (each with one element).
>>
>> What options do you use when you load the file in parallel?
>>
>> You can add also an option like -O DEBUG_IO=2, it will output more debug
>> info during reading
>>
>>  So what options do you pass when loading the file?
>>
>> Thanks,
>> Iulian
>>
>> ________________________________
>> From: WARNER, JAMES E. (LARC-D309) [james.e.warner at nasa.gov]
>> Sent: Tuesday, April 28, 2015 12:47 PM
>> To: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
>> Subject: Re: [MOAB-dev] Polyhedra elements in parallell
>>
>> Hi Iulian,
>>
>> I fixed the bug I mentioned previously but still get the same MOAB error
>> when reading the mesh with two partitions. Attached here are the
>>original
>> (.p1.) and partitioned (.p2.) meshes. Let me know if you have any
>> suggestions when you get a chance to take a look. Thanks a lot!
>>
>> Best,
>> Jim
>>
>>
>> From: <WARNER>, "WARNER, JAMES E. (LARC-D309)" <james.e.warner at nasa.gov>
>> Date: Monday, April 27, 2015 5:00 PM
>> To: "Grindeanu, Iulian R." <iulian at mcs.anl.gov>, "moab-dev at mcs.anl.gov"
>> <moab-dev at mcs.anl.gov>
>> Subject: Re: [MOAB-dev] Polyhedra elements in parallell
>>
>> Hi Iulian,
>>
>> The face that the two polyhedron elements have in common was given a
>> different ID in each element's connectivityŠ so that is probably the
>>issue.
>> I'm going to look into fixing that and I'll let you know if it's
>>resolved
>> afterwards. Sorry about that!
>>
>> -Jim
>>
>>
>> From: <Grindeanu>, "Iulian R." <iulian at mcs.anl.gov>
>> Date: Monday, April 27, 2015 4:42 PM
>> To: "WARNER, JAMES E. (LARC-D309)" <james.e.warner at nasa.gov>,
>> "moab-dev at mcs.anl.gov" <moab-dev at mcs.anl.gov>
>> Subject: RE: [MOAB-dev] Polyhedra elements in parallell
>>
>>
>> my guess is that mbpart (with default options) distributed all faces to
>> different processors, so maybe meshes are not "complete" in the
>>partition; I
>> think that mbpart will need to change for polyhedral meshes.
>> As Rajeev said, can you send your mesh files (before and after
>>partitioning)
>> ?
>> Thanks,
>> Iulian
>> ________________________________
>> From:moab-dev-bounces at mcs.anl.gov [moab-dev-bounces at mcs.anl.gov] on
>>behalf
>> of WARNER, JAMES E. (LARC-D309) [james.e.warner at nasa.gov]
>> Sent: Monday, April 27, 2015 3:12 PM
>> To: moab-dev at mcs.anl.gov
>> Subject: [MOAB-dev] Polyhedra elements in parallell
>>
>> Hello,
>>
>> I generated a MOAB finite element mesh with polyhedra elements that
>>works
>> fine on one processor. However, when I use mbpart to generate two
>>partitions
>> and run my finite element code in parallel, I get the following error
>>when
>> calling DMMoabLoadFromFile() :
>>
>> "
>> ParallelComm.cpp:2928: moab::ErrorCode
>> moab::ParallelComm::get_local_handles(moab::EntityHandle*, int, const
>> std::vector<long unsigned int>&): Assertion
>>`ID_FROM_HANDLE(from_vec[i]) <
>> (int) new_ents.size()' failed.
>> "
>>
>> Ultimately, I'd like to use MOAB's polyhedra elements to implement
>>arbitrary
>> user-defined finite elements in the code I work on, but as a simple
>>initial
>> test I am trying to create a mesh with just two 3D bricks using
>>MBPOLYHEDRON
>> rather than MBHEX. I first generate 2D quad elements for the six facets
>>of
>> each 3D element and use an array of these facets as the connectivity for
>> each polyhedron element, according to the documentation. I don't get any
>> warning or error messages when generating the input mesh file or when
>> calling mbpart to partition the mesh, only when trying to read it into
>>the
>> finite element code in parallel. And again, it appears to be working
>> correctly in serial.
>>
>> It's possible I am overlooking something in my code for generating the
>>mesh,
>> so if you can provide any info about the error above and what may be
>>causing
>> it I'd appreciate it!
>>
>> Thanks a lot,
>> -Jim
>>
>>
>>
>>