[MOAB-dev] Polyhedra elements in parallell

Tue Apr 28 16:04:26 CDT 2015

Hi Jim,

I am able to reproduce the error when I use ghosting

mpiexec -np 2 mbconvert -O PARALLEL=READ_PART -O PARTITION=PARALLEL_PARTITION -O PARALLEL_RESOLVE_SHARED_ENTS -o PARALLEL=WRITE_PART -O PARALLEL_GHOSTS=3.0.1 twoPolyhedra.p2.h5m jj.h5

It is a bug, thank you for reporting it. It is related to polyhedra elements, most likely.

I am looking at it,
thanks,
Iulian

________________________________
From: WARNER, JAMES E. (LARC-D309) [james.e.warner at nasa.gov]
Sent: Tuesday, April 28, 2015 3:26 PM
To: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
Subject: Re: [MOAB-dev] Polyhedra elements in parallell

Hi Iulian,

Thanks for the quick response…

I am able to run that 'mbconvert' command in parallel as well without error.

Where I am running into an issue is when I try to read the partitioned mesh file (twoPolyhedra.p2.h5m) into a finite element code using the DMMoabLoadFromFile(…) function on two processors. I get the following error after calling DMMoabLoadFromFile() :

"
ParallelComm.cpp:2928: moab::ErrorCode moab::ParallelComm::get_local_handles(moab::EntityHandle*, int, const std::vector<long unsigned int>&): Assertion `ID_FROM_HANDLE(from_vec[i]) < (int) new_ents.size()' failed.
"

However, reading the unpartitioned mesh (twoPolyhedra.p1.h5m) on one processor works fine.

-Jim

From: <Grindeanu>, "Iulian R." <iulian at mcs.anl.gov<mailto:iulian at mcs.anl.gov>>
Date: Tuesday, April 28, 2015 2:58 PM
To: "WARNER, JAMES E. (LARC-D309)" <james.e.warner at nasa.gov<mailto:james.e.warner at nasa.gov>>, "moab-dev at mcs.anl.gov<mailto:moab-dev at mcs.anl.gov>" <moab-dev at mcs.anl.gov<mailto:moab-dev at mcs.anl.gov>>
Subject: RE: [MOAB-dev] Polyhedra elements in parallell

Hi Jim,
I was able to "mbconvert" in parallel your file, with a command like
mpiexec -np 2 mbconvert -O PARALLEL=READ_PART -O PARTITION=PARALLEL_PARTITION -O PARALLEL_RESOLVE_SHARED_ENTS -o PARALLEL=WRITE_PART twoPolyhedra.p2.h5m jj.h5m

mbpart created correctly the partitions, (each with one element).

What options do you use when you load the file in parallel?

You can add also an option like -O DEBUG_IO=2, it will output more debug info during reading

 So what options do you pass when loading the file?

Thanks,
Iulian

________________________________
From: WARNER, JAMES E. (LARC-D309) [james.e.warner at nasa.gov<mailto:james.e.warner at nasa.gov>]
Sent: Tuesday, April 28, 2015 12:47 PM
To: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov<mailto:moab-dev at mcs.anl.gov>
Subject: Re: [MOAB-dev] Polyhedra elements in parallell

Hi Iulian,

I fixed the bug I mentioned previously but still get the same MOAB error when reading the mesh with two partitions. Attached here are the original (.p1.) and partitioned (.p2.) meshes. Let me know if you have any suggestions when you get a chance to take a look. Thanks a lot!

Best,
Jim

From: <WARNER>, "WARNER, JAMES E. (LARC-D309)" <james.e.warner at nasa.gov<mailto:james.e.warner at nasa.gov>>
Date: Monday, April 27, 2015 5:00 PM
To: "Grindeanu, Iulian R." <iulian at mcs.anl.gov<mailto:iulian at mcs.anl.gov>>, "moab-dev at mcs.anl.gov<mailto:moab-dev at mcs.anl.gov>" <moab-dev at mcs.anl.gov<mailto:moab-dev at mcs.anl.gov>>
Subject: Re: [MOAB-dev] Polyhedra elements in parallell

Hi Iulian,

The face that the two polyhedron elements have in common was given a different ID in each element's connectivity… so that is probably the issue. I'm going to look into fixing that and I'll let you know if it's resolved afterwards. Sorry about that!

-Jim

From: <Grindeanu>, "Iulian R." <iulian at mcs.anl.gov<mailto:iulian at mcs.anl.gov>>
Date: Monday, April 27, 2015 4:42 PM
To: "WARNER, JAMES E. (LARC-D309)" <james.e.warner at nasa.gov<mailto:james.e.warner at nasa.gov>>, "moab-dev at mcs.anl.gov<mailto:moab-dev at mcs.anl.gov>" <moab-dev at mcs.anl.gov<mailto:moab-dev at mcs.anl.gov>>
Subject: RE: [MOAB-dev] Polyhedra elements in parallell

my guess is that mbpart (with default options) distributed all faces to different processors, so maybe meshes are not "complete" in the partition; I think that mbpart will need to change for polyhedral meshes.
As Rajeev said, can you send your mesh files (before and after partitioning) ?
Thanks,
Iulian
________________________________
From:moab-dev-bounces at mcs.anl.gov<mailto:moab-dev-bounces at mcs.anl.gov> [moab-dev-bounces at mcs.anl.gov<mailto:moab-dev-bounces at mcs.anl.gov>] on behalf of WARNER, JAMES E. (LARC-D309) [james.e.warner at nasa.gov<mailto:james.e.warner at nasa.gov>]
Sent: Monday, April 27, 2015 3:12 PM
To: moab-dev at mcs.anl.gov<mailto:moab-dev at mcs.anl.gov>
Subject: [MOAB-dev] Polyhedra elements in parallell

Hello,

I generated a MOAB finite element mesh with polyhedra elements that works fine on one processor. However, when I use mbpart to generate two partitions and run my finite element code in parallel, I get the following error when calling DMMoabLoadFromFile() :

"
ParallelComm.cpp:2928: moab::ErrorCode moab::ParallelComm::get_local_handles(moab::EntityHandle*, int, const std::vector<long unsigned int>&): Assertion `ID_FROM_HANDLE(from_vec[i]) < (int) new_ents.size()' failed.
"

Ultimately, I'd like to use MOAB's polyhedra elements to implement arbitrary user-defined finite elements in the code I work on, but as a simple initial test I am trying to create a mesh with just two 3D bricks using MBPOLYHEDRON rather than MBHEX. I first generate 2D quad elements for the six facets of each 3D element and use an array of these facets as the connectivity for each polyhedron element, according to the documentation. I don't get any warning or error messages when generating the input mesh file or when calling mbpart to partition the mesh, only when trying to read it into the finite element code in parallel. And again, it appears to be working correctly in serial.

It's possible I am overlooking something in my code for generating the mesh, so if you can provide any info about the error above and what may be causing it I'd appreciate it!

Thanks a lot,
-Jim

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/moab-dev/attachments/20150428/6234c1c8/attachment-0001.html>