[MOAB-dev] r4632 - in MOAB/trunk: src/io src/parallel src/parallel/moab test/parallel tools/vtkMOABReader

[Tim Tautges]

On 03/24/2011 02:09 AM, Rajeev Jain wrote:
> Hi Tim,
>
> I'm unable to get the parallel version of ACIS based MeshKit running,
> there is an error when running CoreGen (HongJun has the same thing, we
> are trying to fix it.
> ---------------------------------------------------------------------------------------
> ....MeshKit-acis/rgg> coregen
> .....
> Internal Error: invalid error code 1dc52e0 (Ring Index out of range) in
> MPIR_Init_thread:394
> p0_5954: p4_error: interrupt SIGSEGV: 11
> -----------------------------------------------------------------------------------------

Are you sure you're using an mpi_exec or mpirun from the same version of mpi that was used to compile?

> my OCC based MeshKit works and I'll try to see how to parallel mergemesh
> fits into CoreGen.
> Is there an example for using ParallelMergeMesh?
>

Right now that work is pretty preliminary; it should be more ready for testing in about 3 wks though.  There's a test in 
test/parallel/parmerge.cpp, but basically the idea is for each proc to read a separate piece of the mesh, then fire off 
the parallel merge.  Performance isn't going to be very good right now, due to sorting on floating point data; that's 
the part we're improving in the next few weeks.

> Recently, I've found that saving and extrusion are the steps that eat a
> lot of memory and take the most time.
> I'm thinking of parallel save and extrusion?
>

I think it would be better to run the whole coregen process in parallel eventually.  Before we do that, though, we'll 
want to make sure we have enough of a user base to justify it.

> Also, exploring pCAMAL in Cubit 12.2:
> "The /set parallel meshing/ works with the /export parallel/ command to
> generate a boundary mesh suitable for sweeping with the pCAMAL application."

Let's hold off on that for now.

> Rajeev
> P.S. Please let me know the correction/additions to the poster.
>

Will do.

- tim

> ------------------------------------------------------------------------
> *From:* "tautges at mcs.anl.gov" <tautges at mcs.anl.gov>
> *To:* moab-dev at mcs.anl.gov
> *Sent:* Wednesday, March 23, 2011 13:58:18
> *Subject:* [MOAB-dev] r4632 - in MOAB/trunk: src/io src/parallel
> src/parallel/moab test/parallel tools/vtkMOABReader
>
> Author: tautges
> Date: 2011-03-23 13:58:17 -0500 (Wed, 23 Mar 2011)
> New Revision: 4632
>
> Modified:
> MOAB/trunk/src/io/ReadNC.cpp
> MOAB/trunk/src/parallel/ParallelMergeMesh.cpp
> MOAB/trunk/src/parallel/moab/ParallelMergeMesh.hpp
> MOAB/trunk/test/parallel/parmerge.cpp
> MOAB/trunk/tools/vtkMOABReader/vtkMOABReader.cxx
> Log:
> ParallelMergeMesh: updates from Nathan, now it works.
>
> ReadNC: change method used to partition, so it checks available
> parameters against # procs;
> also partition over I first, then J, then K.
>
> vtkMOABReader: handle tags on vertices differently, as data associated
> with points instead
> of with point cells.
>
>
>
> Modified: MOAB/trunk/src/io/ReadNC.cpp
> ===================================================================
> --- MOAB/trunk/src/io/ReadNC.cpp 2011-03-22 05:39:07 UTC (rev 4631)
> +++ MOAB/trunk/src/io/ReadNC.cpp 2011-03-23 18:58:17 UTC (rev 4632)
> @@ -646,25 +646,39 @@
> if (isParallel) {
> // partition *the elements* over the parametric space; 1d partition for
> now, in the k parameter
> // (or j if there isn't one)
> - if (-1 != klMin) {
> - int dk = (kMax - kMin) / myPcomm->proc_config().proc_size();
> - unsigned int extra = (kMax - kMin) % myPcomm->proc_config().proc_size();
> - klMin = kMin + myPcomm->proc_config().proc_rank()*dk +
> +
> + if (-1 != ilMin && (iMax - iMin) > myPcomm->proc_config().proc_size()) {
> + int di = (iMax - iMin) / myPcomm->proc_config().proc_size();
> + unsigned int extra = (iMax - iMin) % myPcomm->proc_config().proc_size();
> + ilMin = iMin + myPcomm->proc_config().proc_rank()*di +
> std::min(myPcomm->proc_config().proc_rank(), extra);
> - klMax = klMin + dk + (myPcomm->proc_config().proc_rank() < extra ? 1 : 0);
> + ilMax = ilMin + di + (myPcomm->proc_config().proc_rank() < extra ? 1 : 0);
>
> - ilMin = iMin; ilMax = iMax;
> + klMin = kMin; klMax = kMax;
> jlMin = jMin; jlMax = jMax;
> }
> - else {
> + else if (-1 != jlMin && (jMax - jMin) >
> myPcomm->proc_config().proc_size()) {
> int dj = (jMax - jMin) / myPcomm->proc_config().proc_size();
> unsigned int extra = (jMax - jMin) % myPcomm->proc_config().proc_size();
> jlMin = jMin + myPcomm->proc_config().proc_rank()*dj +
> std::min(myPcomm->proc_config().proc_rank(), extra);
> jlMax = jlMin + dj + (myPcomm->proc_config().proc_rank() < extra ? 1 : 0);
>
> + klMin = kMin; klMax = kMax;
> ilMin = iMin; ilMax = iMax;
> }
> + else if (-1 != klMin && (kMax - kMin) >
> myPcomm->proc_config().proc_size()) {
> + int dk = (kMax - kMin) / myPcomm->proc_config().proc_size();
> + unsigned int extra = (kMax - kMin) % myPcomm->proc_config().proc_size();
> + klMin = kMin + myPcomm->proc_config().proc_rank()*dk +
> + std::min(myPcomm->proc_config().proc_rank(), extra);
> + klMax = klMin + dk + (myPcomm->proc_config().proc_rank() < extra ? 1 : 0);
> +
> + jlMin = jMin; jlMax = jMax;
> + ilMin = iMin; ilMax = iMax;
> + }
> + else
> + ERRORR(MB_FAILURE, "Couldn't find a suitable partition.");
> }
> #endif
>
>

-- 
================================================================
"You will keep in perfect peace him whose mind is
   steadfast, because he trusts in you."               Isaiah 26:3

              Tim Tautges            Argonne National Laboratory
          (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
          phone: (608) 263-8485      1500 Engineering Dr.
            fax: (608) 263-4499      Madison, WI 53706