[MOAB-dev] commit/MOAB: 7 new changesets

commits-noreply at bitbucket.org commits-noreply at bitbucket.org
Sun Jun 30 11:16:52 CDT 2013


7 new commits in MOAB:

https://bitbucket.org/fathomteam/moab/commits/efa4e04799a9/
Changeset:   efa4e04799a9
Branch:      None
User:        vijaysm
Date:        2013-04-09 01:32:34
Summary:     Fix minor debug print for reduce tags.

Affected #:  1 file

diff --git a/src/parallel/ParallelComm.cpp b/src/parallel/ParallelComm.cpp
index ec908b9..348bad0 100644
--- a/src/parallel/ParallelComm.cpp
+++ b/src/parallel/ParallelComm.cpp
@@ -7234,7 +7234,7 @@ ErrorCode ParallelComm::post_irecv(std::vector<unsign=
ed int>& shared_procs,
       RRA("Failure in waitall in tag exchange.");
     }
  =20
-    myDebug->tprintf(1, "Exiting exchange_tags");
+    myDebug->tprintf(1, "Exiting reduce_tags");
=20
     return MB_SUCCESS;
   }


https://bitbucket.org/fathomteam/moab/commits/9c7b4f0a30b0/
Changeset:   9c7b4f0a30b0
Branch:      None
User:        vijaysm
Date:        2013-04-18 05:30:20
Summary:     Merge remote-tracking branch 'upstream/master'

Affected #:  9 files

diff --git a/.gitignore b/.gitignore
index 52cd4d2..6d29b5e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,11 @@
 MOABConfig.cmake
+moab.files
+config/test-driver
 moab.config
+moab.includes
+moab.creator*
+*.ccm
+*.cub
 aclocal.m4
 autom4te.cache/
 config.h
@@ -68,6 +74,7 @@ share/*
 bin/*
 *~
 examples/HelloMoabPar
+examples/TestExodusII
 itaps/igeom/FBiGeom_protos.h
 itaps/igeom/testSmooth2
 itaps/igeom/testSmoothGeom

diff --git a/examples/HelloMoabPar.cpp b/examples/HelloMoabPar.cpp
index cf308a7..a5a71aa 100644
--- a/examples/HelloMoabPar.cpp
+++ b/examples/HelloMoabPar.cpp
@@ -1,18 +1,18 @@
 /** @example HelloMoabPar.cpp \n
  * \brief Read mesh into MOAB in parallel \n
- * This example shows the simplest way of telling MOAB to read in parallel=
. \n
+ * This example shows the simplest way of telling MOAB to read in parallel=
.
  *
- * 0. Initialize MPI and get the rank and number of processors \n
- * 1. Process arguments (file name and options for parallel read) \n
- * 2. Initialize MOAB \n
- * 3. Load a partitioned file in parallel; \n
- * 4. retrieve shared entities on each processor \n
- * 5. Filter owned entities among shared ones on each processor \n
- * 6. Exchange ghost layers, and repeat the reports \n
+ *    -#  Initialize MPI and get the rank and number of processors \n
+ *    -#  Process arguments (file name and options for parallel read) \n
+ *    -#  Initialize MOAB \n
+ *    -#  Load a partitioned file in parallel; \n
+ *    -#  retrieve shared entities on each processor \n
+ *    -#  Filter owned entities among shared ones on each processor \n
+ *    -#  Exchange ghost layers, and repeat the reports \n
  *
- * To compile: \n
+ * <b>To compile</b>: \n
  *    make HelloMoabPar MOAB_DIR=3D<installdir>  \n
- * To run: mpiexec -np 4 HelloMoabPar \n
+ * <b>To run</b>: mpiexec -np 4 HelloMoabPar \n
  *  (depending on your configuration, LD_LIBRARY_PATH may need to contain =
<hdf5>/lib folder)
  *
  */

diff --git a/examples/ReduceExchangeTags.cpp b/examples/ReduceExchangeTags.=
cpp
index 4fb657b..6c2e7de 100644
--- a/examples/ReduceExchangeTags.cpp
+++ b/examples/ReduceExchangeTags.cpp
@@ -1,27 +1,27 @@
-/** \example ReduceExchangeTags ReduceExchangeTags.cpp
+/** @example ReduceExchangeTags.cpp
  * \brief Example program that shows the use case for performing tag data =
exchange
  * between parallel processors in order to sync data on shared entities. T=
he reduction
  * operation on tag data is also shown where the user can perform any of t=
he actions supported
- * by MPI_Op on data residing on shared entities.
+ * by MPI_Op on data residing on shared entities. \n
  *
- * This example:
- * 0.  Initialize MPI and instantiate MOAB
- * 1.  Get user options: Input mesh file name, tag name (default: USERTAG)=
, tag value (default: 1.0)
- * 2.  Create the root and partition sets
- * 3.  Instantiate ParallelComm and read the mesh file in parallel using a=
ppropriate options
- * 4.  Create two tags: USERTAG_EXC (exchange) and USERTAG_RED (reduction)
- * 5.  Set tag data and exchange shared entity information between process=
ors
- * 5a. Get entities in all dimensions and set local (current rank, dimensi=
on) dependent data for
+ * <b>This example </b>:
+ *    -# Initialize MPI and instantiate MOAB
+ *    -# Get user options: Input mesh file name, tag name (default: USERTA=
G), tag value (default: 1.0)
+ *    -# Create the root and partition sets
+ *    -# Instantiate ParallelComm and read the mesh file in parallel using=
 appropriate options
+ *    -# Create two tags: USERTAG_EXC (exchange) and USERTAG_RED (reductio=
n)
+ *    -# Set tag data and exchange shared entity information between proce=
ssors
+ *      -# Get entities in all dimensions and set local (current rank, dim=
ension) dependent data for
  *     exchange tag (USERTAG_EXC)
- * 5b. Perform exchange of tag data so that data on shared entities are sy=
nced via ParallelCommunicator.
- * 6.  Set tag data and reduce shared entity information between processor=
s using MPI_SUM
- * 6a. Get higher dimensional entities in the current partition and set lo=
cal (current rank)
+ *      -# Perform exchange of tag data so that data on shared entities ar=
e synced via ParallelCommunicator.
+ *    -#  Set tag data and reduce shared entity information between proces=
sors using MPI_SUM
+ *      -#  Get higher dimensional entities in the current partition and s=
et local (current rank)
  *     dependent data for reduce tag (USERTAG_EXC)
- * 6b. Perform the reduction operation (MPI_SUM) on shared entities via Pa=
rallelCommunicator.
- * 7. Destroy the MOAB instance and finalize MPI
+ *      -#  Perform the reduction operation (MPI_SUM) on shared entities v=
ia ParallelCommunicator.
+ *    -#  Destroy the MOAB instance and finalize MPI
  *
- * To run: mpiexec -n 2 ./ReduceExchangeTags <mesh_file><tag_name><tag_val=
ue>
- * Example: mpiexec -n 2 ./ReduceExchangeTags ../MeshFiles/unittest/64bric=
ks_1khex.h5m USERTAG 100
+ * <b>To run:</b> \n mpiexec -n 2 ./ReduceExchangeTags <mesh_file><tag_nam=
e><tag_value> \n
+ * <b>Example:</b> \n mpiexec -n 2 ./ReduceExchangeTags ../MeshFiles/unitt=
est/64bricks_1khex.h5m USERTAG 100 \n
  *
  */
=20

diff --git a/examples/StructuredMeshSimple.cpp b/examples/StructuredMeshSim=
ple.cpp
new file mode 100644
index 0000000..25aa08f
--- /dev/null
+++ b/examples/StructuredMeshSimple.cpp
@@ -0,0 +1,110 @@
+/** @example StructuredMeshSimple.cpp
+ * \brief Show creation and query of structured mesh, serial or parallel, =
through MOAB's structured mesh interface.
+ * This is an example showing creation and query of a 3D structured mesh. =
 In serial, a single N*N*N block of elements
+ * is created; in parallel, each proc gets an N*N*N block, with blocks arr=
anged in a 1d column, sharing vertices
+ * and faces at their interfaces (proc 0 has no left neighbor and proc P-1=
 no right neighbor).
+ * Each square block of hex elements is then referenced by its ijk paramet=
erization.
+ * 1D and 2D examples could be made simply by changing the dimension param=
eter passed into the MOAB functions. \n
+ *
+ * <b>This example </b>:
+ *    -# Instantiate MOAB and get the structured mesh interface
+ *    -# Decide what the local parameters of the mesh will be, based on pa=
rallel/serial and rank.
+ *    -# Create a N^d structured mesh, which includes (N+1)^d vertices and=
 N^d elements.
+ *    -# Get the vertices and elements from moab and check their numbers a=
gainst (N+1)^d and N^d, resp.
+ *    -# Loop over elements in d nested loops over i, j, k; for each (i,j,=
k):
+ *      -# Get the element corresponding to (i,j,k)
+ *      -# Get the connectivity of the element
+ *      -# Get the coordinates of the vertices comprising that element
+ *    -# Release the structured mesh interface and destroy the MOAB instan=
ce
+ *
+ * <b> To run: </b> ./structuredmesh [d [N] ] \n
+ * (default values so can run w/ no user interaction)
+ */
+
+#include "moab/Core.hpp"
+#include "moab/ScdInterface.hpp"
+#include "moab/ProgOptions.hpp"
+#include "moab/CN.hpp"
+#include <iostream>
+#include <vector>
+
+using namespace moab;
+
+int main(int argc, char **argv)=20
+{
+  int N =3D 10, dim =3D 3;
+
+  ProgOptions opts;
+  opts.addOpt<int>(std::string("dim,d"), std::string("Dimension of mesh (d=
efault=3D3)"),
+                   &dim);
+  opts.addOpt<int>(std::string(",n"), std::string("Number of elements on a=
 side (default=3D10)"),
+                   &N);
+  opts.parseCommandLine(argc, argv);
+ =20
+    // 0. Instantiate MOAB and get the structured mesh interface
+  Interface *mb =3D new Core();
+  ScdInterface *scdiface;
+  ErrorCode rval =3D mb->query_interface(scdiface); // get a ScdInterface =
object through moab instance
+  if (MB_SUCCESS !=3D rval) return rval;
+
+    // 1. Decide what the local parameters of the mesh will be, based on p=
arallel/serial and rank.
+  int ilow =3D 0, ihigh =3D N;
+  int rank =3D 0, nprocs =3D 1;
+#ifdef USE_MPI
+  MPI_Comm_size(MPI_COMM_WORLD, &nprocs); MPI_Comm_rank(MPI_COMM_WORLD, &r=
ank);
+  ilow =3D rank*N; ihigh =3D ilow + N;
+#endif =20
+
+    // 2. Create a N^d structured mesh, which includes (N+1)^d vertices an=
d N^d elements.
+  ScdBox *box;
+  rval =3D scdiface->construct_box(HomCoord(ilow, (dim>1?0:-1), (dim>2?0:-=
1)), // use in-line logical tests to handle dimensionality
+                                 HomCoord(ihigh, (dim>1?N:-1), (dim>2?N:-1=
)),=20
+                                 NULL, 0, // NULL coords vector and 0 coor=
ds (don't specify coords for now)
+                                 box);    // box is the structured box obj=
ect providing the parametric
+                                          // structured mesh interface for=
 this rectangle of elements
+  if (MB_SUCCESS !=3D rval) return rval;
+
+    // 3. Get the vertices and elements from moab and check their numbers =
against (N+1)^d and N^d, resp.
+  Range verts, elems;
+  rval =3D mb->get_entities_by_dimension(0, 0, verts); // first '0' specif=
ies "root set", or entire MOAB instance, second the entity dimension being =
requested
+  if (MB_SUCCESS !=3D rval) return rval;
+  rval =3D mb->get_entities_by_dimension(0, dim, elems);
+  if (MB_SUCCESS !=3D rval) return rval;
+
+#define MYSTREAM(a) if (!rank) std::cout << a << std::endl
+ =20
+  if (pow(N,dim) =3D=3D (int) elems.size() && pow(N+1,dim) =3D=3D (int) ve=
rts.size()) { // expected #e and #v are N^d and (N+1)^d, resp.
+#ifdef USE_MPI
+    MYSTREAM("Proc 0: ");
+#endif
+    MYSTREAM("Created " << elems.size() << " " << CN::EntityTypeName(mb->t=
ype_from_handle(*elems.begin()))=20
+             << " elements and " << verts.size() << " vertices." << std::e=
ndl);
+  }
+  else
+    std::cout << "Created the wrong number of vertices or hexes!" << std::=
endl;
+ =20
+    // 4. Loop over elements in 3 nested loops over i, j, k; for each (i,j=
,k):
+  std::vector<double> coords(3*pow(N+1,dim));
+  std::vector<EntityHandle> connect;
+  for (int k =3D 0; k < (dim>2?N:1); k++) {
+    for (int j =3D 0; j < (dim>1?N:1); j++) {
+      for (int i =3D 0; i < N-1; i++) {
+          // 4a. Get the element corresponding to (i,j,k)
+        EntityHandle ehandle =3D box->get_element(i, j, k);
+        if (0 =3D=3D ehandle) return MB_FAILURE;
+          // 4b. Get the connectivity of the element
+        rval =3D mb->get_connectivity(&ehandle, 1, connect); // get the co=
nnectivity, in canonical order
+        if (MB_SUCCESS !=3D rval) return rval;
+          // 4c. Get the coordinates of the vertices comprising that eleme=
nt
+        rval =3D mb->get_coords(connect.data(), connect.size(), coords.dat=
a()); // get the coordinates of those vertices
+        if (MB_SUCCESS !=3D rval) return rval;
+      }
+    }
+  }
+
+    // 5. Release the structured mesh interface and destroy the MOAB insta=
nce
+  mb->release_interface(scdiface); // tell MOAB we're done with the ScdInt=
erface
+  delete mb;
+ =20
+  return 0;
+}

diff --git a/examples/TestExodusII.cpp b/examples/TestExodusII.cpp
index d353117..c425f66 100644
--- a/examples/TestExodusII.cpp
+++ b/examples/TestExodusII.cpp
@@ -1,9 +1,28 @@
 /** @example TestExodusII.cpp
- *  TestExodusII example demonstrates how to retrieve material, dirichlet =
and neumann sets from an Exodus file
- * Sets are traversed to find out the number of entities contained in each=
 set.
- * The entities contained in each set are retrieved, using a Boolean flag =
to indicate that any contained sets
- * should be traversed recursively to include non-set entities in the resu=
lts. */
-
+ * This example demonstrates how to retrieve material, dirichlet and neuma=
nn sets
+ * from an ExodusII file. \n
+ * Sets in MOAB contain entities and have a tag on them to give meaning to=
 the entities.
+ * Tag names: MATERIAL_SET", "DIRICHLET_SET" and "NEUMANN_SET" are reserve=
d and
+ * are associated with their corresponding entity sets.
+ * Sets are traversed to find out the type number of entities contained in=
 each set. \n
+ *
+ * <b>Steps in this example </b>:
+ *    -# Instantiate MOAB
+ *    -# Get input mesh file name and load it.
+ *    -# Loop over the three sets: material, dirichlet and neumann
+ *      -# Get TagHandle and EntitySet(corresponding to the TagHandle)
+ *      -# Loop thru all the EntitySet's
+ *        -# Get the set id and entities in this set
+ *    -# Destroy the MOAB instance
+ *
+ *
+ * <b> To compile: </b>
+ *    make TestExodusII MOAB_DIR=3D<installdir> \n
+ *
+ * <b> To run: </b>
+ *    -# TestExodusII <mesh-file> \n
+ *    -# TestExodusII (This uses the default <mesh-file>: <MOAB_SRC_DIR>/M=
eshFiles/unittest/mbtest2.g)
+ */
 #include <iostream>
=20
 // Include header for MOAB instance and range
@@ -15,7 +34,7 @@ int main(int argc, char **argv) {
   moab::Interface *mb =3D new moab::Core();
=20
   // If no input is specified load ../MeshFiles/unittest/mbtest2.g
-  const char* test_file_name =3D  std::string("../MeshFiles/unittest/mbtes=
t2.g");
+//  const char* test_file_name =3D  "../MeshFiles/unittest/mbtest2.g";
=20
   // get the material set tag handle
   moab::Tag mtag;
@@ -25,11 +44,12 @@ int main(int argc, char **argv) {
=20
   if (argc =3D=3D 1) {
       std::cout << "Running default case, loading ../MeshFiles/unittest/mb=
test2.g" << std::endl;
-      std::cout << "Usage: " << argv[0] << " <filename>" << std::endl;
+      std::cout << "Usage: " << argv[0] << " <filename>\n" << std::endl;
       rval =3D mb->load_file("../MeshFiles/unittest/mbtest2.g");
     }
   else{
       rval =3D mb->load_file(argv[argc-1]);
+      std::cout << "Loaded mesh file: " << argv[argc-1] << std::endl;
     }
=20
   // loop over set types
@@ -59,15 +79,11 @@ int main(int argc, char **argv) {
=20
           std::cout << tag_nms[i] << " " << set_id << " has "
                     << set_ents.size() << " entities:" << std::endl;
+
+          // print the entities contained in this set
           set_ents.print("   ");
           set_ents.clear();
         }
     }
-
-  // do the same for all sets
-  sets.clear();
-  rval =3D mb->get_entities_by_type(0, moab::MBENTITYSET, sets);
-  if (moab::MB_SUCCESS !=3D rval) return 1;
-
   delete mb;
 }

diff --git a/examples/makefile b/examples/makefile
index e1b8ef0..4ec6154 100644
--- a/examples/makefile
+++ b/examples/makefile
@@ -17,7 +17,10 @@ ReduceExchangeTags : ReduceExchangeTags.o
=20
 HelloMoabPar: HelloMoabPar.o
 	${MOAB_CXX} -o $@ $< ${MOAB_LIBS_LINK}=20
-=09
+
+TestExodusII: TestExodusII.o
+	${MOAB_CXX} -o $@ $< ${MOAB_LIBS_LINK}
+
 .cpp.o :
 	${MOAB_CXX} ${MOAB_CXXFLAGS} ${MOAB_INCLUDES} -DMESH_DIR=3D\"${MESH_DIR}\=
" -c $<
=20

diff --git a/itaps/iBase.h b/itaps/iBase.h
index 5172bb8..933b9d6 100644
--- a/itaps/iBase.h
+++ b/itaps/iBase.h
@@ -261,7 +261,7 @@ enum iBase_TagValueType {
  *************************************************************************=
*****/
=20
 /*************************************************************************=
**//**
- * \mainpage The ITAPS Interfaces
+ * \page The ITAPS Interfaces
  *
  * \subpage ibase
  *
@@ -281,6 +281,8 @@ enum iBase_TagValueType {
  *
  * \subpage numhops
  *
+ * \subpage resilient
+ *
  * \page error Error Handling
  *
  * With few exceptions, every iMesh function includes an output argument,

diff --git a/itaps/imesh/iMeshP.h b/itaps/imesh/iMeshP.h
index 958c827..28da490 100644
--- a/itaps/imesh/iMeshP.h
+++ b/itaps/imesh/iMeshP.h
@@ -33,7 +33,7 @@ enum iMeshP_EntStatus
 #define iMeshP_ALL_PARTS -1
=20
=20
-/** \mainpage
+/** \page imeshp  iMeshP: ITAPS Parallel Mesh Interface
 iMeshP.h -- ITAPS Parallel Mesh Interface
=20
 Release 0.1; October 2008
@@ -45,7 +45,7 @@ Release 0.1; October 2008
    subsets; like a "mesh," it does not imply a serial or parallel=20
    implementation.
 -  An application may use one or more meshes. =20
--  Parititions can create subsets of entities from one or more meshes.
+-  Partitions can create subsets of entities from one or more meshes.
 -  Meshes can be subdivided by one or more partitions.
 -  Partitions contain parts.  Parts contain the subsets of entities in the
    partition.

diff --git a/src/io/mhdf/include/mhdf.h b/src/io/mhdf/include/mhdf.h
index e8aaa90..9a7a478 100644
--- a/src/io/mhdf/include/mhdf.h
+++ b/src/io/mhdf/include/mhdf.h
@@ -22,7 +22,7 @@
 extern "C" {
 #endif
=20
-/** \mainpage H5M File Format API=20
+/** \page h5mmain H5M File Format API
  *
  *\section Intro   Introduction
  *
@@ -60,7 +60,7 @@ extern "C" {
  * hexahedral topology) and the number of nodes in the element. =20
  *
  *
- *\section Root   The \c tstt Group
+ *\section Root   The tstt Group
  *
  * All file data is stored in the \c tstt group in the HDF5 root group.
  * The \c tstt group may have an optional scalar integer attribute=20
@@ -87,12 +87,12 @@ extern "C" {
  * - \c Polyhedron
  *=20
  *
- *\section History   The \c history DataSet
+ *\section History   The history DataSet
  *
  * The \c history DataSet is a list of variable-length strings with
- * appliation-defined meaning. =20
+ * application-defined meaning. =20
  *
- *\section Nodes   The \c nodes Group
+ *\section Nodes   The nodes Group
  *
  *
  * The \c nodes group contains a single DataSet and an optional
@@ -111,7 +111,7 @@ extern "C" {
  * \c coordinates table.
  *
  *
- *\section Elements   The \c elements Group=20
+ *\section Elements   The elements Group
  *
  * The \c elements group contains an application-defined number of=20
  * subgroups.  Each subgroup defines one or more mesh elements that
@@ -141,7 +141,7 @@ extern "C" {
  * sequentially in the order that they are defined in the table.
  *
  *
- *\section Sets   The \c sets Group
+ *\section Sets   The sets Group
  *
  * The \c sets group contains the definitions of any entity sets stored
  * in the file.  It contains 1 to 4 DataSets and the optional \c tags=20
@@ -199,7 +199,7 @@ extern "C" {
  * subgroup is described in the \ref Dense "section on dense tag storage".=20
  *
  *=20
- * \section Tags   The \c tags Group
+ * \section Tags   The tags Group
  *
  * The \c tags group contains a sub-group for each tag defined
  * in the file.  These sub-groups contain the definition of the
@@ -283,7 +283,7 @@ extern "C" {
  * dataset. =20
  *
  *
- * \section Dense   The \c tags Sub-Groups
+ * \section Dense   The tags Sub-Groups
  *
  * Data for fixed-length tags may also be stored in the \c tags sub-group
  * of the \c nodes, \c sets, and subgroups of the \c elements group. =20


https://bitbucket.org/fathomteam/moab/commits/7964c67d54fa/
Changeset:   7964c67d54fa
Branch:      None
User:        vijaysm
Date:        2013-06-20 15:29:07
Summary:     Merge remote-tracking branch 'upstream/master'

Affected #:  59 files

diff --git a/MeshFiles/unittest/io/Makefile.am b/MeshFiles/unittest/io/Make=
file.am
index ece20cf..8257e5f 100644
--- a/MeshFiles/unittest/io/Makefile.am
+++ b/MeshFiles/unittest/io/Makefile.am
@@ -3,6 +3,7 @@ EXTRA_DIST =3D HommeMapping.nc \
              brick_cubit10.2.cub \
              brick_cubit10.cub \
              camEul26x48x96.t3.nc \
+             fv26x46x72.t.3.nc \
 	     cubtest12.cub \
 	     cubtest.jou \
 	     dum.sat \

diff --git a/MeshFiles/unittest/io/fv26x46x72.t.3.nc b/MeshFiles/unittest/i=
o/fv26x46x72.t.3.nc
new file mode 100644
index 0000000..15ba5b4
Binary files /dev/null and b/MeshFiles/unittest/io/fv26x46x72.t.3.nc differ

diff --git a/config/compiler.m4 b/config/compiler.m4
index ce00d10..6a7e2ed 100644
--- a/config/compiler.m4
+++ b/config/compiler.m4
@@ -102,15 +102,27 @@ if test "xno" !=3D "x$WITH_MPI"; then
   DISTCHECK_CONFIGURE_FLAGS=3D"$DISTCHECK_CONFIGURE_FLAGS --with-mpi=3D\"$=
{withval}\""
  =20
   if test "xyes" =3D=3D "x$WITH_MPI"; then
+    if test "xno" !=3D "x$CHECK_CC"; then =20
     FATHOM_SET_MPI_COMPILER([CC],  [$CC_LIST])
+    fi
+    if test "xno" !=3D "x$CHECK_CXX"; then =20
     FATHOM_SET_MPI_COMPILER([CXX],[$CXX_LIST])
-    FATHOM_SET_MPI_COMPILER([FC],  [$FC_LIST])
-    FATHOM_SET_MPI_COMPILER([F77],[$F77_LIST])
+    fi
+    if test "xno" !=3D "x$CHECK_FC"; then =20
+      FATHOM_SET_MPI_COMPILER([FC],  [$FC_LIST])
+      FATHOM_SET_MPI_COMPILER([F77],[$F77_LIST])
+    fi
   else
+    if test "xno" !=3D "x$CHECK_CC"; then =20
     FATHOM_SET_MPI_COMPILER([CC],  [$CC_LIST],[${WITH_MPI}/bin])
+    fi
+    if test "xno" !=3D "x$CHECK_CXX"; then =20
     FATHOM_SET_MPI_COMPILER([CXX],[$CXX_LIST],[${WITH_MPI}/bin])
-    FATHOM_SET_MPI_COMPILER([FC],  [$FC_LIST],[${WITH_MPI}/bin])
-    FATHOM_SET_MPI_COMPILER([F77],[$F77_LIST],[${WITH_MPI}/bin])
+    fi
+    if test "xno" !=3D "x$CHECK_FC"; then
+      FATHOM_SET_MPI_COMPILER([FC],  [$FC_LIST],[${WITH_MPI}/bin])
+      FATHOM_SET_MPI_COMPILER([F77],[$F77_LIST],[${WITH_MPI}/bin])
+    fi
     WITH_MPI=3Dyes
   fi
 fi
@@ -521,6 +533,12 @@ case "$cc_compiler:$host_cpu" in
     case "$target_vendor" in
       bgp)
         FATHOM_CC_32BIT=3D-q32
+	FATHOM_CC_64BIT=3D-q64
+	AR=3D"ar"
+	NM=3D"nm -B"
+        ;;
+      bgq)
+        FATHOM_CC_32BIT=3D-q32
         FATHOM_CC_64BIT=3D-q64
         AR=3D"ar"
         NM=3D"nm -B"

diff --git a/configure.ac b/configure.ac
index 66fcf81..130e4a9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -41,6 +41,7 @@ else
   AM_SILENT_RULES(no)
 fi
 ])
+AC_C_BIGENDIAN
=20
 # Check if platform is BlueGene
 AC_MSG_CHECKING([if platform is IBM BlueGene])
@@ -1163,6 +1164,8 @@ AC_SUBST([EXPORT_LTFLAGS])
 AC_SUBST([EXPORT_LDFLAGS])
 AC_SUBST([AM_CXXFLAGS])
 AC_SUBST([AM_CFLAGS])
+  AC_SUBST([AM_FCFLAGS])
+  AC_SUBST([AM_FFLAGS])
 AC_SUBST([DISTCHECK_CONFIGURE_FLAGS])
=20
 AC_ARG_VAR([FC], [FORTRAN compiler command])
@@ -1220,10 +1223,10 @@ AC_OUTPUT
 AC_MSG_RESULT([C:   $CC $CFLAGS $CPPFLAGS])
 AC_MSG_RESULT([C++: $CXX $CXXFLAGS $CPPFLAGS])
 if test "x" !=3D "x$FC"; then
-  AC_MSG_RESULT([Fortran: $FC $FCFLAGS])
+  AC_MSG_RESULT([Fortran90: $FC $FCFLAGS])
 fi
 if test "x" !=3D "x$F77"; then
-  AC_MSG_RESULT([Fortran: $F77 $FFLAGS])
+  AC_MSG_RESULT([Fortran77: $F77 $FFLAGS])
 fi
=20
 if test "x$HAVE_HDF5" =3D "xno"; then

diff --git a/doc/MOAB-UG.doc b/doc/MOAB-UG.doc
index 567a315..281d619 100644
Binary files a/doc/MOAB-UG.doc and b/doc/MOAB-UG.doc differ

diff --git a/doc/UG/moabUG.h b/doc/UG/moabUG.h
new file mode 100644
index 0000000..cea942d
--- /dev/null
+++ b/doc/UG/moabUG.h
@@ -0,0 +1,1064 @@
+/*! \page userguide User's Guide (MOAB 4.6)
+=20
+  \subpage team=20
+=20
+  \subpage contents
+=20
+  \subpage figures
+=20
+  \subpage tables
+=20
+  \subpage differences
+
+  \subpage building
+
+  \page team MOAB team members
+ <h2>The MOAB Team, including: </h2>
+=20
+ - Timothy J. Tautges (Argonne National Lab, Univ Wisconsin-Madison)=20
+ - Iulian Grindeanu (Argonne National Lab)=20
+ - Rajeev Jain (Argonne National Lab)
+ - Xiabing Xu (Argonne National Lab)
+
+
+ <h2>Emeritus members:</h2>
+=20
+ - Jason A. Kraftcheck
+ - Brandon M. Smith
+ - Hong-Jun Kim
+ - Jim Porter
+=20
+  \page contents Table of Contents
+=20
+  \ref introduction =20
+
+  \ref interface    =20
+
+     \ref twoone   =20
+
+     \ref twotwo    =20
+
+     \ref twothree      =20
+
+     \ref twofour  =20
+
+  \ref api    =20
+
+  \ref services     =20
+
+    \ref fourone   =20
+
+    \ref fourtwo  =20
+
+    \ref fourthree =20
+
+    \ref fourfour     =20
+
+    \ref fourfive   =20
+
+    \ref foursix =20
+
+  \ref parallel     =20
+
+    \ref fiveone   =20
+
+    \ref fivetwo    =20
+
+    \ref fivethree   =20
+
+    \ref fivefour     =20
+
+  \ref applications  =20
+
+  \ref implementation        =20
+
+  \ref representation    =20
+
+  \ref element   =20
+
+    \ref nineone =20
+
+    \ref ninetwo       =20
+
+    \ref ninethree     =20
+
+  \ref performance  =20
+
+  \ref conclusions   =20
+
+  \ref references=20
+
+  \section introduction 1. Introduction
+
+In scientific computing, systems of partial differential equations (PDEs) =
are solved on computers.  One of the most widely used methods to solve PDEs=
 numerically is to solve over discrete neighborhoods or =E2=80=9Celements=
=E2=80=9D of the domain.  Popular discretization methods include Finite Dif=
ference (FD), Finite Element (FE), and Finite Volume (FV).  These methods r=
equire the decomposition of the domain into a discretized representation, w=
hich is referred to as a =E2=80=9Cmesh=E2=80=9D.  The mesh is one of the fu=
ndamental types of data linking the various tools in the analysis process (=
mesh generation, analysis, visualization, etc.).  Thus, the representation =
of mesh data and operations on those data play a very important role in PDE=
-based simulations.
+=20
+MOAB is a component for representing and evaluating mesh data.  MOAB can s=
tore structured and unstructured mesh, consisting of elements in the finite=
 element =E2=80=9Czoo=E2=80=9D, along with polygons and polyhedra.  The fun=
ctional interface to MOAB is simple, consisting of only four fundamental da=
ta types.  This data is quite powerful, allowing the representation of most=
 types of metadata commonly found on the mesh.  Internally MOAB uses array-=
based storage for fine-grained data, which in many cases provides more effi=
cient access, especially for large portions of mesh and associated data.  M=
OAB is optimized for efficiency in space and time, based on access to mesh =
in chunks rather than through individual entities, while also versatile eno=
ugh to support individual entity access.
+
+The MOAB data model consists of the following four fundamental types: mesh=
 interface instance, mesh entities (vertex, edge, tri, etc.), sets, and tag=
s.  Entities are addressed through handles rather than pointers, to allow t=
he underlying representation of an entity to change without changing the ha=
ndle to that entity.  Sets are arbitrary groupings of mesh entities and oth=
er sets.  Sets also support parent/child relationships as a relation distin=
ct from sets containing other sets.  The directed graph provided by set par=
ent/child relationships is useful for embedding graphs whose nodes include =
collections of mesh entities; this approach has been used to represent a wi=
de variety of application-specific data, including geometric model topology=
, processor partitions, and various types of search trees.  Tags are named =
data which can be assigned to the mesh as a whole, individual entities, or =
sets.  Tags are a mechanism for attaching data to individual entities, and =
sets are a mechanism for describing relations between entities; the combina=
tion of these two mechanisms is a powerful yet simple interface for represe=
nting metadata or application-specific data.
+
+Various mesh-related tools are provided with MOAB or can be used directly =
with MOAB.  These tools can be used for mesh format translation (mbconvert)=
, mesh skinning (Skinner class), solution transfer between meshes (MBCouple=
r tool), ray tracing and other geometric searches (OrientedBoxTreeTool, Ada=
ptiveKDTree), visualization (vtkMOABReader tool), and relation between mesh=
 and geometric models (the separately-packed Lasso tool).  These tools are =
described later in this document.
+
+MOAB is written in the C++ programming language, with applications interac=
ting with MOAB mostly through its moab::Interface class.  All of the MOAB f=
unctions and classes are isolated in and accessed through the moab namespac=
e<sup>1</sup>. The remainder of this report gives class and function names =
without the =E2=80=9Cmoab::=E2=80=9D namespace qualification; unless otherw=
ise noted, the namespace qualifier should be added to all class and functio=
n names referenced here.  MOAB also implements the iMesh interface, which i=
s specified in C but can be called directly from other languages.  Almost a=
ll of the functionality in MOAB can be accessed through the iMesh interface=
.  MOAB is developed and supported on the Linux and MacOS operating systems=
, as well as various HPC operating systems.  MOAB can be used on parallel c=
omputing systems as well, including both clusters and high-end parallel sys=
tems like IBM BG/P and Cray systems.  MOAB is released under a standard LGP=
L open source software license.
+
+MOAB is used in several ways in various applications.  MOAB serves as the =
underlying mesh data representation in several scientific computing applica=
tions [1].  MOAB can also be used as a mesh format translator, using reader=
s and writers included in MOAB.  MOAB has also been used as a bridge to cou=
ple results in multi-physics analysis and to link these applications with o=
ther mesh services [2].
+
+The remainder of this report is organized as follows.  Section 2, =E2=80=
=9CGetting Started=E2=80=9D, provides a few simple examples of using MOAB t=
o perform simple tasks on a mesh.  Section 3 discusses the MOAB data model =
in more detail, including some aspects of the implementation.  Section 4 su=
mmarizes the MOAB function API.  Section 5 describes some of the tools incl=
uded with MOAB, and the implementation of mesh readers/writers for MOAB.  S=
ection 6 describes how to build MOAB-based applications.  Section 7 contain=
s a brief description of MOAB=E2=80=99s relation to the iMesh mesh interfac=
e.  Sections 8 and 9 discuss MOAB's representations of structured and spect=
ral element meshes, respectively.  Section 10 gives helpful hints for acces=
sing MOAB in an efficient manner from applications.  Section 11 gives a con=
clusion and future plans for MOAB development.  Section 12 gives references=
 cited in this report.
+
+Several other sources of information about MOAB may also be of interest to=
 readers.  Meta-data conventions define how sets and /or tags are used toge=
ther to represent various commonly-used simulation constructs; conventions =
used by MOAB are described in Ref [4], which is also included in the MOAB s=
ource distribution.  This document is maintained separately from this docum=
ent, since it is expected to change over time.  The MOAB project maintains =
a wiki [5], which links to most MOAB-related information.  MOAB also uses s=
everal mailing lists [6],[7] for MOAB-related discussions.  Potential users=
 are encouraged to interact with the MOAB team using these mailing lists.
+
+<sup>1</sup> Non-namespaced names are also provided for backward compatibi=
lity, with the =E2=80=9CMB=E2=80=9D prefix added to the class or variable n=
ame.
+
+ \ref contents
+
+ \section interface 2. MOAB Data Model
+The MOAB data model describes the basic types used in MOAB and the languag=
e used to communicate that data to applications.  This chapter describes th=
at data model, along with some of the reasons for some of the design choice=
s in MOAB.
+
+ \ref contents
+
+ \subsection twoone 2.1. MOAB Interface
+MOAB is written in C++.  The primary interface with applications is throug=
h member functions of the abstract base class Interface.  The MOAB library =
is created by instantiating Core, which implements the Interface API.  Mult=
iple instances of MOAB can exist concurrently in the same application; mesh=
 entities are not shared between these instancesi<sup>2</sup>.  MOAB is mos=
t easily viewed as a database of mesh objects accessed through the instance=
.  No other assumptions explicitly made about the nature of the mesh stored=
 there; for example, there is no fundamental requirement that elements fill=
 space or do not overlap each other geometrically.
+=20
+<sup>2</sup> One exception to this statement is when the parallel interfac=
e to MOAB is used; in this case, entity sharing between instances is handle=
d explicitly using message passing.  This is described in more detail in Se=
ction 5 of this document.
+
+ \ref contents
+
+ \subsection twotwo 2.2. Mesh Entities
+MOAB represents the following topological mesh entities: vertex, edge, tri=
angle, quadrilateral, polygon, tetrahedron, pyramid, prism, knife, hexahedr=
on, polyhedron.  MOAB uses the EntityType enumeration to refer to these ent=
ity types (see Table 1).  This enumeration has several special characterist=
ics, chosen intentionally: the types begin with vertex, entity types are gr=
ouped by topological dimension, with lower-dimensional entities appearing b=
efore higher dimensions; the enumeration includes an entity type for sets (=
described in the next section); and MBMAXTYPE is included at the end of thi=
s enumeration, and can be used to terminate loops over type.  In addition t=
o these defined values, the an increment operator (++) is defined such that=
 variables of type EntityType can be used as iterators in loops.
+MOAB refers to entities using =E2=80=9Chandles=E2=80=9D.  Handles are impl=
emented as long integer data types, with the four highest-order bits used t=
o store the entity type (mesh vertex, edge, tri, etc.) and the remaining bi=
ts storing the entity id.  This scheme is convenient for applications becau=
se:
+- Handles sort lexicographically by type and dimension; this can be useful=
 for grouping and iterating over entities by type.
+- The type of an entity is indicated by the handle itself, without needing=
 to call a function.
+- Entities allocated in sequence will typically have contiguous handles; t=
his characteristic can be used to efficiently store and operate on large li=
sts of handles.
+.
+
+This handle implementation is exposed to applications intentionally, becau=
se of optimizations that it enables, and is unlikely to change in future ve=
rsions.
+
+  \subsection tableone Table 1: Values defined for the EntityType enumerat=
ed type.
+<table border=3D"1">
+<tr>
+<td>MBVERTEX =3D 0</td>
+<td>MBPRISM</td>
+</tr>
+<tr>
+<td>MBEDGE</td>
+<td>MBKNIFE</td>
+</tr>
+<tr>
+<td>MBTRI</td>
+<td>MBHEX</td>
+</tr>
+<tr>
+<td>MBQUAD</td>
+<td>MBPOLYHEDRON</td>
+</tr>
+<tr>
+<td>MBPOLYGON</td>
+<td>MBENTITYSET</td>
+</tr>
+<tr>
+<td>MBTET</td>
+<td>MBMAXTYPE</td>
+</tr>
+<tr>
+<td>MBPYRAMID</td>
+<td></td>
+</tr>
+</table>
+
+MOAB defines a special class for storing lists of entity handles, named Ra=
nge.  This class stores handles as a series of (start_handle, end_handle) s=
ubrange tuples.  If a list of handles has large contiguous ranges, it can b=
e represented in almost constant size using Range.  Since entities are typi=
cally created in groups, e.g. during mesh generation or file import, a high=
 degree of contiguity in handle space is typical.  Range provides an interf=
ace similar to C++ STL containers like std::vector, containing iterator dat=
a types and functions for initializing and iterating over entity handles st=
ored in the range.  Range also provides functions for efficient Boolean ope=
rations like subtraction and intersection.  Most API functions in MOAB come=
 in both range-based and vector-based variants.  By definition, a list of e=
ntities stored in an Range is always sorted, and can contain a given entity=
 handle only once.  Range cannot store the handle 0 (zero).
+
+Typical usage of an Range object would look like:
+
+\code
+using namespace moab;
+   int my_function(Range &from_range) {
+          int num_in_range =3D from_range.size();
+          Range to_range;
+          Range::iterator rit;
+    for (rit =3D from_range.begin(); rit !=3D from_range.end(); rit++) {
+            EntityHandle this_ent =3D *rit;
+            to_range.insert(this_ent);
+          }
+        }
+\endcode
+
+Here, the range is iterated similar to how std::vector is iterated.
+
+  \ref contents
+
+ \subsection adjacencies 2.2.1. Adjacencies & AEntities=20
+
+The term adjacencies is used to refer to those entities topologically conn=
ected to a given entity, e.g. the faces bounded by a given edge or the vert=
ices bounding a given region.  The same term is used for both higher-dimens=
ional (or bounded) and lower-dimensional (or bounding) adjacent entities.  =
MOAB provides functions for querying adjacent entities by target dimension,=
 using the same functions for higher- and lower-dimension adjacencies.  By =
default, MOAB stores the minimum data necessary to recover adjacencies betw=
een entities.  When a mesh is initially loaded into MOAB, only entity-verte=
x (i.e. =E2=80=9Cdownward=E2=80=9D) adjacencies are stored, in the form of =
entity connectivity.  When =E2=80=9Cupward=E2=80=9D adjacencies are request=
ed for the first time, e.g. from vertices to regions, MOAB stores all verte=
x-entity adjacencies explicitly, for all entities in the mesh.  Non-vertex =
entity to entity adjacencies are never stored, unless explicitly requested =
by the application.
+
+In its most fundamental form, a mesh need only be represented by its verti=
ces and the entities of maximal topological dimension.  For example, a hexa=
hedral mesh can be represented as the connectivity of the hex elements and =
the vertices forming the hexes.  Edges and faces in a 3D mesh need not be e=
xplicitly represented.  We refer to such entities as =E2=80=9CAEntities=E2=
=80=9D, where =E2=80=98A=E2=80=99 refers to =E2=80=9CAuxiliary=E2=80=9D, =
=E2=80=9CAncillary=E2=80=9D, and a number of other words mostly beginning w=
ith =E2=80=98A=E2=80=99.  Individual AEntities are created only when reques=
ted by applications, either using mesh modification functions or by request=
ing adjacencies with a special =E2=80=9Ccreate if missing=E2=80=9D flag pas=
sed as =E2=80=9Ctrue=E2=80=9D.  This reduces the overall memory usage when =
representing large meshes.  Note entities must be explicitly represented be=
fore they can be assigned tag values or added to entity sets (described in =
following Sections).
+
+\ref contents
+
+ \subsection twothree 2.3. Entity Sets
+Entity sets are used to store arbitrary collections of entities and other =
sets.  Sets are used for a variety of things in mesh-based applications, fr=
om the set of entities discretizing a given geometric model entity to the e=
ntities partitioned to a specific processor in a parallel finite element ap=
plication.  MOAB entity sets can also store parent/child relations with oth=
er entity sets, with these relations distinct from contains relations.  Par=
ent/child relations are useful for building directed graphs with graph node=
s representing collections of mesh entities; this construct can be used, fo=
r example, to represent an interface of mesh faces shared by two distinct c=
ollections of mesh regions.  MOAB also defines one special set, the =E2=80=
=9Croot set=E2=80=9D or the interface itself; all entities are part of this=
 set by definition.  Defining a root set allows the use of a single set of =
MOAB API functions to query entities in the overall mesh as well as its sub=
sets.
+
+MOAB entity sets can be one of two distinct types: list-type entity sets p=
reserve the order in which entities are added to the set, and can store a g=
iven entity handle multiple times in the same set; set-type sets are always=
 ordered by handle, regardless of the order of addition to the set, and can=
 store a given entity handle only once.  This characteristic is assigned wh=
en the set is created, and cannot be changed during the set=E2=80=99s lifet=
ime.
+
+MOAB provides the option to track or not track entities in a set.  When en=
tities (and sets) are deleted by other operations in MOAB, they will also b=
e removed from containing sets for which tracking has been enabled.  This b=
ehavior is assigned when the set is created, and cannot be changed during t=
he set=E2=80=99s lifetime.  The cost of turning tracking on for a given set=
 is sizeof(EntityHandle) for each entity added to the set; MOAB stores cont=
aining sets in the same list which stores adjacencies to other entities.
+
+Using an entity set looks like the following:
+\code
+using namespace moab;
+// load a file using MOAB, putting the loaded mesh into a file set
+EntityHandle file_set;
+ErrorCode rval =3D moab->create_meshset(MESHSET_SET, file_set);
+rval =3D moab->load_file(=E2=80=9Cfname.vtk=E2=80=9D, &file_set);
+Range set_ents;
+// get all the 3D entities in the set
+rval =3D moab->get_entities_by_dimension(file_set, 3, set_ents);
+\endcode
+
+Entity sets are often used in conjunction with tags (described in the next=
 section), and provide a powerful mechanism to store a variety of meta-data=
 with meshes.
+
+\ref contents
+
+ \subsection twofour 2.4. Tags=20
+
+Applications of a mesh database often need to attach data to mesh entities=
.  The types of attached data are often not known at compile time, and can =
vary across individual entities and entity types.  MOAB refers to this atta=
ched data as a =E2=80=9Ctag=E2=80=9D.  Tags can be thought of loosely as a =
variable, which can be given a distinct value for individual entities, enti=
ty sets, or for the interface itself.  A tag is referenced using a handle, =
similarly to how entities are referenced in MOAB.  Each MOAB tag has the fo=
llowing characteristics, which can be queried through the MOAB interface:
+- Name
+- Size (in bytes)
+- Storage type
+- Data type (integer, double, opaque, entity handle)
+- Handle
+.
+
+The storage type determines how tag values are stored on entities. =20
+
+- Dense: Dense tag values are stored in arrays which match arrays of conti=
guous entity handles.  Dense tags are more efficient in both storage and me=
mory if large numbers of entities are assigned the same tag.  Storage for a=
 given dense tag is not allocated until a tag value is set on an entity; me=
mory for a given dense tag is allocated for all entities in a given sequenc=
e at the same time.
+- Sparse: Sparse tags are stored as a list of (entity handle, tag value) t=
uples, one list per sparse tag, sorted by entity handle.
+- Bit: Bit tags are stored similarly to dense tags, but with special handl=
ing to allow allocation in bit-size amounts per entity.
+.
+
+MOAB also supports variable-length tags, which can have a different length=
 for each entity they are assigned to.  Variable length tags are stored sim=
ilarly to sparse tags.
+
+The data type of a tag can either be one understood at compile time (integ=
er, double, entity handle), in which case the tag value can be saved and re=
stored properly to/from files and between computers of different architectu=
re (MOAB provides a native HDF5-based save/restore format for this purpose;=
 see Section 4.6).  The opaque data type is used for character strings, or =
for allocating =E2=80=9Craw memory=E2=80=9D for use by applications (e.g. f=
or storage application-defined structures or other abstract data types).  T=
hese tags are saved and restored as raw memory, with no special handling fo=
r endian or precision differences.
+
+An application would use the following code to attach a double-precision t=
ag to vertices in a mesh, e.g. to assign a temperature field to those verti=
ces:
+
+\code
+using namespace moab;
+// load a file using MOAB and get the vertices
+ErrorCode rval =3D moab->load_file(=E2=80=9Cfname.vtk=E2=80=9D);
+Range verts;
+rval =3D moab->get_entities_by_dimension(0, 0, verts);
+// create a tag called =E2=80=9CTEMPERATURE=E2=80=9D
+Tag temperature;
+double def_val =3D -1.0d-300, new_val =3D 273.0;
+rval =3D moab->tag_create(=E2=80=9CTEMPERATURE=E2=80=9D, sizeof(double), M=
B_TAG_DENSE,=20
+                        MB_TYPE_DOUBLE, temperature, &def_val);
+// assign a value to vertices
+for (Range::iterator vit =3D verts.begin();=20
+     vit !=3D verts.end(); vit++)=20
+  rval =3D moab->tag_set_data(temperature, &(*rit), 1, &new_val);
+
+\endcode
+
+The semantic meaning of a tag is determined by applications using it.  How=
ever, to promote interoperability between applications, there are a number =
of tag names reserved by MOAB which are intended to be used by convention. =
 Mesh readers and writers in MOAB use these tag conventions, and applicatio=
ns can use them as well to access the same data. Ref. [4] maintains an up-t=
o-date list of conventions for meta-data usage in MOAB.
+
+  \ref contents
+
+  \section api 3. MOAB API Design Philosophy and Summary
+
+This section describes the design philosophy behind MOAB, and summarizes t=
he functions, data types and enumerated variables in the MOAB API.  A compl=
ete description of the MOAB API is available in online documentation in the=
 MOAB distribution [8].
+
+MOAB is designed to operate efficiently on collections of entities.  Entit=
ies are often created or referenced in groups (e.g. the mesh faces discreti=
zing a given geometric face, the 3D elements read from a file), with those =
groups having some form of temporal or spatial locality.  The interface pro=
vides special mechanisms for reading data directly into the native storage =
used in MOAB, and for writing large collections of entities directly from t=
hat storage, to avoid data copies.  MOAB applications structured to take ad=
vantage of that locality will typically operate more efficiently.
+
+MOAB has been designed to maximize the flexibility of mesh data which can =
be represented.  There is no explicit constraint on the geometric structure=
 of meshes represented in MOAB, or on the connectivity between elements.  I=
n particular, MOAB allows the representation of multiple entities with the =
same exact connectivity; however, in these cases, explicit adjacencies must=
 be used to distinguish adjacencies with AEntities bounding such entities.
+
+The number of vertices used to represent a given topological entity can va=
ry, depending on analysis needs; this is often the case in FEA.  For exampl=
e, applications often use =E2=80=9Cquadratic=E2=80=9D or 10-vertex tetrahed=
ral, with vertices at edge midpoints as well as corners.  MOAB does not dis=
tinguish these variants by entity type, referring to all variants as =E2=80=
=9Ctetrahedra=E2=80=9D.  The number of vertices for a given entity is used =
to distinguish the variants, with canonical numbering conventions used to d=
etermine placement of the vertices [9].  This is similar to how such variat=
ions are represented in the Exodus [10] and Patran [11] file formats.  In p=
ractice, we find that this simplifies coding in applications, since in many=
 cases the handling of entities depends only on the number of corner vertic=
es in the element.  Some MOAB API functions provide a flag which determines=
 whether corner or all vertices are requested.
+
+The MOAB API is designed to balance complexity and ease of use.  This bala=
nce is evident in the following general design characteristics:
+
+- Entity lists: Lists of entities are passed to and from MOAB in a variety=
 of forms.  Lists output from MOAB are passed as either STL vector or Range=
 data types.  Either of these constructs may be more efficient in both time=
 and memory, depending on the semantics of the data being requested.  Input=
 lists are passed as either Range=E2=80=99s, or as a pointer to EntityHandl=
e and a size.  The latter allows the same function to be used when passing =
individual entities, without requiring construction of an otherwise unneede=
d STL vector.
+- Entity sets: Most query functions accept an entity set as input.  Applic=
ations can pass zero to indicate a request for the whole interface.  Note t=
hat this convention applies only to query functions; attempts to add or sub=
tract entities to/from the interface using set-based modification functions=
, or to add parents or children to the interface set, will fail.  Allowing =
specification of the interface set in this manner avoids the need for a sep=
arate set of API functions to query the database as a whole.
+- Implicit Booleans in output lists: A number of query functions in MOAB a=
llow specification of a Boolean operation (Interface::INTERSECT or Interfac=
e::UNION).  This operation is applied to the results of the query, often el=
iminating the need for code the application would need to otherwise impleme=
nt.  For example, to find the set of vertices shared by a collection of qua=
drilaterals, the application would pass that list of quadrilaterals to a re=
quest for vertex adjacencies, with Interface::INTERSECT passed for the Bool=
ean flag.  The list of vertices returned would be the same as if the applic=
ation called that function for each individual entity, and computed the int=
ersection of the results over all the quadrilaterals.  Applications may als=
o input non-empty lists to store the results, in which case the intersectio=
n is also performed with entities already in the list.  In many cases, this=
 allows optimizations in both time and memory inside the MOAB implementatio=
n.=20
+.
+
+Since these objectives are at odds with each other, tradeoffs had to be ma=
de between them.  Some specific issues that came up are:
+
+- Using ranges: Where possible, entities can be referenced using either ra=
nges (which allow efficient storage of long lists) or STL vectors (which al=
low list order to be preserved), in both input and output arguments.
+- Entities in sets: Accessing the entities in a set is done using the same=
 functions which access entities in the entire mesh.  The whole mesh is ref=
erenced by specifying a set handle of zero<sup>3</sup>.
+- Entity vectors on input: Functions which could normally take a single en=
tity as input are specified to take a vector of handles instead.  Single en=
tities are specified by taking the address of that entity handle and specif=
ying a list length of one.  This minimizes the number of functions, while p=
reserving the ability to input single entities.<sup>4</sup>
+.
+
+Table 2 lists basic data types and enumerated variables defined and used b=
y MOAB.  Values of the ErrorCode enumeration are returned from most MOAB fu=
nctions, and can be compared to those listed in Appendix [ref-appendix].
+
+MOAB uses several pre-defined tag names to define data commonly found in v=
arious mesh-based analyses.  Ref. [4] describes these meta-data conventions=
 in more detail.  These conventions will be added to as new conventions eme=
rge for using sets and tags in MOAB applications.
+
+  \subsection tabletwo Table 2: Basic data types and enums defined in MOAB.
+
+<table border=3D"1">
+<tr>
+<th>Enum / Type</th>
+<th>Description</th>
+</tr>
+<tr>
+<td>ErrorCode</td>
+<td>Specific error codes returned from MOAB</td>
+</tr>
+<tr>
+<td>EntityHandle</td>
+<td>Type used to represent entity handles</td>
+</tr>
+<tr>
+<td>Tag</td>
+<td>Type used to represent tag handles</td>
+</tr>
+<tr>
+<td>TagType</td>
+<td>Type used to represent tag storage type</td>
+</tr>
+<tr>
+<td>DataType</td>
+<td>Type used to represent tag data type</td>
+</tr>
+</table>
+
+Table 3 lists the various groups of functions that comprise the MOAB API. =
 This is listed here strictly as a reference to the various types of functi=
onality supported by MOAB; for a more detailed description of the scope and=
 syntax of the MOAB API, see the online documentation [8].
+
+  \subsection tablethree Table 3: Groups of functions in MOAB API.  See Re=
f. [8] for more details.
+
+<table border=3D"1">
+<tr>
+<th>Function group</th>
+<th>Examples</th>
+<th>Description</th>
+</tr>
+<tr>
+<td>Constructor, destructor, interface</td>
+<td>Interface, ~Core, query_interface</td>
+<td>Construct/destroy interface; get pointer to read/write interface</td>
+</tr>
+<tr>
+<td>Entity query</td>
+<td>get_entities_by_dimension, get_entities_by_handle</td>
+<td>Get entities by dimension, type, etc.</td>
+</tr>
+<tr>
+<td>Adjacencies</td>
+<td>get_adjacencies, set_adjacencies, add_adjacencies</td>
+<td>Get topologically adjacent entities; set or add explicit adjacencies</=
td>
+</tr>
+<tr>
+<td>Vertex coordinates</td>
+<td>get_coords, set_coords</td>
+<td>Get/set vertex coordinates</td>
+</tr>
+<tr>
+<td>Connectivity</td>
+<td>get_connectivity, set_connectivity</td>
+<td>Get/set connectivity of non-vertex entities</td>
+</tr>
+<tr>
+<td>Sets</td>
+<td>create_meshset, add_entities, add_parent_child</td>
+<td>Create and work with entity sets</td>
+</tr>
+<tr>
+<td>Tags</td>
+<td>tag_get_data, tag_create</td>
+<td>Create, read, write tag data</td>
+</tr>
+<tr>
+<td>Handles</td>
+<td>type_from_handle, id_from_handle</td>
+<td>Go between handles and types/ids</td>
+</tr>
+<tr>
+<td>File handling</td>
+<td>load_mesh, save_mesh</td>
+<td>Read/write mesh files</td>
+</tr>
+<tr>
+<td>Geometric dimension</td>
+<td>get_dimension, set_dimension</td>
+<td>Get/set geometric dimension of mesh</td>
+</tr>
+<tr>
+<td>Mesh modification</td>
+<td>create_vertex, delete_entity</td>
+<td>Create or delete mesh entities</td>
+</tr>
+<tr>
+<td>Information</td>
+<td>list_entities, get_last_error</td>
+<td>Get or print certain information</td>
+</tr>
+<tr>
+<td>High-order nodes</td>
+<td>high_order_node</td>
+<td>Get information on high-order nodes</td>
+</tr>
+<tr>
+<td>Canonical numbering</td>
+<td>side_number</td>
+<td>Get canonical numbering information</td>
+</tr>
+</table>
+
+<sup>3</sup>In iMesh, the whole mesh is specified by a special entity set =
handle, referred to as the =E2=80=9Croot set=E2=80=9D.
+
+<sup>4</sup>Note that STL vectors of entity handles can be input in this m=
anner by using &vector[0] and vector.size() for the 1d vector address and s=
ize, respectively.
+
+ \ref contents
+
+ \section services 4. Related Mesh Services
+
+A number of mesh-based services are often used in conjunction with a mesh =
library.  For example, parallel applications often need to visualize the me=
sh and associated data.  Other services, like spatial interpolation or find=
ing the faces on the =E2=80=9Cskin=E2=80=9D of a 3D mesh, can be implemente=
d more efficiently using knowledge of specific data structures in MOAB.  Se=
veral of these services provided with MOAB are described in this chapter.
+
+ \ref contents
+
+  \subsection fourone 4.1. Visualization
+
+Visualization is one of the most common needs associated with meshes.  The=
 primary tool used to visualize MOAB meshes is VisIt [12].  Users can speci=
fy that VisIt read mesh directly out of the MOAB instance, by specifying th=
e ITAPS-MOABC mesh format and a file readable by MOAB (see xxx).
+
+There are some initial capabilities in VisIt for limited viewing and manip=
ulation of tag data and some types of entity sets.  Tag data is visualized =
using the same mechanisms used to view other field data in VisIt, e.g. usin=
g a pseudocolor plot; sets are viewed using VisIt=E2=80=99s SIL window, acc=
essed by selecting the SIL icon in the data selection window.  xxx shows a =
vertex-based radiation temperature field computed by the Cooper rad-hydro c=
ode [1] for a subset of geometric volumes in a mesh.  =20
+
+Reorganization of VisIt=E2=80=99s set handling is also underway, to increa=
se versatility and flexibility of this important mechanism.
+
+ \ref contents
+
+  \subsection fourtwo 4.2. Parallel Decomposition
+
+To support parallel simulation, applications often need to partition a mes=
h into parts, designed to balance the load and minimize communication betwe=
en sets.  MOAB includes the MBZoltan tool for this purpose, constructed on =
the well-known Zoltan partitioning library [13].  After computing the parti=
tion using Zoltan, MBZoltan stores the partition as either tags on individu=
al entities in the partition, or as tagged sets, one set per part.  Since a=
 partition often exhibits locality similar to how the entities were created=
, storing it as sets (based on Range=E2=80=99s) is often more memory-effici=
ent than an entity tag-based representation.  Xxx shows a partition compute=
d with MBZoltan (and visualized in VisIt).=20
+
+ \ref contents
+
+  \subsection fourthree 4.3. Skinner
+
+An operation commonly applied to mesh is to compute the outermost =E2=80=
=9Cskin=E2=80=9D bounding a contiguous block of elements.  This skin consis=
ts of elements of one fewer topological dimension, arranged in one or more =
topological balls on the boundary of the elements.  The Skinner tool comput=
es the skin of a mesh in a memory-efficient manner.  Skinner uses knowledge=
 about whether vertex-entity adjacencies and AEntities exist to minimize me=
mory requirements and searching time required during the skinning process. =
 This skin can be provided as a single collection of entities, or as sets o=
f entities distinguished by forward and reverse orientation with respect to=
 higher-dimensional entities in the set being skinned.
+
+The following code fragment shows how Skinner can be used to compute the s=
kin of a range of hex elements:
+
+ \code
+using namespace moab;
+Range hexes, faces;
+ErrorCode rval =3D moab->get_entities_by_dimension(0, 3, hexes);
+Skinner myskinner(moab);
+bool verts_too =3D false;
+ErrorCode rval =3D myskinner.find_skin(hexes, verts_too, faces);
+\endcode
+
+Skinner can also skin a mesh based on geometric topology groupings importe=
d with the mesh.  The geometric topology groupings contain information abou=
t the mesh =E2=80=9Cowned=E2=80=9D by each of the entities in the geometric=
 model, e.g. the model vertices, edges, etc.  Links between the mesh sets c=
orresponding to those entities can be inferred directly from the mesh.  Ski=
nning a mesh this way will typically be much faster than doing so on the ac=
tual mesh elements, because there is no need to create and destroy interior=
 faces on the mesh.
+
+ \ref contents
+
+  \subsection fourfour 4.4. Tree Decompositions
+
+MOAB provides several mechanisms for spatial decomposition and searching i=
n a mesh:
+
+- AdaptiveKDTree: Adaptive KD tree, a space-filling decomposition with axi=
s-aligned splitting planes, enabling fast searching.
+- BSPTree: Binary Space Partition tree, with non-axis-aligned partitions, =
for fast spatial searches with slightly better memory efficiency than KD tr=
ees.
+- OrientedBoxTreeTool: Oriented Bounding Box tree hierarchy, useful for fa=
st ray-tracing on collections of mesh facets.
+.
+
+These trees have various space and time searching efficiencies.  All are i=
mplemented based on entity sets and parent/child relations between those se=
ts, allowing storage of a tree decomposition using MOAB=E2=80=99s native fi=
le storage mechanism (see Section 4.6.1).  MOAB=E2=80=99s entity set implem=
entation is specialized for memory efficiency when representing binary tree=
s.  Tree decompositions in MOAB have been used to implement fast ray tracin=
g to support radiation transport [14], solution coupling between meshes [2]=
, and embedded boundary mesh generation [15].  MOAB also includes the DAGMC=
 tool, supporting Monte Carlo radiation transport.
+
+The following code fragment shows very basic use of AdaptiveKDTree.  A ran=
ge of entities is put in the tree; the leaf containing a given point is fou=
nd, and the entities in that leaf are returned.
+\code
+using namespace moab;
+// create the adaptive kd tree from a range of tets
+EntityHandle tree_root
+AdaptiveKDTree myTree(moab);
+ErrorCode rval =3D myTree.build_tree(tets, tree_root);
+
+// get the overall bounding box corners
+double boxmax[3], boxmin;
+rval =3D myTree.get_tree_box(tree_root, boxmax, boxmin);
+
+// get the tree leaf containing point xyz, and the tets in that leaf
+AdaptiveKDTreeIter treeiter;
+rval =3D myTree.leaf_containing_point(tree_root, xyz, treeiter);
+Range leaf_tets;
+rval =3D moab->get_entities_by_dimension(treeiter.handle(), 3,=20
+                                       leaf_tets, false);
+\endcode
+
+More detailed examples of using the various tree decompositions in MOAB ca=
n be found in [ref-treeexamples].
+
+ \ref contents
+
+  \subsection fourfive 4.5. File Reader/Writer Interfaces
+
+Mesh readers and writers communicate mesh into/out of MOAB from/to disk fi=
les.  Reading a mesh often involves importing large sets of data, for examp=
le coordinates of all the nodes in the mesh.  Normally, this process would =
involve reading data from the file into a temporary data buffer, then copyi=
ng data from there into its destination in MOAB.  To avoid the expense of c=
opying data, MOAB has implemented a reader/writer interface that provides d=
irect access to blocks of memory used to represent mesh.
+
+The reader interface, declared in ReadUtilIface, is used to request blocks=
 of memory for storing coordinate positions and element connectivity.  The =
pointers returned from these functions point to the actual memory used to r=
epresent those data in MOAB.  Once data is written to that memory, no furth=
er copying is done.  This not only saves time, but it also eliminates the n=
eed to allocate a large memory buffer for intermediate storage of these dat=
a.=20
+
+MOAB allocates memory for nodes and elements (and their corresponding dens=
e tags) in chunks, to avoid frequent allocation/de-allocation of small chun=
ks of memory.  The chunk size used depends on from where the mesh is being =
created, and can strongly affect the performance (and memory layout) of MOA=
B.  Since dense tags are allocated at the chunk size, this can also affect =
overall memory usage in cases where the mesh size is small but the number o=
f dense tags or dense tag size is large.  When creating vertices and elemen=
ts through the normal MOAB API, default chunk sizes defined in the Sequence=
Manager class are used.  However, most of the file readers in MOAB allocate=
 the exact amount of space necessary to represent the mesh being read.  The=
re are also a few exceptions to this:
+
+- When compiled in parallel, this space is increased by a factor of 1.5, t=
o allow subsequent creation of ghost vertices/elements in the same chunk as=
 the original mesh.
+- The .cub file reader, which creates nodes and elements for individual ge=
ometric entities in separate calls, allocates using the default vertex/elem=
ent sequence sizes, which are defined in the SequenceManager class in MOAB.
+.
+
+Applications calling the reader interface functions directly can specify t=
he allocation chunk size as an optional parameter.
+
+The reader interface consists of the following functions:
+
+- get_node_arrays: Given the number of vertices requested, the number of g=
eometric dimensions, and a requested start id, allocates a block of vertex =
handles and returns pointers to coordinate arrays in memory, along with the=
 actual start handle for that block of vertices.
+- get_element_array: Given the number of elements requested, the number of=
 vertices per element, the element type and the requested start id, allocat=
es the block of elements, and returns a pointer to the connectivity array f=
or those elements and the actual start handle for that block.  The number o=
f vertices per element is necessary because those elements may include high=
er-order nodes, and MOAB stores these as part of the normal connectivity ar=
ray.
+- update_adjacencies: This function takes the start handle for a block of =
elements and the connectivity of those elements, and updates adjacencies fo=
r those elements.  Which adjacencies are updated depends on the options set=
 in AEntityFactory.
+.
+
+The following code fragment illustrates the use of ReadUtilIface to read a=
 mesh directly into MOAB=E2=80=99s native representation.  This code assume=
s that connectivity is specified in terms of vertex indices, with vertex in=
dices starting from 1.
+
+\code
+// get the read iface from moab
+ReadUtilIface *iface;
+ErrorCode rval =3D moab->get_interface("ReadUtilIface", &iface);
+
+// allocate a block of vertex handles and read xyz=E2=80=99s into them
+std::vector<double*> arrays;
+EntityHandle startv, *starth;
+rval =3D iface->get_node_arrays(3, num_nodes, 0, startv, arrays);
+for (int i =3D 0; i < num_nodes; i++)
+  infile >> arrays[0][i] >> arrays[1][i] >> arrays[2][i];
+
+// allocate block of hex handles and read connectivity into them
+rval =3D iface->get_element_array(num_hexes, 8, MBHEX, 0, starth);
+for (int i =3D 0; i < 8*num_hexes; i++)
+  infile >> starth[i];
+
+// change connectivity indices to vertex handles
+for (int i =3D 0; i < 8*num_hexes; i++)
+  starth[i] +=3D startv-1;
+\endcode
+
+The writer interface, declared in WriteUtilIface, provides functions that =
support writing vertex coordinates and element connectivity to storage loca=
tions input by the application.  Assembling these data is a common task for=
 writing mesh, and can be non-trivial when exporting only subsets of a mesh=
.  The writer interface declares the following functions:
+
+- get_node_arrays: Given already-allocated memory and the number of vertic=
es and dimensions, and a range of vertices, this function writes vertex coo=
rdinates to that memory.  If a tag is input, that tag is also written with =
integer vertex ids, starting with 1, corresponding to the order the vertice=
s appear in that sequence (these ids are used to write the connectivity arr=
ay in the form of vertex indices).
+- get_element_array: Given a range of elements and the tag holding vertex =
ids, and a pointer to memory, the connectivity of the specified elements ar=
e written to that memory, in terms of the indices referenced by the specifi=
ed tag.  Again, the number of vertices per element is input, to allow the d=
irect output of higher-order vertices.
+- gather_nodes_from_elements: Given a range of elements, this function ret=
urns the range of vertices used by those elements.  If a bit-type tag is in=
put, vertices returned are also marked with 0x1 using that tag.  If no tag =
is input, the implementation of this function uses its own bit tag for mark=
ing, to avoid using an n2 algorithm for gathering vertices.
+- reorder: Given a permutation vector, this function reorders the connecti=
vity for entities with specified type and number of vertices per entity to =
match that permutation.  This function is needed for writing connectivity i=
nto numbering systems other than that used internally in MOAB.
+.
+
+The following code fragment shows how to use WriteUtilIface to write the v=
ertex coordinates and connectivity indices for a subset of entities.
+
+\code
+using namespace moab;
+// get the write iface from moab
+WriteUtilIface *iface;
+ErrorCode rval =3D moab->get_interface("WriteUtilIface", &iface);
+
+// get all hexes the model, and choose the first 10 of those
+Range tmp_hexes, hexes, verts;
+rval =3D moab->get_entities_by_type(0, MBHEX, tmp_hexes);
+for (int i =3D 0; i < 10; i++) hexes.insert(tmp_hexes[i]);
+rval =3D iface->gather_nodes_from_elements(hexes, 0, verts);
+
+// assign vertex ids
+iface->assign_ids(verts, 0, 1);
+
+// allocate space for coordinates & write them
+std::vector<double*> arrays(3);
+for (int i =3D 0; i < 3; i++) arrays[i] =3D new double[verts.size()];
+iface->get_node_arrays(3, verts.size(), verts, 0, 1, arrays);
+
+// put connect=E2=80=99y in array, in the form of indices into vertex array
+std::vector<int> conn(8*hexes.size());
+iface->get_element_array(hexes.size(), 8, 0, hexes, 0, 1, &conn[0]);
+\endcode
+
+ \ref contents
+
+  \subsection foursix 4.6. File Readers/Writers Packaged With MOAB
+
+MOAB has been designed to efficiently represent data and metadata commonly=
 found in finite element mesh files.  Readers and writers are included with=
 MOAB which import/export specific types of metadata in terms of MOAB sets =
and tags, as described earlier in this document.  The number of readers and=
 writers in MOAB will probably grow over time, and so they are not enumerat=
ed here.  See the src/io/README file in the MOAB source distribution for a =
current list of supported formats.
+
+Because of its generic support for readers and writers, described in the p=
revious section, MOAB is also a good environment for constructing new mesh =
readers and writers.  The ReadTemplate and WriteTemplate classes in src/io =
are useful starting points for constructing new file readers/writers; appli=
cations are encouraged to submit their own readers/writers for inclusion in=
 MOAB=E2=80=99s contrib/io directory in the MOAB source.=20
+
+The usefulness of a file reader/writer is determined not only by its abili=
ty to read and write nodes and elements, but also in its ability to store t=
he various types of meta-data included with the typical mesh.  MOAB readers=
 and writers are distinguished by their ability to preserve meta-data in me=
shes that they read and write.  For example, MOAB=E2=80=99s CUB reader impo=
rts not only the mesh saved from CUBIT, but also the grouping of mesh entit=
ies into sets which reflect the geometric topology of the model used to gen=
erate the mesh.  See [4] for a more detailed description of meta-data conve=
ntions used in MOAB=E2=80=99s file readers and writers, and the individual =
file reader/writer header files in src/io for details about the specific re=
aders and writers.
+
+Three specific file readers in MOAB bear further discussion: MOAB=E2=80=99=
s native HDF5-based file reader/writer; the CUB reader, used to import mesh=
 and meta-data represented in CUBIT; and the CGM reader, which imports geom=
etric models.  These are described next.
+
+ \ref contents
+
+  \subsection native 4.6.1. Native HD5-Based Reader/Writer
+
+A mesh database must be able to save and restore the data stored in its da=
ta model, at least to the extent to which it understands the semantics of t=
hat data.  MOAB defines an HDF5-based file format that can store data embed=
ded in MOAB.  By convention, these files are given an =E2=80=9C.h5m=E2=80=
=9D file extension.  When reading or writing large amounts of data, it is r=
ecommended to use this file format, as it is the most complete and also the=
 most efficient of the file readers/writers in MOAB.=20
+
+  \subsection cub 4.6.2. CUB Reader
+
+CUBIT is a toolkit for generating tetrahedral and hexahedral finite elemen=
t meshes from solid model geometry [16].  This tool saves and restores data=
 in a custom =E2=80=9C.cub=E2=80=9D file, which stores both mesh and geomet=
ry (and data relating the two).  The CUB reader in MOAB can import and inte=
rpret much of the meta-data information saved in .cub files.  Ref. [4] desc=
ribes the conventions used to store this meta-data in the MOAB data model. =
 The information read from .cub files, and stored in the MOAB data model, i=
ncludes:
+
+- Geometric model entities and topology
+- Model entity names and ids
+- Groups, element blocks, nodesets, and sidesets, including model entities=
 stored in them
+- Mesh scheme and interval size information assigned to model entities
+.
+
+Note that although information about model entities is recovered, MOAB by =
default does not depend on a solid modeling engine; this information is sto=
red in the form of entity sets and parent/child relations between them.  Se=
e Ref. [4] for more information.
+
+ \ref contents
+
+  \subsection cgm 4.6.3. CGM Reader
+
+The Common Geometry Module (CGM) [17] is a library for representing solid =
model and other types of solid geometry data.  The CUBIT mesh generation to=
olkit uses CGM for its geometric modeling support, and CGM can restore geom=
etric models in the exact state in which they were represented in CUBIT.  M=
OAB contains a CGM reader, which can be enabled with a configure option.  U=
sing this reader, MOAB can read geometric models, and represent their model=
 topology using entity sets linked by parent/child relations.  The mesh in =
these models comes directly from the modeling engine faceting routines; the=
se are the same facets used to visualize solid models in other graphics eng=
ines.  When used in conjunction with the VisIt visualization tool (see Sect=
ion  ), this provides a solution for visualizing geometric models.  Xxx sho=
ws a model imported using MOAB=E2=80=99s CGM reader and visualized with Vis=
It.=20
+
+ \ref contents
+
+  \section parallel 5. Parallel Mesh Representation and Query
+
+A parallel mesh representation must strike a careful balance between provi=
ding an interface to mesh similar to that of a serial mesh, while also allo=
wing the discovery of parallel aspects of the mesh and performance of paral=
lel mesh-based operations efficiently.  MOAB supports a spatial domain-deco=
mposed view of a parallel mesh, where each subdomain is assigned to a proce=
ssor, lower-dimensional entities on interfaces between subdomains are share=
d between processors, and ghost entities can be exchanged with neighboring =
processors.  Locally, each subdomain, along with any locally-represented gh=
ost entities, are accessed through a local MOAB instance.  Parallel aspects=
 of the mesh, e.g. whether entities are shared, on an interface, or ghost e=
ntities, are embedded in the same data model (entities, sets, tags, interfa=
ce) used in the rest of MOAB.  MOAB provides a suite of parallel functions =
for initializing and communicating with a parallel mesh, along with functio=
ns to query the parallel aspects of the mesh.
+
+  \ref contents
+
+  \subsection fiveone 5.1. Nomenclature & Representation
+
+Before discussing how to access parallel aspects of a mesh, several terms =
need to be defined: =20
+
+<B>Shared entity:</B> An entity shared by one or several other processors.
+
+<B>Multi-shared entity:</B> An entity shared by more than two processors.
+
+<B>Owning Processor:</B> Each shared entity is owned by exactly one proces=
sor.  This processor has the right to set tag values on the entity and have=
 those values propagated to any sharing processors. =20
+
+<B>Part:</B> The collection of entities assigned to a given processor.  Wh=
en reading mesh in parallel, the entities in a Part, along with any lower-d=
imensional entities adjacent to those, will be read onto the assigned proce=
ssor.
+
+<B>Partition:</B> A collection of Parts which take part in parallel collec=
tive communication, usually associated with an MPI communicator.
+
+<B>Interface:</B> A collection of mesh entities bounding entities in multi=
ple parts.  Interface entities are owned by a single processor, but are rep=
resented on all parts/processors they bound.
+
+<B>Ghost entity:</B> A shared, non-interface, non-owned entity.
+
+<B>Parallel status:</B> A characteristic of entities and sets represented =
in parallel. The parallel status, or =E2=80=9Cpstatus=E2=80=9D, is represen=
ted by a bit field stored in an unsigned character, with bit values as desc=
ribed in Table 4.
+
+  \subsection tablefour Table 4: Bits representing various parallel charac=
teristics of a mesh.  Also listed are enumerated values that can be used in=
 bitmask expressions; these enumerated variables are declared in MBParallel=
Conventions.h.
+
+<table border=3D"1">
+<tr>
+<th>Bit</th>
+<th>Name</th>
+<th>Represents</th>
+</tr>
+<tr>
+<td>0x1</td>
+<td>PSTATUS_NOT_OWNED</td>
+<td>Not owned by the local processor</td>
+</tr>
+<tr>
+<td>0x2</td>
+<td>PSTATUS_SHARED</td>
+<td>Shared by exactly two processorstd>
+</tr>
+<tr>
+<td>0x4</td>
+<td>PSTATUS_MULTISHARED</td>
+<td>Shared by three or more processors</td>
+</tr>
+<tr>
+<td>0x8</td>
+<td>PSTATUS_INTERFACE</td>
+<td>Part of lower-dimensional interface shared by multiple processors</td>
+</tr>
+<tr>
+<td>0x10</td>
+<td>PSTATUS_GHOST</td>
+<td>Non-owned, non-interface entities represented locally</td>
+</tr>
+</table>
+
+Parallel functionality is described in the following sections.  First, met=
hods to load a mesh into a parallel representation are described; next, fun=
ctions for accessing parallel aspects of a mesh are described; functions fo=
r communicating mesh and tag data are described.
+
+  \ref contents
+
+  \subsection fivetwo 5.2. Parallel Mesh Initialization
+
+Parallel mesh is initialized in MOAB in several steps:
+
+-#  Establish a local mesh on each processor, either by reading the mesh i=
nto that representation from disk, or by creating mesh locally through the =
normal MOAB interface. =20
+-#  Find vertices, then other entities, shared between processors, based o=
n a globally-consistent vertex numbering stored on the GLOBAL_ID tag. =20
+-#  Exchange ghost or halo elements within a certain depth of processor in=
terfaces with neighboring processors. =20
+.
+
+These steps can be executed by a single call to MOAB=E2=80=99s load_file f=
unction, using the procedure described in Section 5.2.1.  Or, they can be e=
xecuted in smaller increments calling specific functions in MOAB=E2=80=99s =
ParallelComm class, as described in Section 5.2.2.  Closely related to the =
latter method is the handling of communicators, described in more detail in=
 Section.
+
+  \ref contents
+
+  \subsection initialization 5.2.1. Parallel Mesh Initialization by Loadin=
g a File
+
+In the file reading approach, a mesh must contain some definition of the p=
artition (the assignment of mesh, usually regions, to processors).  Partiti=
ons can be derived from other set structures already on the mesh, or can be=
 computed explicitly for that purpose by tools like mbzoltan (see Section 4=
.2).  For example, geometric volumes used to generate the mesh, and region-=
based material type assignments, are both acceptable partitions (see Ref. [=
4] for information about this and other meta-data often accompanying mesh).=
  In addition to defining the groupings of regions into parts, the assignme=
nt of specific parts to processors can be done implicitly or using addition=
al data stored with the partition.
+
+MOAB implements several specific methods for loading mesh into a parallel =
representation:
+
+- READ_PART: each processor reads only the mesh used by its part(s).
+
+- READ_DELETE: each processor reads the entire mesh, then deletes the mesh=
 not used by its part(s).
+
+- BCAST_DELETE: the root processor reads and broadcasts the mesh; each pro=
cessor then deletes the mesh not used by its part(s).
+
+The READ_DELETE and BCAST_DELETE methods are supported for all file types =
MOAB is able to read, while READ_PART is only implemented for MOAB=E2=80=99=
s native HDF5-based file format.
+
+Various other options control the selection of part sets or other details =
of the parallel read process.  For example, the application can specify the=
 tags, and optionally tag values, which identify parts, and whether those p=
arts should be distributed according to tag value or using round-robin assi=
gnment.
+
+The options used to specify loading method, the data used to identify part=
s, and other parameters controlling the parallel read process, are shown in=
 Table 5. =20
+  \subsection tablefive Table 5: Options passed to MOAB=E2=80=99s load_fil=
e function identifying the partition and other parameters controlling the p=
arallel read of mesh data.  Options and values should appear in option stri=
ng as =E2=80=9Coption=3Dval=E2=80=9D, with a delimiter (usually =E2=80=9C;=
=E2=80=9D) between options.
+
+<table border=3D"1">
+<tr>
+<th>Option</th>
+<th>Value</th>
+<th>Description</th>
+</tr>
+<tr>
+<td>PARTITION</td>
+<td><tag_name></td>
+<td>Sets with the specified tag name should be used as part sets</td>
+</tr>
+<tr>
+<td>PARTITION_VAL</td>
+<td><val1, val2-val3, ...></td>
+<td>Integer values to be combined with tag name, with ranges input using v=
al1, val2-val3.  Not meaningful unless PARTITION option is also given.</td>
+</tr>
+<tr>
+<td>PARTITION_DISTRIBUTE</td>
+<td>(none)</td>
+<td>If present, or values are not input using PARTITION_VAL, sets with tag=
 indicated in PARTITION option are partitioned across processors in round-r=
obin fashion.</td>
+</tr>
+<tr>
+<td>PARALLEL_RESOLVE_SHARED_ENTS</td>
+<td><pd.sd></td>
+<td>Resolve entities shared between processors, where partition is made up=
 of pd- dimensional entities, and entities of dimension sd and lower should=
 be resolved.</td>
+</tr>
+<tr>
+<td>PARALLEL_GHOSTS</td>
+<td><gd.bd.nl[.ad]></td>
+<td>Exchange ghost elements at shared inter-processor interfaces.  Ghost e=
lements of dimension gd will be exchanged.  Ghost elements are chosen going=
 through bd-dimensional interface entities.  Number of layers of ghost elem=
ents is specified in nl.  If ad is present, lower-dimensional entities boun=
ding exchanged ghost entities will also be exchanged; allowed values for ad=
 are 1 (exchange bounding edges), 2 (faces), or 3 (edges and faces).</td>
+</tr>
+<tr>
+<td>CPUTIME</td>
+<td>(none)</td>
+<td>Print cpu time required for each step of parallel read & initializatio=
n.</td>
+</tr>
+<tr>
+<td>MPI_IO_RANK</td>
+<td><r></td>
+<td>If read method requires reading mesh onto a single processor, processo=
r with rank r is used to do that read.</td>
+</tr>
+</table>
+
+Several example option strings controlling parallel reading and initializa=
tion are:
+
+<B>=E2=80=9CPARALLEL=3DREAD_DELETE; PARTITION=3DMATERIAL_SET; PARTITION_VA=
L=3D100, 200, 600-700=E2=80=9D:</B> The whole mesh is read by every process=
or; this processor keeps mesh in sets assigned the tag whose name is =E2=80=
=9CMATERIAL_SET=E2=80=9D and whose value is any one of 100, 200, and 600-70=
0 inclusive.
+
+<B>=E2=80=9CPARALLEL=3DREAD_PART; PARTITION=3DPARALLEL_PARTITION, PARTITIO=
N_VAL=3D2=E2=80=9D:</B> Each processor reads only its mesh; this processor,=
 whose rank is 2, is responsible for elements in a set with the PARALLEL_PA=
RTITION tag whose value is 2.  This would by typical input for a mesh which=
 had already been partitioned with e.g. Zoltan or Parmetis.
+
+<B>=E2=80=9CPARALLEL=3DBCAST_DELETE; PARTITION=3DGEOM_DIMENSION, PARTITION=
_VAL=3D3, PARTITION_DISTRIBUTE=E2=80=9D:</B> The root processor reads the f=
ile and broadcasts the whole mesh to all processors.  If a list is construc=
ted with entity sets whose GEOM_DIMENSION tag is 3, i.e. sets corresponding=
 to geometric volumes in the original geometric model, this processor is re=
sponsible for all elements with index R+iP, i >=3D 0 (i.e. a round-robin di=
stribution).
+
+ \ref contents
+
+  \subsection functions 5.2.2. Parallel Mesh Initialization Using Functions
+
+After creating the local mesh on each processor, an application can call t=
he following functions in ParallelComm to establish information on shared m=
esh entities.  See the [ref-directparmesh example] in the MOAB source tree =
for a complete example of how this is done from an application.
+
+- ParallelComm::resolve_shared_entities (collective): Resolves shared enti=
ties between processors, based on GLOBAL_ID tag values of vertices.  Variou=
s forms are available, based on entities to be evaluated and maximum dimens=
ion for which entity sharing should be found.
+
+- ParallelComm::exchange_ghost_cells (collective): Exchange ghost entities=
 with processors sharing an interface with this processor, based on specifi=
ed ghost dimension (dimension of ghost entities exchanged), bridge dimensio=
n, number of layers, and type of adjacencies to ghost entities.  An entity =
is sent as a ghost if it is within that number of layers, across entities o=
f the bridge dimension, with entities owned by the receiving processor, or =
if it is a lower-dimensional entity adjacent to a ghost entity and that opt=
ion is requested.
+.
+
+ \ref contents
+
+  \subsection communicator 5.2.3. Communicator Handling
+
+The ParallelComm constructor takes arguments for an MPI communicator and a=
 MOAB instance.  The ParallelComm instance stores the MPI communicator, and=
 registers itself with the MOAB instance.  Applications can specify the Par=
allelComm index to be used for a given file operation, thereby specifying t=
he MPI communicator for that parallel operation.  For example:
+
+\code
+using namespace moab;
+// pass a communicator to the constructor, getting back the index
+MPI_Comm my_mpicomm;
+int pcomm_index;
+ParallelComm my_pcomm(moab, my_mpicomm, &pcomm_index);
+
+// write the pcomm index into a string option
+char load_opt[32];
+sprintf(load_opt, "PARALLEL=3DBCAST_DELETE;PARALLEL_COMM=3D%d",=20
+   pcomm_index);
+
+// specify that option in a parallel read operation
+ErrorCode rval =3D moab->load_file(load_opt, fname, ...)
+\endcode
+
+In the above code fragment, the ParallelComm instance with index pcomm_ind=
ex will be used in the parallel file read, so that the operation executes o=
ver the specified MPI communicator.  If no ParallelComm instance is specifi=
ed for a parallel file operation, a default instance will be defined, using=
 MPI_COMM_WORLD.
+
+Applications needing to retrieve a ParallelComm instance created previousl=
y and stored with the MOAB instance, e.g. by a different code component, ca=
n do so using a static function on ParallelComm:
+
+\code
+ParallelComm *my_pcomm =3D ParallelComm::get_pcomm(moab, pcomm_index);
+\endcode
+
+ParallelComm also provides the ParallelComm::get_all_pcomm function, for r=
etrieving all ParallelComm instances stored with a MOAB instance.  For synt=
ax and usage of this function, see the MOAB online documentation for Parall=
elComm.hpp [8].
+
+ \ref contents
+
+  \subsection fivethree 5.3. Parallel Mesh Query Functions
+
+Various functions are commonly used in parallel mesh-based applications.  =
Functions marked as being collective must be called collectively for all pr=
ocessors that are members of the communicator associated with the ParallelC=
omm instance used for the call.
+
+<B>ParallelComm::get_pstatus:</B>  Get the parallel status for the entity.
+
+<B>ParallelComm::get_pstatus_entities:</B> Get all entities whose pstatus =
satisfies (pstatus & val).
+
+<B>ParallelComm::get_owner:</B> Get the rank of the owning processor for t=
he specified entity.
+
+<B>ParallelComm::get_owner_handle:</B> Get the rank of the owning processo=
r for the specified entity, and the entity's handle on the owning processor.
+
+<B>ParallelComm::get_sharing_data:</B> Get the sharing processor(s) and ha=
ndle(s) for an entity or entities.  Various overloaded versions are availab=
le, some with an optional =E2=80=9Coperation=E2=80=9D argument, where Inter=
face::INTERSECT or Interface::UNION can be specified.  This is similar to t=
he operation arguments to Interface::get_adjacencies.
+
+<B>ParallelComm::get_shared_entities:</B> Get entities shared with the giv=
en processor, or with all processors.  This function has optional arguments=
 for specifying dimension, whether interface entities are requested, and wh=
ether to return just owned entities.
+
+<B>ParallelComm::get_interface_procs:</B> Return all processors with whom =
this processor shares an interface.
+
+<B>ParallelComm::get_comm_procs:</B> Return all processors with whom this =
processor communicates.
+
+  \ref contents
+
+  \subsection fivefour 5.4. Parallel Mesh Communication
+
+Once a parallel mesh has been initialized, applications can call the Paral=
lelComm::exchange_tags function for exchanging tag values between processor=
s.  This function causes the owning processor to send the specified tag val=
ues for all shared, owned entities to other processors sharing those entiti=
es.  Asynchronous communication is used to hide latency, and only point-to-=
point communication is used in these calls.
+
+  \ref contents
+
+  \section applications 6. Building MOAB-Based Applications
+
+There are two primary mechanisms supported by MOAB for building applicatio=
ns, one based on MOAB-defined make variables, and the other based on the us=
e of libtool and autoconf.  Both assume the use of a =E2=80=9Cmake=E2=80=9D=
-based build system. =20
+
+The easiest way to incorporate MOAB into an application=E2=80=99s build pr=
ocess is to include the =E2=80=9Cmoab.make=E2=80=9D file into the applicati=
on=E2=80=99s Makefile, adding the make variables MOAB_INCLUDES and MOAB_LIB=
S_LINK to application=E2=80=99s compile and link commands, respectively.  M=
OAB_INCLUDES contains compiler options specifying the location of MOAB incl=
ude files, and any preprocessor definitions required by MOAB.  MOAB_LIBS_LI=
NK contains both the options telling where libraries can be found, and the =
link options which incorporate those libraries into the application.  Any l=
ibraries depended on by the particular configuration of MOAB are included i=
n that definition, e.g. the HDF5 library.  Using this method to incorporate=
 MOAB is the most straightforward; for example, the following Makefile is u=
sed to build one of the example problems packaged with the MOAB source:
+\code
+include ${MOAB_LIB_DIR}/moab.make
+
+GetEntities : GetEntities.o
+	${CXX} $< ${MOAB_LIBS_LINK} -o $@
+
+.cpp.o :=20
+	${CXX} ${MOAB_INCLUDES} -c $<
+\endcode
+
+Here, the MOAB_LIB_DIR environment variable or make argument definition sp=
ecifies where the MOAB library is installed; this is also the location of t=
he moab.make file.  Once that file has been included, MOAB_INCLUDES and MOA=
B_LIBS_LINK can be used, as shown.
+
+Other make variables are defined in the moab.make file which simplify buil=
ding applications:
+
+- MOAB_LIBDIR, MOAB_INCLUDEDIR: the directories into which MOAB libraries =
and include files will be installed, respectively.  Note that some include =
files are put in a subdirectory named =E2=80=9Cmoab=E2=80=9D below that dir=
ectory, to reflect namespace naming conventions used in MOAB.
+
+- MOAB_CXXFLAGS, MOAB_CFLAGS, MOAB_LDFLAGS: Options passed to the C++ and =
C compilers and the linker, respectively.
+
+- MOAB_CXX, MOAB_CC, MOAB_FC: C++, C, and Fortran compilers specified to M=
OAB at configure time, respectively.
+.
+
+The second method for incorporating MOAB into an application=E2=80=99s bui=
ld system is to use autoconf and libtool.  MOAB is configured using these t=
ools, and generates the =E2=80=9C.la=E2=80=9D files that hold information o=
n library dependencies that can be used in application build systems also b=
ased on autoconf and libtool.  Further information on this subject is beyon=
d the scope of this User=E2=80=99s Guide; see the =E2=80=9C.la=E2=80=9D fil=
es as installed by MOAB, and contact the MOAB developer=E2=80=99s mailing l=
ist [6] for more details.
+
+  \ref contents
+
+  \section implementation  7. iMesh (ITAPS Mesh Interface) Implementation =
in MOAB
+
+iMesh is a common API to mesh data developed as part of the Interoperable =
Tools for Advanced Petascale Simulations (ITAPS) project [18].  Application=
s using the iMesh interface can operate on any implementation of that inter=
face, including MOAB.  MOAB-based applications can take advantage of other =
services implemented on top of iMesh, including the MESQUITE mesh improveme=
nt toolkit [19] and the GRUMMP mesh improvement library [20].
+
+MOAB=E2=80=99s native interface is accessed through the Interface abstract=
 C++ base class.  Wrappers are not provided in other languages; rather, app=
lications wanting to access MOAB from those languages should do so through =
iMesh.  In most cases, the data models and functionality available through =
MOAB and iMesh are identical.  However, there are a few differences, subtle=
 and not-so-subtle, between the two:
+
+<B>SPARSE tags used by default:</B> MOAB=E2=80=99s iMesh implementation cr=
eates SPARSE tags by default, because of semantic requirements of other tag=
-related functions in iMesh.  To create DENSE tags through iMesh, use the i=
Mesh_createTagWithOptions extension function (see below).
+
+<B>Higher-order elements:</B> ITAPS currently handles higher-order element=
s (e.g. a 10-node tetrahedron) usi[21]<sup>5</sup>.  As described in [sec-e=
ntities], MOAB supports higher-order entities by allowing various numbers o=
f vertices to define topological entities like quadrilateral or tetrahedron=
.  Applications can specify flags to the connectivity and adjacency functio=
ns specifying whether corner or all vertices are requested.
+
+<B>Self-adjacencies:</B> In MOAB=E2=80=99s native interface, entities are =
always self-adjacent<sup>6</sup>; that is, adjacencies of equal dimension r=
equested from an entity will always include that entity, while from iMesh w=
ill not include that entity.
+
+<B>Option strings:</B> The iMesh specification requires that options in th=
e options string passed to various functions (e.g. iMesh_load) be prepended=
 with the implementation name required to parse them, and delimited with sp=
aces.  Thus, a MOAB-targeted option would appear as =E2=80=9Cmoab:PARALLEL=
=3DREAD_PART moab:PARTITION=3DMATERIAL_SET=E2=80=9D.
+
+To provide complete MOAB support from other languages through iMesh, a col=
lection of iMesh extension functions are also available.  A general descrip=
tion of these extensions appears below; for a complete description, see the=
 online documentation for iMesh-extensions.h [8].
+
+- Recursive get_entities functions: There are many cases where sets includ=
e other sets (see [4] for more information).  MOAB provides iMesh_getEntiti=
esRec, and other recursive-supporting functions, to get all non-set entitie=
s of a given type or topology accessible from input set(s).  Similar functi=
ons are available for number of entities of a given type/topology.
+
+- Get entities by tag, and optionally tag value: It is common to search fo=
r entities with a given tag, and possibly tag value(s); functions like iMes=
h_getEntitiesByTag are provided for this purpose.
+
+- Options to createTag: To provide more control over the tag type, the iMe=
sh_createTagWithOptions is provided.  The storage type is controlled with t=
he =E2=80=9C
+
+- MBCNType: Canonical numbering evaluations are commonly needed by applica=
tions, e.g. to apply boundary conditions locally.  The MBCN package provide=
s these evaluations in terms of entity types defined in MOAB [9]; the getMB=
CNType is required to translate between iMesh_Topology and MBCN type.
+
+- Iterator step: Step an iterator a specified number of entities; allows a=
dvancement of an iterator without needing to allocate memory to hold the en=
tity handles stepped over.
+
+- Direct access to tag storage: The Interface::tag_iterate function allows=
 an application get a pointer to the memory used to store a given tag.  For=
 dense tags on contiguous ranges of entities, this provides more efficient =
access to tags.  The iMesh functionn iMesh_tagIterate provides access to th=
is functionlity.  See examples/TagIterateC.c and examples/TagIterateF.F for=
 examples of how to use this from C and Fortran, respectively.=20
+.
+
+As required by the iMesh specification, MOAB generates the =E2=80=9CiMesh-=
Defs.inc=E2=80=9D file and installs it with the iMesh and MOAB libraries.  =
This file defines make variables which can be used to build iMesh-based app=
lications.  The method used here is quite similar to that used for MOAB its=
elf (see Section 6).  In particular, the IMESH_INCLUDES and IMESH_LIBS make=
 variables can be used with application compile and link commands, respecti=
vely, with other make variables similar to those provided in moab.make also=
 available.
+
+Note that using the iMesh interface from Fortran-based applications requir=
es a compiler that supports Cray pointers, along with the pass-by-value (%V=
AL) extension.  Almost all compilers support those extensions; however, if =
using the gcc series of compilers, you must use gfortran 4.3 or later.
+
+<sup>5</sup>There are currently no implementations of this interface.
+
+<sup>6</sup>iMesh and MOAB both define adjacencies using the topological c=
oncept of closure.  Since the closure of an entity includes the entity itse=
lf, the d-dimensional entities on the closure of a given entity should incl=
ude the entity itself.
+
+ \ref contents
+
+  \section representation 8. Structured Mesh Representation
+
+A structured mesh is defined as a D-dimensional mesh whose interior vertic=
es have 2D connected edges.   Structured mesh can be stored without connect=
ivity, if certain information is kept about the parametric space of each st=
ructured block of mesh.  MOAB can represent structured mesh with implicit c=
onnectivity, saving approximately 57% of the storage cost compared to an un=
structured representation<sup>7</sup>.  Since connectivity must be computed=
 on the fly, these queries execute a bit slower than those for unstructured=
 mesh.  More information on the theory behind MOAB's structured mesh repres=
entation can be found in=20
+
+Currently, MOAB's structured mesh representation can only be used by creat=
ing structured mesh at runtime; that is, structured mesh is saved/restored =
in an unstructured format in MOAB's HDF5-based native save format.  For mor=
e details on how to use MOAB's structured mesh representation, see the scds=
eq_test.cpp source file in the test/ directory.
+
+<sup>7</sup> This assumes vertex coordinates are represented explicitly, a=
nd that there are approximately the same number of vertices and hexahedra i=
n a structured hex mesh.
+
+ \ref contents
+
+  \section element 9. Spectral Element Meshes
+
+The Spectral Element Method (SEM) is a high-order method, using a polynomi=
al Legendre interpolation basis with Gauss-Lobatto quadrature points, in co=
ntrast to the Lagrange basis used in (linear) finite elements [20].  SEM ob=
tains exponential convergence with decreasing mesh characteristic sizes, an=
d codes implementing this method typically have high floating-point intensi=
ty, making the method highly efficient on modern CPUs.  Most Nth-order SEM =
codes require tensor product cuboid (quad/hex) meshes, with each d-dimensio=
nal element containing (N+1)d degrees of freedom (DOFs).  There are various=
 methods for representing SEM meshes and solution fields on them; this docu=
ment discusses these methods and the tradeoffs between them.  The mesh part=
s of this discussion are given in terms of the iMesh mesh interface and its=
 implementation by the MOAB mesh library [21].
+
+The figure above shows a two-dimensional 3rd-order SEM mesh consisting of =
four quadrilaterals.  For this mesh, each quadrilateral has (N+1)^2=3D16 DO=
Fs, with corner and edge degrees of freedom shared between neighboring quad=
rilaterals.
+
+  \ref contents
+
+  \subsection nineone 9.1. Representations
+
+There are various representations of this mesh in a mesh database like MOA=
B, depending on how DOFs are related to mesh entities and tags on those ent=
ities.  We mention several possible representations:
+
+-# Corner vertices, element-based DOFs: Each quadrilateral is defined by f=
our vertices, ordered in CCW order typical of FE meshes.  DOFs are stored a=
s tags on quadrilaterals, with size (N+1)^2 values, ordered lexicographical=
ly (i.e. as a 2D array tag(i,j) with i varying faster than j.)  In the figu=
re above, the connectivity for face 1 would be (1, 4, 16, 13), and DOFs wou=
ld be ordered (1..16).  Note that in this representation, tag values for DO=
Fs shared by neighboring elements must be set multiple times, since there a=
re as many copies of these DOFs as elements sharing them.
+-# High-order FE-like elements: Each DOF is represented by a mesh vertex. =
Quadrilaterals each have (N+1)^2 vertices, ordered as they would be for hig=
h-order finite elements (corner vertices first, then mid-edge and mid-face =
elements; see [22]).  Mid -face, -edge, and -region vertices for a given ed=
ge/face/region would be ordered lexicographically, according to positive di=
rection in a corresponding reference element.  In the figure above, the con=
nectivity array for face 1 would be (1, 4, 16, 13, 2, 3, 8, 12, 14, 15, 5, =
9, 6, 7, 10, 11).  DOF values are stored as tags on vertices.  Since DOFs a=
re uniquely associated with vertices and vertices are shared by neighboring=
 elements, tag values only need to be set once.  Full vertex-quadrilateral =
adjacencies are available, for all vertices.
+-# Linear FE-like elements, one vertex per DOF, array with DOF vertices: E=
ach quadrilateral is defined by four (corner) vertices, with additional ver=
tices representing mid-edge and mid-face DOFs.  An additional =E2=80=9CDOF =
array=E2=80=9D tag is assigned to each quadrilateral, storing the array of =
vertices representing the (N+1)^2 DOFs for the quadrilateral, ordered lexic=
ographically.  For the figure above, the connectivity array for face 1 woul=
d be (1, 4, 16, 13), and the DOF array would be (1..16), assuming that vert=
ex handles are integers as shown in the figure.  DOF values are stored as t=
ags on vertices, and lexicographically-ordered arrays of DOFs can be retrie=
ved using the DOF array tag as input to the tag_get_data function in MOAB. =
 Adjacency functions would only be meaningful for corner vertices, but tag =
values would only need to be set once per DOF.
+-# High-order FE-like elements, array with DOF vertices: This is a combina=
tion of options 2 and 3.  The advantage would be full vertex-quad adjacency=
 support and direct availability of lexicographically-ordered vertex arrays=
, at the expense of more memory.
+-# Convert to linear mesh: Since a spectral element is a cuboid with highe=
r-order vertices, it can always be converted to N^2 linear cuboids using th=
e high-order vertices as corners of the finer quads/hexes.  This is how rea=
ders in ParaView and VisIt typically import spectral meshes (CAM-SE also ex=
ports connectivity in this form).
+
+As a convenience for applications, functions could also be provided for im=
portant tasks, like assembling the vertex handles for an entity in lexograp=
hic order (useful for option 2 above), and getting an array of tag values i=
n lexicographic order (for option 3 above).
+
+  \ref contents
+
+  \subsection ninetwo 9.2. Tradeoffs
+
+There are various competing tradeoffs in the various representation types.=
  These include:
+
+- Adjacencies: being able to retrieve the element(s) using a given (corner=
 or higher-order) vertex.
+- Connectivity list: being able to retrieve the connectivity of a given el=
ement, consisting of all (corner + higher-order) vertices in the element, u=
sually in lexicographical order.  This is closely linked with being able to=
 access the connectivity list as a const*, i.e. using the list straight fro=
m memory without needing to copy it.
+- Memory vs. time: There is a memory vs. execution time tradeoff between d=
uplicating interface vertex solution/tag variables in neighboring elements =
(more memory but more time-efficient and allows direct access to tag storag=
e by applications) versus using vertex-based tags (less memory but requires=
 assembly of variables into lexicographically-ordered arrays, and prevents =
direct access from applications).
+.
+
+The lower-memory option (storing variables on vertices and assembling into=
 lexicographically-ordered arrays for application use) usually ends up cost=
ing more in memory anyway, since applications must allocate their own stora=
ge for these arrays.  On the other hand, certain applications will always c=
hoose to do that, instead of sharing storage with MOAB for these variables.=
  In the case where applications do share memory with MOAB, other tools wou=
ld need to interpret the lexicographically-ordered field arrays specially, =
instead of simply treating the vertex tags as a point-based field.
+
+  \ref contents
+
+  \subsection ninethree 9.3. MOAB Representation
+In choosing the right MOAB representation for spectral meshes, we are tryi=
ng to balance a) minimal memory usage, b) access to properly-ordered and -a=
ligned tag storage, and c) maximal compatibility with tools likely to use M=
OAB.  The solution we propose is to use a representation most like option 2=
) above, with a few optional behaviors based on application requirements. =20
+
+In brief, we propose to represent elements using the linear, FE-ordered co=
nnectivity list (containing only corner vertices from the spectral element)=
, with field variables written to either vertices, lexicographically-ordere=
d arrays on elements, or both, and with a lexicographically-ordered array (=
stored on tag SPECTRAL_VERTICES) of all (corner+higher-order) vertices stor=
ed on elements.  In the either/or case, the choice will be evident from the=
 tag size and the entities on which the tag is set.  In the both case, the =
tag name will have a =E2=80=9C-LEX=E2=80=9D suffix for the element tags, an=
d the size of the element tag will be (N+1)^2 times that of the vertex-base=
d tag.  Finally, the file set containing the spectral elements (or the root=
 set, if no file set was input to the read) will contain a =E2=80=9CSPECTRA=
L_ORDER=E2=80=9D tag whose value is N.  These conventions are described in =
the =E2=80=9CMetadata Information=E2=80=9D document distributed with the MO=
AB source code.
+
+  \ref contents
+
+  \section performance 10. Performance and Using MOAB Efficiently from App=
lications
+
+MOAB is designed to operate efficiently on groups of entities and for larg=
e meshes.  Applications will be most efficient when they operate on entitie=
s in groups, especially groups which are close in their order of creation. =
 The MOAB API is structured to encourage operations on groups of entities. =
 Conversely, MOAB will not perform as well as other libraries if there are =
frequent deletion and creation of entities.  For those types of application=
s, a mesh library using a C++ object-based representation is more appropria=
te.  In this section, performance of MOAB when executing a variety of tasks=
 is described, and compared to that of other representations.  Of course, t=
hese metrics are based on the particular models and environments where they=
 are run, and may or may not be representative of other application types.
+
+One useful measure of MOAB performance is in the representation and query =
of a large mesh.  MOAB includes a performance test, located in the test/per=
f directory, in which a single rectangular region of hexahedral elements is=
 created then queried; the following steps are performed:
+
+- Create the vertices and hexes in the mesh
+- For each vertex, get the set of connected hexahedra
+- For each hex, get the connected vertices, their coordinates, average the=
m, and assign them as a tag on the hexes
+.
+
+This test can be run on your system to determine the runtime and memory pe=
rformance for these queries in MOAB.
+
+  \ref contents
+
+  \section conclusions 11. Conclusions and Future Plans
+
+MOAB, a Mesh-Oriented datABase, provides a simple but powerful data abstra=
ction to structured and unstructured mesh, and makes that abstraction avail=
able through a function API.  MOAB provides the mesh representation for the=
 VERDE mesh verification tool, which demonstrates some of the powerful mesh=
 metadata representation capabilities in MOAB.  MOAB includes modules that =
import mesh in the ExodusII, CUBIT .cub and Vtk file formats, as well as th=
e capability to write mesh to ExodusII, all without licensing restrictions =
normally found in ExodusII-based applications.  MOAB also has the capabilit=
y to represent and query structured mesh in a way that optimizes storage sp=
ace using the parametric space of a structured mesh; see Ref. [17] for deta=
ils.
+
+Initial results have demonstrated that the data abstraction provided by MO=
AB is powerful enough to represent many different kinds of mesh data found =
in real applications, including geometric topology groupings and relations,=
 boundary condition groupings, and inter-processor interface representation=
.  Our future plans are to further explore how these abstractions can be us=
ed in the design through analysis process.
+
+  \ref contents
+
+  \section references 12. References
+
+[1]	M. Fatenejad and G.A. Moses, =E2=80=9CCooper radiation hydrodynamics c=
ode..=E2=80=9D
+
+[2]	T.J. Tautges and A. Caceres, =E2=80=9CScalable parallel solution coupl=
ing for multiphysics reactor simulation,=E2=80=9D Journal of Physics: Confe=
rence Series,  vol. 180, 2009.
+
+[3]	T.J. Tautges, MOAB Meta-Data Information, 2010.
+
+[4]	T.J. Tautges, =E2=80=9CMOAB - ITAPS =E2=80=93 Trac.=E2=80=9D, http://t=
rac.mcs.anl.gov/projects/ITAPS/wiki/MOAB
+
+[5]	=E2=80=9CMOAB Developers Email List.=E2=80=9D, moab-dev at mcs.anl.gov.
+
+[6]	=E2=80=9CMOAB Users Email List.=E2=80=9D, moab at mcs.anl.gov.
+
+[7]	=E2=80=9CMOAB online documentation.=E2=80=9D, http://gnep.mcs.anl.gov:=
8010/moab-docs/
+
+[8]	T.J. Tautges, =E2=80=9CCanonical numbering systems for finite-element =
codes,=E2=80=9D Communications in Numerical Methods in Engineering,  vol. O=
nline, Mar. 2009.
+
+[9]	L.A. Schoof and V.R. Yarberry, EXODUS II: A Finite Element Data Model,=
  Albuquerque, NM: Sandia National Laboratories, 1994.
+
+[10]	M. PATRAN, =E2=80=9CPATRAN User=E2=80=99s Manual,=E2=80=9D 2005.
+
+[11]	VisIt User's Guide.
+
+[12]	K. Devine, E. Boman, R. Heaphy, B. Hendrickson, and C. Vaughan, =E2=
=80=9CZoltan Data Management Services for Parallel Dynamic Applications,=E2=
=80=9D Computing in Science and Engineering,  vol. 4, 2002, pp. 90=E2=80=93=
97.
+
+[13]	T.J. Tautges, P.P.H. Wilson, J. Kraftcheck, B.F. Smith, and D.L. Hend=
erson, =E2=80=9CAcceleration Techniques for Direct Use of  CAD-Based Geomet=
ries in Monte Carlo Radiation Transport,=E2=80=9D International Conference =
on Mathematics, Computational Methods & Reactor Physics (M&C 2009),  Sarato=
ga Springs, NY: American Nuclear Society, 2009.
+
+[14]	H. Kim and T. Tautges, =E2=80=9CEBMesh: An Embedded Boundary Meshing =
Tool,=E2=80=9D in preparation.
+
+[15]	G.D. Sjaardema, T.J. Tautges, T.J. Wilson, S.J. Owen, T.D. Blacker, W=
.J. Bohnhoff, T.L. Edwards, J.R. Hipp, R.R. Lober, and S.A. Mitchell, CUBIT=
 mesh generation environment Volume 1: Users manual, Sandia National Labora=
tories, May 1994, 1994.
+
+[16]	T.J. Tautges, =E2=80=9CCGM: A geometry interface for mesh generation,=
 analysis and other applications,=E2=80=9D Engineering with Computers,  vol=
. 17, 2001, pp. 299-314.
+
+[17]	T. J. Tautges, MOAB-SD: Integrated structured and unstructured mesh r=
epresentation, Engineering with Computers, vol. 20, no. 3, pp. 286-293, 200=
4.
+
+[18]	=E2=80=9CInteroperable Technologies for Advanced Petascale Simulation=
s (ITAPS),=E2=80=9D Interoperable Technologies for Advanced Petascale Simul=
ations (ITAPS).
+
+[19]	P. Knupp, =E2=80=9CMesh quality improvement for SciDAC applications,=
=E2=80=9D Journal of Physics: Conference Series,  vol. 46, 2006, pp. 458-46=
2.
+
+[20]	M. O. Deville, P. F. Fischer, and E. H. Mund, High-order methods for =
incompressible fluid flow. Cambridge, UK; New York: Cambridge University Pr=
ess, 2002.
+
+[21]	T. J. Tautges, =E2=80=9CMOAB Wiki.=E2=80=9D [Online]. Available: http=
://trac.mcs.anl.gov/projects/ITAPS/wiki/MOAB. [Accessed: 30-Oct-2012].
+
+[22]	T. J. Tautges, =E2=80=9CCanonical numbering systems for finite-elemen=
t codes,=E2=80=9D International Journal for Numerical Methods in Biomedical=
 Engineering, vol. 26, no. 12, pp. 1559=E2=80=931572, 2010.
+
+
+  \ref contents
+
+  \page differences Differences Between iMesh and MOAB
+
+  The data models used in MOAB and iMesh are quite similar, but not identi=
cal.The most significant differences are the following:
+
+- Tags: MOAB differentiates between DENSE, SPARSE, and BIT tags, using dif=
ferent storage models for each, while iMesh uses a single tag concept.  iMe=
sh allows application to query whether an entity has been given a tag of a =
specified type; this query is incompatible with the concept of a DENSE tag =
with a default value.  Thus, MOAB=E2=80=99s iMesh implementation creates SP=
ARSE tags by default, and tags created and accessed through this interface =
will use more memory than DENSE tags created through MOAB=E2=80=99s native =
interface.  To mitigate this problem, MOAB implements an extension of the i=
Mesh_createTag function which allows specification of the tag type (DENSE, =
SPARSE, etc.) to be created.  See later in this section for more informatio=
n.
+
+- Higher-order nodes: ITAPS currently handles higher-order elements (e.g. =
a 10-node tetrahedron) using a special =E2=80=9CShape=E2=80=9D interface.  =
In this interface, higher-order nodes are only accessible through the AEnti=
ties which they resolve.  MOAB=E2=80=99s iMesh implementation provides acce=
ss to higher-order nodes in the same manner described in Section  , by vary=
ing the number of vertices defining each entity.  As a result, if higher-or=
der entities are used in a model, the functions returning connectivity and =
vertex adjacencies always return all vertices, rather than providing an opt=
ion to return just corner vertices.
+
+- Self-adjacencies: iMesh specifies that entities are not self-adjacent; t=
hat is, requesting adjacencies of the same dimension/type results in an err=
or.  MOAB does not consider this an error, returning the entity itself.
+
+- Adjacency table and AEntities: iMesh uses the concept of an =E2=80=9Cadj=
acency table=E2=80=9D to determine which AEntities are available and create=
d by default.  MOAB uses input arguments to the get_adjacencies functions t=
o control whether AEntities are created.  These flags provide finer-grained=
 control over AEntities, but make it slightly less convenient to ensure tha=
t AEntities of a given dimension are always created.
+.
+
+ \page figures List of Figures
+=20
+  This page is intended to be empty.
+=20
+  \page tables List of Tables
+=20
+     \ref tableone
+
+     \ref tabletwo
+
+     \ref tablethree
+
+     \ref tablefour
+
+     \ref tablefive
+=20
+  \page building Building & Installing
+=20
+  MOAB uses an autoconf and libtool-based build process by default.  The p=
rocedure used to build MOAB from scratch depends on whether the source code=
 was obtained from a =E2=80=9Ctarball=E2=80=9D or directly from the Subvers=
ion repository.  Assuming the latter, the following steps should be execute=
d for building and installing MOAB:
+  - Locate and build any required dependencies.  MOAB can be built with no=
 dependencies on other libraries; this may be useful for applications only =
needing basic mesh representation and not needing to export mesh to formats=
 implemented in other libraries.  MOAB=E2=80=99s native save/restore capabi=
lity is built on HDF5-based files; applications needing to save and restore=
 files from MOAB reliably should use this library.  MOAB also uses ExodusII=
, a netCDF-based file format developed at Sandia National Laboratories [10]=
.  Applications needing to execute these tests should also build netCDF.  N=
ote that MOAB uses netCDF=E2=80=99s C++ interface, which is not enabled by =
default in netCDF but can be enabled using the =E2=80=9C=E2=80=93enable-cxx=
=E2=80=9D option to netCDF=E2=80=99s configure script.
+  - Unpack source code into <moab>, and change current working directory t=
o that location.
+  - Execute =E2=80=9Cautoreconf =E2=80=93fi=E2=80=9D.
+  - Run configure script, by executing =E2=80=9C./configure <options>=E2=
=80=9D.  Recommended options:
+       -# =E2=80=93prefix=3D<install_dir>: directory below which MOAB libr=
ary and include files will be installed; can either be the directory used f=
or MOAB source (<moab> from step 1), or a different directory.
+       -# =E2=80=93hdf5-dir=3D<hdf5_dir>: directory whose =E2=80=9Cinclude=
=E2=80=9D and =E2=80=9Clib=E2=80=9D subdirectories hold HDF5 include and li=
brary, respectively.  MOAB uses HDF5 for its native save/restore format (se=
e Section 4.6.1).
+       -# =E2=80=93netcdf-dir=3D: directory whose =E2=80=9Cinclude=E2=80=
=9D and =E2=80=9Clib=E2=80=9D subdirectories hold netCDF include and librar=
y, respectively.  MOAB uses netCDF-based files for many of its build tests.=
  If the location of netCDF cannot be found, MOAB=E2=80=99s build tests wil=
l not function properly, but MOAB will still be usable.
+       .
+  - Run =E2=80=9Cmake check=E2=80=9D; this runs a series of build tests, t=
o verify that the MOAB build was successful.  Note this check will fail if =
netCDF is not used, but MOAB itself will still be usable from applications.
+  - Run =E2=80=9Cmake install=E2=80=9D; this copies include files and libr=
aries to subdirectories of the directory specified in the =E2=80=9Cprefix=
=E2=80=9D option.
+  .
+
+These steps are sufficient for building MOAB against HDF5 and netCDF.  By =
default, a small number of standard MOAB-based applications are also built,=
 including mbconvert (a utility for reading and writing files), mbsize (for=
 querying basic information about a mesh), and the iMesh interface (see Sec=
tion 7).  Other utilities can be enabled using various other options to the=
 configure script; for a complete list of build options, execute =E2=80=9C.=
/configure =E2=80=93help=E2=80=9D.
+=20
+ */
+

diff --git a/doc/user.dox.in b/doc/user.dox.in
index 4fed19e..6b7730b 100644
--- a/doc/user.dox.in
+++ b/doc/user.dox.in
@@ -305,7 +305,8 @@ WARN_LOGFILE           =3D
 # with spaces.
=20
 INPUT                  =3D @top_srcdir@/src @top_srcdir@/src/moab \
-                         @top_srcdir@/src/parallel/moab \
+                         @top_srcdir@/doc/UG/moabUG.h \
+                         @top_srcdir@/src/parallel/moab  \
                          @top_srcdir@/src/MBTagConventions.hpp \
                          @top_srcdir@/src/MBCN.h \
                          @top_srcdir@/src/MBEntityType.h \

diff --git a/examples/DirectAccessNoHoles.cpp b/examples/DirectAccessNoHole=
s.cpp
new file mode 100644
index 0000000..340efae
--- /dev/null
+++ b/examples/DirectAccessNoHoles.cpp
@@ -0,0 +1,186 @@
+/** @example DirectAccessNoHoles.cpp \n
+ * \brief Use direct access to MOAB data to avoid calling through API \n
+ *
+ * This example creates a 1d row of quad elements, such that all quad and =
vertex handles
+ * are contiguous in the handle space and in the database.  Then it shows =
how to get access
+ * to pointers to MOAB-native data for vertex coordinates, quad connectivi=
ty, tag storage,
+ * and vertex to quad adjacency lists.  This allows applications to access=
 this data directly
+ * without going through MOAB's API.  In cases where the mesh is not chang=
ing (or only mesh
+ * vertices are moving), this can save significant execution time in appli=
cations.
+ *
+ *  ----------------------
+ *  |      |      |      |      =20
+ *  |      |      |      | ...
+ *  |      |      |      |
+ *  ----------------------
+ *
+ *    -#  Initialize MOAB \n
+ *    -#  Create a quad mesh, as depicted above
+ *    -#  Create 2 dense tags (tag1, tag2) for avg position to assign to q=
uads, and # verts per quad (tag3)
+ *    -#  Get connectivity, coordinate, tag1 iterators
+ *    -#  Iterate through quads, computing midpoint based on vertex positi=
ons, set on quad-based tag1
+ *    -#  Iterate through vertices, summing positions into tag2 on connect=
ed quads and incrementing vertex count
+ *    -#  Iterate through quads, normalizing tag2 by vertex count and comp=
aring values of tag1 and tag2
+ *
+ * <b>To compile</b>: \n
+ *    make DirectAccessNoHoles MOAB_DIR=3D<installdir>  \n
+ * <b>To run</b>: ./DirectAccessNoHoles [-nquads <# quads>]\n
+ *
+ */
+
+#include "moab/Core.hpp"
+#include "moab/ProgOptions.hpp"
+#include "moab/ReadUtilIface.hpp"
+#include <map>
+#include <iostream>
+#include <assert.h>
+
+// Error routines for use with MOAB API
+#define CHKERR(CODE, MSG) do {if (MB_SUCCESS !=3D (CODE)) {std::string err=
str;  mbImpl->get_last_error(errstr);   \
+        std::cerr << errstr << std::endl; std::cerr << MSG << std::endl; r=
eturn CODE;}} while(false)
+
+using namespace moab;
+
+ErrorCode create_mesh_no_holes(Interface *mbImpl, int nquads);
+
+int main(int argc, char **argv)
+{
+    // get MOAB instance
+  Interface *mbImpl =3D new Core;
+  if (NULL =3D=3D mbImpl) return 1;
+
+  int nquads =3D 1000;
+ =20
+    // parse options
+  ProgOptions opts;
+  opts.addOpt<int>(std::string("nquads,n"), std::string("Number of quads i=
n the mesh (default =3D 1000"), &nquads);
+  opts.parseCommandLine(argc, argv);
+
+    // create simple structured mesh with hole, but using unstructured rep=
resentation
+  ErrorCode rval =3D create_mesh_no_holes(mbImpl, nquads); CHKERR(rval, "T=
rouble creating mesh.");
+ =20
+    // get all vertices and non-vertex entities
+  Range verts, quads;
+  rval =3D mbImpl->get_entities_by_handle(0, quads); CHKERR(rval, "Getting=
 all entities.");
+  verts =3D quads.subset_by_dimension(0);
+  quads -=3D verts;
+
+    // create tag1 (element-based avg), tag2 (vertex-based avg), tag3 (# c=
onnected verts)
+  Tag tag1, tag2, tag3;
+  rval =3D mbImpl->tag_get_handle("tag1", 3, MB_TYPE_DOUBLE, tag1, MB_TAG_=
DENSE | MB_TAG_CREAT); CHKERR(rval, "Creating tag1.");
+  double def_val[3] =3D {0.0, 0.0, 0.0}; // need a default value for tag2 =
because we sum into it
+  rval =3D mbImpl->tag_get_handle("tag2", 3, MB_TYPE_DOUBLE, tag2, MB_TAG_=
DENSE | MB_TAG_CREAT, def_val); CHKERR(rval, "Creating tag2.");
+  int def_val_int =3D 0;  // need a default value for tag3 because we incr=
ement it
+  rval =3D mbImpl->tag_get_handle("tag3", 1, MB_TYPE_INTEGER, tag3, MB_TAG=
_DENSE | MB_TAG_CREAT, &def_val_int); CHKERR(rval, "Creating tag3.");
+ =20
+    // Get pointers to connectivity, coordinate, tag, and adjacency arrays=
; each of these returns a count,
+    // which should be compared to the # entities you expect to verify the=
re's only one chunk (no holes)
+  int count, vpere;
+  EntityHandle *conn_ptr;
+  rval =3D mbImpl->connect_iterate(quads.begin(), quads.end(), conn_ptr, v=
pere, count); CHKERR(rval, "connect_iterate.");
+  assert(count =3D=3D (int) quads.size()); // should end up with just one =
contiguous chunk of quads
+
+  double *x_ptr, *y_ptr, *z_ptr;
+  rval =3D mbImpl->coords_iterate(verts.begin(), verts.end(), x_ptr, y_ptr=
, z_ptr, count); CHKERR(rval, "coords_iterate.");
+  assert(count =3D=3D (int) verts.size()); // should end up with just one =
contiguous chunk of vertices
+
+  double *tag1_ptr, *tag2_ptr;
+  int *tag3_ptr;
+  rval =3D mbImpl->tag_iterate(tag1, quads.begin(), quads.end(), count, (v=
oid*&)tag1_ptr); CHKERR(rval, "tag1_iterate.");
+  assert(count =3D=3D (int) quads.size()); // should end up with just one =
contiguous chunk of quads
+  rval =3D mbImpl->tag_iterate(tag2, quads.begin(), quads.end(), count, (v=
oid*&)tag2_ptr); CHKERR(rval, "tag2_iterate.");
+  assert(count =3D=3D (int) quads.size()); // should end up with just one =
contiguous chunk of quads
+  rval =3D mbImpl->tag_iterate(tag3, quads.begin(), quads.end(), count, (v=
oid*&)tag3_ptr); CHKERR(rval, "tag3_iterate.");
+  assert(count =3D=3D (int) quads.size()); // should end up with just one =
contiguous chunk of quads
+
+  const std::vector<EntityHandle> **adjs_ptr;
+  rval =3D mbImpl->adjacencies_iterate(verts.begin(), verts.end(), adjs_pt=
r, count); CHKERR(rval, "adjacencies_iterate.");
+  assert(count =3D=3D (int) verts.size()); // should end up with just one =
contiguous chunk of vertices
+ =20
+    // Start_ handles used to compute indices into vertex/quad arrays; can=
 use direct subtraction because we know
+    // there aren't any holes in the handle spaces for verts or quads
+  EntityHandle start_vert =3D *verts.begin(), start_quad =3D *quads.begin(=
);
+
+    // iterate over elements, computing tag1 from coords positions
+  for (int i =3D 0; i < nquads; i++) {
+    tag1_ptr[3*i+0] =3D tag1_ptr[3*i+1] =3D tag1_ptr[3*i+2] =3D 0.0; // in=
itialize position for this element
+    for (int j =3D 0; j < vpere; j++) { // loop over vertices in this elem=
ent
+      int v_index =3D conn_ptr[vpere*i+j] - start_vert; // vert index is j=
ust the offset from start vertex
+      tag1_ptr[3*i+0] +=3D x_ptr[v_index];
+      tag1_ptr[3*i+1] +=3D y_ptr[v_index]; // sum vertex positions into ta=
g1...
+      tag1_ptr[3*i+2] +=3D z_ptr[v_index];
+    }
+    tag1_ptr[3*i+0] /=3D vpere;
+    tag1_ptr[3*i+1] /=3D vpere; // then normalize
+    tag1_ptr[3*i+2] /=3D vpere;
+  } // loop over elements in chunk
+   =20
+    // Iterate through vertices, summing positions into tag2 on connected =
elements and incrementing vertex count
+  for (int v =3D 0; v < count; v++) {
+    const std::vector<EntityHandle> *avec =3D *(adjs_ptr+v);
+    for (std::vector<EntityHandle>::const_iterator ait =3D avec->begin(); =
ait !=3D avec->end(); ait++) {
+        // *ait is the quad handle, its index is computed by subtracting t=
he start quad handle
+      int a_ind =3D *ait - start_quad;
+      tag2_ptr[3*a_ind+0] +=3D x_ptr[v];   // tag on each element is 3 dou=
bles, x/y/z
+      tag2_ptr[3*a_ind+1] +=3D y_ptr[v];
+      tag2_ptr[3*a_ind+2] +=3D z_ptr[v];
+      tag3_ptr[a_ind]++;  // increment the vertex count
+    }
+  }
+       =20
+    // Normalize tag2 by vertex count (tag3); loop over elements using sam=
e approach as before
+    // At the same time, compare values of tag1 and tag2
+  int n_dis =3D 0;
+  for (Range::iterator q_it =3D quads.begin(); q_it !=3D quads.end(); q_it=
++) {
+    int i =3D *q_it - start_quad;
+    for (int j =3D 0; j < 3; j++) tag2_ptr[3*i+j] /=3D (double)tag3_ptr[i]=
;  // normalize by # verts
+    if (tag1_ptr[3*i] !=3D tag2_ptr[3*i] || tag1_ptr[3*i+1] !=3D tag2_ptr[=
3*i+1] || tag1_ptr[3*i+2] !=3D tag2_ptr[3*i+2]) {
+      std::cout << "Tag1, tag2 disagree for element " << *q_it + i << std:=
:endl;
+      n_dis++;
+    }
+  }
+  if (!n_dis) std::cout << "All tags agree, success!" << std::endl;
+
+    // Ok, we're done, shut down MOAB
+  delete mbImpl;
+
+  return 0;
+}
+
+ErrorCode create_mesh_no_holes(Interface *mbImpl, int nquads)=20
+{
+    // first make the mesh, a 1d array of quads with left hand side x =3D =
elem_num; vertices are numbered in layers
+  ReadUtilIface *read_iface;
+  ErrorCode rval =3D mbImpl->query_interface(read_iface); CHKERR(rval, "qu=
ery_interface");
+  std::vector<double *> coords;
+  EntityHandle start_vert, start_elem, *connect;
+    // create verts, num is 2(nquads+1) because they're in a 1d row; will =
initialize coords in loop over quads later
+  rval =3D read_iface->get_node_coords (3, 2*(nquads+1), 0, start_vert, co=
ords); CHKERR(rval, "get_node_arrays");
+    // create quads
+  rval =3D read_iface->get_element_connect(nquads, 4, MBQUAD, 0, start_ele=
m, connect); CHKERR(rval, "get_element_connect");
+  for (int i =3D 0; i < nquads; i++) {
+    coords[0][2*i] =3D coords[0][2*i+1] =3D (double) i; // x values are al=
l i
+    coords[1][2*i] =3D 0.0; coords[1][2*i+1] =3D 1.0; // y coords
+    coords[2][2*i] =3D coords[2][2*i+1] =3D (double) 0.0; // z values, all=
 zero (2d mesh)
+    EntityHandle quad_v =3D start_vert + 2*i;
+    connect[4*i+0] =3D quad_v;
+    connect[4*i+1] =3D quad_v+2;
+    connect[4*i+2] =3D quad_v+3;
+    connect[4*i+3] =3D quad_v+1;
+  }
+ =20
+    // last two vertices
+  coords[0][2*nquads] =3D coords[0][2*nquads+1] =3D (double) nquads;
+  coords[1][2*nquads] =3D 0.0; coords[1][2*nquads+1] =3D 1.0; // y coords
+  coords[2][2*nquads] =3D coords[2][2*nquads+1] =3D (double) 0.0; // z val=
ues, all zero (2d mesh)
+ =20
+    // call a vertex-quad adjacencies function to generate vertex-element =
adjacencies in MOAB
+  Range dum_range;
+  rval =3D mbImpl->get_adjacencies(&start_vert, 1, 2, false, dum_range); C=
HKERR(rval, "get_adjacencies");
+  assert(!dum_range.empty());
+
+  return MB_SUCCESS;
+}
+
+   =20
+ =20

diff --git a/examples/DirectAccessNoHolesF90.F90 b/examples/DirectAccessNoH=
olesF90.F90
new file mode 100644
index 0000000..3cf5fa7
--- /dev/null
+++ b/examples/DirectAccessNoHolesF90.F90
@@ -0,0 +1,231 @@
+! @example DirectAccessNoHolesF90.F90 \n
+! \brief Use direct access to MOAB data to avoid calling through API, in F=
ortran90 \n
+!
+! This example creates a 1d row of quad elements, such that all quad and v=
ertex handles
+! are contiguous in the handle space and in the database.  Then it shows h=
ow to get access
+! to pointers to MOAB-native data for vertex coordinates, quad connectivit=
y, and tag storage
+! (vertex to quad adjacency lists aren't accessible from Fortran because t=
hey are std::vector's). =20
+! This allows applications to access this data directly
+! without going through MOAB's API.  In cases where the mesh is not changi=
ng (or only mesh
+! vertices are moving), this can save significant execution time in applic=
ations.
+!
+! Using direct access (or MOAB in general) from Fortran is complicated in =
two specific ways:
+! 1) There is no way to use arrays with specified dimension starting/endin=
g values with ISO_C_BINDING;
+!    therefore, all arrays passed back from MOAB/iMesh must use 1-based in=
dices; this makes it a bit
+!    more difficult to traverse the direct arrays.  In this example, I exp=
licitly use indices that
+!    look like my_array(1+v_ind...) to remind users of this.
+! 2) Arithmetic on handles is complicated by the fact that Fortran has no =
unsigned integer type.  I get
+!    around this by assigning to a C-typed variable first, before the hand=
le arithmetic.  This seems to
+!    work fine.  C-typed variables are part of the Fortran95 standard.
+!
+!  ----------------------
+!  |      |      |      |      =20
+!  |      |      |      | ...
+!  |      |      |      |
+!  ----------------------
+!
+!    -#  Initialize MOAB \n
+!    -#  Create a quad mesh, as depicted above
+!    -#  Create 2 dense tags (tag1, tag2) for avg position to assign to qu=
ads, and # verts per quad (tag3)
+!    -#  Get connectivity, coordinate, tag1 iterators
+!    -#  Iterate through quads, computing midpoint based on vertex positio=
ns, set on quad-based tag1
+!    -#  Iterate through vertices, summing positions into tag2 on connecte=
d quads and incrementing vertex count
+!    -#  Iterate through quads, normalizing tag2 by vertex count and compa=
ring values of tag1 and tag2
+!
+! <b>To compile</b>: \n
+!    make DirectAccessNoHolesF90 MOAB_DIR=3D<installdir>  \n
+! <b>To run</b>: ./DirectAccessNoHolesF90 [-nquads <# quads>]\n
+!
+!
+
+#define CHECK(a) \
+  if (a .ne. iBase_SUCCESS) call exit(a)
+
+module freem
+  interface
+     subroutine c_free(ptr) bind(C, name=3D'free')
+       use ISO_C_BINDING
+       type(C_ptr), value, intent(in) :: ptr
+     end subroutine c_free
+  end interface
+end module freem
+
+program DirectAccessNoHolesF90
+  use ISO_C_BINDING
+  use freem
+  implicit none
+#include "iMesh_f.h"
+
+  ! declarations
+  iMesh_Instance imesh
+  iBase_EntitySetHandle root_set
+  integer ierr, nquads, nquads_tmp, nverts
+  iBase_TagHandle tag1h, tag2h, tag3h
+  TYPE(C_PTR) verts_ptr, quads_ptr, conn_ptr, x_ptr, y_ptr, z_ptr, tag1_pt=
r, tag2_ptr, tag3_ptr, &
+       tmp_quads_ptr, offsets_ptr  ! pointers that are equivalence'd to ar=
rays using c_f_pointer
+  real(C_DOUBLE), pointer :: x(:), y(:), z(:), tag1(:), tag2(:) ! arrays e=
quivalence'd to pointers
+  integer, pointer :: tag3(:), offsets(:)                       ! arrays e=
quivalence'd to pointers
+  iBase_EntityHandle, pointer :: quads(:), verts(:), conn(:), tmp_quads(:)=
  ! arrays equivalence'd to pointers
+  iBase_EntityArrIterator viter, qiter
+  integer(C_SIZE_T) :: startv, startq, v_ind, e_ind  ! needed to do handle=
 arithmetic
+  integer count, vpere, e, i, j, v, offsets_size, tmp_quads_size, n_dis
+  character*120 opt
+
+  ! initialize interface and get root set
+  call iMesh_newMesh("", imesh, ierr); CHECK(ierr)
+  call iMesh_getRootSet(%VAL(imesh), root_set, ierr); CHECK(ierr)
+
+  ! create mesh
+  nquads_tmp =3D 1000
+  call create_mesh_no_holes(imesh, nquads_tmp, ierr); CHECK(ierr)
+
+  ! get all vertices and quads as arrays
+  nverts =3D 0
+  call iMesh_getEntities(%VAL(imesh), %VAL(root_set), %VAL(0), %VAL(iBase_=
VERTEX), &
+       verts_ptr, nverts, nverts, ierr); CHECK(ierr)
+  call c_f_pointer(verts_ptr, verts, [nverts])
+  nquads =3D 0
+  call iMesh_getEntities(%VAL(imesh), %VAL(root_set), %VAL(2), %VAL(iMesh_=
QUADRILATERAL), &
+       quads_ptr, nquads, nquads, ierr); CHECK(ierr)
+  call c_f_pointer(quads_ptr, quads, [nquads])
+
+  ! First vertex/quad is at start of vertex/quad list, and is offset for v=
ertex/quad index calculation
+  startv =3D verts(1); startq =3D quads(1)
+  call c_free(quads_ptr)  ! free memory we allocated in MOAB
+
+  ! create tag1 (element-based avg), tag2 (vertex-based avg)
+  opt =3D 'moab:TAG_STORAGE_TYPE=3DDENSE moab:TAG_DEFAULT_VALUE=3D0'
+  call iMesh_createTagWithOptions(%VAL(imesh), 'tag1', opt, %VAL(3), %VAL(=
iBase_DOUBLE), &
+       tag1h, ierr, %VAL(4), %VAL(56)); CHECK(ierr)
+  call iMesh_createTagWithOptions(%VAL(imesh), 'tag2', opt, %VAL(3), %VAL(=
iBase_DOUBLE), &
+       tag2h, ierr, %VAL(4), %VAL(56)); CHECK(ierr)
+
+  ! Get iterator to verts, then pointer to coordinates
+  call iMesh_initEntArrIter(%VAL(imesh), %VAL(root_set), %VAL(iBase_VERTEX=
), %VAL(iMesh_ALL_TOPOLOGIES), &
+       %VAL(nverts), %VAL(0), viter, ierr); CHECK(ierr)
+  call iMesh_coordsIterate(%VAL(imesh), %VAL(viter), x_ptr, y_ptr, z_ptr, =
count, ierr); CHECK(ierr)
+  CHECK(count-nverts) ! check that all verts are in one contiguous chunk
+  call c_f_pointer(x_ptr, x, [nverts]); call c_f_pointer(y_ptr, y, [nverts=
]); call c_f_pointer(z_ptr, z, [nverts])
+
+  ! Get iterator to quads, then pointers to connectivity and tags
+  call iMesh_initEntArrIter(%VAL(imesh), %VAL(root_set), %VAL(iBase_FACE),=
 %VAL(iMesh_ALL_TOPOLOGIES), &
+       %VAL(nquads), %VAL(0), qiter, ierr); CHECK(ierr)
+  call iMesh_connectIterate(%VAL(imesh), %VAL(qiter), conn_ptr, vpere, cou=
nt, ierr); CHECK(ierr)
+  call c_f_pointer(conn_ptr, conn, [vpere*nquads])
+  call iMesh_tagIterate(%VAL(imesh), %VAL(tag1h), %VAL(qiter), tag1_ptr, c=
ount, ierr); CHECK(ierr)
+  call c_f_pointer(tag1_ptr, tag1, [3*nquads])
+  call iMesh_tagIterate(%VAL(imesh), %VAL(tag2h), %VAL(qiter), tag2_ptr, c=
ount, ierr); CHECK(ierr)
+  call c_f_pointer(tag2_ptr, tag2, [3*nquads])
+ =20
+  ! iterate over elements, computing tag1 from coords positions; use (1+..=
. in array indices for 1-based indexing
+  do i =3D 0, nquads-1
+     tag1(1+3*i+0) =3D 0.0; tag1(1+3*i+1) =3D 0.0; tag1(1+3*i+2) =3D 0.0 !=
 initialize position for this element
+     do j =3D 0, vpere-1 ! loop over vertices in this quad
+        v_ind =3D conn(1+vpere*i+j) ! assign to v_ind to allow handle arit=
hmetic
+        v_ind =3D v_ind - startv ! vert index is just the offset from star=
t vertex
+        tag1(1+3*i+0) =3D tag1(1+3*i+0) + x(1+v_ind)
+        tag1(1+3*i+1) =3D tag1(1+3*i+1) + y(1+v_ind) ! sum vertex position=
s into tag1...
+        tag1(1+3*i+2) =3D tag1(1+3*i+2) + z(1+v_ind)
+     end do
+     tag1(1+3*i+0) =3D tag1(1+3*i+0) / vpere;
+     tag1(1+3*i+1) =3D tag1(1+3*i+1) / vpere; ! then normalize
+     tag1(1+3*i+2) =3D tag1(1+3*i+2) / vpere;
+  end do ! loop over elements in chunk
+   =20
+  ! Iterate through vertices, summing positions into tag2 on connected ele=
ments; get adjacencies all at once,
+  ! could also get them per vertex (time/memory tradeoff)
+  tmp_quads_size =3D 0
+  offsets_size =3D 0
+  call iMesh_getEntArrAdj(%VAL(imesh), verts, %VAL(nverts), %VAL(iMesh_QUA=
DRILATERAL), &
+       tmp_quads_ptr, tmp_quads_size, tmp_quads_size, &
+       offsets_ptr, offsets_size, offsets_size, ierr); CHECK(ierr)=20
+  call c_f_pointer(tmp_quads_ptr, tmp_quads, [tmp_quads_size])
+  call c_f_pointer(offsets_ptr, offsets, [offsets_size])
+  call c_free(verts_ptr)  ! free memory we allocated in MOAB
+  do v =3D 0, nverts-1
+     do e =3D 1+offsets(1+v), 1+offsets(1+v+1)-1  ! 1+ because offsets are=
 in terms of 0-based arrays
+        e_ind =3D tmp_quads(e) ! assign to e_ind to allow handle arithmetic
+        e_ind =3D e_ind - startq ! e_ind is the quad handle minus the star=
ting quad handle
+        tag2(1+3*e_ind+0) =3D tag2(1+3*e_ind+0) + x(1+v)
+        tag2(1+3*e_ind+1) =3D tag2(1+3*e_ind+1) + y(1+v)   ! tag on each e=
lement is 3 doubles, x/y/z
+        tag2(1+3*e_ind+2) =3D tag2(1+3*e_ind+2) + z(1+v)
+     end do
+  end do
+  call c_free(tmp_quads_ptr)  ! free memory we allocated in MOAB
+  call c_free(offsets_ptr)    ! free memory we allocated in MOAB
+
+  ! Normalize tag2 by vertex count (vpere); loop over elements using same =
approach as before
+  ! At the same time, compare values of tag1 and tag2
+  n_dis =3D 0
+  do e =3D 0, nquads-1
+     do j =3D 0, 2
+        tag2(1+3*e+j) =3D tag2(1+3*e+j) / vpere;  ! normalize by # verts
+     end do
+     if (tag1(1+3*e) .ne. tag2(1+3*e) .or. tag1(1+3*e+1) .ne. tag2(1+3*e+1=
) .or. tag1(1+3*e+2) .ne. tag2(1+3*i+2)) then
+        print *, "Tag1, tag2 disagree for element ", e
+        print *, "tag1: ", tag1(1+3*e), tag1(1+3*e+1), tag1(1+3*e+2)
+        print *, "tag2: ", tag2(1+3*e), tag2(1+3*e+1), tag2(1+3*e+2)
+        n_dis =3D n_dis + 1
+     end if
+  end do
+  if (n_dis .eq. 0) print *, 'All tags agree, success!'
+
+    ! Ok, we are done, shut down MOAB
+  call iMesh_dtor(%VAL(imesh), ierr)
+  return
+end program DirectAccessNoHolesF90
+
+subroutine create_mesh_no_holes(imesh, nquads, ierr)=20
+  use ISO_C_BINDING
+  use freem
+  implicit none
+#include "iMesh_f.h"
+
+  iMesh_Instance imesh
+  integer nquads, ierr
+
+  real(C_DOUBLE), pointer :: coords(:,:)
+  TYPE(C_PTR) connect_ptr, tmp_ents_ptr, stat_ptr
+  iBase_EntityHandle, pointer :: connect(:), tmp_ents(:)
+  integer, pointer :: stat(:)
+  integer nverts, tmp_size, stat_size, i
+
+  ! first make the mesh, a 1d array of quads with left hand side x =3D ele=
m_num; vertices are numbered in layers
+  ! create verts, num is 2(nquads+1) because they're in a 1d row
+  nverts =3D 2*(nquads+1)
+  allocate(coords(0:2, 0:nverts-1))
+  do i =3D 0, nquads-1
+     coords(0,2*i) =3D i; coords(0,2*i+1) =3D i ! x values are all i
+     coords(1,2*i) =3D 0.0; coords(1,2*i+1) =3D 1.0 ! y coords
+     coords(2,2*i) =3D 0.0; coords(2,2*i+1) =3D 0.0 ! z values, all zero (=
2d mesh)
+  end do
+  ! last two vertices
+  coords(0, nverts-2) =3D nquads; coords(0, nverts-1) =3D nquads
+  coords(1, nverts-2) =3D 0.0; coords(1, nverts-1) =3D 1.0 ! y coords
+  coords(2, nverts-2) =3D 0.0; coords(2, nverts-1) =3D 0.0 ! z values, all=
 zero (2d mesh)
+  tmp_size =3D 0
+  call iMesh_createVtxArr(%VAL(imesh), %VAL(nverts), %VAL(iBase_INTERLEAVE=
D), &
+       coords, %VAL(3*nverts), tmp_ents_ptr, tmp_size, tmp_size, ierr); CH=
ECK(ierr)
+  call c_f_pointer(tmp_ents_ptr, tmp_ents, [nverts])
+  deallocate(coords)
+  allocate(connect(0:4*nquads-1))
+  do i =3D 0, nquads-1
+     connect(4*i+0) =3D tmp_ents(1+2*i)
+     connect(4*i+1) =3D tmp_ents(1+2*i+2)
+     connect(4*i+2) =3D tmp_ents(1+2*i+3)
+     connect(4*i+3) =3D tmp_ents(1+2*i+1)
+  end do
+  stat_size =3D 0
+  stat_ptr =3D C_NULL_PTR
+  ! re-use tmp_ents here; we know it'll always be larger than nquads so it=
's ok
+  call iMesh_createEntArr(%VAL(imesh), %VAL(iMesh_QUADRILATERAL), connect,=
 %VAL(4*nquads), &
+       tmp_ents_ptr, tmp_size, tmp_size, stat_ptr, stat_size, stat_size, i=
err); CHECK(ierr)
+
+  ierr =3D iBase_SUCCESS
+
+  call c_free(tmp_ents_ptr)
+  call c_free(stat_ptr)
+  deallocate(connect)
+
+  return
+end subroutine create_mesh_no_holes

diff --git a/examples/DirectAccessWithHoles.cpp b/examples/DirectAccessWith=
Holes.cpp
new file mode 100644
index 0000000..524c30f
--- /dev/null
+++ b/examples/DirectAccessWithHoles.cpp
@@ -0,0 +1,239 @@
+/** @example DirectAccess.cpp \n
+ * \brief Use direct access to MOAB data to avoid calling through API \n
+ *
+ * This example creates a 1d row of quad elements, with a user-specified n=
umber of "holes" (missing quads) in the row:
+ *
+ *  ----------------------      ----------------------      --------
+ *  |      |      |      |      |      |      |      |      |      |      =20
+ *  |      |      |      |(hole)|      |      |      |(hole)|      | ...
+ *  |      |      |      |      |      |      |      |      |      |
+ *  ----------------------      ----------------------      --------
+ *
+ * This makes (nholes+1) contiguous runs of quad handles in the handle spa=
ce
+ * This example shows how to use the xxx_iterate functions in MOAB (xxx =
=3D coords, connect, tag, adjacencies) to get=20
+ * direct pointer access to MOAB internal storage, which can be used witho=
ut calling through the MOAB API.
+ *
+ *    -#  Initialize MOAB \n
+ *    -#  Create a quad mesh with holes, as depicted above
+ *    -#  Create 2 dense tags (tag1, tag2) for avg position to assign to q=
uads, and # verts per quad (tag3)
+ *    -#  Get connectivity, coordinate, tag1 iterators
+ *    -#  Iterate through quads, computing midpoint based on vertex positi=
ons, set on quad-based tag1
+ *    -#  Set up map from starting quad handle for a chunk to struct of (t=
ag1_ptr, tag2_ptr, tag3_ptr), pointers to
+ *        the dense tag storage for those tags for the chunk
+ *    -#  Iterate through vertices, summing positions into tag2 on connect=
ed quads and incrementing vertex count
+ *    -#  Iterate through quads, normalizing tag2 by vertex count and comp=
aring values of tag1 and tag2
+ *
+ * <b>To compile</b>: \n
+ *    make DirectAccessWithHoles MOAB_DIR=3D<installdir>  \n
+ * <b>To run</b>: ./DirectAccess [-nquads <# quads>] [-holes <# holes>]\n
+ *
+ */
+
+#include "moab/Core.hpp"
+#include "moab/ProgOptions.hpp"
+#include "moab/ReadUtilIface.hpp"
+#include <map>
+#include <iostream>
+#include <assert.h>
+
+// Error routines for use with MOAB API
+#define CHKERR(CODE, MSG) do {if (MB_SUCCESS !=3D (CODE)) {std::string err=
str;  mbImpl->get_last_error(errstr);   \
+        std::cerr << errstr << std::endl; std::cerr << MSG << std::endl; r=
eturn CODE;}} while(false)
+
+using namespace moab;
+
+ErrorCode create_mesh_with_holes(Interface *mbImpl, int nquads, int nholes=
);
+
+struct tag_struct {double *avg_ptr; int *nv_ptr;};
+
+int main(int argc, char **argv)
+{
+    // get MOAB instance
+  Interface *mbImpl =3D new Core;
+  if (NULL =3D=3D mbImpl) return 1;
+
+  int nquads =3D 1000, nholes =3D 1;
+ =20
+    // parse options
+  ProgOptions opts;
+  opts.addOpt<int>(std::string("nquads,n"), std::string("Number of quads i=
n the mesh (default =3D 1000"), &nquads);
+  opts.addOpt<int>(std::string("holes,H"), std::string("Number of holes in=
 the element handle space (default =3D 1"), &nholes);
+  opts.parseCommandLine(argc, argv);
+  if (nholes >=3D nquads) {
+    std::cerr << "Number of holes needs to be < number of elements." << st=
d::endl;
+    return 1;
+  }
+
+    // create simple structured mesh with hole, but using unstructured rep=
resentation
+  ErrorCode rval =3D create_mesh_with_holes(mbImpl, nquads, nholes); CHKER=
R(rval, "Trouble creating mesh.");
+ =20
+    // get all vertices and non-vertex entities
+  Range verts, elems;
+  rval =3D mbImpl->get_entities_by_handle(0, elems); CHKERR(rval, "Getting=
 all entities.");
+  verts =3D elems.subset_by_dimension(0);
+  elems -=3D verts;
+
+    // create tag1 (element-based avg), tag2 (vertex-based avg), tag3 (# c=
onnected verts)
+  Tag tag1, tag2, tag3;
+  rval =3D mbImpl->tag_get_handle("tag1", 3, MB_TYPE_DOUBLE, tag1, MB_TAG_=
DENSE | MB_TAG_CREAT); CHKERR(rval, "Creating tag1.");
+  double def_val[3] =3D {0.0, 0.0, 0.0}; // need a default value for tag2 =
because we sum into it
+  rval =3D mbImpl->tag_get_handle("tag2", 3, MB_TYPE_DOUBLE, tag2, MB_TAG_=
DENSE | MB_TAG_CREAT, def_val); CHKERR(rval, "Creating tag2.");
+  int def_val_int =3D 0;  // need a default value for tag3 because we incr=
ement it
+  rval =3D mbImpl->tag_get_handle("tag3", 1, MB_TYPE_INTEGER, tag3, MB_TAG=
_DENSE | MB_TAG_CREAT, &def_val_int); CHKERR(rval, "Creating tag3.");
+ =20
+    // Get connectivity, coordinate, tag, and adjacency iterators
+  EntityHandle *conn_ptr;
+  double *x_ptr, *y_ptr, *z_ptr, *tag1_ptr, *tag2_ptr;
+  int *tag3_ptr;
+ =20
+    // First vertex is at start of range (ranges are sorted), and is offse=
t for vertex index calculation
+  EntityHandle first_vert =3D *verts.begin();
+
+    // When iterating over elements, each chunk can have a different # ver=
tices; also, count tells you how many
+    // elements are in the current chunk
+  int vpere, count;
+ =20
+    // Get coordinates iterator, just need this once because we know verts=
 handle space doesn't have holes
+  rval =3D mbImpl->coords_iterate(verts.begin(), verts.end(), x_ptr, y_ptr=
, z_ptr, count); CHKERR(rval, "coords_iterate.");
+  assert(count =3D=3D (int) verts.size()); // should end up with just one =
contiguous chunk of vertices
+
+    // Iterate through elements, computing midpoint based on vertex positi=
ons, set on element-based tag1
+    // Control loop by iterator over elem range
+  Range::iterator e_it =3D elems.begin();
+
+  while (e_it !=3D elems.end()) {
+      // get conn_ptr, tag1_ptr for next contiguous chunk of element handl=
es, and coords pointers for all verts
+    rval =3D mbImpl->connect_iterate(e_it, elems.end(), conn_ptr, vpere, c=
ount); CHKERR(rval, "connect_iterate.");
+    rval =3D mbImpl->tag_iterate(tag1, e_it, elems.end(), count, (void*&)t=
ag1_ptr); CHKERR(rval, "tag1_iterate.");
+
+      // iterate over elements in this chunk=20
+    for (int i =3D 0; i < count; i++) {
+      tag1_ptr[0] =3D tag1_ptr[1] =3D tag1_ptr[2] =3D 0.0; // initialize p=
osition for this element
+      for (int j =3D 0; j < vpere; j++) { // loop over vertices in this el=
ement
+        int v_index =3D conn_ptr[j] - first_vert; // vert index is just th=
e offset from first vertex
+        tag1_ptr[0] +=3D x_ptr[v_index];
+        tag1_ptr[1] +=3D y_ptr[v_index]; // sum vertex positions into tag1=
...
+        tag1_ptr[2] +=3D z_ptr[v_index];
+      }
+      tag1_ptr[0] /=3D vpere;
+      tag1_ptr[1] /=3D vpere; // then normalize
+      tag1_ptr[2] /=3D vpere;
+     =20
+        // done with this element; advance connect_ptr and tag1_ptr to nex=
t element
+      conn_ptr +=3D vpere;
+      tag1_ptr +=3D 3;
+    } // loop over elements in chunk
+   =20
+      // done with chunk; advance range iterator by count; will skip over =
gaps in range
+    e_it +=3D count;
+  } // while loop over all elements
+ =20
+    // Iterate through vertices, summing positions into tag2 on connected =
elements and incrementing vertex count
+    // Iterate over chunks the same as elements, even though we know we ha=
ve only one chunk here, just to show
+    // how it's done
+
+    // Create a std::map from EntityHandle (first entity handle in chunk) =
to=20
+    // tag_struct (ptrs to start of avg/#verts tags for that chunk); then =
for a given entity handle, we can quickly
+    // find the chunk it's in using map::lower_bound; could have set up th=
is map in earlier loop over elements, but do
+    // it here for clarity
+ =20
+  std::map< EntityHandle, tag_struct> elem_map;
+  e_it =3D elems.begin();
+  while (e_it !=3D elems.end()) {
+    tag_struct ts =3D {NULL, NULL};
+    rval =3D mbImpl->tag_iterate(tag2, e_it, elems.end(), count, (void*&)t=
s.avg_ptr); CHKERR(rval, "tag2_iterate.");
+    rval =3D mbImpl->tag_iterate(tag3, e_it, elems.end(), count, (void*&)t=
s.nv_ptr); CHKERR(rval, "tag3_iterate.");
+    elem_map[*e_it] =3D ts;
+    e_it +=3D count;
+  }
+
+    // call a vertex-quad adjacencies function to generate vertex-element =
adjacencies in MOAB
+  Range::iterator v_it =3D verts.begin();
+  Range dum_range;
+  rval =3D mbImpl->get_adjacencies(&(*v_it), 1, 2, false, dum_range); CHKE=
RR(rval, "get_adjacencies");
+  const std::vector<EntityHandle> **adjs_ptr;
+  while (v_it !=3D verts.end()) {
+      // get coords ptrs, adjs_ptr; can't set tag2_ptr by direct access, b=
ecause of hole in element handle space
+    rval =3D mbImpl->coords_iterate(v_it, verts.end(), x_ptr, y_ptr, z_ptr=
, count); CHKERR(rval, "coords_iterate.");
+    rval =3D mbImpl->adjacencies_iterate(v_it, verts.end(), adjs_ptr, coun=
t); CHKERR(rval, "adjacencies_iterate.");
+   =20
+    for (int v =3D 0; v < count; v++) {
+      const std::vector<EntityHandle> *avec =3D *(adjs_ptr+v);
+      for (std::vector<EntityHandle>::const_iterator ait =3D avec->begin()=
; ait !=3D avec->end(); ait++) {
+          // get chunk that this element resides in; upper_bound points to=
 the first element strictly > key, so get that and decrement
+          // (would work the same as lower_bound with an if-test and decre=
ment)
+        std::map<EntityHandle, tag_struct>::iterator mit =3D elem_map.uppe=
r_bound(*ait); mit--;
+          // index of *ait in that chunk
+        int a_ind =3D *ait - (*mit).first;
+        tag_struct ts =3D (*mit).second;
+        ts.avg_ptr[3*a_ind]   +=3D x_ptr[v];   // tag on each element is 3=
 doubles, x/y/z
+        ts.avg_ptr[3*a_ind+1] +=3D y_ptr[v];
+        ts.avg_ptr[3*a_ind+2] +=3D z_ptr[v];
+        ts.nv_ptr[a_ind]++;  // increment the vertex count
+      }
+    }
+
+    v_it +=3D count;
+  }
+       =20
+    // Normalize tag2 by vertex count; loop over elements using same appro=
ach as before
+    // At the same time, compare values of tag1 and tag2
+  e_it =3D elems.begin();
+  while (e_it !=3D elems.end()) {
+      // get conn_ptr, tag1_ptr for next contiguous chunk of element handl=
es, and coords pointers for all verts
+    rval =3D mbImpl->tag_iterate(tag1, e_it, elems.end(), count, (void*&)t=
ag1_ptr); CHKERR(rval, "tag1_iterate.");
+    rval =3D mbImpl->tag_iterate(tag2, e_it, elems.end(), count, (void*&)t=
ag2_ptr); CHKERR(rval, "tag2_iterate.");
+    rval =3D mbImpl->tag_iterate(tag3, e_it, elems.end(), count, (void*&)t=
ag3_ptr); CHKERR(rval, "tag3_iterate.");
+
+      // iterate over elements in this chunk=20
+    for (int i =3D 0; i < count; i++) {
+      for (int j =3D 0; j < 3; j++) tag2_ptr[3*i+j] /=3D (double)tag3_ptr[=
i];  // normalize by # verts
+      if (tag1_ptr[3*i] !=3D tag2_ptr[3*i] || tag1_ptr[3*i+1] !=3D tag2_pt=
r[3*i+1] || tag1_ptr[3*i+2] !=3D tag2_ptr[3*i+2])=20
+        std::cout << "Tag1, tag2 disagree for element " << *e_it + i << st=
d::endl;
+    }
+
+    e_it +=3D count;
+  }
+
+    // Ok, we're done, shut down MOAB
+  delete mbImpl;
+
+  return 0;
+}
+
+ErrorCode create_mesh_with_holes(Interface *mbImpl, int nquads, int nholes=
)=20
+{
+    // first make the mesh, a 1d array of quads with left hand side x =3D =
elem_num; vertices are numbered in layers
+  ReadUtilIface *read_iface;
+  ErrorCode rval =3D mbImpl->query_interface(read_iface); CHKERR(rval, "qu=
ery_interface");
+  std::vector<double *> coords;
+  EntityHandle start_vert, start_elem, *connect;
+    // create verts, num is 4(nquads+1) because they're in a 1d row; will =
initialize coords in loop over elems later
+  rval =3D read_iface->get_node_coords (3, 2*(nquads+1), 0, start_vert, co=
ords); CHKERR(rval, "get_node_arrays");
+    // create elems
+  rval =3D read_iface->get_element_connect(nquads, 4, MBQUAD, 0, start_ele=
m, connect); CHKERR(rval, "get_element_connect");
+  for (int i =3D 0; i < nquads; i++) {
+    coords[0][2*i] =3D coords[0][2*i+1] =3D (double) i; // x values are al=
l i
+    coords[1][2*i] =3D 0.0; coords[1][2*i+1] =3D 1.0; // y coords
+    coords[2][2*i] =3D coords[2][2*i+1] =3D (double) 0.0; // z values, all=
 zero (2d mesh)
+    EntityHandle quad_v =3D start_vert + 2*i;
+    for (int j =3D 0; j < 4; j++) connect[4*i+j] =3D quad_v+j; // connecti=
vity of each quad is a sequence starting from quad_v
+  }
+    // last two vertices
+  coords[0][2*nquads] =3D coords[0][2*nquads+1] =3D (double) nquads;
+  coords[1][2*nquads] =3D 0.0; coords[1][2*nquads+1] =3D 1.0; // y coords
+  coords[2][2*nquads] =3D coords[2][2*nquads+1] =3D (double) 0.0; // z val=
ues, all zero (2d mesh)
+ =20
+    // now delete nholes elements, spaced approximately equally through me=
sh, so contiguous size is about nquads/(nholes+1)
+    // reinterpret start_elem as the next element to be deleted
+  int de =3D nquads / (nholes + 1);
+  for (int i =3D 0; i < nholes; i++) {
+    start_elem +=3D de;
+    rval =3D mbImpl->delete_entities(&start_elem, 1); CHKERR(rval, "delete=
_entities");
+  }
+ =20
+  return MB_SUCCESS;
+}
+
+   =20
+ =20

diff --git a/examples/HelloMOAB.cpp b/examples/HelloMOAB.cpp
index ee81de9..ebab971 100644
--- a/examples/HelloMOAB.cpp
+++ b/examples/HelloMOAB.cpp
@@ -8,13 +8,15 @@
=20
=20
 #include "moab/Core.hpp"
+#include <iostream>
+#include <assert.h>
=20
 using namespace moab;
 using namespace std;
=20
 string test_file_name =3D string(MESH_DIR) + string("/3k-tri-sphere.vtk");
=20
-int main( int argc, char** argv[] )
+int main( int argc, char** argv )
 {
   Interface *iface =3D new Core;
=20
@@ -24,7 +26,7 @@ int main( int argc, char** argv[] )
     test_file_name =3D argv[1];
   } =20
     //load the mesh from vtk file
-  ErrorCode rval =3D iface->load_mesh( test_file_name );
+  ErrorCode rval =3D iface->load_mesh( test_file_name.c_str() );
   assert( rval =3D=3D MB_SUCCESS);
=20
     //get verts entities

diff --git a/examples/makefile b/examples/makefile
index 4ec6154..3402f96 100644
--- a/examples/makefile
+++ b/examples/makefile
@@ -1,7 +1,8 @@
 # MOAB_DIR points to top-level install dir, below which MOAB's lib/ and in=
clude/ are located
 include ${MOAB_DIR}/lib/moab.make
+include ${MOAB_DIR}/lib/iMesh-Defs.inc
=20
-.SUFFIXES: .o .cpp
+.SUFFIXES: .o .cpp .F90
=20
 # MESH_DIR is the top-level MOAB source directory, used to locate mesh fil=
es that come with MOAB source
 MESH_DIR=3D"../MeshFiles/unittest"
@@ -21,6 +22,18 @@ HelloMoabPar: HelloMoabPar.o
 TestExodusII: TestExodusII.o
 	${MOAB_CXX} -o $@ $< ${MOAB_LIBS_LINK}
=20
+DirectAccessWithHoles: DirectAccessWithHoles.o
+	${MOAB_CXX} -o $@ $< ${MOAB_LIBS_LINK}
+
+DirectAccessNoHoles: DirectAccessNoHoles.o
+	${MOAB_CXX} -o $@ $< ${MOAB_LIBS_LINK}
+
+DirectAccessNoHolesF90: DirectAccessNoHolesF90.o
+	${MOAB_CXX} -o $@ $< ${IMESH_LIBS}
+
 .cpp.o :
 	${MOAB_CXX} ${MOAB_CXXFLAGS} ${MOAB_INCLUDES} -DMESH_DIR=3D\"${MESH_DIR}\=
" -c $<
=20
+.F90.o :
+	${IMESH_FC} ${IMESH_FCFLAGS} ${IMESH_INCLUDES} ${IMESH_FCDEFS} -DMESH_DIR=
=3D\"${MESH_DIR}\" -c $<
+

diff --git a/itaps/iBase_f.h.in b/itaps/iBase_f.h.in
index 4a70fd3..90fde77 100644
--- a/itaps/iBase_f.h.in
+++ b/itaps/iBase_f.h.in
@@ -50,7 +50,27 @@
       parameter (iBase_REGION =3D 3)
       parameter (iBase_ALL_TYPES =3D 4)
=20
-
+      integer iBase_AdjacencyCost_MIN
+      integer iBase_UNAVAILABLE
+      integer iBase_ALL_ORDER_1
+      integer iBase_ALL_ORDER_LOGN
+      integer iBase_ALL_ORDER_N
+      integer iBase_SOME_ORDER_1
+      integer iBase_SOME_ORDER_LOGN
+      integer iBase_SOME_ORDER_N
+      integer iBase_AVAILABLE
+      integer iBase_AdjacencyCost_MAX
+
+      parameter (iBase_AdjacencyCost_MIN =3D 0)
+      parameter (iBase_UNAVAILABLE =3D 0)
+      parameter (iBase_ALL_ORDER_1 =3D 1)
+      parameter (iBase_ALL_ORDER_LOGN =3D 2)
+      parameter (iBase_ALL_ORDER_N =3D 3)
+      parameter (iBase_SOME_ORDER_1 =3D 4)
+      parameter (iBase_SOME_ORDER_LOGN =3D 5)
+      parameter (iBase_SOME_ORDER_N =3D 6)
+      parameter (iBase_AVAILABLE =3D 7)
+      parameter (iBase_AdjacencyCost_MAX =3D 7)
=20
       integer iBase_NEW
       integer iBase_ALREADY_EXISTED

diff --git a/itaps/imesh/FindAdjacencyF90.F90 b/itaps/imesh/FindAdjacencyF9=
0.F90
index 78578a7..8115ef1 100644
--- a/itaps/imesh/FindAdjacencyF90.F90
+++ b/itaps/imesh/FindAdjacencyF90.F90
@@ -69,7 +69,7 @@ program findadjacency
=20
      vert_uses =3D vert_uses + iverts_size
=20
-     if (iverts_size .ne. 0) call free(rpverts)
+     if (iverts_size .ne. 0) call iMesh_freeMemory(%VAL(mesh), rpverts)
   end do
=20
   ! now get adjacencies in one big block
@@ -81,9 +81,9 @@ program findadjacency
        offsets_size, ierr)
   CHECK("Failure in getEntArrAdj")
=20
-  if (allverts_size .ne. 0) call free(rpallverts);
-  if (offsets_size .ne. 0) call free(ipoffsets);
-  if (ents_size .ne. 0) call free(rpents);
+  if (allverts_size .ne. 0) call iMesh_freeMemory(%VAL(mesh), rpallverts);
+  if (offsets_size .ne. 0) call iMesh_freeMemory(%VAL(mesh), ipoffsets);
+  if (ents_size .ne. 0) call iMesh_freeMemory(%VAL(mesh), rpents);
=20
   ! compare results of two calling methods
   if (allverts_size .ne. vert_uses) then
@@ -119,6 +119,9 @@ program findadjacency
 !
   write(*, *) "num coords: ", ents_size, " few coords: ", (coords(i), i=3D=
0, ents_size/100) =20
=20
+  call iMesh_freeMemory(%VAL(mesh), verths);
+  call iMesh_freeMemory(%VAL(mesh), pcoord);
+
   call iMesh_dtor(%VAL(mesh), ierr)
   CHECK("Failed to destroy interface")
=20

This diff is so big that we needed to truncate the remainder.

https://bitbucket.org/fathomteam/moab/commits/c424740b8ae1/
Changeset:   c424740b8ae1
Branch:      None
User:        vijaysm
Date:        2013-06-24 19:28:02
Summary:     Merge remote-tracking branch 'upstream/master'

Affected #:  3 files

diff --git a/doc/DG/moabDG.h b/doc/DG/moabDG.h
new file mode 100644
index 0000000..d2e14f0
--- /dev/null
+++ b/doc/DG/moabDG.h
@@ -0,0 +1,259 @@
+/*! \page developerguide Developer's Guide (MOAB 4.6)
+=20
+  \subpage dg-contents
+=20
+  \subpage dg-figures
+
+  \page dg-contents Table of Contents
+
+  \ref sequence
+
+  \ref manager
+
+  \ref s-mesh
+
+  \ref sets
+
+  \section sequence  1. EntitySequence & SequenceData
+
+  \subsection figure1 Figure 1: EntitySequences For One SequenceData
+
+The <I>SequenceData</I> class manages as set of arrays of per-entity value=
s. Each
+<I>SequenceData</I> has a start and end handle denoting the block of entit=
ies for which
+the arrays contain data. The arrays managed by a <I>SequenceData</I> insta=
nce are
+divided into three groups:
+
+- Type-specific data (connectivity, coordinates, etc.): zero or more array=
s.
+- Adjacency data: zero or one array.
+- Dense tag data: zero or more arrays.
+.
+
+The abstract <I>EntitySequence</I> class is a non-strict subset of a <I>Se=
quenceData</I>.
+It contains a pointer to a <I>SequenceData</I> and the start and end handl=
es to indi-
+cate the subset of the referenced <I>SequenceData</I>. The <I>EntitySequen=
ce</I> class is
+used to represent the regions of valid (or allocated) handles in a <I>Sequ=
enceData</I>.
+A <I>SequenceData</I> is expected to be referenced by one or more <I>Entit=
ySequence</I>
+instances.
+
+Initial <I>EntitySequence</I> and <I>SequenceData</I> pairs are typically =
created in one
+of two configurations. When reading from a file, a <I>SequenceData</I> wil=
l be created
+to represent all of a single type of entity contained in a file. As all en=
tries in the <I>SequenceData</I> correspond to valid handles (entities read=
 from the file) a single
+<I>EntitySequence</I> instance corresponding to the entire <I>SequenceData=
</I> is initially
+created. The second configuration arises when allocating a single entity. =
If no
+entities have been allocated yet, a new <I>SequenceData</I> must be create=
d to store
+the entity data. It is created with a constant size (e.g. 4k entities). Th=
e new
+<I>EntitySequence</I> corresponds to only the first entity in the <I>Seque=
nceData</I>: the
+one allocated entity. As subsequent entities are allocated, the <I>EntityS=
equence</I>
+is extended to cover more of the corresponding <I>SequenceData</I>.
+
+Concrete subclasses of the <I>EntitySequence</I> class are responsible for=
 rep-
+resenting specific types of entities using the array storage provided by t=
he
+<I>SequenceData</I> class. They also handle allocating <I>SequenceData</I>=
 instances with
+appropriate arrays for storing a particular type of entity. Each concrete =
subclass
+typically provides two constructors corresponding to the two initial alloc=
ation
+configurations described in the previous paragraph. <I>EntitySequence</I> =
imple-
+mentations also provide a split method, which is a type of factory method.=
 It
+modifies the called sequence and creates a new sequence such that the rang=
e of
+entities represented by the original sequence is split.
+
+The <I>VertexSequence</I> class provides an <I>EntitySequence</I> for stor=
ing ver-
+tex data. It references a SequenceData containing three arrays of doubles
+for storing the blocked vertex coordinate data. The <I>ElementSequence</I>=
 class
+extends the <I>EntitySequence</I> interface with element-specific function=
ality. The
+<I>UnstructuredElemSeq</I> class is the concrete implementation of <I>Elem=
entSequence</I>
+used to represent unstructured elements, polygons, and polyhedra. <I>MeshS=
etSequence</I>
+is the <I>EntitySequence</I> used for storing entity sets.
+
+Each <I>EntitySequence</I> implementation also provides an implementation =
of
+the values per entity method. This value is used to determine if an exist-
+ing <I>SequenceData</I> that has available entities is suitable for storin=
g a particular
+entity. For example, <I>UnstructuredElemSeq</I> returns the number of node=
s per el-
+ement from values per entity. When allocating a new element with a specific
+number of nodes, this value is used to determine if that element may be st=
ored
+in a specific <I>SequenceData</I>. For vertices, this value is always zero=
. This could
+be changed to the number of coordinates per vertex, allowing representatio=
n of
+mixed-dimension data. However, API changes would be required to utilize su=
ch
+a feature. Sequences for which the corresponding data cannot be used to st=
ore
+new entities (e.g. structured mesh discussed in a later section) will retu=
rn -1 or
+some other invalid value.
+
+ \ref dg-contents
+
+  \section manager 2. TypeSequenceManager & SequenceManager=20
+
+The <I>TypeSequenceManager</I> class maintains an organized set of <I>Enti=
tySequence</I>
+instances and corresponding <I>SequenceData</I> instances. It is used to m=
anage
+all such instances for entities of a single <I>EntityType</I>. <I>TypeSequ=
enceManager</I>
+enforces the following four rules on its contained data:
+
+-# No two <I>SequenceData</I> instances may overlap. =20
+-# No two <I>EntitySequence</I> instances may overlap.
+-# Every <I>EntitySequence</I> must be a subset of a <I>SequenceData</I>.
+-# Any pair of <I>EntitySequence</I> instances referencing the same <I>Seq=
uenceData</I> must be separated by at least one unallocated handle.
+.
+
+  \subsection figure2 Figure 2: SequenceManager and Related Classes
+
+The first three rules are required for the validity of the data model. The
+fourth rule avoids unnecessary inefficiency. It is implemented by merging =
such
+adjacent sequences. In some cases, other classes (e.g. <I>SequenceManager<=
/I>) can
+modify an <I>EntitySequence</I> such that the fourth rule is violated. In =
such cases,
+the <I>TypeSequenceManager::notify</I> prepended or <I>TypeSequenceManager=
::notify</I> appended
+method must be called to maintain the integrity of the data<sup>1</sup>. T=
he above rules
+(including the fourth) are assumed in many other methods of the <I>TypeSeq=
uenceManager</I>
+class, such that those methods will fail or behave unexpectedly if the man=
aged
+data does not conform to the rules.
+
+<I>TypeSequenceManager</I> contains three principal data structures. The f=
irst is
+a <I>std::set</I> of <I>EntitySequence</I> pointers sorted using a custom =
comparison op-
+erator that queries the start and end handles of the referenced sequences.=
 The
+comparison operation is defined as: <I>a->end_handle() < b->start_handle()=
</I>.
+This method of comparison has the advantage that a sequence corresponding =
to
+a specific handle can be located by searching the set for a =E2=80=9Cseque=
nce=E2=80=9D beginning
+and ending with the search value. The lower bound and find methods pro-
+vided by the library are guaranteed to return the sequence, if it exists. =
Using
+such a comparison operator will result in undefined behavior if the set co=
ntains
+overlapping sequences. This is acceptable, as rule two above prohibits such
+a configuration. However, some care must be taken in writing and modifying
+methods in <I>TypeSequenceManager</I> so as to avoid having overlapping se=
quences
+as a transitory state of some operation.
+
+The second important data member of <I>TypeSequenceManager</I> is a pointer
+to the last referenced <I>EntitySequence</I>. This =E2=80=9Ccached=E2=80=
=9D value is used to speed up
+searches by entity handle. This pointer is never null unless the sequence =
is empty.
+This rule is maintained to avoid unnecessary branches in fast query paths.=
 In
+cases where the last referenced sequence is deleted, <I>TypeSequenceManage=
r</I> will
+typically assign an arbitrary sequence (e.g. the first one) to the last re=
ferenced
+pointer.
+
+The third data member of <I>TypeSequenceManager</I> is a <I>std::set</I> o=
f <I>SequenceData</I>
+instances that are not completely covered by a <I>EntitySequence</I> insta=
nce<sup>2</sup>.
+This list is searched when allocating new handles. <I>TypeSequenceManager<=
/I> also
+embeds in each <I>SequenceData</I> instance a reference to the first corre=
sponding
+<I>EntitySequence</I> so that it may be located quickly from only the <I>S=
equenceData</I>
+pointer.
+
+The <I>SequenceManager</I> class contains an array of <I>TypeSequenceManag=
er</I> in-
+stances, one for each <I>EntityType</I>. It also provides all type-specifi=
c operations
+such as allocating the correct <I>EntitySequence</I> subtype for a given <=
I>EntityType</I>.
+
+<sup>1</sup>This source of potential error can be eliminated with changes =
to the entity set representation. This is discussed in a later section.
+
+<sup>2</sup>Given rule four for the data managed by a <I>TypeSequenceManag=
er</I>, any <I>SequenceData</I> for which all handles are allocated will be=
 referenced by exactly one <I>EntitySequence</I>.
+
+  \ref dg-contents
+
+ \section s-mesh 3. Structured Mesh
+
+Structured mesh storage is implemented using subclasses of <I>SequenceData=
</I>:
+<I>ScdElementData</I> and <I>ScdVertexData</I>. The <I>StructuredElementSe=
q</I> class is
+used to access the structured element connectivity. A standard <I>VertexSe=
quence</I>
+instance is used to access the ScdVertexData because the vertex data stora=
ge
+is the same as for unstructured mesh.
+
+  \ref dg-contents
+
+  \section sets 4. Entity Sets
+
+- MeshSetSequence
+
+The <I>MeshSetSequence</I> class is the same as most other subclasses of <=
I>EntitySequence</I>
+in that it utilizes SequenceData to store its data. A single array in the =
<I>SequenceData</I>
+is used to store instances of the MeshSet class, one per allocated <I>Enti=
tyHandle</I>.
+<I>SequenceData</I> allocates all of its managed arrays using malloc and f=
ree as
+simple arrays of bytes. <I>MeshSetSequence</I> does in-place construction =
and de-
+struction of <I>MeshSet</I> instances within that array. This is similar t=
o what is
+done by <I>std::vector</I> and other container classes that may own more s=
torage
+than is required at a given time for contained objects.
+
+- MeshSet
+
+  \subsection figure3 Figure 3: SequenceManager and Related Classes
+
+The <I>MeshSet</I> class is used to represent a single entity set instance=
 in MOAB.
+The class is optimized to minimize storage (further possible improvements =
in
+storage size are discussed later.)
+
+Figure 3 shows the memory layout of an instance of the <I>MeshSet</I> clas=
s.
+The flags member holds the set creation bit flags: <I>MESHSET_TRACK_OWNER<=
/I>,
+<I>MESHSET_SET</I>, and <I>MESHSET_ORDERED</I>. The presence of the <I>MES=
HSET_TRACK_OWNER</I>
+indicates that reverse links from the contained entities back to the ownin=
g set
+should be maintained in the adjacency list of each entity. The <I>MESHSET_=
SET</I>
+and <I>MESHSET_ORDERED</I> bits are mutually exclusive, and as such most c=
ode only
+tests for the <I>MESHSET_ORDERED</I>, meaning that in practice the <I>MESH=
SET_SET</I> bit is
+ignored. <I>MESHSET_ORDERED</I> indicates that the set may contain duplica=
te handles
+and that the order that the handles are added to the set should be preserv=
ed.
+In practice, such sets are stored as a simple list of handles. <I>MESHSET_=
SET</I> (or in
+practice, the lack of <I>MESHSET_ORDERED</I>) indicates that the order of =
the handles
+need not be preserved and that the set may not contain duplicate handles. =
Such
+sets are stored in a sorted range-compacted format similar to that of the =
Range
+class.
+
+The memory for storing contents, parents, and children are each handled in
+the same way. The data in the class is composed of a 2-bit =E2=80=98size=
=E2=80=99 field and two
+values, where the two values may either be two handles or two pointers. Th=
e size
+bit-fields are grouped together to reduce the required amount of memory. I=
f the
+numerical value of the 2-bit size field is 0 then the corresponding list i=
s empty.
+If the 2-bit size field is either 1 or 2, then the contents of the corresp=
onding list
+are stored directly in the corresponding two data fields of the MeshSet ob=
ject.
+If the 2-bit size field has a value of 3 (11 binary), then the correspondi=
ng two
+data fields store the begin and end pointers of an external array of handl=
es.
+The number of handles in the external array can be obtained by taking the
+difference of the start and end pointers. Note that unlike <I>std::vector<=
/I>, we
+do not store both an allocated and used size. We store only the =E2=80=98u=
sed=E2=80=99 size
+and call std::realloc whenever the used size is modified, thus we rely on =
the
+std::malloc implementation in the standard C library to track =E2=80=98all=
ocated=E2=80=99 size
+for us. In practice this performs well but does not return memory to the =
=E2=80=98system=E2=80=99
+when lists shrink (unless they shrink to zero). This overall scheme could =
exhibit
+poor performance if the size of one of the data lists in the set frequentl=
y changes
+between less than two and more than two handles, as this will result in fr=
equent
+releasing and re-allocating of the memory for the corresponding array.
+
+If the <I>MESHSET_ORDERED</I> flag is not present, then the set contents l=
ist (parent
+and child lists are unaffected) is stored in a range-compacted format. In =
this
+format the number of handles stored in the array is always a multiple of t=
wo.
+Each consecutive pair of handles indicate the start and end, inclusive, of=
 a range
+of handles contained in the set. All such handle range pairs are stored in=
 sorted
+order and do not overlap. Nor is the end handle of one range ever one less=
 than
+the start handle of the next. All such =E2=80=98adjacent=E2=80=99 range pa=
irs are merged into a
+single pair. The code for insertion and removal of handles from range-form=
atted
+set content lists is fairly complex. The implementation will guarantee tha=
t a
+given call to insert entities into a range or remove entities from a range=
 is never
+worse than O(ln n) + O(m + n), where =E2=80=98n=E2=80=99 is the number of =
handles to insert
+and =E2=80=98m=E2=80=99 is the number of handles already contained in the =
set. So it is generally
+much more efficient to build Ranges of handles to insert (and remove) and =
call
+MOAB to insert (or remove) the entire list at once rather than making may
+calls to insert (or remove) one or a few handles from the contents of a se=
t.
+The set storage could probably be further minimized by allowing up to six
+handles in one of the lists to be elided. That is, as there are six potent=
ial =E2=80=98slots=E2=80=99
+in the MeshSet object then if two of the lists are empty it should be poss=
ible to store up to six values of the remaining list directly in the MeshSe=
t object.
+However, the additional runtime cost of such complexity could easily outwe=
igh
+any storage advantage. Further investigation into this has not been done b=
ecause
+the primary motivation for the storage optimization was to support binary =
trees.
+
+Another possible optimization of storage would be to remove the <I>MeshSet=
</I>
+object entirely and instead store the data in a =E2=80=98blocked=E2=80=99 =
format. The corre-
+sponding <I>SequenceData</I> would contain four arrays: flags, parents, ch=
ildren, and
+contents instead of a single array of <I>MeshSet</I> objects. If this were=
 done then
+no storage need ever be allocated for parent or child links if none of the=
 sets
+in a <I>SequenceData</I> has parent or child links. The effectiveness of t=
he storage
+reduction would depend greatly on how sets get grouped into <I>SequenceDat=
as</I>.
+This alternate storage scheme might also allow for better cache utilizatio=
n as it
+would group like data together. It is often the case that application code=
 that
+is querying the contents of one set will query the contents of many but ne=
ver
+query the parents or children of any set. Or that an application will quer=
y only
+parent or child links of a set without every querying other set properties=
. The
+downside of this solution is that it makes the implementation a little les=
s mod-
+ular and maintainable because the existing logic contained in the <I>MeshS=
et</I> class
+would need to be spread throughout the <I>MeshSetSequence</I> class.
+
+  \ref dg-contents
+
+  \page dg-figures List of Figures
+
+  \ref figure1
+
+  \ref figure2
+
+  \ref figure3
+*/

diff --git a/doc/user.dox.in b/doc/user.dox.in
index 6b7730b..ed65208 100644
--- a/doc/user.dox.in
+++ b/doc/user.dox.in
@@ -306,6 +306,7 @@ WARN_LOGFILE           =3D
=20
 INPUT                  =3D @top_srcdir@/src @top_srcdir@/src/moab \
                          @top_srcdir@/doc/UG/moabUG.h \
+                         @top_srcdir@/doc/DG \
                          @top_srcdir@/src/parallel/moab  \
                          @top_srcdir@/src/MBTagConventions.hpp \
                          @top_srcdir@/src/MBCN.h \

diff --git a/src/moab/Interface.hpp b/src/moab/Interface.hpp
index 6a04ff7..d7ad8aa 100644
--- a/src/moab/Interface.hpp
+++ b/src/moab/Interface.hpp
@@ -22,7 +22,7 @@
  * together to describe geometric topology, boundary condition, and inter-=
processor interface=20
  * groupings in a mesh.
  *
- * MOAB's API is documented in the moab::Interface class.  The User's Guid=
e is located in
+ * MOAB's API is documented in the moab::Interface class.  The User's Guid=
e and Developer's Guide are located in
  * <a href=3D"pages.html">related pages</a>.  Questions and comments shoul=
d be sent to moab-dev=20
  * _at_ mcs.anl.gov.
  */


https://bitbucket.org/fathomteam/moab/commits/6cf1eee99a24/
Changeset:   6cf1eee99a24
Branch:      None
User:        vijaysm
Date:        2013-06-29 05:02:24
Summary:     Fixing all the errors in CMake build system for MOAB. Now succ=
essfully compiles and links the sources. Tests in bin: moab_test and tag_te=
st run successfully.

Affected #:  4 files

diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 38b4130..a1993c9 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -31,8 +31,8 @@
     PolyElementSeq.cpp
     Range.cpp
     RangeSeqIntersectIter.cpp
-    ReadUtil.cpp
     ReaderWriterSet.cpp
+    ReadUtil.cpp
     ScdElementData.cpp
     ScdInterface.cpp
     ScdVertexData.cpp
@@ -56,11 +56,40 @@
     VertexSequence.cpp
     WriteUtil.cpp
   )
+
+  add_subdirectory( io )
+  set ( MOABIO_LIB_SRCS
+    io/IODebugTrack.cpp
+    io/ExoIIUtil.cpp
+    io/FileTokenizer.cpp
+    io/GmshUtil.cpp
+    io/ReadABAQUS.cpp
+    io/ReadGmsh.cpp
+    io/ReadIDEAS.cpp
+    io/ReadMCNP5.cpp
+    io/ReadNASTRAN.cpp
+    io/ReadSmf.cpp
+    io/ReadSms.cpp
+    io/ReadSTL.cpp
+    io/ReadTemplate.cpp
+    io/ReadTetGen.cpp
+    io/ReadTxt.cpp
+    io/ReadVtk.cpp
+    io/SMF_State.cpp
+    io/Tqdcfr.cpp
+    io/VtkUtil.cpp
+    io/WriteAns.cpp
+    io/WriteGMV.cpp
+    io/WriteGmsh.cpp
+    io/WriteSTL.cpp
+    io/WriteSmf.cpp
+    io/WriteTemplate.cpp
+    io/WriteVtk.cpp
+  )
   include_directories(
     ${MOAB_SOURCE_DIR}/src
-    ${MOAB_BINARY_DIR}/src
     ${MOAB_SOURCE_DIR}/src/io
-    ${MOAB_BINARY_DIR}/src/io
+    ${MOAB_SOURCE_DIR}/src/parallel
   )
=20
   if ( MOAB_USE_MPI )
@@ -82,12 +111,73 @@
   set_source_files_properties( ${MOAB_LIB_SRCS}
     COMPILE_FLAGS "-DIS_BUILDING_MB ${MOAB_DEFINES}"
   )
+
+#  target_link_libraries( MOAB moabio )
+  if ( MOAB_USE_NETCDF )
+    find_package( NetCDF )
+    if ( NetCDF_FOUND )
+      set ( MOAB_DEFINES "${MOAB_DEFINES} -DNETCDF_FILE" )
+      set ( MOABIO_LIB_SRCS
+        ${MOABIO_LIB_SRCS}
+        io/ReadNCDF.cpp
+        io/WriteNCDF.cpp
+        io/WriteSLAC.cpp
+      )
+      include_directories(
+        ${NetCDF_INCLUDE_DIRECTORIES}
+      )
+    endif ( NetCDF_FOUND )
+  endif ( MOAB_USE_NETCDF )
+
+  if ( MOAB_USE_HDF )
+    # HDF5
+    find_package( HDF5 )
+    if ( HDF5_FOUND )
+      set ( MOAB_DEFINES "${MOAB_DEFINES} -DHDF5_FILE" )
+      check_function_exists( H5Pset_fapl_mpio MOAB_HDF_HAVE_PARALLEL )
+      set ( MOABIO_LIB_SRCS
+        ${MOABIO_LIB_SRCS}
+        io/HDF5Common.cpp
+        io/ReadHDF5.cpp
+        io/ReadHDF5Dataset.cpp
+        io/ReadHDF5VarLen.cpp
+        io/WriteHDF5.cpp
+      )
+
+#      if ( MOAB_USE_MPI AND MPI_FOUND )
+#        set ( MOABIO_LIB_SRCS
+#          ${MOABIO_LIB_SRCS}
+#          src/parallel/WriteHDF5Parallel.cpp
+#        )
+#      endif (MOAB_USE_MPI AND MPI_FOUND)
+
+      include_directories(
+        ${HDF5_INCLUDE_DIRECTORIES}
+        io/mhdf/include
+      )
+      add_subdirectory( io/mhdf )
+    endif ( HDF5_FOUND )
+  endif ( MOAB_USE_HDF )
+
+  set_source_files_properties( ${MOABIO_LIB_SRCS}
+    COMPILE_FLAGS "-DIS_BUILDING_MB ${MOAB_DEFINES}"
+  )
+
   add_library( MOAB
     ${MOAB_LIB_SRCS}
+    ${MOABIO_LIB_SRCS}
   )
- =20
-  add_subdirectory( io )
-  target_link_libraries( MOAB moabio )
+
+  if ( MOAB_USE_NETCDF AND NetCDF_FOUND )
+    MESSAGE( "Adding NetCDF dependency libraries: ${NetCDF_LIBRARIES}" )
+    target_link_libraries( MOAB ${NetCDF_LIBRARIES} )
+  endif ( MOAB_USE_NETCDF AND NetCDF_FOUND )
+
+  if ( MOAB_USE_HDF AND HDF5_FOUND )
+    MESSAGE( "Adding NetCDF dependency libraries: ${HDF5_LIBRARIES}" )
+#    add_dependencies( MOAB mhdf  )
+    target_link_libraries( MOAB mhdf ${HDF5_LIBRARIES} )
+  endif ( MOAB_USE_HDF AND HDF5_FOUND )
=20
   if ( MOAB_USE_MPI AND MPI_FOUND )
     add_subdirectory( parallel )

diff --git a/src/io/CMakeLists.txt b/src/io/CMakeLists.txt
deleted file mode 100644
index d6c3e44..0000000
--- a/src/io/CMakeLists.txt
+++ /dev/null
@@ -1,90 +0,0 @@
-
-  # MOAB Library
-  set ( MOABIO_LIB_SRCS
-    ExoIIUtil.cpp
-    FileTokenizer.cpp
-    GmshUtil.cpp
-    ReadABAQUS.cpp
-    ReadGmsh.cpp
-    ReadIDEAS.cpp
-    ReadMCNP5.cpp
-    ReadNASTRAN.cpp
-    ReadSmf.cpp
-    ReadSms.cpp
-    ReadSTL.cpp
-    ReadTemplate.cpp
-    ReadTetGen.cpp
-    ReadTxt.cpp
-    ReadVtk.cpp
-    SMF_State.cpp
-    Tqdcfr.cpp
-    VtkUtil.cpp
-    WriteAns.cpp
-    WriteGMV.cpp
-    WriteGmsh.cpp
-    WriteSTL.cpp
-    WriteSmf.cpp=20
-    WriteTemplate.cpp
-    WriteVtk.cpp
-  )
-  include_directories(
-    ${MOAB_SOURCE_DIR}/src
-    ${MOAB_BINARY_DIR}/src
-    ${MOAB_SOURCE_DIR}/src/parallel
-    ${MOAB_BINARY_DIR}/src/parallel
-  )
-
-  if ( MOAB_USE_NETCDF )
-    find_package( NetCDF )
-    if ( NetCDF_FOUND )
-      set ( MOAB_DEFINES "${MOAB_DEFINES} -DNETCDF_FILE" )
-      set ( MOABIO_LIB_SRCS
-        ${MOABIO_LIB_SRCS}
-        ReadNCDF.cpp
-        WriteNCDF.cpp
-        WriteSLAC.cpp
-      )
-      include_directories(
-        ${NetCDF_INCLUDE_DIRECTORIES}
-      )
-    endif ( NetCDF_FOUND )
-  endif ( MOAB_USE_NETCDF )
-
-  if ( MOAB_USE_HDF )
-    # HDF5
-    find_package( HDF5 )
-    if ( HDF5_FOUND )
-      set ( MOAB_DEFINES "${MOAB_DEFINES} -DHDF5_FILE" )
-      check_function_exists( H5Pset_fapl_mpio MOAB_HDF_HAVE_PARALLEL )
-      set ( MOABIO_LIB_SRCS
-        ${MOABIO_LIB_SRCS}
-        ReadHDF5.cpp
-        WriteHDF5.cpp
-      )
-      include_directories(
-        ${HDF5_INCLUDE_DIRECTORIES}
-        ${MOAB_SOURCE_DIR}/mhdf/include
-      )
-      add_subdirectory( mhdf )
-    endif ( HDF5_FOUND )
-  endif ( MOAB_USE_HDF )
-
-  set_source_files_properties( ${MOABIO_LIB_SRCS}
-    COMPILE_FLAGS "-DIS_BUILDING_MB ${MOAB_DEFINES}"
-  )
-  add_library( moabio
-    ${MOABIO_LIB_SRCS}
-  )
-
-  if ( MOAB_USE_HDF AND HDF5_FOUND )
-    target_link_libraries( moabio
-      mhdf
-      ${HDF5_LIBRARIES}
-    )
-  endif ( MOAB_USE_HDF AND HDF5_FOUND )
-
-  if ( MOAB_USE_NETCDF AND NetCDF_FOUND )
-    target_link_libraries( moabio
-      ${NetCDF_LIBRARIES}
-    )
-  endif ( MOAB_USE_NETCDF AND NetCDF_FOUND )

diff --git a/src/parallel/CMakeLists.txt b/src/parallel/CMakeLists.txt
index 598b6fc..7583885 100644
--- a/src/parallel/CMakeLists.txt
+++ b/src/parallel/CMakeLists.txt
@@ -5,8 +5,14 @@ set( moab_PARALLEL_SRCS
   ReadParallel.cpp
   gs.cpp
   TupleList.cpp
+  SharedSetData.cpp
 )
=20
+include_directories(
+    ${MOAB_SOURCE_DIR}/src
+    ${MOAB_SOURCE_DIR}/src/parallel
+  )
+
 if ( MOAB_USE_HDF AND MOAB_HDF_HAVE_PARALLEL )
   set( moab_PARALLEL_SRCS
     ${moab_PARALLEL_SRCS}
@@ -45,38 +51,3 @@ target_link_libraries( MOABpar
   ${MPI_EXTRA_LIBRARY}
 )
=20
-if ( MOAB_USE_MPI AND MPI_FOUND )
-  add_executable ( mbparallelcomm_test mbparallelcomm_test.cpp )
-  target_link_libraries( mbparallelcomm_test MOAB )
-  set_source_files_properties( mbparallelcomm_test.cpp
-    COMPILE_FLAGS "-DIS_BUILDING_MB ${MOAB_DEFINES}" )
-  add_test( TestParallelComm-BcastDelete
-    ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 2 ${MPIEXEC_PREFLAGS}
-    ${EXECUTABLE_OUTPUT_PATH}/mbparallelcomm_test ${MPIEXEC_POSTFLAGS} 0 $=
{MOAB_SOURCE_DIR}/parallel/ptest.cub )
-  add_test( TestParallelComm-ReadDelete
-    ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 2 ${MPIEXEC_PREFLAGS}
-    ${EXECUTABLE_OUTPUT_PATH}/mbparallelcomm_test ${MPIEXEC_POSTFLAGS} -1 =
${MOAB_SOURCE_DIR}/parallel/ptest.cub )
-  add_test( TestParallelComm-ReadParallel
-    ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 2 ${MPIEXEC_PREFLAGS}
-    ${EXECUTABLE_OUTPUT_PATH}/mbparallelcomm_test ${MPIEXEC_POSTFLAGS} -2 =
${MOAB_SOURCE_DIR}/parallel/ptest.cub )
-  add_test( TestParallelComm-Broadcast
-    ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 2 ${MPIEXEC_PREFLAGS}
-    ${EXECUTABLE_OUTPUT_PATH}/mbparallelcomm_test ${MPIEXEC_POSTFLAGS} -3 =
${MOAB_SOURCE_DIR}/parallel/ptest.cub )
-
-  add_executable ( parallel_unit_tests parallel_unit_tests.cpp )
-  target_link_libraries( parallel_unit_tests MOAB )
-  add_test( TestParallel
-    ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 2 ${MPIEXEC_PREFLAGS}
-    ${EXECUTABLE_OUTPUT_PATH}/parallel_unit_tests ${MPIEXEC_POSTFLAGS} ${M=
OAB_SOURCE_DIR}/parallel/ptest.cub )
-  set_source_files_properties( parallel_unit_tests.cpp
-    COMPILE_FLAGS "-DIS_BUILDING_MB ${MOAB_DEFINES}" )
-
-  if ( MOAB_USE_HDF )
-    add_executable( mhdf_parallel mhdf_parallel.c )
-    target_link_libraries( mhdf_parallel MOAB MOABpar mhdf )
-    add_test( TestMHDFParallel ${EXECUTABLE_OUTPUT_PATH}/mhdf_parallel )
-    set_source_files_properties( mhdf_parallel.c
-      COMPILE_FLAGS "-DIS_BUILDING_MB ${MOAB_DEFINES}" )
-  endif ( MOAB_USE_HDF )
-endif ( MOAB_USE_MPI AND MPI_FOUND )
-

diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt
index 030c3b4..3b1d28d 100644
--- a/test/CMakeLists.txt
+++ b/test/CMakeLists.txt
@@ -2,9 +2,7 @@
=20
   include_directories(
     ${MOAB_SOURCE_DIR}/src
-    ${MOAB_BINARY_DIR}/src
     ${MOAB_SOURCE_DIR}/src/parallel
-    ${MOAB_BINARY_DIR}/src/parallel
   )
=20
   add_executable( moab_test MBTest.cpp )
@@ -43,3 +41,39 @@
   add_test( TestBSPTree ${EXECUTABLE_OUTPUT_PATH}/bsp_tree_test )
=20
=20
+if ( MOAB_USE_MPI AND MPI_FOUND )
+
+  add_executable ( mbparallelcomm_test mbparallelcomm_test.cpp )
+  target_link_libraries( mbparallelcomm_test MOAB )
+  set_source_files_properties( mbparallelcomm_test.cpp
+    COMPILE_FLAGS "-DIS_BUILDING_MB ${MOAB_DEFINES}" )
+  add_test( TestParallelComm-BcastDelete
+    ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 2 ${MPIEXEC_PREFLAGS}
+    ${EXECUTABLE_OUTPUT_PATH}/mbparallelcomm_test ${MPIEXEC_POSTFLAGS} 0 $=
{MOAB_SOURCE_DIR}/parallel/ptest.cub )
+  add_test( TestParallelComm-ReadDelete
+    ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 2 ${MPIEXEC_PREFLAGS}
+    ${EXECUTABLE_OUTPUT_PATH}/mbparallelcomm_test ${MPIEXEC_POSTFLAGS} -1 =
${MOAB_SOURCE_DIR}/parallel/ptest.cub )
+  add_test( TestParallelComm-ReadParallel
+    ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 2 ${MPIEXEC_PREFLAGS}
+    ${EXECUTABLE_OUTPUT_PATH}/mbparallelcomm_test ${MPIEXEC_POSTFLAGS} -2 =
${MOAB_SOURCE_DIR}/parallel/ptest.cub )
+  add_test( TestParallelComm-Broadcast
+    ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 2 ${MPIEXEC_PREFLAGS}
+    ${EXECUTABLE_OUTPUT_PATH}/mbparallelcomm_test ${MPIEXEC_POSTFLAGS} -3 =
${MOAB_SOURCE_DIR}/parallel/ptest.cub )
+
+  if ( MOAB_USE_HDF )
+    add_executable( mhdf_parallel mhdf_parallel.c )
+    target_link_libraries( mhdf_parallel MOAB MOABpar mhdf )
+    add_test( TestMHDFParallel ${EXECUTABLE_OUTPUT_PATH}/mhdf_parallel )
+    set_source_files_properties( mhdf_parallel.c
+      COMPILE_FLAGS "-DIS_BUILDING_MB ${MOAB_DEFINES}" )
+  endif ( MOAB_USE_HDF )
+
+  add_executable ( parallel_unit_tests parallel_unit_tests.cpp )
+  target_link_libraries( parallel_unit_tests MOAB )
+  add_test( TestParallel
+    ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 2 ${MPIEXEC_PREFLAGS}
+    ${EXECUTABLE_OUTPUT_PATH}/parallel_unit_tests ${MPIEXEC_POSTFLAGS} ${M=
OAB_SOURCE_DIR}/parallel/ptest.cub )
+  set_source_files_properties( parallel_unit_tests.cpp
+    COMPILE_FLAGS "-DIS_BUILDING_MB ${MOAB_DEFINES}" )
+
+endif ( MOAB_USE_MPI AND MPI_FOUND )


https://bitbucket.org/fathomteam/moab/commits/ff1b2d51a7da/
Changeset:   ff1b2d51a7da
Branch:      None
User:        vijaysm
Date:        2013-06-29 05:58:07
Summary:     Merge remote-tracking branch 'upstream/master'

Affected #:  5 files

diff --git a/MeshFiles/unittest/mpas_p8.h5m b/MeshFiles/unittest/mpas_p8.h5m
new file mode 100644
index 0000000..75662d2
Binary files /dev/null and b/MeshFiles/unittest/mpas_p8.h5m differ

diff --git a/src/Skinner.cpp b/src/Skinner.cpp
index fb80e58..5869777 100644
--- a/src/Skinner.cpp
+++ b/src/Skinner.cpp
@@ -453,16 +453,6 @@ ErrorCode Skinner::find_skin_noadj(const Range &source=
_entities,
   if(mTargetDim < 0 || source_dim > 3)
     return MB_FAILURE;
=20
-  //Core *this_core =3D dynamic_cast<Core*>(thisMB);
-  //bool use_adjs =3D false;
-  //if (!this_core->a_entity_factory()->vert_elem_adjacencies() &&
-  //  create_vert_elem_adjs)
-  //  this_core->a_entity_factory()->create_vert_elem_adjacencies();
-  //
-  //if (this_core->a_entity_factory()->vert_elem_adjacencies())
-  //  use_adjs =3D true;
-  //
-  //else=20
   initialize();
=20
   Range::const_iterator iter, end_iter;
@@ -477,7 +467,8 @@ ErrorCode Skinner::find_skin_noadj(const Range &source_=
entities,
   EntityHandle sub_conn[32];
   std::vector<EntityHandle> tmp_conn_vec;
   int num_nodes, num_sub_nodes, num_sides;
-  int sub_indices[32];
+  int sub_indices[32];// Also, assume that no polygon has more than 32 nod=
es
+  // we could increase that, but we will not display it right in visit moa=
b h5m , anyway
   EntityType sub_type;
=20
   // for each source entity
@@ -490,130 +481,95 @@ ErrorCode Skinner::find_skin_noadj(const Range &sour=
ce_entities,
    =20
     type =3D thisMB->type_from_handle(*iter);
     Range::iterator seek_iter;
-    Range dum_elems, dum_sub_elems;
    =20
-    // get connectivity of each n-1 dimension entity
-    num_sides =3D CN::NumSubEntities( type, mTargetDim );
+    // treat separately polygons (also, polyhedra will need special handli=
ng)
+    if (MBPOLYGON =3D=3D type)
+    {
+      num_sides =3D num_nodes;
+      sub_type =3D MBEDGE;
+      num_sub_nodes =3D 2;
+    }
+    else// get connectivity of each n-1 dimension entity
+      num_sides =3D CN::NumSubEntities( type, mTargetDim );
     for(int i=3D0; i<num_sides; i++)
     {
-      CN::SubEntityNodeIndices( type, num_nodes, mTargetDim, i, sub_type, =
num_sub_nodes, sub_indices );
-      assert((size_t)num_sub_nodes <=3D sizeof(sub_indices)/sizeof(sub_ind=
ices[0]));
-      for(int j=3D0; j<num_sub_nodes; j++)
-        sub_conn[j] =3D conn[sub_indices[j]];
-     =20
-//      if (use_adjs) {
-//        dum_elems.clear();
-//        result =3D thisMB->get_adjacencies(sub_conn, num_sub_nodes, sour=
ce_dim, false,
-//                                         dum_elems);
-//        if (MB_SUCCESS !=3D result) return result;
-//        dum_elems =3D intersect( dum_elems, source_entities );
-//        if (dum_elems.empty()) {
-//          assert(false);  // should never happen
-//          return MB_FAILURE;
-//        }
-//        // if (dum_elems.size() > 2) {=20
-//          // source entities do not form a valid source-dim patch (t-jun=
ction).
-//          // do we care?
-//        // }
-//       =20
-//        if (1 =3D=3D dum_elems.size()) {
-//            // this sub_element is on the skin
-//
-//            // check for existing entity
-//          dum_sub_elems.clear();
-//          result =3D thisMB->get_adjacencies(sub_conn, num_sub_nodes, mT=
argetDim, false,
-//                                           dum_sub_elems);
-//          if (MB_SUCCESS !=3D result) return result;
-//          if (dum_sub_elems.empty()) {
-//              // need to create one
-//            EntityHandle tmphndl=3D0;
-//            int indices[MAX_SUB_ENTITY_VERTICES];
-//            EntityType new_type;
-//            int num_new_nodes;
-//            CN::SubEntityNodeIndices( type, num_nodes, mTargetDim, i, ne=
w_type, num_new_nodes, indices );
-//            for(int j=3D0; j<num_new_nodes; j++)
-//              sub_conn[j] =3D conn[indices[j]];
-//       =20
-//            result =3D thisMB->create_element(new_type, sub_conn, =20
-//                                            num_new_nodes, tmphndl);
-//            forward_target_entities.insert(tmphndl);
-//          }
-//          else {
-//              // else find the relative sense of this entity to the sour=
ce_entity in this set
-//            int side_no, sense =3D 0, offset;
-//            if (source_entities.find(*dum_elems.begin()) =3D=3D source_e=
ntities.end()) {
-//              result =3D thisMB->side_number(*dum_elems.rbegin(), *dum_s=
ub_elems.begin(),
-//                                           side_no, sense, offset);
-//            }
-//            else {
-//              result =3D thisMB->side_number(*dum_elems.begin(), *dum_su=
b_elems.begin(),
-//                                           side_no, sense, offset);
-//            }
-//            if (-1 =3D=3D sense) reverse_target_entities.insert(*dum_sub=
_elems.begin());
-//            else if (1 =3D=3D sense) forward_target_entities.insert(*dum=
_sub_elems.begin());
-//            else return MB_FAILURE;
-//          }
-//        }
-//      }
-//      else {
-       =20
-          // see if we can match this connectivity with
-          // an existing entity
-        find_match( sub_type, sub_conn, num_sub_nodes, match, direct );
+      if(MBPOLYGON=3D=3Dtype)
+      {
+        sub_conn[0] =3D conn[i];
+        sub_conn[1] =3D conn[(i+1)%num_sides];
+      }
+      else
+      {
+        CN::SubEntityNodeIndices( type, num_nodes, mTargetDim, i, sub_type=
, num_sub_nodes, sub_indices );
+        assert((size_t)num_sub_nodes <=3D sizeof(sub_indices)/sizeof(sub_i=
ndices[0]));
+        for(int j=3D0; j<num_sub_nodes; j++)
+          sub_conn[j] =3D conn[sub_indices[j]];
+      }
        =20
-          // if there is no match, create a new entity
-        if(match =3D=3D 0)
+        // see if we can match this connectivity with
+        // an existing entity
+      find_match( sub_type, sub_conn, num_sub_nodes, match, direct );
+
+        // if there is no match, create a new entity
+      if(match =3D=3D 0)
+      {
+        EntityHandle tmphndl=3D0;
+        int indices[MAX_SUB_ENTITY_VERTICES];
+        EntityType new_type;
+        int num_new_nodes;
+        if(MBPOLYGON=3D=3Dtype)
+        {
+          new_type =3D MBEDGE;
+          num_new_nodes =3D 2;
+        }
+        else
         {
-          EntityHandle tmphndl=3D0;
-          int indices[MAX_SUB_ENTITY_VERTICES];
-          EntityType new_type;
-          int num_new_nodes;
           CN::SubEntityNodeIndices( type, num_nodes, mTargetDim, i, new_ty=
pe, num_new_nodes, indices );
           for(int j=3D0; j<num_new_nodes; j++)
             sub_conn[j] =3D conn[indices[j]];
-          result =3D thisMB->create_element(new_type, sub_conn, num_new_no=
des,
-                                          tmphndl);
-          assert(MB_SUCCESS =3D=3D result);
-          add_adjacency(tmphndl, sub_conn, CN::VerticesPerEntity(new_type)=
);
-          forward_target_entities.insert(tmphndl);
         }
-          // if there is a match, delete the matching entity
-          // if we can.=20
-        else
+        result =3D thisMB->create_element(new_type, sub_conn, num_new_node=
s,
+                                        tmphndl);
+        assert(MB_SUCCESS =3D=3D result);
+        add_adjacency(tmphndl, sub_conn, CN::VerticesPerEntity(new_type));
+        forward_target_entities.insert(tmphndl);
+      }
+        // if there is a match, delete the matching entity
+        // if we can.
+      else
+      {
+        if ( (seek_iter =3D forward_target_entities.find(match)) !=3D forw=
ard_target_entities.end())
         {
-          if ( (seek_iter =3D forward_target_entities.find(match)) !=3D fo=
rward_target_entities.end())
+          forward_target_entities.erase(seek_iter);
+          remove_adjacency(match);
+          if(/*!use_adjs &&*/ entity_deletable(match))
           {
-            forward_target_entities.erase(seek_iter);
-            remove_adjacency(match);
-            if(/*!use_adjs &&*/ entity_deletable(match))
-            {
-              result =3D thisMB->delete_entities(&match, 1);
-              assert(MB_SUCCESS =3D=3D result);
-            }
+            result =3D thisMB->delete_entities(&match, 1);
+            assert(MB_SUCCESS =3D=3D result);
+          }
+        }
+        else if ( (seek_iter =3D reverse_target_entities.find(match)) !=3D=
 reverse_target_entities.end())
+        {
+          reverse_target_entities.erase(seek_iter);
+          remove_adjacency(match);
+          if(/*!use_adjs &&*/ entity_deletable(match))
+          {
+            result =3D thisMB->delete_entities(&match, 1);
+            assert(MB_SUCCESS =3D=3D result);
           }
-          else if ( (seek_iter =3D reverse_target_entities.find(match)) !=
=3D reverse_target_entities.end())
+        }
+        else
+        {
+          if(direct =3D=3D FORWARD)
           {
-            reverse_target_entities.erase(seek_iter);
-            remove_adjacency(match);
-            if(/*!use_adjs &&*/ entity_deletable(match))
-            {
-              result =3D thisMB->delete_entities(&match, 1);
-              assert(MB_SUCCESS =3D=3D result);
-            }
+            forward_target_entities.insert(match);
           }
           else
           {
-            if(direct =3D=3D FORWARD)
-            {
-              forward_target_entities.insert(match);
-            }
-            else
-            {
-              reverse_target_entities.insert(match);
-            }
+            reverse_target_entities.insert(match);
           }
         }
-      //}
+      }
     }
   }
=20
@@ -1510,6 +1466,34 @@ ErrorCode Skinner::create_side( EntityHandle elem,
   rval =3D thisMB->get_connectivity( elem, conn, len, false, &storage );
   if (MB_SUCCESS !=3D rval) return rval;
 =20
+  // treat separately MBPOLYGON; we want to create the edge in the
+  // forward sense always ; so figure out the sense first, then get out
+  if (MBPOLYGON=3D=3Dtype && 1=3D=3Dd && MBEDGE=3D=3Dside_type)
+  {
+    // first find the first vertex in the conn list
+    int i=3D0;
+    for (i=3D0; i<len; i++)
+    {
+      if (conn[i]=3D=3Dside_conn[0])
+        break;
+    }
+    if (len =3D=3D i)
+      return MB_FAILURE; // not found, big error
+    int prevIndex =3D (i+len-1)%len;
+    int nextIndex =3D (i+1)%len;
+    EntityHandle conn2[2] =3D {side_conn[0], side_conn[1]};
+    if (conn[prevIndex]=3D=3Dside_conn[1])
+    {
+      // reverse, so the edge will be forward
+      conn2[0]=3Dside_conn[1];
+      conn2[1]=3Dside_conn[0];
+    }
+    else if  ( conn[nextIndex]!=3Dside_conn[1])
+      return MB_FAILURE; // it is not adjacent to the polygon
+
+    return thisMB->create_element( MBEDGE, conn2, 2, side_elem );
+
+  }
   // Find which side we are creating and get indices of all nodes
   // (including higher-order, if any.)
   CN::SideNumber( type, conn, side_conn, ncorner, d, side, sense, offset );

diff --git a/test/parallel/par_intx_sph.cpp b/test/parallel/par_intx_sph.cpp
index 98d0f7a..bf5cbb8 100644
--- a/test/parallel/par_intx_sph.cpp
+++ b/test/parallel/par_intx_sph.cpp
@@ -42,9 +42,12 @@ using namespace moab;
 // some input data
 double EPS1=3D0.2; // this is for box error
 std::string input_mesh_file("Homme_2pt.h5m"); // input file, partitioned c=
orrectly
+std::string mpas_file("mpas_p8.h5m");
 double CubeSide =3D 6.; // the above file starts with cube side 6; radius =
depends on cube side
+double radius;
 void test_intx_in_parallel();
 void test_intx_in_parallel_elem_based();
+void test_intx_mpas();
=20
 int main(int argc, char **argv)
 {
@@ -73,7 +76,10 @@ int main(int argc, char **argv)
     }
   }
   //result +=3D RUN_TEST(test_intx_in_parallel);
+  radius =3D CubeSide/2*sqrt(3.);
   result +=3D RUN_TEST(test_intx_in_parallel_elem_based);
+  radius =3D1.;
+  result +=3D RUN_TEST(test_intx_mpas);
=20
   MPI_Finalize();
   return result;
@@ -92,9 +98,9 @@ ErrorCode  manufacture_lagrange_mesh_on_sphere(Interface =
* mb, EntityHandle eule
    *   circumscribed sphere radius
    *   radius =3D length * math.sqrt(3) /2
    */
-  double radius =3D CubeSide/2*sqrt(3.);// our value depends on cube side
+  //radius =3D CubeSide/2*sqrt(3.);// our value depends on cube side
   Range quads;
-  rval =3D mb->get_entities_by_type(euler_set, MBQUAD, quads);
+  rval =3D mb->get_entities_by_dimension(euler_set, 2, quads);
   CHECK_ERR(rval);
=20
   Range connecVerts;
@@ -177,7 +183,7 @@ void test_intx_in_parallel()
 =20
   Intx2MeshOnSphere worker(&mb);
=20
-  double radius=3D CubeSide/2 * sqrt(3.) ; // input
+  //double radius=3D CubeSide/2 * sqrt(3.) ; // input
   worker.SetRadius(radius);
   worker.set_box_error(EPS1);//
   //worker.SetEntityType(MBQUAD);
@@ -244,7 +250,7 @@ void test_intx_in_parallel_elem_based()
=20
   Intx2MeshOnSphere worker(&mb);
=20
-  double radius=3D CubeSide/2 * sqrt(3.) ; // input
+  //double radius=3D CubeSide/2 * sqrt(3.) ; // input
   worker.SetRadius(radius);
   worker.set_box_error(EPS1);//
   //worker.SetEntityType(MBQUAD);
@@ -282,3 +288,79 @@ void test_intx_in_parallel_elem_based()
       "  intersection area:" << intx_area << " rel error: " << fabs((intx_=
area-arrival_area)/arrival_area) << "\n";
   CHECK_ERR(rval);
 }
+
+void test_intx_mpas()
+{
+  std::string opts =3D std::string("PARALLEL=3DREAD_PART;PARTITION=3DPARAL=
LEL_PARTITION")+
+          std::string(";PARALLEL_RESOLVE_SHARED_ENTS");
+  Core moab;
+  Interface & mb =3D moab;
+  EntityHandle euler_set;
+  ErrorCode rval;
+  rval =3D mb.create_meshset(MESHSET_SET, euler_set);
+  CHECK_ERR(rval);
+  std::string example(TestDir + "/" +  mpas_file);
+
+  rval =3D mb.load_file(example.c_str(), &euler_set, opts.c_str());
+
+  ParallelComm* pcomm =3D ParallelComm::get_pcomm(&mb, 0);
+  CHECK_ERR(rval);
+
+  rval =3D pcomm->check_all_shared_handles();
+  CHECK_ERR(rval);
+
+  // everybody will get a DP tag, including the non owned entities; so exc=
hange tags is not required for LOC (here)
+  rval =3D manufacture_lagrange_mesh_on_sphere(&mb, euler_set);
+  CHECK_ERR(rval);
+
+  int rank =3D pcomm->proc_config().proc_rank();
+
+  std::stringstream ste;
+  ste<<"initial" << rank<<".vtk";
+  mb.write_file(ste.str().c_str(), 0, 0, &euler_set, 1);
+
+  Intx2MeshOnSphere worker(&mb);
+
+  //double radius=3D CubeSide/2 * sqrt(3.) ; // input
+  worker.SetRadius(radius);
+  worker.set_box_error(EPS1);//
+  //worker.SetEntityType(MBQUAD);
+
+  worker.SetErrorTolerance(radius*1.e-8);
+  std::cout << "error tolerance epsilon_1=3D"<< radius*1.e-8 << "\n";
+  //  worker.locate_departure_points(euler_set);
+
+  // we need to make sure the covering set is bigger than the euler mesh
+  EntityHandle covering_lagr_set;
+  rval =3D mb.create_meshset(MESHSET_SET, covering_lagr_set);
+  CHECK_ERR(rval);
+
+  rval =3D worker.create_departure_mesh_2nd_alg(euler_set, covering_lagr_s=
et);
+  CHECK_ERR(rval);
+
+  std::stringstream ss;
+  ss<<"partial" << rank<<".vtk";
+  mb.write_file(ss.str().c_str(), 0, 0, &covering_lagr_set, 1);
+  rval =3D enforce_convexity(&mb, covering_lagr_set);
+  CHECK_ERR(rval);
+  std::stringstream ss2;
+  ss2<<"partialConvex" << rank<<".vtk";
+  mb.write_file(ss2.str().c_str(), 0, 0, &covering_lagr_set, 1);
+  EntityHandle outputSet;
+  rval =3D mb.create_meshset(MESHSET_SET, outputSet);
+  CHECK_ERR(rval);
+  rval =3D worker.intersect_meshes(covering_lagr_set, euler_set, outputSet=
);
+  CHECK_ERR(rval);
+
+  //std::string opts_write("PARALLEL=3DWRITE_PART");
+  //rval =3D mb.write_file("manuf.h5m", 0, opts_write.c_str(), &outputSet,=
 1);
+  std::string opts_write("");
+  std::stringstream outf;
+  outf<<"intersect" << rank<<".h5m";
+  rval =3D mb.write_file(outf.str().c_str(), 0, 0, &outputSet, 1);
+  double intx_area =3D area_on_sphere(&mb, outputSet, radius);
+  double arrival_area =3D area_on_sphere(&mb, euler_set, radius) ;
+  std::cout<< "On rank : " << rank << " arrival area: " << arrival_area<<
+      "  intersection area:" << intx_area << " rel error: " << fabs((intx_=
area-arrival_area)/arrival_area) << "\n";
+  CHECK_ERR(rval);
+}

diff --git a/tools/mbcslam/CslamUtils.cpp b/tools/mbcslam/CslamUtils.cpp
index a819825..1166a7a 100644
--- a/tools/mbcslam/CslamUtils.cpp
+++ b/tools/mbcslam/CslamUtils.cpp
@@ -876,17 +876,20 @@ ErrorCode enforce_convexity(Interface * mb, EntityHan=
dle lset)
     return rval;
=20
   std::vector<double> coords;
-  coords.resize(3*10); // at most 10 vertices per polygon
+  coords.resize(3*MAXEDGES); // at most 10 vertices per polygon
   // we should create a queue with new polygons that need processing for r=
eflex angles
   //  (obtuse)
   std::queue<EntityHandle> newPolys;
   int brokenPolys=3D0;
-  for (Range::iterator eit =3D inputRange.begin(); eit !=3D inputRange.end=
()
-                                       || !newPolys.empty() ; eit++)
+  Range::iterator eit =3D inputRange.begin();
+  while (eit !=3D inputRange.end() || !newPolys.empty())
   {
     EntityHandle eh;
     if (eit !=3D inputRange.end())
+    {
       eh =3D *eit;
+      eit++;
+    }
     else
     {
       eh =3D newPolys.front();

diff --git a/tools/mbcslam/Intx2Mesh.cpp b/tools/mbcslam/Intx2Mesh.cpp
index 4c9c5d5..fadbaa3 100644
--- a/tools/mbcslam/Intx2Mesh.cpp
+++ b/tools/mbcslam/Intx2Mesh.cpp
@@ -1189,9 +1189,10 @@ ErrorCode Intx2Mesh::create_departure_mesh_2nd_alg(E=
ntityHandle & euler_set, Ent
   TLv.initialize(2, 0, 0, 3, numv); // to proc, GLOBAL ID, DP points
   TLv.enableWriteAccess();
=20
-  TLq.initialize(6, 0, 0, 0, numq); // to proc, elem GLOBAL ID, connectivi=
ty[4] (global ID v)
+  int sizeTuple =3D 2+MAXEDGES; // max edges is now 10 :) for polygons
+  TLq.initialize(12, 0, 0, 0, numq); // to proc, elem GLOBAL ID, connectiv=
ity[10] (global ID v)
   TLq.enableWriteAccess();
-  std::cout << "from proc " << my_rank << " send " << numv << " vertices a=
nd " << numq << " quads\n";
+  std::cout << "from proc " << my_rank << " send " << numv << " vertices a=
nd " << numq << " elements\n";
=20
   for (int to_proc=3D0; to_proc<numprocs; to_proc++)
   {
@@ -1213,25 +1214,31 @@ ErrorCode Intx2Mesh::create_departure_mesh_2nd_alg(=
EntityHandle & euler_set, Ent
       TLv.inc_n();
     }
     // also, prep the quad for sending ...
-    Range Q =3D range_to_P.subset_by_type(MBQUAD);
+    Range Q =3D range_to_P.subset_by_dimension(2);
     for (Range::iterator it=3DQ.begin(); it!=3DQ.end(); it++)
     {
       EntityHandle q=3D*it;
       int global_id;
       rval =3D mb->tag_get_data(gid, &q, 1, &global_id);
-      ERRORR(rval, "can't get gid for quad");
+      ERRORR(rval, "can't get gid for polygon");
       int n=3DTLq.get_n();
-      TLq.vi_wr[6*n] =3D to_proc; //
-      TLq.vi_wr[6*n+1] =3D global_id; // global id of element, used to ide=
ntify it ...
+      TLq.vi_wr[sizeTuple*n] =3D to_proc; //
+      TLq.vi_wr[sizeTuple*n+1] =3D global_id; // global id of element, use=
d to identify it ...
       const EntityHandle * conn4;
       int num_nodes;
-      rval =3D mb->get_connectivity(q, conn4, num_nodes);
+      rval =3D mb->get_connectivity(q, conn4, num_nodes);// could be up to=
 10;
       ERRORR(rval, "can't get connectivity for quad");
+      if (num_nodes > MAXEDGES)
+        ERRORR(MB_FAILURE, "too many nodes in a polygon");
       for (int i=3D0; i<num_nodes; i++)
       {
         EntityHandle v =3D conn4[i];
         unsigned int index =3D local_verts.find(v)-local_verts.begin();
-        TLq.vi_wr[6*n+2+i] =3D gids[index];
+        TLq.vi_wr[sizeTuple*n+2+i] =3D gids[index];
+      }
+      for (int k=3Dnum_nodes; k<MAXEDGES; k++)
+      {
+        TLq.vi_wr[sizeTuple*n+2+k] =3D 0; // fill the rest of node ids wit=
h 0; we know that the node ids start from 1!
       }
       TLq.inc_n();
=20
@@ -1264,7 +1271,7 @@ ErrorCode Intx2Mesh::create_departure_mesh_2nd_alg(En=
tityHandle & euler_set, Ent
       // now, all dep points should be at their place
       // look in the local list of q for this proc, and create all those q=
uads and vertices if needed
   Range & local=3DRto[my_rank];
-  Range local_q =3D local.subset_by_type(MBQUAD);
+  Range local_q =3D local.subset_by_dimension(2);
   // the local should have all the vertices in local_verts
   for (Range::iterator it=3Dlocal_q.begin(); it!=3Dlocal_q.end(); it++)
   {
@@ -1273,7 +1280,7 @@ ErrorCode Intx2Mesh::create_departure_mesh_2nd_alg(En=
tityHandle & euler_set, Ent
     const EntityHandle * conn4;
     rval =3D mb->get_connectivity(q, conn4, nnodes);
     ERRORR(rval, "can't get connectivity of local q ");
-    EntityHandle new_conn[4];
+    EntityHandle new_conn[MAXEDGES];
     for (int i=3D0; i<nnodes; i++)
     {
       EntityHandle v1=3Dconn4[i];
@@ -1290,39 +1297,59 @@ ErrorCode Intx2Mesh::create_departure_mesh_2nd_alg(=
EntityHandle & euler_set, Ent
       }
       new_conn[i] =3D globalID_to_handle[gids[index]];
     }
-    EntityHandle new_quad;
-    rval =3D mb->create_element(MBQUAD, new_conn, 4, new_quad);
+    EntityHandle new_element;
+    //
+    EntityType entType =3D MBQUAD;
+    if (nnodes >4)
+      entType =3D MBPOLYGON;
+    if (nnodes <4)
+      entType =3D MBTRI;
+
+    rval =3D mb->create_element(entType, new_conn, nnodes, new_element);
     ERRORR(rval, "can't create new quad ");
-    rval =3D mb->add_entities(covering_lagr_set, &new_quad, 1);
-    ERRORR(rval, "can't add new quad to dep set");
+    rval =3D mb->add_entities(covering_lagr_set, &new_element, 1);
+    ERRORR(rval, "can't add new element to dep set");
     int gid_el;
     // get the global ID of the initial quad
     rval=3Dmb->tag_get_data(gid, &q, 1, &gid_el);
     ERRORR(rval, "can't get element global ID ");
-    globalID_to_eh[gid_el]=3Dnew_quad;
+    globalID_to_eh[gid_el]=3Dnew_element;
   }
-  // now look at all quads received through; we do not want to duplicate t=
hem
-  n=3DTLq.get_n();// number of quads received by this processor
+  // now look at all elements received through; we do not want to duplicat=
e them
+  n=3DTLq.get_n();// number of elements received by this processor
   for (int i=3D0; i<n; i++)
   {
-    int globalIdEl =3D TLq.vi_rd[6*i+1];
+    int globalIdEl =3D TLq.vi_rd[sizeTuple*i+1];
     // do we already have a quad with this global ID, represented?
     if (globalID_to_eh.find(globalIdEl)=3D=3DglobalID_to_eh.end())
     {
       // construct the conn quad
-      EntityHandle new_conn[4];
-      for (int j=3D0; j<4; j++)
+      EntityHandle new_conn[MAXEDGES];
+      int nnodes;
+      for (int j=3D0; j<MAXEDGES; j++)
       {
-        int vgid =3D TLq.vi_rd[6*i+2+j];// vertex global ID
-        assert(globalID_to_handle.find(vgid)!=3DglobalID_to_handle.end());
-        new_conn[j]=3DglobalID_to_handle[vgid];
+        int vgid =3D TLq.vi_rd[sizeTuple*i+2+j];// vertex global ID
+        if (vgid=3D=3D0)
+          new_conn[j] =3D 0;
+        else
+        {
+          assert(globalID_to_handle.find(vgid)!=3DglobalID_to_handle.end()=
);
+          new_conn[j]=3DglobalID_to_handle[vgid];
+          nnodes =3D j+1;// nodes are at the beginning, and are variable n=
umber
+        }
       }
-      EntityHandle new_quad;
-      rval =3D mb->create_element(MBQUAD, new_conn, 4, new_quad);
-      ERRORR(rval, "can't create new quad ");
-      globalID_to_eh[globalIdEl]=3Dnew_quad;
-      rval =3D mb->add_entities(covering_lagr_set, &new_quad, 1);
-      ERRORR(rval, "can't add new quad to dep set");
+      EntityHandle new_element;
+      //
+      EntityType entType =3D MBQUAD;
+      if (nnodes >4)
+        entType =3D MBPOLYGON;
+      if (nnodes <4)
+        entType =3D MBTRI;
+      rval =3D mb->create_element(entType, new_conn, nnodes, new_element);
+      ERRORR(rval, "can't create new element ");
+      globalID_to_eh[globalIdEl]=3Dnew_element;
+      rval =3D mb->add_entities(covering_lagr_set, &new_element, 1);
+      ERRORR(rval, "can't add new element to dep set");
     }
   }
   return MB_SUCCESS;


https://bitbucket.org/fathomteam/moab/commits/271c3fcc6d79/
Changeset:   271c3fcc6d79
Branch:      master
User:        vijaysm
Date:        2013-06-30 18:06:47
Summary:     Merge branch 'master' of bitbucket.org:fathomteam/moab

Affected #:  1 file

diff --git a/doc/DG/moabDG.h b/doc/DG/moabDG.h
index d2e14f0..1091094 100644
--- a/doc/DG/moabDG.h
+++ b/doc/DG/moabDG.h
@@ -1,10 +1,45 @@
-/*! \page developerguide Developer's Guide (MOAB 4.6)
-=20
-  \subpage dg-contents
-=20
-  \subpage dg-figures
+/*! \page developerguide Developer's Guide (MOAB 4.6)
+=20
+  \subpage dg-contents
+=20
+  \subpage dg-figures
+
+  \subpage dg-tables
+*/
+
+/*!  \page dg-contents Table of Contents
+
+  \subpage sequence-info
+
+  \subpage meta-data-info
+*/
+
+/*!  \page dg-figures List of Figures
+
+    \ref figure1
+
+    \ref figure2
+
+    \ref figure3
+*/
+
+/*!  \page dg-tables List of Tables
+
+    \ref table1
+
+    \ref table2
+
+    \ref table3
+
+    \ref table4
+
+    \ref table5
+
+    \ref table6
+
+*/
=20
-  \page dg-contents Table of Contents
+/*!  \page sequence-info Sequence Info
=20
   \ref sequence
=20
@@ -14,10 +49,9 @@
=20
   \ref sets
=20
-  \section sequence  1. EntitySequence & SequenceData
+  \section sequence  EntitySequence & SequenceData=20
=20
   \subsection figure1 Figure 1: EntitySequences For One SequenceData
-
 The <I>SequenceData</I> class manages as set of arrays of per-entity value=
s. Each
 <I>SequenceData</I> has a start and end handle denoting the block of entit=
ies for which
 the arrays contain data. The arrays managed by a <I>SequenceData</I> insta=
nce are
@@ -77,9 +111,9 @@ a feature. Sequences for which the corresponding data ca=
nnot be used to store
 new entities (e.g. structured mesh discussed in a later section) will retu=
rn -1 or
 some other invalid value.
=20
- \ref dg-contents
+ \ref sequence-info "Top"
=20
-  \section manager 2. TypeSequenceManager & SequenceManager=20
+  \section manager TypeSequenceManager & SequenceManager=20
=20
 The <I>TypeSequenceManager</I> class maintains an organized set of <I>Enti=
tySequence</I>
 instances and corresponding <I>SequenceData</I> instances. It is used to m=
anage
@@ -141,119 +175,920 @@ such as allocating the correct <I>EntitySequence</I=
> subtype for a given <I>Enti
=20
 <sup>2</sup>Given rule four for the data managed by a <I>TypeSequenceManag=
er</I>, any <I>SequenceData</I> for which all handles are allocated will be=
 referenced by exactly one <I>EntitySequence</I>.
=20
-  \ref dg-contents
-
- \section s-mesh 3. Structured Mesh
-
-Structured mesh storage is implemented using subclasses of <I>SequenceData=
</I>:
-<I>ScdElementData</I> and <I>ScdVertexData</I>. The <I>StructuredElementSe=
q</I> class is
-used to access the structured element connectivity. A standard <I>VertexSe=
quence</I>
-instance is used to access the ScdVertexData because the vertex data stora=
ge
-is the same as for unstructured mesh.
-
-  \ref dg-contents
-
-  \section sets 4. Entity Sets
-
-- MeshSetSequence
-
-The <I>MeshSetSequence</I> class is the same as most other subclasses of <=
I>EntitySequence</I>
-in that it utilizes SequenceData to store its data. A single array in the =
<I>SequenceData</I>
-is used to store instances of the MeshSet class, one per allocated <I>Enti=
tyHandle</I>.
-<I>SequenceData</I> allocates all of its managed arrays using malloc and f=
ree as
-simple arrays of bytes. <I>MeshSetSequence</I> does in-place construction =
and de-
-struction of <I>MeshSet</I> instances within that array. This is similar t=
o what is
-done by <I>std::vector</I> and other container classes that may own more s=
torage
-than is required at a given time for contained objects.
-
-- MeshSet
-
-  \subsection figure3 Figure 3: SequenceManager and Related Classes
-
-The <I>MeshSet</I> class is used to represent a single entity set instance=
 in MOAB.
-The class is optimized to minimize storage (further possible improvements =
in
-storage size are discussed later.)
-
-Figure 3 shows the memory layout of an instance of the <I>MeshSet</I> clas=
s.
-The flags member holds the set creation bit flags: <I>MESHSET_TRACK_OWNER<=
/I>,
-<I>MESHSET_SET</I>, and <I>MESHSET_ORDERED</I>. The presence of the <I>MES=
HSET_TRACK_OWNER</I>
-indicates that reverse links from the contained entities back to the ownin=
g set
-should be maintained in the adjacency list of each entity. The <I>MESHSET_=
SET</I>
-and <I>MESHSET_ORDERED</I> bits are mutually exclusive, and as such most c=
ode only
-tests for the <I>MESHSET_ORDERED</I>, meaning that in practice the <I>MESH=
SET_SET</I> bit is
-ignored. <I>MESHSET_ORDERED</I> indicates that the set may contain duplica=
te handles
-and that the order that the handles are added to the set should be preserv=
ed.
-In practice, such sets are stored as a simple list of handles. <I>MESHSET_=
SET</I> (or in
-practice, the lack of <I>MESHSET_ORDERED</I>) indicates that the order of =
the handles
-need not be preserved and that the set may not contain duplicate handles. =
Such
-sets are stored in a sorted range-compacted format similar to that of the =
Range
-class.
-
-The memory for storing contents, parents, and children are each handled in
-the same way. The data in the class is composed of a 2-bit =E2=80=98size=
=E2=80=99 field and two
-values, where the two values may either be two handles or two pointers. Th=
e size
-bit-fields are grouped together to reduce the required amount of memory. I=
f the
-numerical value of the 2-bit size field is 0 then the corresponding list i=
s empty.
-If the 2-bit size field is either 1 or 2, then the contents of the corresp=
onding list
-are stored directly in the corresponding two data fields of the MeshSet ob=
ject.
-If the 2-bit size field has a value of 3 (11 binary), then the correspondi=
ng two
-data fields store the begin and end pointers of an external array of handl=
es.
-The number of handles in the external array can be obtained by taking the
-difference of the start and end pointers. Note that unlike <I>std::vector<=
/I>, we
-do not store both an allocated and used size. We store only the =E2=80=98u=
sed=E2=80=99 size
-and call std::realloc whenever the used size is modified, thus we rely on =
the
-std::malloc implementation in the standard C library to track =E2=80=98all=
ocated=E2=80=99 size
-for us. In practice this performs well but does not return memory to the =
=E2=80=98system=E2=80=99
-when lists shrink (unless they shrink to zero). This overall scheme could =
exhibit
-poor performance if the size of one of the data lists in the set frequentl=
y changes
-between less than two and more than two handles, as this will result in fr=
equent
-releasing and re-allocating of the memory for the corresponding array.
-
-If the <I>MESHSET_ORDERED</I> flag is not present, then the set contents l=
ist (parent
-and child lists are unaffected) is stored in a range-compacted format. In =
this
-format the number of handles stored in the array is always a multiple of t=
wo.
-Each consecutive pair of handles indicate the start and end, inclusive, of=
 a range
-of handles contained in the set. All such handle range pairs are stored in=
 sorted
-order and do not overlap. Nor is the end handle of one range ever one less=
 than
-the start handle of the next. All such =E2=80=98adjacent=E2=80=99 range pa=
irs are merged into a
-single pair. The code for insertion and removal of handles from range-form=
atted
-set content lists is fairly complex. The implementation will guarantee tha=
t a
-given call to insert entities into a range or remove entities from a range=
 is never
-worse than O(ln n) + O(m + n), where =E2=80=98n=E2=80=99 is the number of =
handles to insert
-and =E2=80=98m=E2=80=99 is the number of handles already contained in the =
set. So it is generally
-much more efficient to build Ranges of handles to insert (and remove) and =
call
-MOAB to insert (or remove) the entire list at once rather than making may
-calls to insert (or remove) one or a few handles from the contents of a se=
t.
-The set storage could probably be further minimized by allowing up to six
-handles in one of the lists to be elided. That is, as there are six potent=
ial =E2=80=98slots=E2=80=99
-in the MeshSet object then if two of the lists are empty it should be poss=
ible to store up to six values of the remaining list directly in the MeshSe=
t object.
-However, the additional runtime cost of such complexity could easily outwe=
igh
-any storage advantage. Further investigation into this has not been done b=
ecause
-the primary motivation for the storage optimization was to support binary =
trees.
-
-Another possible optimization of storage would be to remove the <I>MeshSet=
</I>
-object entirely and instead store the data in a =E2=80=98blocked=E2=80=99 =
format. The corre-
-sponding <I>SequenceData</I> would contain four arrays: flags, parents, ch=
ildren, and
-contents instead of a single array of <I>MeshSet</I> objects. If this were=
 done then
-no storage need ever be allocated for parent or child links if none of the=
 sets
-in a <I>SequenceData</I> has parent or child links. The effectiveness of t=
he storage
-reduction would depend greatly on how sets get grouped into <I>SequenceDat=
as</I>.
-This alternate storage scheme might also allow for better cache utilizatio=
n as it
-would group like data together. It is often the case that application code=
 that
-is querying the contents of one set will query the contents of many but ne=
ver
-query the parents or children of any set. Or that an application will quer=
y only
-parent or child links of a set without every querying other set properties=
. The
-downside of this solution is that it makes the implementation a little les=
s mod-
-ular and maintainable because the existing logic contained in the <I>MeshS=
et</I> class
-would need to be spread throughout the <I>MeshSetSequence</I> class.
-
-  \ref dg-contents
-
-  \page dg-figures List of Figures
-
-  \ref figure1
-
-  \ref figure2
-
-  \ref figure3
+  \ref sequence-info "Top"
+
+ \section s-mesh Structured Mesh
+
+Structured mesh storage is implemented using subclasses of <I>SequenceData=
</I>:
+<I>ScdElementData</I> and <I>ScdVertexData</I>. The <I>StructuredElementSe=
q</I> class is
+used to access the structured element connectivity. A standard <I>VertexSe=
quence</I>
+instance is used to access the ScdVertexData because the vertex data stora=
ge
+is the same as for unstructured mesh.
+
+  \ref sequence-info "Top"
+
+  \section sets Entity Sets
+
+- MeshSetSequence
+
+The <I>MeshSetSequence</I> class is the same as most other subclasses of <=
I>EntitySequence</I>
+in that it utilizes SequenceData to store its data. A single array in the =
<I>SequenceData</I>
+is used to store instances of the MeshSet class, one per allocated <I>Enti=
tyHandle</I>.
+<I>SequenceData</I> allocates all of its managed arrays using malloc and f=
ree as
+simple arrays of bytes. <I>MeshSetSequence</I> does in-place construction =
and de-
+struction of <I>MeshSet</I> instances within that array. This is similar t=
o what is
+done by <I>std::vector</I> and other container classes that may own more s=
torage
+than is required at a given time for contained objects.
+
+- MeshSet
+
+  \subsection figure3 Figure 3: SequenceManager and Related Classes
+
+The <I>MeshSet</I> class is used to represent a single entity set instance=
 in MOAB.
+The class is optimized to minimize storage (further possible improvements =
in
+storage size are discussed later.)
+
+Figure 3 shows the memory layout of an instance of the <I>MeshSet</I> clas=
s.
+The flags member holds the set creation bit flags: <I>MESHSET_TRACK_OWNER<=
/I>,
+<I>MESHSET_SET</I>, and <I>MESHSET_ORDERED</I>. The presence of the <I>MES=
HSET_TRACK_OWNER</I>
+indicates that reverse links from the contained entities back to the ownin=
g set
+should be maintained in the adjacency list of each entity. The <I>MESHSET_=
SET</I>
+and <I>MESHSET_ORDERED</I> bits are mutually exclusive, and as such most c=
ode only
+tests for the <I>MESHSET_ORDERED</I>, meaning that in practice the <I>MESH=
SET_SET</I> bit is
+ignored. <I>MESHSET_ORDERED</I> indicates that the set may contain duplica=
te handles
+and that the order that the handles are added to the set should be preserv=
ed.
+In practice, such sets are stored as a simple list of handles. <I>MESHSET_=
SET</I> (or in
+practice, the lack of <I>MESHSET_ORDERED</I>) indicates that the order of =
the handles
+need not be preserved and that the set may not contain duplicate handles. =
Such
+sets are stored in a sorted range-compacted format similar to that of the =
Range
+class.
+
+The memory for storing contents, parents, and children are each handled in
+the same way. The data in the class is composed of a 2-bit =E2=80=98size=
=E2=80=99 field and two
+values, where the two values may either be two handles or two pointers. Th=
e size
+bit-fields are grouped together to reduce the required amount of memory. I=
f the
+numerical value of the 2-bit size field is 0 then the corresponding list i=
s empty.
+If the 2-bit size field is either 1 or 2, then the contents of the corresp=
onding list
+are stored directly in the corresponding two data fields of the MeshSet ob=
ject.
+If the 2-bit size field has a value of 3 (11 binary), then the correspondi=
ng two
+data fields store the begin and end pointers of an external array of handl=
es.
+The number of handles in the external array can be obtained by taking the
+difference of the start and end pointers. Note that unlike <I>std::vector<=
/I>, we
+do not store both an allocated and used size. We store only the =E2=80=98u=
sed=E2=80=99 size
+and call std::realloc whenever the used size is modified, thus we rely on =
the
+std::malloc implementation in the standard C library to track =E2=80=98all=
ocated=E2=80=99 size
+for us. In practice this performs well but does not return memory to the =
=E2=80=98system=E2=80=99
+when lists shrink (unless they shrink to zero). This overall scheme could =
exhibit
+poor performance if the size of one of the data lists in the set frequentl=
y changes
+between less than two and more than two handles, as this will result in fr=
equent
+releasing and re-allocating of the memory for the corresponding array.
+
+If the <I>MESHSET_ORDERED</I> flag is not present, then the set contents l=
ist (parent
+and child lists are unaffected) is stored in a range-compacted format. In =
this
+format the number of handles stored in the array is always a multiple of t=
wo.
+Each consecutive pair of handles indicate the start and end, inclusive, of=
 a range
+of handles contained in the set. All such handle range pairs are stored in=
 sorted
+order and do not overlap. Nor is the end handle of one range ever one less=
 than
+the start handle of the next. All such =E2=80=98adjacent=E2=80=99 range pa=
irs are merged into a
+single pair. The code for insertion and removal of handles from range-form=
atted
+set content lists is fairly complex. The implementation will guarantee tha=
t a
+given call to insert entities into a range or remove entities from a range=
 is never
+worse than O(ln n) + O(m + n), where =E2=80=98n=E2=80=99 is the number of =
handles to insert
+and =E2=80=98m=E2=80=99 is the number of handles already contained in the =
set. So it is generally
+much more efficient to build Ranges of handles to insert (and remove) and =
call
+MOAB to insert (or remove) the entire list at once rather than making may
+calls to insert (or remove) one or a few handles from the contents of a se=
t.
+The set storage could probably be further minimized by allowing up to six
+handles in one of the lists to be elided. That is, as there are six potent=
ial =E2=80=98slots=E2=80=99
+in the MeshSet object then if two of the lists are empty it should be poss=
ible to store up to six values of the remaining list directly in the MeshSe=
t object.
+However, the additional runtime cost of such complexity could easily outwe=
igh
+any storage advantage. Further investigation into this has not been done b=
ecause
+the primary motivation for the storage optimization was to support binary =
trees.
+
+Another possible optimization of storage would be to remove the <I>MeshSet=
</I>
+object entirely and instead store the data in a =E2=80=98blocked=E2=80=99 =
format. The corre-
+sponding <I>SequenceData</I> would contain four arrays: flags, parents, ch=
ildren, and
+contents instead of a single array of <I>MeshSet</I> objects. If this were=
 done then
+no storage need ever be allocated for parent or child links if none of the=
 sets
+in a <I>SequenceData</I> has parent or child links. The effectiveness of t=
he storage
+reduction would depend greatly on how sets get grouped into <I>SequenceDat=
as</I>.
+This alternate storage scheme might also allow for better cache utilizatio=
n as it
+would group like data together. It is often the case that application code=
 that
+is querying the contents of one set will query the contents of many but ne=
ver
+query the parents or children of any set. Or that an application will quer=
y only
+parent or child links of a set without every querying other set properties=
. The
+downside of this solution is that it makes the implementation a little les=
s mod-
+ular and maintainable because the existing logic contained in the <I>MeshS=
et</I> class
+would need to be spread throughout the <I>MeshSetSequence</I> class.
+
+  \ref sequence-info "Top"
 */
+
+/*! \page meta-data-info I/O and Meta-Data Storage Conventions in MOAB
+
+  <Center><H3> Timothy J. Tautges </H3></Center>
+
+    \ref meta-introduction
+
+    \ref meta-conventions
+
+    \ref meta-options
+
+    \ref meta-references
+
+    \ref appendixA
+
+    \ref appendixB
+
+    \ref appendixC
+
+    \ref appendixD
+
+    \ref appendixE
+
+  \section meta-introduction  Introduction
+
+The Mesh-Oriented datABase (MOAB) is a library for representing finite ele=
ment and other types of mesh data [1].  Various types of meta-data are ofte=
n used in conjunction with a mesh.  Examples include boundary condition gro=
upings, material types, and provenance information for the mesh.  Because t=
he data model used in MOAB is so abstract, conventions are useful for descr=
ibing how meta-data is stored into that data model.  This document describe=
s those conventions for several types of data commonly found in meshes stor=
ed in MOAB.  Because the data models used by MOAB and iMesh, the ITAPS mesh=
 interface [2], are so similar, the conventions described here apply almost=
 unmodified to iMesh as well as to MOAB.
+
+The meshes represented in MOAB originate in a variety of forms, including =
mesh read from files of various formats (e.g. CUBIT =E2=80=9C.cub=E2=80=9D =
file, VTK, etc.) as well as mesh written into MOAB directly by various soft=
ware libraries (e.g. MeshKit).  Although there is no standard for naming or=
 storing meta-data with a mesh, there is a great deal of commonality in the=
 types of meta-data typically found with mesh data.  This document describe=
s conventions that have been established for commonly encountered meta-data=
.  Various mesh readers implemented in MOAB attempt to read meta-data from =
a file and write it into the MOAB data model using these conventions.  Alth=
ough there is no requirement to store a given type of meta-data in the form=
 described here, a number of services have been written to handle meta-data=
 using these conventions, no matter the source of the meta-data being proce=
ssed.
+
+Several specific tools are often used in concert with MOAB and bear specia=
l mention here.  The CUBIT toolkit generates finite element meshes, and sav=
es them to a native save file (referred to as a =E2=80=9C.cub=E2=80=9D file=
) which MOAB is able to read.  Reading CUBIT meshes into MOAB through the .=
cub file format is preferred over other formats, since most other mesh form=
ats written by CUBIT do not save most meta-data.  The MeshKit library also =
generates mesh using CGM and MOAB, and uses the same conventions for storin=
g meshes into MOAB.  Finally, MOAB includes a CGM reader which can read a g=
eometric model into a faceted representation in MOAB.  Meta-data from all t=
hese tools are stored in MOAB using the conventions described here.=20
+
+The MOAB data model consists of the following basic types:
+- <B>Entity</B>: The basic elements of topology, e.g. vertex, edge, triang=
le, tetrahedron, etc.  MOAB represents all types in the finite element zoo,=
 plus polygons and polyhedra.
+- <B>Entity Set</B>: An arbitrary collection of entities and other sets.  =
Sets can have parent/child relations with other sets, and these relations a=
re distinct from =E2=80=9Ccontains=E2=80=9D relations.
+- <B>Interface</B>: The interface object through which other entities are =
accessed, in the sense of object-oriented-programming.  iMesh refers to the=
 interface as the =E2=80=9Croot=E2=80=9D set.
+- <B>Tag</B>: A piece of data that can be assigned a distinct value to eac=
h entity and entity set, and to the interface itself.  Tags have a prescrib=
ed name, size in bytes, and data type; allowed data types are integer, doub=
le, entity handle, and byte or opaque.
+.
+
+The following section describes each meta-data tag convention in detail; t=
hese conventions are also summarized in Table 1.
+
+  \ref meta-data-info "Top"
+
+  \section meta-conventions  Meta-Data Conventions
+
+Meta-data is stored in MOAB and iMesh in the form of tags applied to eithe=
r entities or entity sets.  For meta-data represented as entity sets, the c=
ontents of those sets are determined by the convention, with tags on those =
sets identifying them with the convention and adding any other semantic dat=
a.
+
+Each meta-data convention is described in a subsection below.  Each conven=
tion begins with a short description of:
+- Whether tags associated with the convention are assigned to entities or =
entity sets
+- The tag(s) associated with the convention; information for each tag incl=
udes the name, the data type (I=3Dinteger, D=3Ddouble, C=3Dcharacter, H=3Dh=
andle), and the tag length.  Tag lengths are specified after an asterisk (*=
); for example, C*32 implies a tag with character type and length 32.  Unsp=
ecified lengths correspond to length one.
+.
+
+<H3>Name</H3>
+
+(Data: Entity sets, entities; Tag(s): NAME/C*32)
+
+Character strings are used in many different contexts in applications.  MO=
AB uses the =E2=80=9CNAME=E2=80=9D tag to store character strings used to n=
ame entities.  This tag is of byte-type and is of length 32 bytes.  Note th=
at the string stored in this tag may or may not be terminated with a NULL c=
haracter.  It is always prudent account for missing NULL terminator, to avo=
id buffer overflow errors in the application.  Applications are free to def=
ine their own version of the NAME tag with a longer length, though this def=
inition may conflict with other services attempting to use this tag with th=
e conventional size.  Applications needing a string tag with a longer or va=
riable length can also use MOAB=E2=80=99s variable-length tag type, though =
this will not be compatible with iMesh.
+
+<H3>Title </H3>
+
+(Data: Entity sets (file or instance); Tag(s): TITLE/C*strlen)
+
+The title tag is meant to hold the overall identifier of a mesh, written a=
t generation time or read from a file generated with a non-MOAB tool.  The =
tag length is variable, and is set by the application directly (by calling =
the tag_create function) or indirectly (by embedding the title in a file re=
ad by MOAB).
+
+<H3> Global Identifier </H3>
+
+(Data: Entity sets, entities; Tag(s): GLOBAL_ID/I)
+
+Global identifiers are used in many different contexts in applications.  G=
eometric model entities are identified by dimension and id, e.g. =E2=80=9CV=
olume 1=E2=80=9D.  Mesh vertices and elements are identified similarly in m=
esh generation codes.  Boundary conditions and material types are identifie=
d similarly.  This tag is used to store such information.  This tag is curr=
ently stored in a 32-byte integer, though this may change in the future.
+
+<H3> Geometric Model Information </H3>
+
+(Data: Entity sets; Tag(s): GEOM_DIMENSION/I, GLOBAL_ID/I, NAME/C*32, CATE=
GORY/C*32, GEOM_SENSE_2(EH[2]), GEOM_SENSE_N_ENTS(EH*N), GEOM_SENSE_N_SENSE=
S(I*N))
+
+Mesh generation is often performed starting from a geometric model, repres=
ented in some form of CAD engine.  Many of the meshes used by MOAB are gene=
rated based on the CGM library.  Geometric models contain both topological =
information (the topological entities in the geometric model) and shape inf=
ormation (the geometric shape of those entities), as well as other meta-dat=
a written to the entities in a model.  When a mesh is read from a CUBIT .cu=
b file, meta-data from the geometric model is read and represented in the M=
OAB data model, as described below. <B> Note that although MOAB reads and r=
epresents meta-data associated with the geometric model, it does not repres=
ent the geometric model itself.</B>  Therefore, shape-related information, =
e.g. the arc length of an edge or surface normal at a given point, can be r=
etrieved only from the model represented in CGM or another geometric modeli=
ng engine.
+
+The information contained in a geometric model, read into and represented =
in MOAB, consists of:
+- Model entities (vertex, edge, face, volume)
+- Topological relationships between model entities
+- Groups of model entities
+- Model entity/group ids
+- Model entity/group names
+.
+The storage of this information into MOAB's data model is described for ea=
ch type is described below.
+
+- <B>Entities </B>
+
+Entities in the geometric model (VERTEX, EDGE, FACE, VOLUME) are each repr=
esented by an entity set<sup>1</sup>.  These sets are tagged with the =E2=
=80=9CGEOM_DIMENSION=E2=80=9D tag, with integer value equal to the topologi=
cal dimension of the entity (VERTEX =3D 0, EDGE =3D 1, etc.)  These sets co=
ntain the mesh owned by the corresponding entity in the geometric model.  N=
ote this does not include mesh owned by bounding entities; thus, the set fo=
r a FACE will not contain the mesh vertices owned by bounding EDGEs in the =
geometric model.  These sets may or may not contain mesh entities of interm=
ediate dimension, e.g. mesh edges owned by a FACE or faces owned by a VOLUM=
E, depending on the application generating the mesh or the file from which =
the mesh was read.  These sets are all set-types, i.e. the order of entitie=
s in the sets is not significant, except in the case of EDGE sets, where or=
der of the mesh vertices and edges corresponds to the relative order of ver=
tices and edges at the time of mesh generation.  In MOAB, these sets are no=
n-tracking by default, i.e. entities do not have knowledge of which geometr=
y sets they are members of.
+
+<sup>1</sup>Body-type entities from CUBIT are not explicitly represented i=
n MOAB.
+
+- <B> Topological Relationships </B>
+
+In the geometric model, each FACE is bounded by zero or more EDGEs; other =
topological relationships between geometric entities exist in a similar man=
ner.  These relationships are embedded in the data model using parent/child=
 relations between entity sets.  For example, the entity set corresponding =
to a FACE will have child sets, each corresponding to a bounding EDGE, and =
parent sets, each corresponding to a VOLUME bounded by that FACE.  The rela=
tive order of sets in those parent/child lists is not significant, thus, =
=E2=80=9Cloops=E2=80=9D bounding a FACE cannot reliably be inferred from th=
is data.
+
+- <B> Groups </B>
+
+Geometric entities are sometimes assigned to application-specific groups. =
 These groups are represented using entity sets, tagged with a =E2=80=9CGRO=
UP=E2=80=9D tag whose value equals the group id.  Group sets are =E2=80=9Cs=
et=E2=80=9D-type, and are not tracking sets.  These sets contain the sets c=
orresponding to geometric entities contained in the groups in the geometric=
 model, as well as any mesh entities assigned to the group.
+
+<H4> Sense </H4>
+
+A geometric face has a natural orientation, indicated by the direction of =
the normal to the face; similarly, edges have a natural orientation determi=
ned by the direction of the tangent.  When faces bound regions, or edges bo=
und faces, they do so with a sense; if a region includes a face with forwar=
d sense, that means the face's natural normal direction points out of the v=
olume.  If a face includes an edge with forward sense, that means that if o=
ne moves along the edge in the direction of its tangent, the material of th=
e face is on the left hand side.  The sense of a face (edge) with respect t=
o a region (face) it bounds is stored using tags on the face (edge).
+
+Most models allow a face to be part of only two regions.  Therefore, to st=
ore the sense of a face with respect to regions including it, a tag with tw=
o values is used.  This tag is named GEOM_SENSE_2, and has 2 EntityHandle v=
alues.  The first value corresponds to the entity set for the region for wh=
ich that face has a forward sense, and the second to the region for which t=
hat face has a reverse sense.
+
+Edges can bound more than two faces.  Therefore, two variable-length tags =
are used, one to store the EntityHandles of the faces the edge bounds, and =
the other to store the sense with which the edge bounds the corresponding f=
ace.  These tags are named GEOM_SENSE_N_ENTS and GEOM_SENSE_N_SENSES, respe=
ctively.  These are stored as variable-length tags; see the MOAB user's gui=
de for information on how to work with tags of this type.=20
+
+The following sense values are used:
+- 0: forward
+- 1: reverse
+- -1: unnknown
+
+<H3> Material Type </H3>
+
+(Data: Entity sets; Tag(s): MATERIAL_SET/I)
+
+Most finite element and other PDE-based analysis codes require a material =
type for each cell or element in the simulation.  MOAB uses entity sets to =
store this information, in the form of entity sets.  The MATERIAL_SET tag i=
s used to identify these sets.  The value of this tag is conventionally an =
integer; in most cases this stores a user-assigned identifier associated wi=
th that material.=20
+
+CUBIT assigns material types using what it calls =E2=80=9Celement blocks=
=E2=80=9D, with each element block given a user-assigned id number and opti=
onally a name.  The CUBIT and Exodus file readers in MOAB read element bloc=
ks into MATERIAL_SET sets. =20
+
+In CUBIT, materials are typically assigned by assigning geometric volumes =
to element blocks.  Therefore, material sets often contain entity sets corr=
esponding to those volumes.  Thus, a materrial set in MOAB is unlikely to c=
ontain mesh entities directly; rather, that set contains other sets which c=
ontain mesh entities.  In these cases, mesh entities can be retrieved by pa=
ssing a =E2=80=9Crecursive=E2=80=9D flag to the appropriate function (MOAB)=
, or by calling the getEntitiesRec extension function (iMesh) provided by M=
OAB.
+
+<H3> Boundary Conditions (Dirichlet, Neumann)</H3>
+
+Data: Entity sets; Tag(s): DIRICHLET_SET/I, NEUMANN_SET/I)
+
+Boundary conditions are often specified in terms of geometric model entiti=
es, similar to material types.  MOAB uses entity sets to store this informa=
tion as well.  The DIRICHLET_SET and NEUMANN_SET tags are used to represent=
 Dirichlet- and Neumann-type boundary condition sets, resp.  By convention,=
 Neumann sets usually contain (indirectly) intermediate-dimension entities =
like edges in a 2D mesh or faces in a 3D mesh, while Dirichlet sets usually=
 contain vertices.  In addition, Neumann sets are represented as sets of fa=
ces, rather than as sides of elements.  Faces can be ordered =E2=80=9Cforwa=
rd=E2=80=9D or =E2=80=9Creverse=E2=80=9D with respect to one of the boundin=
g elements, depending on whether the right-hand normal points into or out o=
f the element.  Forward-sense faces are added to the Neumann set.  Reverse-=
sense faces are put into a separate set; that set is tagged with the NEUSET=
_SENSE tag, with value =3D -1; and that reverse set is added to the Neumman=
n set.
+
+<H3> Parallel Mesh Constructs </H3>
+
+(Data: Entity sets, entities; Tag(s): PARALLEL_PART/I, PARALLEL_PARTITION/=
I, PSTATUS/C*1, PARALLEL_SHARED_PROC/I, PARALLEL/SHARED_HANDLE/H, PARALLEL_=
SHARED_PROCS/I*NP, PARALLEL_SHARED_HANDLES/H*NP)
+
+On a parallel computer, MOAB can represent the mesh on each processor as w=
ell as information about entities shared with neighboring processors.  Some=
 of this information is also relevant even when the mesh is represented on =
a serial machine.  MOAB uses several tag and set conventions to describe th=
e parallel nature of a mesh.  This information is summarized here; for a mo=
re complete description of MOAB=E2=80=99s parallel mesh representation and =
functionality, see [ref-moabpar].
+
+- <B> Parallel partition, parts </B>
+
+Most parallel mesh applications use a domain decomposition approach, where=
 each processor solves for a subset of the domain.  The set of entities sol=
ved by a given processor is referred to as a part, and the collection of pa=
rts together is called the partition.  MOAB stores each part in an entity s=
et, marked with the PARALLEL_PART tag, whose value is the rank of the proce=
ssor assigned that part; an entity set which contains all part sets is give=
n the PARALLEL_PARTITION tag, whose value is currently meaningless.  The MB=
Zoltan tool included as a tool in MOAB can partition a mesh for parallel so=
lution, and writes the partition to the mesh in the form of parts and parti=
tions.  Both these types of sets can be accessed in a serial mesh, e.g. for=
 visualization.
+
+- <B> Part interfaces </B>
+
+When a partitioned mesh has been loaded on a parallel computer, the part o=
n a given processor may share portions of its boundary with parts on other =
processors.  These shared regions are called part interfaces, and are also =
represented using entity sets.  These sets are marked with the PARALLEL_INT=
ERFACE tag, whose value is currently meaningless.
+
+- <B> Shared processor and handle </B>
+
+For entities shared between processors, it is helpful to know locally whic=
h other processor shares an entity, and what the entity=E2=80=99s handle is=
 on the remote processor.  There are two cases which are useful to distingu=
ish, first where an entity is shared with only one other processor (referre=
d to as shared), and second when a processor is shared by more than one oth=
er processor (referred to as multi-shared).   Shared entities are given the=
 PARALLEL_SHARED_PROC and PARALLEL_SHARED_HANDLE tags, which store the rank=
 of the sharing processor and the handle of the entity on that processor, r=
espectively.  Multi-shared entities are marked with the PARALLEL_SHARED_PRO=
CS and PARALLEL_SHARED_HANDLES tags; these tags have a length NP assigned a=
t compile time in MOAB, with default values of -1 for processor rank and ze=
ro for handle (which are each invalid values for the corresponding data).  =
The processors/handles sharing a given entity are then written on the front=
 of the arrays.  So, for example, an entity on processor rank 0, shared by =
processors 1 and 2, would have a PARALLEL_SHARED_PROCS tag whose values wou=
ld be [1, 2, -1, -1, =E2=80=A6], with PARALLEL_SHARED_HANDLES values of [m,=
 n, 0, 0, =E2=80=A6], where m and n would be the handles of that entity on =
processors 1 and 2.  The shared versions of these tags are =E2=80=9Cdense=
=E2=80=9D, with default values which denote unshared entities.  The multi-s=
hared tags are sparse tags in MOAB, with no default value.
+
+- <B> Parallel status </B>
+
+In addition to the tags above, MOAB also defines the PSTATUS tag, whose bi=
ts contain information about the parallel status of a given entity.  Starti=
ng with least significant bit, these bits represent whether an entity is 1)=
 not owned, 2) shared, 3) multi-shared, 4) interface, 5) a ghost entity.  T=
he first bit being set indicates =E2=80=9Cnot owned=E2=80=9D so that the de=
fault value for this tag, of zero, corresponds to an owned, unshared entity=
, which will be the state of most entities on a given processor.
+
+<H3>Structured Mesh Parameters </H3>
+
+MOAB has a structured mesh interface for creating structured mesh (see =E2=
=80=9CScdInterface.hpp=E2=80=9D header file in MOAB source code).  Along wi=
th an internal representation that is more memory-efficient (since it does =
not need to store connectivity), MOAB also creates and tags entity sets wit=
h structured mesh parameters, which can be accessed through the normal tag =
and set interface.  The following tags are used:
+- <B>BOX_DIMS</B>: This tag stores the ijk coordinates of the lower and up=
per corner of the structured mesh box(es).
+- <B>GLOBAL_BOX_DIMS</B>: If specified when the structured mesh is created=
, a tag with this name stores the global box dimensions (which may be diffe=
rent than the local box dimensions).
+- <B>BOX_PERIODIC</B>: Stores whether the box is periodic in the i (BOX_PE=
RIODIC[0]) and j (BOX_PERIODIC[1]) directions.
+- <B>__BOX_SET</B>: Pointer to the ScdBox instance corresponding to this e=
ntity set.<sup>2</sup>
+.
+Although the structured mesh is not saved as such in HDF5-format files, th=
e entity sets and corresponding tags will be saved and restored.
+
+<sup>2</sup>The double-underscore in the tag name implies that this tag wi=
ll not be saved in a file, in this case because the ScdBox instances are no=
t preserved in a file.
+
+<H3>Spectral Mesh Constructs </H3>
+
+The Spectral Element Method (SEM) is a high-order method, using a polynomi=
al Legendre interpolation basis with Gauss-Lobatto quadrature points, in co=
ntrast to the Lagrange basis used in (linear) finite elements.  A spectral =
mesh with order O contains quadrilateral or hexahedral elements comprised o=
f (O+1)d vertices.  Spectral meshes are usually represented in one of two w=
ays, either as coarse elements which point to an array of higher-order vert=
ices (and with corner vertices represented in the normal manner), or as lin=
ear quads/hexes formed from the higher-order vertices, with each original c=
oarse quad/hex represented by Od fine quads/hexes.  Similarly, the spectral=
 variables, which are normally computed at fine vertex positions, are store=
d either on those vertices, or in lexicographically-ordered arrays on eleme=
nts (with tag values repeated on neighboring elements).  MOAB can read spec=
tral meshes from a variety of formats (at this time, including CAM-SE, HOMM=
E, and Nek5000).  Which of the above two representations are controlled by =
read options and are indicated by certain tags:
+- SPECTRAL_MESH: read option indicating that spectral elements should be r=
epresented as coarse linear quads/hexes and each element containing an arra=
y of lexicographically-ordered vertex handles
+- TAG_SPECTRAL_ELEMENTS: read option; if given, spectral variables are rep=
resented as lexicographically-ordered arrays on elements
+- TAG_SPECTRAL_VERTICES: read option; if given, spectral variables are rep=
resented as tags on vertices
+- CONN=3D<filename>: in CAM-SE, the connectivity of the spectral mesh is s=
tored by default in a file named =E2=80=9CHommeMapping.nc=E2=80=9D; this op=
tion can be given to read the connectivity from a different file
+- SPECTRAL_VERTICES: tag name for array of vertex handles
+- SPECTRAL_ORDER: tag name for spectral order, written to file set or (if =
no file set given) to interface after a spectral mesh is read
+.
+
+  \ref meta-data-info "Top"
+
+  \section meta-options Reader/Writer Options
+
+All mesh file readers and writers in MOAB take an option string as an argu=
ment.  By default, the semicolon (=E2=80=9C;=E2=80=9D) delimits individual =
options in the option string.  Options used in multiple readers are describ=
ed in this section; the options enabled in specific readers/writers are des=
cribed in the corresponding appendix at the end of this document.
+
+<H3>variable=3D<var_name>[,...]</H3>
+
+By default, all field data stored with the mesh is read with the mesh, and=
 stored as tags on the associated mesh entities.  This option lists specifi=
c variables that should be read along with the mesh (note also the =E2=80=
=9Cnomesh=E2=80=9D option, described elsewhere in this document).  The vari=
able name listed will be read into a tag with the same name.  For time-depe=
ndent variables, the time step number will be appended to the variable name=
 to form the tag name.  If no =E2=80=9Ctimestep=E2=80=9D or =E2=80=9Ctimeva=
l=E2=80=9D option is given, all time steps will be read, resulting in sever=
al tags being created.  If the =E2=80=9Cnomesh=E2=80=9D option is given, th=
e application must pass the entity set resulting from the original mesh rea=
d in to the function, that this set must contain the mesh read only from th=
at file.  The mesh in the file is checked against the mesh in the set to ve=
rify that the two correspond.  The special name =E2=80=9CMOAB_ALL_VARIABLES=
=E2=80=9D can be used to indicate that all variables should be read.  Multi=
ple variable names can be specified, separated from each other by commas.
+
+<H3>nomesh </H3>
+
+Indicates that no mesh should be read from the file.  This option is used =
in conjunction with the =E2=80=9Cvariable=3D=E2=80=9D option, to read varia=
bles and assign them as tags to a previously-read mesh.  If this option is =
used, applications should pass an entity set to the read function, which sh=
ould contain the mesh previously read from the file.
+
+<H3>timestep=3D<step_number>[, ...] </H3>
+
+Read the time step number whose time value is equal to or greater than the=
 specified time value, for the specified variable(s).  Tag names for the va=
riable(s) will be formed by appending the time step number to the variable =
name.  Multiple time step values can be specified, separated from each othe=
r by commas.
+
+  \ref meta-data-info "Top"
+
+  \section meta-references References
+
+[1]	T.J. Tautges, R. Meyers, K. Merkley, C. Stimpson, and C. Ernst, MOAB: =
A Mesh-Oriented Database, Sandia National Laboratories, 2004.
+
+[2]	L. Diachin, A. Bauer, B. Fix, J. Kraftcheck, K. Jansen, X. Luo, M. Mil=
ler, C. Ollivier-Gooch, M.S. Shephard, T. Tautges, and H. Trease, =E2=80=9C=
Interoperable mesh and geometry tools for advanced petascale simulations,=
=E2=80=9D Journal of Physics: Conference Series,  vol. 78, 2007, p. 012015.
+
+\ref meta-data-info "Top"
+
+  \section appendixA Appendix A: Summary
+
+  \subsection table1 Table 1: Summary of MOAB meta-data conventions.
+
+<table border=3D"1">
+<tr>
+<th>Convention</th>
+<th>Applies to (E=3Dent, S=3Dset)</th>
+<th>Tag(s) (type/length)</th>
+<th>Description</th>
+</tr>
+<tr>
+<td>Name</td>
+<td>E, S</td>
+<td>NAME/C*32</td>
+<td></td>
+</tr>
+<tr>
+<td>Title</td>
+<td>S</td>
+<td>TITLE/C*strlen</td>
+<td>Title of mesh</td>
+</tr>
+<tr>
+<td>Global identifier</td>
+<td>E, S</td>
+<td>GLOBAL_ID/I</td>
+<td></td>
+</tr>
+<tr>
+<td>Geometric topology</td>
+<td>S</td>
+<td>GEOM_DIMENSION/I, GLOBAL_ID/I,
+NAME/C*32,
+CATEGORY/C*32.
+GEOM_SENSE_2/EH[2],
+GEOM_SENSE_N_ENTS/EH*N,
+GEOM_SENSE_N_SENSES/I*N</td>
+<td>Sets contain mesh owned by that entity; parent/child links to bounded/=
bounding entities in geometric model</td>
+</tr>
+<tr>
+<td>Material type</td>
+<td>S</td>
+<td>MATERIAL_SET/I</td>
+<td>Set contains entities or sets assigned a common material type</td>
+</tr>
+<tr>
+<td>Boundary condition</td>
+<td>S</td>
+<td>DIRICHLET_SET/I, NEUMANN_SET/I</td>
+<td>Set contains entities or sets assigned a particular boundary condition=
; neumann sets usually contain edges (2D) or faces (3D)</td>
+</tr>
+<tr>
+<td>Parallel mesh constructs</td>
+<td>E, S</td>
+<td>PARALLEL_PART/I, PARALLEL_PARTITION/I, PSTATUS/C*1, PARALLEL_SHARED_PR=
OC/I, PARALLEL/SHARED_HANDLE/H, PARALLEL_SHARED_PROCS/I*NP, PARALLEL_SHARED=
_HANDLES/H*NP</td>
+<td> Data which describes parallel mesh</td>
+</tr>
+<tr>
+<td>Structured mesh constructs</td>
+<td>S</td>
+<td>BOX_DIMS/I*6, GLOBAL_BOX_DIMS/I*6, BOX_PERIODIC/2*I, __BOX_SET/O</td>
+<td>Data describing structured mesh </td>
+</tr>
+<tr>
+<td>Spectral mesh constructs </td>
+<td>E, S</td>
+<td>SPECTRAL_ORDER/I, SPECTRAL_VERTICES/I*(O+1)^2</td>
+<td>Data marking spectral mesh constructs</td>
+</tr>
+</table>
+=20
+  \ref meta-introduction "Back to Introduction"
+
+  \subsection table2 Table 2: Summary of MOAB conventional tag names, type=
s, and purposes.  Data types are I=3Dinteger, D=3Ddouble, C=3Dcharacter, H=
=3Dentity handle,O=3Dopaque.  Data type with *x denote length of x elements=
 of that data type.
+
+<Table border=3D"1">
+<tr>
+<th>Tag name</th>
+<th>Data type</th>
+<th>Applies to (E=3Dentity, S=3Dset)</th>
+<th>Purpose</th>
+</tr>
+<tr>
+<td>BOX_DIMS</td>
+<td>I*6</td>
+<td>S</td>
+<td>Lower and upper ijk dimensions of box, ordered (ilo, jlo, klo, ihi, jh=
i, khi)</td>
+</tr>
+<tr>
+<td>BOX_PERIODIC</td>
+<td>I*2</td>
+<td>S</td>
+<td>Indicates whether box is periodic in i (BOX_PERIODIC[0]) or j (BOX_PER=
IODIC[1])</td>
+</tr>
+<tr>
+<td>__BOX_SET</td>
+<td>O</td>
+<td>S</td>
+<td>Pointer to corresponding ScdBox instance</td>
+</tr>
+<tr>
+<td>CATEGORY</td>
+<td>C*32</td>
+<td>S</td>
+<td>String describing purpose of set; examples include =E2=80=9Cgroup=E2=
=80=9D, =E2=80=9Cvertex=E2=80=9D, =E2=80=9Cedge=E2=80=9D, =E2=80=9Csurface=
=E2=80=9D, =E2=80=9Cvolume=E2=80=9D</td>
+</tr>
+<tr>
+<td>DIRICHLET_SET </td>
+<td>I</td>
+<td>SO</td>
+<td>Entities or sets with common boundary condition</td>
+</tr>
+<tr>
+<td>GEOM_DIMENSION</td>
+<td>I</td>
+<td>S</td>
+<td>Identifies mesh entities resolving a given geometric model entity</td>
+</tr>
+<tr>
+<td>GEOM_SENSE_2</td>
+<td>EH*2</td>
+<td>S</td>
+<td> Stored on face-type geometric topology sets, values store regions hav=
ing forward and reverse sense</td>
+</tr>
+<tr>
+<td>GEOM_SENSE_N_ENTS</td>
+<td>EH*N</td>
+<td>S</td>
+<td>Stored on edge-type geometric topology sets, values store faces whose =
senses are stored in GEOM_SENSE_N_SENSES.</td>
+</tr>
+<tr>
+<td>GEOM_SENSE_N_SENSES</td>
+<td>I*N</td>
+<td>S</td>
+<td>Stored on edge-type geometric topology sets, values store senses of th=
e edge with respect to faces stored in GEOM_SENSE_N_ENTS.</td>
+</tr>
+<tr>
+<td>GLOBAL_ID</td>
+<td>I</td>
+<td>E,S</td>
+<td>Application-specific entity id</td>
+</tr>
+<tr>
+<td>MATERIAL_SET</td>
+<td>I</td>
+<td>S</td>
+<td>Entities or sets grouped by material type</td>
+</tr>
+<tr>
+<td>NAME</td>
+<td>C*32</td>
+<td>E, S</td>
+<td>User-assigned entity name(s); multiple names delimited with ?</td>
+</tr>
+<tr>
+<td>NEUMANN_SET</td>
+<td>I</td>
+<td>S</td>
+<td>Entities or sets with common boundary condition </td>
+</tr>
+<tr>
+<td>PARALLEL_PART </td>
+<td>I</td>
+<td>S</td>
+<td>Represent a part in a partition</td>
+</tr>
+<tr>
+<td>PARALLEL_PARTITION</td>
+<td>I</td>
+<td>S</td>
+<td>Represents a partition of the mesh for parallel solution, which is a c=
ollection of parts</td>
+</tr>
+<tr>
+<td>__PARALLEL_SHARED_HANDLEd</td>
+<td>H</td>
+<td>E, S</td>
+<td> Handle of this entity/set on sharing processor</td>
+</tr>
+<tr>
+<td>__PARALLEL_SHARED_PROC</td>
+<td>I</td>
+<td>E,S</td>
+<td>Rank of other processor sharing this entity/set </td>
+</tr>
+</table>
+
+\ref meta-data-info "Top"
+
+  \section appendixB Appendix B: CCMIO (Star-CD, Star-CCM+) Reader/Writer =
Conventions
+
+  \subsection table3 Table 3: Translation between CCMIO options and MOAB t=
ags.
+<Table border=3D"1">
+<tr>
+<th> Set Type</th>
+<th>CCMIO Construct</th>
+<th>MOAB Tag Name, Type</th>
+</tr>
+<tr>
+<td rowspan=3D"2">File set / Interface</td>
+<td>Title (option)</td>
+<td>=E2=80=9CTitle=E2=80=9D (C*32)</td>
+</tr>
+<tr>
+<td>CreatingProgram</td>
+<td>=E2=80=9CCreatingProgram=E2=80=9D (C*32)</td>
+</tr>
+<tr>
+<td rowspan=3D"13">Material sets</td>
+<td>Index</td>
+<td>MATERIAL_SET</td>
+</tr>
+<tr>
+<td>Label<sup>1</sup></td>
+<td>NAME</td>
+</tr>
+<tr>
+<td>MaterialId</td>
+<td>=E2=80=9CMaterialId=E2=80=9D (I)</td>
+</tr>
+<tr>
+<td>Radiation</td>
+<td>=E2=80=9CRadiation=E2=80=9D (I)</td>
+</tr>
+<tr>
+<td>PorosityId</td>
+<td>=E2=80=9CPorosityId=E2=80=9D (I)</td>
+</tr>
+<tr>
+<td>SpinId</td>
+<td>=E2=80=9CSpinId=E2=80=9D (I)</td>
+</tr>
+<tr>
+<td>GroupId</td>
+<td>=E2=80=9CGroupId=E2=80=9D (I)</td>
+</tr>
+<tr>
+<td>ColorIdx</td>
+<td>=E2=80=9CColorIdx=E2=80=9D (I)</td>
+</tr>
+<tr>
+<td>ProcessorId</td>
+<td>=E2=80=9CProcessorId=E2=80=9D (I)</td>
+</tr>
+<tr>
+<td>LightMaterial</td>
+<td>=E2=80=9CLightMaterial=E2=80=9D (I)</td>
+</tr>
+<tr>
+<td>FreeSurfaceMaterial</td>
+<td>=E2=80=9CThickness=E2=80=9D (F)</td>
+</tr>
+<tr>
+<td>Thickness</td>
+<td>=E2=80=9CThickness=E2=80=9D (F)</td>
+</tr>
+<tr>
+<td>MaterialType</td>
+<td>=E2=80=9CMaterialType=E2=80=9D (C*32)</td>
+</tr>
+<tr>
+<td rowspan=3D"5">Neumann sets</td>
+<td>Index</td>
+<td>NEUMANN_SET</td>
+</tr>
+<tr>
+<td>Label</td>
+<td>NEUMANN_SET</td>
+</tr>
+<tr>
+<td>BoundaryName</td>
+<td>NAME</td>
+</tr>
+<tr>
+<td>BoundaryType</td>
+<td>=E2=80=9CBoundaryType=E2=80=9D (C*32)</td>
+</tr>
+<tr>
+<td>ProstarRegionNumber</td>
+<td>=E2=80=9CProstarRegionNumber=E2=80=9D (I)</td>
+</tr>
+</table>
+
+Notes:
+1. If no name is present, labels the material group with =E2=80=9CMaterial=
X=E2=80=9D, where X is the index of that group.
+
+\ref meta-data-info "Top"
+
+  \section appendixC Appendix C: ExodusII Reader/Writer Conventions=20
+
+  \subsection table4 Table 4: Translation between ExodusII constructs and =
MOAB tags.
+<Table border=3D"1">
+<tr>
+<th> Data Type</th>
+<th>ExodusII Construct</th>
+<th>MOAB Tag Name, Type</th>
+</tr>
+<tr>
+<td></td>
+<td>QA records</td>
+<td>=E2=80=9CqaRecord=E2=80=9D (C*(v))<sup>2</sup></td>
+</tr>
+<tr>
+<td rowspan=3D"2">Material sets</td>
+<td>Block number</td>
+<td>MATERIAL_SET</td>
+</tr>
+<tr>
+<td>Block element type</td>
+<td>Entity type, # vertices per entity</td>
+</tr>
+<tr>
+<td rowspan=3D"2">Dirichlet sets<sup>3</sup></td>
+<td>Nodeset number</td>
+<td>DIRICHLET_SET</td>
+</tr>
+<tr>
+<td>Distribution factors</td>
+<td>=E2=80=9CdistFactor=E2=80=9D (D*(v))<sup>1</sup></td>
+</tr>
+<tr>
+<td>Neumann sets</td>
+<td>Sideset number</td>
+<td>NEUMANN_SET</td>
+</tr>
+<tr>
+<td rowspan=3D"2">Neumann sets, reverse faces3<sup>3</sup></td>
+<td>Distribution factors</td>
+<td>=E2=80=9CdistFactor=E2=80=9D (D*(v))<sup>1</sup></td>
+</tr>
+<tr>
+<td>Sides</td>
+<td>SENSE</td>
+</tr>
+<tr>
+<td>Nodes, elements</td>
+<td>node_num_map, elem_map</td>
+<td>GLOBAL_ID on nodes/elements</td>
+</tr>
+</table>
+
+Notes:
+-# Variable-length tag used for distribution factors; length for each set =
is the number of entities in
+each set, such that there is one distribution factor for each entity in th=
e set.
+-# QA records are stored as variable-length tags on file set specified on =
read. Tag is a
+concatenation of QA record strings into a single string, with '\0' used to=
 delimit lines.
+-# MOAB represents sidesets as sets of faces, rather than as sides of elem=
ents. Faces can be
+ordered =E2=80=9Cforward=E2=80=9D or =E2=80=9Creverse=E2=80=9D with respec=
t to one of the bounding elements, depending on
+whether the right-hand normal points into or out of the element. Forward-s=
ense faces are added
+to the Neumann set. Reverse-sense faces are put into a separate set; that =
set is tagged with the SENSE tag, with value =3D -1; and that reverse set i=
s added to the Neummann set.
+.
+
+  \ref meta-data-info "Top"
+
+  \section appendixD Appendix D: NC (Climate Data) Reader/Writer Conventio=
ns
+
+The climate data reader in MOAB reads files with the '.nc' filename extens=
ion. By default, this reader
+reads the whole mesh in the file and creates it as structured mesh in MOAB=
, with the mesh accessible
+through MOAB's structured mesh interface. By default, all variables and ti=
mesteps are read from the
+file, and written as tags on the mesh vertices from that file. This behavi=
or is controlled by the
+=E2=80=9Cvariable=E2=80=9D, =E2=80=9Cnomesh=E2=80=9D, =E2=80=9Ctimestep=E2=
=80=9D, and =E2=80=9Ctimeval=E2=80=9D options described earlier in this doc=
ument. If MOAB
+is compiled for parallel execution and configured with a pnetcdf reader, t=
he mesh is read in parallel,
+with a 1D or 2D decomposition designed to balance read performance and com=
munication interface
+size (for details on the partitioning method used, see the src/io/ReadNC.c=
pp source file).
+
+Mesh is put into the entity set provided to the load_file function. This e=
ntity set is also annotated with
+various tags representing information read from the file. These tags are d=
escribed in Table 5.
+
+Several other things to note about reading climate data files into MOAB:
+- Time-dependent variables: MOAB currently has no mechanism for time-depen=
dent tags. Therefore, time-dependent variables are represented using one ta=
g per timestep, with the tag name set as the variable name plus the timeste=
p index. Thus, the first few timesteps for the variable TEMPERATURE would b=
e represented in tags named TEMPERATURE0, TEMPERATURE1, etc.
+- Cell- and face-centered variables: The climate data reader currently doe=
s not do cell- and face-
+centered variables correctly.
+.
+  \subsection table5 Table 5: Summary of MOAB conventional tag names, type=
s, and purposes. Data types are I=3Dinteger, D=3Ddouble, C=3Dcharacter, H=
=3Dentity handle. Data type with *x denote length of x elements of that dat=
a type; data type with *var denote variable-length tag. Tag names with two =
underscores prepended (=E2=80=9C__=E2=80=9D) denote tags not written to a f=
ile by MOAB.
+
+<Table border=3D"1">
+<tr>
+<th> Tag name </th>
+<th>Data type </th>
+<th> Applies to (E=3Dentity, S=3Dset) </th>
+<th>Purpose </th>
+</tr>
+<tr>
+<td>__NUM_DIMS </td>
+<td>I</td>
+<td>S</td>
+<td>The number of dimensions in the netcdf file.</td>
+</tr>
+<tr>
+<td>__NUM_VARS</td>=20
+<td>I</td>
+<td>S</td>
+<td>The number of variables in the netcdf file.</td>
+</tr>
+<tr>
+<td>__DIM_NAMES </td>
+<td>C*var </td>
+<td>S </td>
+<td>The dimension names, concatenated into a
+character string, with '\0' terminating each name.
+ </td>
+</tr>
+<tr>
+<td>__DIM_NAMES=20
+</td>
+<td>C*var</td>
+<td>S</td>
+<td>The variable names, concatenated into a character
+string, with '\0' terminating each name.
+</td>
+</tr>
+<tr>
+<td><dim_name>=20
+</td>
+<td>(I or=20
+D)*va=20
+</td>
+<td>S</td>
+<td>For each dimension, the values for the dimension.
+The data type for this tag corresponds to that in the
+netcdf file. The length of this tag is the number of
+values stored for the dimension in the netcdf file.</td>
+</tr>
+<tr>
+<td>__<dim_name>_LOC_MIN MAX</td>=20
+<td>2*(I or D)</td>
+<td>S</td>
+<td>The indices (0-based) of the local min and max
+values of dimension stored locally. For spatial
+dimensions like lon or lat, this will store the
+minimum and maximum indices in the loca</td>
+</tr>
+<tr>
+<td >__<dim_name>_LOC_VAL </td>=20
+<td>(I or D)*var </td>
+<td>S</td>
+<td>The indices (0-based) of the dimension stored
+locally. This tag only makes sense for dimensions
+that can be read in multiple pieces, such as time.
+Only one of __<dim_name>_LOC_VALS and
+_LOC_MIN_MAX can be used for a given
+dimension.
+</tr>
+<tr>
+<td>__<var_name>_DIMS=20
+</td>
+<td>C*n=20
+</td>
+<td>S</td>
+<td>For each variable, the tag handles for the
+dimensions defining this variable, in netcdf
+ordering (last dimension varying fastest). The
+length of this tag, n, is # dimensions for the
+variable * sizeof(TagHandle).
+</td>
+</tr>
+<tr>
+<td><var_name><timestep_ind>=20
+</td>
+<td>(data type)</td>
+<td>E</td>
+<td>Values of the variable for timestep <timestep_ind>
+for vertices. The data type of this tag corresponds
+to that of the variable from the netcdf file.
+Timestep index is 0-based.
+</td>
+</tr>
+<tr>
+<td>__GLOBAL_ATTRIBS=20
+</td>
+<td>C*Var=20
+</td>
+<td>S</td>
+<td>The global attributes, concatenated into a character
+string, with =E2=80=98\0=E2=80=99 terminating each attribute name, =E2=80=
=98;=E2=80=99
+       separating the data type and value, and =E2=80=98;=E2=80=99
+          separating one name/data type/value from the next.
+</td>
+</tr>
+<tr>
+<td>__GLOBAL_ATTRIBS_LEN=20
+</td>
+<td>I*Var=20
+</td>
+<td>S</td>
+<td>A vector of integers, marking the end position of
+each attribute (name/data type/value) in __GLOBAL_ATTRIBS tag.
+</td>
+</tr>
+<tr>
+<td>__<var_name>_ATTRIBS=20
+</td>
+<td>C*Var
+</td>
+<td>S</td>
+<td>The variable attributes, concatenated into a
+character string, with =E2=80=98\0=E2=80=99 terminating each attribute
+   name, =E2=80=98;=E2=80=99 separating the data type and value, and =E2=
=80=98;=E2=80=99
+          separating one name/data type/value from the next.
+</td>
+</tr>
+<tr>
+<td>__<var_name>_ATTRIBS_LEN=20
+</td>
+<td>I*Var
+</td>
+<td>S</td>
+<td>A vector of integers, marking the end position of
+each attribute (name/data type/value) in
+__<var_name>_ATTRIBS tags
+</td>
+</tr>
+</table>
+
+  \ref meta-data-info "Top"
+
+  \section appendixE Appendix E: Nek5000 Reader/Writer Conventions
+
+Nek5000, or Nek, is a code that uses the spectral element method to model =
fluid, heat transfer,
+electromagnetics, and other physics. Nek uses unstructured hexahedral mesh=
es, with each hex element
+resolved by a structured grid of =E2=80=9CGauss Lebato Legendre=E2=80=9D (=
GLL) points. Nek can read meshes through
+MOAB, and can output physics variables and GLL points through MOAB as well.
+
+Since fluid is a single material in Nek, no material sets are needed. Boun=
dary conditions are mapped to
+Nek's cbc array using Neumann sets and a user-provided =E2=80=9Cusr_moab2n=
ek=E2=80=9D subroutine (for an example
+of this subroutine, see examples/moab/pipe.usr in the Nek source code). GL=
L point locations and fluid
+variables on those points are stored in tags on the hex elements. All hex =
elements have the same
+number of GLL points. The number of GLL points in each direction is stored=
 in a tag on the mesh
+instance. These tags are described in Table 6.
+
+GLL point locations and fluid variables are stored in lexicographic order,=
 similar to their storage order
+inside the Nek code.
+
+  \subsection table6 Table 6: Summary of MOAB conventional tag names, type=
s, and purposes for Nek. Data types are I=3Dinteger, D=3Ddouble, C=3Dcharac=
ter, H=3Dentity handle. Data type with *x denote length of x elements of th=
at data type; data type with *var denote variable-length tag. Tag names wit=
h two underscores prepended (=E2=80=9C__=E2=80=9D) denote tags not written =
to a file by MOAB.
+<Table border=3D"1">
+<tr>
+<th> Tag name </th>
+<th> Data Type</th>
+<th>Applies to (E=3Dentity, S=3Dset)</th>
+<th>Purpose</th>
+</tr>
+<tr>
+<td>SEM_DIMS</td>
+<td>I*3</td>
+<td>S</td>
+<td>The dimensions of the GLL mesh in each hex
+element.
+</td>
+</tr>
+<tr>
+<td>SEM_X</td>
+<td>D*nx*ny*nz</td>
+<td>E</td>
+<td>X position of GLL points (having nx*ny*nz
+values)
+</td>
+</tr>
+<tr>
+<td>SEM_Y</td>
+<td>D*nx*ny*nz</td>
+<td>E</td>
+<td>Y position of GLL points (having nx*ny*nz values)</td>
+</tr>
+<tr>
+<td>SEM_Z</td>
+<td>D*nx*ny*nz</td>
+<td>E</td>
+<td>Z position of GLL points (having nx*ny*nz values)</td>
+</tr>
+<tr>
+<td>VEL_X</td>
+<td>D*nx*ny*nz</td>
+<td>E</td>
+<td>Fluid velocities in the x direction for GLL point
+array (having nx*ny*nz values)</td>
+</tr>
+<tr>
+<td>VEL_Y</td>
+<td>D*nx*ny*nz</td>
+<td>E</td>
+<td>Fluid velocities in the y direction for GLL point
+array (having nx*ny*nz values)</td>
+</tr>
+<tr>
+<td>VEL_Z</td>
+<td>D*nx*ny*nz</td>
+<td>E</td>
+<td>Fluid velocities in the z direction for GLL point
+array (having nx*ny*nz values)</td>
+</tr>
+<tr>
+<td>TEMP</td>
+<td>D*nx*ny*nz</td>
+<td>E</td>
+<td>Fluid temperature for GLL point array (having
+nx*ny*nz values)
+</td>
+</tr>
+<tr>
+<td>PRESS</td>
+<td>D*nx*ny*nz</td>
+<td>E</td>
+<td>Fluid pressure for GLL point array (having
+nx*ny*nz values)
+</td>
+</tr>
+</table>
+  \ref meta-data-info "Top"
+	*/

Repository URL: https://bitbucket.org/fathomteam/moab/

--

This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.


More information about the moab-dev mailing list