itaps-parallel deletion of ghost entities

[Devine, Karen D]

On 11/4/08 11:09 AM, "Jason Kraftcheck" <kraftche at cae.wisc.edu> wrote:

> Communiction of ghost data as specified in iMeshP.h:
>
> iMeshP_addGhostOf : asynchronous communication
> iMeshP_rmvGhostOf : unspecified

The draft says removal is point-to-point, non-blocking, push-and-forget.
(The comment is with the Add function; I'm fixing that now as part of
Doxygenizing the interface.)

> iMeshP_createGhostEnts : collective & blocking
> iMeshP_deleteGhostEnts : none
>
> Also, the API is somewhat confusing in that iMesh_createGhostEnts appears to
> a) communicate ghost entities when called and b) create rules defining later
> communication by iMeshP_syncMeshAll.
>
> I have two issues with this setup.
>   1) MOAB will need to do communication to remove ghosted entities unless
>      we leave things in an invalid state until the next call to syncMeshAll.
>      And if we have to discover these kinds of changes during syncMeshAll
>      it would probably be more robust to drop these functions entirely.
>      iMesh_deleteEnt or iMesh_deleteEntArr should suffice.

IMeshP_rmvGhostOf will perform communication as noted above.
IMeshP_deleteGhostEnts removes all ghosts; it should probably be a
collective operation so that all processors know to remove all ghosts (can't
remove all ghosts on one part without other parts knowing  that they are
gone.)  Good catch; I will fix this in the interface.

>   2) iMeshP_createGhostEnts should either establish rules for later
>      communication by iMeshP_syncMeshAll *or* communicate data
>      immediately, not both.
>      2a) I prefer that we drop the iMeshP_createGhostEnts and the
>          idea of deferred ghosting.  The application can just as easily
>          call iMeshP_createGhostEnts again to update ghosting data after
>          modifying the mesh.  Simpler API, less internal state, etc.

You're right; the documentation isn't quite clear.  iMeshP_createGhostEnts
should create ghosts right away -- it should do the communication when it is
called.  But it also has to store the ghosting rules so that, if
repartitioning or mesh modification of an existing mesh with ghosting
occurs, the ghosts can be re-established after the repartitioning or mesh
modification.  Re-establishing the ghosts in this second case is done by the
end of iMeshP_syncMeshAll or iMeshP_syncPartitionAll; it can be done
incrementally as, say, the mesh modification is done, but we guarantee only
that ghosting is re-established by the end of the sync functions.

>
> - jason
>
>
>
>