itaps-parallel iMeshP_getNumOfTypeAll and iMeshP_getNumOfTopoAll

[Devine, Karen D]

Re-reading my response, I realize it may not be clear.

Each part would compute a contribution to the result from its owned entities.  These contributions would then be collected (e.g., MPI_Allreduce with MPI_SUM) across all parts in the partition.  The reduced result would be returned by the function.

For a given mesh with the root set, the answer returned by this function would be the same, regardless of the number of parts over which the mesh was distributed.

Sorry for the confusion.
Karen


________________________________________
From: Devine, Karen D
Sent: Monday, November 10, 2008 9:21 PM
To: Jason Kraftcheck; itaps-parallel at mcs.anl.gov
Subject: RE: itaps-parallel iMeshP_getNumOfTypeAll and iMeshP_getNumOfTopoAll

I suspect our intent was for each part to return info about only owned entities.
This function was meant to answer common questions like, "What is the global
number of elements in my mesh?"

The entity set was included for consistency with the similar iMesh interface functions.
With it, you could answer questions like, "What is the total number of faces having
a given boundary condition imposed?"  Or "What is the total number of elements in
a given material set in my multi-material simulation?"
Perhaps others can offer use cases.

The function should be easier to implement if we specify that each part returns only info
about owned entities.   The ambiguity about copies would be removed.

Karen

________________________________________
From: itaps-parallel-bounces at lists.mcs.anl.gov [itaps-parallel-bounces at lists.mcs.anl.gov] On Behalf Of Jason Kraftcheck [kraftche at cae.wisc.edu]
Sent: Monday, November 10, 2008 3:15 PM
To: itaps-parallel at mcs.anl.gov
Subject: itaps-parallel iMeshP_getNumOfTypeAll and iMeshP_getNumOfTopoAll

iMeshP_getNumOfTypeAll and iMeshP_getNumOfTopoAll are too general to
implement efficiently.  And the behavior is unclear.  Consider the following
case:
  - 2 processors
  - a vertex is on the interface, and therefore has a copy on each
  - in one proc, the 'entity_set_handle' does not contain the vertex
  - on the other, it does

Is this vertex counted?  Either way, it is not possible for every processor
to send just a count.  It is necessary to gather data from each processor
for each entity for which that processor has a copy to determine if/when to
count vertices like the above.  Further, it isn't even possible to provide
an optimized case for the root set (e.g. count owned entities on each proc)
because there's no guarantee that because the root set is passed in one
processor that it is on any other processor.

This seems like yet another case of an ill-considered and overly-complex
API.  If the intention is to get global counts (e.g. entity_set_handle ==
root_set), then I think we should just drop the entity_set_handle argument.
 If there is a realistic case for having the entity_set_handle argument, then
a) We should have functions both with and without it so that the without
case doesn't incur the horrible performance penalty of the 'with' case (or
alternately mandate that if it is the root set on one proc, it must be the
root set on all procs.)
b) It would be nice if whomever does have a use case for the 'with' case
share it, so that we have some basis upon which to decide the 'all vs any'
thing.

- jason