itaps-parallel Assignments for parallel interface, due before 2/5 phone conf

[Mark Shephard]

Karen,

One quick input on:

 > -  "When we specify a part handle, must we always also specify a 
partition handle?  That is, should a part be uniquely identified by the 
pair (partition_handle, part_handle), just as entities are uniquely 
identified by (mesh_instance, entity_handle)?   "No" is the most 
convenient answer to this question; if parts are uniquely identified by 
only the part handle, we can do the entity-set argument overloading that 
we discussed at bootcamp.  We can also remove the partition_handle 
argument from many of the functions.  I had previously argued for use of 
the tuple, but many things would be easier if we didn't need it.  Can 
all the implementations generate part handles that are unique within an 
iMesh_Instance, rather than within a partition?"


What follows does not directly indicates an answer to your question. 
However, today one of our research meetings where we are working on 
getting mesh adapt to 23K processors on the Blue Gene, we found that 
having a global numbering for the equivalent of part boundary entities 
might be a problem for scaling on Blue Gene like machines. Without 
trying to explaining the specific problem we came across (it would take 
a while) because we currently have global part ID's, the one potential 
solution to avoid needing to wasting memory proportional to at least the 
number of processors (an issue when we will have 1M or more of them) was 
to assign an owner to each part interface in which case the order of 
memory usage reduces from the total number of parts boundaries on each 
processor something like the number of part boundaries times the number 
of parts sharing those boundaries. We find that for properly partitioned 
meshes, both of these numbers tend to go no higher that a fined number 
independent of the number of processors. Thus, I am starting to worry 
about anything that has memory requirements proportional to the number 
processors in the system being required.

Mark



Devine, Karen D wrote:
> In the last phone conf, someone suggested that we put together a draft of
> the syntax for the capabilities on which we agreed.  So I added syntax from
> Carl's, Onkar's and Tim's past emails to our Combined document.  For
> readability, I turned the file into a C header file; the discussions from
> the Combined document are now comments in the C file.  The file is attached.
> 
> For now, I'd rather not discuss:
> -  values of "prefix_".
> -  my use of "//" for comments being non-standard for some C compilers.
> -  my inconsistent use of "const" (unless I used it for a return argument --
> oops!)
> -  whether or not this file would actually compile.
> 
> Here are your assignments:
> 
> ALL:  Please give your opinions of the questions below. Some are simple
> questions resulting from the fact that I'm not used to ITAPS code
> conventions.
> 
> Mark M. and Lori:
> -  Review all functions at a high level to identify gaps and duplications in
> capability.  Suggest functions that should be added or removed.
> -  Review all functions at a high level to determine whether our syntax is
> generally consistent with ITAPS' conventions and generally consistent within
> the document.  You shouldn't catch every detail; just let us know whether
> someone who is comfortable with other ITAPS interfaces would find this
> interface comfortable as well.
> 
> Mark M and Onkar:
> -  Review all functions at a high level and indicate whether their use of
> communication will be definite, likely, or nonexistent.  If they incur
> communication, will the communication be global (e.g., MPI_Allreduce) or
> point-to-point (e.g., MPI_Send/MPI_Recv)?  We'll eventually modify the
> function names to indicate whether or not they incur communication.
> 
> Onkar:
> -  Review in detail the Entity Functionality section.  This section has the
> entity copying capability, including the add/remove copies that you
> discussed in the last phone conf.
> 
> Vitus:
> -  Propose the communication functions needed to support
> prefix_sendEntArrToParts and prefix_receiveEntArrToParts.
> 
> Carl:
> -  Should functions prefix_getEntSets and prefix_getNumEntSets be added to
> Carl's sixteen functions?  They appear to be analogous to
> prefix_getEntities and prefix_getNumOfType.
> -  Review in detail the Part Functionality section.
> 
> Tim:
> -  Review in detail the Partition Functionality section.
> 
> ------------------------
> Questions for everyone:
> 
> -  When we specify a part handle, must we always also specify a partition
> handle?  That is, should a part be uniquely identified by the pair
> (partition_handle, part_handle), just as entities are uniquely identified by
> (mesh_instance, entity_handle)?   "No" is the most convenient answer to this
> question; if parts are uniquely identified by only the part handle, we can
> do the entity-set argument overloading that we discussed at bootcamp.  We
> can also remove the partition_handle argument from many of the functions.  I
> had previously argued for use of the tuple, but many things would be easier
> if we didn't need it.  Can all the implementations generate part handles
> that are unique within an iMesh_Instance, rather than within a partition?
> 
> -  With iterators over parts or part boundaries, is it necessary to have
> separate interface functions for getNext, reset, and end, or will the
> standard iMesh functions for getNext, reset, and end suffice?  That is, once
> an iterator is initiated, do we need to continue passing the part handle
> and, perhaps, neighbor-part id, to the iterator, or can the iterator
> "remember" these just as it remembers entity type and topology?  If we can
> re-use the iMesh getNext, reset and end, we can remove six functions from
> Carl's sixteen functions, bringing the count back to his original estimate
> of ten.
> 
> -  Which functions should operate on single parts and/or entities, and which
> should have array versions (function_name and function_nameArr)?  I included
> both wherever we had a comment "Need single-part and array-part versions" in
> the combined document, but I won't be offended if we remove some of the
> functions.
> 
> -  Since global IDs for entities are not used anywhere in the interface, I
> propose their generation and use should be a service on top of the interface
> rather than part of the interface.  What say you?
> 
>