itaps-parallel Revised iMeshP interface

[Mark Miller]

On Mon, 2008-09-15 at 13:25 -0600, Devine, Karen D wrote:
> I revised the iMeshP interface and committed it to the svn repository at
> RPI:  https:// svn.scorec.rpi.edu/svn/TSTT/Interface/iMeshP
> 
> Here's what remains to be done.
> 
> For the slides due this week:
> -  Draft an interface for "pulling" tag data from shared part-boundary
> entities to the owners.  For the slides, we'll need a function name and a
> description of what it does.  That is, we'll have to decide whether it
> performs a numerical operation (e.g., MPI_SUM) or just gathers the received
> data into an array on which the application can perform operations.  Would
> anyone like to make the first proposal with rough syntax?  If not, I'll
> volunteer someone ---  or worse yet for the implementers, I'll draft a
> proposal myself.  :)

If the interface permits the caller to pass an MPI_Op pointer (which is
all MPI_SUM really is), then I think that would permit the ITAPs
implementation to take advantage of optimizations from an MPI
implementation as well. For example, an MPI_reduce operation can be
implemented many, many ways that allow the numerical operation being
performed to be scheduled optimally relative the communication necessary
to perform it. I am not saying any MPI implementation actually does
that. I am just saying the interface makes it possible. And if the
interface DID NOT permit the caller to pass an MPI_Op pointer, then such
optimizations would NOT be possible.

> 
> BTW:  This is an issue we identified earlier and forgot about; this note was
> in the draft:
> "Tim envisions applications (e.g., FETI methods) updating tag data
> in their copies that they would like to accumulate back to the
> owner.  Xiaolin said that he writes in his ghosts, but doesn't send
> those values back to the owner.  Currently, we have the ability
> to send tag data only from owners to ghosts.  Tim will look at this issue
> and propose a solution."
> 
> For SC08:
> -  Need a set of basic common options to iMeshP_load and iMeshP_save to
> allow elementary interoperability with VTK files.  At bootcamp, Tim and Ken
> volunteered to propose a set of common options.

What kind of VTK files are you talkin' here? ascii/binary or both? xml?
the new 'parallel' VTK formats?

> 
> -  At bootcamp, a question arose about what getNumCopies and getCopies would
> return when they were passed a ghost-entity handle.  At bootcamp, we said
> these functions would return errors.  But as I read the document today, I
> found the following:  "Functions getNumCopies, getCopies, getCopyParts, and
> getCopyOnPart have different behavior for ghost and part-boundary copies.
> Ghosts will return only itself and its owner in getCopies; part-boundary
> entities will return copies on other parts as well."  Which behavior do you
> like better?
> 
> Here's a summary of what I changed in DraftInterface.h as a result of the
> bootcamp:
> -  Added iMeshP_load and iMeshP_save.  IMeshP_load will populate both a mesh
> instance (with entities, sets, etc.) and a partition handle (with parts and
> entities).
> -  Changed iMeshP_getNumParts to iMeshP_getNumGlobalParts.
> -  Added iMeshP_getNumLocalParts and iMeshP_getLocalParts.
> -  Changed iMeshP_exchTags (and similar functions) to iMeshP_pushTags and
> clarified documentation on their function.  Removed part ID argument from
> the functions, as updating tags in some copies but not all copies is likely
> a bad idea.
> -  In iMeshP_createGhostEnt, added flag indicating whether non-owned
> part-boundary entities should be ghosted according to the ghosting rules.
> -  Fixed a few comments, but did not thoroughly review all comments.
> 
> 
> 
-- 
Mark C. Miller, Lawrence Livermore National Laboratory
email: mailto:miller86 at llnl.gov
(M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!)
(Th/F)  (530)-753-8511 (!!LLNL BUSINESS ONLY!!)