[MOAB-dev] parallel itaps (more patches)

[Tim Tautges]

Thanks Jed.  I'm forwarding to itaps-parallel too.  A few notes:

1. We spoke at length among the itaps team about iMeshP last week, 
resulting in some changes to the spec which will happen, particularly 
with exchanging tags.  Stay tuned, but the changes will likely improve 
things rather than make things worse.

2. Your comments are relevant as much or more to MOAB's implementation 
of iMeshP than they are to the interface in general, and my 
implementation is partial at best.  I don't do anything special with 
parts or partitions, so that support is questionable.  I will have full 
support by the SC08 conference in November, but probably won't start in 
earnest until Oct 1.

3.  We're still talking about load options, but in the meantime you're 
fine using the options below.  They won't be interoperable, but I'm 
hoping the interoperable version will look something like it (started 
discussing this on Fri).

- tim

Jed Brown wrote:
> I realize that the iMeshP interface has not stabilized, let alone been
> fully implemented yet, but I'm beginning to use it anyway :-)
> That is, I'm loading a mesh with
> 
>   PARALLEL=BCAST_DELETE;PARTITION=PARALLEL_PARTITION;PARTITION_DISTRIBUTE;PARALLEL_RESOLVE_SHARED_ENTS;
> 
> and interrogating it using iMeshP to create a global ordering from which
> I can build a function space.  It would be handy to have
> iMeshP_ExchTags() (inconsistent case in original) implemented for
> managing adaptive p-refinement. (I think this is not difficult so I'll
> look into it if nobody beats me to it.)  While implementing this stuff
> in my code, I found more inconsistencies and made a few minor bug fixes.
> 
> 
> The latest iMeshP.h mentioned on the itaps-parallel mailing list 
> 
>   http://www-unix.mcs.anl.gov/web-mail-archive/lists/itaps-parallel/2008/07/msg00016.html
> 
> does not provide a way to get a PartHandle.  You can create one, but
> cannot access existing parts without significant obfuscation (find an
> entity, get the part ID of the owning Part, then get the handle
> associated with the part ID).  The MOAB implementation has
> iMeshP_getPartsOnRank() which is only implemented when rank is the local
> rank (I modified the function to return NOT_IMPLEMENTED rather than
> erroneous part handles when the rank is remote).  There is a comment at
> the bottom of the (mailing list version of) iMeshP.h that
> iMeshP_getPartsOnRank() may not be a good idea due to memory costs to
> provide this without communication.  For my usage, I would be happy with
> the purely local
> 
> void iMeshP_getParts(iMesh_Instance instance,
>                      iMeshP_PartitionHandle partition_handle,
> 		     iMeshP_PartHandle **part_handles,
> 		     int *part_handles_allocated,
> 		     int *part_handles_size,
> 		     int *err);
> 
> The 'const' annotations are meaningless for pass-by-value, but still
> appear throughout the ITAPS headers.  Is there a reason for this?
> 
> In iMeshP_getEntStatus(), the returned status is not dependent on the
> PartHandle that is passed in.  This is probably more useful, but it's
> certainly not what the spec intended.  (Of course all implemented
> functions ignore the PartitionHandle but that is clearly a deeper
> issue.)
> 
> Bug fixes include off-by-one and indexing beyond an array.  I ran into
> another incorrectly spec'd type in iMesh_extensions.h.
> 
> Since my previous patches have not hit the repository yet, the attached
> patch is an aggregate.
> 
> Jed
> 

-- 
================================================================
"You will keep in perfect peace him whose mind is
   steadfast, because he trusts in you."               Isaiah 26:3

              Tim Tautges            Argonne National Laboratory
          (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
          phone: (608) 263-8485      1500 Engineering Dr.
            fax: (608) 263-4499      Madison, WI 53706