itaps-parallel Neighbor definition

[Tim Tautges]

Thanks Mark, I agree with all that.  Slight comments below.

Mark Miller wrote:
> Hi Tim,
> 
> This is very clear to me and I agree with it. But, since some of the
> discussion after Lori left the conversation is missing, I'd like to take
> a stab at adding that here simply in order to take a snapshot of the
> history of this discussion...
> 
> We basically went full circle in our discussion. The conversation
> started with the notion that part neighbors are based upon the sharing
> of mesh entities. An objection was raised because we can
> often think of circumstances in which two parts need to exchange
> data across the 'boundary' between them even when they DO NOT
> necessarily share mesh entities. Rercusive Coordinate Bisection and
> Finite Volume computations were raised as two examples. In the
> former, RCB, I believe we also made arguments that there does in
> fact exist a 'shared mesh entity' analog in that there are mesh edges
> that are 'cut' by the RCB algorithm and it is those edges that wind
> up being 'shared' between parts [I did not consider this yesterday
> but what if such edges are never actually instantiated in iMesh?
> Would this make the argument theoretically correct but practically
> speaking not implementable?]

I wondered about this and ended up removing a clause about shared 
entities being explicitly represented for this reason.  For example, an 
application may request 2d neighbors for a part.  In this case these 
entities should be instantiated in some way (at least enough to assign a 
handle for them).  1d neighbors wouldn't need to be instantiated in this 
case, even though technically they'd still exist.

> 
> We observed that there are likely to be many situations in which
> processors (or the parts they own) need to communicate that have
> nothing to do with underlying mesh topology. So the question arose,
> do we expect our Part/Partition model and implementation to support
> a more general notion of 'neighbor' or one that is specific to
> mesh topology. Ultimately, we argeed on the latter.
> 
> We also argued that a notion of neighbor based solely on mesh
> topology is a 'sufficient bootstrap' to enable applications requiring a
> more general notion of 'neighbor' to affect their communication
> patterns. [As an aside, I don't think we considered this argument
> in great detail and I wonder if it is in fact true in all cases that
> more general notions of neighbor can always be derived from the
> topologically specific one being proposed here. I'm inclined NOT to
> believe that. At the same time, I think it is also arguable that such
> situations are too 'application specific' to be worried about
> explicitly supporting in the Part/Parition model we are developing.]
> 

This is sort of a circular argument, but I'll make it anyway... more 
general notions of neighbors that involve topology will be derivable 
from topology, and those that aren't are likely not derivable from 
mesh-intrinsic information, and are therefore application-specific.

> We also agreed that our intention in developing the Part/Partition
> model on top of iMesh is to support the topologically common
> operations most applications will need and not necessarily ALL
> possible communication patterns applications might use.
> 
>  One question we need to re-visit is what are the implications
> (storage and/or performance), if neighbor relationships other than
> those determined by topology, are not supported?
> 

Since these relationships should be storable under our current data 
model, I think applications can decide whether to use memory or cpu time 
to recover them.  This does beg the question though about whether we can 
tag parts (you all probably know my preference on that).

- tim

> Mark
> 
> Tim Tautges wrote:
> 
>> A question arose in yesterday's telecon on what constitutes a part
>> neighbor.  After some discussion, we came up with the following
>> definition and rationale.
>>
>> Def: for a given part, a part neighbor of dimension d is any other part
>> which shares mesh entities of dimension d with that part (i.e. the
>> entity is represented on both parts).
>>
>> Rationale:
>> - it is desirable to tie the definition of a part neighbor with
>> information intrinsic to the mesh, i.e. one could derive information
>> about part neighbors using only information about the mesh entities and
>> the partition of entities over processors
>>
>> - in general, applications will have other criteria determining which
>> other processors a given processor communicates with; that information
>> can be stored in iMesh using sets and tags already, and applications
>> will presumably be able to pass those things to communication functions
>> in the itaps parallel interface (thus driving the application-specific
>> communication patterns)
>>
>> Question: are ghosted/copied entities between parts considered shared?
>> I would assert that ghosting/copying should be between processors, not
>> parts, so this question is not relevant with respect to part neighbors.
>>
>> - tim
>>
>> --
>> ================================================================
>> "You will keep in perfect peace him whose mind is
>>    steadfast, because he trusts in you."               Isaiah 26:3
>>
>>               Tim Tautges            Argonne National Laboratory
>>           (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
>>           phone: (608) 263-8485      1500 Engineering Dr.
>>             fax: (608) 263-4499      Madison, WI 53706
> 
> --
> Mark C. Miller, Lawrence Livermore National Laboratory
> email: mailto:miller86 at llnl.gov
> (M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!)
> (Th/F)  (530)-753-8511 (!!LLNL BUSINESS ONLY!!)
> 
> 
> 

-- 
================================================================
"You will keep in perfect peace him whose mind is
   steadfast, because he trusts in you."               Isaiah 26:3

              Tim Tautges            Argonne National Laboratory
          (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
          phone: (608) 263-8485      1500 Engineering Dr.
            fax: (608) 263-4499      Madison, WI 53706