itaps-parallel Questions about iMeshP

[Mark Shephard]

I do not remember the issue of what is meant by multiple partitions of a 
mesh instance being really quantified. If I recall that was the starting 
question on this.

The inconsistency I recall was that a partition is a set of parts where 
iMesh can deal with each part locally. I thought the question that then 
gave us was of having two copies of a mesh (two set of iMesh on parts) 
in one mesh instance which is implied by 2 partitions of that mesh instance.

(I actually do not remember hearing of a use case for wanting two 
complete partitions of the same mesh (not the same as repartitioning). 
Was it some odd multiphysics case where both physics used the same mesh, 
but wanted different partitions of it?)

Mark

Tim Tautges wrote:
> 
> 
> Jason Kraftcheck wrote:
>> The responses below are my interpretation of these things only.
>>
>>
>> Saurabh Tendulkar wrote:
>>> Hi,
>>>
>>> We are trying to implement the iMeshP interface to our software. We have
>>> the following questions and are hoping someone could help us with these:
>>>
>>> 1. What does the mesh instance represent in iMeshP? Is it essentially
>>> the on-process subset of entities of some global mesh? Where does the
>>> control of the global mesh lie in this case? It does not appear to be
>>> with the partition, as there can be multiple partitions of the same 
>>> mesh.
>>>
>>
>> What does "control of the global mesh" entail?  What specific 
>> functionality
>> is missing from the API?  If it isn't an API issue, then put the 
>> "control of
>> the global mesh" wherever it best fits in your implementation.
>>
> 
> I've interpreted the global mesh as a set of mesh instances, one per 
> process, with the collection established by the application.  The v0.8 
> iMeshP.h describes the global mesh as being associated with a partition 
> and an MPI communicator, but that description doesn't allow for multiple 
> active partitions, which in some cases I think will be necessary.
> 
>>
>>> 2. It appears that the mesh contains the entities and it is the
>>> partition that controls how the entities lie in parts. This seems to
>>> mean that an entity can be in the mesh but not in a part (for example,
>>> when a part that contains entities is deleted). Is this fundamentally
>>> disallowed? Because various aspects of iMeshP are not defined if this is
>>> allowed. For example, if the entity was owned by the part that was
>>> deleted, who owns the entity now? And, how does one know whether an
>>> entity can be deleted from the mesh (next question)?
>>>
> 
> I think part ownership of entities should only be interpreted in the 
> context of a parallel distribution of mesh, rather than being something 
> intrinsic to the representation of those entities in an iMeshP 
> instance.  So, an entity can exist outside a part.  Since we handle 
> modification at a fairly low level in other parts of iMesh, I would 
> expect that if an application requests deletion of a part, the entities 
> in that part have either been moved to another part or removed from the 
> partition.
> 
>>
>> I think we have to allow for this case, even though it may result in some
>> iMeshP calls failing for such entities.  If there are multiple active
>> partitions, can we practically require that all entities be in all
>> partitions?  Even those of different dimension?  If I have a partition 
>> that
>> distributes volume elements and a different one that distributes surface
>> elements, what part of the volume mesh partition is a surface element in?
>>
>>
>>> 3. Who controls the deletion of entities when they are no longer needed
>>> on-process - eg. if they are migrated away, especially while keeping in
>>> mind that there may be multiple partitions (the entity may be needed on
>>> one and not on another).
>>>
>>
> 
> Yeah, good question.  I don't think anyone's far enough along 
> implementing or using iMeshP to have experience with that.
> 
>> I'm not sure about this.  Given that an entity may exist only in some 
>> subset
>> of the partitions, the two options appear to be: a) the application is
>> responsible for deletion or the entity is never deleted or b) the
>> implementation does some kind of reference counting such that it can 
>> delete
>> the entity when it is a member of no partition.
>>
>>> 4. It is mentioned that a part can act as an entityset. Since entitysets
>>> can have parent-child and containment relationships with other
>>> entitysets, does this mean that parts need to have them too? If so, it
>>> is undefined what this means with respect to the partitioning. If not,
>>> what is the expected course of action when a part is used in an iMesh
>>> entityset function call which would result in a hierarchical 
>>> relationship?
>>>
>>
>> The only API requirement is that the functions for querying the 
>> contents of
>> sets also work for parts.  Whether or not those functions that 
>> *modify* sets
>> (either contents or parent/child links) can be called on a part is
>> implementation-dependent.
> 
> Actually, the v0.8 iMeshP.h states that *all* functions in the serial 
> interface taking sets should also work for parts.  So, I think that 
> includes things like parent/child relations.  Same goes for contains 
> relations.  I think that does imply the need for a recursive getEntities 
> function (which I have in a set of extensions to iMesh, BTW).
> 
> - tim
> 
>>
>> - jason
>>
>