itaps-parallel Questions about iMeshP

[Devine, Karen D]

The issue of multiple partitions of a mesh is one that we agreed to support,
but deferred on the details in the interest of completing the more basic
iMeshP interface.  However, we were careful to avoid features in iMeshP that
would prevent having multiple partitions of a mesh.

While this capability is not needed by every application, I believe it is
still important.  With Symmetrix' actively implementing iMeshP, it is time
to fill in some of the details on multiple partitions of a mesh.  I recall
that we agreed to have a way to designate a partition as the primary
partition, with other partitions being secondary.  But we did not go much
further than that.

How is your availability in January for a phone conference discussing this
issue?

Karen


On 12/22/09 6:00 AM, "Mark Shephard" <shephard at scorec.rpi.edu> wrote:

> The key thing to scaling parallel contact was understanding current work
> load and communication patterns as the problem evolved with the changing
> contact and accounting for it in dynamic partitioning. The use of
> "multiple partitions" as part of the implementation may be a way to
> account for the evolving communications patterns. Note that this is in
> fact not a different thing than accounting for a combination of variable
> weights and evolving computational model such as seen in parallel mesh
> adaptation. We happen to use a predictive load balancing tool for that
> process and do not need to formally have two partitions of the mesh at
> one time.
> 
> The above is not saying that iMeshP should or should not support
> multiple partitions of a mesh instance. However, it does indicate that
> multiple partitions of a mesh instance is not actually needed to
> properly account for things like contact.
> 
> The open issue I still see is that is still sounds to me like having
> multiple partitions of a mesh instance will require substantial extra
> data and logic that I would not want to pay why I have simple ways to
> avoid it.
> 
> Mark
> 
> 
> 
> Tim Tautges wrote:
>> That is correct, and is the reason for using a completely different one
>> (and was the key element of the breakthough on parallel contact, as I
>> understand it).
>> 
>> - tim
>> 
>> Mark Beall wrote:
>>> For the known examples of needing multiple partitionings is there
>>> generally any correlation between the partitions (meaning that
>>> entities in one partitioning end up on the same processors as entities
>>> in the other partitioning)? I assuming there is a very low correlation
>>> between the partitionings, almost by definition (since, otherwise, why
>>> have the other partitioning). Is that correct?
>>> 
>>> mark
>>> 
>>> On Dec 21, 2009, at 11:56 AM, Tim Tautges wrote:
>>> 
>>>> Interesting.  I've thought about PIC codes in the context, but always
>>>> wondered whether apps would consider it overkill to do the particles
>>>> with vertices (which I think is what you're implying below?  Or not?)
>>>> 
>>>> - tim
>>>> 
>>>> Devine, Karen D wrote:
>>>>> Thanks, Tim, for providing a good example.  Here's another:  a
>>>>> particle-in-cell code can partition the mesh one way for force
>>>>> calculations
>>>>> and a different way for particle calculations.  The force
>>>>> calculations use a
>>>>> static partitioning; the particle calculations use a dynamic
>>>>> partitioning
>>>>> that attempts to maintain particle balance.
>>>>> Karen
>>>>> On 12/18/09 9:54 AM, "Tim Tautges" <tautges at mcs.anl.gov> wrote:
>>>>>> The best example, brought up by both Karen and me IIRC, is of
>>>>>> large-deformation transient dynamics with contact, where
>>>>>> the FE solution of the dynamics is solved on 3d elements on one
>>>>>> partition, and
>>>>>> the contact solution is solved on faces
>>>>>> on a different partition.  Many believe it was this capability that
>>>>>> made truly
>>>>>> scalable parallel FEM with contact even
>>>>>> possible.  See e.g.
>>>>>> 
>>>>>> Transient dynamics simulations: parallel algorithms for contact
>>>>>> detection and
>>>>>> smoothed particle hydrodynamics,
>>>>>> Proceedings of the 1996 ACM/IEEE conference on Supercomputing, 1996.
>>>>>> 
>>>>>> That's just one reference, there are many others for that
>>>>>> particular work.
>>>>>> 
>>>>>> I think the details of multiple *active* partitions still need some
>>>>>> work, both
>>>>>> in use cases and in how they behave under
>>>>>> iMeshP.  Normally I wouldn't advocate such an unexplored thing
>>>>>> being part of
>>>>>> the initial interface definition, but in
>>>>>> this case I think it's an important enough capability that it's
>>>>>> justified.
>>>>>> 
>>>>>> - tim
>>>>>> 
>>>>>> Mark Beall wrote:
>>>>>>> As a more general context to the questions that Saurabh asked, the
>>>>>>> main
>>>>>>> questions we have about iMeshP are issues related to having multiple
>>>>>>> partitions. Although we won't be supporting that for now (since our
>>>>>>> software doesn't support that), it would be helpful in understanding
>>>>>>> that need if someone could give some examples of how this
>>>>>>> functionality
>>>>>>> would be used.
>>>>>>> 
>>>>>>> mark
>>>>>>> 
>>>>>>> 
>>>>>> --
>>>>>> ================================================================
>>>>>> "You will keep in perfect peace him whose mind is
>>>>>>   steadfast, because he trusts in you."               Isaiah 26:3
>>>>>> 
>>>>>>              Tim Tautges            Argonne National Laboratory
>>>>>>          (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
>>>>>>          phone: (608) 263-8485      1500 Engineering Dr.
>>>>>>            fax: (608) 263-4499      Madison, WI 53706
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> --
>>>> ================================================================
>>>> "You will keep in perfect peace him whose mind is
>>>>  steadfast, because he trusts in you."               Isaiah 26:3
>>>> 
>>>>             Tim Tautges            Argonne National Laboratory
>>>>         (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
>>>>         phone: (608) 263-8485      1500 Engineering Dr.
>>>>           fax: (608) 263-4499      Madison, WI 53706
>>>> 
>>> 
>>> 
>> 
> 
>