itaps-parallel Questions about iMeshP

[Mark Beall]

Not that it's a big deal, but it's "Sim" as in Simulation, not "Sym"  
as in Symmetry. I only mention this since, if someone went to www.symmetrix.com 
  rather than www.simmetrix.com they would probably be very confused  
as why anyone from here is on this mailing list :)

mark

On Dec 23, 2009, at 10:36 AM, Devine, Karen D wrote:

>
> The issue of multiple partitions of a mesh is one that we agreed to  
> support,
> but deferred on the details in the interest of completing the more  
> basic
> iMeshP interface.  However, we were careful to avoid features in  
> iMeshP that
> would prevent having multiple partitions of a mesh.
>
> While this capability is not needed by every application, I believe  
> it is
> still important.  With Symmetrix' actively implementing iMeshP, it  
> is time
> to fill in some of the details on multiple partitions of a mesh.  I  
> recall
> that we agreed to have a way to designate a partition as the primary
> partition, with other partitions being secondary.  But we did not go  
> much
> further than that.
>
> How is your availability in January for a phone conference  
> discussing this
> issue?
>
> Karen
>
>
> On 12/22/09 6:00 AM, "Mark Shephard" <shephard at scorec.rpi.edu> wrote:
>
>> The key thing to scaling parallel contact was understanding current  
>> work
>> load and communication patterns as the problem evolved with the  
>> changing
>> contact and accounting for it in dynamic partitioning. The use of
>> "multiple partitions" as part of the implementation may be a way to
>> account for the evolving communications patterns. Note that this is  
>> in
>> fact not a different thing than accounting for a combination of  
>> variable
>> weights and evolving computational model such as seen in parallel  
>> mesh
>> adaptation. We happen to use a predictive load balancing tool for  
>> that
>> process and do not need to formally have two partitions of the mesh  
>> at
>> one time.
>>
>> The above is not saying that iMeshP should or should not support
>> multiple partitions of a mesh instance. However, it does indicate  
>> that
>> multiple partitions of a mesh instance is not actually needed to
>> properly account for things like contact.
>>
>> The open issue I still see is that is still sounds to me like having
>> multiple partitions of a mesh instance will require substantial extra
>> data and logic that I would not want to pay why I have simple ways to
>> avoid it.
>>
>> Mark
>>
>>
>>
>> Tim Tautges wrote:
>>> That is correct, and is the reason for using a completely  
>>> different one
>>> (and was the key element of the breakthough on parallel contact,  
>>> as I
>>> understand it).
>>>
>>> - tim
>>>
>>> Mark Beall wrote:
>>>> For the known examples of needing multiple partitionings is there
>>>> generally any correlation between the partitions (meaning that
>>>> entities in one partitioning end up on the same processors as  
>>>> entities
>>>> in the other partitioning)? I assuming there is a very low  
>>>> correlation
>>>> between the partitionings, almost by definition (since,  
>>>> otherwise, why
>>>> have the other partitioning). Is that correct?
>>>>
>>>> mark
>>>>
>>>> On Dec 21, 2009, at 11:56 AM, Tim Tautges wrote:
>>>>
>>>>> Interesting.  I've thought about PIC codes in the context, but  
>>>>> always
>>>>> wondered whether apps would consider it overkill to do the  
>>>>> particles
>>>>> with vertices (which I think is what you're implying below?  Or  
>>>>> not?)
>>>>>
>>>>> - tim
>>>>>
>>>>> Devine, Karen D wrote:
>>>>>> Thanks, Tim, for providing a good example.  Here's another:  a
>>>>>> particle-in-cell code can partition the mesh one way for force
>>>>>> calculations
>>>>>> and a different way for particle calculations.  The force
>>>>>> calculations use a
>>>>>> static partitioning; the particle calculations use a dynamic
>>>>>> partitioning
>>>>>> that attempts to maintain particle balance.
>>>>>> Karen
>>>>>> On 12/18/09 9:54 AM, "Tim Tautges" <tautges at mcs.anl.gov> wrote:
>>>>>>> The best example, brought up by both Karen and me IIRC, is of
>>>>>>> large-deformation transient dynamics with contact, where
>>>>>>> the FE solution of the dynamics is solved on 3d elements on one
>>>>>>> partition, and
>>>>>>> the contact solution is solved on faces
>>>>>>> on a different partition.  Many believe it was this capability  
>>>>>>> that
>>>>>>> made truly
>>>>>>> scalable parallel FEM with contact even
>>>>>>> possible.  See e.g.
>>>>>>>
>>>>>>> Transient dynamics simulations: parallel algorithms for contact
>>>>>>> detection and
>>>>>>> smoothed particle hydrodynamics,
>>>>>>> Proceedings of the 1996 ACM/IEEE conference on Supercomputing,  
>>>>>>> 1996.
>>>>>>>
>>>>>>> That's just one reference, there are many others for that
>>>>>>> particular work.
>>>>>>>
>>>>>>> I think the details of multiple *active* partitions still need  
>>>>>>> some
>>>>>>> work, both
>>>>>>> in use cases and in how they behave under
>>>>>>> iMeshP.  Normally I wouldn't advocate such an unexplored thing
>>>>>>> being part of
>>>>>>> the initial interface definition, but in
>>>>>>> this case I think it's an important enough capability that it's
>>>>>>> justified.
>>>>>>>
>>>>>>> - tim
>>>>>>>
>>>>>>> Mark Beall wrote:
>>>>>>>> As a more general context to the questions that Saurabh  
>>>>>>>> asked, the
>>>>>>>> main
>>>>>>>> questions we have about iMeshP are issues related to having  
>>>>>>>> multiple
>>>>>>>> partitions. Although we won't be supporting that for now  
>>>>>>>> (since our
>>>>>>>> software doesn't support that), it would be helpful in  
>>>>>>>> understanding
>>>>>>>> that need if someone could give some examples of how this
>>>>>>>> functionality
>>>>>>>> would be used.
>>>>>>>>
>>>>>>>> mark
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> ================================================================
>>>>>>> "You will keep in perfect peace him whose mind is
>>>>>>>  steadfast, because he trusts in you."               Isaiah 26:3
>>>>>>>
>>>>>>>             Tim Tautges            Argonne National Laboratory
>>>>>>>         (tautges at mcs.anl.gov)      (telecommuting from UW- 
>>>>>>> Madison)
>>>>>>>         phone: (608) 263-8485      1500 Engineering Dr.
>>>>>>>           fax: (608) 263-4499      Madison, WI 53706
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> ================================================================
>>>>> "You will keep in perfect peace him whose mind is
>>>>> steadfast, because he trusts in you."               Isaiah 26:3
>>>>>
>>>>>            Tim Tautges            Argonne National Laboratory
>>>>>        (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
>>>>>        phone: (608) 263-8485      1500 Engineering Dr.
>>>>>          fax: (608) 263-4499      Madison, WI 53706
>>>>>
>>>>
>>>>
>>>
>>
>>
>
>