itaps-parallel Questions about iMeshP

[Devine, Karen D]

Sorry, Mark B.  I shouldn't respond to email before I've had my coffee.  :)

Thanks for your patience.
Karen



On 12/23/09 9:23 AM, "Mark Beall" <mbeall at simmetrix.com> wrote:

> Not that it's a big deal, but it's "Sim" as in Simulation, not "Sym"
> as in Symmetry. I only mention this since, if someone went to
> www.symmetrix.com
>   rather than www.simmetrix.com they would probably be very confused
> as why anyone from here is on this mailing list :)
> 
> mark
> 
> On Dec 23, 2009, at 10:36 AM, Devine, Karen D wrote:
> 
>> 
>> The issue of multiple partitions of a mesh is one that we agreed to
>> support,
>> but deferred on the details in the interest of completing the more
>> basic
>> iMeshP interface.  However, we were careful to avoid features in
>> iMeshP that
>> would prevent having multiple partitions of a mesh.
>> 
>> While this capability is not needed by every application, I believe
>> it is
>> still important.  With Symmetrix' actively implementing iMeshP, it
>> is time
>> to fill in some of the details on multiple partitions of a mesh.  I
>> recall
>> that we agreed to have a way to designate a partition as the primary
>> partition, with other partitions being secondary.  But we did not go
>> much
>> further than that.
>> 
>> How is your availability in January for a phone conference
>> discussing this
>> issue?
>> 
>> Karen
>> 
>> 
>> On 12/22/09 6:00 AM, "Mark Shephard" <shephard at scorec.rpi.edu> wrote:
>> 
>>> The key thing to scaling parallel contact was understanding current
>>> work
>>> load and communication patterns as the problem evolved with the
>>> changing
>>> contact and accounting for it in dynamic partitioning. The use of
>>> "multiple partitions" as part of the implementation may be a way to
>>> account for the evolving communications patterns. Note that this is
>>> in
>>> fact not a different thing than accounting for a combination of
>>> variable
>>> weights and evolving computational model such as seen in parallel
>>> mesh
>>> adaptation. We happen to use a predictive load balancing tool for
>>> that
>>> process and do not need to formally have two partitions of the mesh
>>> at
>>> one time.
>>> 
>>> The above is not saying that iMeshP should or should not support
>>> multiple partitions of a mesh instance. However, it does indicate
>>> that
>>> multiple partitions of a mesh instance is not actually needed to
>>> properly account for things like contact.
>>> 
>>> The open issue I still see is that is still sounds to me like having
>>> multiple partitions of a mesh instance will require substantial extra
>>> data and logic that I would not want to pay why I have simple ways to
>>> avoid it.
>>> 
>>> Mark
>>> 
>>> 
>>> 
>>> Tim Tautges wrote:
>>>> That is correct, and is the reason for using a completely
>>>> different one
>>>> (and was the key element of the breakthough on parallel contact,
>>>> as I
>>>> understand it).
>>>> 
>>>> - tim
>>>> 
>>>> Mark Beall wrote:
>>>>> For the known examples of needing multiple partitionings is there
>>>>> generally any correlation between the partitions (meaning that
>>>>> entities in one partitioning end up on the same processors as
>>>>> entities
>>>>> in the other partitioning)? I assuming there is a very low
>>>>> correlation
>>>>> between the partitionings, almost by definition (since,
>>>>> otherwise, why
>>>>> have the other partitioning). Is that correct?
>>>>> 
>>>>> mark
>>>>> 
>>>>> On Dec 21, 2009, at 11:56 AM, Tim Tautges wrote:
>>>>> 
>>>>>> Interesting.  I've thought about PIC codes in the context, but
>>>>>> always
>>>>>> wondered whether apps would consider it overkill to do the
>>>>>> particles
>>>>>> with vertices (which I think is what you're implying below?  Or
>>>>>> not?)
>>>>>> 
>>>>>> - tim
>>>>>> 
>>>>>> Devine, Karen D wrote:
>>>>>>> Thanks, Tim, for providing a good example.  Here's another:  a
>>>>>>> particle-in-cell code can partition the mesh one way for force
>>>>>>> calculations
>>>>>>> and a different way for particle calculations.  The force
>>>>>>> calculations use a
>>>>>>> static partitioning; the particle calculations use a dynamic
>>>>>>> partitioning
>>>>>>> that attempts to maintain particle balance.
>>>>>>> Karen
>>>>>>> On 12/18/09 9:54 AM, "Tim Tautges" <tautges at mcs.anl.gov> wrote:
>>>>>>>> The best example, brought up by both Karen and me IIRC, is of
>>>>>>>> large-deformation transient dynamics with contact, where
>>>>>>>> the FE solution of the dynamics is solved on 3d elements on one
>>>>>>>> partition, and
>>>>>>>> the contact solution is solved on faces
>>>>>>>> on a different partition.  Many believe it was this capability
>>>>>>>> that
>>>>>>>> made truly
>>>>>>>> scalable parallel FEM with contact even
>>>>>>>> possible.  See e.g.
>>>>>>>> 
>>>>>>>> Transient dynamics simulations: parallel algorithms for contact
>>>>>>>> detection and
>>>>>>>> smoothed particle hydrodynamics,
>>>>>>>> Proceedings of the 1996 ACM/IEEE conference on Supercomputing,
>>>>>>>> 1996.
>>>>>>>> 
>>>>>>>> That's just one reference, there are many others for that
>>>>>>>> particular work.
>>>>>>>> 
>>>>>>>> I think the details of multiple *active* partitions still need
>>>>>>>> some
>>>>>>>> work, both
>>>>>>>> in use cases and in how they behave under
>>>>>>>> iMeshP.  Normally I wouldn't advocate such an unexplored thing
>>>>>>>> being part of
>>>>>>>> the initial interface definition, but in
>>>>>>>> this case I think it's an important enough capability that it's
>>>>>>>> justified.
>>>>>>>> 
>>>>>>>> - tim
>>>>>>>> 
>>>>>>>> Mark Beall wrote:
>>>>>>>>> As a more general context to the questions that Saurabh
>>>>>>>>> asked, the
>>>>>>>>> main
>>>>>>>>> questions we have about iMeshP are issues related to having
>>>>>>>>> multiple
>>>>>>>>> partitions. Although we won't be supporting that for now
>>>>>>>>> (since our
>>>>>>>>> software doesn't support that), it would be helpful in
>>>>>>>>> understanding
>>>>>>>>> that need if someone could give some examples of how this
>>>>>>>>> functionality
>>>>>>>>> would be used.
>>>>>>>>> 
>>>>>>>>> mark
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> --
>>>>>>>> ================================================================
>>>>>>>> "You will keep in perfect peace him whose mind is
>>>>>>>>  steadfast, because he trusts in you."               Isaiah 26:3
>>>>>>>> 
>>>>>>>>             Tim Tautges            Argonne National Laboratory
>>>>>>>>         (tautges at mcs.anl.gov)      (telecommuting from UW-
>>>>>>>> Madison)
>>>>>>>>         phone: (608) 263-8485      1500 Engineering Dr.
>>>>>>>>           fax: (608) 263-4499      Madison, WI 53706
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> ================================================================
>>>>>> "You will keep in perfect peace him whose mind is
>>>>>> steadfast, because he trusts in you."               Isaiah 26:3
>>>>>> 
>>>>>>            Tim Tautges            Argonne National Laboratory
>>>>>>        (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
>>>>>>        phone: (608) 263-8485      1500 Engineering Dr.
>>>>>>          fax: (608) 263-4499      Madison, WI 53706
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 
> 
>