itaps-parallel Mesh Instances

[Tim Tautges]

Hi all,

First, and I think this is most important: in reality, we're targeting 
the API at two primary uses (and several others): a) applications 
accessing mesh data stored in some implementation of the API, and b) 
applications wrapping their representation of mesh for iMesh-enabled 
tools to work on.  Both those uses lean towards single-iMesh-instance 
applications, and up to now we've focused heavily on that (and 
intentionally, because the question has come up before).  I think this 
is a really important issue to resolve again at a high level; that is, 
are we developing these higher-level interfaces (like parallel mesh, but 
also fields eventually) to explicitly treat multiple underlying 
implementations of the same iXxx interfaces.  I'd still assert that from 
an interface point of view, we can do most of what applications will 
want while maintaining our primary focus on the single-instance case.  I 
also assert that we're not explicitly ruling out the multi-instance 
case, which can still be done (though not quite as easily).

The single instance vs. multiple instance question pertains to both 
serial and parallel applications.  Treating parallel mesh adds another 
hierarchy to the data at least, even for a single iMesh instance per 
processor.  Whether to take the multiple iMesh instances per processor 
into iPartition explicitly, or to accomplish that by assuming some 
multiplexer that iPartition doesn't know the details of, is the question.

I think there are two primary reasons for keeping iPartition focused on 
one iMesh instance per processor.  First, from a complexity point of 
view, keeping the interfaces separated makes the sum less complex. 
Second, most of our current applications are single-instance and likely 
to stay that way, and for the future ones (esp. multiphysics), it's 
still not clear that treating multiple iMesh instances per iPartition, 
without a multiplexer, would be the best way to structure a solution.

For example, I'm helping to develop a parallel multiphysics reactor 
simulation code at ANL, composed from existing components for heat 
transfer & fluid flow, neutronics, and soon structural mechanics.  Each 
of these components already operates in parallel, on its own mesh. 
Among other things, one of the physics-based things we have to do is map 
solutions between meshes, possibly with those meshes having completely 
different distributions on possibly non-overlapping sets of processors. 
  To map solutions between meshes efficiently and accurately, you start 
needing lots of mesh-based support, things like data sorting and 
searching and interpolation for example.  That motivates the need for an 
iMesh instance in which the solution remapper can construct its own data 
structures.  If one wants this remapping component to be interoperable, 
it should be constructed on top of iPartition also.

Now, by definition as iPartition is currently being discussed (in the 
multiple instance case), the application still has to deal with multiple 
instances of iMesh, whether iPartition does or not.  Having multiple 
instances treated by iPartition isn't going to relieve the application, 
or the solution remapper, from having to also deal with that issue. 
Given that, the question becomes can we accomplish what we need to with 
iPartition using a single iMesh instance, with the various applications 
we have in mind, and is that solution easier without being too 
restrictive.  So far, I think the answer is yes.

There's a lot more detail I could go into and won't, given the length of 
this message already.  I'll also add a few specific replies to the 
various email discussions as well.

- tim



Parallel mesh: this adds

Devine, Karen D. wrote:
> In yesterday's meeting, Lori asked me to add our understanding of iMesh
> instances to the background and terminology section of the combined
> document.  I am unable to do so until we come to some agreement on what mesh
> instances are and how they are to be used.
> 
> Let's start with getting agreement for serial applications.
> 
> In the iMesh user guide, I cannot find a definition of what a mesh instance
> represents.  To me as a new(ish) ITAPS user, it looks like it represents a
> single mesh.  If our interface were a C++ interface, I would say that iMesh
> is a class, and that we'd create one iMesh object for each mesh we used in
> the application.  Then we'd invoke methods on each iMesh object as needed.
> For examples, in a loosely coupled physics implementation on two meshes,
> we'd invoke meshOne's methods to do the first set of physics, and meshTwo's
> methods to do the second set of physics.
> 
> In our C interface, we can't invoke methods on an object.  What we do
> instead is what most people do when writing an object-oriented C interface:
> they pass the object to the interface functions.  It is like passing C++'s
> "this" pointer to C functions.
> 
> I think this is a natural way for users of multiple meshes to view the iMesh
> instance, and I don't see anything in the documentation to contradict it.  I
> also think we need to allow multiple iMesh instances to allow multiple
> implementations to be used at the same time via the multiplexer.
> 
> Let's get this serial scenario understood before we move on to parallel.
> 
> Karen
> 
> ------ Forwarded Message
> From: Mark Miller <miller86 at llnl.gov>
> Organization: Lawrence Livermore National Lab
> Date: Fri, 16 Nov 2007 00:21:51 -0800
> To: "itaps-parallel at mcs.anl.gov" <itaps-parallel at mcs.anl.gov>
> Subject: itaps-parallel comments on today's meeting
> 
> I think it is important to understand in detail Tim's insistence that
> there should be only one iMesh_instance object per 'parallel task'
> even to the point of aglomerating multiple different meshes of
> an application into that iMesh_instance and then distinguishing
> between them via a multiplexer. I think I understand some of his
> rationale for this but not all of it. And, I am sure I don't
> understand how multiplexing helps as my understanding is that
> the multiplexer is the iMesh instance (at least in Jason's example
> he sent us).
> 
> ------ End of Forwarded Message
> 
> 
> 

-- 
================================================================
"You will keep in perfect peace him whose mind is
  steadfast, because he trusts in you."               Isaiah 26:3

             Tim Tautges            Argonne National Laboratory
         (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
         phone: (608) 263-8485      1500 Engineering Dr.
           fax: (608) 263-4499      Madison, WI 53706