itaps-parallel Proposal for handling queries with parts, sets, and partitions

[Carl Ollivier-Gooch]

Hello, all.

During our last telecon, I had what I thought was an idea that would 
unify the parallel query for sets stuff.  At the time, the idea was 
vaguely formed, and I hadn't thought through all of the consequences.  I 
may still not have, but I've at least got it to the point of being ready 
to drag it out into the light of day.

Essentially, the issue is that we want to have much the same set of 
queries for mesh instances, parts, and partitions, but we haven't worked 
out how to handle this without exploding the number of functions.

In the serial interface, we have lots of queries that take both a mesh 
instance and an entity set argument.  In the now more-or-less deprecated 
SIDL paradigm, the mesh instance in an object (in the C++ sense), and we
unified global and entity set calls by creating a root set that is,
essentially, shorthand for "everything in the mesh instance".

In parallel, the mesh instance resides at the process level in a
partition / process / part heirarchy (black / red, green in the attached
picture; yes, it's a very obvious illustration, but perhaps useful for
reference) .  We also want to be able to query partitions and parts
(although for partitions we only want to query for numbers of things,
not lists of things).  Also, we'd like to able to query sets other than
the root set (like that lovely blue set that spans a bunch of parts...).

I propose that we think about both parts and mesh instances as coverings 
of the global mesh (they are), and that we also think about parts, mesh
instances, and the partition as containing -all- data about some 
(topologically and/or geometrically compact subset) or all of the global 
mesh (as opposed to entity sets, which are deliberately more selective). 
  Yes, I know they're all collections of entities, but stick with me 
while I justify why I think this semantic distinction is worthwhile. 
Also yes, we haven't decided yet (have we?) whether a part will 
officially contain everything or just the entities that were 
partitioned; I don't -think- that any of what I'm about to propose is 
adversely affected, either way.

Finally, it's worth noting that, while a partition contains many parts
distributed over many processes, each process is associated with a
unique partition (MPI communicator or equivalent) and each part is
associated both with a unique process and a unique partition.  So
specifying both part handle and partition handle is actually redundant.

Given that backgroun, I propose that we overload our current query
functions so that any place where a mesh instance is currently usable, a
part handle or partition handle is also usable, and continue to use
either a bona fide entity set or the (global placeholder) root set as
the second argument.  In this scenario, we would have, for instance:

             / partition handle
getNumOfType| mesh instance    , root set handle   , type, result, err )
             \ part handle        entity set handle

The other iMesh function that is obviously of this type is getNumOfTopo.

A somewhat larger collection of functions would be able to take both
mesh instances and part handles as the first argument, but not partition
handles: getAllVtxCoords, getVtxCoordIndex, getEntities.

My guess is that iterator functions will fall into the second category.

Many of the set queries (number of sets, number of children, identities 
of sets and children) are a bit more ambiguous here, to my surprise: 
essentially, if you ask for the number of sets contained in a given set 
for a particular part, you presumably mean the number of contained sets 
that intersect that part.  While I don't have a problem with that 
definition, I'm not sure it's an entirely straightforward one, either 
conceptually or in implementation.  But I haven't thought about it in a 
great deal of detail, either...  One way to finesse this would be to 
create sets at the part level rather than the mesh instance level; then 
all is easy again.

Now, there are a couple of beneath-the-hood requirements for
implementations here:

1.  All part handles must be unique, even in the presence of multiple
     partitions.  Pointer-type handles will easily satisfy this;
     integer-type handles may need to reserve some bits for partition ID
     and some for part ID.

2.  In most (all?) implementations, finding the result for calls with
     (part handle, entity set handle...) will require an implementation
     to do some sort of intersection internally.  This will prove
     especially challenging for iterator in the presence of mesh
     modification and/or migration.

I recognize also that this paradigm shift potentially ambiguates the use
of calls like addEntToSet for adding entities to a part; we can either
overload those functions (probably renamed to addEntToCollection) or
create new addEntToPart functions (I think there are only going to be
four of these: add/rmv single/array).

Okay, so there's my proposal.  I'll now stand back and let people poke
holes in the idea; that may not prove to be too difficult.

Carl

-- 
------------------------------------------------------------------------
Dr. Carl Ollivier-Gooch, P.Eng.                   Voice: +1-604-822-1854
Associate Professor                                 Fax: +1-604-822-2403
Department of Mechanical Engineering             email: cfog at mech.ubc.ca
University of British Columbia              http://www.mech.ubc.ca/~cfog
Vancouver, BC  V6T 1Z4                  http://tetra.mech.ubc.ca/ANSLab/
------------------------------------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: schematic.jpg
Type: image/jpeg
Size: 60179 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/itaps-parallel/attachments/20071213/84c242a6/attachment.jpg>