itaps-parallel Notes from today's meeting

[seol at scorec.rpi.edu]

I think it's worthy to explain the distributed mesh design of the FMDB
before we have further iMeshP discussion.

The FMDB has 3-level hierarchy, "DB instance - mesh instance - part
handle", where all entities are assigned to a part handle when it's
created.

- one global FMDB instance per machine
- one mesh instance per process(or)
  : global FMDB instance can access N mesh instances on N process(or) machine
- multiple parts per mesh instance
  : mesh instance can access all parts contained in it through part handle
- each part has an entity container
  : when an entity is created, the part handle is required so the entity
is contained in the part.
  : mesh instance can access all mesh entities through part handle
- multiple parts are supported also in a serial mesh
  : when a mesh file is loaded in a serial machine, one part is created on
a mesh instance if no specific request for multiple parts is made.

We do not support "partition" at this moment. If we will allow multiple
partitions per process, I would rather prefer the term "part-subpart" to
"partition-part".

Assuming one mesh instance per process (the FMDB design), if we think the
definition of "partition" as a group of mesh instances which meet specific
criteria (e.g. mesh instances on neighboring processors for expediting the
communication), we will need more in depth investigation for what it
really should be.

The FMDB design described above works seamlessly both in parallel and
serial environment. IOW, we view a serial mesh as a special case of
parallel mesh, which means a serial mesh is a distributed mesh running on
one process. So conceptually and technically, we do not have any
difference between serial and parallel programs. By all means, the FMDB
provides a single set of API which works both in parallel and serial,
which is quite opposite to what we are doing for iMesh and iMeshP.

I understand our history for providing two different mesh API for serial
and parallel but not so sure if the end user will be ok with having two
different set of API's with understanding of our excuse(?) of "history".

Seegyoung


>
> Notes from 4/12/10 phone conference attended by
>
>   RPI:  Mark S., S. Yung (spelling?), Ting
>   Uwisc:  Tim T, Jason
>   UBC:  Carl
>   SNL:  Vitus, Karen
>   LLNL:  Mark M.
>   Simmetrix:  Mark B., Saurabh Tendulkar
>
> Agreed:  We will meet again on 4/19 at 8:30am PST.
>
> Action Items:
>
> 1.  Before our next meeting, write your view of the first big question
> below
> and email it to the list.  Include pros, cons, how apps will use it, how
> it
> will affect implementations, etc.  Mail it to
> itaps-parallel at lists.mcs.anl.gov.
>
> 2.  Read other peoples' write-ups.  Definitely reply for clarification of
> details, but let's save the discussion for the phone call where it is
> easier
> to follow.
>
> My notes from the meeting:
>
> Agreed:  From the iMeshP perspective, an application will typically have,
> for a given mesh, one iMesh_instance per process, containing all entities
> stored on that process.  The iMesh_instance may have a partition
> associated
> with it, with zero, one or multiple parts per process.  The underlying
> implementation may implement this design in any way desired.
>
> If I understand correctly, Simmetrix intends to implement multiple parts
> per
> process by having, internally, multiple "meshes," each representing a
> part.
> The iMesh functions would return results from the union of these meshes;
> the
> iMeshP part-specific functions would return results from individual
> meshes.
> (Does that sound correct, Mark B. and Saurabh?)
>
> The first big question:  Can an entity exist in an iMesh_instance that has
> a
> partition without being owned by any part?
> Discussion:  There are several points in iMeshP operations where an entity
> may not (yet) have a part assignment.  For example, entities are created
> with iMesh_createEnt; they can then be added to a part with
> iMesh_addEntToSet (overloaded with a part handle).  But in-between the two
> function calls, the entity exists without being in a part.  Some think
> this
> state is acceptable, with the understanding that parallel operations are
> not
> possible on entities that are not in parts.  But this state causes
> problems
> for some implementations, which assume an entity is always in some part
> (with a default part assignment set during entity creation).  If we
> require
> that an entity always be assigned to some part, we'll likely need
> additional
> interface functions, such as iMeshP_createEnt that creates an entity and
> assigns it to a part.  Even if we don't have the requirement, we could
> consider adding such functions as a convenience.
>
> The second big question:  How should we enable multiple partitions of a
> single mesh in iMeshP?
>
> The third question:  Should iMeshP_destroyPart should return an error if
> the
> part to be destroyed is not empty?
> The answer to this question depends, to some extent, on our answer to the
> first big question above.
> Cons:  Requires users to explicitly remove entities from parts before
>        deleting the parts.  This operation requires three function
>        calls:  iMeshP_getEntities, iMesh_rmvEntFromSet (overloaded with
>        part handle), iMeshP_destroyPart.
> Pros:  Prevents users from shooting themselves in the foot.
>        Users would possibly call iMeshP_getEntities and
>        iMesh_rmvEntFromSet anyway to do migration to new part.
>
>
>