[MOAB-dev] MOAB Concurrency

[Tim Tautges]

Parallel file read/write depends on which file reader/writer is being used.  Currently, only the HDF5-based reader and 
writer support what I call a "true parallel" read/write, i.e. one where only the part of the model needed on a given 
processor is read, and where each processor contributes a part of the total model stored in a single file*.  We have 
plans to broaden our support for parallel read/write this summer to other file types, but what I say in the next 
paragraph will still hold.

As for concurrent read/write, we have no plans to support this.  We view parallel file read/write as a collective 
operation (i.e. involves all processors in a given communicator) that is atomic (processes do only that until they're 
all done).  I think concurrent read and write are probably more akin to database operations, which are far enough 
outside our "sweet spot" that I can't see much need for it.

Depending on the use case you have in mind, there could be alternatives.  For example, in many climate datasets, each 
timestep is stored in a different file, and we implement a "nomesh" option for reading just the tags from all but the 
first file.  We currently restrict this to files that have the same mesh definition, but I'm sure there are workarounds 
in cases where the mesh changes.  Also, if the differences were larger, you could just read the new mesh as a separate 
mesh, and reconcile the two meshes in the database in some other way, e.g. based on an id or something.

It would help to know more about the use cases here.

- tim

* - this isn't quite true; we do have a .nc file reader that does concurrent read too; otherwise the restrictions are 
the same, though.

On 02/15/2012 03:34 PM, Christopher Mueller wrote:
> Hi All,
>
> I'm trying to get a little clarification on exactly how concurrency works with MOAB. When one process is reading a mesh
> (or more likely, data on a mesh) from a file and another process attempts to write to that file, what happens? Does the
> writer get blocked until the reader is through? Can the file be modified by the writer while a reader has an in-memory
> representation of the file? Is there a mechanism for the reader to know if the underlying file has changed (is this what
> <https://redmine.scorec.rpi.edu/anonsvn/itaps/software/tags/1.3/tools/doxygen/html/group__MeshModification.html#ga392c572f006e88ebe62bffade3be8b9>iMesh_areEHValid
> <https://redmine.scorec.rpi.edu/anonsvn/itaps/software/tags/1.3/tools/doxygen/html/group__MeshModification.html> does)?
>
> Does the same answer apply (with roles reversed) if the reader process starts after the writer process?
>
> Are there strategies in place for dealing with these situations?
>
> Thanks in advance,
> Chris

-- 
================================================================
"You will keep in perfect peace him whose mind is
   steadfast, because he trusts in you."               Isaiah 26:3

              Tim Tautges            Argonne National Laboratory
          (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
  phone (gvoice): (608) 354-1459      1500 Engineering Dr.
             fax: (608) 263-4499      Madison, WI 53706