[MOAB-dev] MOAB Concurrency

[Christopher Mueller]

Tim,

Thanks for the information.  Lots of good food for thought!

Chris

From: Tim Tautges <tautges at mcs.anl.gov<mailto:tautges at mcs.anl.gov>>
Date: Wed, 15 Feb 2012 18:50:01 -0500
To: Christopher Mueller <cmueller at asascience.com<mailto:cmueller at asascience.com>>
Cc: "moab-dev at mcs.anl.gov<mailto:moab-dev at mcs.anl.gov>" <moab-dev at mcs.anl.gov<mailto:moab-dev at mcs.anl.gov>>, Guy De Wardener <gdewardener at asascience.com<mailto:gdewardener at asascience.com>>, David Stuebe <DStuebe at asascience.com<mailto:DStuebe at asascience.com>>, don brittain <don.brittain at instanteffects.com<mailto:don.brittain at instanteffects.com>>
Subject: Re: [MOAB-dev] MOAB Concurrency

Parallel file read/write depends on which file reader/writer is being used.  Currently, only the HDF5-based reader and
writer support what I call a "true parallel" read/write, i.e. one where only the part of the model needed on a given
processor is read, and where each processor contributes a part of the total model stored in a single file*.  We have
plans to broaden our support for parallel read/write this summer to other file types, but what I say in the next
paragraph will still hold.

As for concurrent read/write, we have no plans to support this.  We view parallel file read/write as a collective
operation (i.e. involves all processors in a given communicator) that is atomic (processes do only that until they're
all done).  I think concurrent read and write are probably more akin to database operations, which are far enough
outside our "sweet spot" that I can't see much need for it.

Depending on the use case you have in mind, there could be alternatives.  For example, in many climate datasets, each
timestep is stored in a different file, and we implement a "nomesh" option for reading just the tags from all but the
first file.  We currently restrict this to files that have the same mesh definition, but I'm sure there are workarounds
in cases where the mesh changes.  Also, if the differences were larger, you could just read the new mesh as a separate
mesh, and reconcile the two meshes in the database in some other way, e.g. based on an id or something.

It would help to know more about the use cases here.

- tim

* - this isn't quite true; we do have a .nc file reader that does concurrent read too; otherwise the restrictions are
the same, though.

On 02/15/2012 03:34 PM, Christopher Mueller wrote:
Hi All,

I'm trying to get a little clarification on exactly how concurrency works with MOAB. When one process is reading a mesh
(or more likely, data on a mesh) from a file and another process attempts to write to that file, what happens? Does the
writer get blocked until the reader is through? Can the file be modified by the writer while a reader has an in-memory
representation of the file? Is there a mechanism for the reader to know if the underlying file has changed (is this what
<https://redmine.scorec.rpi.edu/anonsvn/itaps/software/tags/1.3/tools/doxygen/html/group__MeshModification.html#ga392c572f006e88ebe62bffade3be8b9>iMesh_areEHValid
<https://redmine.scorec.rpi.edu/anonsvn/itaps/software/tags/1.3/tools/doxygen/html/group__MeshModification.html> does)?

Does the same answer apply (with roles reversed) if the reader process starts after the writer process?

Are there strategies in place for dealing with these situations?

Thanks in advance,
Chris

--
================================================================
"You will keep in perfect peace him whose mind is
   steadfast, because he trusts in you."               Isaiah 26:3

              Tim Tautges            Argonne National Laboratory
          (tautges at mcs.anl.gov<mailto:tautges at mcs.anl.gov>)      (telecommuting from UW-Madison)
  phone (gvoice): (608) 354-1459      1500 Engineering Dr.
             fax: (608) 263-4499      Madison, WI 53706


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/moab-dev/attachments/20120216/01b1f9a9/attachment.htm>