[mpich-discuss] ROMIO: Need information on File realms

[Rob Latham]

On Thu, Aug 26, 2010 at 11:04:17AM +0200, Pascal Deveze wrote:
> Hi,
> 
> I recently saw that there are new files in adio/common
> (ad_aggregate_new.c, ad_io_coll.c, ...). The are implementing a "new
> 2 phase method" using "file realms".
> This is very interesting to me, but I do not have any information.
> Are there some paper (architecture, white paper or High Level
> Design) describing this new method ?

The code is in ROMIO but only enabled if you set the "romio_cb_pfr"
hint: (PFR == persistent file realms)

Here are all the hints relevant to file realms.  There are quite a
few:

- romio_cb_pfr: set this to use file realms.  if not set, file domains
  will continue to be calculated in the traditional manner.

- romio_cb_fr_types: the "file realm types" can be  "AAR", "FSZ" or
  "USR".  The paper Wei-keng mentioned explains this better. 

  I am slightly embarrassed to admit that while I added hint parsing
  for the rest of these hints, I never did add the hint parsing for
  this one.  You will be stuck with AAR. 

- romio_cb_fr_alignment: easier hint to explain.  Align file realms
  to the given byte boundary.  Certain file systems perform much
  better when writes are aligned to block boundaries

- romio_cb_ds_threshold: normally, two-phase does data sieving 
  if the write request contains any holes.  Set this hint (a
  datatype's size-to-extent ratio), and datatypes less than this ratio will skip the data sieving optimization and instead service the request piecewise.  

- romio_cb_alltoall: the communication phase of two-phase can involve
  either point-to-point communication, or use MPI_Alltoall if this
  hint is set to 'enable' or 'automatic'

If you experiment with this code, I would love to hear your results.
You have a knack for finding bugs :>

==rob

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA