[EXTERNAL] Re: metadata consistency

[Wei-keng Liao]

This 8-byte write only happens in collective APIs, and if the write is
to a record variable and the number of records increases. I guess this
is a very common use case. But please note that this write is done by
root process, as the header is accessible by root only. Many modern
parallel file systems (not PVFS) will cache this write. Hence, I would
say performance should not be affected significantly.

The one we probably should worry about is the memory synchronization
for the number of records. It requires an MPI_Allreduce().
This may be significant for large runs. But, I would consider this
is an unavoidable cost.

A question to PnetCDF users. Say, if your program stops in the middle
of PnetCDF I/O and files are not closed, is it acceptable to see the
number of records from the file header smaller than the data written
in the file body? Or would you just simply discard the files?
The answer will determine what default settings should be used.

Wei-keng

On Jul 18, 2013, at 12:33 PM, Rob Latham wrote:

> On Thu, Jul 18, 2013 at 11:47:57AM -0500, Wei-keng Liao wrote:
>> There is an issue for flushing the number of records to file I would like
>> discuss here. In r1364, since the number of records is part of file header,
>> it is flushed to file only when NC_SHARE is used. Otherwise, it will be done
>> at file close time. However, because flushing it only requires to write a
>> 4-byte (CDF and CDF2) or 8-byte integer (CDF5), flushing it should cost
>> much less than other header changes. Other cases will require to flush the
>> entire header. I can change the flushing to be performed no matter
>> NC_SHARE is set or not. Please let me know your preference on when the
>> number of records should be flushed, as this can affect what you will
>> see from the file header if the program stops/exists before file close.
> 
> As our HDF5 friends can attest, metadata consistency is a big deal. 
> 
> I'm nervous, though, about introducing an additional 8 byte write to
> every record variable I/O operation.   I guess we should try it and
> see what the costs are.  
> 
> ==rob
> 
> -- 
> Rob Latham
> Mathematics and Computer Science Division
> Argonne National Lab, IL USA