[EXTERNAL] Re: metadata consistency

[Wei-keng Liao]

In r1364, I add a note that describes the metadata synchronization semantics.
In summary, checking consistency for file header cached in memory will
perform only when the environment variable PNETCDF_SAFE_MODE is set to 1.
FLUSHING the dirty header to file will only perform when the file is opened
in NC_SHARE mode. Please send me your comments if this works for you.

There is an issue for flushing the number of records to file I would like
discuss here. In r1364, since the number of records is part of file header,
it is flushed to file only when NC_SHARE is used. Otherwise, it will be done
at file close time. However, because flushing it only requires to write a
4-byte (CDF and CDF2) or 8-byte integer (CDF5), flushing it should cost
much less than other header changes. Other cases will require to flush the
entire header. I can change the flushing to be performed no matter
NC_SHARE is set or not. Please let me know your preference on when the
number of records should be flushed, as this can affect what you will
see from the file header if the program stops/exists before file close.

Wei-keng

On Jul 11, 2013, at 4:32 PM, Sjaardema, Gregory D wrote:

> I use the put_att_int and put_att_double in data mode to change the value
> of an existing attribute.  The values that are changed are a timestamp
> indicating the "last written time" and a value listing the maximum name
> length of any names on my databaseŠ  I'm good with making the call
> collective, but requiring it to be made in define mode seems like a major
> change.
> 
> --Greg
> 
> On 7/11/13 1:18 PM, "Wei-keng Liao" <wkliao at ece.northwestern.edu> wrote:
> 
>> In netCDF, the put_att_<type> is allowed in data mode only when
>> it is used to change an existing attribute. I consider
>> this case rare, and thus am asking the community if this is
>> a common practice.
>> 
>> The solution of "rank 0 wins" does not solve the read case.
>> For example, one process change a time stamp (as a global attribute)
>> in data mode. This change will not be made aware to other processes.
>> So, when other processes call get_att(), they will get the old value.
>> Will this be OK?
>> 
>> One milder solution is to make the APIs collective, if they are
>> called in data mode.
>> 
>> Wei-keng
>> 
>> On Jul 11, 2013, at 1:21 PM, Rob Latham wrote:
>> 
>>> On Thu, Jul 11, 2013 at 12:09:54PM -0500, Wei-keng Liao wrote:
>>>> One minor correction. The last API in the list I intent to say
>>>> is the family of ncmpi_put_att_<type> APIs. The <type> can be one
>>>> of text, uchar, schar, short, int, ...
>>>> 
>>> 
>>> I'm a little wary of changing our semantics in such a mature piece of
>>> software.  I think you are right that most people are already doing
>>> this, but it makes me a bit nervous.
>>> 
>>> the put_att_<type> change has me the most nervous.
>>> 
>>> It's not as nice as your proposal, but could we just say "rank 0 wins
>>> if there is ever inconsistent metadata" ?
>>> 
>>> ==rob
>>> 
>>>> Wei-keng
>>>> 
>>>> On Jul 11, 2013, at 12:03 PM, Wei-keng Liao wrote:
>>>> 
>>>>> Dear PnetCDF users,
>>>>> 
>>>>> I am working on strengthening the PnetCDF's metadata consistency and
>>>>> would like to change/limit the usage of APIs that modified the
>>>>> metadata (file header) of a netCDF file. These APIs are:
>>>>>  ncmpi_rename_dim(),
>>>>>  ncmpi_rename_var(),
>>>>>  ncmpi_copy_att(),
>>>>>  ncmpi_rename_att(), and
>>>>>  ncmpii_put_att().              <------- correction !
>>>>> 
>>>>> (The consistency here is referring to the consistency of file header
>>>>> data stored in memory across all MPI processes.)
>>>>> 
>>>>> In netCDF, the above APIs are allowed in data mode if the space
>>>>> required to store the new metadata (attributes, names, etc.) is
>>>>> less than the old one. Otherwise, they must be called in the define
>>>>> mode.
>>>>> 
>>>>> In PnetCDF, I would like to change that to allow these APIs only
>>>>> in define mode. If your applications require the above APIs to
>>>>> be called in data mode, please do let me know.
>>>>> 
>>>>> Here is my reason for the above change. In data mode, if metadata
>>>>> is changed on one process's memory (or even the change is written
>>>>> to the file by that process because NC_SHARE is set), there is no
>>>>> way to propagate the change from this process to other processes,
>>>>> until ncmpi_close() or ncmpi_sync() is called. If allowing these
>>>>> APIs in define mode only, we can rely on ncmpi_endef() to
>>>>> ensure/check the consistency.
>>>>> 
>>>>> Please let me know if your applications will have a problem with
>>>>> such change.
>>>>> 
>>>>> (My plan is to make NC_SHARE the default mode for PnetCDF as
>>>>> PnetCDF IS developed to handle parallel access to shared files.
>>>>> The above suggested API changes is the first step of my plan.)
>>>>> 
>>>>> 
>>>>> Wei-keng
>>>>> 
>>>> 
>>> 
>>> -- 
>>> Rob Latham
>>> Mathematics and Computer Science Division
>>> Argonne National Lab, IL USA
>> 
> 
>