[parallel-netcdf] #42: MPI-agnostic "ncmpidiff" tool?

[Wei-keng Liao]

Hi, Carl

cdfdiff and ncmpidiff were designed to be more like unix command “diff”.
It reports whether the 2 NetCDF files are different and the “locations”
of differences.

NCO’s ncbo does calculate the element-wise differences and
other statistics values, and save them to a new file.

I believe cprnc is similar, but reports the differences to stdout.

One advantage of cdfdiff is it does not need PnetCDF or NetCDF
library to compile or run.

FYI. I updated cdfdiff to report the array indices of first
array element that is found different. Here is an example output.

  cdfdiff tst_rec.nc1 tst_rec.nc2 ----
  DIFF: file format (CDF-1) != (CDF-2)
  DIFF: variable "rec_var" of type "NC_FLOAT" at element [1, 1]
  DIFF: variable "fix_var" of type "NC_FLOAT" at element [1, 2]
  Number of differences in header: 1
  Number of differences in variables: 2

Command line option -b is for verbose mode, which also reports
what objects are the same.

Wei-keng

> On Jun 12, 2019, at 12:09 PM, Carl Ponder <cponder at nvidia.com> wrote:
> 
> On 6/11/19 1:05 PM, Wei-keng Liao wrote:
>> I have implemented a serial version of ncmpidiff, named "cdfdiff”. It is available in PnetCDF repo and will be added to the next release.
>> https://github.com/Parallel-NetCDF/PnetCDF/tree/master/src/utils/ncmpidiff
>> Command “make” or “make cdfdiff” will create it. Note it only compares the classic NetCDF files.
> Wei-Keng -- do you have some explanation of the tool output?
> I have this, for example:
> DIFF: global attribute "history" length (31) != (30)
> DIFF: global attribute "config_run_duration" of type "NC_CHAR" (3)
> DIFF: global attribute "file_id" of type "NC_CHAR" (3)
> DIFF: variable "ru" of type "NC_FLOAT" (158)
> DIFF: variable "ru_save" of type "NC_FLOAT" (-185)
> DIFF: variable "rw" of type "NC_FLOAT" (-91)
> DIFF: variable "rw_save" of type "NC_FLOAT" (101)
> DIFF: variable "rtheta_p" of type "NC_FLOAT" (77)
> DIFF: variable "rtheta_p_save" of type "NC_FLOAT" (-47)
> DIFF: variable "rho_p" of type "NC_FLOAT" (42)
> DIFF: variable "rho_p_save" of type "NC_FLOAT" (-35)
> DIFF: variable "rho_zz_old_split" of type "NC_FLOAT" (83)
> DIFF: variable "ruAvg" of type "NC_FLOAT" (-151)
> DIFF: variable "ruAvg_split" of type "NC_FLOAT" (15)
> DIFF: variable "wwAvg" of type "NC_FLOAT" (-93)
> DIFF: variable "wwAvg_split" of type "NC_FLOAT" (-70)
> Number of differences in header 3
> Number of differences in variables 13
> Most of these are arrays, is this reporting the largest absolute difference across all the elements?
> 
>         -- Carl
> 
> 
> This email message is for the sole use of the intended recipient(s) and may contain confidential information.  Any unauthorized review, use, disclosure or distribution is prohibited.  If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.