pnetcdf bug?

[Wei-keng Liao]

Hi, Bill

I can reproduce what you are seeing.

If the differences happen only to those missing array elements (fill values),
then this is because PnetCDF supports the fill mode only in 1.6.1.
Please note the way fill mode is used differs from netCDF. See the release note
and example codes in
http://trac.mcs.anl.gov/projects/parallel-netcdf/wiki/ReleaseNotes-1.6.1

Please let me know if this is the case.

Wei-keng

On Oct 27, 2015, at 12:41 PM, Bill Sacks wrote:

> I have put the attachment on a public ftp server:
> 
> ftp ftp.cgd.ucar.edu
> 
> user name: anonymous
> password: (your email address)
> 
> cd pub/sacks
> get pnetcdf_bug.tar.gz
> 
> Thanks,
> Bill
> 
>> On Oct 27, 2015, at 11:11 AM, Wei-keng Liao <wkliao at eecs.northwestern.edu> wrote:
>> 
>> Hi, Bill
>> 
>> Bug NCF-234 should not be the cause, as you are using netCDF 4.3.3.1.
>> The fix has been applied to 4.3.0. I will take a look and get back to you.
>> 
>> Somehow your attachment did not come through my mail system.
>> I check PnetCDF mail archive and it does not appear there either.
>> http://lists.mcs.anl.gov/pipermail/parallel-netcdf/2015-October/001746.html
>> 
>> Maybe the file is too big? If that is the case, please send it to me directly.
>> Thanks
>> 
>> Wei-keng
>> 
>> On Oct 27, 2015, at 10:36 AM, Bill Sacks wrote:
>> 
>>> I wonder if this could be related to this (fixed) bug:
>>> 
>>> https://bugtracking.unidata.ucar.edu/browse/NCF-234
>>> 
>>> As with that one, it's possible that the problem is actually in netCDF and not in pnetcdf. Does anyone have an idea for how to determine if this is a pnetcdf problem or a netcdf problem? Or should I go ahead and post this to the netcdf bug list as well?
>>> 
>>> Charlie: I'm feeling more and more that NCO is probably off the hook here: sorry for dragging you into this initially :-)
>>> 
>>> Bill
>>> 
>>> 
>>>> On Oct 27, 2015, at 9:21 AM, Bill Sacks <wsacks at gmail.com> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> I have run into what appears to be a bug in pnetcdf: I have a file written by pnetcdf (via CESM). When I try to append a variable onto it using ncks -A, the new variable gets written properly, but a different variable on the file gets garbage values put into it. If the original file is written with standard netcdf rather than pnetcdf, the problem does not occur.
>>>> 
>>>> I am attaching a tar file that contains files needed to see the problem. It contains two restart files written by CESM (file names beginning check_ncks...): one written with pnetcdf and one with standard netcdf (the latter has "netcdf" in its name). It also contains a third file from which I was trying to copy variables onto this file.
>>>> 
>>>> To reproduce:
>>>> 
>>>> cp check_ncks_problem_noInterp_1027.clm2.r.0001-01-01-01800.nc test.nc
>>>> ncks -A -v COL_Z_p,LEVGRND_CLASS_p finidat_interp_dest.nc test.nc 
>>>> ncdump -v plant_nalloc check_ncks_problem_noInterp_1027.clm2.r.0001-01-01-01800.nc > dump1
>>>> ncdump -v plant_nalloc test.nc > dump2
>>>> diff dump1 dump2 | less
>>>> 
>>>> Notice that many points that were FillValue have been replaced by garbage. 
>>>> 
>>>> If you do the same thing, but using check_ncks_problem_noInterp_netcdf_1027.clm2.r.0001-01-01-01800.nc, then the dumps are identical.
>>>> 
>>>> I originally filed a bug report with NCO <https://sourceforge.net/p/nco/bugs/84/>, but Charlie Zender and Jim Edwards both feel that this is most likely a problem in the writing of the original file, which points to a possible pnetcdf problem.
>>>> 
>>>> CESM was built with
>>>> 
>>>>        module load netcdf-mpi/4.3.3.1
>>>>        module load pnetcdf/1.6.0
>>>> 
>>>> (on NCAR's yellowstone machine).
>>>> 
>>>> Thank you,
>>>> Bill
>>>> 
>>>> --
>>>> Bill Sacks
>>>> CESM Software Engineering Group
>>>> National Center for Atmospheric Research
>>>> (303) 497-1762
>>>> 
>>>> 
>>>> 
>>> 
>> 
>