<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi Wei-keng,<div class=""><br class=""></div><div class="">Thanks very much for looking into this. I'm happy to submit a bug to the netCDF group if you think that's the best next step.</div><div class=""><br class=""></div><div class="">Superficially, this sure sounds similar to <a href="https://bugtracking.unidata.ucar.edu/browse/NCF-234" class="">https://bugtracking.unidata.ucar.edu/browse/NCF-234</a> – but maybe there are details that make it differ.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Bill</div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Oct 27, 2015, at 1:11 PM, Wei-keng Liao <<a href="mailto:wkliao@eecs.northwestern.edu" class="">wkliao@eecs.northwestern.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">Hi, Bill<br class=""><br class="">I checked the file starting offsets for the two newly added variables.<br class="">It appears that ncks (netCDF underneath) does not respect the offset<br class="">alignment used in the files created by PnetCDF.<br class=""><br class="">Your file created by netCDF has no alignment in between two adjacent variables.<br class="">The other file created by PnetCDF has an alignment of 512 bytes.<br class="">So, when ncks adds 2 new variables, I found the file offsets of the<br class="">two new variables overlap with the last variable of the existing file.<br class="">This indicates a bug in netCDF library, as ncks does not use PnetCDF library.<br class=""><br class="">I will dig into netCDF library to see what happens internally.<br class=""><br class="">Wei-keng<br class=""><br class="">On Oct 27, 2015, at 1:41 PM, Bill Sacks wrote:<br class=""><br class=""><blockquote type="cite" class="">Looking back at my notes, it seems that this problem sometimes appears in differences in actual values – i.e., it doesn't appear to just be a difference in where there are fill values.<br class=""><br class="">Thank you,<br class="">Bill<br class=""><br class=""><blockquote type="cite" class="">On Oct 27, 2015, at 12:30 PM, Wei-keng Liao <<a href="mailto:wkliao@eecs.northwestern.edu" class="">wkliao@eecs.northwestern.edu</a>> wrote:<br class=""><br class="">Hi, Bill<br class=""><br class="">I can reproduce what you are seeing.<br class=""><br class="">If the differences happen only to those missing array elements (fill values),<br class="">then this is because PnetCDF supports the fill mode only in 1.6.1.<br class="">Please note the way fill mode is used differs from netCDF. See the release note<br class="">and example codes in<br class=""><a href="http://trac.mcs.anl.gov/projects/parallel-netcdf/wiki/ReleaseNotes-1.6.1" class="">http://trac.mcs.anl.gov/projects/parallel-netcdf/wiki/ReleaseNotes-1.6.1</a><br class=""><br class="">Please let me know if this is the case.<br class=""><br class="">Wei-keng<br class=""><br class="">On Oct 27, 2015, at 12:41 PM, Bill Sacks wrote:<br class=""><br class=""><blockquote type="cite" class="">I have put the attachment on a public ftp server:<br class=""><br class="">ftp ftp.cgd.ucar.edu<br class=""><br class="">user name: anonymous<br class="">password: (your email address)<br class=""><br class="">cd pub/sacks<br class="">get pnetcdf_bug.tar.gz<br class=""><br class="">Thanks,<br class="">Bill<br class=""><br class=""><blockquote type="cite" class="">On Oct 27, 2015, at 11:11 AM, Wei-keng Liao <wkliao@eecs.northwestern.edu> wrote:<br class=""><br class="">Hi, Bill<br class=""><br class="">Bug NCF-234 should not be the cause, as you are using netCDF 4.3.3.1.<br class="">The fix has been applied to 4.3.0. I will take a look and get back to you.<br class=""><br class="">Somehow your attachment did not come through my mail system.<br class="">I check PnetCDF mail archive and it does not appear there either.<br class="">http://lists.mcs.anl.gov/pipermail/parallel-netcdf/2015-October/001746.html<br class=""><br class="">Maybe the file is too big? If that is the case, please send it to me directly.<br class="">Thanks<br class=""><br class="">Wei-keng<br class=""><br class="">On Oct 27, 2015, at 10:36 AM, Bill Sacks wrote:<br class=""><br class=""><blockquote type="cite" class="">I wonder if this could be related to this (fixed) bug:<br class=""><br class="">https://bugtracking.unidata.ucar.edu/browse/NCF-234<br class=""><br class="">As with that one, it's possible that the problem is actually in netCDF and not in pnetcdf. Does anyone have an idea for how to determine if this is a pnetcdf problem or a netcdf problem? Or should I go ahead and post this to the netcdf bug list as well?<br class=""><br class="">Charlie: I'm feeling more and more that NCO is probably off the hook here: sorry for dragging you into this initially :-)<br class=""><br class="">Bill<br class=""><br class=""><br class=""><blockquote type="cite" class="">On Oct 27, 2015, at 9:21 AM, Bill Sacks <wsacks@gmail.com> wrote:<br class=""><br class="">Hi,<br class=""><br class="">I have run into what appears to be a bug in pnetcdf: I have a file written by pnetcdf (via CESM). When I try to append a variable onto it using ncks -A, the new variable gets written properly, but a different variable on the file gets garbage values put into it. If the original file is written with standard netcdf rather than pnetcdf, the problem does not occur.<br class=""><br class="">I am attaching a tar file that contains files needed to see the problem. It contains two restart files written by CESM (file names beginning check_ncks...): one written with pnetcdf and one with standard netcdf (the latter has "netcdf" in its name). It also contains a third file from which I was trying to copy variables onto this file.<br class=""><br class="">To reproduce:<br class=""><br class="">cp check_ncks_problem_noInterp_1027.clm2.r.0001-01-01-01800.nc test.nc<br class="">ncks -A -v COL_Z_p,LEVGRND_CLASS_p finidat_interp_dest.nc test.nc <br class="">ncdump -v plant_nalloc check_ncks_problem_noInterp_1027.clm2.r.0001-01-01-01800.nc > dump1<br class="">ncdump -v plant_nalloc test.nc > dump2<br class="">diff dump1 dump2 | less<br class=""><br class="">Notice that many points that were FillValue have been replaced by garbage. <br class=""><br class="">If you do the same thing, but using check_ncks_problem_noInterp_netcdf_1027.clm2.r.0001-01-01-01800.nc, then the dumps are identical.<br class=""><br class="">I originally filed a bug report with NCO <https://sourceforge.net/p/nco/bugs/84/>, but Charlie Zender and Jim Edwards both feel that this is most likely a problem in the writing of the original file, which points to a possible pnetcdf problem.<br class=""><br class="">CESM was built with<br class=""><br class=""> module load netcdf-mpi/4.3.3.1<br class=""> module load pnetcdf/1.6.0<br class=""><br class="">(on NCAR's yellowstone machine).<br class=""><br class="">Thank you,<br class="">Bill<br class=""><br class="">--<br class="">Bill Sacks<br class="">CESM Software Engineering Group<br class="">National Center for Atmospheric Research<br class="">(303) 497-1762<br class=""><br class=""><br class=""><br class=""></blockquote><br class=""></blockquote><br class=""></blockquote><br class=""></blockquote><br class=""></blockquote><br class=""></blockquote><br class=""></div></blockquote></div><br class=""></div></body></html>