writing large variables

[John Clyne]

Thanks for the quick response, Rob. I've poked the Unidata folks as well to see if they have any updates on their CDF-5 support plans. One followup question: Is it possible to output large variables from pnetcdf without using CDF-5? netCDF seems to support this in a CDF-2 format, albeit with restrictions. For our application we can live with those restrictions.'

Thanks again for your help.

Best,

jc

On Jan 16, 2013, at 7:54 AM, Rob Latham wrote:

> On Tue, Jan 15, 2013 at 05:09:05PM -0700, John Clyne wrote:
>> Is it possible to write a large variable (>4GB) to a file with pnetcdf and read back the variable from the resulting file with netCDF? Outputting a large variable with pnetcdf appears to require passing the NC_64BIT_DATA flag (not NC_64BIT_OFFSET) to nc_create_par() - without this flag an error is generated. The file is written successfully, but when NC_64BIT_DATA is used the file is unrecognized by netcdf. For example:
>> 
>> yslogin2[43] ncdump -h vx.0000.nc0
>> ncdump: vx.0000.nc0: NetCDF: Unknown file format
>> 
>> From what I can gather from the web the NC_64BIT_DATA results in the generation of a CDF-5 formatted file. Is there support for CDF-5 in netCDF, or any other options for mixing pnetcdf and netCDF?
> 
> Hi John:  the short answer is there is no "unidata netCDF" support for
> pnetcdf's CDF-5 (giant variables) file format.  
> 
> I've been working with Unidata  on and off over the last few years to
> find a way that we could use NetCDF-4's "netcdf on pnetcdf" feature to
> support CDF-5, but that support right now only exists as a series of
> patches yet to be incorporated into Unidata's tree.
> 
> ==rob
> 
> -- 
> Rob Latham
> Mathematics and Computer Science Division
> Argonne National Lab, IL USA

John Clyne
National Center for Atmospheric Research
303.497.1236 (w), 303.809.1922 (c)
clyne at ucar.edu