Pnetcdf New File Format

[kgao@eecs]

Hi, Katie,
	Thank you for the testing and reporting.
	I fixed the ncmpidump and the second array size problems  problems
and
committed the changes to svn server. The patch file is attached.
	
	Thanks and regards,
Kui
        
	

On Wed, 2008-08-13 at 17:01 -0500, kgao0827 at gmail wrote:
> 
> -----Original Message-----
> From: owner-parallel-netcdf at mcs.anl.gov
> [mailto:owner-parallel-netcdf at mcs.anl.gov] On Behalf Of Wei-keng Liao
> Sent: Wednesday, August 13, 2008 12:58 PM
> To: Katie Antypas
> Cc: parallel-netcdf at mcs.anl.gov
> Subject: Re: Pnetcdf New File Format
> 
> Hi, Katie,
> 
> Thank you for reporting the ncmpidump and the second array size problems. 
> The additional arrays should not have the 2GB limintation. We are working 
> on them right now. We will let you know once the fix is uploaded.
> 
> regards,
> 
> Wei-keng
> 
> On Tue, 12 Aug 2008, Katie Antypas wrote:
> 
> > Thanks for the responses.  I think I was off by an exponent.  I knew 
> > there was the subarray limit, but had it at 2^32 rather than 2^31.
> > 
> > I have a few other things to report too.
> > 
> > First, I believe I found a bug in the ncmpidump tool.  For a 
> > multidimensional variable of size (x,y,z), ncmpidump reports the 
> > variable dimensions as (x,x,y) and which leads the data output to be 
> > incorrect as well.
> > 
> > Secondly, I have successfully created a variable size of > 4GB using a 
> > 3d-multidimensional variable, without using the UNLIMITED dimension, so 
> > this is good news.  I tested creating a variable up to 32 GB without 
> > problems. Next I tried to put additional variables into the file, but it

> > appears that any variables after the first one are still cut off at 4GB.

> > Is this a known issue?
> > 
> > I am working with a group who needs to write out about ~50 16GB 
> > variables for a production run.  There are a number of different work 
> > arounds but right now we have to focus on performance.  The current 
> > implementation puts each variable in a separate file, but we are finding

> > the overhead of opening and closing files to be high which is why I am 
> > trying to test putting all or at least some of the variables in the same

> > file.
> > 
> > Katie
> > 
> > Wei-keng Liao wrote:
> > > Hi, Katie,
> > >
> > > The development of the new pnetcdf 64bit integer format is still in
its
> > > very early stage. Dr. Kui Gao here at Northwestern is working on it.
> > >
> > > The reason of why 1D array with more than 2^31 elements is not work is
> > > because the current pnetcdf's implementation calls
> > > MPI_Type_create_subarray() and its arguments are defined as int (32bit
> > > integers). Rob R., Rob L. and we are all aware of this problem. Kui is
> > > implementing an alternative approach now. Once this is done, we will
> > > update the source asap.
> > >
> > > As for the integer type, my suggestion is to use MPI_Offset, rather
than
> > > int64_t. int64_t is for internal implementation. Using MPI_Offset in
> > > application programs is more portable and consistent.
> > >
> > > Wei-keng
> > >
> > >
> > > On Mon, 11 Aug 2008, Katie Antypas wrote:
> > >
> > >    
> > > > Hi Rob,
> > > >
> > > > I am testing out the new pnetcdf file format which is supposed to
> allow
> > > > variable sizes greater than 4GB (as long as they have fewer than
2^31
> > > > elements).  I am running on the Franklin Cray XT4 system at NERSC
with
> > > > the PGI compilers and have not successfully created a 1d variable
size
> > > > larger than 4GB.  (I'll try multi-dimensions soon.)
> > > >
> > > > I pass "NC_CLOBBER | NC_64BIT_DATA" to ncmpi_create now.  Also, I
was
> > > > getting type warnings and noticed a new type int64_t in places where
> > > > type int and MPI_Offset were used before in a number of calls.  What
> is
> > > > your recommendation to users regarding the int64_t type?  Should
they
> > > > replace MPI_Offset with int64_t?
> > > >
> > > > In either case though, if I do or don't make any changes to the
ncmpi
> > > > calls (namely ncmpi_create, ncmpi_def_var,
ncmpi_put_vara_double_all)
> > > > then I can only create a variable up to 2GB variable size which is
> under
> > > > 2^31 NC_DOUBLE elements in a 1 dimensional array.  (If we were
> counting
> > > > bytes, rather than doubles my test would be at the 2^31 elements
> limit.)
> > > > I didn't think this was the way elements were counted though?
> > > >
> > > > Katie
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >      
> > >
> > >    
> > 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch.txt
Type: text/x-patch
Size: 11730 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/parallel-netcdf/attachments/20080814/8a20faaa/attachment.bin>