Writing large files from pnetcdf on IBM

[John Michalakes]

Hi Robert,

> -----Original Message-----
> From: Robert Latham [mailto:robl at mcs.anl.gov]
> Sent: Tuesday, November 28, 2006 9:13 AM
> To: John Michalakes
> Cc: parallel-netcdf at mcs.anl.gov
> Subject: Re: Writing large files from pnetcdf on IBM
>
>
[...]
>
> What happens if you use the 'ncmpidump' tool that comes with
> parallel-netcdf?  (I'm curious if the problem is with the on-disk file
> or with the dumping tool)

/contrib/pnetcdf/bin/ncmpidump: wrfrst_d02_2005-04-02_18_00_00: Not a netCDF
file

>
> One thing to check is the type of netcdf file you are creating.  You
> are already passing in the appropriate flag.  I'd like to take a peek
> at the first few bytes of the datafile header.  One of these
> approaches should do the trick:
>
> $ xxd my_netcdf_file.nc | head
> $ od -x my_netcdf_file.nc | head
> $ hexdump my_netcdf_file.nc | head
>

bluevista:/ptmp/michalak/run_hemi 157 > od -x wrfrst_d02_2005-04-02_18_00_00 |
head
0000000  4801 6000 0000 0001 0000 0000 0000 0000
0000020  0000 0000 0000 0000 0000 0000 0000 0000
*
14414620  4036 b693 4036 6a47 4036 1e26 4035 d230
14414640  4035 8665 4035 3ac5 4034 ef51 4034 a407
14414660  4034 58e9 4034 0df5 4033 c32d 4033 7890
14414700  4033 2e1e 4032 e3d6 4032 99b9 4032 4fc8
14414720  4032 0603 4031 bc68 4031 72f8 4031 29b3
14414740  4030 e099 4030 97ab 4030 4ee7 4030 064f
14414760  402f bde1 402f 759f 402f 2d88 402e e59c

(For comparison, with a file known to work:

bluevista:/ptmp/michalak/run_hemi 158 > od -x wrfrst_d01_2005-04-03_00_00_00 |
head
0000000  4344 4601 0000 0001 0000 000a 0000 0009
0000020  0000 0004 5469 6d65 0000 0000 0000 000a
0000040  4461 7465 5374 724c 656e 0000 0000 0013
0000060  0000 000e 7765 7374 5f65 6173 745f 7374
0000100  6167 0000 0000 0138 0000 000b 736f 7574
0000120  685f 6e6f 7274 6800 0000 0137 0000 000a
0000140  626f 7474 6f6d 5f74 6f70 0000 0000 0064
0000160  0000 0009 7765 7374 5f65 6173 7400 0000
0000200  0000 0137 0000 0010 736f 7574 685f 6e6f
0000220  7274 685f 7374 6167 0000 0138 0000 000f)


> Let us know what that gives you.
>
> If you have access to the 'nccconfig.h' header, you should see
> '#define SIZEOF_OFF_T 8' in there as well as
> '#define SIZEOF_MPI_OFFSET 8'

/* The number of bytes in a off_t */
#define SIZEOF_OFF_T 8

/* The number of bytes in an MPI_Offset */
#define SIZEOF_MPI_OFFSET 8


>
> Thanks for the report.  We should be able to get this figured out
> shortly.

Thanks for your help.

John

-----------------------------------------------------------
John Michalakes, Software Engineer        michalak at ucar.edu
NCAR, MMM Division                   voice: +1 303 497 8199
3450 Mitchell Lane                     fax: +1 303 497 8181
Boulder, Colorado 80301 U.S.A.        cell: +1 720 209 2320
        http://www.mmm.ucar.edu/individual/michalakes
-----------------------------------------------------------