Big file support on BG with 32-bit off_t type

[John Michalakes]

Working on Blue Gene, we have a code that uses pnetcdf to successfully write a
large file (> 2GB, file type "2"). The pnetcdf library and code are compiled
with 32-bit addressing so that sizeof(off_t) is 4. The file appears to be
correct based on persual with ncdump.

We are unable to read the file back in, however, using pnetcdf with another
program also compiled with 32-bit addressing.  The error return is:  NC_ESMALL.

I believe the section of pnetcdf code returning the error is ncmpii_hdr_get_NC,
in header.c:

  /* check version number in last byte of magic */
  if (magic[sizeof(ncmagic)-1] == 0x1) {
          getbuf.version = 1;
  } else if (magic[sizeof(ncmagic)-1] == 0x2) {
          getbuf.version = 2;
          fSet(ncp->flags, NC_64BIT_OFFSET);
          if (sizeof(off_t) != 8) {
                  /* take the easy way out: if we can't support all CDF-2
                   * files, return immediately */
                  free(getbuf.base);
                  return NC_ESMALL;

This is around line 1142.

Question: is this error condition correct and necessary? How does 32-bit pnetcdf
manage to *write* the large file (apparently) correctly?

Will pnetcdf compiled for OBJECT_MODE 64 work properly on Blue Gene?

Thank you,

John

-----------------------------------------------------------
John Michalakes, Software Engineer        michalak at ucar.edu
NCAR, MMM Division                   voice: +1 303 497 8199
3450 Mitchell Lane                     fax: +1 303 497 8181
Boulder, Colorado 80301 U.S.A.        cell: +1 720 209 2320
        http://www.mmm.ucar.edu/individual/michalakes
-----------------------------------------------------------