p-netcdf or netcdf4?

[Jim Edwards]

To clarify what I said - pio uses pnetcdf or netcdf4 as backends, it does
not use pnetcdf as interfaced via netcdf4 rather it has it's own pnetcdf
interface - I'm not sure that would make any difference as far as memory or
performance, just want to be clear about how I'm using it.   Because the
memory issue that we've seen with netcdf4 is a show stopper with us we
haven't really looked at the performance of netcdf4/hdf5.

Jim

On Tue, Jun 15, 2010 at 1:43 PM, michael <michael.bane at manchester.ac.uk>wrote:

> On Tue, 2010-06-15 at 13:59 -0500, Rob Latham wrote:
> > On Tue, Jun 15, 2010 at 07:51:21PM +0100, michael wrote:
> > > All sounds good! And I keep aiming to upgrade my p-netcdf installation
> > > sometime.
> > >
> > > However, I must hands up to some confusion: at a recent w/shop I was
> > > informed netcdf-4 has its own parallel implementation and that I should
> > > use that and forget p-netcdf (!). Can you clarify this for me?
> >
> > I get asked this not-infrequently, so I wrote up a short response:
> >
> > http://trac.mcs.anl.gov/projects/parallel-netcdf/wiki/PnetcdfAndNetcdf4
> >
> > The relationship between the Netcdf4 developers and the
> > parallel-netcdf developers is not nearly as contentious as that
> > workshop would have you believe.  Both have their strengths.  I
> > sympathise that more options can be more confusing.
> >
> > ==rob
> >
>
> Rob, glad to see it's far from contentious. Just to clarify that page
> (noting I'm not expect in the diffs from netcdf v3 to v4...):
>
>  if I have netcdf v3 installed I can
>   (a) replace some netcdf (v3) calls with p-netcdf calls, link to
> p-netcdf, and get some parallelism (for an underlying parallel file sys)
>
>  if I have netcdf v4 installed I can
>   (a) amend my code from v3 to v4 then use the HDF5 "layer" to get some
> parallelism
>   (b) amend my code from v3 to v4, ensure I've p-netcdf installed, then
> use the p-netcdf "layer" to get some parallelism
>
> And I think Jim's just said (b) uses 10 times less memory than (a) but I
> didn't pick up if there's much difference in I/O speed?
>
> Many thanks, M
> --
> Dr. Michael K Bane
> Senior Development Officer
> RESEARCH COMPUTING SERVICES
> Univ. of Manchester M13 9PL
> http://www.rcs.manchester.ac.uk
> http://twitter.com/mkbane_mcr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/parallel-netcdf/attachments/20100615/f4994f55/attachment.htm>