To clarify what I said - pio uses pnetcdf or netcdf4 as backends, it does not use pnetcdf as interfaced via netcdf4 rather it has it's own pnetcdf interface - I'm not sure that would make any difference as far as memory or performance, just want to be clear about how I'm using it. Because the memory issue that we've seen with netcdf4 is a show stopper with us we haven't really looked at the performance of netcdf4/hdf5.<br>
<br>Jim <br><br><div class="gmail_quote">On Tue, Jun 15, 2010 at 1:43 PM, michael <span dir="ltr"><<a href="mailto:michael.bane@manchester.ac.uk">michael.bane@manchester.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div></div><div class="h5">On Tue, 2010-06-15 at 13:59 -0500, Rob Latham wrote:<br>
> On Tue, Jun 15, 2010 at 07:51:21PM +0100, michael wrote:<br>
> > All sounds good! And I keep aiming to upgrade my p-netcdf installation<br>
> > sometime.<br>
> ><br>
> > However, I must hands up to some confusion: at a recent w/shop I was<br>
> > informed netcdf-4 has its own parallel implementation and that I should<br>
> > use that and forget p-netcdf (!). Can you clarify this for me?<br>
><br>
> I get asked this not-infrequently, so I wrote up a short response:<br>
><br>
> <a href="http://trac.mcs.anl.gov/projects/parallel-netcdf/wiki/PnetcdfAndNetcdf4" target="_blank">http://trac.mcs.anl.gov/projects/parallel-netcdf/wiki/PnetcdfAndNetcdf4</a><br>
><br>
> The relationship between the Netcdf4 developers and the<br>
> parallel-netcdf developers is not nearly as contentious as that<br>
> workshop would have you believe. Both have their strengths. I<br>
> sympathise that more options can be more confusing.<br>
><br>
> ==rob<br>
><br>
<br>
</div></div>Rob, glad to see it's far from contentious. Just to clarify that page<br>
(noting I'm not expect in the diffs from netcdf v3 to v4...):<br>
<br>
if I have netcdf v3 installed I can<br>
(a) replace some netcdf (v3) calls with p-netcdf calls, link to<br>
p-netcdf, and get some parallelism (for an underlying parallel file sys)<br>
<br>
if I have netcdf v4 installed I can<br>
(a) amend my code from v3 to v4 then use the HDF5 "layer" to get some<br>
parallelism<br>
(b) amend my code from v3 to v4, ensure I've p-netcdf installed, then<br>
use the p-netcdf "layer" to get some parallelism<br>
<br>
And I think Jim's just said (b) uses 10 times less memory than (a) but I<br>
didn't pick up if there's much difference in I/O speed?<br>
<br>
Many thanks, M<br>
<div><div></div><div class="h5">--<br>
Dr. Michael K Bane<br>
Senior Development Officer<br>
RESEARCH COMPUTING SERVICES<br>
Univ. of Manchester M13 9PL<br>
<a href="http://www.rcs.manchester.ac.uk" target="_blank">http://www.rcs.manchester.ac.uk</a><br>
<a href="http://twitter.com/mkbane_mcr" target="_blank">http://twitter.com/mkbane_mcr</a><br>
<br>
</div></div></blockquote></div><br>