Parallel-NetCDF included in NetCDF-4 ?

[Wei-keng Liao]

Hi, Louis

You can use either NetCDF or PnetCDF library to write/read files in parallel.
Please note the APIs are different between the two libraries.

If the NetCDF library is built with PnetCDF support, then you can write a
program using NetCDF APIs to perform parallel I/O. In this case, the parallel
I/O is carried out by PnetCDF library underneath.

Similarly, the NetCDF library can be built with HDF5 support and the parallel
I/O is carried out by HDF5 library underneath.

The software release of NetCDF libraries do not include PnetCDF source codes
nor HDF5 source codes. You must obtain and build them separately.

There are two kinds of file formats supported by NetCDF library, the netCDF
classical file format and HDF5 format. When using PnetCDF underneath, the
file format will be classical and using HDF5 underneath, the HDF5 format.

There are some parallel I/O features in PnetCDF that have not appeared in
NetCDF yet, such as request aggregation through nonblocking APIs. The
aggregation feature often result in a good I/O performance.

Let me know if you have further questions.

Wei-keng

On Jun 6, 2017, at 1:49 PM, Monier, Louis wrote:

> Dear Parallel-NetCDF users, 
> 
> I’m Louis Monier, a young undergraduate from France who joined Zhiming Kuang’s research group in Harvard university for 4 months (internship). I’m working on the optimization of a  code (related to atmospheric modelling) with the use of parallel NetCDF to speed up the output process. 
> 
> I’m a beginner in parallel computing. I would have a important question for you. I can’t find a clear answer on the internet: 
> 
> I’ve seen on the Parallel-NetCDF web site (http://trac.mcs.anl.gov/projects/parallel-netcdf) that starting from version 4.4.0, the NetCDF library includes both sequential and parallel access. So here is my question: 
> If I want to speed up my output process using a NetCDF file with a parallel write process, should I use the PNETCDF library or NETCDF library (starting from version 4.4.0) ? 
> In other words, is PNETCDF library included in NETCDF library from version 4.4.0 ? 
> 
> Indeed, it’s possible to find some parallel-NetCDF examples. Some are using NETCDF library, other are using PNETCDF library. I don’t which one is best to use. 
> 
> Thank you to inform me on this subject. 
> 
> Best, 
> 
> Louis Monier