Problem in compiling example programs

[Wei-keng Liao]

Hi, Lyndon,

What do you do with the dumped data?
If you can describe your I/O requirement in more detailed,
maybe we can find a way not to dump the data into the text form.
In most of the case, users use the binary data directly.

I would say parallel netCDF is very scalable. Note a good parallel
I/O performance also requires parallel file system support.
There are several example programs in the software distribution.
Hope they can be helpful for you.


Wei-keng

On Jul 1, 2013, at 7:53 PM, Lyndon Lu wrote:

> Hi Wei-keng,
> 
>      The file size for some of netCDF files we have is bigger than 2GB. The converting process will take longer.
> 
>       I will rewrite our old mpi programs with pnetcdf. Could you please tell me how much is improved in performance comparing to using netcdf?
> 
>       Thanks.
> 
> Cheers,
> Lyndon.
> 
> 
> On Tue, Jul 2, 2013 at 2:34 AM, Wei-keng Liao <wkliao at ece.northwestern.edu> wrote:
> Hi Lyndon,
> 
> Thanks for pointing this out. We have not spent much time to
> improve this utility.
> 
> Do you mind telling us why you need to run this in parallel?
> Note that ncdump/ncmpidump is to convert netCDF file to text form?
> The standard output will be the I/O bottleneck. Unscalable performance
> should be expected. If you can tell us what you intend to use this
> utility, we might come up with a better solution.
> 
> 
> Wei-keng
> 
> On Jul 1, 2013, at 1:54 AM, Lyndon Lu wrote:
> 
> > Hi Wei-keng,
> >
> >
> >     Thanks! I was expecting that the ncmpidump will take less time than using ncdump because it will run over multiple processors. In fact, it took much longer even though it can generate multiple copies of ouput. So, it may be not useful to us. Probably, it should be designed to work in the way: creating a single copy of output, but will take  less time by getting it run over multiple processors, which I guess that it would be more useful than present way.
> >
> > Cheers,
> > Lyndon.
> >
> >
> > On Mon, Jul 1, 2013 at 2:11 PM, Wei-keng Liao <wkliao at ece.northwestern.edu> wrote:
> > Hi, Lyndon
> >
> > ncmpidump is built as an MPI application.
> > Hence you should run it like all other MPI applications.
> > For example, mpiexec -n 2 ncmpidump file.nc
> > If you run it this way, you will see two copies of the
> > dumps, each printed from one of the 2 MPI processes.
> >
> > I often use it to print the header of a netCDF file in
> > CDF-5 format, running on one process. The Unidata's
> > ncdump currently cannot recognize CDF-5 format.
> >
> > If you have a different expectation on the outputs
> > produced from ncmpidump, please let us know.
> >
> > Wei-keng
> >
> > On Jun 30, 2013, at 9:40 PM, Lyndon Lu wrote:
> >
> > > Hi Guys,
> > >
> > >     I am also wondering what's advantage of using ncmpidump (under sub-dir bin) over ncdump? It seems that there is not an option in the command line to set the number of cpus for ncmpidump.
> > >
> > > Cheers,
> > > Lyndon.
> > >
> > >
> > > On Mon, Jul 1, 2013 at 9:55 AM, Wei-keng Liao <wkliao at ece.northwestern.edu> wrote:
> > > Thanks, Rajeev. The fix has been added to the SVN repo.
> > >
> > > Hi, Lyndon,
> > > Thanks for giving our latest SVN a try. (File flex_f.f90 is
> > > yet to be official released.) Your testing helps us improve
> > > the software for next release. If you find any further
> > > problem or questions, please let us know.
> > >
> > > Wei-keng
> > >
> > > On Jun 30, 2013, at 2:09 AM, Rajeev Thakur wrote:
> > >
> > > > It should be MPI_INTEGER instead of MPI_INT in Fortran.
> > > >
> > > > On Jun 30, 2013, at 1:45 AM, Lyndon Lu wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >>    When running make under sub-dir examples, one of fortran program cannot be compiled successfully ( see below) and other programs seem to be fine.
> > > >>
> > > >> /opt/openmpi/1.4.3/bin/mpif90 -c -g -I../src/lib  -I../src/libf  flex_f.f90
> > > >> flex_f.f90(117): error #6404: This name does not have a type, and must have an explicit type.   [MPI_INT]
> > > >>               MPI_INT, subarray, err)
> > > >> ---------------^
> > > >> flex_f.f90(115): error #6285: There is no matching specific subroutine for this generic subroutine call.   [MPI_TYPE_CREATE_SUBARRAY]
> > > >>          call MPI_Type_create_subarray(2, array_of_sizes, &
> > > >> ---------------^
> > > >> compilation aborted for flex_f.f90 (code 1)
> > > >> make: *** [flex_f.o] Error 1
> > > >>
> > > >>     Does anyone know what's wrong with this program flex_f.90? I am using pnetcdf 1.3.1/netcdf 4.3.0/hdf5 1.8.11.
> > > >>
> > > >>      In addition, could someone tell me what's advantage of using ncmpidump over ncdump? It seems that there is not an option in the command line to set the number of cpus for ncmpidump.
> > > >>
> > > >>
> > > >>      Thanks.
> > > >>
> > > >> Cheers,
> > > >> Lyndon
> > > >
> > >
> > >
> >
> >
> 
>