Problem in compiling example programs

[Lyndon Lu]

Hi Wei-keng,

    "What do you do with the dumped data?"

    Sometimes I need to compare two netcdf files. It seems that there is
not a utility provided by netCDF. I notice that pnetcdf provides a utility
ncmpidiff. Does it do netcdf files comparison? Is it faster?

Cheers,
Lyndon.


On Tue, Jul 2, 2013 at 2:02 PM, Wei-keng Liao
<wkliao at ece.northwestern.edu>wrote:

> Hi, Lyndon,
>
> What do you do with the dumped data?
> If you can describe your I/O requirement in more detailed,
> maybe we can find a way not to dump the data into the text form.
> In most of the case, users use the binary data directly.
>
> I would say parallel netCDF is very scalable. Note a good parallel
> I/O performance also requires parallel file system support.
> There are several example programs in the software distribution.
> Hope they can be helpful for you.
>
>
> Wei-keng
>
> On Jul 1, 2013, at 7:53 PM, Lyndon Lu wrote:
>
> > Hi Wei-keng,
> >
> >      The file size for some of netCDF files we have is bigger than 2GB.
> The converting process will take longer.
> >
> >       I will rewrite our old mpi programs with pnetcdf. Could you please
> tell me how much is improved in performance comparing to using netcdf?
> >
> >       Thanks.
> >
> > Cheers,
> > Lyndon.
> >
> >
> > On Tue, Jul 2, 2013 at 2:34 AM, Wei-keng Liao <
> wkliao at ece.northwestern.edu> wrote:
> > Hi Lyndon,
> >
> > Thanks for pointing this out. We have not spent much time to
> > improve this utility.
> >
> > Do you mind telling us why you need to run this in parallel?
> > Note that ncdump/ncmpidump is to convert netCDF file to text form?
> > The standard output will be the I/O bottleneck. Unscalable performance
> > should be expected. If you can tell us what you intend to use this
> > utility, we might come up with a better solution.
> >
> >
> > Wei-keng
> >
> > On Jul 1, 2013, at 1:54 AM, Lyndon Lu wrote:
> >
> > > Hi Wei-keng,
> > >
> > >
> > >     Thanks! I was expecting that the ncmpidump will take less time
> than using ncdump because it will run over multiple processors. In fact, it
> took much longer even though it can generate multiple copies of ouput. So,
> it may be not useful to us. Probably, it should be designed to work in the
> way: creating a single copy of output, but will take  less time by getting
> it run over multiple processors, which I guess that it would be more useful
> than present way.
> > >
> > > Cheers,
> > > Lyndon.
> > >
> > >
> > > On Mon, Jul 1, 2013 at 2:11 PM, Wei-keng Liao <
> wkliao at ece.northwestern.edu> wrote:
> > > Hi, Lyndon
> > >
> > > ncmpidump is built as an MPI application.
> > > Hence you should run it like all other MPI applications.
> > > For example, mpiexec -n 2 ncmpidump file.nc
> > > If you run it this way, you will see two copies of the
> > > dumps, each printed from one of the 2 MPI processes.
> > >
> > > I often use it to print the header of a netCDF file in
> > > CDF-5 format, running on one process. The Unidata's
> > > ncdump currently cannot recognize CDF-5 format.
> > >
> > > If you have a different expectation on the outputs
> > > produced from ncmpidump, please let us know.
> > >
> > > Wei-keng
> > >
> > > On Jun 30, 2013, at 9:40 PM, Lyndon Lu wrote:
> > >
> > > > Hi Guys,
> > > >
> > > >     I am also wondering what's advantage of using ncmpidump (under
> sub-dir bin) over ncdump? It seems that there is not an option in the
> command line to set the number of cpus for ncmpidump.
> > > >
> > > > Cheers,
> > > > Lyndon.
> > > >
> > > >
> > > > On Mon, Jul 1, 2013 at 9:55 AM, Wei-keng Liao <
> wkliao at ece.northwestern.edu> wrote:
> > > > Thanks, Rajeev. The fix has been added to the SVN repo.
> > > >
> > > > Hi, Lyndon,
> > > > Thanks for giving our latest SVN a try. (File flex_f.f90 is
> > > > yet to be official released.) Your testing helps us improve
> > > > the software for next release. If you find any further
> > > > problem or questions, please let us know.
> > > >
> > > > Wei-keng
> > > >
> > > > On Jun 30, 2013, at 2:09 AM, Rajeev Thakur wrote:
> > > >
> > > > > It should be MPI_INTEGER instead of MPI_INT in Fortran.
> > > > >
> > > > > On Jun 30, 2013, at 1:45 AM, Lyndon Lu wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >>    When running make under sub-dir examples, one of fortran
> program cannot be compiled successfully ( see below) and other programs
> seem to be fine.
> > > > >>
> > > > >> /opt/openmpi/1.4.3/bin/mpif90 -c -g -I../src/lib  -I../src/libf
>  flex_f.f90
> > > > >> flex_f.f90(117): error #6404: This name does not have a type, and
> must have an explicit type.   [MPI_INT]
> > > > >>               MPI_INT, subarray, err)
> > > > >> ---------------^
> > > > >> flex_f.f90(115): error #6285: There is no matching specific
> subroutine for this generic subroutine call.   [MPI_TYPE_CREATE_SUBARRAY]
> > > > >>          call MPI_Type_create_subarray(2, array_of_sizes, &
> > > > >> ---------------^
> > > > >> compilation aborted for flex_f.f90 (code 1)
> > > > >> make: *** [flex_f.o] Error 1
> > > > >>
> > > > >>     Does anyone know what's wrong with this program flex_f.90? I
> am using pnetcdf 1.3.1/netcdf 4.3.0/hdf5 1.8.11.
> > > > >>
> > > > >>      In addition, could someone tell me what's advantage of using
> ncmpidump over ncdump? It seems that there is not an option in the command
> line to set the number of cpus for ncmpidump.
> > > > >>
> > > > >>
> > > > >>      Thanks.
> > > > >>
> > > > >> Cheers,
> > > > >> Lyndon
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/parallel-netcdf/attachments/20130704/d35ef9fb/attachment.html>