[pnetcdf-devel] [parallel-netcdf] #31: Problem installing pnetcdf on 64 bit Ubuntu machine with intel compilers
Wei-keng Liao
wkliao at eecs.northwestern.edu
Thu Jun 8 19:12:29 CDT 2017
Hi, Steve
> MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
This error is expected when using older versions of MPI that fail to return
a few MPI error codes correctly. It is safe to ignore this error. It
should disappear when using newer version of MPI libraries.
Other errors may cause by using an "mpiexec" or "mpirun" that is not from
the same build of mpicc. Please check the mpiexec or mpirun command location
and see if they are the same as the mpicc used.
Please try the commands below and see if they work.
make -s distclean
./configure MPICC=mpiicc MPICXX=mpiicpc MPIF77=mpiifort MPIF90=mpiifort TEST_MPIRUN="mpirun -n NP"
make -s
make -s tests
make -s check
make -s ptest
The above configure command assumes the Intel MPI compilers are used.
Wei-keng
On Jun 8, 2017, at 1:22 PM, Steve Brenner wrote:
> Hi Wei-keng
>
> I tried your patch an the make step seems to run ok. I did notice however that as the output for make check scrolls down the screen there are several lines such as
>
> ./nf_test -c -d .
> ./nf_test -d .
> MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
> *** TESTING F77 ./nf_test for CDF-1 ------ pass
> ./nf_test -c -2 -d .
> ./nf_test -2 -d .
> MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
> *** TESTING F77 ./nf_test for CDF-2 ------ pass
> ./nf_test -c -5 -d .
> ./nf_test -5 -d .
> MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
> *** TESTING F77 ./nf_test for CDF-5 ------ pass
>
>
> and in make ptest it seems to pass most of the tests but also encounters errors such as
>
> Error at test_erange_get_double_int line 4053: expect NC_ERANGE but got NC_NOERR
> Error at test_erange_get_double_int line 4060: expect rbuf[0]=-2147483647 but got -1
> Error at test_erange_get_double_int line 4060: expect rbuf[1]=-2147483647 but got -1
> Error at test_erange_get_double_int line 4060: expect rbuf[2]=-2147483647 but got -1
> Error at test_erange_get_double_int line 4060: expect rbuf[3]=-2147483647 but got -1
> Error at test_erange_get_double_int line 4060: expect rbuf[4]=-2147483647 but got -1
> Error at test_erange_get_double_int line 4060: expect rbuf[5]=-2147483647 but got -1
> Error at test_erange_get_double_int line 4060: expect rbuf[6]=-2147483647 but got -1
> Error at test_erange_get_double_int line 4060: expect rbuf[7]=-2147483647 but got -1
> Error at test_erange_get_double_int line 4060: expect rbuf[8]=-2147483647 but got -1
> Error at test_erange_get_double_int line 4060: expect rbuf[9]=-2147483647 but got -1
> MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
> Error at test_erange_get_int_uint line 4177: Unknown error in file operation
> Error at test_erange_get_int_uint line 4178: NetCDF: Not a valid ID
> Error at test_erange_get_int_uint line 4183: expect NC_ERANGE but got NC_EBADID
> Error at test_erange_get_int_uint line 4190: expect rbuf[8]=4294967295 but got 2147483649
> Error at test_erange_get_int_uint line 4190: expect rbuf[9]=4294967295 but got 2147483649
> Error at test_erange_get_int_uint line 4194: NetCDF: Not a valid ID
> fail with 538 mismatches
> make[2]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/testcases'
> make[2]: Entering directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/CXX'
> *** TESTING C++ nctst for APIs with different netCDF formats ------ MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
> Unknown error in file operation
> file: ncmpiFile.cpp line:46 error code=-204 Error!
> unexpected file format
> read Error
> file: nctst.cpp line:119 error code=0 Error!
> fail with 2 mismatches
> *** TESTING C++ nctst for APIs with different netCDF formats ------ read Error: unlimited dimension frtime
> file: nctst.cpp line:156 error code=0 Error!
> fail with 1 mismatches
> *** TESTING C++ nctst for APIs with different netCDF formats ------ MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
> Unknown error in file operation
> file: ncmpiFile.cpp line:46 error code=-204 Error!
> fail with 1 mismatches
> *** TESTING C++ nctst for APIs with different netCDF formats ------ MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
> Unknown error in file operation
> file: ncmpiFile.cpp line:46 error code=-204 Error!
> fail with 1 mismatches
> *** TESTING C++ test_classic for creation of classic format file ------ pass
> *** TESTING C++ test_classic for creation of classic format file ------ pass
> *** TESTING C++ test_classic for creation of classic format file ------ pass
> *** TESTING C++ test_classic for creation of classic format file ------ pass
> make[2]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/CXX'
> make[2]: Entering directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/nf_test'
> make[2]: Nothing to be done for 'ptest4'.
> make[2]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/nf_test'
> make[2]: Entering directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/nf90_test'
> make[2]: Nothing to be done for 'ptest4'.
> make[2]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/nf90_test'
> make[2]: Entering directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/F90'
> *** TESTING F90 ./f90tst_parallel ------ pass
> *** TESTING F90 ./f90tst_parallel ------ pass
> *** TESTING F90 ./f90tst_parallel ------ pass
> *** TESTING F90 ./f90tst_parallel ------ pass
> *** TESTING F90 ./f90tst_parallel2 for strided access ------ pass
> *** TESTING F90 ./f90tst_parallel2 for strided access ------ pass
> *** TESTING F90 ./f90tst_parallel2 for strided access ------ pass
> *** TESTING F90 ./f90tst_parallel2 for strided access ------ pass
> *** TESTING F90 ./f90tst_parallel3 ------ pass
> *** TESTING F90 ./f90tst_parallel3 ------ pass
> *** TESTING F90 ./f90tst_parallel3 ------ pass
> *** TESTING F90 ./f90tst_parallel3 ------ pass
> Error: Unknown error in file operation
> 2
> MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file
> *** TESTING F90 ./f90tst_parallel4 ------ pass
> *** TESTING F90 ./f90tst_parallel4 ------ pass
> *** TESTING F90 ./f90tst_parallel4 ------ pass
> Makefile:111: recipe for target 'ptest4' failed
> make[2]: *** [ptest4] Error 2
> make[2]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/F90'
> Makefile:73: recipe for target 'ptest' failed
> make[1]: *** [ptest] Error 2
> make[1]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test'
> Makefile:95: recipe for target 'ptest-test' failed
> make: *** [ptest-test] Error 2
>
>
> I honestly don't recall if I also saw these last night since it was rather late at night when I was running the installation and I focused on all of the PASS lines that appeared.
>
> make install successfully creates the header, library, and utility files and places them where they should be, but do the errors that appeared in the make ptest step mean that pnetcdf might now work correctly when I run a job?
>
> I just tried running a test case with the MPI version of the Princeton Ocean Model and it appears to have successfully compiled and executed. In the course of the day I will examine the nc output files to see if they are ok.
>
> Best regards,
> Steve
>
>
> Steve Brenner, Professor
> Department of Geography and Environment
> Bar Ilan University
> Ramat Gan, Israel
>
> Phone: +972 3 5318973
> Secretary: +972 3 531 8340
> Fax: +972 3 7384033
>
> E-mail: steve.brenner at biu.ac.il
> Web site: geoenv.biu.ac.il/en/Steve_Brenner
>
>
>
> From: Wei-keng Liao <wkliao at eecs.northwestern.edu>
> Sent: Thursday, June 8, 2017 18:59
> To: Steve Brenner
> Cc: sbrenner at mail.biu.ac.il; parallel-netcdf at mcs.anl.gov
> Subject: Re: [pnetcdf-devel] [parallel-netcdf] #31: Problem installing pnetcdf on 64 bit Ubuntu machine with intel compilers
>
> Hi, Steve
>
> The error you encountered is actually a bug in Makefile.in.
> The file pnetcdf_pc should be in DIST_GARBAGE, not GARBAGE.
> I have committed a fix to the PnetCDF repo. Below is a patch.
> Thanks for your testing. Please let me know if the patch works
> for you and if you have further questions.
>
> --- Makefile.in (revision 2963)
> +++ Makefile.in (working copy)
> @@ -28,13 +28,14 @@
> benchmarks \
> examples
>
> -GARBAGE = pnetcdf_pc
> +GARBAGE =
>
> DIST_GARBAGE = config.cache \
> config.status \
> config.log \
> macros.make \
> stamp-h \
> + pnetcdf_pc \
> pnetcdf.pc \
> pnetcdf-config \
> check_install
>
> Wei-keng
>
> On Jun 8, 2017, at 4:09 AM, Steve Brenner wrote:
>
> > Wei-keng
> >
> > I have done a bit more digging and tried building it again but this time instead of using CC=icc CXX=icpc FC=ifort I now used CC=mpicc CXX=mpicxx MPIF90=mpifort which it picks up from /usr/local/bin. It now completes the build and the two tests (make check and make ptest). When I ran make install it appears to run except for the last few steps where it says
> >
> > rm -f pnetcdf.pc
> > sed -e "s%INSTALL_PREFIX%/usr/local%g" pnetcdf_pc > pnetcdf.pc
> > sed: can't read pnetcdf_pc: No such file or directory
> > Makefile:147: recipe for target 'install_PKGCONFIG' failed
> > make: *** [install_PKGCONFIG] Error 2
> >
> > When I look in /usr/local/include and /usr/local/lib the pnetcdf file are there. I will try running a test code tomorrow.
> >
> > Best regards
> >
> > Steve Brenner, Professor
> > Department of Geography and Environment
> > Bar Ilan University
> > Ramat Gan, Israel
> >
> > Phone: +972 3 5318973
> > Secretary: +972 3 531 8340
> > Fax: +972 3 7384033
> >
> > E-mail: steve.brenner at biu.ac.il
> > Web site: geoenv.biu.ac.il/en/Steve_Brenner
> >
> >
> >
> > From: Wei-keng Liao <wkliao at eecs.northwestern.edu>
> > Sent: Thursday, June 8, 2017 10:09
> > To: sbrenner at mail.biu.ac.il
> > Cc: parallel-netcdf at mcs.anl.gov
> > Subject: Re: [pnetcdf-devel] [parallel-netcdf] #31: Problem installing pnetcdf on 64 bit Ubuntu machine with intel compilers
> >
> > I do not have access to the exactly same environment, but just now built
> > PnetCDF 1.8.1 on an x86_64 with Ubuntu 14.04, using mpich 3.2. built with
> > icc/ifort 16.0.3 and 17.0.2. The build was successful.
> >
> > If the build hung at the compile time, then I would guess it most likely
> > is an issue of the compiler. Can check your MPI fortran compiler to
> > make sure it compiles other F90 programs without problems.
> >
> > Wei-keng
> >
> > On Jun 8, 2017, at 12:48 AM, parallel-netcdf wrote:
> >
> > > #31: Problem installing pnetcdf on 64 bit Ubuntu machine with intel compilers
> > > -------------------------------------+--------------------------------------
> > > Reporter: sbrenner@… | Owner: robl
> > > Type: defect/bug | Status: new
> > > Priority: major | Milestone:
> > > Component: parallel-netcdf | Version: 1.8.1
> > > Keywords: |
> > > -------------------------------------+--------------------------------------
> > > I am trying to install pnetcdf on a i7 machine running Ubuntu linux 16.04
> > > with the intel 2017 c, c++ and fortran compliers. It configues ok but in
> > > the make step it hangs when trying to compile pnetcdf.f90 (no error is
> > > returned, it simple is stuck in the compilation step). I have tried both
> > > versions 1.8.1 and 1.7.0. Any suggestions?
> > >
> > > --
> > > Ticket URL: <http://trac.mcs.anl.gov/projects/parallel-netcdf/ticket/31>
> > > parallel-netcdf <https://trac.mcs.anl.gov/projects/parallel-netcdf>
> Parallel netCDF: A Parallel I/O Library for NetCDF File Access
> trac.mcs.anl.gov
> Parallel netCDF: A Parallel I/O Library for NetCDF File Access. Parallel netCDF (PnetCDF) is jointly developed by Northwestern University and Argonne National Laboratory.
>
> > Parallel netCDF: A Parallel I/O Library for NetCDF File Access
> > trac.mcs.anl.gov
> > Parallel netCDF: A Parallel I/O Library for NetCDF File Access. Parallel netCDF (PnetCDF) is jointly developed by Northwestern University and Argonne National Laboratory.
> >
> > >
> > > _______________________________________________
> > > pnetcdf-devel mailing list
> > > pnetcdf-devel at lists.mcs.anl.gov
> > > https://lists.mcs.anl.gov/mailman/listinfo/pnetcdf-devel
> >
>
More information about the parallel-netcdf
mailing list