<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:10pt;color:#000000;font-family:Arial Unicode MS;" dir="ltr">
<p>Wei-keng</p>
<p><br>
</p>
<p>I tried what you suggested. The make check step in general seems to work but does still produce some
<br>
</p>
<p><br>
</p>
<p><span>MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file</span></p>
<p><br>
</p>
<p>although is appears to pass the various tests. The same happens with make ptest which produces some "invalid file" errors. Otherwise it appears to pass the tests including the section called</p>
<p><br>
</p>
<p><span> Parallel testing on 4 MPI processes</span><br>
</p>
<p><br>
</p>
<p>I also ran my test case with the Princeton Ocean Model again which generates output files using pnetcdf and that seems to work ok, although I read them back with regular netcdf.<br>
</p>
<p><br>
</p>
<p>Best regards,</p>
<p>Steve<br>
</p>
<p><br>
</p>
<div id="Signature">
<div id="divtagdefaultwrapper" style="font-size: 10pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: Arial Unicode MS,"EmojiFont","Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;">
<div style="font-family:Tahoma; font-size:13px">Steve Brenner, Professor<br>
Department of Geography and Environment<br>
Bar Ilan University<br>
Ramat Gan, Israel<br>
<br>
Phone: +972 3 5318973<br>
Secretary: +972 3 531 8340<br>
Fax: +972 3 7384033<br>
<br>
E-mail: steve.brenner@biu.ac.il<br>
Web site: geoenv.biu.ac.il/en/Steve_Brenner<br>
<br>
</div>
</div>
</div>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Wei-keng Liao <wkliao@eecs.northwestern.edu><br>
<b>Sent:</b> Sunday, June 11, 2017 08:06<br>
<b>To:</b> Steve Brenner<br>
<b>Cc:</b> parallel-netcdf@mcs.anl.gov<br>
<b>Subject:</b> Re: [pnetcdf-devel] [parallel-netcdf] #31: Problem installing pnetcdf on 64 bit Ubuntu machine with intel compilers</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">Hi, Steve<br>
<br>
I see. I revise the build commands below for your case.<br>
<br>
PnetCDF library built with OpenMPI is regularly tested. So far,<br>
I have not encountered major problems for 1.8.1 release.<br>
<br>
If you would like to give the following commands a try, I will<br>
be interested in knowing whether they resolve your problem.<br>
Then I can safely close your ticket.<br>
<br>
make -s distclean<br>
./configure --with-mpi=/path/to/OpenMPI TEST_MPIRUN="/path/to/OpenMPI/bin/mpiexec -n NP"<br>
make -s<br>
make -s check<br>
make -s ptest<br>
<br>
<br>
When OpenMPI is used, the following error messages can be safely ignored.<br>
<br>
---- During make -s ----<br>
src/lib/ncmpidtype.c: In function 'ncmpii_type_filter':<br>
src/lib/ncmpidtype.c:196: warning: 'ompi_mpi_lb' is deprecated (declared at OpenMPI/include/mpi.h:913)<br>
src/lib/ncmpidtype.c:205: warning: 'ompi_mpi_ub' is deprecated (declared at OpenMPI/include/mpi.h:914)<br>
<br>
---- During make -s check ----<br>
MPI error (MPI_File_delete) : MPI_ERR_IO: input/output error<br>
<br>
<br>
<br>
Wei-keng<br>
<br>
On Jun 9, 2017, at 5:50 PM, Steve Brenner wrote:<br>
<br>
> Wei-keng<br>
> <br>
> I use Openmpi and built it with the intel compilers in order to be consistent with other models that I am running which work successfully with this combination. I do not use the intel supplied mpi libraries since that did not work in the other models I am
using. Perhaps to avoid any problems with the particular application/model I am trying to run which uses pnetcdf I will consider replacing any calls to pnetcdf to regular netcdf which I know is fully functional on my system and which I also use in other applications.<br>
> <br>
> Best regards,<br>
> Steve<br>
> <br>
> Steve Brenner, Professor<br>
> Department of Geography and Environment<br>
> Bar Ilan University<br>
> Ramat Gan, Israel<br>
> <br>
> Phone: +972 3 5318973<br>
> Secretary: +972 3 531 8340<br>
> Fax: +972 3 7384033<br>
> <br>
> E-mail: steve.brenner@biu.ac.il<br>
> Web site: geoenv.biu.ac.il/en/Steve_Brenner<br>
> <br>
> <br>
> <br>
> From: Wei-keng Liao <wkliao@eecs.northwestern.edu><br>
> Sent: Friday, June 9, 2017 03:12<br>
> To: Steve Brenner<br>
> Cc: parallel-netcdf@mcs.anl.gov<br>
> Subject: Re: [pnetcdf-devel] [parallel-netcdf] #31: Problem installing pnetcdf on 64 bit Ubuntu machine with intel compilers<br>
> <br>
> Hi, Steve<br>
> <br>
> > MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file<br>
> This error is expected when using older versions of MPI that fail to return<br>
> a few MPI error codes correctly. It is safe to ignore this error. It<br>
> should disappear when using newer version of MPI libraries.<br>
> <br>
> Other errors may cause by using an "mpiexec" or "mpirun" that is not from<br>
> the same build of mpicc. Please check the mpiexec or mpirun command location<br>
> and see if they are the same as the mpicc used.<br>
> <br>
> Please try the commands below and see if they work.<br>
> <br>
> make -s distclean<br>
> ./configure MPICC=mpiicc MPICXX=mpiicpc MPIF77=mpiifort MPIF90=mpiifort TEST_MPIRUN="mpirun -n NP"<br>
> make -s<br>
> make -s tests<br>
> make -s check<br>
> make -s ptest<br>
> <br>
> The above configure command assumes the Intel MPI compilers are used.<br>
> <br>
> Wei-keng<br>
> <br>
> On Jun 8, 2017, at 1:22 PM, Steve Brenner wrote:<br>
> <br>
> > Hi Wei-keng<br>
> > <br>
> > I tried your patch an the make step seems to run ok. I did notice however that as the output for make check scrolls down the screen there are several lines such as<br>
> > <br>
> > ./nf_test -c -d .<br>
> > ./nf_test -d .<br>
> > MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file<br>
> > *** TESTING F77 ./nf_test for CDF-1 ------ pass<br>
> > ./nf_test -c -2 -d .<br>
> > ./nf_test -2 -d .<br>
> > MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file<br>
> > *** TESTING F77 ./nf_test for CDF-2 ------ pass<br>
> > ./nf_test -c -5 -d .<br>
> > ./nf_test -5 -d .<br>
> > MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file<br>
> > *** TESTING F77 ./nf_test for CDF-5 ------ pass<br>
> > <br>
> > <br>
> > and in make ptest it seems to pass most of the tests but also encounters errors such as<br>
> > <br>
> > Error at test_erange_get_double_int line 4053: expect NC_ERANGE but got NC_NOERR<br>
> > Error at test_erange_get_double_int line 4060: expect rbuf[0]=-2147483647 but got -1<br>
> > Error at test_erange_get_double_int line 4060: expect rbuf[1]=-2147483647 but got -1<br>
> > Error at test_erange_get_double_int line 4060: expect rbuf[2]=-2147483647 but got -1<br>
> > Error at test_erange_get_double_int line 4060: expect rbuf[3]=-2147483647 but got -1<br>
> > Error at test_erange_get_double_int line 4060: expect rbuf[4]=-2147483647 but got -1<br>
> > Error at test_erange_get_double_int line 4060: expect rbuf[5]=-2147483647 but got -1<br>
> > Error at test_erange_get_double_int line 4060: expect rbuf[6]=-2147483647 but got -1<br>
> > Error at test_erange_get_double_int line 4060: expect rbuf[7]=-2147483647 but got -1<br>
> > Error at test_erange_get_double_int line 4060: expect rbuf[8]=-2147483647 but got -1<br>
> > Error at test_erange_get_double_int line 4060: expect rbuf[9]=-2147483647 but got -1<br>
> > MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file<br>
> > Error at test_erange_get_int_uint line 4177: Unknown error in file operation<br>
> > Error at test_erange_get_int_uint line 4178: NetCDF: Not a valid ID<br>
> > Error at test_erange_get_int_uint line 4183: expect NC_ERANGE but got NC_EBADID<br>
> > Error at test_erange_get_int_uint line 4190: expect rbuf[8]=4294967295 but got 2147483649<br>
> > Error at test_erange_get_int_uint line 4190: expect rbuf[9]=4294967295 but got 2147483649<br>
> > Error at test_erange_get_int_uint line 4194: NetCDF: Not a valid ID<br>
> > fail with 538 mismatches<br>
> > make[2]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/testcases'<br>
> > make[2]: Entering directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/CXX'<br>
> > *** TESTING C++ nctst for APIs with different netCDF formats ------ MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file<br>
> > Unknown error in file operation<br>
> > file: ncmpiFile.cpp line:46 error code=-204 Error!<br>
> > unexpected file format<br>
> > read Error <br>
> > file: nctst.cpp line:119 error code=0 Error!<br>
> > fail with 2 mismatches<br>
> > *** TESTING C++ nctst for APIs with different netCDF formats ------ read Error: unlimited dimension frtime
<br>
> > file: nctst.cpp line:156 error code=0 Error!<br>
> > fail with 1 mismatches<br>
> > *** TESTING C++ nctst for APIs with different netCDF formats ------ MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file<br>
> > Unknown error in file operation<br>
> > file: ncmpiFile.cpp line:46 error code=-204 Error!<br>
> > fail with 1 mismatches<br>
> > *** TESTING C++ nctst for APIs with different netCDF formats ------ MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file<br>
> > Unknown error in file operation<br>
> > file: ncmpiFile.cpp line:46 error code=-204 Error!<br>
> > fail with 1 mismatches<br>
> > *** TESTING C++ test_classic for creation of classic format file ------ pass<br>
> > *** TESTING C++ test_classic for creation of classic format file ------ pass<br>
> > *** TESTING C++ test_classic for creation of classic format file ------ pass<br>
> > *** TESTING C++ test_classic for creation of classic format file ------ pass<br>
> > make[2]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/CXX'<br>
> > make[2]: Entering directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/nf_test'<br>
> > make[2]: Nothing to be done for 'ptest4'.<br>
> > make[2]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/nf_test'<br>
> > make[2]: Entering directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/nf90_test'<br>
> > make[2]: Nothing to be done for 'ptest4'.<br>
> > make[2]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/nf90_test'<br>
> > make[2]: Entering directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/F90'<br>
> > *** TESTING F90 ./f90tst_parallel ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel2 for strided access ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel2 for strided access ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel2 for strided access ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel2 for strided access ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel3 ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel3 ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel3 ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel3 ------ pass<br>
> > Error: Unknown error in file operation<br>
> > 2<br>
> > MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file<br>
> > *** TESTING F90 ./f90tst_parallel4 ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel4 ------ pass<br>
> > *** TESTING F90 ./f90tst_parallel4 ------ pass<br>
> > Makefile:111: recipe for target 'ptest4' failed<br>
> > make[2]: *** [ptest4] Error 2<br>
> > make[2]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test/F90'<br>
> > Makefile:73: recipe for target 'ptest' failed<br>
> > make[1]: *** [ptest] Error 2<br>
> > make[1]: Leaving directory '/home/sbrenner/temp/parallel-netcdf-1.8.1/test'<br>
> > Makefile:95: recipe for target 'ptest-test' failed<br>
> > make: *** [ptest-test] Error 2<br>
> > <br>
> > <br>
> > I honestly don't recall if I also saw these last night since it was rather late at night when I was running the installation and I focused on all of the PASS lines that appeared.<br>
> > <br>
> > make install successfully creates the header, library, and utility files and places them where they should be, but do the errors that appeared in the make ptest step mean that pnetcdf might now work correctly when I run a job?<br>
> > <br>
> > I just tried running a test case with the MPI version of the Princeton Ocean Model and it appears to have successfully compiled and executed. In the course of the day I will examine the nc output files to see if they are ok.<br>
> > <br>
> > Best regards,<br>
> > Steve<br>
> > <br>
> > <br>
> > Steve Brenner, Professor<br>
> > Department of Geography and Environment<br>
> > Bar Ilan University<br>
> > Ramat Gan, Israel<br>
> > <br>
> > Phone: +972 3 5318973<br>
> > Secretary: +972 3 531 8340<br>
> > Fax: +972 3 7384033<br>
> > <br>
> > E-mail: steve.brenner@biu.ac.il<br>
> > Web site: geoenv.biu.ac.il/en/Steve_Brenner<br>
> > <br>
> > <br>
> > <br>
> > From: Wei-keng Liao <wkliao@eecs.northwestern.edu><br>
> > Sent: Thursday, June 8, 2017 18:59<br>
> > To: Steve Brenner<br>
> > Cc: sbrenner@mail.biu.ac.il; parallel-netcdf@mcs.anl.gov<br>
> > Subject: Re: [pnetcdf-devel] [parallel-netcdf] #31: Problem installing pnetcdf on 64 bit Ubuntu machine with intel compilers<br>
> > <br>
> > Hi, Steve<br>
> > <br>
> > The error you encountered is actually a bug in Makefile.in.<br>
> > The file pnetcdf_pc should be in DIST_GARBAGE, not GARBAGE.<br>
> > I have committed a fix to the PnetCDF repo. Below is a patch.<br>
> > Thanks for your testing. Please let me know if the patch works<br>
> > for you and if you have further questions.<br>
> > <br>
> > --- Makefile.in (revision 2963)<br>
> > +++ Makefile.in (working copy)<br>
> > @@ -28,13 +28,14 @@<br>
> > benchmarks \<br>
> > examples<br>
> > <br>
> > -GARBAGE = pnetcdf_pc<br>
> > +GARBAGE =<br>
> > <br>
> > DIST_GARBAGE = config.cache \<br>
> > config.status \<br>
> > config.log \<br>
> > macros.make \<br>
> > stamp-h \<br>
> > + pnetcdf_pc \<br>
> > pnetcdf.pc \<br>
> > pnetcdf-config \<br>
> > check_install<br>
> > <br>
> > Wei-keng<br>
> > <br>
> > On Jun 8, 2017, at 4:09 AM, Steve Brenner wrote:<br>
> > <br>
> > > Wei-keng<br>
> > > <br>
> > > I have done a bit more digging and tried building it again but this time instead of using CC=icc CXX=icpc FC=ifort I now used CC=mpicc CXX=mpicxx MPIF90=mpifort which it picks up from /usr/local/bin. It now completes the build and the two tests (make
check and make ptest). When I ran make install it appears to run except for the last few steps where it says<br>
> > > <br>
> > > rm -f pnetcdf.pc<br>
> > > sed -e "s%INSTALL_PREFIX%/usr/local%g" pnetcdf_pc > pnetcdf.pc<br>
> > > sed: can't read pnetcdf_pc: No such file or directory<br>
> > > Makefile:147: recipe for target 'install_PKGCONFIG' failed<br>
> > > make: *** [install_PKGCONFIG] Error 2<br>
> > > <br>
> > > When I look in /usr/local/include and /usr/local/lib the pnetcdf file are there. I will try running a test code tomorrow.<br>
> > > <br>
> > > Best regards<br>
> > > <br>
> > > Steve Brenner, Professor<br>
> > > Department of Geography and Environment<br>
> > > Bar Ilan University<br>
> > > Ramat Gan, Israel<br>
> > > <br>
> > > Phone: +972 3 5318973<br>
> > > Secretary: +972 3 531 8340<br>
> > > Fax: +972 3 7384033<br>
> > > <br>
> > > E-mail: steve.brenner@biu.ac.il<br>
> > > Web site: geoenv.biu.ac.il/en/Steve_Brenner<br>
> > > <br>
> > > <br>
> > > <br>
> > > From: Wei-keng Liao <wkliao@eecs.northwestern.edu><br>
> > > Sent: Thursday, June 8, 2017 10:09<br>
> > > To: sbrenner@mail.biu.ac.il<br>
> > > Cc: parallel-netcdf@mcs.anl.gov<br>
> > > Subject: Re: [pnetcdf-devel] [parallel-netcdf] #31: Problem installing pnetcdf on 64 bit Ubuntu machine with intel compilers<br>
> > > <br>
> > > I do not have access to the exactly same environment, but just now built<br>
> > > PnetCDF 1.8.1 on an x86_64 with Ubuntu 14.04, using mpich 3.2. built with<br>
> > > icc/ifort 16.0.3 and 17.0.2. The build was successful.<br>
> > > <br>
> > > If the build hung at the compile time, then I would guess it most likely<br>
> > > is an issue of the compiler. Can check your MPI fortran compiler to<br>
> > > make sure it compiles other F90 programs without problems.<br>
> > > <br>
> > > Wei-keng<br>
> > > <br>
> > > On Jun 8, 2017, at 12:48 AM, parallel-netcdf wrote:<br>
> > > <br>
> > > > #31: Problem installing pnetcdf on 64 bit Ubuntu machine with intel compilers<br>
> > > > -------------------------------------+--------------------------------------<br>
> > > > Reporter: sbrenner@… | Owner: robl <br>
> > > > Type: defect/bug | Status: new <br>
> > > > Priority: major | Milestone: <br>
> > > > Component: parallel-netcdf | Version: 1.8.1<br>
> > > > Keywords: | <br>
> > > > -------------------------------------+--------------------------------------<br>
> > > > I am trying to install pnetcdf on a i7 machine running Ubuntu linux 16.04<br>
> > > > with the intel 2017 c, c++ and fortran compliers. It configues ok but in<br>
> > > > the make step it hangs when trying to compile pnetcdf.f90 (no error is<br>
> > > > returned, it simple is stuck in the compilation step). I have tried both<br>
> > > > versions 1.8.1 and 1.7.0. Any suggestions?<br>
> > > > <br>
> > > > -- <br>
> > > > Ticket URL: <<a href="http://trac.mcs.anl.gov/projects/parallel-netcdf/ticket/31" id="LPlnk682570" previewremoved="true">http://trac.mcs.anl.gov/projects/parallel-netcdf/ticket/31</a>><br>
> > > > parallel-netcdf <<a href="https://trac.mcs.anl.gov/projects/parallel-netcdf" id="LPlnk259902" previewremoved="true">https://trac.mcs.anl.gov/projects/parallel-netcdf</a>>
<div id="LPBorder_GT_14971658021060.8838988484941974" style="margin-bottom: 20px; overflow: auto; width: 100%; text-indent: 0px;">
<table id="LPContainer_14971658021040.02547991403130989" style="width: 90%; background-color: rgb(255, 255, 255); position: relative; overflow: auto; padding-top: 20px; padding-bottom: 20px; margin-top: 20px; border-top: 1px dotted rgb(200, 200, 200); border-bottom: 1px dotted rgb(200, 200, 200);" role="presentation" cellspacing="0">
<tbody>
<tr style="border-spacing: 0px;" valign="top">
<td id="TextCell_14971658021040.6351126063380932" style="vertical-align: top; position: relative; padding: 0px; display: table-cell;" colspan="2">
<div id="LPRemovePreviewContainer_14971658021040.618891286465562"></div>
<div id="LPTitle_14971658021050.30060235764230225" style="top: 0px; color: rgb(0, 114, 198); font-weight: 400; font-size: 21px; font-family: "wf_segoe-ui_light","Segoe UI Light","Segoe WP Light","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; line-height: 21px;">
<a id="LPUrlAnchor_14971658021050.27622058083454626" style="text-decoration: none;" href="https://trac.mcs.anl.gov/projects/parallel-netcdf" target="_blank">Parallel netCDF: A Parallel I/O Library for NetCDF File Access</a></div>
<div id="LPMetadata_14971658021050.31619939106091455" style="margin: 10px 0px 16px; color: rgb(102, 102, 102); font-weight: 400; font-family: "wf_segoe-ui_normal","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; font-size: 14px; line-height: 14px;">
trac.mcs.anl.gov</div>
<div id="LPDescription_14971658021060.9561786380929193" style="display: block; color: rgb(102, 102, 102); font-weight: 400; font-family: "wf_segoe-ui_normal","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; font-size: 14px; line-height: 20px; max-height: 100px; overflow: hidden;">
Parallel netCDF: A Parallel I/O Library for NetCDF File Access. Parallel netCDF (PnetCDF) is jointly developed by Northwestern University and Argonne National Laboratory.</div>
</td>
</tr>
</tbody>
</table>
</div>
<br>
> Parallel netCDF: A Parallel I/O Library for NetCDF File Access<br>
> trac.mcs.anl.gov<br>
> Parallel netCDF: A Parallel I/O Library for NetCDF File Access. Parallel netCDF (PnetCDF) is jointly developed by Northwestern University and Argonne National Laboratory.<br>
> <br>
> > Parallel netCDF: A Parallel I/O Library for NetCDF File Access<br>
> > trac.mcs.anl.gov<br>
> > Parallel netCDF: A Parallel I/O Library for NetCDF File Access. Parallel netCDF (PnetCDF) is jointly developed by Northwestern University and Argonne National Laboratory.<br>
> > <br>
> > > Parallel netCDF: A Parallel I/O Library for NetCDF File Access<br>
> > > trac.mcs.anl.gov<br>
> > > Parallel netCDF: A Parallel I/O Library for NetCDF File Access. Parallel netCDF (PnetCDF) is jointly developed by Northwestern University and Argonne National Laboratory.<br>
> > > <br>
> > > > <br>
> > > > _______________________________________________<br>
> > > > pnetcdf-devel mailing list<br>
> > > > pnetcdf-devel@lists.mcs.anl.gov<br>
> > > > <a href="https://lists.mcs.anl.gov/mailman/listinfo/pnetcdf-devel" id="LPlnk821175" previewremoved="true">
https://lists.mcs.anl.gov/mailman/listinfo/pnetcdf-devel</a><br>
> > > <br>
> > <br>
> <br>
<br>
</div>
</span></font></div>
</div>
</body>
</html>