[petsc-users] Undefined reference in PETSc 3.13+ with old MPI version

Sun Apr 11 13:45:07 CDT 2021

Hi Junchao,

I also ported the changes you have made to PETSc 3.13.6 and configured with Intel 14.0 and OpenMPI 1.6.5, it works too. 

There is a similar problem in PETSc 3.14+ version as MPI_Iallreduce is only available in OpenMPI V1.7+. I would not say this is a bug, it just requires a newer MPI version. 

/home/danyangs/soft/petsc/petsc-3.14.6/intel-14.0.2-openmpi-1.6.5/lib/libpetsc.so: undefined reference to `MPI_Iallreduce' 

Thanks again for all your help,

Danyang

From: Junchao Zhang <junchao.zhang at gmail.com>
Date: Sunday, April 11, 2021 at 7:54 AM
To: Danyang Su <danyang.su at gmail.com>
Cc: Barry Smith <bsmith at petsc.dev>, "petsc-users at mcs.anl.gov" <petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] Undefined reference in PETSc 3.13+ with old MPI version

Thanks, Glad to know you have a workaround.
--Junchao Zhang

On Sat, Apr 10, 2021 at 10:06 PM Danyang Su <danyang.su at gmail.com> wrote:

Hi Junchao,

I cannot configure your branch with same options due to the error in sowing. I had similar error before on other clusters with very old openmpi version. Problem was solved when openmpi was updated to a newer one. 

At this moment, I configured a PETSc version with Openmpi 2.1.6 version and it seems working properly. 

Thanks and have a good rest of the weekend,

Danyang

From: Danyang Su <danyang.su at gmail.com>
Date: Saturday, April 10, 2021 at 4:08 PM
To: Junchao Zhang <junchao.zhang at gmail.com>
Cc: Barry Smith <bsmith at petsc.dev>, "petsc-users at mcs.anl.gov" <petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] Undefined reference in PETSc 3.13+ with old MPI version

Hi Junchao,

The configuration is successful. The error comes from the last step when I run 

make PETSC_DIR=/home/danyangs/soft/petsc/petsc-3.13.6 PETSC_ARCH=linux-intel-openmpi check

*******************Error detected during compile or link!*******************

See http://www.mcs.anl.gov/petsc/documentation/faq.html

/home/danyangs/soft/petsc/petsc-3.13.6/src/snes/tutorials ex5f

*********************************************************

mpif90 -fPIC -O3 -march=native -mtune=nativels   -I/home/danyangs/soft/petsc/petsc-3.13.6/include -I/home/danyangs/soft/petsc/petsc-3.13.6/linux-intel-openmpi/include      ex5f.F90  -Wl,-rpath,/home/danyangs/soft/petsc/petsc-3.13.6/linux-intel-openmpi/lib -L/home/danyangs/soft/petsc/petsc-3.13.6/linux-intel-openmpi/lib -Wl,-rpath,/home/danyangs/soft/petsc/petsc-3.13.6/linux-intel-openmpi/lib -L/home/danyangs/soft/petsc/petsc-3.13.6/linux-intel-openmpi/lib -Wl,-rpath,/global/software/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64 -L/global/software/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64 -Wl,-rpath,/global/software/intel/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/global/software/intel/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -Wl,-rpath,/global/software/openmpi-1.6.5/intel/lib64 -L/global/software/openmpi-1.6.5/intel/lib64 -Wl,-rpath,/global/software/intel/composerxe/mkl/lib/intel64 -L/global/software/intel/composerxe/mkl/lib/intel64 -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/global/software/intel/composerxe/lib/intel64 -lpetsc -lHYPRE -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common -lpord -lscalapack -lsuperlu -lflapack -lfblas -lX11 -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5 -lparmetis -lmetis -lstdc++ -ldl -lmpi_f90 -lmpi_f77 -lmpi -lm -lnuma -lrt -lnsl -lutil -limf -lifport -lifcore -lsvml -lipgo -lintlc -lpthread -lgcc_s -lirc_s -lstdc++ -ldl -o ex5f

ifort: command line warning #10159: invalid argument for option '-m'

/home/danyangs/soft/petsc/petsc-3.13.6/linux-intel-openmpi/lib/libpetsc.so: undefined reference to `MPI_Win_allocate'

/home/danyangs/soft/petsc/petsc-3.13.6/linux-intel-openmpi/lib/libpetsc.so: undefined reference to `MPI_Win_attach'

/home/danyangs/soft/petsc/petsc-3.13.6/linux-intel-openmpi/lib/libpetsc.so: undefined reference to `MPI_Win_create_dynamic'

gmake[4]: *** [ex5f] Error 1

Thanks,

Danyang

From: Junchao Zhang <junchao.zhang at gmail.com>
Date: Saturday, April 10, 2021 at 3:57 PM
To: Danyang Su <danyang.su at gmail.com>
Cc: Barry Smith <bsmith at petsc.dev>, "petsc-users at mcs.anl.gov" <petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] Undefined reference in PETSc 3.13+ with old MPI version

You sent a wrong one. This configure.log was from a successful configuration. Note FOPTFLAGS="-O3 -march=native -mtune=nativels" looks suspicious.

--Junchao Zhang

On Sat, Apr 10, 2021 at 5:32 PM Danyang Su <danyang.su at gmail.com> wrote:

Hi Junchao,

Thanks for looking into this problem. The configuration log is attached.

All the best,

Danyang

From: Junchao Zhang <junchao.zhang at gmail.com>
Date: Saturday, April 10, 2021 at 2:36 PM
To: Danyang Su <danyang.su at gmail.com>
Cc: Barry Smith <bsmith at petsc.dev>, "petsc-users at mcs.anl.gov" <petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] Undefined reference in PETSc 3.13+ with old MPI version

Hi, Danyang, 

Send the configure.log.  Also, PETSc does not need MPI_Win_allocate etc to work. I will have a look.

--Junchao Zhang

On Sat, Apr 10, 2021 at 2:47 PM Danyang Su <danyang.su at gmail.com> wrote:

Hi Barry,

I tried this option before but get ‘Error running configure on OpenMPI’

*******************************************************************************

         UNABLE to CONFIGURE with GIVEN OPTIONS    (see configure.log for details):

-------------------------------------------------------------------------------

Error running configure on OPENMPI

*******************************************************************************

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/configure.py", line 456, in petsc_configure

    framework.configure(out = sys.stdout)

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/framework.py", line 1253, in configure

    self.processChildren()

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/framework.py", line 1242, in processChildren

    self.serialEvaluation(self.childGraph)

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/framework.py", line 1217, in serialEvaluation

    child.configure()

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/package.py", line 1144, in configure

    self.executeTest(self.configureLibrary)

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/base.py", line 140, in executeTest

    ret = test(*args,**kargs)

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/package.py", line 902, in configureLibrary

    for location, directory, lib, incl in self.generateGuesses():

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/package.py", line 476, in generateGuesses

    d = self.checkDownload()

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/packages/OpenMPI.py", line 56, in checkDownload

    return self.getInstallDir()

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/package.py", line 365, in getInstallDir

    installDir = self.Install()

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/packages/OpenMPI.py", line 63, in Install

    installDir = config.package.GNUPackage.Install(self)

  File "/global/home/danyangs/soft/petsc/petsc-3.14.6/config/BuildSystem/config/package.py", line 1667, in Install

    raise RuntimeError('Error running configure on ' + self.PACKAGE)

================================================================================

Finishing configure run at Sat, 10 Apr 2021 11:57:20 -0700

================================================================================

Thanks,

Danyang

From: Barry Smith <bsmith at petsc.dev>
Date: Saturday, April 10, 2021 at 10:31 AM
To: Danyang Su <danyang.su at gmail.com>
Cc: "petsc-users at mcs.anl.gov" <petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] Undefined reference in PETSc 3.13+ with old MPI version

  Depending on the network you can remove the ./configure options --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90  and use instead --with-cc=icc --with-cxx=icpc and--with-fc=ifort --download-openmpi 

  Barry

On Apr 10, 2021, at 12:18 PM, Danyang Su <danyang.su at gmail.com> wrote:

Dear PETSc developers and users,

I am trying to install the latest PETSc version on an ancient cluster. The OpenMPI version is 1.6.5 and Compiler is Intel 14.0, which are the newest on that cluster. I have no problem to install PETSc up to version 3.12.5. However, if I try to use PETSc 3.13+, there are three undefined reference errors in MPI_Win_allocate, MPI_Win_attach and MPI_Win_create_dynamic. I know these three functions are available from OpenMPI 2.0+. Because the cluster is not in technical support anymore, there is no way I can install new OpenMPI version or do some update. Is it possible to disable these three functions in PETSc 3.13+ version?

The errors occur in ‘make check’ step:

/home/dsu/soft/petsc/petsc-3.13.0/linux-intel-openmpi/lib/libpetsc.so: undefined reference to `MPI_Win_allocate'

/home/dsu/soft/petsc/petsc-3.13.0/linux-intel-openmpi/lib/libpetsc.so: undefined reference to `MPI_Win_attach'

/home/dsu/soft/petsc/petsc-3.13.0/linux-intel-openmpi/lib/libpetsc.so: undefined reference to `MPI_Win_create_dynamic'

The configuration used is shown below:

./configure --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 --download-mumps --download-scalapack --download-parmetis --download-metis --download-fblaslapack --download-hypre --download-superlu --download-hdf5=yes --with-debugging=0 COPTFLAGS="-O3 -march=native -mtune=native" CXXOPTFLAGS="-O3 -march=native -mtune=native" FOPTFLAGS="-O3 -march=native -mtune=nativels"

Thanks,

Danyang

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210411/e13c5186/attachment-0001.html>