incorrect AMD math library link order

Tue Mar 12 11:34:25 CDT 2024

Hi, Max

Since I do not have an access to the AMD environment, I will just make
suggestions based on what I am seeing from the two files you provided.

I notice -freciprocal-math is used, which may be the reason libtool
thinks -lamdlibm is required. Could you remove it and try again?

Other unrelated issues:
1. --enable-mpi-io-test is not a valid configure option of PnetCDF
2. -fPIC can be removed. If you want to build shared library, just add
   --enable-shared to the configure command line

Wei-keng

On Mar 12, 2024, at 6:57 AM, Max R. Dechantsreiter <max at performancejones.com> wrote:

Hi Wei-Keng,

I ran my entire build script (including tests) for PnetCDF 1.12.3.
Attached is that script, with embedded annotations; along with its
config.log, as you requested.  Please note that for this build I
did not set LIBS or invoke libamdlibm in any way - yet linkage was
with'-lm -lamdlibm', making me wonder how libtool came to that.
(This AMD library does not appear in my build of PnetCDF 1.12.3
with GCC toolchain.)

While I doubt linking with '-lm -lamdlibm' will affect performance
very much, it is annoying that I cannot gain full control over the
build.  Meanwhile I will continue building my application with the
PnetCDF that I have now.

Thank you,

Max
---

On Mon, Mar 11, 2024 at 05:53:00PM +0000, Wei-Keng Liao wrote:
This sounds most like the problem of libtool.
Libtool figures out all the library dependency and adds those required.

The reason I asked for the testings is to rule out the possibility of
interference from environment variable settings, such as LD_LIBRARY_PATH
or modules loaded. PnetCDF only needs MPI compilers, so the minimal
configure command-line option is --with-mpi

Could you please send me the file "config.log"?

Wei-keng

On Mar 11, 2024, at 11:36 AM, Max R. Dechantsreiter <max at performancejones.com> wrote:

Hi Wei-Keng,

Could you modify build_pnetcdf.txt to compile a simple F90 program
(example programs can be found from MPICH repo in
https://urldefense.com/v3/__https://github.com/pmodels/mpich/tree/main/examples__;!!Dq0X2DkFhyF93HkjWTBQKhk!Qe-o1JhyzY0PqJpYUc01_4WRS9WF7O7reBH9vDAZtx2kXnt1lYpf1m85yJ2aQgdn2FjM2uQWiGP1XLSCw6pMtqM$ )

This is to see if the environment variables set in build_pnetcdf.txt
also work without PnetCDF? For example, run this command:
  mpifort pi3f90.f90

I don't quite understand this - I know my AOCC OpenMPI
works, because I have used it, set up the same way, to
build other tools/applications (e.g., HDF5).

If that works, could you try this configure command, without running
build_pnetcdf.txt? i.e. no environment variables are set.
 ./configure --with-mpi=$HOME/AOCC/opt/amd/ompi/5.0.2

I presume by this you meant I should set the environment
for AOCC tools and OpenMPI, but not set the 'configure'
variables (such as LIBS).  Then with

./configure --with-mpi=$HOME/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0

I still have libamdlibm after libm:

libtool: link: /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/bin/mpicc -g -O2 -o ncmpidump ncmpidump.o vardata.o dumplib.o  ../../../src/libs/.libs/libpnetcdf.a -L/home/maxd/AOCC/opt/amd/ompi/hwloc-2.10.0_aocc-4.2.0/lib /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libmpi_usempif08.so /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libmpi_usempi_ignore_tkr.so /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libmpi_mpifh.so -L/home/maxd/AOCC/opt/amd/aocc/4.2.0/lib /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libmpi.so /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libopen-pal.so -lpthread -lrt /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libpmix.so -lutil -ldl -levent_core -levent_pthreads /home/maxd/AOCC/opt/amd/ompi/hwloc-2.10.0_aocc-4.2.0/lib/libhwloc.so -lm -lamdlibm -pthread -Wl,-rpath -Wl,/home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib -Wl,-rpath -Wl,/home/maxd/AOCC/opt/amd/ompi/hwloc-2.10.0_aocc-4.2.0/lib -Wl,-rpath -Wl,/home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib -Wl,-rpath -Wl,/home/maxd/AOCC/opt/amd/ompi/hwloc-2.10.0_aocc-4.2.0/lib

How is libtool (or configure) deciding to use libamdlibm -
is it based on the compiler?

Best regards,

Max
---

If the above failed, please send me file config.log.

Wei-keng

On Mar 11, 2024, at 10:37 AM, Max R. Dechantsreiter <max at performancejones.com> wrote:

Hi Wei-keng,

...Same problem.  I am attaching my build procedure.

I even tried hacking the Makefile and libtool, to no effect.

Best regards,

Max
---

On Mon, Mar 11, 2024 at 03:12:27PM +0000, Wei-Keng Liao wrote:
Hi, Max

Could you give the release candidate 1.13.0.rc1 a try?
The configure utility files have been updated since 1.12.3,
which may fix the problem.

https://urldefense.com/v3/__https://github.com/Parallel-NetCDF/Parallel-NetCDF.github.io/raw/master/Release/pnetcdf-1.13.0.rc1.tar.gz__;!!Dq0X2DkFhyF93HkjWTBQKhk!QaDTsobUHYLC8K-towJw_kn5rctCrB6wv3OdDy6h4_9yKnUNepXT1b2RCV-EcqmdpdkcMAcwrT2IN4hDGUZIk9s$

Wei-keng

On Mar 11, 2024, at 6:36 AM, Max R. Dechantsreiter <max at performancejones.com> wrote:

Hello Wei-Keng,

Building PnetCDF 1.12.3 with AOCC compilers, I found that
libamdlm is linked after libm ('-lm -lamdlibm') instead of
before as it should be; so far I have not found any way to
change this.  If I set LIBS='-lamdlibm' and configure, both
FCLIBS and FLIBS in the resulting Makefile are assigned

-L/home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib -L/usr/lib/gcc/x86_64-linux-gnu/10 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/home/maxd/AOCC/opt/amd/aocc/4.2.0/bin/../lib -L/lib -L/usr/lib -L/home/maxd/AOCC/opt/amd/aocl/4.2.0/aocc/lib -L/home/maxd/AOCC/opt/amd/aocc/4.2.0/lib -L/home/maxd/AOCC/opt/amd/aocc/4.2.0/lib32 -L. -lamdlibm -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lflangmain -lflang -lflangrti -lpgmath -lquadmath -lompstub -lm -lrt -lpthread

which is promising.  However, in the link it's wrong:

libtool: link: /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/bin/mpif90 -I../../src/binding/f90 -ffree-form -I. -DN_DIM=3 -DMAXBLOCKS=100 -DIONMAX=13 -march=broadwell -mtune=broadwell -O3 -freciprocal-math -fPIC -o flash_benchmark_io get_mfluid_property.o flash_release.o flash_benchmark_io.o checkpoint_ncmpi_parallel.o plotfile_ncmpi_parallel.o  -L../../src/libs /home/maxd/AOCC/Build/pnetcdf-1.12.3/src/libs/.libs/libpnetcdf.a -L/home/maxd/AOCC/opt/amd/ompi/hwloc-2.10.0_aocc-4.2.0/lib /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libmpi_usempif08.so /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libmpi_usempi_ignore_tkr.so /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libmpi_mpifh.so -L/home/maxd/AOCC/opt/amd/aocc/4.2.0/lib /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libmpi.so /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libopen-pal.so -lpthread -lrt /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib/libpmix.so -lutil -ldl -levent_core -levent_pthreads /home/maxd/AOCC/opt/amd/ompi/hwloc-2.10.0_aocc-4.2.0/lib/libhwloc.so -lm -lamdlibm -pthread -rpath /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib -rpath /home/maxd/AOCC/opt/amd/ompi/hwloc-2.10.0_aocc-4.2.0/lib -rpath /home/maxd/AOCC/opt/amd/ompi/openmpi-5.0.2_aocc-4.2.0/lib -rpath /home/maxd/AOCC/opt/amd/ompi/hwloc-2.10.0_aocc-4.2.0/lib

Any suggestion for how to remedy this?

Best regards,

Max
---
Max R. Dechantsreiter
President
Performance Jones L.L.C.
max at performancejones.com
Skype: PerformanceJones (UTC+01:00)
+1 414 446-3100 (telephone/voicemail)
https://urldefense.com/v3/__http://www.linkedin.com/in/benchmarking__;!!Dq0X2DkFhyF93HkjWTBQKhk!XoFfRmNF4v6X3b67Y51DU9xey_PcSw1VYyTqFCWAsLtD1l0FSMXpDqzdhbHEA9T8wvV7jXjXnH3AaDZYp90LdZg$

<build_pnetcdf.txt>

<build_pnetcdf.txt><config.log>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/parallel-netcdf/attachments/20240312/eb51e571/attachment-0001.html>