<div dir="ltr">I do see this:<br><div><br></div><div>15:43 /sw/spock/spack-envs/views/rocm-4.1.0/lib$ nm libhsa-runtime64.so | grep -n hsa_signal_load_scacquir<br>349:0000000000074de0 T hsa_signal_load_scacquire<br>1491:000000000004bef0 t _ZN4rocr3HSA25hsa_signal_load_scacquireE12hsa_signal_s<br>15:43 /sw/spock/spack-envs/views/rocm-4.1.0/lib$<br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 3:45 PM Mark Adams <<a href="mailto:mfadams@lbl.gov">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">No diffs. I added this:<div><br><div>diff --git a/config/BuildSystem/config/packages/zlib.py b/config/BuildSystem/config/packages/zlib.py<br>index fbf9bdf4a0..b76d362536 100644<br>--- a/config/BuildSystem/config/packages/zlib.py<br>+++ b/config/BuildSystem/config/packages/zlib.py<br>@@ -25,6 +25,7 @@ class Configure(config.package.Package):<br> self.pushLanguage('C')<br> args.append('CC="'+self.getCompiler()+'"')<br> args.append('CFLAGS="'+self.updatePackageCFlags(self.getCompilerFlags())+'"')<br>+ args.append('LDFLAGS="'+self.getLinkerFlags()+'"')<br> args.append('prefix="'+self.installDir+'"')<br> self.popLanguage()<br> args=' '.join(args)<br>lines 1-12/12 (END)<br></div><div><br></div><div>but it still fails.</div><div><br></div><div>15:20 jczhang/fix-kokkos-includes *= /gpfs/alpine/csc314/scratch/adams/petsc$ cd /gpfs/alpine/csc314/scratch/adams/petsc/arch-spock-opt-cray-kokkos/externalpackages/zlib-1.2.11 && CC="cc" CFLAGS="-fPIC -fstack-protector -Qunused-arguments -g -O0 -I${ROCM_PATH}/include" prefix="/gpfs/alpine/csc314/scratch/adams/petsc/arch-spock-opt-cray-kokkos" ./configure && /usr/bin/gmake -j8 -l307.2 && /usr/bin/gmake install<br>Checking for shared library support...<br>Building shared library libz.so.1.2.11 with cc.<br>Checking for size_t... Yes.<br>Checking for off64_t... Yes.<br>Checking for fseeko... Yes.<br>Checking for strerror... No.<br>Checking for unistd.h... Yes.<br>Checking for stdarg.h... Yes.<br>Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf().<br>Checking for vsnprintf() in stdio.h... No.<br> WARNING: vsnprintf() not found, falling back to vsprintf(). zlib<br> can build but will be open to possible buffer-overflow security<br> vulnerabilities.<br>Checking for return value of vsprintf()... Yes.<br>Checking for attribute(visibility) support... Yes.<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -c -o example.o test/example.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o adler32.o adler32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o crc32.o crc32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o deflate.o deflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o infback.o infback.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o inffast.o inffast.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o inflate.o inflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o inftrees.o inftrees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o trees.o trees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o zutil.o zutil.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o compress.o compress.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o uncompr.o uncompr.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzclose.o gzclose.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzlib.o gzlib.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzread.o gzread.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzwrite.o gzwrite.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -c -o minigzip.o test/minigzip.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/adler32.o adler32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/crc32.o crc32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/deflate.o deflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/infback.o infback.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/inffast.o inffast.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/inflate.o inflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/inftrees.o inftrees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/trees.o trees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/zutil.o zutil.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/compress.o compress.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/uncompr.o uncompr.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzclose.o gzclose.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzlib.o gzlib.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzread.o gzread.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzwrite.o gzwrite.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -D_FILE_OFFSET_BITS=64 -c -o example64.o test/example.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -D_FILE_OFFSET_BITS=64 -c -o minigzip64.o test/minigzip.c<br>ar rc libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o gzwrite.o<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o example example.o -L. libz.a<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o minigzip minigzip.o -L. libz.a<br>cc -shared -Wl,-soname,libz.so.1,--version-script,zlib.map -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o libz.so.1.2.11 adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo -lc<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o minigzip64 minigzip64.o -L. libz.a<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o example64 example64.o -L. libz.a<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]<br><br>ld.lld: ld.lld: error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br>/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]ld.lld<br>: ld.lld: error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br>/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>gmake: *** [Makefile:292: minigzip] Error 1<br>gmake: *** Waiting for unfinished jobs....<br>gmake: *** [Makefile:289: example] Error 1<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>gmake: *** [Makefile:304: minigzip64] Error 1<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>gmake: *** [Makefile:301: example64] Error 1<br>rm -f libz.so libz.so.1<br>ln -s libz.so.1.2.11 libz.so<br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 2:03 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Wed, Jul 7, 2021 at 1:40 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 1:26 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><br></div> You will need to pass the -L arguments appropriately to zlib's ./configure so it can link its shared library appropriately. That is, the zlib configure requires the value obtained with L'+os.environ['ROCM_PATH'],+'/lib -lhsa-runtime64', <br></div></blockquote><div><br></div><div>It's not clear to me how to do that. I added the -L to my configure script. It is not clear to me how to modify Matt's command.</div></div></div></blockquote><div><br></div><div>Can you try this?</div><div><br></div><div>knepley/feature-orientation-rethink *$:/PETSc3/petsc/petsc-dev$ git diff config/BuildSystem/config/packages/zlib.py<br>diff --git a/config/BuildSystem/config/packages/zlib.py b/config/BuildSystem/config/packages/zlib.py<br>index fbf9bdf4a0a..b76d3625364 100644<br>--- a/config/BuildSystem/config/packages/zlib.py<br>+++ b/config/BuildSystem/config/packages/zlib.py<br>@@ -25,6 +25,7 @@ class Configure(config.package.Package):<br><br> self.pushLanguage('C')<br> args.append('CC="'+self.getCompiler()+'"')<br> args.append('CFLAGS="'+self.updatePackageCFlags(self.getCompilerFlags())+'"')<br>+ args.append('LDFLAGS="'+self.getLinkerFlags()+'"')<br> args.append('prefix="'+self.installDir+'"')<br> self.popLanguage()<br> args=' '.join(args)<br></div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><br><blockquote type="cite"><div>On Jul 7, 2021, at 12:18 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:</div><br><div><div dir="ltr">Well, still getting these hsa errors:<br><div><br></div><div>13:07 jczhang/fix-kokkos-includes= /gpfs/alpine/csc314/scratch/adams/petsc$ !136<br>cd /gpfs/alpine/csc314/scratch/adams/petsc/arch-spock-opt-cray-kokkos/externalpackages/zlib-1.2.11 && CC="cc" CFLAGS="-fPIC -fstack-protector -Qunused-arguments -g -O0 -I${ROCM_PATH}/include" prefix="/gpfs/alpine/csc314/scratch/adams/petsc/arch-spock-opt-cray-kokkos" ./configure && /usr/bin/gmake -j8 -l307.2 && /usr/bin/gmake install<br>Checking for shared library support...<br>Building shared library libz.so.1.2.11 with cc.<br>Checking for size_t... Yes.<br>Checking for off64_t... Yes.<br>Checking for fseeko... Yes.<br>Checking for strerror... No.<br>Checking for unistd.h... Yes.<br>Checking for stdarg.h... Yes.<br>Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf().<br>Checking for vsnprintf() in stdio.h... No.<br> WARNING: vsnprintf() not found, falling back to vsprintf(). zlib<br> can build but will be open to possible buffer-overflow security<br> vulnerabilities.<br>Checking for return value of vsprintf()... Yes.<br>Checking for attribute(visibility) support... Yes.<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -c -o example.o test/example.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o adler32.o adler32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o crc32.o crc32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o deflate.o deflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o infback.o infback.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o inffast.o inffast.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o inflate.o inflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o inftrees.o inftrees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o trees.o trees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o zutil.o zutil.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o compress.o compress.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o uncompr.o uncompr.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzclose.o gzclose.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzlib.o gzlib.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzread.o gzread.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzwrite.o gzwrite.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -c -o minigzip.o test/minigzip.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/adler32.o adler32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/crc32.o crc32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/deflate.o deflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/infback.o infback.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/inffast.o inffast.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/inflate.o inflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/inftrees.o inftrees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/trees.o trees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/zutil.o zutil.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/compress.o compress.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/uncompr.o uncompr.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzclose.o gzclose.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzlib.o gzlib.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzread.o gzread.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzwrite.o gzwrite.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -D_FILE_OFFSET_BITS=64 -c -o example64.o test/example.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -D_FILE_OFFSET_BITS=64 -c -o minigzip64.o test/minigzip.c<br>ar rc libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o gzwrite.o<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o example example.o -L. libz.a<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o minigzip minigzip.o -L. libz.a<br>cc -shared -Wl,-soname,libz.so.1,--version-script,zlib.map -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o libz.so.1.2.11 adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo -lc<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o minigzip64 minigzip64.o -L. libz.a<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o example64 example64.o -L. libz.a<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]<br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br><br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>gmake: *** [Makefile:289: example] Error 1<br>gmake: *** Waiting for unfinished jobs....<br>gmake: *** [Makefile:292: minigzip] Error 1<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>gmake: *** [Makefile:304: minigzip64] Error 1<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>gmake: *** [Makefile:301: example64] Error 1<br>rm -f libz.so libz.so.1<br>ln -s libz.so.1.2.11 libz.so<br>ln -s libz.so.1.2.11 libz.so.1<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 1:05 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thanks, it was missing the /<br><div><br></div><div> '--LDFLAGS=-L'+os.environ['ROCM_PATH'],+'/lib -lhsa-runtime64',<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 12:48 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Did you look in /sw/spock/spack-envs/views/rocm-4.1.0lib ?<div><br></div><div> Matt</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 12:29 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Ok, I tried that but now I get this error.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 12:13 PM Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" target="_blank">stefano.zampini@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">There's an extra comma</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il Mer 7 Lug 2021, 18:08 Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Humm, I get this error (I just copied your whole file into here):<div><br></div><div>12:06 jczhang/fix-kokkos-includes= /gpfs/alpine/csc314/scratch/adams/petsc$ ~/arch-spock-dbg-cray-kokkos.py<br>Traceback (most recent call last):<br> File "/ccs/home/adams/arch-spock-dbg-cray-kokkos.py", line 27, in <module><br> '--LDFLAGS=-L'+os.environ['ROCM_PATH'],+'lib -lhsa-runtime64',<br>TypeError: bad operand type for unary +: 'str'<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 11:08 AM Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" rel="noreferrer" target="_blank">stefano.zampini@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>Mark<div><br></div><div>On Spock, you can use <a href="https://gitlab.com/petsc/petsc/-/blob/main/config/examples/arch-olcf-spock.py" rel="noreferrer" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/config/examples/arch-olcf-spock.py</a> as a template for your configuration. You need to add libraries as LDFLAGS to resolve the hsa symbols<br><div><br><blockquote type="cite"><div>On Jul 7, 2021, at 5:04 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" rel="noreferrer" target="_blank">mfadams@lbl.gov</a>> wrote:</div><br><div><div dir="ltr">Thanks,<br><div><br></div><div>08:30 jczhang/fix-kokkos-includes= /gpfs/alpine/csc314/scratch/adams/petsc$ cd /gpfs/alpine/csc314/scratch/adams/petsc/arch-spock-opt-cray-kokkos/externalpackages/zlib-1.2.11 && CC="cc" CFLAGS="-fPIC -fstack-protector -Qunused-arguments -g -O0 -I${ROCM_PATH}/include" prefix="/gpfs/alpine/csc314/scratch/adams/petsc/arch-spock-opt-cray-kokkos" ./configure && /usr/bin/gmake -j8 -l307.2 && /usr/bin/gmake install<br>Checking for shared library support...<br>Building shared library libz.so.1.2.11 with cc.<br>Checking for size_t... Yes.<br>Checking for off64_t... Yes.<br>Checking for fseeko... Yes.<br>Checking for strerror... No.<br>Checking for unistd.h... Yes.<br>Checking for stdarg.h... Yes.<br>Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf().<br>Checking for vsnprintf() in stdio.h... No.<br> WARNING: vsnprintf() not found, falling back to vsprintf(). zlib<br> can build but will be open to possible buffer-overflow security<br> vulnerabilities.<br>Checking for return value of vsprintf()... Yes.<br>Checking for attribute(visibility) support... Yes.<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -c -o example.o test/example.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o adler32.o adler32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o crc32.o crc32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o deflate.o deflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o infback.o infback.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o inffast.o inffast.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o inflate.o inflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o inftrees.o inftrees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o trees.o trees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o zutil.o zutil.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o compress.o compress.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o uncompr.o uncompr.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzclose.o gzclose.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzlib.o gzlib.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzread.o gzread.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -c -o gzwrite.o gzwrite.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -c -o minigzip.o test/minigzip.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/adler32.o adler32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/crc32.o crc32.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/deflate.o deflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/infback.o infback.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/inffast.o inffast.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/inflate.o inflate.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/inftrees.o inftrees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/trees.o trees.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/zutil.o zutil.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/compress.o compress.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/uncompr.o uncompr.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzclose.o gzclose.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzlib.o gzlib.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzread.o gzread.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -DPIC -c -o objs/gzwrite.o gzwrite.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -D_FILE_OFFSET_BITS=64 -c -o example64.o test/example.c<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -I. -D_FILE_OFFSET_BITS=64 -c -o minigzip64.o test/minigzip.c<br>ar rc libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o gzwrite.o<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o example example.o -L. libz.a<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o minigzip minigzip.o -L. libz.a<br>cc -shared -Wl,-soname,libz.so.1,--version-script,zlib.map -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -fPIC -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o libz.so.1.2.11 adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo -lc<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o minigzip64 minigzip64.o -L. libz.a<br>cc -fPIC -fstack-protector -Qunused-arguments -g -O0 -I/sw/spock/spack-envs/views/rocm-4.1.0/include -D_LARGEFILE64_SOURCE=1 -DNO_STRERROR -DNO_vsnprintf -DHAVE_HIDDEN -o example64 example64.o -L. libz.a<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]<br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br><br>ld.lldld.lld: : error: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]/opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br><br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>gmake: *** [Makefile:292: minigzip] Error 1<br>gmake: *** Waiting for unfinished jobs....<br>gmake: *** [Makefile:289: example] Error 1<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>gmake: *** [Makefile:304: minigzip64] Error 1<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_allocate [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agent_iterate_memory_pools [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_iterate_agents [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_load_scacquire [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_unlock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_signal_destroy [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_get_info [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_lock [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_memory_pool_free [--no-allow-shlib-undefined]<br>ld.lld: error: /opt/cray/pe/mpich/8.1.4/gtl/lib/libmpi_gtl_hsa.so: undefined reference to hsa_amd_agents_allow_access [--no-allow-shlib-undefined]<br>clang-11: error: linker command failed with exit code 1 (use -v to see invocation)<br>gmake: *** [Makefile:301: example64] Error 1<br>rm -f libz.so libz.so.1<br>ln -s libz.so.1.2.11 libz.so<br>ln -s libz.so.1.2.11 libz.so.1<br>11:03 2 jczhang/fix-kokkos-includes= /gpfs/alpine/csc314/scratch/adams/petsc/arch-spock-opt-cray-kokkos/externalpackages/zlib-1.2.11$<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 9:18 AM Matthew Knepley <<a href="mailto:knepley@gmail.com" rel="noreferrer" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">It is hard to see the error. I suspect it is something crazy with the install. Can you run the build by hand?<div><br></div><div>cd /gpfs/alpine/csc314/scratch/adams/petsc/arch-spock-opt-cray-kokkos/externalpackages/zlib-1.2.11 && CC="cc" CFLAGS="-fPIC -fstack-protector -Qunused-arguments -g -O0 -I${ROCM_PATH}/include" prefix="/gpfs/alpine/csc314/scratch/adams/petsc/arch-spock-opt-cray-kokkos" ./configure && /usr/bin/gmake -j8 -l307.2 && /usr/bin/gmake install<br></div><div><br></div><div>and see what happens, and what the error code is?</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 8:48 AM Mark Adams <<a href="mailto:mfadams@lbl.gov" rel="noreferrer" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Also, this is in jczhang/fix-kokkos-includes</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 7, 2021 at 8:46 AM Mark Adams <<a href="mailto:mfadams@lbl.gov" rel="noreferrer" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Apparently the same error with --download-zlib=/yourselectedlocation/zlib-1.2.11.tar.gz</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jul 6, 2021 at 11:53 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" rel="noreferrer" target="_blank">bsmith@petsc.dev</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">$ curl <a href="http://www.zlib.net/zlib-1.2.11.tar.gz" rel="noreferrer" target="_blank">http://www.zlib.net/zlib-1.2.11.tar.gz</a> > zlib-1.2.11.tar.gz</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> % Total % Received % Xferd Average Speed Time Time Time Current</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"> Dload Upload Total Spent Left Speed</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">100 593k 100 593k 0 0 835k 0 --:--:-- --:--:-- --:--:-- 834k</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(200,20,201)"><span style="font-variant-ligatures:no-common-ligatures">~/Src/petsc</span><span style="font-variant-ligatures:no-common-ligatures"><b> (barry/2021-07-03/demonstrate-network-parallel-build=)</b></span><span style="font-variant-ligatures:no-common-ligatures"> arch-demonstrate-network-parallel-build</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">$ tar -zxf zlib-1.2.11.tar.gz </span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(200,20,201)"><span style="font-variant-ligatures:no-common-ligatures">~/Src/petsc</span><span style="font-variant-ligatures:no-common-ligatures"><b> (barry/2021-07-03/demonstrate-network-parallel-build=)</b></span><span style="font-variant-ligatures:no-common-ligatures"> arch-demonstrate-network-parallel-build</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">$ ls zlib-1.2.11</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">CMakeLists.txt adler32.c deflate.c gzread.c inflate.h </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">os400</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">watcom</span><span style="font-variant-ligatures:no-common-ligatures"> zlib.h</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">ChangeLog </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">amiga</span><span style="font-variant-ligatures:no-common-ligatures"> deflate.h gzwrite.c inftrees.c </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">qnx</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">win32</span><span style="font-variant-ligatures:no-common-ligatures"> zlib.map</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">FAQ compress.c </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">doc</span><span style="font-variant-ligatures:no-common-ligatures"> infback.c inftrees.h </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">test</span><span style="font-variant-ligatures:no-common-ligatures"> zconf.h zlib.pc.cmakein</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">INDEX </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(180,36,25)">configure</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">examples</span><span style="font-variant-ligatures:no-common-ligatures"> inffast.c <a href="http://make_vms.com/" rel="noreferrer" target="_blank">make_vms.com</a> treebuild.xml zconf.h.cmakein <a href="http://zlib.pc.in/" rel="noreferrer" target="_blank">zlib.pc.in</a></span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">Makefile </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">contrib</span><span style="font-variant-ligatures:no-common-ligatures"> gzclose.c inffast.h </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">msdos</span><span style="font-variant-ligatures:no-common-ligatures"> trees.c <a href="http://zconf.h.in/" rel="noreferrer" target="_blank">zconf.h.in</a> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(180,36,25)">zlib2ansi</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">Makefile.in crc32.c gzguts.h inffixed.h </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">nintendods</span><span style="font-variant-ligatures:no-common-ligatures"> trees.h zlib.3 zutil.c</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures">README crc32.h gzlib.c inflate.c </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(64,11,217)">old</span><span style="font-variant-ligatures:no-common-ligatures"> uncompr.c zlib.3.pdf zutil.h</span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div style="margin:0px;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo"><br></div><div><br><blockquote type="cite"><div>On Jul 6, 2021, at 7:57 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" rel="noreferrer" target="_blank">mfadams@lbl.gov</a>> wrote:</div><br><div><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jul 6, 2021 at 6:42 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" rel="noreferrer" target="_blank">bsmith@petsc.dev</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
Mark,<br>
<br>
You can try what the configure error message should be suggesting (it is not clear if that is being printed to your screen or no).<br>
<br>
ERROR: Unable to download package ZLIB from: <a href="http://www.zlib.net/zlib-1.2.11.tar.gz" rel="noreferrer noreferrer" target="_blank">http://www.zlib.net/zlib-1.2.11.tar.gz</a></blockquote><div><br></div><div>My browser can not open this and I could not see a download button on this site.</div><div><br></div><div>Can you download this?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
* If URL specified manually - perhaps there is a typo?<br>
* If your network is disconnected - please reconnect and rerun ./configure<br>
* Or perhaps you have a firewall blocking the download<br>
* You can run with --with-packages-download-dir=/adirectory and ./configure will instruct you what packages to download manually<br>
* or you can download the above URL manually, to /yourselectedlocation/zlib-1.2.11.tar.gz<br>
and use the configure option:<br>
--download-zlib=/yourselectedlocation/zlib-1.2.11.tar.gz<br>
<br>
Barry<br>
<br>
<br>
> On Jul 6, 2021, at 4:29 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" rel="noreferrer" target="_blank">mfadams@lbl.gov</a>> wrote:<br>
> <br>
> I am getting some sort of error in build zlib on Spock at ORNL.<br>
> Other libraries are downloaded and I am sure the network is fine.<br>
> Any ideas?<br>
> Thanks,<br>
> Mark<br>
> <configure.log><br>
<br>
</blockquote></div></div>
</div></blockquote></div><br></div></blockquote></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div>
</blockquote></div>
</div></blockquote></div><br></div></div></blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div>
<span id="gmail-m_4213878651678470358gmail-m_-8688883016868545400gmail-m_2789541683600680410gmail-m_7460271565475788618cid:f_kqtqvopr0"><configure.log></span></div></blockquote></div><br></div></blockquote></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div>