[petsc-users] configure error
Barry Smith
bsmith at petsc.dev
Sat May 15 23:31:56 CDT 2021
Looks like nvcc_wrapper gets confused when both -arch and -gencode are used. But handles either of them separately correctly.
Mark,
PETSc configure now handles the -arch and --with-kokkos-cuda-arch= business automatically so you do not, and generally shouldn't pass -arch sm_70 in --CUDAFLAGS anymore or use the --with-kokkos-cuda-arch option anymore.
If you need to set the -arch to something earlier than the system supports (rare) you can now use -with-cuda-gencodearch to set that instead of using CUDAFLAGS.
Barry
$ NVCC_WRAPPER_DEFAULT_COMPILER=mpicxx /home/bsmith/petsc/arch-main/bin/nvcc_wrapper --show --expt-extended-lambda -c -Xcompiler -fPIC -g -gencode arch=compute_70,code=sm_70 -I/home/bsmith/soft/gnu-mpich/include -Wno-deprecated-gpu-targets -I/home/bsmith/petsc/include -I/home/bsmith/petsc/arch-main/include -I/nfs/apps/spacks/2021-02-09/opt/spack/linux-centos7-x86_64/gcc-7.3.0/cuda-11.2.0-ikruu5mo6dtt3avvmwsejouqhxu4btdm/include /home/bsmith/petsc/src/ts/utils/dmplexlandau/kokkos/landau.kokkos.cxx -o arch-main/obj/ts/utils/dmplexlandau/kokkos/landau.o
nvcc --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -Wno-deprecated-gpu-targets -ccbin mpicxx -g -I/home/bsmith/soft/gnu-mpich/include -I/home/bsmith/petsc/include -I/home/bsmith/petsc/arch-main/include -I/nfs/apps/spacks/2021-02-09/opt/spack/linux-centos7-x86_64/gcc-7.3.0/cuda-11.2.0-ikruu5mo6dtt3avvmwsejouqhxu4btdm/include -Xcompiler -fPIC -x cu /home/bsmith/petsc/src/ts/utils/dmplexlandau/kokkos/landau.kokkos.cxx -c -o arch-main/obj/ts/utils/dmplexlandau/kokkos/landau.o
~/petsc (main=) arch-main
$ NVCC_WRAPPER_DEFAULT_COMPILER=mpicxx /home/bsmith/petsc/arch-main/bin/nvcc_wrapper --show --expt-extended-lambda -c -arch=sm_70 -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -O3 -gencode arch=compute_80,code=sm_80 -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include -Wno-deprecated-gpu-targets
nvcc --expt-extended-lambda -arch=sm_70 -Wno-deprecated-gpu-targets -ccbin mpicxx -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -O3 -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include -Xcompiler -rdynamic,-fPIC,arch=compute_80,code=sm_80 -c
~/petsc (main=) arch-main
$ NVCC_WRAPPER_DEFAULT_COMPILER=mpicxx /home/bsmith/petsc/arch-main/bin/nvcc_wrapper --show --expt-extended-lambda -c -Xcompiler -fPIC -g -arch=sm_70 -gencode arch=compute_70,code=sm_70 -I/home/bsmith/soft/gnu-mpich/include -Wno-deprecated-gpu-targets -I/home/bsmith/petsc/include -I/home/bsmith/petsc/arch-main/include -I/nfs/apps/spacks/2021-02-09/opt/spack/linux-centos7-x86_64/gcc-7.3.0/cuda-11.2.0-ikruu5mo6dtt3avvmwsejouqhxu4btdm/include /home/bsmith/petsc/src/ts/utils/dmplexlandau/kokkos/landau.kokkos.cxx -o arch-main/obj/ts/utils/dmplexlandau/kokkos/landau.o
nvcc --expt-extended-lambda -arch=sm_70 -Wno-deprecated-gpu-targets -ccbin mpicxx -g -I/home/bsmith/soft/gnu-mpich/include -I/home/bsmith/petsc/include -I/home/bsmith/petsc/arch-main/include -I/nfs/apps/spacks/2021-02-09/opt/spack/linux-centos7-x86_64/gcc-7.3.0/cuda-11.2.0-ikruu5mo6dtt3avvmwsejouqhxu4btdm/include -Xcompiler -fPIC,arch=compute_70,code=sm_70 -x cu /home/bsmith/petsc/src/ts/utils/dmplexlandau/kokkos/landau.kokkos.cxx -c -o arch-main/obj/ts/utils/dmplexlandau/kokkos/landau.o
> On May 15, 2021, at 8:44 PM, Junchao Zhang <junchao.zhang at gmail.com> wrote:
>
> Add --show to the failing nvcc_wrapper command and see how nvcc_wrapper parses the options
>
> PATH=/global/common/sw/cray/sles15/x86_64/zlib/1.2.11/gcc/8.2.0/pep2pal/bin:/global/common/sw/cray/cnl7/haswell/cmake/3.18.2/bin:/opt/cray/rca/2.2.20-7.0.1.1_4.62__g8e3fb5b.ari/bin:/opt/cray/alps/6.6.58-7.0.1.1_6.20__g437d88db.ari/sbin:/opt/cray/alps/default/bin:/opt/cray/job/2.2.4-7.0.1.1_3.48__g36b56f4.ari/bin:/opt/cray/pe/craype/2.6.2/bin:/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/compilers/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/cuda/bin:/usr/common/software/sles15_cgpu/cuda/11.1.1/bin:/usr/common/software/sles15_cgpu/gcc/8.3.0/bin:/opt/esslurm/bin:/opt/cray/pe/modules/3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:`dirname <http://3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:dirname> /usr/common/software/sles15_cgpu/cuda/11.1.1/bin/nvcc` NVCC_WRAPPER_DEFAULT_COMPILER=/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin/mpicxx /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/bin/nvcc_wrapper --expt-extended-lambda -c -arch=sm_70 -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -O3 -gencode arch=compute_80,code=sm_80 -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include -Wno-deprecated-gpu-targets -I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/include -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include -I/usr/common/software/sles15_cgpu/cuda/11.1.1/include /global/u2/m/madams/petsc/src/sys/objects/kokkos/kinit.kokkos.cxx -o arch-cori-gpu-opt-kokkos-gcc/obj/sys/objects/kokkos/kinit.o --show
>
> --Junchao Zhang
>
>
> On Sat, May 15, 2021 at 7:14 PM Mark Adams <mfadams at lbl.gov <mailto:mfadams at lbl.gov>> wrote:
> Thanks,
> Now I get an error in make all. THis was working a few weeks ago.
> Make.log was empty but here is the output.
> ....
> CC arch-cori-gpu-opt-kokkos-gcc/obj/sys/error/ftn-custom/zerrf.o
> CC arch-cori-gpu-opt-kokkos-gcc/obj/sys/f90-src/f90_cwrap.o
> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/hdf5io.o
> PATH=/global/common/sw/cray/sles15/x86_64/zlib/1.2.11/gcc/8.2.0/pep2pal/bin:/global/common/sw/cray/cnl7/haswell/cmake/3.18.2/bin:/opt/cray/rca/2.2.20-7.0.1.1_4.62__g8e3fb5b.ari/bin:/opt/cray/alps/6.6.58-7.0.1.1_6.20__g437d88db.ari/sbin:/opt/cray/alps/default/bin:/opt/cray/job/2.2.4-7.0.1.1_3.48__g36b56f4.ari/bin:/opt/cray/pe/craype/2.6.2/bin:/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/compilers/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/cuda/bin:/usr/common/software/sles15_cgpu/cuda/11.1.1/bin:/usr/common/software/sles15_cgpu/gcc/8.3.0/bin:/opt/esslurm/bin:/opt/cray/pe/modules/3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:`dirname <http://3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:dirname> /usr/common/software/sles15_cgpu/cuda/11.1.1/bin/nvcc` NVCC_WRAPPER_DEFAULT_COMPILER=/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin/mpicxx /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/bin/nvcc_wrapper --expt-extended-lambda -c -arch=sm_70 -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -O3 -gencode arch=compute_80,code=sm_80 -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include -Wno-deprecated-gpu-targets -I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/include -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include -I/usr/common/software/sles15_cgpu/cuda/11.1.1/include /global/u2/m/madams/petsc/src/sys/objects/kokkos/kinit.kokkos.cxx -o arch-cori-gpu-opt-kokkos-gcc/obj/sys/objects/kokkos/kinit.o
> g++: error: arch=compute_80: No such file or directory
> g++: error: code=sm_80: No such file or directory
> gmake[3]: *** [gmakefile:188: arch-cori-gpu-opt-kokkos-gcc/obj/sys/objects/kokkos/kinit.o] Error 1
> gmake[3]: *** Waiting for unfinished jobs....
> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/isio.o
> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/ftn-auto/pmapf.o
> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/pmap.o
> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/psort.o
> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/isltog.o
> FC arch-cori-gpu-opt-kokkos-gcc/obj/vec/f90-mod/petscvecmod.o
> CUDAC arch-cori-gpu-opt-kokkos-gcc/obj/sys/memory/cuda/mcudahost.o
> CUDAC.dep arch-cori-gpu-opt-kokkos-gcc/obj/sys/memory/cuda/mcudahost.o
> gmake[2]: *** [/global/homes/m/madams/petsc/lib/petsc/conf/rules:50: libs] Error 2
> **************************ERROR*************************************
> Error during compile, check arch-cori-gpu-opt-kokkos-gcc/lib/petsc/conf/make.log
> Send it and arch-cori-gpu-opt-kokkos-gcc/lib/petsc/conf/configure.log to petsc-maint at mcs.anl.gov <mailto:petsc-maint at mcs.anl.gov>
> ********************************************************************
> gmake[1]: *** [makefile:40: all] Error 1
> make: *** [GNUmakefile:9: all] Error 2
>
> On Sat, May 15, 2021 at 5:30 PM Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
>
> Bitbucket is off-line. Do you need fortran stubs at this moment? If not use --with-fortran-interface=0
>
>
>> On May 15, 2021, at 2:51 PM, Mark Adams <mfadams at lbl.gov <mailto:mfadams at lbl.gov>> wrote:
>>
>> I can build a non-kokkos PETSc here (Cori/GPU) but I get this error with Kokkos.
>> Any suggestions?
>> Thanks,
>> Mark
>>
>> ============================================================================================= ============================================================================================= Trying to download git://https://bitbucket.org/petsc/pkg-sowing.git <https://bitbucket.org/petsc/pkg-sowing.git> for SOWING ============================================================================================= ============================================================================================= Trying to download git://https://bitbucket.org/petsc/pkg-sowing.git <https://bitbucket.org/petsc/pkg-sowing.git> for SOWING ============================================================================================= ============================================================================================= Trying to download https://bitbucket.org/petsc/pkg-sowing/get/v1.1.26-p1.tar.gz <https://bitbucket.org/petsc/pkg-sowing/get/v1.1.26-p1.tar.gz> for SOWING ============================================================================================= *******************************************************************************
>> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):
>> -------------------------------------------------------------------------------
>> Error during download/extract/detection of SOWING:
>> Unable to clone sowing
>> Could not execute "['git clone https://bitbucket.org/petsc/pkg-sowing.git <https://bitbucket.org/petsc/pkg-sowing.git> /global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing']":
>> Cloning into '/global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing'...
>> fatal: unable to access 'https://bitbucket.org/petsc/pkg-sowing.git/ <https://bitbucket.org/petsc/pkg-sowing.git/>': The requested URL returned error: 503
>> Unable to download package SOWING from: git://https://bitbucket.org/petsc/pkg-sowing.git <https://bitbucket.org/petsc/pkg-sowing.git>
>> * If URL specified manually - perhaps there is a typo?
>> * If your network is disconnected - please reconnect and rerun ./configure
>> * Or perhaps you have a firewall blocking the download
>> * You can run with --with-packages-download-dir=/adirectory and ./configure will instruct you what packages to download manually
>> * or you can download the above URL manually, to /yourselectedlocation
>> and use the configure option:
>> --download-sowing=/yourselectedlocation
>> Unable to clone sowing
>> Could not execute "['git clone https://bitbucket.org/petsc/pkg-sowing.git <https://bitbucket.org/petsc/pkg-sowing.git> /global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing']":
>> Cloning into '/global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing'...
>> fatal: unable to access 'https://bitbucket.org/petsc/pkg-sowing.git/ <https://bitbucket.org/petsc/pkg-sowing.git/>': The requested URL returned error: 503
>> Unable to download package SOWING from: git://https://bitbucket.org/petsc/pkg-sowing.git <https://bitbucket.org/petsc/pkg-sowing.git>
>> * If URL specified manually - perhaps there is a typo?
>> * If your network is disconnected - please reconnect and rerun ./configure
>> * Or perhaps you have a firewall blocking the download
>> * You can run with --with-packages-download-dir=/adirectory and ./configure will instruct you what packages to download manually
>> * or you can download the above URL manually, to /yourselectedlocation
>> and use the configure option:
>> --download-sowing=/yourselectedlocation
>> file could not be opened successfully
>> Downloaded package SOWING from: https://bitbucket.org/petsc/pkg-sowing/get/v1.1.26-p1.tar.gz <https://bitbucket.org/petsc/pkg-sowing/get/v1.1.26-p1.tar.gz> is not a tarball.
>> [or installed python cannot process compressed files]
>> * If you are behind a firewall - please fix your proxy and rerun ./configure
>> For example at LANL you may need to set the environmental variable http_proxy (or HTTP_PROXY?) to http://proxyout.lanl.gov <http://proxyout.lanl.gov/>
>> * You can run with --with-packages-download-dir=/adirectory and ./configure will instruct you what packages to download manually
>> * or you can download the above URL manually, to /yourselectedlocation/v1.1.26-p1.tar.gz
>> and use the configure option:
>> --download-sowing=/yourselectedlocation/v1.1.26-p1.tar.gz
>> *******************************************************************************
>> <configure.log>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210515/a7c74037/attachment-0001.html>
More information about the petsc-users
mailing list