------------------------------------------------------------------ PETSc Performance Summary: ------------------------------------------------------------------ /wclustre/accelsim/sajid/packages/synergia2/build/examples/fodo_cxx/fodo_cxx on a named wcgpu05.fnal.gov with 1 processor, by sasyed Wed Oct 5 12:02:12 2022 Using Petsc Development GIT revision: 9c433d7f88f2542481f0a4ae019dc41737212a1b GIT Date: 2022-10-03 15:36:49 -0500 Max Max/Min Avg Total Time (sec): 4.262e+01 1.000 4.262e+01 Objects: 5.800e+01 1.000 5.800e+01 Flops: 1.408e+07 1.000 1.408e+07 1.408e+07 Flops/sec: 3.303e+05 1.000 3.303e+05 3.303e+05 MPI Msg Count: 0.000e+00 0.000 0.000e+00 0.000e+00 MPI Msg Len (bytes): 7.718e+03 1.000 0.000e+00 7.718e+03 MPI Reductions: 2.000e+00 1.000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flop ------ --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total Count %Total Avg %Total Count %Total 0: Main Stage: 4.2623e+01 100.0% 1.4079e+07 100.0% 0.000e+00 0.0% 0.000e+00 100.0% 2.000e+00 100.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flop: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent AvgLen: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flop in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors) GPU Mflop/s: 10e-6 * (sum of flop on GPU over all processors)/(max GPU time over all processors) CpuToGpu Count: total number of CPU to GPU copies per processor CpuToGpu Size (Mbytes): 10e-6 * (total size of CPU to GPU copies per processor) GpuToCpu Count: total number of GPU to CPU copies per processor GpuToCpu Size (Mbytes): 10e-6 * (total size of GPU to CPU copies per processor) GPU %F: percent flops on GPU in this event ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flop --- Global --- --- Stage ---- Total GPU - CpuToGpu - - GpuToCpu - GPU Max Ratio Max Ratio Max Ratio Mess AvgLen Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s Mflop/s Count Size Count Size %F --------------------------------------------------------------------------------------------------------------------------------------------------------------- --- Event Stage 0: Main Stage PetscBarrier 2 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 BuildTwoSided 2 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 cuBLAS Init 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 5 0 0 0 0 5 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 DCtxCreate 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 DCtxSetUp 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 DCtxSetDevice 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 VecScale 400 1.0 nan nan 1.31e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 93 0 0 0 0 93 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 100 VecCopy 400 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 VecSet 400 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 VecScatterBegin 800 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 400 1.05e+02 0 0.00e+00 0 VecScatterEnd 800 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 VecCUDACopyTo 400 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 400 1.05e+02 0 0.00e+00 0 VecCUDACopyFrom 400 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 400 1.05e+02 0 MatSolve 400 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 19 0 0 0 0 19 0 0 0 0 -nan -nan 0 0.00e+00 400 1.05e+02 0 MatLUFactor 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 3 0 0 0 0 3 0 0 0 0 -nan -nan 0 0.00e+00 1 1.56e+00 0 MatLUFactorNum 1 1.0 nan nan 9.72e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 7 0 0 0 0 7 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 100 MatILUFactorSym 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 MatAssemblyBegin 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 MatAssemblyEnd 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 MatGetRowIJ 2 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 MatGetOrdering 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 MatCUSPARSCopyTo 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 1 2.47e+00 0 0.00e+00 0 MatCUSPARSCopyFr 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 1 1.56e+00 0 MatSetPreallCOO 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 1 2.47e+00 0 0.00e+00 0 MatSetValuesCOO 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 SFSetGraph 4 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 SFSetUp 2 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 SFPack 800 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 SFUnpack 800 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 KSPSetUp 1 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 -nan -nan 0 0.00e+00 0 0.00e+00 0 KSPSolve 400 1.0 8.2571e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 19 0 0 0 0 19 0 0 0 0 0 -nan 0 0.00e+00 400 1.05e+02 0 PCSetUp 2 1.0 nan nan 9.72e+05 1.0 0.0e+00 0.0e+00 0.0e+00 3 7 0 0 0 3 7 0 0 0 -nan -nan 0 0.00e+00 1 1.56e+00 100 PCApply 400 1.0 nan nan 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 19 0 0 0 0 19 0 0 0 0 -nan -nan 0 0.00e+00 400 1.05e+02 0 --- Event Stage 1: Unknown --------------------------------------------------------------------------------------------------------------------------------------------------------------- Object Type Creations Destructions. Reports information only for process 0. --- Event Stage 0: Main Stage Container 3 0 PetscDeviceContext 1 0 Vector 20 20 Distributed Mesh 2 2 Matrix 3 3 Index Set 12 12 IS L to G Mapping 2 2 Star Forest Graph 8 8 Discrete System 2 2 Weak Form 2 2 Krylov Solver 1 1 Preconditioner 1 1 Viewer 1 0 --- Event Stage 1: Unknown ======================================================================================================================== Average time to get PetscTime(): 2.38e-08 #No PETSc Option Table entries Compiled without FORTRAN kernels Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4 Configure options: --prefix=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/petsc-main-nqr2wd2jteywlase7lpcxi2yk6x4u26h --with-ssl=0 --download-c2html=0 --download-sowing=0 --download-hwloc=0 --with-cc=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openmpi-4.1.4-yasdsobvx3cy3qoa3xeczttt247wrnlb/bin/mpicc --with-cxx=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openmpi-4.1.4-yasdsobvx3cy3qoa3xeczttt247wrnlb/bin/mpic++ --with-fc=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openmpi-4.1.4-yasdsobvx3cy3qoa3xeczttt247wrnlb/bin/mpif90 --with-precision=double --with-scalar-type=real --with-shared-libraries=1 --with-debugging=0 --with-openmp=0 --with-64-bit-indices=0 --with-blaslapack-lib=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openblas-0.3.20-kj6iljgbzsw3yy45jmclolcl6kfxlwna/lib/libopenblas.so --with-batch=1 --with-x=0 --with-clanguage=C --with-cuda=1 --with-cuda-dir=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/cuda-11.7.0-ibtl5btimvgk3rehaweoeto4brxp53qt --with-hip=0 --with-metis=1 --with-metis-include=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/metis-5.1.0-47t544fqfubgzukq7zpkhhn2reqevxw3/include --with-metis-lib=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/metis-5.1.0-47t544fqfubgzukq7zpkhhn2reqevxw3/lib/libmetis.so --with-hypre=1 --with-hypre-include=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hypre-develop-nzomfnuxjsgb7ldfzlo7emjmff2lnuam/include --with-hypre-lib=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hypre-develop-nzomfnuxjsgb7ldfzlo7emjmff2lnuam/lib/libHYPRE.so --with-parmetis=1 --with-parmetis-include=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/parmetis-4.0.3-py7o626nunn4vhnjvpxmbv2j3valyhxg/include --with-parmetis-lib=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/parmetis-4.0.3-py7o626nunn4vhnjvpxmbv2j3valyhxg/lib/libparmetis.so --with-kokkos=0 --with-kokkos-kernels=0 --with-superlu_dist=1 --with-superlu_dist-include=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/superlu-dist-develop-3y547oztppl7hwkh7go46lg34wwfrytl/include --with-superlu_dist-lib=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/superlu-dist-develop-3y547oztppl7hwkh7go46lg34wwfrytl/lib/libsuperlu_dist.so --with-ptscotch=0 --with-suitesparse=0 --with-hdf5=1 --with-hdf5-include=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hdf5-1.12.1-qe2ggdtrbgnayp4elx2p72ror3bnrx27/include --with-hdf5-lib="/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hdf5-1.12.1-qe2ggdtrbgnayp4elx2p72ror3bnrx27/lib/libhdf5_hl.so /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hdf5-1.12.1-qe2ggdtrbgnayp4elx2p72ror3bnrx27/lib/libhdf5.so" --with-zlib=1 --with-zlib-include=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/zlib-1.2.12-fusk4kklde3cd6tcoaqk5ebo4ddyezas/include --with-zlib-lib=/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/zlib-1.2.12-fusk4kklde3cd6tcoaqk5ebo4ddyezas/lib/libz.so --with-mumps=0 --with-trilinos=0 --with-fftw=0 --with-valgrind=0 --with-gmp=0 --with-libpng=0 --with-giflib=0 --with-mpfr=0 --with-netcdf=0 --with-pnetcdf=0 --with-moab=0 --with-random123=0 --with-exodusii=0 --with-cgns=0 --with-memkind=0 --with-p4est=0 --with-saws=0 --with-yaml=0 --with-hwloc=0 --with-libjpeg=0 --with-scalapack=0 --with-strumpack=0 --with-mmg=0 --with-parmmg=0 --with-tetgen=0 --with-cuda-arch=70 ----------------------------------------- Libraries compiled on 2022-10-05 00:24:42 on wcgpu04.fnal.gov Machine characteristics: Linux-3.10.0-1160.71.1.el7.x86_64-x86_64-with-glibc2.17 Using PETSc directory: /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/petsc-main-nqr2wd2jteywlase7lpcxi2yk6x4u26h Using PETSc arch: ----------------------------------------- Using C compiler: /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openmpi-4.1.4-yasdsobvx3cy3qoa3xeczttt247wrnlb/bin/mpicc -fPIC -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-stringop-overflow -fstack-protector -fvisibility=hidden -g -O Using Fortran compiler: /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openmpi-4.1.4-yasdsobvx3cy3qoa3xeczttt247wrnlb/bin/mpif90 -fPIC -Wall -ffree-line-length-none -ffree-line-length-0 -Wno-lto-type-mismatch -Wno-unused-dummy-argument -g -O ----------------------------------------- Using include paths: -I/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/petsc-main-nqr2wd2jteywlase7lpcxi2yk6x4u26h/include -I/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hypre-develop-nzomfnuxjsgb7ldfzlo7emjmff2lnuam/include -I/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/superlu-dist-develop-3y547oztppl7hwkh7go46lg34wwfrytl/include -I/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hdf5-1.12.1-qe2ggdtrbgnayp4elx2p72ror3bnrx27/include -I/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/parmetis-4.0.3-py7o626nunn4vhnjvpxmbv2j3valyhxg/include -I/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/metis-5.1.0-47t544fqfubgzukq7zpkhhn2reqevxw3/include -I/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/zlib-1.2.12-fusk4kklde3cd6tcoaqk5ebo4ddyezas/include -I/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/cuda-11.7.0-ibtl5btimvgk3rehaweoeto4brxp53qt/include ----------------------------------------- Using C linker: /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openmpi-4.1.4-yasdsobvx3cy3qoa3xeczttt247wrnlb/bin/mpicc Using Fortran linker: /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openmpi-4.1.4-yasdsobvx3cy3qoa3xeczttt247wrnlb/bin/mpif90 Using libraries: -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/petsc-main-nqr2wd2jteywlase7lpcxi2yk6x4u26h/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/petsc-main-nqr2wd2jteywlase7lpcxi2yk6x4u26h/lib -lpetsc -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hypre-develop-nzomfnuxjsgb7ldfzlo7emjmff2lnuam/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hypre-develop-nzomfnuxjsgb7ldfzlo7emjmff2lnuam/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/superlu-dist-develop-3y547oztppl7hwkh7go46lg34wwfrytl/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/superlu-dist-develop-3y547oztppl7hwkh7go46lg34wwfrytl/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openblas-0.3.20-kj6iljgbzsw3yy45jmclolcl6kfxlwna/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openblas-0.3.20-kj6iljgbzsw3yy45jmclolcl6kfxlwna/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hdf5-1.12.1-qe2ggdtrbgnayp4elx2p72ror3bnrx27/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hdf5-1.12.1-qe2ggdtrbgnayp4elx2p72ror3bnrx27/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/parmetis-4.0.3-py7o626nunn4vhnjvpxmbv2j3valyhxg/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/parmetis-4.0.3-py7o626nunn4vhnjvpxmbv2j3valyhxg/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/metis-5.1.0-47t544fqfubgzukq7zpkhhn2reqevxw3/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/metis-5.1.0-47t544fqfubgzukq7zpkhhn2reqevxw3/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/zlib-1.2.12-fusk4kklde3cd6tcoaqk5ebo4ddyezas/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/zlib-1.2.12-fusk4kklde3cd6tcoaqk5ebo4ddyezas/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/cuda-11.7.0-ibtl5btimvgk3rehaweoeto4brxp53qt/lib64 -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/cuda-11.7.0-ibtl5btimvgk3rehaweoeto4brxp53qt/lib64 -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/cuda-11.7.0-ibtl5btimvgk3rehaweoeto4brxp53qt/lib64/stubs -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openmpi-4.1.4-yasdsobvx3cy3qoa3xeczttt247wrnlb/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/openmpi-4.1.4-yasdsobvx3cy3qoa3xeczttt247wrnlb/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hwloc-2.8.0-xw52q7mge746x4d6gv6mp3fkmn4lefnf/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/hwloc-2.8.0-xw52q7mge746x4d6gv6mp3fkmn4lefnf/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/ucx-1.13.1-zzxyxwmvcmtvk7hrizwhki4l6v6qz7y2/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/ucx-1.13.1-zzxyxwmvcmtvk7hrizwhki4l6v6qz7y2/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/libevent-2.1.12-5t7kr2s7soh6yna6r3miudfpqacktrgx/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/libevent-2.1.12-5t7kr2s7soh6yna6r3miudfpqacktrgx/lib -Wl,-rpath,/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/pmix-4.1.2-u3idu6rhzskyxdek45z2hpa4wequcsdx/lib -L/wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-11.3.0/pmix-4.1.2-u3idu6rhzskyxdek45z2hpa4wequcsdx/lib -Wl,-rpath,/srv/software/gnu11/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0 -L/srv/software/gnu11/11.3.0/lib/gcc/x86_64-pc-linux-gnu/11.3.0 -Wl,-rpath,/srv/software/gnu11/11.3.0/lib64 -L/srv/software/gnu11/11.3.0/lib64 -Wl,-rpath,/srv/software/gnu11/11.3.0/lib -L/srv/software/gnu11/11.3.0/lib -lHYPRE -lsuperlu_dist -lopenblas -lhdf5_hl -lhdf5 -lparmetis -lmetis -lm -lz -lcudart -lnvToolsExt -lcufft -lcublas -lcusparse -lcusolver -lcurand -lcuda -lstdc++ -ldl -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lpthread -lquadmath -lstdc++ -ldl -----------------------------------------