<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
Hi all, we are trying to compile PETSc in Frontier using the structured matrix hierarchical solver strumpack, which uses GPU and might be a good candidate for our Poisson discretization. The list of libs I used for PETSc in this case is: $. /configure
</div>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</div>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 16px 0px 16px 0px; direction: ltr" ><tr><td>
<table border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%; border-radius:4px; border-top:4px solid #90a4ae;background-color:#D0D8DC;"><tr><td valign="top">
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px">
<tr><td style="color:#000000; font-family: 'Arial', sans-serif; font-weight:bold; font-size:14px; direction: ltr">
This Message Is From an External Sender
</td></tr>
<tr><td style="color:#000000; font-weight:normal; font-family: 'Arial', sans-serif; font-size:12px; direction: ltr">
This message came from outside your organization.
</td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<![if !((ie)|(mso))]>
<div dir="ltr" id="pfptBannermzm7v80" style="all: revert !important; display:block !important; text-align: left !important; margin:16px 0px 16px 0px !important; padding:8px 16px 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; background-color: #D0D8DC !important; background-color: #D0D8DC; border-top: 4px solid #90a4ae !important; border-top: 4px solid #90a4ae;">
<div id="pfptBannermzm7v80" style="all: unset !important; float:left !important; display:block !important; margin: 0px 0px 1px 0px !important; max-width: 600px !important;">
<div id="pfptBannermzm7v80" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-weight:bold !important; font-weight:bold; font-size:14px !important; line-height:18px !important; line-height:18px">
This Message Is From an External Sender
</div>
<div id="pfptBannermzm7v80" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-weight:normal; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-size:12px !important; line-height:18px !important; line-height:18px; margin-top:2px !important;">
This message came from outside your organization.
</div>
</div>
<div style="clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important; height: 0px"> </div>
</div>
<![endif]>
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</div>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<html>
<head><!-- BaNnErBlUrFlE-HeAdEr-start -->
<style>
#pfptBannermzm7v80 { all: revert !important; display: block !important;
visibility: visible !important; opacity: 1 !important;
background-color: #D0D8DC !important;
max-width: none !important; max-height: none !important }
.pfptPrimaryButtonmzm7v80:hover, .pfptPrimaryButtonmzm7v80:focus {
background-color: #b4c1c7 !important; }
.pfptPrimaryButtonmzm7v80:active {
background-color: #90a4ae !important; }
</style>
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
Hi all, we are trying to compile PETSc in Frontier using the structured matrix hierarchical solver strumpack, which uses GPU and might be a good candidate for our Poisson discretization.</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
The list of libs I used for PETSc in this case is:</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
$./configure COPTFLAGS="-O3" CXXOPTFLAGS="-O3" FOPTFLAGS="-O3" FCOPTFLAGS="-O3" HIPOPTFLAGS="-O3 --offload-arch=gfx90a" --with-debugging=0 --with-cc=cc --with-cxx=CC --with-fc=ftn --with-hip --with-hip-arch=gfx908 --with-hipc=hipcc --LIBS="-L${MPICH_DIR}/lib
-lmpi ${CRAY_XPMEM_POST_LINK_OPTS} -lxpmem ${PE_MPICH_GTL_DIR_amd_gfx90a} ${PE_MPICH_GTL_LIBS_amd_gfx90a}" --download-kokkos --download-kokkos-kernels --download-suitesparse --download-hypre --download-superlu_dist
<b>--download-strumpack --download-metis --download-slate --download-magma --download-parmetis --download-ptscotch --download-zfp --download-butterflypack --with-openmp-dir=/opt/cray/pe/gcc/12.2.0/snos --download-scalapack</b> --download-cmake --force</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
I'm getting an error at configure time:</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
...</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
Trying to download <a href="https://urldefense.us/v3/__https://github.com/liuyangzhuan/ButterflyPACK__;!!G_uCfscf7eWS!cW5KuKKMbmDa8n59SJGArXdSxVT_-V0qH3vt1-LE-CAr4wShO2pTXN3GvI0bVCwUh6RWH6z2URqBczHnVEyXXKAJ2LN7JnSj$">https://github.com/liuyangzhuan/ButterflyPACK</a> for BUTTERFLYPACK</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
Configuring BUTTERFLYPACK with CMake; this may take several minutes</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
Compiling and installing BUTTERFLYPACK; this may take several minutes</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
Trying to download <a href="https://urldefense.us/v3/__https://github.com/pghysels/STRUMPACK__;!!G_uCfscf7eWS!cW5KuKKMbmDa8n59SJGArXdSxVT_-V0qH3vt1-LE-CAr4wShO2pTXN3GvI0bVCwUh6RWH6z2URqBczHnVEyXXKAJ2FeUr7dA$">https://github.com/pghysels/STRUMPACK</a> for STRUMPACK</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
Configuring STRUMPACK with CMake; this may take several minutes</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
Compiling and installing STRUMPACK; this may take several minutes</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
=============================================================================================</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
<br>
</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
*********************************************************************************************</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
---------------------------------------------------------------------------------------------</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
Error running make on STRUMPACK</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
*********************************************************************************************</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
Looking in the configure.log file I see error like this related to strumpack compilation:</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
/<span style="font-family: "Courier New", monospace;">opt/cray/pe/craype/2.7.19/bin/CC -D__HIP_PLATFORM_AMD__=1 -D__HIP_PLATFORM_HCC__=1 -Dstrumpack_EXPORTS -I/autofs/nccs-svm1_home1/vanellam/Software/petsc/arch-linux-frontier-opt-gcc-direct/externalpackages/git.strumpack/src
-I/autofs/nccs-svm1_home1/vanellam/Software/petsc/arch-linux-frontier-opt-gcc-direct/externalpackages/git.strumpack/petsc-build -isystem /autofs/nccs-svm1_home1/vanellam/Software/petsc/arch-linux-frontier-opt-gcc-direct/include -isystem /opt/rocm-5.4.0/include
-isystem /opt/rocm-5.4.0/hip/include -isystem /opt/rocm-5.4.0/llvm/lib/clang/15.0.0/.. -Wno-lto-type-mismatch -Wno-psabi -O3 -fPIC -fopenmp -Wno-lto-type-mismatch -Wno-psabi -O3 -fPIC -fopenmp -fPIC -Wall -Wno-overloaded-virtual -fopenmp -x hip --offload-arch=gfx900
--offload-arch=gfx906 --offload-arch=gfx908 --offload-arch=gfx90a --offload-arch=gfx1030 -MD -MT CMakeFiles/strumpack.dir/src/clustering/NeighborSearch.cpp.o -MF CMakeFiles/strumpack.dir/src/clustering/NeighborSearch.cpp.o.d -o CMakeFiles/strumpack.dir/src/clustering/NeighborSearch.cpp.o
-c /autofs/nccs-svm1_home1/vanellam/Software/petsc/arch-linux-frontier-opt-gcc-direct/externalpackages/git.strumpack/src/clustering/NeighborSearch.cpp</span></div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
gmake[2]: Leaving directory '/autofs/nccs-svm1_home1/vanellam/Software/petsc/arch-linux-frontier-opt-gcc-direct/externalpackages/git.strumpack/petsc-build'</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
gmake[1]: Leaving directory '/autofs/nccs-svm1_home1/vanellam/Software/petsc/arch-linux-frontier-opt-gcc-direct/externalpackages/git.strumpack/petsc-build'</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
stdout:</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
g++: error: unrecognized command-line option '--offload-arch=gfx900'</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
g++: error: unrecognized command-line option '--offload-arch=gfx900'</div>
<div style="font-family: "Courier New", monospace; font-size: 12pt; color: black;">
g++: error: unrecognized command-line option '--offload-arch=gfx900'</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
It seems the configure is picking up CC (g++) instead of hipcc as the compiler for strumpack.</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
I wonder if anyone has come across this issue or has any suggestions?</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
Thanks!</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
Marcos</div>
</body>
</html>