<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
Hi Marcos,
<div><br>
</div>
<div>Yes, I compiled with clang instead of icc (no particular reason for this; I tend to use gcc/clang). I use mpich4.1.1, which I first built with clang and ifort: </div>
<div><br>
</div>
<div>
<p style="margin: 0px; font-stretch: normal; line-height: normal;">FC=ifort</p>
<p style="margin: 0px; font-stretch: normal; line-height: normal;">./configure --prefix=/usr/local/mpich4 --enable-two-level-namespace</p>
<p style="margin: 0px; font-stretch: normal; line-height: normal;"><br>
</p>
</div>
<div>Samar</div>
<div>
<div><br>
<blockquote type="cite">
<div>On May 15, 2023, at 6:07 PM, Vanella, Marcos (Fed) <marcos.vanella@nist.gov> wrote:</div>
<br class="Apple-interchange-newline">
<div>
<div class="elementToProof" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
Hi Samar, what MPI library do you use? Did you compile it with clang instead of icc?</div>
<div class="elementToProof" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
Thanks,</div>
<div class="elementToProof" style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
Marcos<br>
</div>
<div id="appendonsend" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
</div>
<hr tabindex="-1" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline-block; width: 562.515625px;">
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"></span>
<div id="divRplyFwdMsg" dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span class="Apple-converted-space"> </span>Samar Khatiwala <samar.khatiwala@earth.ox.ac.uk><br>
<b>Sent:</b><span class="Apple-converted-space"> </span>Monday, May 15, 2023 1:05 PM<br>
<b>To:</b><span class="Apple-converted-space"> </span>Matthew Knepley <knepley@gmail.com><br>
<b>Cc:</b><span class="Apple-converted-space"> </span>Vanella, Marcos (Fed) <marcos.vanella@nist.gov>; petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov><br>
<b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Compiling PETSC with Intel OneAPI compilers and OpenMPI</font>
<div> </div>
</div>
<div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; line-break: after-white-space;">
Hi, for what it’s worth, clang + ifort from OneAPI 2023 update 1 works fine for me on both Intel and M2 Macs. So it might just be a matter of upgrading.
<div>
<div><br>
</div>
<div>Samar</div>
<div><br>
<blockquote type="cite">
<div>On May 15, 2023, at 5:53 PM, Matthew Knepley <knepley@gmail.com> wrote:</div>
<br class="x_Apple-interchange-newline">
<div>
<div dir="ltr">Send us
<div><br>
</div>
<div> $PETSC_ARCH/include/petscconf.h</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Mon, May 15, 2023 at 12:49 PM Vanella, Marcos (Fed) <<a href="mailto:marcos.vanella@nist.gov">marcos.vanella@nist.gov</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">
<div class="x_msg-7961065678739807522">
<div dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
Hi Matt, I configured the lib like this:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<span style="font-family: "Courier New", monospace;">$ ./configure --with-blaslapack-dir=/opt/intel/oneapi/mkl/2022.2.1 --with-debugging=0 --with-shared-libraries=0 --download-make</span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
and compiled. I still get some check segfault error. See below:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<span style="font-family: "Courier New", monospace;">$ make PETSC_DIR=/Users/mnv/Documents/Software/petsc-3.19.1 PETSC_ARCH=arch-darwin-c-opt check</span>
<div><span style="font-family: "Courier New", monospace;">Running check examples to verify correct installation</span></div>
<div><span style="font-family: "Courier New", monospace;">Using PETSC_DIR=/Users/mnv/Documents/Software/petsc-3.19.1 and PETSC_ARCH=arch-darwin-c-opt</span></div>
<div><span style="font-family: "Courier New", monospace;">*******************Error detected during compile or link!*******************</span></div>
<div><span style="font-family: "Courier New", monospace;">See<span class="Apple-converted-space"> </span><a href="https://petsc.org/release/faq/" originalsrc="https://petsc.org/release/faq/" shash="waTQbTdPO2QrsfHBOoRMTqS5Aqfc1L4Ap1vdh4QI2wlfL5giC0jkTtzLf1s/FRpPrsGkF6Am+dnnSNg58juz3kzgbgLESB0h71h4I3tQtWFbAiiLHOARFe+Qh/nhVeJire7Zv//z6ie0XBfN8LuDOY3cxl+icmFZK+loQp+UTLQ=" target="_blank">https://petsc.org/release/faq/</a></span></div>
<div><span style="font-family: "Courier New", monospace;">/Users/mnv/Documents/Software/petsc-3.19.1/src/snes/tutorials ex19</span></div>
<div><span style="font-family: "Courier New", monospace;">*********************************************************************************</span></div>
<div><span style="font-family: "Courier New", monospace;">mpicc -Wl,-bind_at_load -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first -Wl,-no_compact_unwind -fPIC -wd1572 -Wno-unknown-pragmas -g
-O3 -I/Users/mnv/Documents/Software/petsc-3.19.1/include -I/Users/mnv/Documents/Software/petsc-3.19.1/arch-darwin-c-opt/include -I/opt/X11/include -std=c99 ex19.c -L/Users/mnv/Documents/Software/petsc-3.19.1/arch-darwin-c-opt/lib -Wl,-rpath,/opt/intel/oneapi/mkl/2022.2.1/lib
-L/opt/intel/oneapi/mkl/2022.2.1/lib -Wl,-rpath,/opt/X11/lib -L/opt/X11/lib -L/opt/openmpi414_oneapi22u3/lib -Wl,-rpath,/opt/intel/oneapi/compiler/2022.2.1/mac/compiler/lib -L/opt/intel/oneapi/tbb/2021.7.1/lib -L/opt/intel/oneapi/ippcp/2021.6.2/lib -L/opt/intel/oneapi/ipp/2021.6.2/lib
-L/opt/intel/oneapi/dnnl/2022.2.1/cpu_iomp/lib -L/opt/intel/oneapi/dal/2021.7.1/lib -L/opt/intel/oneapi/compiler/2022.2.1/mac/compiler/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -Wl,-rpath,/opt/intel/oneapi/compiler/2022.2.1/mac/bin/intel64/../../compiler/lib
-L/opt/intel/oneapi/compiler/2022.2.1/mac/bin/intel64/../../compiler/lib -Wl,-rpath,/Library/Developer/CommandLineTools/usr/lib/clang/14.0.3/lib/darwin -L/Library/Developer/CommandLineTools/usr/lib/clang/14.0.3/lib/darwin -lpetsc -lmkl_intel_lp64 -lmkl_core
-lmkl_sequential -lpthread -lX11 -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -limf -lm -lz -lifport -lifcoremt -lsvml -lipgo -lirc -lpthread -lclang_rt.osx -lmpi -lopen-rte -lopen-pal -limf -lm -lz -lsvml -lirng -lc++ -lipgo
-ldecimal -lirc -lclang_rt.osx -lmpi -lopen-rte -lopen-pal -limf -lm -lz -lsvml -lirng -lc++ -lipgo -ldecimal -lirc -lclang_rt.osx -o ex19</span></div>
<div><span style="font-family: "Courier New", monospace;">icc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended
compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.</span></div>
<div><span style="font-family: "Courier New", monospace;">In file included from /Users/mnv/Documents/Software/petsc-3.19.1/include/petscsys.h(44),</span></div>
<div><span style="font-family: "Courier New", monospace;"> from /Users/mnv/Documents/Software/petsc-3.19.1/include/petscvec.h(9),</span></div>
<div><span style="font-family: "Courier New", monospace;"> from /Users/mnv/Documents/Software/petsc-3.19.1/include/petscmat.h(7),</span></div>
<div><span style="font-family: "Courier New", monospace;"> from /Users/mnv/Documents/Software/petsc-3.19.1/include/petscpc.h(7),</span></div>
<div><span style="font-family: "Courier New", monospace;"> from /Users/mnv/Documents/Software/petsc-3.19.1/include/petscksp.h(7),</span></div>
<div><span style="font-family: "Courier New", monospace;"> from /Users/mnv/Documents/Software/petsc-3.19.1/include/petscsnes.h(7),</span></div>
<div><span style="font-family: "Courier New", monospace;"> from ex19.c(68):</span></div>
<div><span style="font-family: "Courier New", monospace;">/Users/mnv/Documents/Software/petsc-3.19.1/include/petscsystypes.h(68): warning #2621: attribute "warn_unused_result" does not apply here</span></div>
<div><span style="font-family: "Courier New", monospace;"> PETSC_ERROR_CODE_TYPEDEF enum PETSC_ERROR_CODE_NODISCARD {</span></div>
<div><span style="font-family: "Courier New", monospace;"> ^</span></div>
<div><br>
</div>
<div><span style="font-family: "Courier New", monospace;">Possible error running C/C++ src/snes/tutorials/ex19 with 1 MPI process</span></div>
<div><span style="font-family: "Courier New", monospace;">See<span class="Apple-converted-space"> </span><a href="https://petsc.org/release/faq/" originalsrc="https://petsc.org/release/faq/" shash="waTQbTdPO2QrsfHBOoRMTqS5Aqfc1L4Ap1vdh4QI2wlfL5giC0jkTtzLf1s/FRpPrsGkF6Am+dnnSNg58juz3kzgbgLESB0h71h4I3tQtWFbAiiLHOARFe+Qh/nhVeJire7Zv//z6ie0XBfN8LuDOY3cxl+icmFZK+loQp+UTLQ=" target="_blank">https://petsc.org/release/faq/</a></span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37807] *** Process received signal ***</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37807] Signal: Segmentation fault: 11 (11)</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37807] Signal code: Address not mapped (1)</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37807] Failing at address: 0x7f</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37807] *** End of error message ***</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">Primary job terminated normally, but 1 process returned</span></div>
<div><span style="font-family: "Courier New", monospace;">a non-zero exit code. Per user-direction, the job has been aborted.</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">mpiexec noticed that process rank 0 with PID 0 on node excess exited on signal 11 (Segmentation fault: 11).</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">Possible error running C/C++ src/snes/tutorials/ex19 with 2 MPI processes</span></div>
<div><span style="font-family: "Courier New", monospace;">See<span class="Apple-converted-space"> </span><a href="https://petsc.org/release/faq/" originalsrc="https://petsc.org/release/faq/" shash="waTQbTdPO2QrsfHBOoRMTqS5Aqfc1L4Ap1vdh4QI2wlfL5giC0jkTtzLf1s/FRpPrsGkF6Am+dnnSNg58juz3kzgbgLESB0h71h4I3tQtWFbAiiLHOARFe+Qh/nhVeJire7Zv//z6ie0XBfN8LuDOY3cxl+icmFZK+loQp+UTLQ=" target="_blank">https://petsc.org/release/faq/</a></span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37831] *** Process received signal ***</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37831] Signal: Segmentation fault: 11 (11)</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37831] Signal code: Address not mapped (1)</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37831] Failing at address: 0x7f</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37831] *** End of error message ***</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37832] *** Process received signal ***</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37832] Signal: Segmentation fault: 11 (11)</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37832] Signal code: Address not mapped (1)</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37832] Failing at address: 0x7f</span></div>
<div><span style="font-family: "Courier New", monospace;">[excess:37832] *** End of error message ***</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">Primary job terminated normally, but 1 process returned</span></div>
<div><span style="font-family: "Courier New", monospace;">a non-zero exit code. Per user-direction, the job has been aborted.</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">mpiexec noticed that process rank 1 with PID 0 on node excess exited on signal 11 (Segmentation fault: 11).</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">Possible error running Fortran example src/snes/tutorials/ex5f with 1 MPI process</span></div>
<div><span style="font-family: "Courier New", monospace;">See<span class="Apple-converted-space"> </span><a href="https://petsc.org/release/faq/" originalsrc="https://petsc.org/release/faq/" shash="waTQbTdPO2QrsfHBOoRMTqS5Aqfc1L4Ap1vdh4QI2wlfL5giC0jkTtzLf1s/FRpPrsGkF6Am+dnnSNg58juz3kzgbgLESB0h71h4I3tQtWFbAiiLHOARFe+Qh/nhVeJire7Zv//z6ie0XBfN8LuDOY3cxl+icmFZK+loQp+UTLQ=" target="_blank">https://petsc.org/release/faq/</a></span></div>
<div><span style="font-family: "Courier New", monospace;">forrtl: severe (174): SIGSEGV, segmentation fault occurred</span></div>
<div><span style="font-family: "Courier New", monospace;">Image PC Routine Line Source </span></div>
<div><span style="font-family: "Courier New", monospace;">libifcoremt.dylib 000000010B7F7FE4 for__signal_handl Unknown Unknown</span></div>
<div><span style="font-family: "Courier New", monospace;">libsystem_platfor 00007FF8024C25ED _sigtramp Unknown Unknown</span></div>
<div><span style="font-family: "Courier New", monospace;">ex5f 00000001087AFA38 PetscGetArchType Unknown Unknown</span></div>
<div><span style="font-family: "Courier New", monospace;">ex5f 000000010887913B PetscErrorPrintfI Unknown Unknown</span></div>
<div><span style="font-family: "Courier New", monospace;">ex5f 000000010878D227 PetscInitialize_C Unknown Unknown</span></div>
<div><span style="font-family: "Courier New", monospace;">ex5f 000000010879D289 petscinitializef_ Unknown Unknown</span></div>
<div><span style="font-family: "Courier New", monospace;">ex5f 0000000108713C09 petscsys_mp_petsc Unknown Unknown</span></div>
<div><span style="font-family: "Courier New", monospace;">ex5f 0000000108710B5D MAIN__ Unknown Unknown</span></div>
<div><span style="font-family: "Courier New", monospace;">ex5f 0000000108710AEE main Unknown Unknown</span></div>
<div><span style="font-family: "Courier New", monospace;">dyld 00007FF80213B41F start Unknown Unknown</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">Primary job terminated normally, but 1 process returned</span></div>
<div><span style="font-family: "Courier New", monospace;">a non-zero exit code. Per user-direction, the job has been aborted.</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">mpiexec detected that one or more processes exited with non-zero status, thus causing</span></div>
<div><span style="font-family: "Courier New", monospace;">the job to be terminated. The first process to do so was:</span></div>
<div><br>
</div>
<div><span style="font-family: "Courier New", monospace;"> Process name: [[48108,1],0]</span></div>
<div><span style="font-family: "Courier New", monospace;"> Exit code: 174</span></div>
<div><span style="font-family: "Courier New", monospace;">--------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;">Completed test examples</span></div>
<div><span style="font-family: "Courier New", monospace;">Error while running make check</span></div>
<div><span style="font-family: "Courier New", monospace;">make[1]: *** [check] Error 1</span></div>
<div><span style="font-family: "Courier New", monospace;">make: *** [check] Error 2</span></div>
<br>
</div>
<div id="x_m_-7961065678739807522appendonsend"></div>
<hr style="display: inline-block; width: 540.671875px;">
<div id="x_m_-7961065678739807522divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span class="Apple-converted-space"> </span>Vanella, Marcos (Fed) <<a href="mailto:marcos.vanella@nist.gov" target="_blank">marcos.vanella@nist.gov</a>><br>
<b>Sent:</b><span class="Apple-converted-space"> </span>Monday, May 15, 2023 12:20 PM<br>
<b>To:</b><span class="Apple-converted-space"> </span>Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Compiling PETSC with Intel OneAPI compilers and OpenMPI</font>
<div> </div>
</div>
<div dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
Thank you Matt I'll try this and let you know.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
Marcos<br>
</div>
<div id="x_m_-7961065678739807522x_appendonsend"></div>
<hr style="display: inline-block; width: 540.671875px;">
<div id="x_m_-7961065678739807522x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span class="Apple-converted-space"> </span>Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Sent:</b><span class="Apple-converted-space"> </span>Monday, May 15, 2023 12:08 PM<br>
<b>To:</b><span class="Apple-converted-space"> </span>Vanella, Marcos (Fed) <<a href="mailto:marcos.vanella@nist.gov" target="_blank">marcos.vanella@nist.gov</a>><br>
<b>Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Compiling PETSC with Intel OneAPI compilers and OpenMPI</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div dir="ltr">On Mon, May 15, 2023 at 11:19 AM Vanella, Marcos (Fed) via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
</div>
<div>
<blockquote style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">
<div>
<div dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
Hello, I'm trying to compile the PETSc library version 3.19.1 with OpenMPI 4.1.4 and the OneAPI 2022 Update 2 Intel Compiler suite on a Mac with OSX Ventura 13.3.1.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
I can compile PETSc in debug mode with this configure and make lines. I can run the PETSC tests, which seem fine.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
When I compile the library in optimized mode, either using -O3 or O1, for example configuring with:</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I hate to yell "compiler bug" when this happens, but it sure seems like one. Can you just use</div>
<div><br>
</div>
<div> --with-debugging=0</div>
<div><br>
</div>
<div>without the custom COPTFLAGS, CXXOPTFLAGS, FOPTFLAGS? If that works, it is almost</div>
<div>certainly a compiler bug. If not, then we can go in the debugger and see what is failing.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">
<div>
<div dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<span style="font-family: "Courier New", monospace;">$ ./configure --prefix=/opt/petsc-oneapi22u3 --with-blaslapack-dir=/opt/intel/oneapi/mkl/2022.2.1 COPTFLAGS='-m64 -O1 -g -diag-disable=10441' CXXOPTFLAGS='-m64 -O1 -g -diag-disable=10441' FOPTFLAGS='-m64
-O1 -g' LDFLAGS='-m64' --with-debugging=0 --with-shared-libraries=0 --download-make</span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
and using mpicc (icc), mpif90 (ifort) from Open MPI, the static lib compiles. Yet, I see right off the bat this segfault error in the first PETSc example:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<span style="font-family: "Courier New", monospace;">$ make PETSC_DIR=/Users/mnv/Documents/Software/petsc-3.19.1 PETSC_ARCH=arch-darwin-c-opt test</span>
<div><span style="font-family: "Courier New", monospace;">/Users/mnv/Documents/Software/petsc-3.19.1/arch-darwin-c-opt/bin/make --no-print-directory -f /Users/mnv/Documents/Software/petsc-3.19.1/gmakefile.test PETSC_ARCH=arch-darwin-c-opt PETSC_DIR=/Users/mnv/Documents/Software/petsc-3.19.1
test</span></div>
<div><span style="font-family: "Courier New", monospace;">/opt/intel/oneapi/intelpython/latest/bin/python3 /Users/mnv/Documents/Software/petsc-3.19.1/config/gmakegentest.py --petsc-dir=/Users/mnv/Documents/Software/petsc-3.19.1 --petsc-arch=arch-darwin-c-opt
--testdir=./arch-darwin-c-opt/tests</span></div>
<div><span style="font-family: "Courier New", monospace;">Using MAKEFLAGS: --no-print-directory -- PETSC_ARCH=arch-darwin-c-opt PETSC_DIR=/Users/mnv/Documents/Software/petsc-3.19.1</span></div>
<div><span style="font-family: "Courier New", monospace;"> CC arch-darwin-c-opt/tests/sys/classes/draw/tests/ex1.o</span></div>
<div><span style="font-family: "Courier New", monospace;">In file included from /Users/mnv/Documents/Software/petsc-3.19.1/include/petscsys.h(44),</span></div>
<div><span style="font-family: "Courier New", monospace;"> from /Users/mnv/Documents/Software/petsc-3.19.1/src/sys/classes/draw/tests/ex1.c(4):</span></div>
<div><span style="font-family: "Courier New", monospace;">/Users/mnv/Documents/Software/petsc-3.19.1/include/petscsystypes.h(68): warning #2621: attribute "warn_unused_result" does not apply here</span></div>
<div><span style="font-family: "Courier New", monospace;"> PETSC_ERROR_CODE_TYPEDEF enum PETSC_ERROR_CODE_NODISCARD {</span></div>
<div><span style="font-family: "Courier New", monospace;"> ^</span></div>
<div><br>
</div>
<div><span style="font-family: "Courier New", monospace;"> CLINKER arch-darwin-c-opt/tests/sys/classes/draw/tests/ex1</span></div>
<div><span style="font-family: "Courier New", monospace;"> TEST arch-darwin-c-opt/tests/counts/sys_classes_draw_tests-ex1_1.counts</span></div>
<div><span style="font-family: "Courier New", monospace;">not ok sys_classes_draw_tests-ex1_1<span class="Apple-converted-space"> </span><b># Error code: 139</b></span></div>
<div><b><span style="font-family: "Courier New", monospace;"># [excess:98681] *** Process received signal ***</span></b></div>
<div><b><span style="font-family: "Courier New", monospace;"># [excess:98681] Signal: Segmentation fault: 11 (11)</span></b></div>
<div><b><span style="font-family: "Courier New", monospace;"># [excess:98681] Signal code: Address not mapped (1)</span></b></div>
<div><b><span style="font-family: "Courier New", monospace;"># [excess:98681] Failing at address: 0x7f</span></b></div>
<div><b><span style="font-family: "Courier New", monospace;"># [excess:98681] *** End of error message ***</span></b></div>
<div><span style="font-family: "Courier New", monospace;"># --------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;"># Primary job terminated normally, but 1 process returned</span></div>
<div><span style="font-family: "Courier New", monospace;"># a non-zero exit code. Per user-direction, the job has been aborted.</span></div>
<div><span style="font-family: "Courier New", monospace;"># --------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;"># --------------------------------------------------------------------------</span></div>
<div><span style="font-family: "Courier New", monospace;"># mpiexec noticed that process rank 0 with PID 0 on node excess exited on signal 11 (Segmentation fault: 11).</span></div>
<div><span style="font-family: "Courier New", monospace;"># --------------------------------------------------------------------------</span></div>
<span style="font-family: "Courier New", monospace;"> ok sys_classes_draw_tests-ex1_1 # SKIP Command failed so no diff</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<span style="font-family: "Courier New", monospace;"><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<span style="font-family: Calibri, Helvetica, sans-serif;">I see the same segfault error in all PETSc examples.<span class="Apple-converted-space"> </span><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<span style="font-family: Calibri, Helvetica, sans-serif;">Any help is mostly appreciated, I'm starting to work with PETSc. Our plan is to use the linear solver from PETSc for the Poisson equation on our numerical scheme and test this on a GPU cluster. So also,
any guideline on how to interface PETSc with a fortran code and personal experience is also most appreciated!</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<span style="font-family: Calibri, Helvetica, sans-serif;"><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<span style="font-family: Calibri, Helvetica, sans-serif;">Marcos<br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<span style="font-family: Calibri, Helvetica, sans-serif;"><br>
</span><span style="font-family: "Courier New", monospace;"></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
<span>--<span class="Apple-converted-space"> </span></span><br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
<div><br>
</div>
<div><a href="http://www.cse.buffalo.edu/~knepley/" originalsrc="http://www.cse.buffalo.edu/~knepley/" shash="e8gR3CxAr6PfMQvleElonmrskKgf9UaUQjnIxu34zGXxOXbjSNG5RFSM4+/c/ESKVMkoSpAsvl/oxzrLixW8+T+7DLGltdnH+BfCPnNPq50c4Wb1noyWVlM+dYrHqyHsi1MHZX/wy08VP2r7YWtxKDaDyv0EJZTHoLVl+7trO3Q=" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
<span class="x_gmail_signature_prefix">--<span class="Apple-converted-space"> </span></span><br>
<div dir="ltr" class="x_gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
<div><br>
</div>
<div><a href="http://www.cse.buffalo.edu/~knepley/" originalsrc="http://www.cse.buffalo.edu/~knepley/" shash="H2qTUG/5BR62MoPtjlLpfla4urWdJ6uSH8//1P5XaNc6ntHChs8hTiN+IDGNQXL2PzEUUvnKUmznGeExnfm93CDeKTtdm2QO82mXEqTKtTqRn4fKrJK8meKj3UUJYrAcqjOQdMVTCsCgh8B13sSHE4kc7fzG+wzty3dcDE4WJMQ=" target="_blank">https://www.cse.buffalo.edu/~knepley/</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</body>
</html>