<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;">Hello,</span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We are unable to build PETSc using the <span style="color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;">NVIDIA HPC SDK and
<span style="caret-color:rgb(32, 31, 30);background-color:rgb(255, 255, 255);display:inline !important">
--with-scalar-type=complex</span>. Below is our procedure:</span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<p style="caret-color:rgb(32, 31, 30);color:rgb(32, 31, 30);font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:15px;margin:0px">
<span style="margin:0px;font-family:Calibri, Helvetica, sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures">$ module load<span class="Apple-converted-space"> </span></span><span style="margin:0px;font-family:Calibri, Helvetica, sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures"><span data-markjs="true" class="marko3o8a0796" data-ogac="" data-ogab="" data-ogsc="" data-ogsb="" style="margin:0px">nvhpc</span>/21.11</span></p>
<p style="caret-color:rgb(32, 31, 30);color:rgb(32, 31, 30);font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:15px;margin:0px">
<span style="margin:0px;font-family:Calibri, Helvetica, sans-serif;font-size:12pt;color:black">$ module load openmpi/<span data-markjs="true" class="marko3o8a0796" data-ogac="" data-ogab="" data-ogsc="" data-ogsb="" style="margin:0px">nvhpc</span>-21.11/4.1.2/64</span></p>
<div style="margin:0px;font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:15px;color:rgb(32, 31, 30);caret-color:rgb(32, 31, 30);min-height:21px">
<span style="margin:0px;font-family:Calibri, Helvetica, sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures">$ git clone -b release https://gitlab.com/petsc/petsc.git petsc; </span><span style="margin:0px;font-family:Calibri, Helvetica, sans-serif;font-size:12pt;color:black">cd
petsc</span><br>
<span style="margin:0px;font-variant-ligatures:no-common-ligatures"></span></div>
<p style="caret-color:rgb(32, 31, 30);color:rgb(32, 31, 30);font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:15px;margin:0px">
<span style="margin:0px;font-family:Calibri, Helvetica, sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures">$ ./configure --with-debugging=1 --with-scalar-type=complex PETSC_ARCH=openmpi-power</span></p>
<p style="caret-color:rgb(32, 31, 30);color:rgb(32, 31, 30);font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:15px;margin:0px">
<span style="margin:0px;font-family:Calibri, Helvetica, sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures">$ make PETSC_DIR=/home/$USER/software/petsc PETSC_ARCH=openmpi-power all</span></p>
<p style="caret-color:rgb(32, 31, 30);color:rgb(32, 31, 30);font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:15px;margin:0px">
<span style="margin:0px;font-family:Calibri, Helvetica, sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures">$ make PETSC_DIR=/home/$USER/software/petsc PETSC_ARCH=openmpi-power check</span></p>
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
"make check" fails with a segmentation fault when running ex19. The fortran test ex5f passes.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="caret-color:rgb(0, 0, 0);background-color:rgb(255, 255, 255);display:inline !important">The procedure above fails on x86_64 and POWER both running RHEL8. It also fails using nvhpc 20.7.</span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
The procedure above works for "real" instead of "complex".</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
A "hello world" MPI code using a complex data type works with our nvhpc modules.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
The procedure above works successfully when GCC and an Open MPI library built using GCC is used.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
The only trouble is the combination of PETSc with nvhpc and complex. Any known issues?</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
The build log for the procedure above is here:</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<a href="https://tigress-web.princeton.edu/~jdh4/petsc_nvhpc_complex_17dec2021.log">https://tigress-web.princeton.edu/~jdh4/petsc_nvhpc_complex_17dec2021.log</a><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Jon</div>
</body>
</html>