<div dir="ltr">Hi, Jon,<div>  I could reproduce the error exactly.  I will have a look.</div><div>  Thanks for reporting.<br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">--Junchao Zhang</div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Dec 17, 2021 at 2:56 PM Jonathan D. Halverson <<a href="mailto:halverson@princeton.edu">halverson@princeton.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">




<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif;font-size:12pt">Hello,</span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
We are unable to build PETSc using the <span style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif;font-size:12pt">NVIDIA HPC SDK and
<span style="background-color:rgb(255,255,255);display:inline">
--with-scalar-type=complex</span>. Below is our procedure:</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<p style="color:rgb(32,31,30);font-size:15px;margin:0px">
<span style="margin:0px;font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures">$ module load<span> </span></span><span style="margin:0px;font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures"><span style="margin:0px">nvhpc</span>/21.11</span></p>
<p style="color:rgb(32,31,30);font-size:15px;margin:0px">
<span style="margin:0px;font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:black">$ module load openmpi/<span style="margin:0px">nvhpc</span>-21.11/4.1.2/64</span></p>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);min-height:21px">
<span style="margin:0px;font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures">$ git clone -b release <a href="https://gitlab.com/petsc/petsc.git" target="_blank">https://gitlab.com/petsc/petsc.git</a> petsc; </span><span style="margin:0px;font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:black">cd
 petsc</span><br>
<span style="margin:0px;font-variant-ligatures:no-common-ligatures"></span></div>
<p style="color:rgb(32,31,30);font-size:15px;margin:0px">
<span style="margin:0px;font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures">$ ./configure --with-debugging=1 --with-scalar-type=complex PETSC_ARCH=openmpi-power</span></p>
<p style="color:rgb(32,31,30);font-size:15px;margin:0px">
<span style="margin:0px;font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures">$ make PETSC_DIR=/home/$USER/software/petsc PETSC_ARCH=openmpi-power all</span></p>
<p style="color:rgb(32,31,30);font-size:15px;margin:0px">
<span style="margin:0px;font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:black;font-variant-ligatures:no-common-ligatures">$ make PETSC_DIR=/home/$USER/software/petsc PETSC_ARCH=openmpi-power check</span></p>
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
"make check" fails with a segmentation fault when running ex19. The fortran test ex5f passes.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="background-color:rgb(255,255,255);display:inline">The procedure above fails on x86_64 and POWER both running RHEL8. It also fails using nvhpc 20.7.</span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
The procedure above works for "real" instead of "complex".</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
A "hello world" MPI code using a complex data type works with our nvhpc modules.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
The procedure above works successfully when GCC and an Open MPI library built using GCC is used.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
The only trouble is the combination of PETSc with nvhpc and complex. Any known issues?</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
The build log for the procedure above is here:</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<a href="https://tigress-web.princeton.edu/~jdh4/petsc_nvhpc_complex_17dec2021.log" target="_blank">https://tigress-web.princeton.edu/~jdh4/petsc_nvhpc_complex_17dec2021.log</a><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Jon</div>
</div>

</blockquote></div>