<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div><br class=""></div><div> On some systems, like my Mac, the environmental variable CPATH is set to the appropriate directory for mkl.h when MKL is initialized with </div><div><br class=""></div><div>source /opt/intel/compilers_and_libraries/mac/bin/compilervars.sh intel64</div><div><br class=""></div><div>hence the compiler can magically find mkl.h but this doesn't seem to be universal, I think that is why Felix's system cannot find the include, I found a Linux machine at ANL that produces the same problem Felix has. Anyways I'm fixing the PETSc install will just work even if CPATH is not set.</div><div><br class=""></div><div> Barry</div><div><br class=""></div><div><br class=""></div><div><br class=""></div><div><br class=""></div><div><br class=""><blockquote type="cite" class=""><div class="">On Aug 3, 2020, at 10:21 PM, Xiaoye S. Li <<a href="mailto:xsli@lbl.gov" class="">xsli@lbl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="gmail_default" style="font-family: verdana, sans-serif; font-size: small;">Regarding MKL, superlu only uses some BLAS routines in MKL. If you have alternative BLAS, you don't need to use MKL.</div><div class="gmail_default" style="font-family: verdana, sans-serif; font-size: small;"><br class=""></div><div class="gmail_default" style="font-family: verdana, sans-serif; font-size: small;">But, in my experience, on an Intel system, the compiler can correctly include the path with mkl.h, that is, I don't need to do anything special. </div><div class="gmail_default" style="font-family: verdana, sans-serif; font-size: small;">I think mpiicc should work.</div><div class="gmail_default" style="font-family: verdana, sans-serif; font-size: small;"><br class=""></div><div class="gmail_default" style="font-family: verdana, sans-serif; font-size: small;">Sherry</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 3, 2020 at 8:46 AM <<a href="mailto:flw@rzg.mpg.de" class="">flw@rzg.mpg.de</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Barry,<br class="">
Thanks for the branch (and thanks to Sherry as well). I tried to use <br class="">
the configure example " arch-ci-linux-intel-mkl-single.py" (adding the <br class="">
--with-batch flag, since I am running on a cluster), but I get the <br class="">
following error message:<br class="">
TESTING: check from <br class="">
config.libraries(config/BuildSystem/config/libraries.py:157) <br class="">
<br class="">
<br class="">
<br class="">
*******************************************************************************<br class="">
UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log <br class="">
for details):<br class="">
-------------------------------------------------------------------------------<br class="">
Downloaded ptscotch could not be used. Please check install in <br class="">
/u/flw/petsc-hash-pkgs/073aec050<br class="">
<br class="">
(see configure1.log)<br class="">
<br class="">
Next I tried a minimalistic version using Intel MPI compilers:<br class="">
./configure --download-superlu_dist --download-metis <br class="">
--download-parmetis --download-ptscotch --with-precision=single <br class="">
--with-batch --with-fc=mpiifort --with-cc=mpiicc --with-cxx=mpiicpc<br class="">
<br class="">
There I got the following error:<br class="">
=============================================================================== Compiling and installing SUPERLU_DIST; this may take several minutes =============================================================================== <br class="">
*******************************************************************************<br class="">
UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log <br class="">
for details):<br class="">
-------------------------------------------------------------------------------<br class="">
Error running make on SUPERLU_DIST<br class="">
<br class="">
(see configure2.log)<br class="">
<br class="">
<br class="">
Next I tried<br class="">
./configure --download-superlu_dist --download-metis <br class="">
--download-parmetis --download-ptscotch --with-precision=single <br class="">
--with-batch <br class="">
--with-mpi-dir=/mpcdf/soft/SLE_12/packages/x86_64/intel_parallel_studio/2018.4/impi/2018.4.274/intel64<br class="">
<br class="">
Same error message (see configure3.log)<br class="">
<br class="">
<br class="">
It seems that there is something going on with mkl in case I want to <br class="">
use Intel compilers for C and C++ (compiling with gcc and g++ seems to <br class="">
work )<br class="">
<br class="">
Do you know what is going on there? (I am running on the Draco <br class="">
cluster, if that helps <br class="">
(<a href="https://www.mpcdf.mpg.de/services/computing/draco" rel="noreferrer" target="_blank" class="">https://www.mpcdf.mpg.de/services/computing/draco</a>))<br class="">
<br class="">
<br class="">
Best regards,<br class="">
Felix<br class="">
<br class="">
Zitat von Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank" class="">bsmith@petsc.dev</a>>:<br class="">
<br class="">
> Felix,<br class="">
><br class="">
> The branch is ready. Just use<br class="">
><br class="">
> git checkout barry/2020-07-28/superlu_dist-single<br class="">
> ./configure --download-superlu_dist --download-metis <br class="">
> --download-parmetis --download-ptscotch --with-precision=single <br class="">
> and whatever else you use<br class="">
><br class="">
> Barry<br class="">
><br class="">
> It will automatically get the needed branch of SuperLU_DIST that <br class="">
> Sherry prepared.<br class="">
><br class="">
><br class="">
>> On Jul 27, 2020, at 2:10 PM, <a href="mailto:flw@rzg.mpg.de" target="_blank" class="">flw@rzg.mpg.de</a> wrote:<br class="">
>><br class="">
>> Hi Shery,<br class="">
>> Yes, ideally we would like to compile PETSc in single precision and <br class="">
>> simply run a single precision version of SUPERLU_DIST just like <br class="">
>> e.g. MUMPS.<br class="">
>><br class="">
>> Best regards and thanks,<br class="">
>> Felix<br class="">
>> Zitat von "Xiaoye S. Li" <<a href="mailto:xsli@lbl.gov" target="_blank" class="">xsli@lbl.gov</a>>:<br class="">
>><br class="">
>>> Barry,<br class="">
>>><br class="">
>>> I have a branch 'Mixed-precision' working with single precision FP32. I<br class="">
>>> assume Felix wants to use superlu_dist from petsc. How do you want to<br class="">
>>> incorporate it in petsc?<br class="">
>>><br class="">
>>> <a href="https://github.com/xiaoyeli/superlu_dist" rel="noreferrer" target="_blank" class="">https://github.com/xiaoyeli/superlu_dist</a><br class="">
>>><br class="">
>>> PS1: in this version, FP32 only works on CPU. FP64 and complex-FP64 all<br class="">
>>> work on GPU.<br class="">
>>><br class="">
>>> PS2: currently there is no mixed-precision yet, but it is the branch we are<br class="">
>>> adding mix-prec support. Will take a while before merging to master.<br class="">
>>><br class="">
>>> Sherry<br class="">
>>><br class="">
>>><br class="">
>>> On Wed, Jul 22, 2020 at 6:04 AM <<a href="mailto:flw@rzg.mpg.de" target="_blank" class="">flw@rzg.mpg.de</a>> wrote:<br class="">
>>><br class="">
>>>> Hi Barry,<br class="">
>>>> for now I just want to run everything in single on CPUs only with<br class="">
>>>> SUPERLU_DIST. Maybe we will also incorporate GPUs in the future, but<br class="">
>>>> there are no immediate plans yet. So if you could provide the support,<br class="">
>>>> that would be awesome.<br class="">
>>>><br class="">
>>>> Best regards,<br class="">
>>>> Felix<br class="">
>>>><br class="">
>>>> Zitat von Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank" class="">bsmith@petsc.dev</a>>:<br class="">
>>>><br class="">
>>>> > Felix,<br class="">
>>>> ><br class="">
>>>> > What are your needs, do you want this for CPUs or for GPUs? Do<br class="">
>>>> > you wish to run all your code in single precision or just the<br class="">
>>>> > SuperLU_Dist solver while the rest of your code double?<br class="">
>>>> ><br class="">
>>>> > If you want to run everything on CPUs using single precision<br class="">
>>>> > then adding the support is very easy, we can provide that for you<br class="">
>>>> > any time. The other cases will require more thought.<br class="">
>>>> ><br class="">
>>>> > Barry<br class="">
>>>> ><br class="">
>>>> ><br class="">
>>>> >> On Jul 21, 2020, at 8:58 AM, <a href="mailto:flw@rzg.mpg.de" target="_blank" class="">flw@rzg.mpg.de</a> wrote:<br class="">
>>>> >><br class="">
>>>> >> Dear PETSc support team,<br class="">
>>>> >> some time ago you told me that you are planning on releasing a<br class="">
>>>> >> version that supports SUPERLU_DIST in single-precision soon. Can<br class="">
>>>> >> you tell me roughly what time frame you had in mind?<br class="">
>>>> >><br class="">
>>>> >> Best regards,<br class="">
>>>> >> Felix<br class="">
>>>> >><br class="">
>>>><br class="">
>>>><br class="">
>>>><br class="">
>>>><br class="">
>><br class="">
>><br class="">
>><br class="">
<br class="">
<br class="">
</blockquote></div>
</div></blockquote></div><br class=""></body></html>