[petsc-users] SUPERLU_DIST in single precision

Tue Aug 4 08:09:32 CDT 2020

PS:
I saw you committed a new version for the SUPERLU_DIST installer.
There is a missing ")" at the end of line 42.
Also I think one also has to add
self.addToArgs(args,'-DCMAKE_CXX_FLAGS:STRING','-I'+self.blasLapack.include)

as well (for me only adding the C flag raised the same error as  
before; with this, it works for me even if CPATH isn't set)
Thank you again very much :)

Best regards,
Felix

Zitat von Barry Smith <bsmith at petsc.dev>:

> On some systems, like my Mac, the environmental variable CPATH is  
> set to the appropriate directory for mkl.h when MKL is initialized  
> with
>
> source /opt/intel/compilers_and_libraries/mac/bin/compilervars.sh intel64
>
> hence the compiler can magically find mkl.h but this doesn't seem to  
> be universal, I think that is why Felix's system cannot find the  
> include, I found a Linux machine at ANL that produces the same  
> problem Felix has. Anyways I'm fixing the PETSc install will just  
> work even if CPATH is not set.
>
>    Barry
>
>
>
>
>
>> On Aug 3, 2020, at 10:21 PM, Xiaoye S. Li <xsli at lbl.gov> wrote:
>>
>> Regarding MKL, superlu only uses some BLAS routines in MKL.  If you  
>> have alternative BLAS, you don't need to use MKL.
>>
>> But, in my experience, on an Intel system, the compiler can  
>> correctly include the path with mkl.h, that is, I don't need to do  
>> anything special.
>> I think mpiicc should work.
>>
>> Sherry
>>
>> On Mon, Aug 3, 2020 at 8:46 AM <flw at rzg.mpg.de  
>> <mailto:flw at rzg.mpg.de>> wrote:
>> Hi Barry,
>> Thanks for the branch (and thanks to Sherry as well). I tried to use
>> the configure example " arch-ci-linux-intel-mkl-single.py" (adding the
>> --with-batch flag, since I am running on a cluster), but I get the
>> following error message:
>> TESTING: check from
>> config.libraries(config/BuildSystem/config/libraries.py:157)
>>
>>
>>
>> *******************************************************************************
>>           UNABLE to CONFIGURE with GIVEN OPTIONS    (see configure.log
>> for details):
>> -------------------------------------------------------------------------------
>> Downloaded ptscotch could not be used. Please check install in
>> /u/flw/petsc-hash-pkgs/073aec050
>>
>> (see configure1.log)
>>
>> Next I tried a minimalistic version using Intel MPI compilers:
>> ./configure --download-superlu_dist --download-metis
>> --download-parmetis --download-ptscotch  --with-precision=single
>> --with-batch --with-fc=mpiifort --with-cc=mpiicc --with-cxx=mpiicpc
>>
>> There I got the following error:
>> ===============================================================================                                                                                                                                                                     Compiling and installing SUPERLU_DIST; this may take several minutes                                                                                                                                                                     
>> ===============================================================================
>> *******************************************************************************
>>           UNABLE to CONFIGURE with GIVEN OPTIONS    (see configure.log
>> for details):
>> -------------------------------------------------------------------------------
>> Error running make on  SUPERLU_DIST
>>
>> (see configure2.log)
>>
>>
>> Next I tried
>> ./configure --download-superlu_dist --download-metis
>> --download-parmetis --download-ptscotch  --with-precision=single
>> --with-batch
>> --with-mpi-dir=/mpcdf/soft/SLE_12/packages/x86_64/intel_parallel_studio/2018.4/impi/2018.4.274/intel64
>>
>> Same error message (see configure3.log)
>>
>>
>> It seems that there is something going on with mkl in case I want to
>> use Intel compilers for C and C++ (compiling with gcc and g++ seems to
>> work )
>>
>> Do you know what is going on there?  (I am running on the Draco
>> cluster, if that helps
>> (https://www.mpcdf.mpg.de/services/computing/draco  
>> <https://www.mpcdf.mpg.de/services/computing/draco>))
>>
>>
>> Best regards,
>> Felix
>>
>> Zitat von Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>>:
>>
>> > Felix,
>> >
>> >     The branch is ready. Just use
>> >
>> >     git checkout barry/2020-07-28/superlu_dist-single
>> >   ./configure --download-superlu_dist --download-metis
>> > --download-parmetis --download-ptscotch  --with-precision=single
>> > and whatever else you use
>> >
>> >   Barry
>> >
>> >   It will automatically get the needed branch of SuperLU_DIST that
>> > Sherry prepared.
>> >
>> >
>> >> On Jul 27, 2020, at 2:10 PM, flw at rzg.mpg.de  
>> <mailto:flw at rzg.mpg.de> wrote:
>> >>
>> >> Hi Shery,
>> >> Yes, ideally we would like to compile PETSc in single precision and
>> >> simply run a single precision version of SUPERLU_DIST just like
>> >> e.g. MUMPS.
>> >>
>> >> Best regards and thanks,
>> >> Felix
>> >> Zitat von "Xiaoye S. Li" <xsli at lbl.gov <mailto:xsli at lbl.gov>>:
>> >>
>> >>> Barry,
>> >>>
>> >>> I have a branch 'Mixed-precision' working with single precision FP32. I
>> >>> assume Felix wants to use superlu_dist from petsc.  How do you want to
>> >>> incorporate it in petsc?
>> >>>
>> >>> https://github.com/xiaoyeli/superlu_dist  
>> <https://github.com/xiaoyeli/superlu_dist>
>> >>>
>> >>> PS1:  in this version, FP32 only works on CPU.  FP64 and  
>> complex-FP64 all
>> >>> work on GPU.
>> >>>
>> >>> PS2: currently there is no mixed-precision yet, but it is the  
>> branch we are
>> >>> adding mix-prec support.  Will take a while before merging to master.
>> >>>
>> >>> Sherry
>> >>>
>> >>>
>> >>> On Wed, Jul 22, 2020 at 6:04 AM <flw at rzg.mpg.de  
>> <mailto:flw at rzg.mpg.de>> wrote:
>> >>>
>> >>>> Hi Barry,
>> >>>> for now I just want to run everything in single on CPUs only with
>> >>>> SUPERLU_DIST. Maybe we will also incorporate GPUs in the future, but
>> >>>> there are no immediate plans yet. So if you could provide the support,
>> >>>> that would be awesome.
>> >>>>
>> >>>> Best regards,
>> >>>> Felix
>> >>>>
>> >>>> Zitat von Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>>:
>> >>>>
>> >>>> > Felix,
>> >>>> >
>> >>>> >     What are your needs, do you want this for CPUs or for GPUs?  Do
>> >>>> > you wish to run all your code in single precision or just the
>> >>>> > SuperLU_Dist solver while the rest of your code double?
>> >>>> >
>> >>>> >     If you want to run everything on CPUs using single precision
>> >>>> > then adding the support is very easy, we can provide that for you
>> >>>> > any time. The other cases will require more thought.
>> >>>> >
>> >>>> >     Barry
>> >>>> >
>> >>>> >
>> >>>> >> On Jul 21, 2020, at 8:58 AM, flw at rzg.mpg.de  
>> <mailto:flw at rzg.mpg.de> wrote:
>> >>>> >>
>> >>>> >> Dear PETSc support team,
>> >>>> >> some time ago you told me that you are planning on releasing a
>> >>>> >> version that supports SUPERLU_DIST in single-precision soon. Can
>> >>>> >> you tell me roughly what time frame you had in mind?
>> >>>> >>
>> >>>> >> Best regards,
>> >>>> >> Felix
>> >>>> >>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>
>> >>
>> >>
>>
>>