[petsc-users] SUPERLU_DIST in single precision

Barry Smith bsmith at petsc.dev
Tue Aug 4 10:55:57 CDT 2020 

  Good. I have cleaned up the code and rebased the branch so if you use it in the future you need to do

  git fetch
  git checkout master 
  git branch -D barry/2020-07-28/superlu_dist-single
 git checkout barry/2020-07-28/superlu_dist-single
 
  Do not just do a git pull in barry/2020-07-28/superlu_dist-single that will make a mess


> On Aug 4, 2020, at 8:09 AM, flw at rzg.mpg.de wrote:
> 
> PS:
> I saw you committed a new version for the SUPERLU_DIST installer.
> There is a missing ")" at the end of line 42.
> Also I think one also has to add
> self.addToArgs(args,'-DCMAKE_CXX_FLAGS:STRING','-I'+self.blasLapack.include)
> 
> as well (for me only adding the C flag raised the same error as before; with this, it works for me even if CPATH isn't set)
> Thank you again very much :)
> 
> Best regards,
> Felix
> 
> Zitat von Barry Smith <bsmith at petsc.dev>:
> 
>> On some systems, like my Mac, the environmental variable CPATH is set to the appropriate directory for mkl.h when MKL is initialized with
>> 
>> source /opt/intel/compilers_and_libraries/mac/bin/compilervars.sh intel64
>> 
>> hence the compiler can magically find mkl.h but this doesn't seem to be universal, I think that is why Felix's system cannot find the include, I found a Linux machine at ANL that produces the same problem Felix has. Anyways I'm fixing the PETSc install will just work even if CPATH is not set.
>> 
>>   Barry
>> 
>> 
>> 
>> 
>> 
>>> On Aug 3, 2020, at 10:21 PM, Xiaoye S. Li <xsli at lbl.gov> wrote:
>>> 
>>> Regarding MKL, superlu only uses some BLAS routines in MKL.  If you have alternative BLAS, you don't need to use MKL.
>>> 
>>> But, in my experience, on an Intel system, the compiler can correctly include the path with mkl.h, that is, I don't need to do anything special.
>>> I think mpiicc should work.
>>> 
>>> Sherry
>>> 
>>> On Mon, Aug 3, 2020 at 8:46 AM <flw at rzg.mpg.de <mailto:flw at rzg.mpg.de>> wrote:
>>> Hi Barry,
>>> Thanks for the branch (and thanks to Sherry as well). I tried to use
>>> the configure example " arch-ci-linux-intel-mkl-single.py" (adding the
>>> --with-batch flag, since I am running on a cluster), but I get the
>>> following error message:
>>> TESTING: check from
>>> config.libraries(config/BuildSystem/config/libraries.py:157)
>>> 
>>> 
>>> 
>>> *******************************************************************************
>>>          UNABLE to CONFIGURE with GIVEN OPTIONS    (see configure.log
>>> for details):
>>> -------------------------------------------------------------------------------
>>> Downloaded ptscotch could not be used. Please check install in
>>> /u/flw/petsc-hash-pkgs/073aec050
>>> 
>>> (see configure1.log)
>>> 
>>> Next I tried a minimalistic version using Intel MPI compilers:
>>> ./configure --download-superlu_dist --download-metis
>>> --download-parmetis --download-ptscotch  --with-precision=single
>>> --with-batch --with-fc=mpiifort --with-cc=mpiicc --with-cxx=mpiicpc
>>> 
>>> There I got the following error:
>>> ===============================================================================                                                                                                                                                                     Compiling and installing SUPERLU_DIST; this may take several minutes                                                                                                                                                                    ===============================================================================
>>> *******************************************************************************
>>>          UNABLE to CONFIGURE with GIVEN OPTIONS    (see configure.log
>>> for details):
>>> -------------------------------------------------------------------------------
>>> Error running make on  SUPERLU_DIST
>>> 
>>> (see configure2.log)
>>> 
>>> 
>>> Next I tried
>>> ./configure --download-superlu_dist --download-metis
>>> --download-parmetis --download-ptscotch  --with-precision=single
>>> --with-batch
>>> --with-mpi-dir=/mpcdf/soft/SLE_12/packages/x86_64/intel_parallel_studio/2018.4/impi/2018.4.274/intel64
>>> 
>>> Same error message (see configure3.log)
>>> 
>>> 
>>> It seems that there is something going on with mkl in case I want to
>>> use Intel compilers for C and C++ (compiling with gcc and g++ seems to
>>> work )
>>> 
>>> Do you know what is going on there?  (I am running on the Draco
>>> cluster, if that helps
>>> (https://www.mpcdf.mpg.de/services/computing/draco <https://www.mpcdf.mpg.de/services/computing/draco>))
>>> 
>>> 
>>> Best regards,
>>> Felix
>>> 
>>> Zitat von Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>>:
>>> 
>>> > Felix,
>>> >
>>> >     The branch is ready. Just use
>>> >
>>> >     git checkout barry/2020-07-28/superlu_dist-single
>>> >   ./configure --download-superlu_dist --download-metis
>>> > --download-parmetis --download-ptscotch  --with-precision=single
>>> > and whatever else you use
>>> >
>>> >   Barry
>>> >
>>> >   It will automatically get the needed branch of SuperLU_DIST that
>>> > Sherry prepared.
>>> >
>>> >
>>> >> On Jul 27, 2020, at 2:10 PM, flw at rzg.mpg.de <mailto:flw at rzg.mpg.de> wrote:
>>> >>
>>> >> Hi Shery,
>>> >> Yes, ideally we would like to compile PETSc in single precision and
>>> >> simply run a single precision version of SUPERLU_DIST just like
>>> >> e.g. MUMPS.
>>> >>
>>> >> Best regards and thanks,
>>> >> Felix
>>> >> Zitat von "Xiaoye S. Li" <xsli at lbl.gov <mailto:xsli at lbl.gov>>:
>>> >>
>>> >>> Barry,
>>> >>>
>>> >>> I have a branch 'Mixed-precision' working with single precision FP32. I
>>> >>> assume Felix wants to use superlu_dist from petsc.  How do you want to
>>> >>> incorporate it in petsc?
>>> >>>
>>> >>> https://github.com/xiaoyeli/superlu_dist <https://github.com/xiaoyeli/superlu_dist>
>>> >>>
>>> >>> PS1:  in this version, FP32 only works on CPU.  FP64 and complex-FP64 all
>>> >>> work on GPU.
>>> >>>
>>> >>> PS2: currently there is no mixed-precision yet, but it is the branch we are
>>> >>> adding mix-prec support.  Will take a while before merging to master.
>>> >>>
>>> >>> Sherry
>>> >>>
>>> >>>
>>> >>> On Wed, Jul 22, 2020 at 6:04 AM <flw at rzg.mpg.de <mailto:flw at rzg.mpg.de>> wrote:
>>> >>>
>>> >>>> Hi Barry,
>>> >>>> for now I just want to run everything in single on CPUs only with
>>> >>>> SUPERLU_DIST. Maybe we will also incorporate GPUs in the future, but
>>> >>>> there are no immediate plans yet. So if you could provide the support,
>>> >>>> that would be awesome.
>>> >>>>
>>> >>>> Best regards,
>>> >>>> Felix
>>> >>>>
>>> >>>> Zitat von Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>>:
>>> >>>>
>>> >>>> > Felix,
>>> >>>> >
>>> >>>> >     What are your needs, do you want this for CPUs or for GPUs?  Do
>>> >>>> > you wish to run all your code in single precision or just the
>>> >>>> > SuperLU_Dist solver while the rest of your code double?
>>> >>>> >
>>> >>>> >     If you want to run everything on CPUs using single precision
>>> >>>> > then adding the support is very easy, we can provide that for you
>>> >>>> > any time. The other cases will require more thought.
>>> >>>> >
>>> >>>> >     Barry
>>> >>>> >
>>> >>>> >
>>> >>>> >> On Jul 21, 2020, at 8:58 AM, flw at rzg.mpg.de <mailto:flw at rzg.mpg.de> wrote:
>>> >>>> >>
>>> >>>> >> Dear PETSc support team,
>>> >>>> >> some time ago you told me that you are planning on releasing a
>>> >>>> >> version that supports SUPERLU_DIST in single-precision soon. Can
>>> >>>> >> you tell me roughly what time frame you had in mind?
>>> >>>> >>
>>> >>>> >> Best regards,
>>> >>>> >> Felix
>>> >>>> >>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>
>>> >>
>>> >>
>>> 
>>> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200804/6a83616c/attachment.html>

More information about the petsc-users mailing list