[petsc-users] SUPERLU_DIST in single precision
flw at rzg.mpg.de
flw at rzg.mpg.de
Wed Aug 5 01:33:10 CDT 2020
Thanks Barry and Sherry, it works like a charm :)
Best regards,
Felix
Zitat von Barry Smith <bsmith at petsc.dev>:
> Good. I have cleaned up the code and rebased the branch so if you
> use it in the future you need to do
>
> git fetch
> git checkout master
> git branch -D barry/2020-07-28/superlu_dist-single
> git checkout barry/2020-07-28/superlu_dist-single
>
> Do not just do a git pull in barry/2020-07-28/superlu_dist-single
> that will make a mess
>
>
>> On Aug 4, 2020, at 8:09 AM, flw at rzg.mpg.de wrote:
>>
>> PS:
>> I saw you committed a new version for the SUPERLU_DIST installer.
>> There is a missing ")" at the end of line 42.
>> Also I think one also has to add
>> self.addToArgs(args,'-DCMAKE_CXX_FLAGS:STRING','-I'+self.blasLapack.include)
>>
>> as well (for me only adding the C flag raised the same error as
>> before; with this, it works for me even if CPATH isn't set)
>> Thank you again very much :)
>>
>> Best regards,
>> Felix
>>
>> Zitat von Barry Smith <bsmith at petsc.dev>:
>>
>>> On some systems, like my Mac, the environmental variable CPATH is
>>> set to the appropriate directory for mkl.h when MKL is initialized
>>> with
>>>
>>> source /opt/intel/compilers_and_libraries/mac/bin/compilervars.sh intel64
>>>
>>> hence the compiler can magically find mkl.h but this doesn't seem
>>> to be universal, I think that is why Felix's system cannot find
>>> the include, I found a Linux machine at ANL that produces the same
>>> problem Felix has. Anyways I'm fixing the PETSc install will just
>>> work even if CPATH is not set.
>>>
>>> Barry
>>>
>>>
>>>
>>>
>>>
>>>> On Aug 3, 2020, at 10:21 PM, Xiaoye S. Li <xsli at lbl.gov> wrote:
>>>>
>>>> Regarding MKL, superlu only uses some BLAS routines in MKL. If
>>>> you have alternative BLAS, you don't need to use MKL.
>>>>
>>>> But, in my experience, on an Intel system, the compiler can
>>>> correctly include the path with mkl.h, that is, I don't need to
>>>> do anything special.
>>>> I think mpiicc should work.
>>>>
>>>> Sherry
>>>>
>>>> On Mon, Aug 3, 2020 at 8:46 AM <flw at rzg.mpg.de
>>>> <mailto:flw at rzg.mpg.de>> wrote:
>>>> Hi Barry,
>>>> Thanks for the branch (and thanks to Sherry as well). I tried to use
>>>> the configure example " arch-ci-linux-intel-mkl-single.py" (adding the
>>>> --with-batch flag, since I am running on a cluster), but I get the
>>>> following error message:
>>>> TESTING: check from
>>>> config.libraries(config/BuildSystem/config/libraries.py:157)
>>>>
>>>>
>>>>
>>>> *******************************************************************************
>>>> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log
>>>> for details):
>>>> -------------------------------------------------------------------------------
>>>> Downloaded ptscotch could not be used. Please check install in
>>>> /u/flw/petsc-hash-pkgs/073aec050
>>>>
>>>> (see configure1.log)
>>>>
>>>> Next I tried a minimalistic version using Intel MPI compilers:
>>>> ./configure --download-superlu_dist --download-metis
>>>> --download-parmetis --download-ptscotch --with-precision=single
>>>> --with-batch --with-fc=mpiifort --with-cc=mpiicc --with-cxx=mpiicpc
>>>>
>>>> There I got the following error:
>>>> =============================================================================== Compiling and installing SUPERLU_DIST; this may take several minutes
>>>> ===============================================================================
>>>> *******************************************************************************
>>>> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log
>>>> for details):
>>>> -------------------------------------------------------------------------------
>>>> Error running make on SUPERLU_DIST
>>>>
>>>> (see configure2.log)
>>>>
>>>>
>>>> Next I tried
>>>> ./configure --download-superlu_dist --download-metis
>>>> --download-parmetis --download-ptscotch --with-precision=single
>>>> --with-batch
>>>> --with-mpi-dir=/mpcdf/soft/SLE_12/packages/x86_64/intel_parallel_studio/2018.4/impi/2018.4.274/intel64
>>>>
>>>> Same error message (see configure3.log)
>>>>
>>>>
>>>> It seems that there is something going on with mkl in case I want to
>>>> use Intel compilers for C and C++ (compiling with gcc and g++ seems to
>>>> work )
>>>>
>>>> Do you know what is going on there? (I am running on the Draco
>>>> cluster, if that helps
>>>> (https://www.mpcdf.mpg.de/services/computing/draco
>>>> <https://www.mpcdf.mpg.de/services/computing/draco>))
>>>>
>>>>
>>>> Best regards,
>>>> Felix
>>>>
>>>> Zitat von Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>>:
>>>>
>>>> > Felix,
>>>> >
>>>> > The branch is ready. Just use
>>>> >
>>>> > git checkout barry/2020-07-28/superlu_dist-single
>>>> > ./configure --download-superlu_dist --download-metis
>>>> > --download-parmetis --download-ptscotch --with-precision=single
>>>> > and whatever else you use
>>>> >
>>>> > Barry
>>>> >
>>>> > It will automatically get the needed branch of SuperLU_DIST that
>>>> > Sherry prepared.
>>>> >
>>>> >
>>>> >> On Jul 27, 2020, at 2:10 PM, flw at rzg.mpg.de
>>>> <mailto:flw at rzg.mpg.de> wrote:
>>>> >>
>>>> >> Hi Shery,
>>>> >> Yes, ideally we would like to compile PETSc in single precision and
>>>> >> simply run a single precision version of SUPERLU_DIST just like
>>>> >> e.g. MUMPS.
>>>> >>
>>>> >> Best regards and thanks,
>>>> >> Felix
>>>> >> Zitat von "Xiaoye S. Li" <xsli at lbl.gov <mailto:xsli at lbl.gov>>:
>>>> >>
>>>> >>> Barry,
>>>> >>>
>>>> >>> I have a branch 'Mixed-precision' working with single
>>>> precision FP32. I
>>>> >>> assume Felix wants to use superlu_dist from petsc. How do you want to
>>>> >>> incorporate it in petsc?
>>>> >>>
>>>> >>> https://github.com/xiaoyeli/superlu_dist
>>>> <https://github.com/xiaoyeli/superlu_dist>
>>>> >>>
>>>> >>> PS1: in this version, FP32 only works on CPU. FP64 and
>>>> complex-FP64 all
>>>> >>> work on GPU.
>>>> >>>
>>>> >>> PS2: currently there is no mixed-precision yet, but it is the
>>>> branch we are
>>>> >>> adding mix-prec support. Will take a while before merging to master.
>>>> >>>
>>>> >>> Sherry
>>>> >>>
>>>> >>>
>>>> >>> On Wed, Jul 22, 2020 at 6:04 AM <flw at rzg.mpg.de
>>>> <mailto:flw at rzg.mpg.de>> wrote:
>>>> >>>
>>>> >>>> Hi Barry,
>>>> >>>> for now I just want to run everything in single on CPUs only with
>>>> >>>> SUPERLU_DIST. Maybe we will also incorporate GPUs in the future, but
>>>> >>>> there are no immediate plans yet. So if you could provide
>>>> the support,
>>>> >>>> that would be awesome.
>>>> >>>>
>>>> >>>> Best regards,
>>>> >>>> Felix
>>>> >>>>
>>>> >>>> Zitat von Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>>:
>>>> >>>>
>>>> >>>> > Felix,
>>>> >>>> >
>>>> >>>> > What are your needs, do you want this for CPUs or for GPUs? Do
>>>> >>>> > you wish to run all your code in single precision or just the
>>>> >>>> > SuperLU_Dist solver while the rest of your code double?
>>>> >>>> >
>>>> >>>> > If you want to run everything on CPUs using single precision
>>>> >>>> > then adding the support is very easy, we can provide that for you
>>>> >>>> > any time. The other cases will require more thought.
>>>> >>>> >
>>>> >>>> > Barry
>>>> >>>> >
>>>> >>>> >
>>>> >>>> >> On Jul 21, 2020, at 8:58 AM, flw at rzg.mpg.de
>>>> <mailto:flw at rzg.mpg.de> wrote:
>>>> >>>> >>
>>>> >>>> >> Dear PETSc support team,
>>>> >>>> >> some time ago you told me that you are planning on releasing a
>>>> >>>> >> version that supports SUPERLU_DIST in single-precision soon. Can
>>>> >>>> >> you tell me roughly what time frame you had in mind?
>>>> >>>> >>
>>>> >>>> >> Best regards,
>>>> >>>> >> Felix
>>>> >>>> >>
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>
>>>> >>
>>>> >>
>>>>
>>>>
>>
>>
>>
More information about the petsc-users
mailing list