[petsc-dev] [EXTERNAL] Re: building on Fugaku

[Mark Adams]

On Fri, Apr 16, 2021 at 12:31 PM Sreepathi, Sarat <sarat at ornl.gov> wrote:

> Hi Mark,
>
>
>
> I couldn’t follow all the issues in the other thread.  Anyway, you are in
> luck.
>
> I have pushed for GNU compiler support with Fujitsu MPI since November
> after hitting multiple issues. Finally, got an update from Fugaku helpdesk
> earlier this week.
>
> As you are aware, you will probably get better performance with GNU than
> Fujitsu on A64FX. My latest experiments confirmed that **again** for
> climate workloads.
>
>
>
> See "Users Guide - Language and development environment Ver. 1.02" -
> Chapter 4 for details.
>
>
> https://www.fugaku.r-ccs.riken.jp/doc_root/en/user_guides/lang_1.02/GCC_MPI/index.html
>
>
>
> Additionally, please note that you will need to `spack load gcc at 10.2.0`
> to load the latest GNU compiler.
>
> To summarize, you need to do the following on a compute node to get the
> requisite environment.
>
> ```
>
> . /vol0004/apps/oss/spack/share/spack/setup-env.sh
>
> spack load fujitsu-mpi%gcc
>
> spack load gcc at 10.2.0
>
> ```
>

Yes, I go this this AM from Kokkos:

   . /vol0004/apps/oss/spack/share/spack/setup-env.sh
    spack load gcc at 10.2.0%gcc at 8.3.1 arch=linux-rhel8-a64fx
    spack load fujitsu-mpi at 4.5.0%gcc at 8.3.1 arch=linux-rhel8-a64fx

Looks like the same thing + @8.3.1 arch=linux-rhel8-a64fx

It has passed PETSc 'make check'

Thanks,


> Hope this helps (cc’ing petsc-dev),
> Sarat.
>
> *From:* Mark Adams <mfadams at lbl.gov>
> *Sent:* Friday, April 16, 2021 10:27 AM
> *To:* Sreepathi, Sarat <sarat at ornl.gov>
> *Subject:* Re: [EXTERNAL] Re: building on Fugaku
>
>
>
> Sarat,
>
> I am making progress. Kokkos is working w/o OMP,  but Kokkos Kernels + OMP
> is barfing and Kokkos people suggested GNU.
>
> But one of my third party libraries needs MPI. Do you know of some sort of
> MPI library on Fugaku?
>
> It just has to work with one process.
>
> Thanks,
>
> Mark
>
>
>
> On Wed, Apr 14, 2021 at 3:21 PM Sreepathi, Sarat <sarat at ornl.gov> wrote:
>
> Hi Mark,
>
>
>
> From earlier notes, I used the Kokkos “develop” branch.
>
>
>
> cmake -DCMAKE_BUILD_TYPE=Release \
>     -DCMAKE_INSTALL_PREFIX=${KOKKOS_SRC_DIR}/install \
>     -DBUILD_TESTING=OFF \
>     -DKokkos_ENABLE_LIBDL=OFF \
>     -DKokkos_ENABLE_OPENMP=ON \
>     -DKokkos_ENABLE_SERIAL=ON \
>     -DKokkos_ENABLE_AGGRESSIVE_VECTORIZATION=ON \
>     ..
>
> -Sarat.
>
>
>
> *From:* Mark Adams <mfadams at lbl.gov>
> *Sent:* Wednesday, April 14, 2021 3:09 PM
> *To:* Sreepathi, Sarat <sarat at ornl.gov>
> *Cc:* Richard Tran Mills <rtmills at anl.gov>
> *Subject:* [EXTERNAL] Re: building on Fugaku
>
>
>
> Sarat,
>
>
>
> I have made some progress but am getting a C++ error:
>
>
>
> CMake Warning at cmake/kokkos_test_cxx_std.cmake:47 (MESSAGE):
>   C++14 is not supported as a compiler feature.  We will choose custom
> flags
>   for now, but this behavior has been deprecated.  Please open an issue at
>   https://github.com/kokkos/kokkos/issues reporting that Fujitsu failed
> for
>   14, preferrably including your CMake command.
>
>
>
> I am using:
>
>
>
> 04:08 adams/landau-gpu-opt= ~/petsc$ mpiFCC --version
> FCC (FCC) 4.5.0 20210304
> simulating gcc version 6.1
>
>
>
> Do you have any ideas?
>
>
>
> Thanks,
>
> Mark
>
>
>
> On Wed, Apr 14, 2021 at 7:55 AM Mark Adams <mfadams at lbl.gov> wrote:
>
> Hi Sarat,
>
>
>
> My colleague, Rich Mills, and I are coming back to Fugaku after a while,
> and I wanted to ask you how you build, with Kokkos, on Fugaku.
>
> I have tried Arm and Fugitzu compilers and was able to get Fugitzu to work
> a few months ago, but I wanted to see what you use. Especially for Kokkos.
>
> Can you share with us your build details?
>
>
>
> Thanks,
>
> Mark
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20210416/c21a0294/attachment.html>