[petsc-dev] Error on Fugaku

[Mark Adams]

Satish,

For the fujitsu compiler OMP is -Kopenmp.

Sarat (cc'ed) tells me that he built Kokkos with:

cmake -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX=${KOKKOS_SRC_DIR}/install \
    -DBUILD_TESTING=OFF \
    -DKokkos_ENABLE_LIBDL=OFF \
    -DKokkos_ENABLE_OPENMP=ON \
    -DKokkos_ENABLE_SERIAL=ON \
    -DKokkos_ENABLE_AGGRESSIVE_VECTORIZATION=ON \
    ..

How might I make this happen in PETSc?

Thanks,
Mark


On Wed, Apr 14, 2021 at 2:44 PM Satish Balay <balay at mcs.anl.gov> wrote:

> On Wed, 14 Apr 2021, Mark Adams wrote:
>
> > I have this building now.
> > Do you know anything about OpenMP?
> > I can add --with-openmp
> > That should get Kokkos to be made with OpenMP.
> > Should PETSc deal with the compilers correctly?
>
> Well it tries the following compiler options for openmp.
>
>     oflags = ["-fopenmp", # Gnu
>               "-qsmp=omp",# IBM XL C/C++
>               "-h omp",   # Cray. Must come after XL because XL interprets
> this option as meaning "-soname omp"
>               "-mp",      # Portland Group
>               "-Qopenmp", # Intel windows
>               "-openmp",  # Intel
>               "-xopenmp", # Sun
>               "+Oopenmp", # HP
>               "/openmp"   # Microsoft Visual Studio
>               ]
>
> I don't know what the flag for fugaku compiler is.
>
> Satish
>
>
> > Thanks,
> > Mark
> >
> > On Wed, Apr 14, 2021 at 1:45 PM Mark Adams <mfadams at lbl.gov> wrote:
> >
> > > Thanks,
> > > If you feel inspired you could try Kokkos :||
> > > I am in a parking lot waiting for my daughter but can try this when I
> get
> > > home,
> > > Thanks again,
> > > Mark
> > >
> > > On Wed, Apr 14, 2021 at 1:33 PM Satish Balay <balay at mcs.anl.gov>
> wrote:
> > >
> > >> I think I allocated a single node - and did the build on it.
> > >>
> > >> Now I'm getting an error - don't know what changed..
> > >>
> > >> login6$ pjsub --interact -L "node=1" -L "rscunit=rscunit_ft01" -L
> > >> "rscgrp=eap-int" -L "elapse=1:00:00" --sparam "wait-time=600"
> > >> [ERR.] PJM 0059 pjsub rscgrp=eap-int is disabled.
> > >>
> > >> Ok - the following worked..
> > >>
> > >> login6$ pjsub --interact -L "node=1" -L "rscunit=rscunit_ft01" -L
> > >> "elapse=1:00:00" --sparam "wait-time=600"
> > >> [INFO] PJM 0000 pjsub Job 6301572 submitted.
> > >> [INFO] PJM 0081 .connected.
> > >> [INFO] PJM 0082 pjsub Interactive job 6301572 started.
> > >> [a04201 at j31-3110s petsc]$
> > >>
> > >> Ok - trying this build now.
> > >>
> > >> [a04201 at j31-3110s petsc]$ cat
> > >> ~/petsc.save/arch-arm/lib/petsc/conf/reconfigure-arch-arm.py
> > >> #!/usr/bin/python3
> > >> if __name__ == '__main__':
> > >>   import sys
> > >>   import os
> > >>   sys.path.insert(0, os.path.abspath('config'))
> > >>   import configure
> > >>   configure_options = [
> > >>     '--with-blaslapack-lib=-lfjlapack',
> > >>     '--with-debugging=0',
> > >>     'CC=mpifcc',
> > >>     'CXX=mpiFCC',
> > >>     'FC=mpifrt',
> > >>     'PETSC_ARCH=arch-arm',
> > >>   ]
> > >>   configure.petsc_configure(configure_options)
> > >> [a04201 at j31-3110s petsc]$
> > >> ~/petsc.save/arch-arm/lib/petsc/conf/reconfigure-arch-arm.py
> > >> --download-p4est --download-zlib
> > >> <snip>
> > >> p4est:
> > >>   Includes: -I/vol0004/ra010009/a04201/petsc/arch-arm/include
> > >>   Library:  -Wl,-rpath,/vol0004/ra010009/a04201/petsc/arch-arm/lib
> > >> -L/vol0004/ra010009/a04201/petsc/arch-arm/lib -lp4est -lsc
> > >>
> > >> Ok - this worked for me.
> > >>
> > >> Satish
> > >>
> > >> On Wed, 14 Apr 2021, Mark Adams wrote:
> > >>
> > >> > Do you recall what nodes you use to build on a "compute" node, to
> avoid
> > >> > cross compilation?
> > >> >
> > >> > On Wed, Apr 14, 2021 at 12:08 PM Satish Balay <balay at mcs.anl.gov>
> > >> wrote:
> > >> >
> > >> > > looks like p4est cannot be cross-compiled.
> > >> > >
> > >> > > Satish
> > >> > >
> > >> > > On Wed, 14 Apr 2021, Mark Adams wrote:
> > >> > >
> > >> > > > I get this error with p4est on Fugaku.
> > >> > > > It is a Fortran error. Odd.
> > >> > > > Mark
> > >> > > >
> > >> > >
> > >> > >
> > >> >
> > >>
> > >>
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20210414/ffc4b9eb/attachment.html>