[petsc-users] installing petsc via spack on ac922 (a summit like system)
Satish Balay
balay at mcs.anl.gov
Tue Mar 10 13:48:12 CDT 2020
Glad it worked! Thanks for the update.
Satish
On Tue, 10 Mar 2020, Cameron Smith wrote:
> Thank you.
>
> Using the spack spec
>
> petsc at 3.7.7 ~hdf5 ~hypre ~superlu-dist cflags=-qsmp=omp fflags=-qsmp=omp
> cxxflags=-qsmp=omp
>
> and passing the same flags to a manual (non-spack) petsc 3.7.7 build produced
> installs that do not list xlomp_ser in the configure.log.
>
> Thank-you,
> Cameron
>
> On 3/10/20 9:48 AM, Satish Balay wrote:
> > Created an MR for this change
> >
> > https://gitlab.com/petsc/petsc/-/merge_requests/2593
> >
> > Satish
> >
> > On Tue, 10 Mar 2020, Satish Balay via petsc-users wrote:
> >
> >> Another option is to strip out -lxlomp_ser when parsing 'xlc -V' [as its a
> >> common library for all 3 language compilers]. i.e the following change
> >> [this change is on current maint - but similar change can be done with
> >> petsc-3.7]..
> >>
> >> [This is easier than changing the order in which --with-openmp option is
> >> processed]
> >>
> >> Satish
> >> -----------
> >>
> >> diff --git a/config/BuildSystem/config/compilers.py
> >> b/config/BuildSystem/config/compilers.py
> >> index b4cbc183c0..cfffb1a80e 100644
> >> --- a/config/BuildSystem/config/compilers.py
> >> +++ b/config/BuildSystem/config/compilers.py
> >> @@ -308,7 +308,7 @@ class Configure(config.base.Configure):
> >> self.logPrint('already in lflags: '+arg, 4, 'compilers')
> >> continue
> >> # Check for system libraries
> >> - m =
> >> re.match(r'^-l(ang.*|crt[0-9].o|crtbegin.o|c|gcc|gcc_ext(.[0-9]+)*|System|cygwin|crt[0-9].[0-9][0-9].[0-9].o)$',
> >> arg)
> >> + m =
> >> re.match(r'^-l(ang.*|crt[0-9].o|crtbegin.o|c|gcc|gcc_ext(.[0-9]+)*|System|cygwin|xlomp_ser|crt[0-9].[0-9][0-9].[0-9].o)$',
> >> arg)
> >> if m:
> >> self.logPrint('Skipping system library: '+arg, 4, 'compilers')
> >> continue
> >> @@ -687,7 +687,7 @@ class Configure(config.base.Configure):
> >> self.logPrint('already in lflags: '+arg, 4, 'compilers')
> >> continue
> >> # Check for system libraries
> >> - m =
> >> re.match(r'^-l(ang.*|crt[0-9].o|crtbegin.o|c|gcc|gcc_ext(.[0-9]+)*|System|cygwin|crt[0-9].[0-9][0-9].[0-9].o)$',
> >> arg)
> >> + m =
> >> re.match(r'^-l(ang.*|crt[0-9].o|crtbegin.o|c|gcc|gcc_ext(.[0-9]+)*|System|cygwin|xlomp_ser|crt[0-9].[0-9][0-9].[0-9].o)$',
> >> arg)
> >> if m:
> >> self.logPrint('Skipping system library: '+arg, 4, 'compilers')
> >> continue
> >> @@ -1085,7 +1085,7 @@ Otherwise you need a different combination of C, C++,
> >> and Fortran compilers")
> >> self.logPrint('Already in lflags so skipping: '+arg, 4,
> >> 'compilers')
> >> continue
> >> # Check for system libraries
> >> - m =
> >> re.match(r'^-l(ang.*|crt[0-9].o|crtbegin.o|c|gcc|gcc_ext(.[0-9]+)*|System|cygwin|crt[0-9].[0-9][0-9].[0-9].o)$',
> >> arg)
> >> + m =
> >> re.match(r'^-l(ang.*|crt[0-9].o|crtbegin.o|c|gcc|gcc_ext(.[0-9]+)*|System|cygwin|xlomp_ser|crt[0-9].[0-9][0-9].[0-9].o)$',
> >> arg)
> >> if m:
> >> self.logPrint('Found system library therefore skipping: '+arg,
> >> 4, 'compilers')
> >> continue
> >>
> >> On Tue, 10 Mar 2020, Cameron Smith wrote:
> >>
> >>> That makes sense. Thank you.
> >>>
> >>> -Cameron
> >>>
> >>> On 3/10/20 9:25 AM, Satish Balay wrote:
> >>>> PETSc configure attempts to check for compiler libraries [so that one
> >>>> can mix and use c,c++,fortran codes with a c linker] by running
> >>>> compilers in verbose mode - and parsing the output. [i.e
> >>>> checkCLibraries(),
> >>>> checkFortranLibraries() ..]
> >>>>
> >>>> Here IBM compiler is using different library internally based on 'xlc -V'
> >>>> vs
> >>>> 'xlc -fopenmp -V'. For most other compilers - its just an additional
> >>>> library
> >>>> [with corresponding include file]
> >>>>
> >>>> So -fopenmp needs to be set before this step [checkCLibraries()] in
> >>>> configure.
> >>>>
> >>>> PETSc configure treats -with-openmp as an additional package - and
> >>>> process
> >>>> this option after the above checkCLibraries() check.
> >>>>
> >>>> However CFLAGS etc get processed and set before the call to
> >>>> checkCLibraries().
> >>>>
> >>>> My suggestion is a workaround to get -fopenmp option set before
> >>>> checkCLibraries() are called.
> >>>>
> >>>> Satish
> >>>>
> >>>> On Tue, 10 Mar 2020, Cameron Smith wrote:
> >>>>
> >>>>> Thank you. I'll give that a shot.
> >>>>>
> >>>>> Out of curiosity, how does passing the openmp flags relate to the
> >>>>> '--with-openmp' option described here:
> >>>>>
> >>>>> https://www.mcs.anl.gov/petsc/documentation/installation.html
> >>>>>
> >>>>> under 'Installing packages that utilize OpenMP'? Is this just passing
> >>>>> the
> >>>>> openmp flags into compile/link commands of the packages that petsc
> >>>>> builds
> >>>>> (via, --download-<package> options) and not to the petsc compile/link?
> >>>>>
> >>>>> -Cameron
> >>>>>
> >>>>> On 3/10/20 9:01 AM, Satish Balay wrote:
> >>>>>> BTW: You might be able to do the same via spack.
> >>>>>>
> >>>>>> spack install petsc at 3.7.7 ~hdf5 ~hypre ~superlu-dist cflags=-fopenmp
> >>>>>> fflags=-fopenmp cxxflags=-fopenmp
> >>>>>>
> >>>>>> Satish
> >>>>>>
> >>>>>> On Tue, 10 Mar 2020, Satish Balay via petsc-users wrote:
> >>>>>>
> >>>>>>> Cameron,
> >>>>>>>
> >>>>>>> You can try changing following petsc configure options and see if that
> >>>>>>> works.. [i.e build petsc manually]
> >>>>>>>
> >>>>>>> CFLAGS=-fopenmp FFLAGS=-fopenmp CXXFLAGS=-fopenmp
> >>>>>>>
> >>>>>>> Satish
> >>>>>>>
> >>>>>>> On Tue, 10 Mar 2020, Cameron Smith wrote:
> >>>>>>>
> >>>>>>>> Thank you Mark.
> >>>>>>>>
> >>>>>>>> The configure.log is attached.
> >>>>>>>>
> >>>>>>>> Please let me know if any other info is needed.
> >>>>>>>>
> >>>>>>>> -Cameron
> >>>>>>>>
> >>>>>>>> On 3/10/20 7:31 AM, Mark Adams wrote:
> >>>>>>>>> Hi Cameron,
> >>>>>>>>>
> >>>>>>>>> This can go on the list and we always want the configure.log file.
> >>>>>>>>>
> >>>>>>>>> I build on Summit, but have not used the XL compilers. I've built
> >>>>>>>>> 3.7.7
> >>>>>>>>> with
> >>>>>>>>> GNU and PGI. (XGC usually wants PGI)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Mar 9, 2020 at 11:27 PM Cameron Smith <smithc11 at rpi.edu
> >>>>>>>>> <mailto:smithc11 at rpi.edu>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hello,
> >>>>>>>>>
> >>>>>>>>> I'm installing petsc 3.7.7 on a summit like system with the
> >>>>>>>>> following
> >>>>>>>>> spack spec:
> >>>>>>>>>
> >>>>>>>>> petsc at 3.7.7 ~hdf5 ~hypre ~superlu-dist
> >>>>>>>>>
> >>>>>>>>> with the XL 16.1.1 compiler and Spectrum MPI 10.3 . This
> >>>>>>>>> install
> >>>>>>>>> produces a
> >>>>>>>>> `/path/to/petsc/install/lib/petsc/conf/petscvariables`
> >>>>>>>>> file
> >>>>>>>>> that contains '-lxlomp_ser' in the 'PETSC_EXTERNAL_LIB_BASIC'
> >>>>>>>>> and
> >>>>>>>>> 'PETSC_WITH_EXTERNAL_LIB' variables.
> >>>>>>>>>
> >>>>>>>>> The application I'm building, XGC, has a makefile based build
> >>>>>>>>> system
> >>>>>>>>> that includes
> >>>>>>>>> '/path/to/petsc/install/lib/petsc/conf/variables'
> >>>>>>>>> which in
> >>>>>>>>> turn includes '/lib/petsc/conf/petscvariables'.
> >>>>>>>>>
> >>>>>>>>> From what I can tell, xlomp_ser is a serial implementation
> >>>>>>>>> of
> >>>>>>>>> the
> >>>>>>>>> openmp library. When XGC links this library it satisfies the
> >>>>>>>>> openmp
> >>>>>>>>> symbols XGC wants and at run time results in openmp API calls
> >>>>>>>>> like
> >>>>>>>>> 'omp_get_max_threads()' returning 1 regardless of the
> >>>>>>>>> OMP_NUM_THREADS
> >>>>>>>>> setting.
> >>>>>>>>>
> >>>>>>>>> Do you know how I can build petsc, with or without spack, and
> >>>>>>>>> avoid
> >>>>>>>>> this
> >>>>>>>>> library being listed in 'lib/petsc/conf/petscvariables'?
> >>>>>>>>>
> >>>>>>>>> If this should go to a petsc mailing list or git repo issues
> >>>>>>>>> page
> >>>>>>>>> I
> >>>>>>>>> can
> >>>>>>>>> send it there.
> >>>>>>>>>
> >>>>>>>>> Thank-you,
> >>>>>>>>> Cameron
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>
> >>
>
More information about the petsc-users
mailing list