[petsc-dev] [petsc-users] Using PETSC with an openMP program

[Jed Brown]

Adrián Amor <aamor at pa.uc3m.es> writes:

> 1. I have compiled PETSC with these options:
> ./configure --with-cc=mpiicc --with-cxx=mpiicpc --with-fc=mpiifort
> --with-blas-lapack-dir=/opt/intel/mkl/lib/intel64/ --with-debugging=1
> --with-scalar-type=complex --with-threadcomm --with-pthreadclasses
> --with-openmp
> --with-openmp-include=/opt/intel/compilers_and_libraries_2016.1.150/linux/compiler/lib/intel64_lin
> --with-openmp-lib=/opt/intel/compilers_and_libraries_2016.1.150/linux/compiler/lib/intel64_lin/libiomp5.a

Matt, --with-threadcomm and --with-pthreadclasses were removed in 2015,
but people still use these options and think they are doing something
because configure doesn't even mention unused options.  We've been
talking about fixing this for at least a decade, and yet here we still
are.  Things like this get fixed in the C code, so it can't be pure
laziness.  Python is supposed to be easy, but this part of BuildSystem
is evidently scary.  I have no idea if fixing this would be a 2-hour job
or if it would snowball into more than 20.  It's hard to estimate time
to make cross-cutting changes, but in my experience, BuildSystem is
especially hard to make estimates about.  I spent an hour or two trying
to assess this particular issue and came up empty-handed some years ago.
I've had several other false starts over the years, trying to make minor
improvements to BuildSystem and finding a web of subtle dependencies.
The PETSc library tends to have well-documented interfaces with minimal
side-effects, but BuildSystem has a lot of similarly-named undocumented
(or inaccurately-documented) interfaces with implications that are hard
to reason about.  I don't have an immediate solution, but this technical
debt is costly.  It works, in many ways functionally better than
alternatives, but it's really hard to work on.