[petsc-dev] problem with hypre with '--with-openmp=1'

[Jed Brown]

Mark Adams <mfadams at lbl.gov> writes:

> On Tue, Jun 26, 2018 at 1:06 PM Satish Balay <balay at mcs.anl.gov> wrote:
>
>> I wonder if these jobs are scheduled in such a way so that they are not
>> oversubscribed.
>>
>> i.e number_mpi_jobs_per_node * number_of_openmp_threads_per_node <=
>> no_of_cores_per_node
>
>
> You are right(ish). This fixed the problem:
>
> export OMP_NUM_THREADS=1

This is a frequent cause of performance woes.  I think PETSc should do
some checking for oversubscription, perhaps using omp_num_threads() and
MPI_Comm_split_type(MPI_COMM_TYPE_SHARED) along with

https://stackoverflow.com/questions/150355/programmatically-find-the-number-of-cores-on-a-machine#150971

Such oversubscription could be reported prominently in -log_view.