[petsc-users] Configuration of Hybrid MPI-OpenMP

[Danyang Su]

Hi All,

Is it possible to configure PETSc with support for hybrid MPI-OpenMP?
My codes have a lot of OpenMP instructions and work well. I have 
implemented PETSc solver in the codes and the solver can also work and 
help to speedup. Currently I have the following question:

1. If I configure PETSc with OpenMP, the OpenMP instructions work well 
and I can see the significant speedup in this part, but the speedup for 
the solver is very small or no speedup. I have asked the question 
before, maybe this is due to the bus bandwidth. Not sure on it.

2. If I configure PETSc with MPI, the speedup for the solver is 
significant, but the OpenMP instructions cannot work. The section with 
OpenMP instructions runs even 10 times slower.

I tried to configure PETSc with both MPI and OpenMP enabled, but failed.

BTW: I have also tried to use "PETSc for WINDOWS2.0" 
(http://www.msic.ch/Software). Both the OpenMP instructions and PETSc 
solver (MPI) can work well, but the solver speedup is not as significant 
as compiling in CYGWIN with PETSc 3.4.3. Usually the speedup of solver 
is 1.5 times slower.

Is it possible to configure PETSc with Hybrid MPI-OPenMP? How?

Thanks and regards,

Danyang