[petsc-users] Documentation for different parallelization options

[Moritz Cygorek]

Hi everyone,

I'm looking for a document/tutorial/howto that describes the different options to compile PETSc with parallelization.

My problem is the following: 
I'm trying to solve a large sparse eigenvalue problem using the Krylov-Schur method implemented in SLEPc
When I install SLEPc/PETSc on my Ubuntu laptop via apt-get, everything works smoothly and parallelization works automatically.
I see this by the fact that the CPU-load of the process (only one process, not using mpiexec) is close to 400% according to "top"
Therefore, it seems that OpenMP is used.

I have access to better computers and I would like to install SLEPc/PETSc there, but I have to configure it manually.
I have tried different options, none of the satisfactory:

When I compile PETSc with the --with-openmp flag, I see that the program never runs with cpu load above 100%.
I use the same command to call the program as on my laptop where everything works. So it seems that openmp is somehow not activated.
An old mailing list entry says that I am supposed to configure PETSc using --with-threadcomm --with-openmp, which I did, but it also didn't help.
However that entry was from 2014 and I found in the list of changes for PETSc in version 3.6:
"Removed all threadcomm support including --with-pthreadclasses and --with-openmpclasses configure arguments"

Does that mean that openmp is no longer supported in newer versions?


Given my resources, I would prefer OpenMP over MPI. Nevertheless, I then spent some time to go full MPI without openmp and to split up the sparse matrix across several processes. When I start the program using mpiexec,
I see indeed that multiple processes are started, but even when I use 12 processes, the computation time is about the same as with only 1 process.
Is there anything I have to tell the EPS solver to activate parallelization?


So, all in all, I can't get to run anything faster on a large multi-core computer than on my old crappy laptop. 


I have no idea how to start debugging and assessing the performance and the documentation on this issue on the website is not very verbose. 
Can you give me a few hints?

Regards,
Moritz