Enabling OpenMP in Petsc

[Albert Hartono]

Hi,

Could someone explain how to build Petsc to enable OpenMP?

I want to include OpenMP directives in src/mat/impls/aij/seq/inode.c,
but I cannot figure out how to build it correctly so that OpenMP
parallelization can be used correctly on the BGP surveyor machine.

What I did is this:

1. I added #include <omp.h> in src/mat/impls/aij/seq/inode.c

2. I added a simple OpenMP test (shown below) in one of the functions in
src/mat/impls/aij/seq/inode.c

ierr=PetscPrintf(PETSC_COMM_WORLD,"***************************\n");
omp_set_num_threads(4);
int nthreads, tid;
#pragma omp parallel private(nthreads, tid)
 {
   tid = omp_get_thread_num();
   ierr=PetscPrintf(PETSC_COMM_WORLD,"Hello from thread = %d\n",tid);
   if (tid == 0) {
     nthreads = omp_get_num_threads();
     ierr=PetscPrintf(PETSC_COMM_WORLD,"Number of threads =
%d\n",nthreads);
   }
 }
ierr=PetscPrintf(PETSC_COMM_WORLD,"***************************\n");

3. To activate the OpenMP test, I used snes/examples/tutorials/ex27.c.
I modified the makefile in the tutorials directory as follows.

line 6:   CFLAGS           = -qsmp=omp
line 124: ex27: ex27.o chkopts
            -${CLINKER} -o ex27 ex27.o ${PETSC_SNES_LIB} -qsmp=omp
            ${RM} ex27.o

4. Then I was able to rebuild the inode.c and ex27 with no errors. And
then run ex27 executable on BGP surveyor machine.

And below is the results.

...
***************************
Hello World from thread = 0
Number of threads = 1
***************************
***************************
Hello World from thread = 0
Number of threads = 1
***************************
...

The results showed the OpenMP problem: only ONE thread was used (although
it was explicitly set to use four threads).

Note that I didn't include -qsmp=omp in the makefile in
src/mat/impls/aij/seq/ since it caused a multiple-definition error problem
(the link was done twice). It may have been linked statically there.

Helps will be greatly appreciated.

Thanks,
--Albert