[petsc-users] [petsc-maint] Speedup problem when using OpenMP?

[Karl Rupp]

Hi again,

as Jed noted, adding --with-threadcomm to configure finally gives you 
the OpenMP parallelization (--with-pthreadclasses also gives you pthread 
parallelization, but this may not be available on your Windows machine). 
With this I get the following execution times on my laptop for e.g. 
VecScale() for n=m=500:

Single-threaded: 9.3448e-01 sec
OpenMP:          4.8728e-01 sec
PThreads:        4.3991e-01 sec

This factor of two is fairly common for VecScale() and indicates a 
saturation of the memory bus. Other operations may saturate later.

Best regards,
Karli




On 11/05/2013 01:26 AM, Danyang Su wrote:
> Hi All,
>
> I have test the same example under Ubuntu12.04 X64.  The PETSc-dev
> version is update to date (GIT Date: 2013-11-01 14:59:20 -0500) and the
> installation is smooth without any error. The speedup of MPI version is
> linear scalable but the speedup of OpenMP version does not change. *From
> the CPU usage, the program still run in one thread when use OpenMP. *
>
> The commands to run the test are as follows:
>
> openmp
> ./ex2f -threadcomm_type openmp -threadcomm_nthreads 4 -m 1000 -n 1000
> -log_summary log_ex2f_1000x1000_ubuntu1204_omp_p4.log
>
> mpi
> mpiexec -n 4 ./ex2f -m 1000 -n 1000 -log_summary
> log_ex2f_1000x1000_ubuntu1204_mpi_p4.log
>
> This problem is so tricky to me. Can anybody confirm if KSP solver is
> parallelized for OpenMP version?
>
> Thanks and regards,
>
> Danyang
>
> On 31/10/2013 4:54 PM, Danyang Su wrote:
>> Hi All,
>>
>> I have a question on the speedup of PETSc when using OpenMP. I can get
>> good speedup when using MPI, but no speedup when using OpenMP.
>> The example is ex2f with m=100 and n=100. The number of available
>> processors is 16 (32 threads) and the OS is Windows Server 2012. The
>> log files for 4 and 8 processors are attached.
>>
>> The commands I used to run with 4 processors are as follows:
>> Run using MPI
>> mpiexec -n 4 Petsc-windows-ex2f.exe -m 100 -n 100 -log_summary
>> log_100x100_mpi_p4.log
>>
>> Run using OpenMP
>> Petsc-windows-ex2f.exe -threadcomm_type openmp -threadcomm_nthreads 4
>> -m 100 -n 100 -log_summary log_100x100_openmp_p4.log
>>
>> The PETSc used for this test is PETSc for Windows
>> http://www.mic-tc.ch/downloads/PETScForWindows.zip, but I guess this
>> is not the problem because the same problem exists when I use
>> PETSc-dev in Cygwin. I don't know if this problem exists in Linux,
>> would anybody help to test?
>>
>> Thanks and regards,
>>
>> Danyang
>