<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hei,</p>
<p>when compiling the attached files using the following compilation
line</p>
<p><i>/usr/lib64/mpi/gcc/openmpi3/bin/mpicxx -DBOOST_ALL_NO_LIB
-DBOOST_FILESYSTEM_DYN_LINK -DBOOST_MPI_DYN_LINK
-DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_SERIALIZATION_DYN_LINK
-DUSE_CUDA
-I/home/roland/Dokumente/C++-Projekte/armadillo_with_PETSc/include
-I/opt/intel/compilers_and_libraries_2020.2.254/linux/mkl/include
-I/opt/armadillo/include -isystem /opt/petsc_release/include
-isystem /opt/fftw3/include -isystem /opt/boost/include
-march=native -fopenmp-simd -DMKL_LP64 -m64 -Wall -Wextra
-pedantic -fPIC -flto -O2 -funroll-loops -funroll-all-loops
-fstrict-aliasing -mavx -march=native -fopenmp -std=gnu++17 -c
<source_files> -o <target_files></i></p>
<p>and linking them with </p>
<p><i>/usr/lib64/mpi/gcc/openmpi3/bin/mpicxx -march=native
-fopenmp-simd -DMKL_LP64 -m64 <target_files> -o
bin/armadillo_with_PETSc
-Wl,-rpath,/opt/boost/lib:/opt/fftw3/lib64:/opt/petsc_release/lib
/usr/lib64/libgsl.so /usr/lib64/libgslcblas.so -lgfortran
/opt/intel/mkl/lib/intel64/libmkl_rt.so
/opt/boost/lib/libboost_filesystem.so.1.72.0
/opt/boost/lib/libboost_mpi.so.1.72.0
/opt/boost/lib/libboost_program_options.so.1.72.0
/opt/boost/lib/libboost_serialization.so.1.72.0
/opt/fftw3/lib64/libfftw3.so /opt/fftw3/lib64/libfftw3_mpi.so
/opt/petsc_release/lib/libpetsc.so
/usr/lib64/gcc/x86_64-suse-linux/9/libgomp.so</i></p>
<p>my output is </p>
<p><i>Arma and PETSc/MatScale are
equal: 0</i><i><br>
</i><i>Arma-time for a matrix size of [1024,
8192]: 24955</i><i><br>
</i><i>PETSc-time, pointer for a matrix size of [1024,
8192]: 28283</i><i><br>
</i><i>PETSc-time, MatScale for a matrix size of [1024,
8192]: 23138</i></p>
<p>but when removing the explicit call to openmp (i.e. removing <i>-fopenmp</i>
and <i>/usr/lib64/gcc/x86_64-suse-linux/9/libgomp.so</i>) my
result is <br>
</p>
<p><i>Arma and PETSc/MatScale are
equal: 1</i><i><br>
</i><i>Arma-time for a matrix size of [1024,
8192]: 24878</i><i><br>
</i><i>PETSc-time, pointer for a matrix size of [1024,
8192]: 18942</i><i><br>
</i><i>PETSc-time, MatScale for a matrix size of [1024,
8192]: 23350</i></p>
<p>even though both times the executable is linked to <br>
</p>
<p><i> libmkl_intel_lp64.so =>
/opt/intel/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64_lin/libmkl_intel_lp64.so
(0x00007f9eebd70000)</i><i><br>
</i><i> libmkl_core.so =>
/opt/intel/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64_lin/libmkl_core.so
(0x00007f9ee77aa000)</i><i><br>
</i><i> libmkl_intel_thread.so =>
/opt/intel/compilers_and_libraries_2020.2.254/linux/mkl/lib/intel64_lin/libmkl_intel_thread.so
(0x00007f9ee42c3000)</i><i><br>
</i><i> libiomp5.so =>
/opt/intel/compilers_and_libraries_2020.2.254/linux/compiler/lib/intel64_lin/libiomp5.so
(0x00007f9ee3ebd000)</i><i><br>
</i><i> libgomp.so.1 => /usr/lib64/libgomp.so.1
(0x00007f9ea98bd000)</i></p>
<p>via the petsc-library. Why does the execution time vary by so
much, and why does my result change when calling MatScale (i.e.
returning wrong results) when explicitly linking to OpenMP? <br>
</p>
<p>Thanks!</p>
<p>Regards,</p>
<p>Roland<br>
</p>
</body>
</html>