<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div dir="ltr">
<div dir="ltr"><br>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Jan 25, 2019 at 8:07 PM Mohammad Gohardoust via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div dir="ltr">
<div style="color:rgb(103,78,167)"></div>
<div style="color:rgb(103,78,167)">Hi,</div>
<div style="color:rgb(103,78,167)"><br>
</div>
<div style="color:rgb(103,78,167)">I am trying to modify a "pure MPI" code for solving water movement equation in soils which employs KSP iterative solvers. This code gets really slow in the hpc I am testing it as I increase the number of calculating nodes
(each node has 28 cores) even from 1 to 2. I went for implementing "MPI-OpenMP" solutions like MUMPS. I did this inside the petsc by:</div>
<div style="color:rgb(103,78,167)"><br>
</div>
<div style="color:rgb(103,78,167);margin-left:40px">KSPSetType(ksp, KSPPREONLY);</div>
<div style="color:rgb(103,78,167);margin-left:40px">PCSetType(pc, PCLU);</div>
<div style="color:rgb(103,78,167);margin-left:40px">PCFactorSetMatSolverType(pc, MATSOLVERMUMPS);</div>
<div style="color:rgb(103,78,167);margin-left:40px">KSPSolve(ksp, ... <br>
</div>
<div style="color:rgb(103,78,167);margin-left:40px"><br>
</div>
<div style="color:rgb(103,78,167)">and I run it through:</div>
<div style="color:rgb(103,78,167)"><br>
</div>
<div style="color:rgb(103,78,167);margin-left:40px">export OMP_NUM_THREADS=16 && mpirun -n 2 ~/Programs/my_programs<br>
<br>
</div>
</div>
</div>
</blockquote>
<div>On some cases, I saw multithreaded MUMPS could improve its performance about 30%. I guess something was wrong in your tests. You need to compile MUMPS with OpenMP support. If you installed MUMPS through PETSc, you need petsc configure option --with-openmp=1.
In addition, you need a multithreaded BLAS. You can do it through --download-openblas</div>
<div><br>
</div>
<div>But first of all, you should add --log_view to report your performance results. </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div dir="ltr">
<div style="color:rgb(103,78,167);margin-left:40px"></div>
<div style="color:rgb(103,78,167)"></div>
<div style="color:rgb(103,78,167)">The code is working (in my own PC) but it is too slow (maybe about 50 times slower). Since I am not an expert, I like to know is this what I should expect from MUMPS!?</div>
<div style="color:rgb(103,78,167)"><br>
</div>
<div style="color:rgb(103,78,167)">Thanks,</div>
<div style="color:rgb(103,78,167)">Mohammad<br>
</div>
<div style="color:rgb(103,78,167)"></div>
<div style="color:rgb(103,78,167)"></div>
<br>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>