<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Jan:</div><div class="gmail_quote">I can reproduce reported behavior using</div><div class="gmail_quote">petsc/src/ksp/ksp/examples/tutorials/ex10.c on your mat.dat and rhs.dat.</div><div class="gmail_quote"><br></div><div class="gmail_quote">Using petsc sequential lu with default ordering 'nd', I get</div><div class="gmail_quote"><div class="gmail_quote">./ex10 -f0 mat.dat -rhs rhs.dat -pc_type lu</div><div class="gmail_quote">Number of iterations =   0</div><div class="gmail_quote">Residual norm 0.0220971</div><div class="gmail_quote"><br></div><div class="gmail_quote">Changing to</div><div class="gmail_quote">./ex10 -f0 mat.dat -rhs rhs.dat -pc_type lu -pc_factor_mat_ordering_type natural</div><div class="gmail_quote">Number of iterations =   1</div><div class="gmail_quote">  Residual norm < 1.e-12</div><div class="gmail_quote"><br></div><div class="gmail_quote">Back to superlu_dist, I get</div><div class="gmail_quote"><div class="gmail_quote">mpiexec -n 3 ./ex10 -f0 /homes/hzhang/tmp/mat.dat -rhs /homes/hzhang/tmp/rhs.dat -pc_type lu -pc_factor_mat_solver_package superlu_dist</div><div class="gmail_quote">Number of iterations =   4</div><div class="gmail_quote">Residual norm 25650.8</div><div class="gmail_quote"><br></div><div class="gmail_quote">which uses default ordering  (-ksp_view)</div><div class="gmail_quote"><div class="gmail_quote">Row permutation LargeDiag</div><div class="gmail_quote">Column permutation METIS_AT_PLUS_A</div><div class="gmail_quote"><br></div><div class="gmail_quote">Run it with</div><div class="gmail_quote"><div class="gmail_quote">mpiexec -n 3 ./ex10 -f0 mat.dat -rhs rhs.dat -pc_type lu -pc_factor_mat_solver_package superlu_dist -mat_superlu_dist_rowperm NATURAL -mat_superlu_dist_colperm NATURAL</div><div class="gmail_quote">Number of iterations =   1</div><div class="gmail_quote">  Residual norm < 1.e-12</div><div class="gmail_quote"><br></div><div class="gmail_quote">i.e., your problem is sensitive to matrix ordering, which I do not know why.<br></div><div class="gmail_quote"><br></div><div class="gmail_quote">I checked condition number of your mat.dat using superlu:</div><div class="gmail_quote"><div class="gmail_quote">./ex10 -f0 mat.dat -rhs rhs.dat -pc_type lu -pc_factor_mat_solver_package superlu -mat_superlu_conditionnumber</div><div class="gmail_quote">  Recip. condition number = 1.137938e-03</div><div class="gmail_quote">Number of iterations =   1</div><div class="gmail_quote">  Residual norm < 1.e-12</div></div></div></div></div><div class="gmail_quote"><br></div><div class="gmail_quote">As you see, matrix is well-conditioned. Why is it so sensitive to matrix ordering?</div><div class="gmail_quote"><br></div><div class="gmail_quote">Hong</div><div class="gmail_quote"><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Using attached petsc4py code, matrix and right-hand side, SuperLU_dist<br>
returns totally wrong solution for mixed Laplacian:<br>
<br>
$ tar -xzf report.tar.gz<br>
$ python test-solve.py -pc_factor_mat_solver_package mumps -ksp_final_residual<br>
KSP final norm of residual 3.81865e-15<br>
$ python test-solve.py -pc_factor_mat_solver_package umfpack -ksp_final_residual<br>
KSP final norm of residual 3.68546e-14<br>
$ python test-solve.py -pc_factor_mat_solver_package superlu_dist -ksp_final_residual<br>
KSP final norm of residual 1827.72<br>
<br>
Moreover final residual is random when run using mpirun -np 3. Maybe<br>
a memory corruption issue? This is reproducible using PETSc 3.6.2 (and<br>
SuperLU_dist configured by PETSc) and much older, see<br>
<a href="http://fenicsproject.org/pipermail/fenics-support/2014-March/000439.html" rel="noreferrer" target="_blank">http://fenicsproject.org/pipermail/fenics-support/2014-March/000439.html</a><br>
but has never been reported upstream.<br>
<br>
The code for assembling the matrix and rhs using FEniCS is also<br>
included for the sake of completeness.<br>
<span class=""><font color="#888888"><br>
Jan<br>
</font></span></blockquote></div><br></div></div>