<div dir="ltr">I actually was able to solve my own problem...for some reason, I need to do <div><br><div><div>PCSetType(pc, PCLU);</div><div>PCFactorSetMatSolverPackage(pc, MATSOLVERSUPERLU_DIST);</div><div>KSPSetTolerances(ksp, 1.e-15, PETSC_DEFAULT, PETSC_DEFAULT, PETSC_DEFAULT);</div>
</div></div><div><br></div><div>instead of the ordering I initially had, though I'm not really clear on what the issue was. However, there seems to be some loss of accuracy as I increase the number of processes. Is this expected, or can I force a lower tolerance somehow? I am able to compare the solutions to a reference solution, and the error increases as I increase the processes. This is the solution in sequential:</div>
<div><br></div><div><div>superlu_1process = [</div><div>-6.8035811950925553e-06</div><div>1.6324030474375778e-04</div><div>5.4145340579614926e-02</div><div>1.6640521936281516e-04</div><div>-1.7669374392923965e-04</div><div>
-2.8099208957838207e-04</div><div>5.3958133511222223e-02</div><div>-5.4077899123806263e-02</div><div>-5.3972905090366369e-02</div><div>-1.9485020474821160e-04</div><div>5.4239813043824400e-02</div><div>4.4883984259948430e-04];</div>
</div><div><br></div><div><div>superlu_2process = [</div><div>-6.8035811950509821e-06</div><div>1.6324030474371623e-04</div><div>5.4145340579605655e-02</div><div>1.6640521936281687e-04</div><div>-1.7669374392923807e-04</div>
<div>-2.8099208957839834e-04</div><div>5.3958133511212911e-02</div><div>-5.4077899123796964e-02</div><div>-5.3972905090357078e-02</div><div>-1.9485020474824480e-04</div><div>5.4239813043815172e-02</div><div>4.4883984259953320e-04];</div>
</div><div><br></div><div>superlu_4process= [<br></div><div><div>-6.8035811952565206e-06</div><div>1.6324030474386164e-04</div><div>5.4145340579691455e-02</div><div>1.6640521936278326e-04</div><div>-1.7669374392921441e-04</div>
<div>-2.8099208957829171e-04</div><div>5.3958133511299078e-02</div><div>-5.4077899123883062e-02</div><div>-5.3972905090443085e-02</div><div>-1.9485020474806352e-04</div><div>5.4239813043900860e-02</div><div>4.4883984259921287e-04];</div>
</div><div><br></div><div>This is some finite element solution and I can compute the error of the solution against an exact solution in the functional L2 norm:</div><div><br></div><div>error with 1 process: 1.71340e-02 (accepted value)</div>
<div>error with 2 processes: 2.65018e-02 </div><div>error with 3 processes: 3.00164e-02 </div>
<div>error with 4 processes: 3.14544e-02 </div><div><br></div><div><br></div><div>Is there a way to remedy this?</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 30, 2014 at 2:37 AM, Justin Dong <span dir="ltr"><<a href="mailto:jsd1@rice.edu" target="_blank">jsd1@rice.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi,<div><br></div><div>I'm trying to solve a linear system in parallel using SuperLU but for some reason, it's not giving me the correct solution. I'm testing on a small example so I can compare the sequential and parallel cases manually. I'm absolutely sure that my routine for generating the matrix and right-hand side in parallel is working correctly.</div>
<div><br></div><div>Running with 1 process and PCLU gives the correct solution. Running with 2 processes and using SUPERLU_DIST does not give the correct solution (I tried with 1 process too but according to the superlu documentation, I would need SUPERLU for sequential?). This is the code for solving the system:</div>
<div><br></div><div><div> /* solve the system */</div><div><span style="white-space:pre-wrap"> </span>KSPCreate(PETSC_COMM_WORLD, &ksp);</div><div><span style="white-space:pre-wrap"> </span>KSPSetOperators(ksp, Aglobal, Aglobal, DIFFERENT_NONZERO_PATTERN);</div>
<div><span style="white-space:pre-wrap"> </span>KSPSetType(ksp,KSPPREONLY);</div><div><br></div><div><span style="white-space:pre-wrap"> </span>KSPGetPC(ksp, &pc);</div><div><br></div><div><span style="white-space:pre-wrap"> </span>KSPSetTolerances(ksp, 1.e-13, PETSC_DEFAULT, PETSC_DEFAULT, PETSC_DEFAULT);</div>
<div><span style="white-space:pre-wrap"> </span>PCFactorSetMatSolverPackage(pc, MATSOLVERSUPERLU_DIST);</div><div><br></div><div><span style="white-space:pre-wrap"> </span>KSPSolve(ksp, bglobal, bglobal);</div></div>
<div><br></div><div>Sincerely,</div><div>Justin</div><div><br></div><div><br></div></div>
</blockquote></div><br></div>