<div dir="ltr">Very good point. Thank you for the clarification. <br></div><br><div class="gmail_quote"><div dir="ltr">On Sun, Jun 24, 2018 at 8:46 PM Mark Adams <<a href="mailto:mfadams@lbl.gov">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Note, you converged because of the relative tolerance so initial guess does not really matter except that the initial residual on the second time step may have more high frequency content then your first initial residual.</div><br><div class="gmail_quote"><div dir="ltr">On Sun, Jun 24, 2018 at 6:37 AM Najeeb Ahmad <<a href="mailto:nahmad16@ku.edu.tr" target="_blank">nahmad16@ku.edu.tr</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Massimiliano,<div><br></div><div>Thanks for your feedback. Indeed i am using previous solution for the subsequent ones and probably that is the reason for the faster convergence for the subsequent iterations. As Matt suggested, i need to improve my preconditioning to make first iteration converge faster.</div><div><br></div><div>Thanks</div></div><br><div class="gmail_quote"><div dir="ltr">On Sun, Jun 24, 2018 at 3:26 PM Massimiliano Leoni <<a href="mailto:leoni.massimiliano1@gmail.com" target="_blank">leoni.massimiliano1@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>
<div style="font-family:'Noto Sans';font-size:10pt;font-weight:400;font-style:normal">
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">Hi, are the various solves a time stepping procedure? Could it be that it is re-using the solution at a previous iteration as an initial guess for the subsequent ones, and for this reason it converges sooner?</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">Massimiliano</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">On sabato 23 giugno 2018 13:36:59 CEST Najeeb Ahmad wrote:<br></p>
<p style="margin-top:12px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">Hi,</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><br></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">I am trying to solve a  <span style="background-color:#ffffff">64008000</span> x  <span style="background-color:#ffffff">64008000</span> system using PETSc. The problem I am facing is that I am getting a very large iteration count for the first solution (order of thousands). For the subsequent soluitons, the iteration count is less. For instance, here is the iteration count for the first solution when my code is run on 32 processes.</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><br></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">...</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">2772 KSP Residual norm 1.198554448053e-07</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">2773 KSP Residual norm 1.190214006391e-07</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">2774 KSP Residual norm 1.181931531360e-07</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">2775 KSP Residual norm 1.173706620648e-07</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">2776 KSP Residual norm 1.165538874733e-07</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">2777 KSP Residual norm 1.157427896857e-07</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">2778 KSP Residual norm 1.149373293013e-07</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">2779 KSP Residual norm 1.141374671925e-07</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">2780 KSP Residual norm 1.133431645025e-07</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">Line<span style="background-color:#ffffff">ar solve converged due to CONVERGED_RTOL iterations 2780</span> </p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><br></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">For the subsequent solutions, the iteration count is as follows:</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">...</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">116 KSP Residual norm 2.657138978992e-04</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">117 KSP Residual norm 2.767485246885e-04</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">118 KSP Residual norm 2.071574619161e-04</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">119 KSP Residual norm 2.273562103015e-04</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">120 KSP Residual norm 1.693847273193e-04</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">121 KSP Residual norm 1.803833437167e-04</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">122 KSP Residual norm 1.443434809819e-04</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">123 KSP Residual norm 1.390878278231e-04</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">124 KSP Residual norm 1.247521972621e-04</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">125 KSP Residual norm 1.059224678494e-04</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">Linear solve converged due to CONVERGED_RTOL iterations 125</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><br></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">The solver setup for my code is as follows:<br></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><br></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"> ierr = KSPCreate(PETSC_COMM_WORLD,&ksp);CHKERRQ(ierr);</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">  ierr = KSPSetOperators(ksp,A,A);CHKERRQ(ierr);</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">  ierr = KSPSetType(ksp, KSPCG);CHKERRQ(ierr);</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">  ierr = KSPGetPC(ksp,&pc);CHKERRQ(ierr);</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">  ierr = PCSetType(pc, PCJACOBI);</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">  ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr);</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">  ierr = KSPSetUp(ksp);CHKERRQ(ierr);</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><br></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">and the command line for running the code is:</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">mpiexec -n 32 ./main -mat_type mpiaij -ksp_atol 1E-7 -ksp_rtol 1E-7 -ksp_converged_reason -ksp_monitor</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><br></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px">Any help is highly appreciated.</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><br></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><br>-- <br></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><span style="font-size:small;font-weight:600;font-style:italic;color:#073763">Najeeb Ahmad</span></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><span style="font-size:small;font-weight:600;font-style:italic;color:#073763"><br></span></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><span style="font-size:small;font-weight:600;color:#783f04">Research and Teaching Assistant</span><span style="font-size:small;font-weight:600"><br></span></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><span style="font-size:small;font-weight:600;color:#000000">PARallel and MultiCORE Computing Laboratory (ParCoreLab)</span><span style="font-size:small;font-weight:600;color:#783f04"> </span></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><span style="font-size:small;font-weight:600">Computer Science and Engineering</span></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><span style="font-size:small;font-weight:600"><br></span></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><span style="font-size:small;font-weight:600">Koç University, Istanbul, Turkey</span></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:40px;margin-right:40px;text-indent:0px"><span style="font-size:small;font-weight:600"><br></span></p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><br><br></p></div></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_3911207778986238191m_-8443747988670971444gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div><div></div><font size="2"><b><span style="color:rgb(7,55,99)"><i>Najeeb Ahmad</i></span></b></font></div><div><font size="2"><b><font color="#073763"><i><br></i></font></b></font><div dir="ltr" style="font-size:small"><div><font size="2"><b><span style="color:rgb(120,63,4)">Research and Teaching Assistant</span><br></b></font></div><div><font size="2"><b><font color="#000000">PARallel and MultiCORE Computing Laboratory (ParCoreLab)</font><span style="color:rgb(120,63,4)"> </span></b></font></div><font size="2"><b>Computer Science and Engineering<br></b></font></div><div style="font-size:small"><font size="2"><b>Koç University, Istanbul, Turkey</b></font></div></div></div></div><div><font size="2"><b><br></b></font></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div><div></div><font size="2"><b><span style="color:rgb(7,55,99)"><i>Najeeb Ahmad</i></span></b></font></div><div><font size="2"><b><font color="#073763"><i><br></i></font></b></font><div dir="ltr" style="font-size:small"><div><font size="2"><b><span style="color:rgb(120,63,4)">Research and Teaching Assistant</span><br></b></font></div><div><font size="2"><b><font color="#000000">PARallel and MultiCORE Computing Laboratory (ParCoreLab)</font><span style="color:rgb(120,63,4)"> </span></b></font></div><font size="2"><b>Computer Science and Engineering<br></b></font></div><div style="font-size:small"><font size="2"><b>Koç University, Istanbul, Turkey</b></font></div></div></div></div><div><font size="2"><b><br></b></font></div></div></div></div></div></div></div></div>