<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, Nov 7, 2017 at 10:55 AM, Bernardo Rocha <span dir="ltr"><<a href="mailto:bernardomartinsrocha@gmail.com" target="_blank">bernardomartinsrocha@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:arial,helvetica,sans-serif;font-size:small">Thanks for the reply.</div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>1) This is block-Jacobi, why not use PCBJACOBI? Is it because you want to select rows?</div></div></div></div></blockquote><div><br></div></span><div style="font-family:arial,helvetica,sans-serif;font-size:small">I'm only using it to understand the performance behavior of PCFieldSplit since I'm also</div><div style="font-family:arial,helvetica,sans-serif;font-size:small">having the same issue in a large and more complex problem.</div><span class="gmail-"><div style="font-family:arial,helvetica,sans-serif;font-size:small">​</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>2) We cannot tell anything without knowing how many iterates were used</div></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>  -ksp_monitor_true_residual -ksp_converged_reason -pc_fieldsplit_[0,1]_ksp_monit<wbr>or_true_residual</div><div><br></div><div>3) We cannot say anything about performance without seeing the log for both runs</div><div>  -log_view</div></div></div></div></blockquote><div><br></div></span><div><div style="font-family:arial,helvetica,sans-serif;font-size:small">I'm sending to you the log files with the recommended command line arguments for the three cases.</div></div></div></div></div></blockquote><div><br></div><div>You did not print out the iterates for the field split solves:</div><div><br></div><div>  -pc_fieldsplit_[0,1]_ksp_monit<wbr>or_true_residual</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div style="font-family:arial,helvetica,sans-serif;font-size:small">1-scalar case</div><div style="font-family:arial,helvetica,sans-serif;font-size:small">2-PCFieldSplit (as we were initially running)</div><div style="font-family:arial,helvetica,sans-serif;font-size:small">3-PCFieldSplit with Preonly/Jacobi in each block, as suggested by Patrick.</div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div><span style="font-size:12.8px">As Patrick pointed out, with Preonly/Jacobi the behavior is closer to what I expected.</span></div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">Please note that the log was taken for 100 calls to KSPSolve, I just simplified it.</div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">What would be the proper way of creating this block preconditioner <br><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">As you can see, the timing with PCFieldSplit is bigger for case 3.<br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">For case 2 it is nearly 2x, as I expected (I don't know if this idea makes sense).</div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">So for the case 2, the reason for the large timing is due to the inner/outer solver?</div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div></div><div><div style="font-family:arial,helvetica,sans-serif;font-size:small">​Does the "machinery" behind the PCFieldSplit for a block preconditioner results</div><div style="font-family:arial,helvetica,sans-serif;font-size:small">in some performance overhead? (neglecting the efficiency of the PC itself)</div></div></div></div></div></blockquote><div><br></div><div>No.</div><div><br></div><div>What you sent makes little sense to me. How do you have 29 iterates for the solve, but 5900<br></div><div>MatMults in the log?</div><div><br></div><div>The number of MatMults in 2) is 4x, not 2x. I suspect that is because convergence of the Krylov</div><div>solver on each block takes the same number of iterates that your global Krylov solver takes. Thus</div><div>you have 2x, but you do 2 outer iterates, which is 4x. Thus you get your 2x time.</div><div><br></div><div>You could try to play games with the inner tolerances (say run the blocks only to 10^-4). However,</div><div>the fact remains that this is not even a credible solver for the problem.</div><div><br></div><div>  Thanks,</div><div><br></div><div>     Matt</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div style="font-family:arial,helvetica,sans-serif;font-size:small">Best regards,</div><div style="font-family:arial,helvetica,sans-serif;font-size:small">Bernardo​</div><br></div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.caam.rice.edu/~mk51/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div>
</div></div>