<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"Andale Mono";
panose-1:2 11 5 9 0 0 0 0 0 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Consolas",serif;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
p.p1, li.p1, div.p1
{mso-style-name:p1;
margin:0in;
background:black;
font-size:9.0pt;
font-family:"Andale Mono";
color:white;}
p.p2, li.p2, div.p2
{mso-style-name:p2;
margin:0in;
background:black;
font-size:9.0pt;
font-family:"Andale Mono";
color:#2FB41D;}
span.s1
{mso-style-name:s1;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1878470989;
mso-list-type:hybrid;
mso-list-template-ids:872828332 -1266141094 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">From:
</span></b><span style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Jed Brown <jed@jedbrown.org><br>
<b>Date: </b>Sunday, March 3, 2024 at 11:35 PM<br>
<b>To: </b>Zou, Ling <lzou@anl.gov>, Barry Smith <bsmith@petsc.dev><br>
<b>Cc: </b>petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov><br>
<b>Subject: </b>Re: [petsc-users] 'Preconditioning' with lower-order method<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">If you're having PETSc use coloring and have confirmed that the stencil is sufficient, then it would be nonsmoothness (again, consider the limiter you've chosen)
preventing quadratic convergence (assuming that doesn't kick in eventually). Note
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerStart<o:p></o:p></span></p>
</div>
<div style="border:none;border-top:solid #90A4AE 3.0pt;padding:0in 0in 0in 0in;display:block!important;text-align:left!important;margin:0px!important;padding:16px!important;border-radius:4px!important;min-width:200px!important;background-color:#D0D8DC!important;border-top:#90a4ae!important" id="pfptBannerdove87h">
<div id="pfptBannerdove87h">
<div id="pfptBannerdove87h">
<p class="MsoNormal" style="line-height:13.5pt;background:#D0D8DC"><b><span style="font-family:"Arial",sans-serif;color:black">This Message Is From an External Sender
<o:p></o:p></span></b></p>
</div>
<div id="pfptBannerdove87h">
<p class="MsoNormal" style="line-height:13.5pt;background:#D0D8DC"><span style="font-family:"Arial",sans-serif;color:black">This message came from outside your organization.
<o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal" style="background:#D0D8DC"><span style="color:black"> </span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerEnd<o:p></o:p></span></p>
</div>
<pre style="white-space:pre-wrap"><span style="font-size:11.0pt;font-family:"Arial",sans-serif">If you're having PETSc use coloring and have confirmed that the stencil is sufficient, then it would be nonsmoothness (again, consider the limiter you've chosen) preventing quadratic convergence (assuming that doesn't kick in eventually).<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre style="margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman""> </span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Yes, I do use coloring, and I do provide sufficient stencil, i.e., neighbor’s neighbor. The sufficiency is confirmed by PETSc’s <b><i>-snes_test_jacobian</i> </b>and <b><i>-snes_test_jacobian_view</i></b> options.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">Note that assembling a Jacobian of a second order TVD operator requires at least second neighbors while the first order needs only first neighbors, thus is much sparser and needs fewer colors to compute. <o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre style="margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman""> </span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">In my code implementation, when marking the Jacobian nonzero pattern, I don’t differentiate FV1 or FV2, I always use the FV2 stencil, so it’s a bit ‘fat’ for the FV1 method, but worked just fine.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">I expect you're either not exploiting that in the timings or something else is amiss. You can run with `-log_view -snes_view -ksp_converged_reason` to get a bit more information about what's happening. <o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre style="margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman""> </span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">The attached is screen output as you suggest. The linear and nonlinear performance of FV2 is both worse from the output.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">FV2:<o:p></o:p></span></pre>
<p class="p1"><span class="s1">Time Step 149, time = 13229.7, dt = 100</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">0, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">7.80968E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Linear solve converged due to CONVERGED_RTOL iterations 26</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">1, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">7.65731E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Linear solve converged due to CONVERGED_RTOL iterations 24</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">2, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">6.85034E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Linear solve converged due to CONVERGED_RTOL iterations 27</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">3, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">6.11873E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Linear solve converged due to CONVERGED_RTOL iterations 25</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">4, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">1.57347E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Linear solve converged due to CONVERGED_RTOL iterations 27</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">5, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">9.03536E-04</span><o:p></o:p></p>
<p class="p1"><span class="s1">SNES Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: newtonls</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">maximum iterations=20, maximum function evaluations=10000</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">tolerances: relative=1e-08, absolute=1e-06, solution=1e-08</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"><span style="color:yellow"> </span>
</span><span class="s1"><span style="color:yellow">total number of linear solver iterations=129</span></span><span style="color:yellow"><o:p></o:p></span></p>
<p class="p1"><span class="apple-converted-space"><span style="color:yellow"> </span>
</span><span class="s1"><span style="color:yellow">total number of function evaluations=144</span></span><span style="color:yellow"><o:p></o:p></span></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">norm schedule ALWAYS</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Jacobian is applied matrix-free with differencing</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Preconditioning Jacobian is built using finite differences with coloring</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">SNESLineSearch Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: bt</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">interpolation: cubic</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">alpha=1.000000e-04</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">maxstep=1.000000e+08, minlambda=1.000000e-12</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">maximum iterations=40</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">KSP Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: gmres</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">restart=100, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">happy breakdown tolerance 1e-30</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">maximum iterations=100, initial guess is zero</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">tolerances:</span><span class="apple-converted-space">
</span><span class="s1">relative=0.0001, absolute=1e-50, divergence=10000.</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">left preconditioning</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">using PRECONDITIONED norm type for convergence test</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">PC Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: ilu</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">out-of-place factorization</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">0 levels of fill</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">tolerance for zero pivot 2.22045e-14</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">using diagonal shift to prevent zero pivot [NONZERO]</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">matrix ordering: rcm</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">factor fill ratio given 1., needed 1.</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Factored matrix follows:</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Mat Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: seqaij</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">rows=8715, cols=8715</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">package used to perform factorization: petsc</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">total: nonzeros=38485, allocated nonzeros=38485</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">not using I-node routines</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">linear system matrix followed by preconditioner matrix:</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Mat Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: mffd</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">rows=8715, cols=8715</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Matrix-free approximation:</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">err=1.49012e-08 (relative error in function evaluation)</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Using wp compute h routine</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Does not compute normU</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Mat Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: seqaij</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">rows=8715, cols=8715</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">total: nonzeros=38485, allocated nonzeros=38485</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">total number of mallocs used during MatSetValues calls=0</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">not using I-node routines</span><o:p></o:p></p>
<p class="p2"><span class="apple-converted-space"> </span><span class="s1">Solve Converged!</span><o:p></o:p></p>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">FV1:<o:p></o:p></span></pre>
<p class="p1"><span class="s1">Time Step 149, time = 13229.7, dt = 100</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">0, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">7.90072E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Linear solve converged due to CONVERGED_RTOL iterations 12</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">1, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">2.01919E-04</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Linear solve converged due to CONVERGED_RTOL iterations 17</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">2, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">1.06960E-05</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Linear solve converged due to CONVERGED_RTOL iterations 15</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">3, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">2.41683E-09</span><o:p></o:p></p>
<p class="p1"><span class="s1">SNES Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: newtonls</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">maximum iterations=20, maximum function evaluations=10000</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">tolerances: relative=1e-08, absolute=1e-06, solution=1e-08</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"><span style="color:yellow"> </span>
</span><span class="s1"><span style="color:yellow">total number of linear solver iterations=44</span></span><span style="color:yellow"><o:p></o:p></span></p>
<p class="p1"><span class="apple-converted-space"><span style="color:yellow"> </span>
</span><span class="s1"><span style="color:yellow">total number of function evaluations=51</span></span><span style="color:yellow"><o:p></o:p></span></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">norm schedule ALWAYS</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Jacobian is applied matrix-free with differencing</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Preconditioning Jacobian is built using finite differences with coloring</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">SNESLineSearch Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: bt</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">interpolation: cubic</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">alpha=1.000000e-04</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">maxstep=1.000000e+08, minlambda=1.000000e-12</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">maximum iterations=40</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">KSP Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: gmres</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">restart=100, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">happy breakdown tolerance 1e-30</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">maximum iterations=100, initial guess is zero</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">tolerances:</span><span class="apple-converted-space">
</span><span class="s1">relative=0.0001, absolute=1e-50, divergence=10000.</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">left preconditioning</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">using PRECONDITIONED norm type for convergence test</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">PC Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: ilu</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">out-of-place factorization</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">0 levels of fill</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">tolerance for zero pivot 2.22045e-14</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">using diagonal shift to prevent zero pivot [NONZERO]</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">matrix ordering: rcm</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">factor fill ratio given 1., needed 1.</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Factored matrix follows:</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Mat Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: seqaij</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">rows=8715, cols=8715</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">package used to perform factorization: petsc</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">total: nonzeros=38485, allocated nonzeros=38485</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">not using I-node routines</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">linear system matrix followed by preconditioner matrix:</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Mat Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: mffd</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">rows=8715, cols=8715</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Matrix-free approximation:</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">err=1.49012e-08 (relative error in function evaluation)</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Using wp compute h routine</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Does not compute normU</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">Mat Object: 1 MPI process</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">type: seqaij</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">rows=8715, cols=8715</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">total: nonzeros=38485, allocated nonzeros=38485</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">total number of mallocs used during MatSetValues calls=0</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">not using I-node routines</span><o:p></o:p></p>
<p class="p2"><span class="apple-converted-space"> </span><span class="s1">Solve Converged!</span><o:p></o:p></p>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">"Zou, Ling via petsc-users" <petsc-users@mcs.anl.gov> writes:<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Barry, thank you.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> I am not sure if I exactly follow you on this:<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> “Are you forming the Jacobian for the first and second order cases inside of Newton?”<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> The problem that we deal with, heat/mass transfer in heterogeneous systems (reactor system), is generally small in terms of size, i.e., # of DOFs (several k to maybe 100k level), so for now, I completely rely on PETSc to compute Jacobian, i.e., finite-differencing.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> That’s a good suggestion to see the time spent during various events.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> What motivated me to try the options are the following observations.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> 2nd order FVM:<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Time Step 149, time = 13229.7, dt = 100<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> NL Step = 0, fnorm = 7.80968E-03<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> NL Step = 1, fnorm = 7.65731E-03<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> NL Step = 2, fnorm = 6.85034E-03<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> NL Step = 3, fnorm = 6.11873E-03<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> NL Step = 4, fnorm = 1.57347E-03<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> NL Step = 5, fnorm = 9.03536E-04<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Solve Converged!<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> 1st order FVM:<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Time Step 149, time = 13229.7, dt = 100<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> NL Step = 0, fnorm = 7.90072E-03<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> NL Step = 1, fnorm = 2.01919E-04<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> NL Step = 2, fnorm = 1.06960E-05<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> NL Step = 3, fnorm = 2.41683E-09<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Solve Converged!<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Notice the obvious ‘stagnant’ in residual for the 2nd order method while not in the 1st order.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> For the same problem, the wall time is 10 sec vs 6 sec. I would be happy if I can reduce 2 sec for the 2nd order method.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> -Ling<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> From: Barry Smith <bsmith@petsc.dev><o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Date: Sunday, March 3, 2024 at 12:06 PM<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> To: Zou, Ling <lzou@anl.gov><o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Cc: petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov><o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Subject: Re: [petsc-users] 'Preconditioning' with lower-order method<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Are you forming the Jacobian for the first and second order cases inside of Newton? You can run both with -log_view to see how much time is spent in the various events (compute function, compute Jacobian, linear solve, .. . ) for the two cases<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> ZjQcmQRYFpfptBannerStart<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> This Message Is From an External Sender<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> This message came from outside your organization.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> ZjQcmQRYFpfptBannerEnd<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Are you forming the Jacobian for the first and second order cases inside of Newton?<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> You can run both with -log_view to see how much time is spent in the various events (compute function, compute Jacobian, linear solve, ...) for the two cases and compare them.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> On Mar 3, 2024, at 11:42 AM, Zou, Ling via petsc-users <petsc-users@mcs.anl.gov> wrote:<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Original email may have been sent to the incorrect place.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> See below.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> -Ling<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> From: Zou, Ling <lzou@anl.gov<mailto:lzou@anl.gov>><o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Date: Sunday, March 3, 2024 at 10:34 AM<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> To: petsc-users <petsc-users-bounces@mcs.anl.gov<mailto:petsc-users-bounces@mcs.anl.gov>><o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Subject: 'Preconditioning' with lower-order method<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Hi all,<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> I am solving a PDE system over a spatial domain. Numerical methods are:<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> * Finite volume method (both 1st and 2nd order implemented)<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> * BDF1 and BDF2 for time integration.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> What I have noticed is that 1st order FVM converges much faster than 2nd order FVM, regardless the time integration scheme. Well, not surprising since 2nd order FVM introduces additional non-linearity.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> I’m thinking about two possible ways to speed up 2nd order FVM, and would like to get some thoughts or community knowledge before jumping into code implementation.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Say, let the 2nd order FVM residual function be F2(x) = 0; and the 1st order FVM residual function be F1(x) = 0.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> 1. Option – 1, multi-step for each time step<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Step 1: solving F1(x) = 0 to obtain a temporary solution x1<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Step 2: feed x1 as an initial guess to solve F2(x) = 0 to obtain the final solution.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> [Not sure if gain any saving at all]<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> 1. Option -2, dynamically changing residual function F(x)<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> In pseudo code, would be something like.<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> snesFormFunction(SNES snes, Vec u, Vec f, void *)<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> {<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> if (snes.nl_it_no < 4) // 4 being arbitrary here<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> f = F1(u);<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> else<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> f = F2(u);<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> }<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> I know this might be a bit crazy since it may crash after switching residual function, still, any thoughts?<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> Best,<o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Arial",sans-serif">> -Ling<o:p></o:p></span></pre>
</div>
</div>
</div>
</body>
</html>