<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Monaco;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Andale Mono";
panose-1:2 11 5 9 0 0 0 0 0 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
p.p1, li.p1, div.p1
{mso-style-name:p1;
margin:0in;
background:black;
font-size:9.0pt;
font-family:"Andale Mono";
color:white;}
p.p2, li.p2, div.p2
{mso-style-name:p2;
margin:0in;
background:black;
font-size:9.0pt;
font-family:"Andale Mono";
color:#2FB41D;}
span.s1
{mso-style-name:s1;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:262609627;
mso-list-template-ids:360095740;}
@list l1
{mso-list-id:839589606;
mso-list-template-ids:1241833920;}
@list l1:level1
{mso-level-start-at:2;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2
{mso-list-id:1934123224;
mso-list-template-ids:-17144466;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Barry, thank you.<o:p></o:p></p>
<p class="MsoNormal">I am not sure if I exactly follow you on this:<o:p></o:p></p>
<p class="MsoNormal">“Are you forming the Jacobian for the first and second order cases inside of Newton?”<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The problem that we deal with, heat/mass transfer in heterogeneous systems (reactor system), is generally small in terms of size, i.e., # of DOFs (several k to maybe 100k level), so for now, I completely rely on PETSc to compute Jacobian,
i.e., finite-differencing.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">That’s a good suggestion to see the time spent during various events.<o:p></o:p></p>
<p class="MsoNormal">What motivated me to try the options are the following observations.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">2<sup>nd</sup> order FVM:<o:p></o:p></p>
<p class="p1"><span class="s1">Time Step 149, time = 13229.7, dt = 100</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">0, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">7.80968E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">1, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">7.65731E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">2, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">6.85034E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">3, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">6.11873E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">4, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">1.57347E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">5, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">9.03536E-04</span><o:p></o:p></p>
<p class="p2"><span class="apple-converted-space"> </span><span class="s1">Solve Converged!</span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">1<sup>st</sup> order FVM:<o:p></o:p></p>
<p class="p1"><span class="s1">Time Step 149, time = 13229.7, dt = 100</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">0, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">7.90072E-03</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">1, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">2.01919E-04</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">2, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">1.06960E-05</span><o:p></o:p></p>
<p class="p1"><span class="apple-converted-space"> </span><span class="s1">NL Step =</span><span class="apple-converted-space">
</span><span class="s1">3, fnorm =</span><span class="apple-converted-space"> </span>
<span class="s1">2.41683E-09</span><o:p></o:p></p>
<p class="p2"><span class="apple-converted-space"> </span><span class="s1">Solve Converged!</span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Notice the obvious ‘stagnant’ in residual for the 2<sup>nd</sup> order method while not in the 1<sup>st</sup> order.
<o:p></o:p></p>
<p class="MsoNormal">For the same problem, the wall time is 10 sec vs 6 sec. I would be happy if I can reduce 2 sec for the 2<sup>nd</sup> order method.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-Ling<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">From:
</span></b><span style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">Barry Smith <bsmith@petsc.dev><br>
<b>Date: </b>Sunday, March 3, 2024 at 12:06 PM<br>
<b>To: </b>Zou, Ling <lzou@anl.gov><br>
<b>Cc: </b>petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov><br>
<b>Subject: </b>Re: [petsc-users] 'Preconditioning' with lower-order method<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">Are you forming the Jacobian for the first and second order cases inside of Newton? You can run both with -log_view to see how much time is spent in the various
events (compute function, compute Jacobian, linear solve, .. . ) for the two cases
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerStart<o:p></o:p></span></p>
</div>
<div style="border:none;border-top:solid #90A4AE 3.0pt;padding:0in 0in 0in 0in;display:block!important;text-align:left!important;margin:0px!important;padding:16px!important;border-radius:4px!important;min-width:200px!important;background-color:#D0D8DC!important;border-top:#90a4ae!important" id="pfptBannerfb8wtog">
<div id="pfptBannerfb8wtog">
<div id="pfptBannerfb8wtog">
<p class="MsoNormal" style="line-height:13.5pt;background:#D0D8DC"><b><span style="font-family:"Arial",sans-serif;color:black">This Message Is From an External Sender
<o:p></o:p></span></b></p>
</div>
<div id="pfptBannerfb8wtog">
<p class="MsoNormal" style="line-height:13.5pt;background:#D0D8DC"><span style="font-family:"Arial",sans-serif;color:black">This message came from outside your organization.
<o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal" style="background:#D0D8DC"><span style="color:black"> </span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerEnd<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"> Are you forming the Jacobian for the first and second order cases inside of Newton?<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> You can run both with -log_view to see how much time is spent in the various events (compute function, compute Jacobian, linear solve, ...) for the two cases and compare them.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Mar 3, 2024, at 11:42 AM, Zou, Ling via petsc-users <petsc-users@mcs.anl.gov> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">Original email may have been sent to the incorrect place.<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">See below.<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">-Ling<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid windowtext 1.0pt;padding:3.0pt 0in 0in 0in;border-color:currentcolor currentcolor">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;font-family:"Aptos",sans-serif">From:<span class="apple-converted-space"> </span></span></b><span style="font-size:12.0pt;font-family:"Aptos",sans-serif">Zou, Ling <<a href="mailto:lzou@anl.gov">lzou@anl.gov</a>><br>
<b>Date:<span class="apple-converted-space"> </span></b>Sunday, March 3, 2024 at 10:34 AM<br>
<b>To:<span class="apple-converted-space"> </span></b>petsc-users <<a href="mailto:petsc-users-bounces@mcs.anl.gov">petsc-users-bounces@mcs.anl.gov</a>><br>
<b>Subject:<span class="apple-converted-space"> </span></b>'Preconditioning' with lower-order method</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal">Hi all,<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">I am solving a PDE system over a spatial domain. Numerical methods are:<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<ul style="margin-top:0in" type="disc">
<li class="MsoListParagraph" style="margin-top:0in;margin-bottom:0in;mso-list:l2 level1 lfo1">
Finite volume method (both 1<sup>st</sup><span class="apple-converted-space"> </span>and 2<sup>nd</sup><span class="apple-converted-space"> </span>order implemented)<span style="font-size:10.0pt"><o:p></o:p></span></li><li class="MsoListParagraph" style="margin-top:0in;margin-bottom:0in;mso-list:l2 level1 lfo1">
BDF1 and BDF2 for time integration.<span style="font-size:10.0pt"><o:p></o:p></span></li></ul>
<div>
<p class="MsoNormal">What I have noticed is that 1<sup>st</sup><span class="apple-converted-space"> </span>order FVM converges much faster than 2<sup>nd</sup><span class="apple-converted-space"> </span>order FVM, regardless the time integration scheme. Well,
not surprising since 2<sup>nd</sup><span class="apple-converted-space"> </span>order FVM introduces additional non-linearity.<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">I’m thinking about two possible ways to speed up 2<sup>nd</sup><span class="apple-converted-space"> </span>order FVM, and would like to get some thoughts or community knowledge before jumping into code implementation.<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Say, let the 2<sup>nd</sup><span class="apple-converted-space"> </span>order FVM residual function be<span class="apple-converted-space"> </span><b><i>F</i></b><sub>2</sub>(<b><i>x</i></b>) = 0; and the 1<sup>st</sup><span class="apple-converted-space"> </span>order
FVM residual function be<span class="apple-converted-space"> </span><b><i>F</i></b><sub>1</sub>(<b><i>x</i></b>) = 0.<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<ol style="margin-top:0in" start="1" type="1">
<li class="MsoListParagraph" style="margin-top:0in;margin-bottom:0in;mso-list:l0 level1 lfo2">
Option – 1, multi-step for each time step<span style="font-size:10.0pt"><o:p></o:p></span></li></ol>
<div style="margin-left:.5in">
<p class="MsoNormal">Step 1: solving<span class="apple-converted-space"> </span><b><i>F</i></b><sub>1</sub>(<b><i>x</i></b>) = 0 to obtain a temporary solution<span class="apple-converted-space"> </span><b><i>x</i></b><sub>1</sub><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal">Step 2: feed<span class="apple-converted-space"> </span><b><i>x</i></b><sub>1</sub><span class="apple-converted-space"> </span>as an initial guess to solve<span class="apple-converted-space"> </span><b><i>F</i></b><sub>2</sub>(<b><i>x</i></b>)
= 0 to obtain the final solution.<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal">[Not sure if gain any saving at all]<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"> <span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<ol style="margin-top:0in" start="2" type="1">
<li class="MsoListParagraph" style="margin-top:0in;margin-bottom:0in;mso-list:l1 level1 lfo3">
Option -2, dynamically changing residual function F(x)<span style="font-size:10.0pt"><o:p></o:p></span></li></ol>
<div style="margin-left:.5in">
<p class="MsoNormal">In pseudo code, would be something like.<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"> <span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco">snesFormFunction(SNES snes, Vec u, Vec f, void *)</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco">{</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco"> if (snes.nl_it_no < 4) // 4 being arbitrary here</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco"> f = F1(u);</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco"> else</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco"> f = F2(u);</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco">}</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco"> </span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal">I know this might be a bit crazy since it may crash after switching residual function, still, any thoughts?<span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco"> </span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco">Best,</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco"> </span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Monaco">-Ling</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>