<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi all,<div>we are using PETSc to solve the steady state Stokes equations with non-linear viscosities using finite difference. Recently we have realized that our true residual norm after the last KSP solve did not match next SNES function norm when solving the linear Stokes equations.</div><div><br></div><div>So to understand this better, we set up two extremely simple linear residuals, one with no coupling between variables (vx, vy, P and T), the other with one coupling term (shown below).</div><div><br></div><div><u style="background-color: rgb(255, 255, 255);">RESIDUAL 1 (NO COUPLING):</u><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> for (j=info->ys; j<info->ys+info->ym; j++) {</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> for (i=info->xs; i<info->xs+info->xm; i++) {</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> f[j][i].P = x[j][i].P - 3000000;</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> f[j][i].vx= 2*x[j][i].vx;</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> f[j][i].vy= 3*x[j][i].vy - 2;</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> f[j][i].T = x[j][i].T;</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> }</span></div><div><br></div><div><u style="background-color: rgb(255, 255, 255);">RESIDUAL 2 (ONE COUPLING TERM):</u><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> for (j=info->ys; j<info->ys+info->ym; j++) {</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> for (i=info->xs; i<info->xs+info->xm; i++) {</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> f[j][i].P = x[j][i].P - 3;</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> f[j][i].vx= x[j][i].vx - 3*x[j][i].vy;</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> f[j][i].vy= x[j][i].vy - 2;</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> f[j][i].T = x[j][i].T;</span><span style="background-color: rgb(255, 255, 255);"> </span><span style="background-color: rgb(255, 255, 255);"> </span><br><span style="background-color: rgb(255, 255, 255);"> }</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> }</span></div><div><br></div><div><br></div><div>and our default set of options is:</div><div><br></div><div><br></div><div><u style="background-color: rgb(255, 255, 255);">OPTIONS:</u><span style="background-color: rgb(255, 255, 255);"> mpiexec -np $np ../Stokes -snes_max_it 4 -ksp_atol 2.0e+2 -ksp_max_it 20 -ksp_rtol 9.0e-1 -ksp_type fgmres -snes_monitor -snes_converged_reason -snes_view -log_summary -options_left 1 -ksp_monitor_true_residual -pc_type none -snes_linesearch_type cp </span></div><div><br></div><div><br></div><div>With the uncoupled residual (Residual 1), we get matching KSP and SNES norm, highlighted below:</div><div><br></div><div><br></div><div><u style="background-color: rgb(255, 255, 255);">Result from Solve - RESIDUAL 1 </u><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 0 SNES Function norm 8.485281374240e+07 </span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 0 KSP unpreconditioned resid norm 8.485281374240e+07 true resid norm 8.485281374240e+07 ||r(i)||/||b|| 1.000000000000e+00</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 1 KSP unpreconditioned resid norm 1.131370849896e+02 true resid norm </span><b style="background-color: rgb(255, 255, 255);">1.131370849896e+02</b><span style="background-color: rgb(255, 255, 255);"> ||r(i)||/||b|| 1.333333333330e-06</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 1 SNES Function norm </span><b style="background-color: rgb(255, 255, 255);">1.131370849896e+02</b><span style="background-color: rgb(255, 255, 255);"> </span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 0 KSP unpreconditioned resid norm 1.131370849896e+02 true resid norm 1.131370849896e+02 ||r(i)||/||b|| 1.000000000000e+00</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 2 SNES Function norm 1.131370849896e+02 </span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);">Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 2</span></div><div><br></div><div><br></div><div>With the coupled residual (Residual 2), the norms do not match, see below</div><div><br></div><div><br></div><div><u style="background-color: rgb(255, 255, 255);">Result from Solve - RESIDUAL 2:</u><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 0 SNES Function norm 1.019803902719e+02 </span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 0 KSP unpreconditioned resid norm 1.019803902719e+02 true resid norm 1.019803902719e+02 ||r(i)||/||b|| 1.000000000000e+00</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 1 KSP unpreconditioned resid norm 8.741176309016e+01 true resid norm 8.741176309016e+01 ||r(i)||/||b|| 8.571428571429e-01</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 1 SNES Function norm 1.697056274848e+02 </span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 0 KSP unpreconditioned resid norm 1.697056274848e+02 true resid norm 1.697056274848e+02 ||r(i)||/||b|| 1.000000000000e+00</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 1 KSP unpreconditioned resid norm 5.828670868165e-12 true resid norm </span><b style="background-color: rgb(255, 255, 255);">5.777940247956e-12</b><span style="background-color: rgb(255, 255, 255);"> ||r(i)||/||b|| 3.404683942184e-14</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 2 SNES Function norm </span><b style="background-color: rgb(255, 255, 255);">3.236770473841e-07 </b><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);">Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 2</span></div><div><span style="background-color: rgb(255, 255, 255);"><br></span></div><div><br></div><div><span style="background-color: rgb(255, 255, 255);">Lastly, if we add -snes_fd to our options, the norms for residual 2 get better - they match after the first iteration but not after the second.</span></div><div><span style="background-color: rgb(255, 255, 255);"><br></span></div><div><span style="background-color: rgb(255, 255, 255);"><br></span></div><div><u style="background-color: rgb(255, 255, 255);">Result from Solve with -snes_fd - RESIDUAL 2</u><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 0 SNES Function norm 8.485281374240e+07 </span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 0 KSP unpreconditioned resid norm 8.485281374240e+07 true resid norm 8.485281374240e+07 ||r(i)||/||b|| 1.000000000000e+00</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 1 KSP unpreconditioned resid norm 2.039607805429e+02 true resid norm 2.039607805429e+02 ||r(i)||/||b|| 2.403700850300e-06</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 1 SNES Function norm 2.039607805429e+02 </span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 0 KSP unpreconditioned resid norm 2.039607805429e+02 true resid norm 2.039607805429e+02 ||r(i)||/||b|| 1.000000000000e+00</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 1 KSP unpreconditioned resid norm 2.529822128436e+01 true resid norm </span><span style="background-color: rgb(255, 255, 255);"><b>2.529822128436e+01</b></span><span style="background-color: rgb(255, 255, 255);"> ||r(i)||/||b|| 1.240347346045e-01</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 2 SNES Function norm </span><b><span style="background-color: rgb(255, 255, 255);">2.549509757105e+01 [SLIGHTLY DIFFERENT]</span></b><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 0 KSP unpreconditioned resid norm 2.549509757105e+01 true resid norm </span><span style="background-color: rgb(255, 255, 255);">2.549509757105e+01</span><span style="background-color: rgb(255, 255, 255);"> ||r(i)||/||b|| 1.000000000000e+00</span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);"> 3 SNES Function norm </span><span style="background-color: rgb(255, 255, 255);">2.549509757105e+01</span><span style="background-color: rgb(255, 255, 255);"> </span><br style="background-color: rgb(255, 255, 255);"><span style="background-color: rgb(255, 255, 255);">Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 3</span></div><div><br></div><div><br><div>Does this mean that our Jacobian is not approximated properly by the default “coloring” method when it has off-diagonal terms?</div></div><div><br></div><div>Thanks a lot,</div><div>Arthur and Eric</div></body></html>