<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div><br></div> Try ksp_monitor_true_residual. Let's see if it is an issue of preconditioned vs unpreconditioned residual.<div><br></div><div><br id="lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On Sep 16, 2024, at 1:05 PM, miguel.salazar <miguel.salazar@corintis.com> wrote:</div><br class="Apple-interchange-newline"><div><meta charset="UTF-8"><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US">Hello,<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US"><o:p> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US">I have this simple example on Firedrake to illustrate my point. I am solving for a two-component poisson equation (uncoupled). Only the first component has a non-zero residual.<br><br>```<br></span>import firedrake as fd<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">from firedrake import inner, grad, dx, sin, pi<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">N = 10<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">mesh = fd.UnitSquareMesh(N, N)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">V = fd.FunctionSpace(mesh, "CG", 1)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">W = V * V<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">u = fd.Function(W)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">v = fd.TestFunction(W)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">a = inner(grad(u[0]), grad(v[0])) * dx + inner(grad(u[1]), grad(v[1])) * dx<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">x = fd.SpatialCoordinate(mesh)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">f = fd.Function(V)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">f.interpolate(fd.Constant(1e4) * sin(x[0] * pi) * sin(2 * x[1] * pi))<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">L = f * v[0] * dx<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">F = a - L<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">bcs = [fd.DirichletBC(W.sub(0), fd.Constant(2.0), (1,))]<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">def snes_firedrake_residual(F, u, bcs):<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> for bcs_ in bcs:<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> bcs_.apply(u)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> residual = fd.assemble(F, bcs=bcs, zero_bc_nodes=True)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> with residual.dat.vec_ro as r:<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> print("Initial residual:", r.norm())<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">snes_firedrake_residual(F, u, bcs)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">problem = fd.NonlinearVariationalProblem(F, u, bcs=bcs)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">solver_mumps_assembled = {<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "ksp_type": "preonly",<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "ksp_monitor": None,<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "pc_type": "python",<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "pc_python_type": "firedrake.AssembledPC",<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "assembled_pc_type": "lu",<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "assembled_pc_factor_mat_solver_type": "mumps",<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "assembled_mat_mumps_icntl_14": 200,<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "assembled_mat_mumps_icntl_24": 1,<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">}<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">solver_fieldsplit = {<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "mat_type": "matfree",<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "snes_type": "newtonls",<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "ksp_type": "fgmres",<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "ksp_rtol": 1e-1,<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "ksp_monitor": None,<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "pc_type": "fieldsplit",<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "pc_fieldsplit_type": "additive",<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "fieldsplit_0": solver_mumps_assembled,<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"> "fieldsplit_1": solver_mumps_assembled,<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">}<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">solver = fd.NonlinearVariationalSolver(problem, solver_parameters=solver_fieldsplit)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">solver.solve()<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US">```<br><br>The PETSc output is as follow<br><br>```<br>Initial residual: 462.13689530272404<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US"> 0 SNES Function norm 4.621368953027e+02<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US"> Residual norms for firedrake_0_ solve.<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US"> 0 KSP Residual norm 4.621368953027e+02<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US"> Residual norms for firedrake_0_fieldsplit_0_ solve.<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US"> <span class="Apple-converted-space"> </span></span><span lang="IT-CH">0 KSP Residual norm 1.000000000000e+00<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="IT-CH"> 1 KSP Residual norm 3.501082228626e-15<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="IT-CH"> <span class="Apple-converted-space"> </span></span><span lang="EN-US">Residual norms for firedrake_0_fieldsplit_1_ solve.<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US"> <span class="Apple-converted-space"> </span></span><span lang="IT-CH">0 KSP Residual norm 0.000000000000e+00<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="IT-CH"> 1 KSP Residual norm 0.000000000000e+00<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="IT-CH"> <span class="Apple-converted-space"> </span></span><span lang="EN-US">1 KSP Residual norm 1.612167203819e-12<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US"> 1 SNES Function norm 1.599350481360e-12<br>```<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US"><o:p> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;">Using the fieldsplit additive preconditioner, the problem converges in a single KSP iteration, as expected. However, I do not understand<span class="Apple-converted-space"> </span><span lang="EN-US">why the</span><span class="Apple-converted-space"> </span>residual of fieldsplit_0 (1e+0) does not coincide with the outer residual (462.13689530272404)<span lang="EN-US">. It should be the case given that only<span class="Apple-converted-space"> </span></span>fieldsplit_0 has<span lang="EN-US"><span class="Apple-converted-space"> </span>a non-zero residual contribution</span>. The fact that it is just 1 is suspicious. Is there something about how the fieldsplit works that I am missing?<span lang="EN-US"><br><br>Thanks,<o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif;"><span lang="EN-US">Miguel<o:p></o:p></span></div></div><div style="caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-size: 10pt; font-family: Arial;"><br class="webkit-block-placeholder"></div><table cellspacing="0" cellpadding="0" border="0" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; letter-spacing: normal; orphans: auto; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; width: 409px;"><tbody><tr><td valign="top" style="font-size: 10pt; font-family: Arial; width: 260px; padding-bottom: 20px;"><br></td><td valign="top" style="font-size: 10pt; font-family: Arial; width: 149px; padding-bottom: 20px;"><br></td></tr><tr><td valign="top" colspan="1" style="width: 260px; font-family: Arial; color: rgb(138, 138, 140); font-size: 8pt; line-height: 14pt; padding: 0px 0px 15px;"><span data-codetwo-visible="Name"><b style="color: rgb(27, 20, 100);">MIGUEL ANGEL SALAZAR DE TROYA<br></b></span><span style="font-size: 8pt; color: rgb(138, 138, 140);"><span data-codetwo-visible="Job title">Head of Software Engineering</span></span><br><span data-codetwo-visible="Email"><a href="mailto:miguel.salazar@corintis.com" style="font-size: 8pt; text-decoration: underline; color: rgb(138, 138, 140);"><em>miguel.salazar@corintis.com</em></a><br></span><span data-codetwo-visible="Phone">Corintis SA<br></span><span data-codetwo-visible="Company address">EPFL Innovation Park Building C<br>1015 Lausanne<br></span></td><td valign="top" colspan="1" style="width: 149px; font-family: Arial; line-height: 14pt; padding: 0px 0px 15px;"><span id="cid:2024-08-1609_02_11-re_mailmigrationfromgoogletooffice365-sebastien.gobel@corintis.com-co_4456a1bf-dd3e-46b7-85f9-21f278e66a79.png"><2024-08-1609_02_11-re_mailmigrationfromgoogletooffice365-sebastien.gobel@corintis.com-co_4456a1bf-dd3e-46b7-85f9-21f278e66a79.png></span></td></tr><tr data-codetwo-visible="Disclaimer"><td valign="top" colspan="2" style="font-size: 7pt; font-family: Arial; width: 409px; color: rgb(255, 255, 255); padding: 3px 10px; background-color: rgb(27, 20, 100);">Here at Corintis we care for your privacy. That is why we have taken appropriate measures to ensure that the data you have provided to us is always secure.</td></tr></tbody></table></div></blockquote></div><br></div></body></html>