<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Hi Barry, </div><div class=""><br class=""></div><div class=""> Attached is the output. </div><div class=""><br class=""></div><div class="">-Manav</div><div class=""><br class=""></div><div class=""><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">|| R ||_2 = 433013 : || R_i ||_2 = ( 2.81069e-07 , 433013 )</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 SNES Function norm 4.330127018922e+05 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 KSP Residual norm 1.746840810717e-02 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 KSP Residual norm 5.983637077441e-12 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Linear solve converged due to CONVERGED_RTOL iterations 1</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">|| R ||_2 = 5.07622e-07 : || R_i ||_2 = ( 5.07622e-07 , 7.12896e-11 )</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 SNES Function norm 5.076218984984e-07 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Time step: 1 : t = 0.001 : xdot-L2 = 2.01301e-06</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">|| R ||_2 = 1.73186e+06 : || R_i ||_2 = ( 5.77273e-07 , 1.73186e+06 )</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 SNES Function norm 1.731856101521e+06 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 KSP Residual norm 1.745842035995e-02 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 KSP Residual norm 2.366595812944e-12 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Linear solve converged due to CONVERGED_RTOL iterations 1</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">|| R ||_2 = 1.29889e+06 : || R_i ||_2 = ( 9.7379e-07 , 1.29889e+06 )</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 SNES Function norm 1.298892076140e+06 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Time step: 2 : t = 0.002 : xdot-L2 = 2.72522e-06</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">|| R ||_2 = 433159 : || R_i ||_2 = ( 1.35694e-06 , 433159 )</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 SNES Function norm 4.331589481273e+05 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 KSP Residual norm 1.744848431182e-02 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 KSP Residual norm 7.650255893811e-12 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Linear solve converged due to CONVERGED_RTOL iterations 1</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">|| R ||_2 = 866074 : || R_i ||_2 = ( 8.42454e-07 , 866074 )</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 SNES Function norm 8.660737893156e+05 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Time step: 3 : t = 0.003 : xdot-L2 = 3.59383e-06</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">|| R ||_2 = 2.59764e+06 : || R_i ||_2 = ( 1.12168e-06 , 2.59764e+06 )</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 SNES Function norm 2.597639281693e+06 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 KSP Residual norm 1.743859969865e-02 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 KSP Residual norm 1.045225058356e-11 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Linear solve converged due to CONVERGED_RTOL iterations 1</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">|| R ||_2 = 2.16477e+06 : || R_i ||_2 = ( 9.73157e-07 , 2.16477e+06 )</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 SNES Function norm 2.164772029312e+06 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Changing dt: old dt = 0.001 new dt = 0.001</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Time step: 4 : t = 0.004 : xdot-L2 = 4.9281e-06</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">|| R ||_2 = 1.29933e+06 : || R_i ||_2 = ( 1.2554e-06 , 1.29933e+06 )</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 SNES Function norm 1.299329216581e+06 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 KSP Residual norm 1.742876625743e-02 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 KSP Residual norm 6.512084951772e-12 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Linear solve converged due to CONVERGED_RTOL iterations 1</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">|| R ||_2 = 1.73215e+06 : || R_i ||_2 = ( 1.00171e-06 , 1.73215e+06 )</span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 SNES Function norm 1.732147071753e+06 </span></div><div style="margin: 0px; line-height: normal; font-family: Monaco; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1</span></div></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><br class=""><div><blockquote type="cite" class=""><div class="">On Apr 20, 2017, at 2:38 PM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" class="">bsmith@mcs.anl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class=""><br class=""> Run with -snes_monitor also and send the output<br class=""><br class=""><br class=""><blockquote type="cite" class="">On Apr 20, 2017, at 2:30 PM, Manav Bhatia <<a href="mailto:bhatiamanav@gmail.com" class="">bhatiamanav@gmail.com</a>> wrote:<br class=""><br class="">Hi, <br class=""><br class=""> I have a time-dependent multiphysics problem that I am trying to solve using pc_fieldsplit. I have defined a nested matrix for the jacobian with the diagonal block matrices explicitly created and the off-diagonal blocks defined using shell matrices, so that the matrix vector product is defined. The nonlinear system of equations at each time-step is solved using an snes construct.<br class=""><br class=""> I have been facing some convergence issues, so I have reduced the problem scope to ensure that the code converges to a single discipline solution when the off-diagonal couplings are ignored. <br class=""><br class=""> Here, I have provided a constant forcing function to discipline two, which is a linear problem, so that I expect convergence in a single iteration. <br class=""><br class=""> The linear solver, defined using pc_fieldsplit seems to be converging without problems. The nonlinear solver convergence in a single time-step with FNORM in the first time step.<br class=""><br class=""> The second time-step onwards, the nonlinear solver does not converge in a single step, and is terminating due to SNORM_RELATIVE. I am not sure why this is happening. <br class=""><br class=""> What is intriguing is that the solution at the end of the n^th time-step is n times the solution after the first time step. In other words, snes at each time-step is taking the same step as was used in the first time-step. <br class=""><br class=""> Not sure sure why this is happening. I would appreciate any advice. <br class=""><br class="">Regards,<br class="">Manav<br class=""><br class="">Time step: 0 : t = 0.000 <br class="">|| R ||_2 = 433013 : || R_i ||_2 = ( 2.81069e-07 , 433013 )<br class=""> 0 KSP Residual norm 1.746840810717e-02 <br class=""> 1 KSP Residual norm 5.983637077441e-12 <br class=""> Linear solve converged due to CONVERGED_RTOL iterations 1<br class="">|| R ||_2 = 5.07622e-07 : || R_i ||_2 = ( 5.07622e-07 , 7.12896e-11 )<br class="">Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 1<br class="">Time step: 1 : t = 0.001 <br class="">|| R ||_2 = 1.73186e+06 : || R_i ||_2 = ( 5.77273e-07 , 1.73186e+06 )<br class=""> 0 KSP Residual norm 1.745842035995e-02 <br class=""> 1 KSP Residual norm 2.366595812944e-12 <br class=""> Linear solve converged due to CONVERGED_RTOL iterations 1<br class="">|| R ||_2 = 1.29889e+06 : || R_i ||_2 = ( 9.7379e-07 , 1.29889e+06 )<br class="">Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1<br class="">Time step: 2 : t = 0.002 : xdot-L2 = 2.72522e-06<br class="">|| R ||_2 = 433159 : || R_i ||_2 = ( 1.35694e-06 , 433159 )<br class=""> 0 KSP Residual norm 1.744848431182e-02 <br class=""> 1 KSP Residual norm 7.650255893811e-12 <br class=""> Linear solve converged due to CONVERGED_RTOL iterations 1<br class="">|| R ||_2 = 866074 : || R_i ||_2 = ( 8.42454e-07 , 866074 )<br class="">Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1<br class="">Time step: 3 : t = 0.003<br class="">|| R ||_2 = 2.59764e+06 : || R_i ||_2 = ( 1.12168e-06 , 2.59764e+06 )<br class=""> 0 KSP Residual norm 1.743859969865e-02 <br class=""> 1 KSP Residual norm 1.045225058356e-11 <br class=""> Linear solve converged due to CONVERGED_RTOL iterations 1<br class="">|| R ||_2 = 2.16477e+06 : || R_i ||_2 = ( 9.73157e-07 , 2.16477e+06 )<br class="">Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE iterations 1<br class=""><br class=""><br class="">Following is the output of snesview: <br class="">SNES Object: 1 MPI processes<br class=""> type: newtonls<br class=""> maximum iterations=50, maximum function evaluations=10000<br class=""> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08<br class=""> total number of linear solver iterations=1<br class=""> total number of function evaluations=2<br class=""> norm schedule ALWAYS<br class=""> SNESLineSearch Object: 1 MPI processes<br class=""> type: basic<br class=""> maxstep=1.000000e+08, minlambda=1.000000e-12<br class=""> tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08<br class=""> maximum iterations=1<br class=""> KSP Object: 1 MPI processes<br class=""> type: gmres<br class=""> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br class=""> GMRES: happy breakdown tolerance 1e-30<br class=""> maximum iterations=10000, initial guess is zero<br class=""> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br class=""> left preconditioning<br class=""> using PRECONDITIONED norm type for convergence test<br class=""> PC Object: 1 MPI processes<br class=""> type: fieldsplit<br class=""> FieldSplit with SYMMETRIC_MULTIPLICATIVE composition: total splits = 2<br class=""> Solver info for each split is in the following KSP objects:<br class=""> Split number 0 Defined by IS<br class=""> KSP Object: (fieldsplit_0_) 1 MPI processes<br class=""> type: preonly<br class=""> maximum iterations=10000, initial guess is zero<br class=""> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br class=""> left preconditioning<br class=""> using NONE norm type for convergence test<br class=""> PC Object: (fieldsplit_0_) 1 MPI processes<br class=""> type: ilu<br class=""> ILU: out-of-place factorization<br class=""> 0 levels of fill<br class=""> tolerance for zero pivot 2.22045e-14<br class=""> matrix ordering: natural<br class=""> factor fill ratio given 1., needed 1.<br class=""> Factored matrix follows:<br class=""> Mat Object: 1 MPI processes<br class=""> type: seqaij<br class=""> rows=108, cols=108<br class=""> package used to perform factorization: petsc<br class=""> total: nonzeros=2800, allocated nonzeros=2800<br class=""> total number of mallocs used during MatSetValues calls =0<br class=""> using I-node routines: found 27 nodes, limit used is 5<br class=""> linear system matrix = precond matrix:<br class=""> Mat Object: (fieldsplit_0_) 1 MPI processes<br class=""> type: seqaij<br class=""> rows=108, cols=108<br class=""> total: nonzeros=2800, allocated nonzeros=2800<br class=""> total number of mallocs used during MatSetValues calls =0<br class=""> using I-node routines: found 27 nodes, limit used is 5<br class=""> Split number 1 Defined by IS<br class=""> KSP Object: (fieldsplit_1_) 1 MPI processes<br class=""> type: preonly<br class=""> maximum iterations=10000, initial guess is zero<br class=""> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br class=""> left preconditioning<br class=""> using NONE norm type for convergence test<br class=""> PC Object: (fieldsplit_1_) 1 MPI processes<br class=""> type: lu<br class=""> LU: out-of-place factorization<br class=""> tolerance for zero pivot 2.22045e-14<br class=""> matrix ordering: nd<br class=""> factor fill ratio given 5., needed 1.15385<br class=""> Factored matrix follows:<br class=""> Mat Object: 1 MPI processes<br class=""> type: seqaij<br class=""> rows=30, cols=30<br class=""> package used to perform factorization: petsc<br class=""> total: nonzeros=540, allocated nonzeros=540<br class=""> total number of mallocs used during MatSetValues calls =0<br class=""> using I-node routines: found 9 nodes, limit used is 5<br class=""> linear system matrix = precond matrix:<br class=""> Mat Object: (fieldsplit_1_) 1 MPI processes<br class=""> type: seqaij<br class=""> rows=30, cols=30<br class=""> total: nonzeros=468, allocated nonzeros=468<br class=""> total number of mallocs used during MatSetValues calls =0<br class=""> using I-node routines: found 10 nodes, limit used is 5<br class=""> linear system matrix = precond matrix:<br class=""> Mat Object: 1 MPI processes<br class=""> type: nest<br class=""> rows=138, cols=138<br class=""> Matrix object: <br class=""> type=nest, rows=2, cols=2 <br class=""> MatNest structure: <br class=""> (0,0) : prefix="fieldsplit_0_", type=seqaij, rows=108, cols=108 <br class=""> (0,1) : type=shell, rows=108, cols=30 <br class=""> (1,0) : type=shell, rows=30, cols=108 <br class=""> (1,1) : prefix="fieldsplit_1_", type=seqaij, rows=30, cols=30 <br class=""><br class=""><br class=""></blockquote><br class=""></div></div></blockquote></div><br class=""></body></html>