<div dir="ltr"><div class="gmail_default" style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">OK, thanks.</font></div><div class="gmail_default" style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)"><br></font></div><div class="gmail_default" style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">I'm using a composite DM, DM_packer. To make a separate KSP, I did the following in the FormJacobian() routine, after assembling the Jacobian matrix A and the RHS for the tangent vector, b:</font></div><div class="gmail_default" style=""><ul style=""><li style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">KSPCreate(PETSC_COMM_WORLD,&ksp)</font></li><li style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">KSPSetDM(ksp,DM_packer)</font></li><li style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">KSPSetDMActive(ksp,PETSC_FALSE) - because I want to set the operators</font></li><li style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">KSPSetOperators(ksp,A,P)</font></li><li style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">VecSet(n,0) - set initial guess to zero</font></li><li style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">KSPSolve(ksp,b,n) - this solve works correctly</font></li><li style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">VecNormalize(n,NULL)</font></li><li style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">MatNullSpaceCreate(PetscObjectComm((PetscObject)A),PETSC_FALSE,1,&n,&nullsp)</font></li><li style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">MatSetNullSpace(A,nullsp)</font></li></ul></div><div class="gmail_default" style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)">Then, with -snes_type newtonls -pc_type none -ksp_monitor -ksp_monitor_true_residual -ksp_view, the output for the KSPSolve described above, i.e. for the tangent vector, looks correct:</font></div><div class="gmail_default" style=""><span class="gmail-s3" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><br></font></span></div><div class="gmail_default" style=""><font face="arial, helvetica, sans-serif" color="#000000" style="background-color:rgb(255,255,255)"><span class="gmail-s3" style="font-variant-ligatures:no-common-ligatures"> 0 KSP</span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures"> preconditioned resid norm 1.000000000000e+03 true resid norm 1.000000000000e+03 ||r(i)||/||b|| 1.000000000000e+00</span><br></font></div><div class="gmail_default" style=""><font face="arial, helvetica, sans-serif" color="#000000" style="background-color:rgb(255,255,255)">...</font></div><div class="gmail_default" style="">
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000">185 KSP preconditioned resid norm 9.900713131874e-03 true resid norm 9.900713131904e-03 ||r(i)||/||b|| 9.900713131904e-06</font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000">Linear solve converged due to CONVERGED_RTOL iterations 185</font></span></p></div><div class="gmail_default" style=""><font color="#000000" face="arial, helvetica, sans-serif" style="background-color:rgb(255,255,255)"><br></font></div><div class="gmail_default" style="">
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif">KSP Object: 1 MPI processes</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>type: gmres</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>happy breakdown tolerance 1e-30</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>maximum iterations=10000, initial guess is zero</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>tolerances:<span class="gmail-Apple-converted-space"> </span>relative=1e-05, absolute=1e-50, divergence=10000.</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>left preconditioning</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>using PRECONDITIONED norm type for convergence test</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000">PC Object: 1 MPI processes</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>type: none</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>linear system matrix = precond matrix:</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>Mat Object: 1 MPI processes</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>type: seqaij</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>rows=78247, cols=78247</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>total: nonzeros=6063481, allocated nonzeros=6063481</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>total number of mallocs used during MatSetValues calls =0</font></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space" style=""> </span>using I-node routines: found 26083 nodes, limit used is 5</font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><br></font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000">But the next KSP, i.e. the one in the SNES, doesn't converge in the true residual:</font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><br></font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000">
</font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"> 0 KSP preconditioned resid norm 2.045599092896e-04 true resid norm 2.803828296212e-04 ||r(i)||/||b|| 1.000000000000e+00</font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal">
</p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000">191 KSP preconditioned resid norm 2.009941278534e-09 true resid norm 1.636010142734e-04 ||r(i)||/||b|| 5.834915586465e-01</font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><br></font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000">KSP Object: 1 MPI processes</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>type: gmres</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>happy breakdown tolerance 1e-30</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>maximum iterations=10000, initial guess is zero</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>tolerances:<span class="gmail-Apple-converted-space"> </span>relative=1e-05, absolute=1e-50, divergence=10000.</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>left preconditioning</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>using PRECONDITIONED norm type for convergence test</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000">PC Object: 1 MPI processes</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>type: none</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><font face="arial, helvetica, sans-serif" color="#000000" style="background-color:rgb(255,255,255)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span></span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">linear</span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures"> system matrix = precond matrix:</span></font></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>Mat Object: 1 MPI processes</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>type: seqaij</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>rows=78247, cols=78247</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>total: nonzeros=6063481, allocated nonzeros=6063481</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>total number of mallocs used during MatSetValues calls =0</font></span></p><p class="gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000"><span class="gmail-Apple-converted-space"> </span>has attached null space</font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" color="#000000">
</font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" style="" color="#000000"><span class="gmail-Apple-converted-space" style=""> </span>using I-node routines: found 26083 nodes, limit used is 5</font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" style="" color="#000000"><br></font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif" style="" color="#000000">My guess about what's going on is that the tangent vector n isn't really in the nullspace of A. Rather, it's in the nullspace of the m x (m+1) submatrix of A. So, An=c e_{m+1}, where c is an arbitrary constant and e_{m+1} is the m+1 th basis vector. The nonlinear function also has an added row, F_{m+1}, which is set to zero in the FormFunction() routine. I don't care about the value of F_{m+1}, but I suppose that if it's included in the true residual, and NOT in the "preconditioned" residual, even with pc_type none, then it will be tricky to diagnose the performance. Should I be using my own routine to evaluate the residual, so that F_{m+1} is not included?</font></span></p></div></div>