<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 12 Jul 2022, at 2:13 PM, Pierre Jolivet <<a href="mailto:pierre@joliv.et" class="">pierre@joliv.et</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On 12 Jul 2022, at 2:04 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" class="">mfadams@lbl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">It looks like the RHS is zero in the 2nd case (0 KSP unpreconditioned resid norm 0.000000000000e+00), but the true residual is the same.</div><div class=""><br class=""></div><div class="">It looks like you added "nest_subvec" to our example.</div></div></div></blockquote><div class=""><br class=""></div><div class="">It’s a new (dumb) example (ex1111).</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="">You can start by looking at the vectors with -vec_view (there is code that you can view vectors explicitly in your code to get more control if this is too noisy)</div></div></div></blockquote><div class=""><br class=""></div><div class="">The Vec are exactly the same (as reported by -ksp_view_rhs) in both cases.</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="">You could send us the diffs in ex111. We may be able to see something wrong.</div></div></div></blockquote><div class=""><br class=""></div><div class="">I believe there is something weird inside KSPGMRES + PCFIELDSPLIT + right preconditioning.</div></div></div></div></blockquote><div><br class=""></div><div>More precisely, this is the faulty line: <a href="https://petsc.org/main/src/ksp/ksp/impls/gmres/gmres.c.html#line119" class="">https://petsc.org/main/src/ksp/ksp/impls/gmres/gmres.c.html#line119</a></div><div>Here is the value of res with -nest_subvec false: 3.162278</div><div>With -nest_subvec true: 2.387784</div><div>The input Vec being</div><div>Vec Object: 1 MPI process<br class=""> type: nest<br class=""> VecNest, rows=2, structure: <br class=""> (0) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 0.720032<br class=""> 0.061794<br class=""> 0.0100223<br class=""> 0.144639<br class=""> 0.397778<br class=""> (1) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class=""></div><div>One can see that either VecNormalize() or VecNest is indeed broken.</div><div><br class=""></div><div>Thanks,</div><div>Pierre</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><div class="">Because if I switch to KSPHPDDM implementation of GMRES, then all is good.</div><div class="">In the below traces, I switched VecSet(part of RHS,0.0) to VecSet(part of RHS,1.0) in ex1111.c to make the error more visible.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Pierre</div><div class=""><br class=""></div><div class="">$ ./ex1111 -pc_type fieldsplit -ksp_converged_reason -fieldsplit_pc_type lu -ksp_pc_side right -ksp_view_final_residual -nest_subvec false -ksp_view_rhs -ksp_view_solution </div>Linear solve did not converge due to DIVERGED_BREAKDOWN iterations 30 < garbage<br class="">KSP final norm of residual 1.75544 < garbage<br class="">Vec Object: 1 MPI process<br class=""> type: nest<br class=""> VecNest, rows=2, structure: <br class=""> (0) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 0.720032<br class=""> 0.061794<br class=""> 0.0100223<br class=""> 0.144639<br class=""> 0.397778<br class=""> (1) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class="">Vec Object: 1 MPI process<br class=""> type: nest<br class=""> VecNest, rows=2, structure: <br class=""> (0) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 1.73518 < garbage<br class=""> 2.07583e-16<br class=""> -8.13135e-16<br class=""> -4.24749e-17<br class=""> 8.18337e-16<br class=""> (1) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> -1.73518 < garbage<br class=""> -1.73518 < garbage<br class=""> -1.73518 < garbage<br class=""> -1.73518 < garbage<br class=""> -1.73518 < garbage<br class=""><br class="">$ ./ex1111 -pc_type fieldsplit -ksp_converged_reason -fieldsplit_pc_type lu -ksp_pc_side right -ksp_view_final_residual -nest_subvec false -ksp_view_rhs -ksp_view_solution -ksp_type hpddm<br class="">Linear solve converged due to CONVERGED_RTOL iterations 1 < correct<br class="">KSP final norm of residual 7.89955e-16<br class="">Vec Object: 1 MPI process<br class=""> type: nest<br class=""> VecNest, rows=2, structure: <br class=""> (0) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 0.720032<br class=""> 0.061794<br class=""> 0.0100223<br class=""> 0.144639<br class=""> 0.397778<br class=""> (1) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class="">Vec Object: 1 MPI process<br class=""> type: nest<br class=""> VecNest, rows=2, structure: <br class=""> (0) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 1. < correct<br class=""> 1.79841e-18<br class=""> -5.45696e-17<br class=""> 8.83987e-18<br class=""> 2.82354e-17<br class=""> (1) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> -1. < correct<br class=""> -1. < correct<br class=""> -1. < correct<br class=""> -1. < correct<br class=""> -1. < correct<br class=""><br class="">$ ./ex1111 -pc_type fieldsplit -ksp_converged_reason -fieldsplit_pc_type lu -ksp_pc_side right -ksp_view_final_residual -nest_subvec true -ksp_view_rhs -ksp_view_solution<br class="">Linear solve converged due to CONVERGED_RTOL iterations 1 < correct<br class="">KSP final norm of residual 2.74152e-16<br class="">Vec Object: 1 MPI process<br class=""> type: nest<br class=""> VecNest, rows=2, structure: <br class=""> (0) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 0.720032<br class=""> 0.061794<br class=""> 0.0100223<br class=""> 0.144639<br class=""> 0.397778<br class=""> (1) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class=""> 1.<br class="">Vec Object: 1 MPI process<br class=""> type: nest<br class=""> VecNest, rows=2, structure: <br class=""> (0) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> 1. < correct<br class=""> -3.75571e-17<br class=""> 2.51808e-16<br class=""> 7.69081e-18<br class=""> -2.65477e-16<br class=""> (1) : type=seq, rows=5 <br class=""> Vec Object: 1 MPI process<br class=""> type: seq<br class=""> -1. < correct<br class=""> -1. < correct<br class=""> -1. < correct<br class=""> -1. < correct<br class=""> -1. < correct<br class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="">Mark</div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jul 11, 2022 at 2:17 PM Pierre Jolivet <<a href="mailto:pierre@joliv.et" class="">pierre@joliv.et</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello,<br class="">
Could anyone help me understand what is going on in the following example, please?<br class="">
I have a VecNest.<br class="">
I either: a) initialize all values to 0.0, then set a specific part of the vector to nonzero or b) initialize a part of the vector to 0.0 and set the other part to nonzero.<br class="">
I don’t see why a) and b) produce different results.<br class="">
<br class="">
$ ./ex1111 -pc_type fieldsplit -ksp_monitor_true_residual -ksp_converged_reason -fieldsplit_pc_type jacobi -ksp_pc_side right -ksp_view_final_residual -nest_subvec true <br class="">
0 KSP unpreconditioned resid norm 8.375635517980e-01 true resid norm 8.375635517980e-01 ||r(i)||/||b|| 1.000000000000e+00<br class="">
1 KSP unpreconditioned resid norm 4.748816884247e-01 true resid norm 4.748816884247e-01 ||r(i)||/||b|| 5.669798875623e-01<br class="">
2 KSP unpreconditioned resid norm 4.713006778679e-01 true resid norm 4.713006778679e-01 ||r(i)||/||b|| 5.627043784990e-01<br class="">
3 KSP unpreconditioned resid norm 7.092979927129e-02 true resid norm 7.092979927129e-02 ||r(i)||/||b|| 8.468587144106e-02<br class="">
4 KSP unpreconditioned resid norm 1.457836310255e-02 true resid norm 1.457836310255e-02 ||r(i)||/||b|| 1.740567992870e-02<br class="">
5 KSP unpreconditioned resid norm 1.625040500524e-14 true resid norm 1.633468028779e-14 ||r(i)||/||b|| 1.950261595401e-14<br class="">
Linear solve converged due to CONVERGED_RTOL iterations 5<br class="">
KSP final norm of residual 1.63347e-14<br class="">
$ ./ex1111 -pc_type fieldsplit -ksp_monitor_true_residual -ksp_converged_reason -fieldsplit_pc_type jacobi -ksp_pc_side right -ksp_view_final_residual -nest_subvec false<br class="">
0 KSP unpreconditioned resid norm 0.000000000000e+00 true resid norm 8.375635517980e-01 ||r(i)||/||b|| inf<br class="">
Linear solve converged due to CONVERGED_ATOL iterations 0<br class="">
KSP final norm of residual 0.837564<br class="">
<br class="">
Thanks,<br class="">
Pierre<br class="">
<br class="">
</blockquote></div></div>
</div></blockquote></div><br class=""></div></div></blockquote></div><br class=""></body></html>