[petsc-dev] VecNest

Pierre Jolivet pierre at joliv.et
Tue Jul 12 07:31:24 CDT 2022 


> On 12 Jul 2022, at 2:13 PM, Pierre Jolivet <pierre at joliv.et> wrote:
> 
> 
> 
>> On 12 Jul 2022, at 2:04 PM, Mark Adams <mfadams at lbl.gov <mailto:mfadams at lbl.gov>> wrote:
>> 
>> It looks like the RHS is zero in the 2nd case (0 KSP unpreconditioned resid norm 0.000000000000e+00), but the true residual is the same.
>> 
>> It looks like you added "nest_subvec" to our example.
> 
> It’s a new (dumb) example (ex1111).
> 
>> You can start by looking at the vectors with -vec_view (there is code that you can view vectors explicitly in your code to get more control if this is too noisy)
> 
> The Vec are exactly the same (as reported by -ksp_view_rhs) in both cases.
> 
>> You could send us the diffs in ex111. We may be able to see something wrong.
> 
> I believe there is something weird inside KSPGMRES + PCFIELDSPLIT + right preconditioning.

More precisely, this is the faulty line: https://petsc.org/main/src/ksp/ksp/impls/gmres/gmres.c.html#line119 <https://petsc.org/main/src/ksp/ksp/impls/gmres/gmres.c.html#line119>
Here is the value of res with -nest_subvec false: 3.162278
With -nest_subvec true: 2.387784
The input Vec being
Vec Object: 1 MPI process
  type: nest
  VecNest, rows=2,  structure: 
  (0) : type=seq, rows=5 
    Vec Object: 1 MPI process
      type: seq
    0.720032
    0.061794
    0.0100223
    0.144639
    0.397778
  (1) : type=seq, rows=5 
    Vec Object: 1 MPI process
      type: seq
    1.
    1.
    1.
    1.
    1.
One can see that either VecNormalize() or VecNest is indeed broken.

Thanks,
Pierre

> Because if I switch to KSPHPDDM implementation of GMRES, then all is good.
> In the below traces, I switched VecSet(part of RHS,0.0) to VecSet(part of RHS,1.0) in ex1111.c to make the error more visible.
> 
> Thanks,
> Pierre
> 
> $ ./ex1111 -pc_type fieldsplit -ksp_converged_reason -fieldsplit_pc_type lu -ksp_pc_side right -ksp_view_final_residual -nest_subvec false -ksp_view_rhs -ksp_view_solution          
> Linear solve did not converge due to DIVERGED_BREAKDOWN iterations 30 < garbage
> KSP final norm of residual 1.75544 < garbage
> Vec Object: 1 MPI process
>   type: nest
>   VecNest, rows=2,  structure: 
>   (0) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     0.720032
>     0.061794
>     0.0100223
>     0.144639
>     0.397778
>   (1) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     1.
>     1.
>     1.
>     1.
>     1.
> Vec Object: 1 MPI process
>   type: nest
>   VecNest, rows=2,  structure: 
>   (0) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     1.73518 < garbage
>     2.07583e-16
>     -8.13135e-16
>     -4.24749e-17
>     8.18337e-16
>   (1) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     -1.73518 < garbage
>     -1.73518 < garbage
>     -1.73518 < garbage
>     -1.73518 < garbage
>     -1.73518 < garbage
> 
> $ ./ex1111 -pc_type fieldsplit -ksp_converged_reason -fieldsplit_pc_type lu -ksp_pc_side right -ksp_view_final_residual -nest_subvec false -ksp_view_rhs -ksp_view_solution -ksp_type hpddm
> Linear solve converged due to CONVERGED_RTOL iterations 1 < correct
> KSP final norm of residual 7.89955e-16
> Vec Object: 1 MPI process
>   type: nest
>   VecNest, rows=2,  structure: 
>   (0) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     0.720032
>     0.061794
>     0.0100223
>     0.144639
>     0.397778
>   (1) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     1.
>     1.
>     1.
>     1.
>     1.
> Vec Object: 1 MPI process
>   type: nest
>   VecNest, rows=2,  structure: 
>   (0) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     1. < correct
>     1.79841e-18
>     -5.45696e-17
>     8.83987e-18
>     2.82354e-17
>   (1) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     -1. < correct
>     -1. < correct
>     -1. < correct
>     -1. < correct
>     -1. < correct
> 
> $ ./ex1111 -pc_type fieldsplit -ksp_converged_reason -fieldsplit_pc_type lu -ksp_pc_side right -ksp_view_final_residual -nest_subvec true -ksp_view_rhs -ksp_view_solution
> Linear solve converged due to CONVERGED_RTOL iterations 1 < correct
> KSP final norm of residual 2.74152e-16
> Vec Object: 1 MPI process
>   type: nest
>   VecNest, rows=2,  structure: 
>   (0) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     0.720032
>     0.061794
>     0.0100223
>     0.144639
>     0.397778
>   (1) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     1.
>     1.
>     1.
>     1.
>     1.
> Vec Object: 1 MPI process
>   type: nest
>   VecNest, rows=2,  structure: 
>   (0) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     1. < correct
>     -3.75571e-17
>     2.51808e-16
>     7.69081e-18
>     -2.65477e-16
>   (1) : type=seq, rows=5 
>     Vec Object: 1 MPI process
>       type: seq
>     -1. < correct
>     -1. < correct
>     -1. < correct
>     -1. < correct
>     -1. < correct
> 
>> Mark
>> 
>> On Mon, Jul 11, 2022 at 2:17 PM Pierre Jolivet <pierre at joliv.et <mailto:pierre at joliv.et>> wrote:
>> Hello,
>> Could anyone help me understand what is going on in the following example, please?
>> I have a VecNest.
>> I either: a) initialize all values to 0.0, then set a specific part of the vector to nonzero or b) initialize a part of the vector to 0.0 and set the other part to nonzero.
>> I don’t see why a) and b) produce different results.
>> 
>> $ ./ex1111 -pc_type fieldsplit -ksp_monitor_true_residual -ksp_converged_reason -fieldsplit_pc_type jacobi -ksp_pc_side right -ksp_view_final_residual -nest_subvec true 
>>   0 KSP unpreconditioned resid norm 8.375635517980e-01 true resid norm 8.375635517980e-01 ||r(i)||/||b|| 1.000000000000e+00
>>   1 KSP unpreconditioned resid norm 4.748816884247e-01 true resid norm 4.748816884247e-01 ||r(i)||/||b|| 5.669798875623e-01
>>   2 KSP unpreconditioned resid norm 4.713006778679e-01 true resid norm 4.713006778679e-01 ||r(i)||/||b|| 5.627043784990e-01
>>   3 KSP unpreconditioned resid norm 7.092979927129e-02 true resid norm 7.092979927129e-02 ||r(i)||/||b|| 8.468587144106e-02
>>   4 KSP unpreconditioned resid norm 1.457836310255e-02 true resid norm 1.457836310255e-02 ||r(i)||/||b|| 1.740567992870e-02
>>   5 KSP unpreconditioned resid norm 1.625040500524e-14 true resid norm 1.633468028779e-14 ||r(i)||/||b|| 1.950261595401e-14
>> Linear solve converged due to CONVERGED_RTOL iterations 5
>> KSP final norm of residual 1.63347e-14
>> $ ./ex1111 -pc_type fieldsplit -ksp_monitor_true_residual -ksp_converged_reason -fieldsplit_pc_type jacobi -ksp_pc_side right -ksp_view_final_residual -nest_subvec false
>>   0 KSP unpreconditioned resid norm 0.000000000000e+00 true resid norm 8.375635517980e-01 ||r(i)||/||b||            inf
>> Linear solve converged due to CONVERGED_ATOL iterations 0
>> KSP final norm of residual 0.837564
>> 
>> Thanks,
>> Pierre
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20220712/e5e2f65d/attachment.html>

More information about the petsc-dev mailing list