[petsc-users] GMRES for outer solver

Mon May 2 14:16:37 CDT 2022

On Mon, May 2, 2022 at 3:09 PM Amneet Bhalla <mail2amneet at gmail.com> wrote:

>
> While trying to use -fp_trap with a debugger on a local system we used the
> debug build of our and PETSc code and all solvers are working fine (left
> and right PC and fgmres also) and no NaNs either.
>

Debugging code tends to initial all variables to 0. Uninitialized vars can
be NaN. That is the first place I would look. You
can usually find that with compiler warnings.

  Thanks,

     Matt

> Interrogating the optimized version of the code now…
>
>
> On Mon, May 2, 2022 at 11:11 AM Matthew Knepley <knepley at gmail.com> wrote:
>
>> On Mon, May 2, 2022 at 1:56 PM Amneet Bhalla <mail2amneet at gmail.com>
>> wrote:
>>
>>> Thanks Matt and Barry. Since we are using MATSHELL, we printed max norm
>>> of || b || and || A*x ||, the two things that have all the physics of the
>>> problem. Both are coming out to be finite (no NANs). Perhaps there is a NaN
>>> in PCSHELL. But this is counter-intuitive because the true residual norm ||
>>> b - A*x || has a NaN where there is no PC application. We checked the
>>> velocity and pressure field in VisIt and they seem to be reasonable and
>>> matching the analytical solution well.  Need to try it in a debugger or
>>> -fp_trap next.
>>>
>>
>> I would recommend -fp_trap. It will just fail at the first NaN so it is
>> really easy to find. You can also send the code and I can run it.
>>
>>
>>> The behavior is similar to what is discussed in this thread:
>>> https://www.mail-archive.com/petsc-users@mcs.anl.gov/msg34602.html
>>>
>>>
>>
>> That turned out to be a bug in their code.
>>
>>   Thanks,
>>
>>     Matt
>>
>>
>>> On Mon, May 2, 2022 at 7:56 AM Barry Smith <bsmith at petsc.dev> wrote:
>>>
>>>>
>>>>
>>>> On May 2, 2022, at 8:12 AM, Matthew Knepley <knepley at gmail.com> wrote:
>>>>
>>>> On Mon, May 2, 2022 at 12:23 AM Ramakrishnan Thirumalaisamy <
>>>> rthirumalaisam1857 at sdsu.edu> wrote:
>>>>
>>>>> Thank you. I have a couple of questions. I am solving the low Mach
>>>>> Navier-Stokes system using a projection preconditioner (pc_shell type) with
>>>>> GMRES being the outer solver and Richardson being the Krylov
>>>>> preconditioner. The solver diverges when ksp_pc_type is "right”:
>>>>>
>>>>> Linear stokes_ solve did not converge due to DIVERGED_NANORINF
>>>>> iterations 0
>>>>>
>>>>
>>>> NaN can always be tracked back. I recommend tracing it back to the
>>>> first NaN produced. My guess is that your equation of state if producing a
>>>> NaN.
>>>>
>>>>
>>>>    You can run in the debugger with -ksp_error_if_not_converged or
>>>> -fp_trap to see when the Nan first appears. If the problem does not appear
>>>> on one rank or you need to use mpiexec to start the program you can use the
>>>> option -start_in_debugger to have the program started up in the debugger
>>>> https://petsc.org/main/docs/manualpages/Sys/PetscAttachDebugger.html
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Also, we have an example of low Mach flow in TS ex76.
>>>>
>>>>   Thanks,
>>>>
>>>>     Matt
>>>>
>>>>
>>>>> and it converges when ksp_pc_type is "left":
>>>>>
>>>>> Residual norms for stokes_ solve.
>>>>>   0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm
>>>>>         -nan ||r(i)||/||b||           -nan
>>>>>   1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm
>>>>>         -nan ||r(i)||/||b||           -nan
>>>>>   2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm
>>>>>         -nan ||r(i)||/||b||           -nan
>>>>> Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2
>>>>>
>>>>>  I am curious to know why this is happening. The solver also diverges
>>>>> with "FGMRES" as the outer solver (which supports only right
>>>>> preconditioning).
>>>>>
>>>>> 2. Is it also possible to not get "-nan" when || b || = 0?
>>>>>
>>>>>
>>>>> Regards,
>>>>> Rama
>>>>>
>>>>> On Sun, May 1, 2022 at 12:12 AM Dave May <dave.mayhem23 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun 1. May 2022 at 07:03, Amneet Bhalla <mail2amneet at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> How about using a fixed number of Richardson iterations as a Krylov
>>>>>>> preconditioner to a GMRES solver?
>>>>>>>
>>>>>>
>>>>>> That is fine.
>>>>>>
>>>>>> Would that lead to a linear operation?
>>>>>>>
>>>>>>
>>>>>> Yes.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Sat, Apr 30, 2022 at 8:21 PM Jed Brown <jed at jedbrown.org> wrote:
>>>>>>>
>>>>>>>> In general, no. A fixed number of Krylov iterations (CG, GMRES,
>>>>>>>> etc.) is a nonlinear operation.
>>>>>>>>
>>>>>>>> A fixed number of iterations of a method with a fixed polynomial,
>>>>>>>> such as Chebyshev, is a linear operation so you don't need a flexible outer
>>>>>>>> method.
>>>>>>>>
>>>>>>>> Ramakrishnan Thirumalaisamy <rthirumalaisam1857 at sdsu.edu> writes:
>>>>>>>>
>>>>>>>> > Hi,
>>>>>>>> >
>>>>>>>> > I have a Krylov solver with a preconditioner that is also a
>>>>>>>> Krylov solver.
>>>>>>>> > I know I can use "fgmres" for the outer solver but can I use
>>>>>>>> gmres for the
>>>>>>>> > outer solver with a fixed number of iterations in the Krylov
>>>>>>>> > preconditioners?
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > Thanks,
>>>>>>>> > Rama
>>>>>>>>
>>>>>>> --
>>>>>>> --Amneet
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>>
>>>> https://www.cse.buffalo.edu/~knepley/
>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>
>>>>
>>>>
>>>
>>> --
>>> --Amneet
>>>
>>>
>>>
>>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> <http://www.cse.buffalo.edu/~knepley/>
>>
> --
> --Amneet
>
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220502/07b91ea3/attachment-0001.html>