[petsc-dev] [petsc-maint] Petsc KSP options that provide the most 'robust' solve possible

[Barry Smith]

> On Feb 3, 2017, at 12:59 AM, Mark McClure <mark.w.mccl at gmail.com> wrote:
> 
> Hi, Jed and Matt. 
> 
> To close the loop, it turns out that the condition number issue had a simple explanation. The BC equation that I had added to the system had very different units from the other equations. Multiplying that row by a constant (effectively, modifying the units of the equation) improved the condition number of the matrix by many orders of magnitude. Big picture, this did not have an apparent effect on any of the numerical performance - the linear solver still had nonconvergence in the same places (when I used a large number of processes) and when I use a smaller number of processors so that the linear solve always converges, the overall numerical scheme is unaffected by whether or not I scale the extra BC equation. I hadn't realized how important it is to make sure the equations are scaled consistently/nondimensionalized when using the convergence number to evaluate whether a system is singular. Interesting experience.
> 
> Thanks again for the help. I'll use the direct solver you suggested as a backup and send the user a warning if they try to use too many processors on a small problem.
> 
> An aside, at first, when I saw that my overall numerical scheme was failing, I didn't realize the problem was that the linear solver was not converging. I spent a fair amount time debugging until I found the issue and learned how to use KSPConvergedReason. Because if KSP doesn't converge, it merely returns incorrect numbers. Without knowing to run KSPConvergedReason, an inexperienced user (like me) many not know about the nonconvergence in the linear solve and could spend a lot of time checking for other issues. It might be worth changing the default behavior for nonconvergence to be that it returns an nan so that the user gets a clear signal that the values coming out of KSP cannot be used.

   Because the linear solvers in PETSc are usually used by the nonlinear solvers and ODE integrators that have recovery methods for failure in a linear solve we moved away from the "crash and burn" on failed linear solver approach; since that makes the recovery more difficult.

    We could consider the following; if the KSP is not created by a SNES or TS it defaults to "crash and burn" on failed linear solve this would help newbies who are only solving linear systems and expecting a "crash and burn" on failure.

   What do people think?

   Barry

> 
> Regards,
> Mark
> 
> 
> 
> On Thu, Feb 2, 2017 at 7:48 AM, Jed Brown <jed at jedbrown.org> wrote:
> Mark McClure <mark.w.mccl at gmail.com> writes:
> 
> > I think you are right that I have an issue with how the BC is implemented.
> > It is a pipe flow simulation that is solving mass and momentum balance
> > simultaneously (corresponding unknowns are pressure and flow rate). I am
> > applying a constant mass flow rate boundary condition. Upon further
> > consideration, it may be that I am not properly providing a boundary
> > condition for the momentum balance equation at the inlet. If I did, the
> > inlet pressure could be readily calculated standalone,
> 
> Momentum inflow is common, but if you also have momentum outflow (i.e.,
> all Dirichlet conditions for momentum) then there is a null space of
> constant pressure -- pressure is only determined up to a constant.
> See the user manual section on solving singular equations.
> 
> > outside the system of equations, and the problematic equation would be
> > removed.
>