[petsc-users] [EXTERNAL] Re: Question about SuperLU

Fri Jun 10 19:35:01 CDT 2022

Could that be due to "numerical zero pivot" (caused due to cancellation and
underflow)?  You can try to force diagonal to be nonzero.

Looking at the options page:
https://petsc.org/main/docs/manualpages/Mat/MATSOLVERSUPERLU_DIST/

You can enable this one:

-mat_superlu_dist_replacetinypivot

replace tiny pivots
(the default is NO, not to replace tiny pivots, including zero pivots.)

Sherry

On Fri, Jun 10, 2022 at 4:31 PM Jorti, Zakariae via petsc-users <
petsc-users at mcs.anl.gov> wrote:

> Hi,
>
> Thank you all for your answers.
> I have tried your suggestions and here is what I found.
> Barry you were right about the first case. But in the second case, I am
> not using a Schur fieldsplit but a multiplicative fieldsplit :
> -fieldsplit_TEBV_fieldsplit_EBV_pc_type fieldsplit
> -fieldsplit_TEBV_fieldsplit_EBV_pc_fieldsplit_type multiplicative
>
> So, there should not be any Schur complement approximation Sp.
>
> When I ran a test with
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ksp_view_pre
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ksp_error_if_not_converged, I
> got this error:
>
>
>     0 SNES Function norm 6.368031218939e-02
>       0 KSP Residual norm 6.368031218939e-02
>       Linear fieldsplit_ni_ solve converged due to CONVERGED_ATOL
> iterations 0
>         Linear fieldsplit_TEBV_fieldsplit_tau_ solve converged due to
> CONVERGED_RTOL iterations 1
>           Linear fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_EP_ solve
> converged due to CONVERGED_RTOL iterations 3
> [0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> [0]PETSC ERROR: Zero pivot in LU factorization:
> https://petsc.org/release/faq/#zeropivot
> [0]PETSC ERROR: Zero pivot in row 1658
> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> [0]PETSC ERROR: Petsc Development GIT revision: v3.16.3-751-g2f43bd9bc3
> GIT Date: 2022-01-26 22:34:02 -0600
> [0]PETSC ERROR: ./main on a macx named pn2032683.lanl.gov by zjorti Fri
> Jun 10 16:17:35 2022
> [0]PETSC ERROR: Configure options PETSC_ARCH=macx --with-fc=0
> --with-mpi-dir=/Users/zjorti/.brew --download-hypre --with-debugging=0
> --with-cxx-dialect=C++11 --download-superlu_dist --download-parmetis
> --download-metis --download-ptscotch --download-cmake
>
>
> Then I tried this flag
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ksp_view_pmat
> binary:BVmat:binary_matlab and checked the resulting matrix in Matlab.
> For Matlab, this is a full rank matrix, and the LU factorization there was
> carried out without any issues.
>
> I also outputted the BV block directly from the Jacobian matrix.
> Once again, according to Matlab, it is a full rank matrix and it computes
> the LU factorization without any problem.
>
>
> ------------------------------
> *From:* Barry Smith <bsmith at petsc.dev>
> *Sent:* Friday, June 10, 2022 7:32 AM
> *To:* Jorti, Zakariae
> *Cc:* petsc-users at mcs.anl.gov; Tang, Xianzhu
> *Subject:* [EXTERNAL] Re: [petsc-users] Question about SuperLU
>
>
>   It is difficult to tell exactly how the preconditioner is being formed
> with the information below it looks like in the
>
> first case: the original B diagonal block and V diagonal block of the
> matrix are being factored separately with SuperLU_DIST
>
> second case: the B block is factored with SuperLU_DIST and an explicit
> approximation to a Schur complement of the V block (Schur complement on
> eliminating the B block) is formed using "Preconditioner for the Schur
> complement formed from Sp, an assembled approximation to S, which uses
> A00's %sdiagonal's inverse" (this is the printout from a KSPView() for this
> part of the preconditioner).
>
>   My guess is you have a "Stokes"-like problem where the V block is
> identically 0 so, of course, the SuperLU_DIST will fail on it. But the
> approximation of the Schur complement onto that block is not singular so
> SuperLU_DIST has no trouble. If I am wrong and the V block is not
> identically 0 then it may be singular (or possibly, but less likely just
> badly order) so that SuperLU_DIST encounters a zero pivot.
>
>   You can run with -ksp_view_pre to have the KSP print the KSP solver
> algorithm details BEFORE the linear solve (hence they would get printed
> despite your failed solve). That would be useful to see exactly what your
> preconditioner is.
>
>    You can use -ksp_view_pmat (with appropriate prefix) to display the
> matrix that is going to be factored. Thus you can quickly verify what V is.
>
>   If you run with -ksp_error_if_not_converged then the solver will stop
> exactly when the zero pivot is encountered; this would include some
> information from SuperLU_DIST which might include the row number etc.
>
>   Notes on PETSc improvements needed.
>
> 1) The man page for KSPCheckSolve() is terribly misleading
>
> 2) It would be nice to have a view that displayed the nested fieldsplit
> preconditioners more clearly
>
>
>
>
>
>
> On Jun 9, 2022, at 5:19 PM, Jorti, Zakariae via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
> Hi,
>
> I am solving non-linear problem that has 5 unknowns {ni, T, E, B, V}, and
> for the preconditioning part, I am using a FieldSplit preconditioner. At
> the last fieldsplit/level, we are left with a {B,V} block that tried to
> precondition in 2 different ways:
> a) SuperLU:
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ksp_type preonly
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_pc_type lu
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_pc_factor_mat_solver_type
> superlu_dist
> b) a Schur-based fieldsplit preconditioner that uses SuperLU for both V
> and B blocks:
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ksp_type gmres
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_pc_type fieldsplit
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_pc_fieldsplit_type schur
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_pc_fieldsplit_schur_precondition
> selfp -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_B_ksp_type
> preonly -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_B_pc_type
> lu
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_B_pc_factor_mat_solver_type
> superlu_dist
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_V_ksp_type preonly
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_V_pc_type lu
> -fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_fieldsplit_V_pc_factor_mat_solver_type
> superlu_dist
>
> Option a) yields the following error:
> "     Linear fieldsplit_ni_ solve converged due to CONVERGED_ATOL
> iterations 0
>         Linear fieldsplit_TEBV_fieldsplit_tau_ solve converged due to
> CONVERGED_RTOL iterations 1
>           Linear fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_EP_ solve
> converged due to CONVERGED_RTOL iterations 5
>           Linear fieldsplit_TEBV_fieldsplit_EBV_fieldsplit_BV_ solve did
> not converge due to DIVERGED_PC_FAILED iterations 0
>                          PC failed due to FACTOR_NUMERIC_ZEROPIVOT "
> whereas options b) seems to be working well.
> Is it possible that the SuperLU on the {V,B} block uses a reordering that
> introduces a zero pivot or could there be another explanation for this
> error?
>
> Many thanks.
> Best,
>
> Zakariae
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220610/c309b949/attachment.html>