[petsc-users] PC Direct Solution failure

[Barry Smith]

> On Jul 21, 2016, at 4:00 PM, Matthew Overholt <overholt at capesim.com> wrote:
> 
> PETSc Users,
>  
> I am doing a KSPPREONLY solution (of the heat transfer equation using FEA) and comparing several packages like PARDISO and MUMPS, and I am encountering a MatSolve() failure that I am having trouble diagnosing.  The matrix inversion fails and I get “nan”.  The failure only happens for certain input files, and its not (just) related to problem size.  By making a slight change to the geometry of the problem I can get it to solve.
>  
> The SuperLu solver is the only one that will give me any error message:
> -ksp_type preonly -pc_type lu –pc_mat_solver_package superlu –info
> I get the error message:
> [0] MatSolve(): MatFactorError 2
> Is that a PCFailedReason of PC_FACTOR_NUMERIC_ZEROPIVOT?

  Yes

typedef enum {MAT_FACTOR_NOERROR,MAT_FACTOR_STRUCT_ZEROPIVOT,MAT_FACTOR_NUMERIC_ZEROPIVOT,MAT_FACTOR_OUTMEMORY,MAT_FACTOR_OTHER} MatFactorError;

  There are a few SuperLU options that could potentially alleviate the problem of the zero pivot: Run with -help to see them all or look at the manual page for MATSOLVERSUPERLU

+ -mat_superlu_equil <FALSE>            - Equil (None)
. -mat_superlu_colperm <COLAMD>         - (choose one of) NATURAL MMD_ATA MMD_AT_PLUS_A COLAMD
. -mat_superlu_iterrefine <NOREFINE>    - (choose one of) NOREFINE SINGLE DOUBLE EXTRA
. -mat_superlu_symmetricmode: <FALSE>   - SymmetricMode (None)
. -mat_superlu_diagpivotthresh <1>      - DiagPivotThresh (None)
. -mat_superlu_pivotgrowth <FALSE>      - PivotGrowth (None)
. -mat_superlu_conditionnumber <FALSE>  - ConditionNumber (None)
. -mat_superlu_rowperm <NOROWPERM>      - (choose one of) NOROWPERM LargeDiag
. -mat_superlu_replacetinypivot <FALSE> - ReplaceTinyPivot (None)

but they may introduce a different problem for a different matrix.

The thing with sparse direct solvers is they can work fine for some matrices but when you change the matrix slightly they don't work, they can also work for some orderings and not for others and if you change the matrix it may be a different ordering is better for that matrix than the ordering for a different matrix. So generally for a particular matrix you might be able to get things to run but I know of no way to bullet proof the direct solver so it will always work when you throw different matrices at it unless you manually change some options.



> If so, is there a way to perturb the pivot in some way?
>  
> In another (non-PETSc) code which uses MKL PARDISO I am able to solve the exact same problem by the same approach without any issues, and that code gives PARDISO a pivot perturbation flag value.

  I'm not surprised. Different sparse solvers will work better on some classes of matrices than others but it is not easy to predict in advance which solver will be best.  Generally for each type of simulation we do we try out the different sparse direct solvers and then pick the one that seems the most robust for that simulation.  Note that since each solver package has its own tuning options this can be a annoying because you need to find the tuning options for each package and see if they help.

  Barry

>  
> Is there a better way to figure out what is happening?  I have been running the code in TotalView with extreme memory checks and everything appears to be ok.
>  
> Thanks,
> Matt Overholt
> CapeSym, Inc.
>  
>  
> 
> 	Virus-free. www.avast.com