[petsc-users] Rock & Hard Place with SuperLU

[Matthew Knepley]

On Mon, Jan 28, 2013 at 7:46 PM, Brian Helenbrook <bhelenbr at clarkson.edu>wrote:

> Hi Again,
>
> Thanks for everybody's prompt help.  I got petsc3.2-p7 running again and
> that works so at least I am able to get results again.
>
> I ran my test case with petsc3.2-p7 and petsc3.3-p5 and turned on ksp_view
> output.   The file "output" is identical except for one line:
>
> tolerance for zero pivot 2.22045e-14 for petsc3.3-p5 and
> tolerance for zero pivot 1e-12 for petsc3.2-p7
>
> This is the file "output" from ksp_view
>
> KSP Object: 2 MPI processes
>   type: preonly
>   maximum iterations=10000, initial guess is zero
>   tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
>   left preconditioning
>   using NONE norm type for convergence test
> PC Object: 2 MPI processes
>   type: lu
>     LU: out-of-place factorization
>     tolerance for zero pivot 2.22045e-14
>     matrix ordering: natural
>     factor fill ratio given 0, needed 0
>       Factored matrix follows:
>         Matrix Object:         2 MPI processes
>           type: mpiaij
>           rows=6957, cols=6957
>           package used to perform factorization: superlu_dist
>           total: nonzeros=0, allocated nonzeros=0
>           total number of mallocs used during MatSetValues calls =0
>             SuperLU_DIST run parameters:
>               Process grid nprow 2 x npcol 1
>               Equilibrate matrix TRUE
>               Matrix input mode 1
>               Replace tiny pivots TRUE
>               Use iterative refinement FALSE
>               Processors in row 2 col partition 1
>               Row permutation LargeDiag
>               Column permutation METIS_AT_PLUS_A
>               Parallel symbolic factorization FALSE
>               Repeated factorization SamePattern_SameRowPerm
>   linear system matrix = precond matrix:
>   Matrix Object:   2 MPI processes
>     type: mpiaij
>     rows=6957, cols=6957
>     total: nonzeros=611043, allocated nonzeros=0
>     total number of mallocs used during MatSetValues calls =0
>       using I-node (on process 0) routines: found 1407 nodes, limit used
> is 5
>
> The residual vector going into the KSP inversion is the same, but the
> inversion gives a different answer
>
> petsc3.3-p5:
>
> jacobian made 4.628e-01 seconds
> matrix inverted 9.669e-01 seconds
> # iterations 1 residual0 1.824e-05 du 4.290e-05 solve time: 1.821e-02
> seconds
>
> petsc3.2-p7:
>
> jacobian made 4.279e-01 seconds
> matrix inverted 6.854e-01 seconds
> # iterations 1 residual0 1.824e-05 du 1.885e-05 solve time: 1.284e-02
> seconds
>
> Where the output is calculated as:
>
> double resmax;
> VecNorm(petsc_f, NORM_2, &resmax );
>
>
> PetscGetTime(&time1);
> err = KSPSolve(ksp,petsc_f,petsc_du);
> CHKERRABORT(MPI_COMM_WORLD,err);
>
>
> double resmax2;
> VecNorm(petsc_du, NORM_2, &resmax2 );
>
>
> KSPGetIterationNumber(ksp,&its);
> PetscGetTime(&time2);
> *gbl->log << "# iterations " << its << " residual0 " << resmax << " du "<< resmax2 << "
> solve time: " << time2-time1 << " seconds" << endl;
>
>
> I can output the jacobian and make sure that is the same, but I'm guessing
> that it is.     Any other suggestions of things to try?   I'd be surprised
> if the zero-pivot thing had much to do with it although I could be wrong.
> I've arranged the matrices to try and make them diagonally dominant going
> in.   Does anyone know how to change that setting of the top of their head
> (i.e. without me reading the manual).
>

Excellent!

1) Output the matrix and rhs in PETSc binary format, either with a call to
Vec/MatView(), or with -ksp_view_binary

2) Run your setup using KSP ex10

3) Send them (or post them) to petsc-maint at mcs.anl.gov and we can also do
that and find the bug

  Thanks,

      Matt


> Thanks again,
>
> Brian
>
>
>
>
>
>
>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20130128/4697bce8/attachment.html>