[petsc-users] Rock & Hard Place with SuperLU

[Brian Helenbrook]

Hi Again,

Thanks for everybody's prompt help.  I got petsc3.2-p7 running again and that works so at least I am able to get results again.  

I ran my test case with petsc3.2-p7 and petsc3.3-p5 and turned on ksp_view output.   The file "output" is identical except for one line:

tolerance for zero pivot 2.22045e-14 for petsc3.3-p5 and
tolerance for zero pivot 1e-12 for petsc3.2-p7

This is the file "output" from ksp_view

KSP Object: 2 MPI processes
  type: preonly
  maximum iterations=10000, initial guess is zero
  tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
  left preconditioning
  using NONE norm type for convergence test
PC Object: 2 MPI processes
  type: lu
    LU: out-of-place factorization
    tolerance for zero pivot 2.22045e-14
    matrix ordering: natural
    factor fill ratio given 0, needed 0
      Factored matrix follows:
        Matrix Object:         2 MPI processes
          type: mpiaij
          rows=6957, cols=6957
          package used to perform factorization: superlu_dist
          total: nonzeros=0, allocated nonzeros=0
          total number of mallocs used during MatSetValues calls =0
            SuperLU_DIST run parameters:
              Process grid nprow 2 x npcol 1 
              Equilibrate matrix TRUE 
              Matrix input mode 1 
              Replace tiny pivots TRUE 
              Use iterative refinement FALSE 
              Processors in row 2 col partition 1 
              Row permutation LargeDiag 
              Column permutation METIS_AT_PLUS_A
              Parallel symbolic factorization FALSE 
              Repeated factorization SamePattern_SameRowPerm
  linear system matrix = precond matrix:
  Matrix Object:   2 MPI processes
    type: mpiaij
    rows=6957, cols=6957
    total: nonzeros=611043, allocated nonzeros=0
    total number of mallocs used during MatSetValues calls =0
      using I-node (on process 0) routines: found 1407 nodes, limit used is 5

The residual vector going into the KSP inversion is the same, but the inversion gives a different answer

petsc3.3-p5:

jacobian made 4.628e-01 seconds
matrix inverted 9.669e-01 seconds
# iterations 1 residual0 1.824e-05 du 4.290e-05 solve time: 1.821e-02 seconds

petsc3.2-p7:

jacobian made 4.279e-01 seconds
matrix inverted 6.854e-01 seconds
# iterations 1 residual0 1.824e-05 du 1.885e-05 solve time: 1.284e-02 seconds

Where the output is calculated as:

	double resmax;
	VecNorm(petsc_f, NORM_2, &resmax );
	
	PetscGetTime(&time1);
	err = KSPSolve(ksp,petsc_f,petsc_du);
	CHKERRABORT(MPI_COMM_WORLD,err);
	
	double resmax2;
	VecNorm(petsc_du, NORM_2, &resmax2 );
	
	KSPGetIterationNumber(ksp,&its);
	PetscGetTime(&time2);
	*gbl->log << "# iterations " << its << " residual0 " << resmax << " du " << resmax2 << " solve time: " << time2-time1 << " seconds" << endl;


I can output the jacobian and make sure that is the same, but I'm guessing that it is.     Any other suggestions of things to try?   I'd be surprised if the zero-pivot thing had much to do with it although I could be wrong.   I've arranged the matrices to try and make them diagonally dominant going in.   Does anyone know how to change that setting of the top of their head (i.e. without me reading the manual).

Thanks again,

Brian








-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20130128/8815a63f/attachment-0001.html>