[petsc-users] KSP solver increases the solution time

[Alejandro Marcos Aragón]

Hi all,

I'm using the standard configuration of the KSP solver, but the time it takes to solve a large system of equations is increasing (because of the increasing number of iterations?). These are my timing lines and the log from the KSP solver in two consecutive solves:

[1]   Solving system... 0.154203 s

KSP Object:
  type: gmres
    GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
    GMRES: happy breakdown tolerance 1e-30
  maximum iterations=10000
  tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
  left preconditioning
  using nonzero initial guess
  using PRECONDITIONED norm type for convergence test
PC Object:
  type: bjacobi
    block Jacobi: number of blocks = 3
    Local solve is same for all blocks, in the following KSP and PC objects:
  KSP Object:(sub_)
    type: preonly
    maximum iterations=10000, initial guess is zero
    tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
    left preconditioning
    using PRECONDITIONED norm type for convergence test
  PC Object:(sub_)
    type: ilu
      ILU: out-of-place factorization
      0 levels of fill
      tolerance for zero pivot 1e-12
      using diagonal shift to prevent zero pivot
      matrix ordering: natural
      factor fill ratio given 1, needed 1
        Factored matrix follows:
          Matrix Object:
            type=seqaij, rows=2020, cols=2020
            package used to perform factorization: petsc
            total: nonzeros=119396, allocated nonzeros=163620
              using I-node routines: found 676 nodes, limit used is 5
    linear system matrix = precond matrix:
    Matrix Object:
      type=seqaij, rows=2020, cols=2020
      total: nonzeros=119396, allocated nonzeros=163620
        using I-node routines: found 676 nodes, limit used is 5
  linear system matrix = precond matrix:
  Matrix Object:
    type=mpiaij, rows=6058, cols=6058
    total: nonzeros=365026, allocated nonzeros=509941
      using I-node (on process 0) routines: found 676 nodes, limit used is 5


[1]   System solved in 51 iterations... 0.543215 s
...
...
...

[1]   Solving system... 0.302414 s

KSP Object:
  type: gmres
    GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
    GMRES: happy breakdown tolerance 1e-30
  maximum iterations=10000
  tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
  left preconditioning
  using nonzero initial guess
  using PRECONDITIONED norm type for convergence test
PC Object:
  type: bjacobi
    block Jacobi: number of blocks = 3
    Local solve is same for all blocks, in the following KSP and PC objects:
  KSP Object:(sub_)
    type: preonly
    maximum iterations=10000, initial guess is zero
    tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
    left preconditioning
    using PRECONDITIONED norm type for convergence test
  PC Object:(sub_)
    type: ilu
      ILU: out-of-place factorization
      0 levels of fill
      tolerance for zero pivot 1e-12
      using diagonal shift to prevent zero pivot
      matrix ordering: natural
      factor fill ratio given 1, needed 1
        Factored matrix follows:
          Matrix Object:
            type=seqaij, rows=2020, cols=2020
            package used to perform factorization: petsc
            total: nonzeros=119396, allocated nonzeros=163620
              using I-node routines: found 676 nodes, limit used is 5
    linear system matrix = precond matrix:
    Matrix Object:
      type=seqaij, rows=2020, cols=2020
      total: nonzeros=119396, allocated nonzeros=163620
        using I-node routines: found 676 nodes, limit used is 5
  linear system matrix = precond matrix:
  Matrix Object:
    type=mpiaij, rows=6058, cols=6058
    total: nonzeros=365026, allocated nonzeros=509941
      using I-node (on process 0) routines: found 676 nodes, limit used is 5

[1]   System solved in 3664 iterations... 42.683 s



As you can see, the second iteration takes more than 40 seconds to solve. Could some explain why this is happening and why he number of iterations is increasing dramatically between solves? Thank you all,

Alejandro M. Aragón, Ph.D.