GMRES left-preconditioned with ILU1 versus ILU0

Fri May 16 11:22:18 CDT 2008

Hi,
I tried to improve the convergence of a rather badly conditioned linear 
system by increasing the ILU(k) level from k=0 to k=1.
And, whereas I got a convergence for k=0, I ended up with an explosive 
divergence for k=1!!!
The PETCS version is 2.3.2-p8.
The matrix type is MPIBAIJ (block size=7,mesh points=23010,non-empty 
blocks=297454)
The common command line options are:

    * KSP="-ksp_type gmres -ksp_max_it 800 -ksp_gmres_restart 800
      -ksp_rtol 1.0e-12 -ksp_left_pc -ksp_gmres_modifiedgramschmidt
      -ksp_gmres_cgs_refinement_type REFINE_NEVER -ksp_singmonitor
      -ksp_compute_eigenvalues": I'm forcing 800 krylovs without restart
      to get the condition number of the pre-conditioned system (if I
      understand correctly the man page of -ksp_singmonitor)
    * PC="-pc_type asm -pc_asm_overlap 2": I'm planning to run with more
      than 1 partition, but for the time being, only one partition is used.
    * BLK_KSP="-sub_ksp_type preonly": Because of GMRES+ILU

For ILU(0), I'm using:

    * BLK_PC="-sub_pc_type ilu -sub_pc_factor_levels 0
      -sub_pc_factor_fill 1.00 -sub_pc_factor_shift_nonzero
      -sub_pc_factor_mat_ordering_type rcm -sub_pc_factor_pivot_in_blocks"

and I got for convergence:
  0 KSP Residual norm 1.687258996558e+00 % max 1 min 1 max/min 1
  1 KSP Residual norm 1.687132576728e+00 % max 67.8829 min 67.8829 max/min 1
  2 KSP Residual norm 1.685760733293e+00 % max 3496.78 min 19.5582 
max/min 178.789
  3 KSP Residual norm 1.668552043995e+00 % max 3604.26 min 12.0073 
max/min 300.174
  4 KSP Residual norm 1.578511835381e+00 % max 3639.92 min 7.35118 
max/min 495.148
....
795 KSP Residual norm 1.465607932165e-09 % max 18209.9 min 0.00612973 
max/min 2.97075e+06
796 KSP Residual norm 1.390602265424e-09 % max 18227.3 min 0.00612913 
max/min 2.97388e+06
797 KSP Residual norm 1.320529491862e-09 % max 18231.9 min 0.0061286 
max/min 2.97489e+06
798 KSP Residual norm 1.253371917713e-09 % max 18234.8 min 0.00612856 
max/min 2.97538e+06
799 KSP Residual norm 1.188955299647e-09 % max 18278.5 min 0.00612594 
max/min 2.98378e+06
800 KSP Residual norm 1.118294486519e-09 % max 18278.5 min 0.00612475 
max/min 2.98437e+06

and the iterative solution compares very well with the one computed 
using complete LU factorization.

For ILU(1), I'm using:

    * BLK_PC="-sub_pc_type ilu -sub_pc_factor_levels 1
      -sub_pc_factor_fill 3.81 -sub_pc_factor_shift_nonzero
      -sub_pc_factor_mat_ordering_type rcm
      -sub_pc_factor_pivot_in_blocks": RCM gives the smallest fill value.

and I got for "convergence":
  0 KSP Residual norm 7.095990612421e+126 % max 1 min 1 max/min 1
  1 KSP Residual norm 3.313547979190e+123 % max 1.68012e+135 min 
1.68012e+135 max/min 1
  2 KSP Residual norm 1.257750994639e+119 % max 6.34518e+135 min 
3.55953e+131 max/min 17825.9
  3 KSP Residual norm 5.233083258710e+118 % max 1.25538e+136 min 
1.42732e+127 max/min 8.79538e+08
  4 KSP Residual norm 1.938981257595e+118 % max 1.44472e+136 min 
4.82369e+125 max/min 2.99506e+10
  5 KSP Residual norm 3.371270926617e+116 % max 1.45841e+136 min 
1.79839e+125 max/min 8.1095e+10
  6 KSP Residual norm 2.179293254483e+115 % max 1.45842e+136 min 
5.16422e+122 max/min 2.82408e+13
  7 KSP Residual norm 2.120598006522e+115 % max 1.46024e+136 min 
3.67337e+122 max/min 3.97521e+13
  8 KSP Residual norm 1.486820601733e+115 % max 1.461e+136 min 
1.02249e+122 max/min 1.42886e+14
  9 KSP Residual norm 7.653834441859e+114 % max 1.46138e+136 min 
4.93314e+121 max/min 2.96237e+14
 10 KSP Residual norm 5.001204920218e+114 % max 1.47243e+136 min 
4.89675e+121 max/min 3.00694e+14

My guess is that the ILU(1) is singular (zero as diagonal elements?), 
but I thought that the options "-sub_pc_factor_shift_nonzero 
-sub_pc_factor_pivot_in_blocks" were taking care of that... I got lost 
in the source files trying to find out who at the end is computing and 
applying the ILU for MPIBAIJ format (replaced by SEQBAIJ with only one 
partition, I'm guessing).

The question is: What am I doing wrong? I never heard that ILU(1) was 
worst than ILU(0)!
Stef.

-- 
___________________________________________________________
Dr. Stephane AUBERT, CEO & CTO
FLUOREM s.a.s
Centre Scientifique Auguste MOIROUX
64 chemin des MOUILLES
F-69130 ECULLY, FRANCE
International:      fax: +33 4.78.33.99.39      tel: +33 4.78.33.99.35
France:             fax: 04.78.33.99.39         tel: 04.78.33.99.35
email: stephane.aubert at fluorem.com
web: www.fluorem.com