Cg/asm doesn't scale

[Nguyen, Hung V ERDC-ITL-MS]

Hello,

I tried to solver the SPD linear system with using cg/asm preconditioner and
found that it doesn't scale well, see table below. Note: it does scale well
with cg/jacobi preconditioner.

Do you know why it doesn't scale? 

Thanks,

-hung

Number Pes   Solver Time (secs)  #it
1	         31.317345         544
2	        263.172225	  6959
4             734.828840        23233
8             805.217591        41250
16            611.813716        49262
32            345.331928        49792
64            212.084555        53771


---
1 :  aprun -n 1  ./test_matrix_read  -ksp_type cg -pc_type asm -pc_asm_type
basic   -sub_pc_type ilu -sub_ksp_type preonly -ksp_rtol 1.0e-12 -ksp_max_it
100000
Time in PETSc solver: 31.317345 seconds
The number of iteration       = 544
The solution residual error = 1.658653e-08
2 norm 7.885361e-07
 infinity norm 6.738382e-09
 1 norm 2.124207e-04

Application 679466 resources: utime 0, stime 0
************************ Beginning new run ************************

2 :  aprun -n 2  ./test_matrix_read  -ksp_type cg -pc_type asm -pc_asm_type
basic   -sub_pc_type ilu -sub_ksp_type preonly -ksp_rtol 1.0e-12 -ksp_max_it
100000
Time in PETSc solver: 263.172225 seconds
The number of iteration       = 6959
The solution residual error = 1.794494e-08
2 norm 6.579571e-07
 infinity norm 8.745052e-09
 1 norm 1.907733e-04

-- Here is info about matrix A:

Computed <structure:nrows> as <178353>
Computed <structure:symmetry> as <0>
Computed <structure:nnzeros> as <3578321>
Computed <structure:max-nnzeros-per-row> as <27>
Computed <structure:min-nnzeros-per-row> as <6>
Computed <structure:left-bandwidth> as <76553>
Computed <structure:right-bandwidth> as <76553>