[petsc-users] Number of Blocks in the Block Jacobi preconditionr

[Wenlei Xie]

Thank you! However as I observed if there are too many blocks (say 200), the performance would decrease drastically. Here are some numbers (collected under ILU as the preconditioner in the sub block -- I am going to test it with LU)
#Blocks       Setup Time          Solving Time (50 Iterations)
100              0.5s       17.5s
200    0.76s           19.1s
300              2.75s       26.5s
500    7.39s                   43.64s
1000            39.13s                 165.83s

It looks like both the Setup Time and the Solving Time increase significantly as the number of blocks are more than 200. Is this a normal result?

Thanks!
Wenlei


On Sep 23, 2012, at 6:03 PM, Jed Brown wrote:

On Sun, Sep 23, 2012 at 4:58 PM, Wenlei Xie <wenleix at cs.cornell.edu<mailto:wenleix at cs.cornell.edu>> wrote:
I am using PETSc to solve linear systems with Block Jacobi preconditioner, and I am currently studying the trade off between the number of blocks.
In my experiments I tried a very sparse 1M*1M matrix and try different number of blocks. I expect the result would be, if I use less blocks (thus each block has larger size), then the convergence rate would increase but the time spend on each iteration would would also increase; while for more blocks (and each block has smaller size), both the convergence rate and the time on each iteration will be decreased. However, the result turns out to that the time on each iteration also decreased as the block size increased. Is it because I misused it?

I am use Richardson method as the Krylov method, and ILU as the solver inside each block matrix. The matrix contains around 3.5M non-zeros entries and I use METIS to partition it into blocks.

The work required for ILU is essentially linear in the subdomain size so the only possible speedup comes from potentially better cache reuse by small blocks. With the default -sub_ksp_type preonly, that isn't significant yet.

To see the effect you expect, use -sub_pc_type lu. The cost of the direct solve (especially in 3D) grows noticeably superlinearly, thus large subdomains become much more expensive than the overhead of decomposing the problem at that granularity.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120923/3e15de83/attachment.html>