<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Thank you! However as I observed if there are too many blocks (say 200), the performance would decrease drastically. Here are some numbers (collected under ILU as the preconditioner in the sub block -- I am going to test it with LU)<div>#Blocks Setup Time Solving Time (50 Iterations)</div><div>100 0.5s<span class="Apple-tab-span" style="white-space:pre"> </span> 17.5s</div><div>200<span class="Apple-tab-span" style="white-space:pre"> </span> 0.76s<span class="Apple-tab-span" style="white-space:pre"> </span> <span class="Apple-tab-span" style="white-space:pre"> </span> 19.1s</div><div>300 2.75s<span class="Apple-tab-span" style="white-space:pre"> </span> 26.5s</div><div>500<span class="Apple-tab-span" style="white-space:pre"> </span> 7.39s 43.64s</div><div>1000 39.13s 165.83s<br><div><br></div><div>It looks like both the Setup Time and the Solving Time increase significantly as the number of blocks are more than 200. Is this a normal result?</div><div><br></div><div>Thanks!</div><div>Wenlei</div><div><br></div><div><br><div><div>On Sep 23, 2012, at 6:03 PM, Jed Brown wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div class="gmail_quote">On Sun, Sep 23, 2012 at 4:58 PM, Wenlei Xie <span dir="ltr"><<a href="mailto:wenleix@cs.cornell.edu" target="_blank">wenleix@cs.cornell.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div id=":b3h">I am using PETSc to solve linear systems with Block Jacobi preconditioner, and I am currently studying the trade off between the number of blocks.<br>
In my experiments I tried a very sparse 1M*1M matrix and try different number of blocks. I expect the result would be, if I use less blocks (thus each block has larger size), then the convergence rate would increase but the time spend on each iteration would would also increase; while for more blocks (and each block has smaller size), both the convergence rate and the time on each iteration will be decreased. However, the result turns out to that the time on each iteration also decreased as the block size increased. Is it because I misused it?<br>
<br>
I am use Richardson method as the Krylov method, and ILU as the solver inside each block matrix. The matrix contains around 3.5M non-zeros entries and I use METIS to partition it into blocks.</div></blockquote></div><br>
<div>The work required for ILU is essentially linear in the subdomain size so the only possible speedup comes from potentially better cache reuse by small blocks. With the default -sub_ksp_type preonly, that isn't significant yet.</div>
<div><br></div><div>To see the effect you expect, use -sub_pc_type lu. The cost of the direct solve (especially in 3D) grows noticeably superlinearly, thus large subdomains become much more expensive than the overhead of decomposing the problem at that granularity.</div>
</blockquote></div><br></div></div></body></html>