<div class="gmail_quote">How small is the Reynolds number? There is a difference between 0.1 and 100, although both may be laminar.</div><div class="gmail_quote"><br></div><div class="gmail_quote">On Fri, May 13, 2011 at 15:50, Henning Sauerland <span dir="ltr"><<a href="mailto:uerland@gmail.com">uerland@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div>The problem is discretized using FEM (more precisely XFEM) with stabilized, trilinear hexahedral elements. As the XFEM approximation space is time-dependant as well as the physical properties at the nodes the resulting system may change quite significantly between time steps. </div>
</blockquote><div><br></div><div>I assume the XFEM basis is just resolving the jump across the interface. Does this mean the size of the matrix is changing as the interface moves? In any case, it looks like you can't amortize setup costs between time steps, so we need a solution short of a direct solve. </div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div></div><div><div>ILU(2) requires less than half the number of KSP iterations, but it scales similar to ILU(0) and requires about 1/3 more time.</div>
</div></blockquote><div><br></div><div>Eventually, we're going to need a coarse level to battle the poor scaling with more processes. Hopefully that will alleviate the need for these more expensive local solves.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div class="im">I guess you are talking about the nonlinear iterations? I was always referring to the KSP iterations and I thought that the ksp iteration count grows with increasing number of processors is more or less solely related to the iterative solver and preconditioner.</div>
</div></blockquote><div><br></div><div>I meant linear iterations. It is mostly dependent on preconditioner. The increased iteration count (when a direct subdomain solver is used, inexact subdomain solves confuse things) is likely due to the fundamental scaling for elliptic problems. There are many ways to improve constants, but you need a coarse level to fix the asymptotics.</div>
<div><br></div><div>Unfortunately, multigrid methods for XFEM are a recent topic. Perhaps the best results I have seen (at conferences) use some geometric information in an otherwise algebraic framework. For this problem (unlike many fracture problems), the influence of the extended basis functions may be local enough that you can build a coarse level using the conventional part of the problem. The first thing I would try is probably to see if a direct solve with the conventional part makes an effective coarse level. If that works, I would see if ML or Hypre can do a reasonable job with that part of the problem.</div>
<div><br></div><div>I have no great confidence that this will work, it's highly dependent on how local the influence of the extended basis functions is. Perhaps you have enough experience with the method to hypothesize.</div>
<div><br></div><div>Note: for the conventional part of the problem, it is still incompressible flow. It sounds like you are using equal-order elements (Q1-Q1 stabilized; PSPG or Dohrmann&Bochev?). For those elements, you will want to interlace the velocity and pressure degrees of freedom, then use a smoother that respects the block size. PETSc's ML and Hypre interfaces forward this information if you set the block size on the matrix. When using ML, you'll probably have to make the smoothers stronger. There are also some "energy minimization" options that may help.</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div class="im"><blockquote type="cite"><div class="gmail_quote">
</div>
</blockquote></div>ibcgs is slightly faster, requiring less number of ksp iterations compared to lgmres. Unfortunately, the iteration count scales very similar to lgmres and generally the lack of robustness of bcgs solvers turns out to problematic for tougher testcases in my experience.</div>
</blockquote></div><br><div>Yes, that suggestion was only an attempt to improve the constants a little.</div>