<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Aug 5, 2013 at 4:14 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div class="h5">On Mon, Aug 5, 2013 at 8:48 AM, Bishesh Khanal <span dir="ltr"><<a href="mailto:bisheshkh@gmail.com" target="_blank">bisheshkh@gmail.com</a>></span> wrote:<br>
</div></div><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Aug 5, 2013 at 3:17 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>On Mon, Aug 5, 2013 at 7:54 AM, Bishesh Khanal <span dir="ltr"><<a href="mailto:bisheshkh@gmail.com" target="_blank">bisheshkh@gmail.com</a>></span> wrote:<br>
</div></div><div class="gmail_extra"><div><div><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Jul 17, 2013 at 9:48 PM, Jed Brown <span dir="ltr"><<a href="mailto:jedbrown@mcs.anl.gov" target="_blank">jedbrown@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Bishesh Khanal <<a href="mailto:bisheshkh@gmail.com" target="_blank">bisheshkh@gmail.com</a>> writes:<br>
<br>
> Now, I implemented two different approaches, each for both 2D and 3D, in<br>
> MATLAB. It works for the smaller sizes but I have problems solving it for<br>
> the problem size I need (250^3 grid size).<br>
> I use staggered grid with p on cell centers, and components of v on cell<br>
> faces. Similar split up of K to cell center and faces to account for the<br>
> variable viscosity case)<br>
<br>
</div>Okay, you're using a staggered-grid finite difference discretization of<br>
variable-viscosity Stokes. This is a common problem and I recommend<br>
starting with PCFieldSplit with Schur complement reduction (make that<br>
work first, then switch to block preconditioner). You can use PCLSC or<br>
(probably better for you), assemble a preconditioning matrix containing<br>
the inverse viscosity in the pressure-pressure block. This diagonal<br>
matrix is a spectrally equivalent (or nearly so, depending on<br>
discretization) approximation of the Schur complement. The velocity<br>
block can be solved with algebraic multigrid. Read the PCFieldSplit<br>
docs (follow papers as appropriate) and let us know if you get stuck.<br></blockquote><div><br></div><div>I was trying to assemble the inverse viscosity diagonal matrix to use as the preconditioner for the Schur complement solve step as you suggested. I've few questions about the ways to implement this in Petsc:<br>
</div><div>A naive approach that I can think of would be to create a vector with its components as reciprocal viscosities of the cell centers corresponding to the pressure variables, and then create a diagonal matrix from this vector. However I'm not sure about:<br>
How can I make this matrix, (say S_p) compatible to the Petsc distribution of the different rows of the main system matrix over different processors ? The main matrix was created using the DMDA structure with 4 dof as explained before. <br>
</div></div>The main matrix correspond to the DMDA with 4 dofs but for the S_p matrix would correspond to only pressure space. Should the distribution of the rows of S_p among different processor not correspond to the distribution of the rhs vector, say h' if it is solving for p with Sp = h' where S = A11 inv(A00) A01 ? <br>
</div></div></blockquote></div><br clear="all"></div></div><div>PETSc distributed vertices, not dofs, so it never breaks blocks. The P distribution is the same as the entire problem divided by 4.</div></div></div></blockquote>
<div><br></div><div>Thanks Matt. So if I create a new DMDA with same grid size but with dof=1 instead of 4, the vertices for this new DMDA will be identically distributed as for the original DMDA ? Or should I inform PETSc by calling a particular function to make these two DMDA have identical distribution of the vertices ?<br>
</div></div></div></div></blockquote><div><br></div></div></div><div>Yes.</div><div class="im"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><div>Even then I think there might be a problem due to the presence of "fictitious pressure vertices". The system matrix (A) contains an identity corresponding to these fictitious pressure nodes, thus when using a -pc_fieldsplit_detect_saddle_point, will detect a A11 zero block of size that correspond to only non-fictitious P-nodes. So the preconditioner S_p for the Schur complement outer solve with Sp = h' will also need to correspond to only the non-fictitious P-nodes. This means its size does not directly correspond to the DMDA grid defined for the original problem. Could you please suggest an efficient way of assembling this S_p matrix ?</div>
</div></div></div></blockquote><div><br></div></div><div>Don't use detect_saddle, but split it by fields -pc_fieldsplit_0_fields 0,1,2 -pc_fieldsplit_1_fields 4</div></div></div></div></blockquote><div><br></div><div>
How can I set this split in the code itself without giving it as a command line option when the system matrix is assembled from the DMDA for the whole system with 4 dofs. (i.e. <i>without</i> using the DMComposite or <i>without</i> using the nested block matrices to assemble different blocks separately and then combine them together). <br>
I need the split to get access to the fieldsplit_1_ksp in my code, because not using detect_saddle_point means I cannot use -fieldsplit_1_ksp_constant_null_space due to the presence of identity for the fictitious pressure nodes present in the fieldsplit_1_ block. I need to use PCFieldSplitGetSubKsp() so that I can set proper null-space basis.<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div> Matt</div><div class="im">
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div><br></div>
<div> Matt</div><span><font color="#888888"><div>
<div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener
</div></font></span></div></div>
</blockquote></div><br></div></div>
</blockquote></div></div><div class="im"><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener
</div></div></div>
</blockquote></div><br></div></div>