<div dir="ltr">On Fri, Mar 1, 2013 at 12:40 PM, John Mousel <span dir="ltr"><<a href="mailto:elafint.john@gmail.com" target="_blank">elafint.john@gmail.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">So am I right in reading this as you suggesting to: <div><br></div><div>1. Compute VecAXPY on all the u(n),u(n-1)...locally</div>
<div>2. Scatter the result of step (1)</div></div></blockquote><div><br></div><div style>Why is this here? You don't need that at all since the local space is "ghosted".</div><div style><br></div><div style>
Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>3. Perform the divergence on the resulting ghosted vector from (2).</div>
<div><br></div><div>I don't see how that is either cleaner or more efficient than just summing up the ghosted portions of the vector since they have not been altered since the solve at the previous time step.</div>
<div>I hope I'm being clear that what I am proposing is just redundant computation to avoid communication?</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Mar 1, 2013 at 11:31 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>On Fri, Mar 1, 2013 at 10:37 AM, John Mousel <span dir="ltr"><<a href="mailto:elafint.john@gmail.com" target="_blank">elafint.john@gmail.com</a>></span> wrote:<br>
</div><div class="gmail_extra"><div class="gmail_quote"><div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Let me be even more specific. I'm implementing a velocity correction scheme in rotational form (reference given below). In the first step, forgetting even a sub-iterative loop, a projecting pressure is solved for from <br>
</div><div><br></div><div>L(p(n+1)) = -div(beta*u(n) + gamma*u(n-1) + exci*u(n-2) + H(n+1) - H(n) + f(n+1) - f(n))</div><div><br></div><div>Here, L and div are Laplacian and divergence operators, and H(n) and f(n) are the non-linear and body force at time level n. In my mind, to form part of the RHS, I can use VecAXPY on all the terms in the div(), and then perform the divergence on that. To avoid communication, I can just use the VecAXPY on the ghosted portion as well, which since I've copied the entire ghosted vector during the VecCopy(u(n),u(n-1)) operation, should make sense.</div>
<div>Is that not a wise thing to do?</div></div></blockquote><div><br></div></div><div>You have some communication of the ghosts before the solve. Why not work completely in the local space</div><div>for this assembly, and then completely in the global space for the solve? We usually organize this way.</div>
<div><br></div><div> Matt</div><div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>John</div><div><br></div><div><span style="color:rgb(33,47,64);font-family:proxima-nova,'Helvetica Neue',Arial,sans-serif;font-size:14px;white-space:pre-wrap">J. L. Guermond, P. Minev, and J. Shen. An overview of projection methods for incompressible flows. Computer Methods in Applied Mechanics and Engineering, 195(44-47):6011–6045, 2006.</span><br style="color:rgb(33,47,64);font-family:proxima-nova,'Helvetica Neue',Arial,sans-serif;font-size:14px;white-space:pre-wrap">
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Mar 1, 2013 at 9:28 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>On Fri, Mar 1, 2013 at 10:24 AM, John Mousel <span dir="ltr"><<a href="mailto:elafint.john@gmail.com" target="_blank">elafint.john@gmail.com</a>></span> wrote:<br>
</div><div class="gmail_extra"><div class="gmail_quote"><div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">I use the data in the vectors for things besides pure vector operations. In the context of a sub-iteration scheme for an incompressible solver, I want to use the values stored in the vector at the previous iteration to construction the non-linear term. This requires copying not just the local portion of the vector in the u(k)->u(k-1) operation, but the ghosts as well. Doesn't copying the entire ghosted vector make sense in this context?</div>
</blockquote><div><br></div></div><div>It seems like a strange work flow. Since none of the vector operations touch these, you will be getting</div><div>stale values anyway. It would make more sense to me if you were using GlobalToLocal() during the iteration.</div>
<div><br></div><div> Matt</div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div>John</div></div><div class="gmail_extra"><br>
<br><div class="gmail_quote">On Fri, Mar 1, 2013 at 8:09 AM, Jed Brown <span dir="ltr"><<a href="mailto:jedbrown@mcs.anl.gov" target="_blank">jedbrown@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>On Wed, Feb 27, 2013 at 4:56 PM, John Mousel <span dir="ltr"><<a href="mailto:john.mousel@gmail.com" target="_blank">john.mousel@gmail.com</a>></span> wrote:<br>
</div><div class="gmail_extra"><div class="gmail_quote"><div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Is there a possibility of adding a wrapper function around a few basic vector operations such as VecCopy, VecAXPY, VECAXPYPZ... to operate on ghosted vectors? I perform a lot of vector operations including the ghost region to avoid communication. </div>
</blockquote><div><br></div></div><div>Can you explain the context where operating with local forms in this way makes sense? Usually you would either be working locally, in which case you copy between local forms (or purely local work vectors) or you are operating globally in which case there is nothing to gain by applying operations to the local form.</div>
</div></div></div>
</blockquote></div><br></div>
</blockquote></div></div><br><br clear="all"><span><font color="#888888"><div><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener
</div></font></span></div></div>
</blockquote></div><br></div></div>
</blockquote></div></div><div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener
</div></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener
</div></div>