<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Jun 6, 2015 at 8:21 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Sat, Jun 6, 2015 at 6:02 PM, Young, Matthew, Adam <span dir="ltr"><<a href="mailto:may@bu.edu" target="_blank">may@bu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">This is a problem from ionospheric plasma physics. The simulation treats ions via a particle-in-cell method and electrons as an inertialess fluid, the justification being that ionospheric
ions are 10^4 times more massive than electrons. We further assume that the plasma is effectively neutral on the length scale of interest (i.e. quasi-neutral) and those assumptions allows us to write an elliptic equation for the electrostatic potential, phi:
Div[n(x) T Grad(phi)]. <span style="font-size:10pt">n(x) is the quasi-neutral plasma density, which is updated via an ion gather at each time step, and T is a tensor of constant coefficients that looks like {{1, kappa, 0},{-kappa, 1, 0},{0, 0, 1+kappa^2}},
where kappa is the ratio of gyrofrequency to collision frequency for electrons (~100 for our problem)*. The RHS is a function of density, ion current (or flux, both of which are related to density), and constant electron fluid parameters. </span><span style="font-size:10pt">Eq
1 of the attached paper shows this equation for the 2-D problem in the plane perpendicular to the ambient magnetic field.</span></div></div></div></blockquote><div><br></div></span><div>Its hard to say anything without knowing what the n(x) functions look like. </div></div></div></div></blockquote><div><br></div><div>I would think its pretty smooth unless it is a perturbation, this does not sound like it has shocks or anything ... but T scares the hell out of me! Very anisotropy on the vertical direction with a sort of skew thing going on in the horizontal plane?</div><div><br></div><div>If you are seeing the convergence rate tank as kappa is increased then you will probably find that ASM smoothers help. Making good subdomains for ASM is a problem. If you have small subdomains per process you can use these (just use bjacobi as the smoother). GAMG can let you use the GAMG aggregates as blocks (this has not been tested for ages and we don't have a regression test it). Try just using processor ASM and use as many processors as you can. If this helps your convergence rate (a lot) then this might be the way to go and we can look at GAMG blocks.</div><div><br></div><div>Mark</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>I would say now that it is easy to try GAMG, so</div><div>that is what I would do first.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div><div class="h5"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">--Matt</div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt"><br>
</div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">*It was a little unfair of me to say earlier that the off-diagonal terms grow as the simulation progresses. It's the density that gradients grow and they are multiplied by kappa.<span><br>
<div><br>
</div>
<div><br>
</div>
<div><br>
<div style="font-family:Tahoma;font-size:13px">
<div style="font-family:Tahoma;font-size:13px">--------------------------------------------------------------
<div>Matthew Young</div>
<div>Graduate Student</div>
<div>Boston University Dept. of Astronomy</div>
<div>--------------------------------------------------------------</div>
<div><br>
</div>
</div>
</div>
</div>
</span><div style="font-family:Times New Roman;color:#000000;font-size:16px">
<hr>
<div style="direction:ltr"><font face="Tahoma" size="2" color="#000000"><b>From:</b> Matthew Knepley [<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>]<br>
<b>Sent:</b> Saturday, June 06, 2015 6:12 PM<br>
<b>To:</b> Young, Matthew, Adam<br>
<b>Cc:</b> Justin Chang; Mark Adams; petsc-users<div><div><br>
<b>Subject:</b> Re: [petsc-users] Guidance on GAMG preconditioning<br>
</div></div></font><br>
</div><div><div>
<div></div>
<div>
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Sat, Jun 6, 2015 at 3:00 PM, Young, Matthew, Adam <span dir="ltr">
<<a href="mailto:may@bu.edu" target="_blank">may@bu.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt"><font size="2">Forgive me for being like a child who wanders into the middle of a movie...</font>
<div><font size="2"><br>
</font></div>
<div><font size="2">I've been attempting to follow this conversation from a beginner's level because I am trying to solve an elliptic PDE with variable coefficients. Both the operator and the RHS change at each time step and the operator has off-diagonal terms
that become dominant as the instability of interest grows. I read somewhere that a direct method is the best for this but I'm intrigued by Justin's comment that GAMG seems to be "the preconditioner to use for elliptic problems". I don't want to highjack this
conversation but it seems like a good chance to ask for your collective advice on resources for understanding my problem. Any thoughts?</font></div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>The problem here is that fast methods do not depend on the operator being elliptic so much as they depend on the operator</div>
<div>falling off away from the diagonal (satisfying a Calderon-Zygmund bound, there are lots of ways of expressing this). When</div>
<div>this ceases to be true, these methods stop being fast.</div>
<div><br>
</div>
<div>So the answer is, when you have complicated coefficient structure, there are no general methods and you need to know more</div>
<div>about exactly what is going on. Where is your problem from?</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">
<div><font size="2">--Matt</font></div>
<div><font size="2"><br>
</font>
<div>
<div style="font-family:Tahoma;font-size:13px">
<div style="font-family:Tahoma;font-size:13px">--------------------------------------------------------------
<div>Matthew Young</div>
<div>Graduate Student</div>
<div>Boston University Dept. of Astronomy</div>
<div>--------------------------------------------------------------</div>
<div><br>
</div>
</div>
</div>
</div>
<div style="font-family:Times New Roman;color:#000000;font-size:16px">
<hr>
<div style="direction:ltr"><font face="Tahoma" size="2" color="#000000"><b>From:</b>
<a href="mailto:petsc-users-bounces@mcs.anl.gov" target="_blank">petsc-users-bounces@mcs.anl.gov</a> [<a href="mailto:petsc-users-bounces@mcs.anl.gov" target="_blank">petsc-users-bounces@mcs.anl.gov</a>] on behalf of Justin Chang [<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>]<br>
<b>Sent:</b> Saturday, June 06, 2015 5:29 AM<br>
<b>To:</b> Mark Adams<br>
<b>Cc:</b> petsc-users<br>
<b>Subject:</b> Re: [petsc-users] Guidance on GAMG preconditioning<br>
</font><br>
</div>
<div>
<div>
<div></div>
<div>
<div dir="ltr">Matt and Mark thank you guys for your responses. <br>
<br>
The reason I brought up GAMG was because it seems to me that this is the preconditioner to use for elliptic problems. However, I am using CG/Jacobi for my larger problems and the solver converges (with -ksp_atol and -ksp_rtol set to 1e-8). Using GAMG I get
rough the same wall-clock time, but significantly fewer solver iterations. <br>
<br>
As I also kind of mentioned in another mail, the ultimate purpose is to compare how this "correction" methodology using the TAO solver (with bounded constraints) performs compared to the original methodology using the KSP solver (without constraints). I have
the A for BLMVM and CG/Jacobi and they are roughly 0.3 and 0.2 respectively (do these sound about right?). Although the AI is higher for TAO , the ratio of actual FLOPS/s over the AI*STREAMS BW is smaller, though I am not sure what conclusions to make of that.
This was also partly why I wanted to see what kind of metrics another KSP solver/preconditioner produces.
<div><br>
</div>
<div>Point being, if I were to draw such comparisons between TAO and KSP, would I get crucified if people find out I am using CG/Jacobi and not GAMG? <br>
<div><br>
</div>
</div>
<div>Thanks,<br>
Justin</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Jun 5, 2015 at 2:02 PM, Mark Adams <span dir="ltr">
<<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div><br>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>The overwhleming cost of AMG is the Galerkin triple-product RAP.</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<div> </div>
</span>
<div>That is overstating it a bit. It can be if you have a hard 3D operator and coarsening slowly is best.</div>
<div><br>
</div>
<div>Rule of thumb is you spend 50% time is the solver and 50% in the setup, which is often mostly RAP (in 3D, 2D is much faster). That way you are within 2x of optimal and it often works out that way anyway.</div>
<span><font color="#888888">
<div><br>
</div>
<div>Mark </div>
</font></span></div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
</div>
</div>
</div>
</div></div></div>
</div>
</div>
</div>
</blockquote></div></div></div><div><div class="h5"><br><br clear="all"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div></div></div>
</blockquote></div><br></div></div>