<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Mar 6, 2014 at 5:49 PM, Mohammad Mirzadeh <span dir="ltr"><<a href="mailto:mirzadeh@gmail.com" target="_blank">mirzadeh@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Matt,</div></blockquote><div><br></div><div>1) Use -ksp_monitor_true_residual</div><div><br></div><div>2) Quit using Hypre. It can easily get rank deficient stuff on its coarse grid, which throws off this analysis.</div>
<div> Use SOR if it converges, otherwise use -pc_type svd</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">
</div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>here's the output:</div><div>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>0<span style="color:rgb(192,192,192)"> </span>KSP<span style="color:rgb(192,192,192)"> </span>Residual<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>3.033840960250e+02<span style="color:rgb(192,192,192)"> </span></pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>1<span style="color:rgb(192,192,192)"> </span>KSP<span style="color:rgb(192,192,192)"> </span>Residual<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>1.018974811826e+01<span style="color:rgb(192,192,192)"> </span></pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>2<span style="color:rgb(192,192,192)"> </span>KSP<span style="color:rgb(192,192,192)"> </span>Residual<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>5.450493684941e-02<span style="color:rgb(192,192,192)"> </span></pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>3<span style="color:rgb(192,192,192)"> </span>KSP<span style="color:rgb(192,192,192)"> </span>Residual<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>3.944064039516e-02<span style="color:rgb(192,192,192)"> </span></pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>4<span style="color:rgb(192,192,192)"> </span>KSP<span style="color:rgb(192,192,192)"> </span>Residual<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>6.286181172600e-05<span style="color:rgb(192,192,192)"> </span></pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>5<span style="color:rgb(192,192,192)"> </span>KSP<span style="color:rgb(192,192,192)"> </span>Residual<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>4.349133658643e-06<span style="color:rgb(192,192,192)"> </span></pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>6<span style="color:rgb(192,192,192)"> </span>KSP<span style="color:rgb(192,192,192)"> </span>Residual<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>9.279429568232e-08<span style="color:rgb(192,192,192)"> </span></pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>7<span style="color:rgb(192,192,192)"> </span>KSP<span style="color:rgb(192,192,192)"> </span>Residual<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>3.032522248740e-09<span style="color:rgb(192,192,192)"> </span></pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>8<span style="color:rgb(192,192,192)"> </span>KSP<span style="color:rgb(192,192,192)"> </span>Residual<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>6.533747246875e-09<span style="color:rgb(192,192,192)"> </span></pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>9<span style="color:rgb(192,192,192)"> </span>KSP<span style="color:rgb(192,192,192)"> </span>Residual<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>6.083185162500e-12<span style="color:rgb(192,192,192)"> </span></pre>
<pre style="margin-top:0px;margin-bottom:0px">Linear<span style="color:rgb(192,192,192)"> </span>solve<span style="color:rgb(192,192,192)"> </span>converged<span style="color:rgb(192,192,192)"> </span>due<span style="color:rgb(192,192,192)"> </span>to<span style="color:rgb(192,192,192)"> </span>CONVERGED_RTOL<span style="color:rgb(192,192,192)"> </span>iterations<span style="color:rgb(192,192,192)"> </span>9</pre>
<pre style="margin-top:0px;margin-bottom:0px">KSP<span style="color:rgb(192,192,192)"> </span>Object:<span style="color:rgb(192,192,192)"> </span>1<span style="color:rgb(192,192,192)"> </span>MPI<span style="color:rgb(192,192,192)"> </span>processes</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>type:<span style="color:rgb(192,192,192)"> </span>bcgs</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>maximum<span style="color:rgb(192,192,192)"> </span>iterations=10000,<span style="color:rgb(192,192,192)"> </span>initial<span style="color:rgb(192,192,192)"> </span>guess<span style="color:rgb(192,192,192)"> </span>is<span style="color:rgb(192,192,192)"> </span>zero</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>tolerances:<span style="color:rgb(192,192,192)"> </span>relative=1e-12,<span style="color:rgb(192,192,192)"> </span>absolute=1e-50,<span style="color:rgb(192,192,192)"> </span>divergence=10000</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>left<span style="color:rgb(192,192,192)"> </span>preconditioning</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>has<span style="color:rgb(192,192,192)"> </span>attached<span style="color:rgb(192,192,192)"> </span>null<span style="color:rgb(192,192,192)"> </span>space</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>using<span style="color:rgb(192,192,192)"> </span>PRECONDITIONED<span style="color:rgb(192,192,192)"> </span>norm<span style="color:rgb(192,192,192)"> </span>type<span style="color:rgb(192,192,192)"> </span>for<span style="color:rgb(192,192,192)"> </span>convergence<span style="color:rgb(192,192,192)"> </span>test</pre>
<pre style="margin-top:0px;margin-bottom:0px">PC<span style="color:rgb(192,192,192)"> </span>Object:<span style="color:rgb(192,192,192)"> </span>1<span style="color:rgb(192,192,192)"> </span>MPI<span style="color:rgb(192,192,192)"> </span>processes</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>type:<span style="color:rgb(192,192,192)"> </span>hypre</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG<span style="color:rgb(192,192,192)"> </span>preconditioning</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Cycle<span style="color:rgb(192,192,192)"> </span>type<span style="color:rgb(192,192,192)"> </span>V</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Maximum<span style="color:rgb(192,192,192)"> </span>number<span style="color:rgb(192,192,192)"> </span>of<span style="color:rgb(192,192,192)"> </span>levels<span style="color:rgb(192,192,192)"> </span>25</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Maximum<span style="color:rgb(192,192,192)"> </span>number<span style="color:rgb(192,192,192)"> </span>of<span style="color:rgb(192,192,192)"> </span>iterations<span style="color:rgb(192,192,192)"> </span>PER<span style="color:rgb(192,192,192)"> </span>hypre<span style="color:rgb(192,192,192)"> </span>call<span style="color:rgb(192,192,192)"> </span>1</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Convergence<span style="color:rgb(192,192,192)"> </span>tolerance<span style="color:rgb(192,192,192)"> </span>PER<span style="color:rgb(192,192,192)"> </span>hypre<span style="color:rgb(192,192,192)"> </span>call<span style="color:rgb(192,192,192)"> </span>0</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Threshold<span style="color:rgb(192,192,192)"> </span>for<span style="color:rgb(192,192,192)"> </span>strong<span style="color:rgb(192,192,192)"> </span>coupling<span style="color:rgb(192,192,192)"> </span>0.5</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Interpolation<span style="color:rgb(192,192,192)"> </span>truncation<span style="color:rgb(192,192,192)"> </span>factor<span style="color:rgb(192,192,192)"> </span>0</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Interpolation:<span style="color:rgb(192,192,192)"> </span>max<span style="color:rgb(192,192,192)"> </span>elements<span style="color:rgb(192,192,192)"> </span>per<span style="color:rgb(192,192,192)"> </span>row<span style="color:rgb(192,192,192)"> </span>0</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Number<span style="color:rgb(192,192,192)"> </span>of<span style="color:rgb(192,192,192)"> </span>levels<span style="color:rgb(192,192,192)"> </span>of<span style="color:rgb(192,192,192)"> </span>aggressive<span style="color:rgb(192,192,192)"> </span>coarsening<span style="color:rgb(192,192,192)"> </span>0</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Number<span style="color:rgb(192,192,192)"> </span>of<span style="color:rgb(192,192,192)"> </span>paths<span style="color:rgb(192,192,192)"> </span>for<span style="color:rgb(192,192,192)"> </span>aggressive<span style="color:rgb(192,192,192)"> </span>coarsening<span style="color:rgb(192,192,192)"> </span>1</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Maximum<span style="color:rgb(192,192,192)"> </span>row<span style="color:rgb(192,192,192)"> </span>sums<span style="color:rgb(192,192,192)"> </span>0.9</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Sweeps<span style="color:rgb(192,192,192)"> </span>down<span style="color:rgb(192,192,192)"> </span>1</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Sweeps<span style="color:rgb(192,192,192)"> </span>up<span style="color:rgb(192,192,192)"> </span>1</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Sweeps<span style="color:rgb(192,192,192)"> </span>on<span style="color:rgb(192,192,192)"> </span>coarse<span style="color:rgb(192,192,192)"> </span>1</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Relax<span style="color:rgb(192,192,192)"> </span>down<span style="color:rgb(192,192,192)"> </span>symmetric-SOR/Jacobi</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Relax<span style="color:rgb(192,192,192)"> </span>up<span style="color:rgb(192,192,192)"> </span>symmetric-SOR/Jacobi</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Relax<span style="color:rgb(192,192,192)"> </span>on<span style="color:rgb(192,192,192)"> </span>coarse<span style="color:rgb(192,192,192)"> </span>symmetric-SOR/Jacobi</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Relax<span style="color:rgb(192,192,192)"> </span>weight<span style="color:rgb(192,192,192)"> </span>(all)<span style="color:rgb(192,192,192)"> </span>1</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Outer<span style="color:rgb(192,192,192)"> </span>relax<span style="color:rgb(192,192,192)"> </span>weight<span style="color:rgb(192,192,192)"> </span>(all)<span style="color:rgb(192,192,192)"> </span>1</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Using<span style="color:rgb(192,192,192)"> </span>CF-relaxation</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Measure<span style="color:rgb(192,192,192)"> </span>type<span style="color:rgb(192,192,192)"> </span>local</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Coarsen<span style="color:rgb(192,192,192)"> </span>type<span style="color:rgb(192,192,192)"> </span>Falgout</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>HYPRE<span style="color:rgb(192,192,192)"> </span>BoomerAMG:<span style="color:rgb(192,192,192)"> </span>Interpolation<span style="color:rgb(192,192,192)"> </span>type<span style="color:rgb(192,192,192)"> </span>classical</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>linear<span style="color:rgb(192,192,192)"> </span>system<span style="color:rgb(192,192,192)"> </span>matrix<span style="color:rgb(192,192,192)"> </span>=<span style="color:rgb(192,192,192)"> </span>precond<span style="color:rgb(192,192,192)"> </span>matrix:</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>Mat<span style="color:rgb(192,192,192)"> </span>Object:<span style="color:rgb(192,192,192)"> </span>1<span style="color:rgb(192,192,192)"> </span>MPI<span style="color:rgb(192,192,192)"> </span>processes</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>type:<span style="color:rgb(192,192,192)"> </span>seqaij</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>rows=263169,<span style="color:rgb(192,192,192)"> </span>cols=263169</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>total:<span style="color:rgb(192,192,192)"> </span>nonzeros=1236141,<span style="color:rgb(192,192,192)"> </span>allocated<span style="color:rgb(192,192,192)"> </span>nonzeros=1244417</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>total<span style="color:rgb(192,192,192)"> </span>number<span style="color:rgb(192,192,192)"> </span>of<span style="color:rgb(192,192,192)"> </span>mallocs<span style="color:rgb(192,192,192)"> </span>used<span style="color:rgb(192,192,192)"> </span>during<span style="color:rgb(192,192,192)"> </span>MatSetValues<span style="color:rgb(192,192,192)"> </span>calls<span style="color:rgb(192,192,192)"> </span>=0</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>has<span style="color:rgb(192,192,192)"> </span>attached<span style="color:rgb(192,192,192)"> </span>null<span style="color:rgb(192,192,192)"> </span>space</pre>
<pre style="margin-top:0px;margin-bottom:0px"><span style="color:rgb(192,192,192)"> </span>not<span style="color:rgb(192,192,192)"> </span>using<span style="color:rgb(192,192,192)"> </span>I-node<span style="color:rgb(192,192,192)"> </span>routines</pre>
<pre style="margin-top:0px;margin-bottom:0px"><br></pre></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Mar 6, 2014 at 3:43 PM, Jed Brown <span dir="ltr"><<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Mohammad Mirzadeh <<a href="mailto:mirzadeh@gmail.com" target="_blank">mirzadeh@gmail.com</a>> writes:<br>
<br>
> Yes. To be precise this is the set of functions I call:<br>
><br>
> ierr = MatNullSpaceCreate(mpicomm, PETSC_FALSE, 1, &null_space,<br>
> &A_null_space); CHKERRXX(ierr);<br>
><br>
> ierr = MatSetNullSpace(A, A_null_space); CHKERRXX(ierr);<br>
><br>
> ierr = KSPSetNullSpace(ksp, A_null_space); CHKERRXX(ierr);<br>
><br>
> ierr = MatNullSpaceRemove(A_null_space, rhs_, NULL); CHKERRXX(ierr);<br>
<br>
</div>Is the matrix symmetric? If not, the right and left null spaces could<br>
be different, in which case this system might be inconsistent.<br>
</blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener
</div></div>