[petsc-users] How to investigate the reason for slow convergence rate?

Sun Jul 22 09:42:11 CDT 2012

On Sun, Jul 22, 2012 at 9:22 AM, Bao Kai <paeanball at gmail.com> wrote:

> Hi, Jed,
>
> The following is the output.  Two equations are solved during each
> time steps. One is using bicgstab is already very fast, the other one
> is using gmres+ml.
>

These equations are different? Why do you think they should take a similar
amount of time?

Most of your execution time is spent in the 31 PCSetUp calls (not all of
which are ML). If only a few iterations are needed, ML setup is not worth
it. Also, if the system is not changing much, you could lag the
preconditioner so that you pay for setup less frequently. This can be done
at the time integration level (e.g. using a Rosenbrock(-W) method) or at
the nonlinear solve by a variety of schemes including modified Newton, MFFD
Jacobian, Quasi-Newton, or NGMRES.

Note that you can use PetscLogStagePush/Pop to profile each solve
separately.

>
> The log_summary output can found in the end of the output.
>
> Best Regards,
> Kai
>
> > Message: 6
> > Date: Sun, 22 Jul 2012 07:16:07 -0500
> > From: Jed Brown <jedbrown at mcs.anl.gov>
> > To: PETSc users list <petsc-users at mcs.anl.gov>
> > Subject: Re: [petsc-users] How to investigate the reason for slow
> >       convergence rate?
> > Message-ID:
> >       <
> CAM9tzSkxKz5FwFJJPZz0UO5eN-S45QsdQxmo7xnH345R1EdTEQ at mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Please send -log_summary output with performance questions.
> >
> > On Sun, Jul 22, 2012 at 4:01 AM, Bao Kai <paeanball at gmail.com> wrote:
> >
> >> Hi, Matt,
> >>
> >> I tried ML6.2 with petsc3.3 with default parameters by only specifying
> >> the preconditioner to be PCML.  The KSP solver is gmres.
> >>
> >> The convergence rate is much faster, while it still took much longer
> >> time in total.
> >>
> >> For example, for the problem with 500^3 mesh ( 125 million unknowns )
> >> with 512nodes(4 processors per node) on bluegene/P , it tooks about
> >> ten iterations to get convergent while the total time used is about
> >> 400 seconds.
> >>
> >>     506   the KSP type is gmres
> >>     507   the PC type is ml
> >>     508  KSP rtol =  0.100000000000000008E-04  abstol =
> >> 0.100000000000000001E-49  dtol =  10000.0000000000000  maxit =  10000
> >>     509  SNES rtol =  0.100000000000000002E-07  abstol =
> >> 0.100000000000000001E-49  stol =  0.100000000000000002E-07  maxit =
> >> 50  maxf=  10000
> >>     510   0 SNES Function norm 5.859593121800e+02
> >>     511     0 KSP Residual norm 8.340827070202e+06
> >>     512     1 KSP Residual norm 7.980806572332e+05
> >>     513     2 KSP Residual norm 1.870896731234e+05
> >>     514     3 KSP Residual norm 6.790580947452e+04
> >>     515     4 KSP Residual norm 2.665552335248e+04
> >>     516     5 KSP Residual norm 1.130212349885e+04
> >>     517     6 KSP Residual norm 4.053599972292e+03
> >>     518     7 KSP Residual norm 1.786770710693e+03
> >>     519     8 KSP Residual norm 7.313571654931e+02
> >>     520     9 KSP Residual norm 3.205683714450e+02
> >>     521    10 KSP Residual norm 1.263243312734e+02
> >>     522    11 KSP Residual norm 3.945082815178e+01
> >>     523   1 SNES Function norm 9.378772067642e-02
> >>     524     0 KSP Residual norm 5.413489711800e+01
> >>     525     1 KSP Residual norm 1.442598710609e+01
> >>     526     2 KSP Residual norm 4.073537172140e+00
> >>     527     3 KSP Residual norm 1.157455598705e+00
> >>     528     4 KSP Residual norm 3.509855901968e-01
> >>     529     5 KSP Residual norm 1.160625342728e-01
> >>     530     6 KSP Residual norm 3.209351890216e-02
> >>     531     7 KSP Residual norm 7.780869881329e-03
> >>     532     8 KSP Residual norm 1.820828886636e-03
> >>     533     9 KSP Residual norm 4.172544590190e-04
> >>     534   2 SNES Function norm 6.747963806680e-07
> >>     535  Number of KSP iteration is  9
> >>     536  SNES solve takes time  406.724867261176314
> >>
> >> But with bcgs and bjacobi, it tooks about 550 KSP iterations ( 2 snes
> >> iterations ) and 69 seconds to get the result.
> >>
> >> For much smaller problems, benefiting from the fast convergence, it
> >> did takes less time to get the result. It seems that the ml can not be
> >> scaled, or I used it in a wrong way.
> >>
> >> Best Regards,
> >> Kai
> >>
> >>
> >> > Message: 2
> >> > Date: Sat, 21 Jul 2012 08:47:10 -0500
> >> > From: Matthew Knepley <knepley at gmail.com>
> >> > To: PETSc users list <petsc-users at mcs.anl.gov>
> >> > Subject: Re: [petsc-users] How to investigate the reason for slow
> >> >       convergence rate?
> >> > Message-ID:
> >> >       <
> >> CAMYG4GkfV6kmTFEKXUadOv+2CrKHk9hRY7UR-cFWf+vcRxCv5g at mail.gmail.com>
> >> > Content-Type: text/plain; charset="iso-8859-1"
> >> >
> >> > On Sat, Jul 21, 2012 at 4:30 AM, Bao Kai <paeanball at gmail.com> wrote:
> >> >
> >> >>
> >> >>> HI, all,
> >> >>
> >> >> I am still suffering from the slow convergence rate of the KSP
> >> >> solution.
> >> >>
> >> >> I changed the code to use Petsc3.3 and then try the gamg
> >> >> precoditioner,
> >> >> the convergence rate is better, while it took more total time because
> >> >> it
> >> >> took much more time for each iteration and some extra time for
> >> >> pre-processing.
> >> >>
> >> >
> >> > Try ML to see if it has better defaults for your problem. If not, you
> >> will
> >> > have to start experimenting with the solver
> >> > parameters.
> >> >
> >> >
> >> >> I am wondering if there are some ways that can help me to investigate
> >> the
> >> >> slow convergence rate for KSP solution so that I can do some
> >> improvement.
> >> >>  Is DMMG will be a good solution?
> >> >>
> >> >
> >> > No.
> >> >
> >> >     Matt
> >> >
> >> >
> >> >> Thank you very much.
> >> >>
> >> >> Best Regards,
> >> >> Kai
> >> >>
> >> >>>
> >> >>> Message: 2
> >> >>> Date: Wed, 11 Jul 2012 15:17:15 -0500
> >> >>> From: Matthew Knepley <knepley at gmail.com>
> >> >>> To: PETSc users list <petsc-users at mcs.anl.gov>
> >> >>> Subject: Re: [petsc-users] Does this mean the matrix is
> >> >>>         ill-conditioned?
> >> >>> Message-ID:
> >> >>>         <CAMYG4Gk7T=
> >> >>> q+w1PKO7G_TW07iDzux90Sncbv9K7d0FD-MDrLRg at mail.gmail.com>
> >> >>> Content-Type: text/plain; charset="iso-8859-1"
> >> >>>
> >> >>> On Wed, Jul 11, 2012 at 12:40 PM, Bao Kai <paeanball at gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>> > Hi, all,
> >> >>> >
> >> >>> > The following is the output from the solution of a Poisson
> equation
> >> >>> > from Darcy's law.
> >> >>> >
> >> >>> > To compute the condition number of matrix, I did not use PC and
> use
> >> >>> > GMRES KSP to do the test.
> >> >>> >
> >> >>> > It seems like that the condition number keep increasing during the
> >> >>> > iterative solution. Does this mean the matrix is ill-conditioned?
> >> >>> >
> >> >>>
> >> >>> Generally yes. Krylov methods take a long time to resolve the
> >> >>> smallest
> >> >>> eigenvalues, so this approximation is not great.
> >> >>>
> >> >>>
> >> >>> > For this test, it did not achieve convergence with 10000
> >> >>> > iterations.
> >> >>> >
> >> >>> > When I use BJOCABI PC and BICGSTAB KSP, it generally takes about
> >> >>> > 600
> >> >>> > times iteration to get the iteration convergent.
> >> >>> >
> >> >>> > Any suggestion for improving the convergence rate will be much
> >> >>> > appreciated.  The solution of this equation has been the
> bottleneck
> >> of
> >> >>> > my code, it takes more than 90% of the total time.
> >> >>> >
> >> >>>
> >> >>> Try ML or GAMG.
> >> >>>
> >> >>>     Matt
> >> >>>
> >> >>>
> >> >>> > Thank you very much.
> >> >>> >
> >> >>> > Best Regards,
> >> >>> > Kai
> >> >>> >
> >> >>>
> >> >>>
> >> >>>
> >> >
> >> >
> >> > --
> >> > What most experimenters take for granted before they begin their
> >> > experiments is infinitely more interesting than any results to which
> >> their
> >> > experiments lead.
> >> > -- Norbert Wiener
> >> > -------------- next part --------------
> >> > An HTML attachment was scrubbed...
> >> > URL:
> >> > <
> >>
> http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120721/eaf7b2ee/attachment-0001.html
> >> >
> >> >
> >> > ------------------------------
> >> >
> >>
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> > <
> http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120722/7f782dd6/attachment.html
> >
> >
> > ------------------------------
> >
> > _______________________________________________
> > petsc-users mailing list
> > petsc-users at mcs.anl.gov
> > https://lists.mcs.anl.gov/mailman/listinfo/petsc-users
> >
> >
> > End of petsc-users Digest, Vol 43, Issue 61
> > *******************************************
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120722/26934ef4/attachment.html>