[petsc-users] Profile a matrix-free solver.

[Song Gao]

Hello, Barry,

Thanks for your prompt reply.  I ran the matrix-based solver with
matrix-based SGS precondioner. I see your point. The profiling table is
below and attached.

So Matmult takes 4% time and PCApply takes 43% time.

 MatMult              636 1.0 9.0361e+00 1.0 9.21e+09 1.0 7.6e+03 1.1e+04
0.0e+00  4 85 52 17  0   4 85 52 17  0  3980
 PCApply              636 1.0 8.7006e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00
1.9e+03 43  0  0  0 24  43  0  0  0 24     0


The way I see it, the matrix-free solver spends most of the time (70%) on
matmult or equivalently rhs evaluation. Every KSP iteration, one rhs
evaluation is performed. This is much more costly than a matrix vector
product in a matrix-based solver. Perhaps this is expected in matrix-free
solver.

I will start look at the rhs evaluation since it takes the most time.

Thanks.
Song Gao



2016-01-14 16:24 GMT-05:00 Barry Smith <bsmith at mcs.anl.gov>:

>
>    So
>
>     KSPSolve is 96 % and MatMult is 70 % + PCApply 24 % = 94 % so this
> makes sense; the solver time is essentially the
> multiply time plus the PCApply time.
>
> compute_rhs         1823 1.0 4.2119e+02 1.0 0.00e+00 0.0 4.4e+04 5.4e+04
> 1.1e+04 71  0100100 39  71  0100100 39     0
> LU-SGS              1647 1.0 1.3590e+02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 23  0  0  0  0  23  0  0  0  0     0
> SURFINT             1823 1.0 1.0647e+02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 17  0  0  0  0  17  0  0  0  0     0
> VOLINT              1823 1.0 2.2373e+02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 35  0  0  0  0  35  0  0  0  0     0
>
>    Depending on the "quality" of the preconditioner (if it is really good)
> one expects the preconditioner time to be larger than the MatMult(). Only
> for simple preconditioners (like Jacobi) does one see it being much less
> than the MatMult().  For matrix based solvers the amount of work in  SGS is
> as large as the amount of work in the MatMult() if not more, so I would
> expect the time of the preconditioner to be higher than the time of the
> multiply.
>
>   So based on knowing almost nothing I think the MatMult_ is taking more
> time then it should unless you are ignoring (skipping) a lot of the terms
> in your matrix-free SGS; then it is probably reasonable.
>
>   Barry
>
>
>
> > On Jan 14, 2016, at 3:01 PM, Song Gao <song.gao2 at mail.mcgill.ca> wrote:
> >
> > Hello,
> >
> > I am profiling a finite element Navier-Stokes solver. It uses the
> Jacobian-free Newton Krylov method and a custom preconditoner LU-SGS (a
> matrix-free version of Symmetic Gauss-Seidel ). The log summary is
> attached. Four events are registered.  compute_rhs is compute rhs (used by
> MatMult_MFFD). SURFINT and VOLINT are parts of compute_rhs. LU-SGS is the
> custom preconditioner. I didn't call PetscLogFlops so these flops are zeros.
> >
> >  I'm wondering, is the percent time of the events reasonable in the
> table? I see 69% time is spent on  matmult_mffd. Is it expected in
> matrix-free method? What might be a good starting point of profiling this
> solver? Thank you in advance.
> >
> >
> > Song Gao
> > <log_summary>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160115/35c76544/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: log_summary_matrix_based_version
Type: application/octet-stream
Size: 9473 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160115/35c76544/attachment.obj>