[petsc-users] Code performance for solving multiple RHS

[Matthew Knepley]

On Wed, Aug 10, 2016 at 9:54 PM, Harshad Ranadive <harshadranadive at gmail.com
> wrote:

> Hi All,
>
> I have currently added the PETSc library with our CFD solver.
>
> In this I need to use KSPSolve(...) multiple time for the same matrix A. I
> have read that PETSc does not support passing multiple RHS vectors in the
> form of a matrix and the only solution to this is calling KSPSolve multiple
> times as in example given here:
> http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/
> examples/tutorials/ex16.c.html
>
> I have followed this technique, but I find that the performance of the
> code is very slow now. I basically have a mesh size of 8-10 Million and I
> need to solve the matrix A very large number of times. I have checked that
> the statement KSPSolve(..) is taking close to 90% of my computation time.
>
> I am setting up the matrix A, KSPCreate, KSPSetup etc just once at the
> start. Only the following statements are executed in a repeated loop
>
> *Loop begin: (say million times !!)*
>
> *   loop over vector length*
> *       VecSetValues( ....)*
> *   end*
>
> *   VecAssemblyBegin( ... )*
> *   VecAssemblyEnd (...)*
>
> *    KSPSolve (...)*
>
> *    VecGetValues*
>
> *Loop end.*
>
> Is there an efficient way of doing this rather than using KSPSolve
> multiple times?
>
> Please note my matrix A never changes during the time steps or across the
> mesh ... So essentially if I can get the inverse once would it be good
> enough?  It has been recommended in the FAQ that matrix inverse should be
> avoided but would it be okay to use in my case?
>
> Also could someone please provide an example of how to use MatLUFactor
> and MatCholeskyFactor() to find the matrix inverse... the arguments below
> were not clear to me.
> *IS row *
> *IS col*
> *const MatFactorInfo *info*
>
> Apologies for a long email and thanks to anyone for help.
>

1) For any questions, we NEED to see the output of -log_view for all cases
you are asking about

2) You could try factoring your matrix

  -pc_type lu -pc_factor_mat_solver_package <mumps,superlu_dist>

and it would be reused across solves, but for large matrices you can run
out of memory.

3) You can factor subdomains with BJacobi or ASM and they would be reused,
but this might be a crap preconditioner.

4) For any questions about convergence, we need to see the output of

  -ksp_view -ksp_monitor_true_residual -ksp_converged_reason

   Matt


> Regards
> Harshad
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160810/6d091405/attachment.html>