[petsc-users] Code performance for solving multiple RHS

[Harshad Ranadive]

Hi Barry,

Thanks for this recommendation.

As you mention, the matrix factorization should be on a single processor.
If the factored matrix A is available on all processors can I then use
MatMatSolve(A,B,X) in parallel? That is could the RHS block matrix 'B' and
solution matrix 'X' be distributed in different processors as is done while
using MatCreateDense(...) ?

Thanks,
Harshad



On Fri, Aug 12, 2016 at 2:09 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
>    If it is sequential, which it probably should be, then you can you
> MatLUFactorSymbolic(), MatLUFactorNumeric() and MatMatSolve() where you put
> a bunch of your right hand side vectors into a dense array; not all million
> of them but maybe 10 to 100 at a time.
>
>    Barry
>
> > On Aug 10, 2016, at 10:18 PM, Harshad Ranadive <
> harshadranadive at gmail.com> wrote:
> >
> > Hi Barry
> >
> > The matrix A is mostly tridiagonal
> >
> > 1 α 0 ......... 0
> >
> > α 1 α 0 .......0
> >
> >
> > 0 α 1 α 0 ....0
> >
> >
> > ....................
> > 0..............α 1
> >
> > In some cases (periodic boundaries) there would be an 'α' in
> right-top-corner and left-bottom corner.
> >
> > I am not using multigrid approach. I just implemented an implicit
> filtering approach (instead of an explicit existing one) which requires the
> solution of the above system.
> >
> > Thanks
> > Harshad
> >
> > On Thu, Aug 11, 2016 at 1:07 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> >
> >   Effectively utilizing multiple right hand sides with the same system
> can result in roughly 2 or at absolute most  3 times  improvement in solve
> time. A great improvement but when you have a million right hand sides not
> a giant improvement.
> >
> >   The first step is to get the best (most efficient) preconditioner for
> you problem. Since you have many right hand sides it obviously pays to
> spend more time building the preconditioner so that each solve is faster.
> If you provide more information on your linear system we might have
> suggestions.  CFD so is your linear system a Poisson problem? Are you using
> geometric or algebraic multigrid with PETSc? It not a Poisson problem how
> can you describe the linear system?
> >
> >   Barry
> >
> >
> >
> > > On Aug 10, 2016, at 9:54 PM, Harshad Ranadive <
> harshadranadive at gmail.com> wrote:
> > >
> > > Hi All,
> > >
> > > I have currently added the PETSc library with our CFD solver.
> > >
> > > In this I need to use KSPSolve(...) multiple time for the same matrix
> A. I have read that PETSc does not support passing multiple RHS vectors in
> the form of a matrix and the only solution to this is calling KSPSolve
> multiple times as in example given here:
> > > http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/
> examples/tutorials/ex16.c.html
> > >
> > > I have followed this technique, but I find that the performance of the
> code is very slow now. I basically have a mesh size of 8-10 Million and I
> need to solve the matrix A very large number of times. I have checked that
> the statement KSPSolve(..) is taking close to 90% of my computation time.
> > >
> > > I am setting up the matrix A, KSPCreate, KSPSetup etc just once at the
> start. Only the following statements are executed in a repeated loop
> > >
> > > Loop begin: (say million times !!)
> > >
> > >    loop over vector length
> > >        VecSetValues( ....)
> > >    end
> > >
> > >    VecAssemblyBegin( ... )
> > >    VecAssemblyEnd (...)
> > >
> > >     KSPSolve (...)
> > >
> > >     VecGetValues
> > >
> > > Loop end.
> > >
> > > Is there an efficient way of doing this rather than using KSPSolve
> multiple times?
> > >
> > > Please note my matrix A never changes during the time steps or across
> the mesh ... So essentially if I can get the inverse once would it be good
> enough?  It has been recommended in the FAQ that matrix inverse should be
> avoided but would it be okay to use in my case?
> > >
> > > Also could someone please provide an example of how to use MatLUFactor
> and MatCholeskyFactor() to find the matrix inverse... the arguments below
> were not clear to me.
> > > IS row
> > > IS col
> > > const MatFactorInfo *info
> > >
> > > Apologies for a long email and thanks to anyone for help.
> > >
> > > Regards
> > > Harshad
> > >
> > >
> > >
> > >
> > >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160812/2173bd22/attachment.html>