[petsc-dev] PETSc LU, Lapack and Preconditioning Matrices

[Matthew Knepley]

On Fri, Dec 16, 2011 at 5:59 PM, Dave Nystrom <Dave.Nystrom at tachyonlogic.com
> wrote:

> Barry Smith writes:
>  >
>  > On Dec 16, 2011, at 9:52 AM, Matthew Knepley wrote:
>  >
>  > > On Fri, Dec 16, 2011 at 9:37 AM, Dave Nystrom <dnystrom1 at comcast.net>
> wrote:
>  > > I'm trying to figure out whether I can do a couple of things with
> petsc.
>  > >
>  > > 1.  It looks like the preconditioning matrix can actually be
> different from
>  > > the full problem matrix.  So I'm wondering if I could provide a
> different
>  > > preconditioning matrix for my problem and then do an LU solve of the
>  > > preconditioning matrix using the -pc_type lu as my preconditioner.
>  > >
>  > > Yes, that is what it is for.
>  > >
>  > > 2.  When I build petsc, I use the --download-f-blas-lapack=yes
> option.  I'm
>  > > wondering if petsc uses lapack under the hood or has the capability
> to use
>  > > lapack under the hood when one uses the -pc_type lu option.  In
> particular,
>  > > since my matrices are band matrices from doing a discretization on a
> 2d
>  > > regular mesh, I'm wondering if the petsc lu solve has the ability to
> use the
>  > > lapack band solver dgbsv or dgbsvx.  Or is it possible to use the
> lapack band
>  > > solver through one of the external packages that petsc can interface
> with.
>  > > I'm interested in this capability for smaller problem sizes that fit
> on a
>  > > single node and that make sense.
>  > >
>  > > We do not have any banded matrix stuff. Its either dense or sparse
> right now.
>  >
>  > Dave,
>  >
>  > As I noted before, your band is so large that lapack type band solvers
>  > don't make sense. Using a general purpose sparse direct solver will be
>  > much more efficient than using the lapack band solver.
>
> Thanks.  I guess I don't really know what a "general purpose sparse direct
> solver" is.  When I do a direct solve with petsc of one of my linear
> systems
> using "-ksp_type preonly -pc_type lu", and letting petsc decide on the
> matrix
> storage type, is petsc using a "general purpose sparse direct solver"?  I
>

No, its dense.


> assume a lapack type band solver would fill in the the band width which in
> my
> case should be a bandwidth of about 2*nx+1.  But I guess I thought that was
> just the price of doing a direct solve on a banded matrix.  Does a "general
>

No one would use a band solver for a sqrt(n) band.


> purpose sparse direct solver" somehow use less fill in?  Could you point me
> to a good reference on general purpose sparse direct solvers?
>

Any of the torrent of papers about SuperLU or MUMPS. Googable.


> I have been finding that for one or two of my linear systems on a single
> node
> that using "-ksp_type preonly -pc_type lu" for small enough mesh sizes is
> the
> most efficient way to solve the system.  So I have been wondering if I
> could
> find a highly tuned and optimized package to do the solve, for instance
> Intel
> mkl, that might run well with multiple threads.  And Intel mkl does have a
> band solver that I assume is highly optimized.  But I was not aware that a
> general purpose sparse direct solver would be computationally more
> efficient
> than a highly tuned band solver.  But then again, I don't really understand
> what the difference is between the two.
>

No, Barry is correct here.


>
> Given that I am having good results with "-ksp_type preonly -pc_type lu"
> with
> petsc for a couple of my systems with small enough problem sizes, would it
> be
> beneficial for me to try one of the external packages such as superlu or
> mumps or some other package?
>

Yes.

   Matt


> Thanks,
>
> Dave
>
>  > Barry
>  >
>  > >
>  > > 3.  I'm also wondering how I might be able to learn more about the
> petsc ilu
>  > > capability.  My impression is that it does ilu(k) and I have tried it
> with
>  > > k>0 but am wondering if one of the options might allow it to do ilut
> and
>  > > whether as k gets big whether ilu(k) approximates lu.  I currently do
> not
>  > > understand the petsc ilu well enough to know how much extra fill I
> get as I
>  > > increase k and where that extra fill might be located for the case of
> a band
>  > > matrix that one gets from discretization on a regular 2d mesh.
>  > >
>  > > We do not do ilu(dt). Its complicated, and we determined that it was
> not worth
>  > > the effort. You can get that from Hypre is you want. Certainly, for
> big enough
>  > > k, ilu(k) is lu but its a slow way to do it.
>  > >
>  > >    Matt
>  > >
>  > > Thanks,
>  > >
>  > > Dave
>  > >
>  > >
>  > >
>  > > --
>  > > What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
>  > > -- Norbert Wiener
>  >
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20111216/ab336462/attachment.html>