[petsc-dev] PETSc LU, Lapack and Preconditioning Matrices

[Dave Nystrom]

Barry Smith writes:
 > 
 > On Dec 16, 2011, at 9:52 AM, Matthew Knepley wrote:
 > 
 > > On Fri, Dec 16, 2011 at 9:37 AM, Dave Nystrom <dnystrom1 at comcast.net> wrote:
 > > I'm trying to figure out whether I can do a couple of things with petsc.
 > > 
 > > 1.  It looks like the preconditioning matrix can actually be different from
 > > the full problem matrix.  So I'm wondering if I could provide a different
 > > preconditioning matrix for my problem and then do an LU solve of the
 > > preconditioning matrix using the -pc_type lu as my preconditioner.
 > > 
 > > Yes, that is what it is for.
 > >  
 > > 2.  When I build petsc, I use the --download-f-blas-lapack=yes option.  I'm
 > > wondering if petsc uses lapack under the hood or has the capability to use
 > > lapack under the hood when one uses the -pc_type lu option.  In particular,
 > > since my matrices are band matrices from doing a discretization on a 2d
 > > regular mesh, I'm wondering if the petsc lu solve has the ability to use the
 > > lapack band solver dgbsv or dgbsvx.  Or is it possible to use the lapack band
 > > solver through one of the external packages that petsc can interface with.
 > > I'm interested in this capability for smaller problem sizes that fit on a
 > > single node and that make sense.
 > > 
 > > We do not have any banded matrix stuff. Its either dense or sparse right now.
 > 
 > Dave,
 > 
 > As I noted before, your band is so large that lapack type band solvers
 > don't make sense. Using a general purpose sparse direct solver will be
 > much more efficient than using the lapack band solver.

Thanks.  I guess I don't really know what a "general purpose sparse direct
solver" is.  When I do a direct solve with petsc of one of my linear systems
using "-ksp_type preonly -pc_type lu", and letting petsc decide on the matrix
storage type, is petsc using a "general purpose sparse direct solver"?  I
assume a lapack type band solver would fill in the the band width which in my
case should be a bandwidth of about 2*nx+1.  But I guess I thought that was
just the price of doing a direct solve on a banded matrix.  Does a "general
purpose sparse direct solver" somehow use less fill in?  Could you point me
to a good reference on general purpose sparse direct solvers?

I have been finding that for one or two of my linear systems on a single node
that using "-ksp_type preonly -pc_type lu" for small enough mesh sizes is the
most efficient way to solve the system.  So I have been wondering if I could
find a highly tuned and optimized package to do the solve, for instance Intel
mkl, that might run well with multiple threads.  And Intel mkl does have a
band solver that I assume is highly optimized.  But I was not aware that a
general purpose sparse direct solver would be computationally more efficient
than a highly tuned band solver.  But then again, I don't really understand
what the difference is between the two.

Given that I am having good results with "-ksp_type preonly -pc_type lu" with
petsc for a couple of my systems with small enough problem sizes, would it be
beneficial for me to try one of the external packages such as superlu or
mumps or some other package?

Thanks,

Dave

 > Barry
 > 
 > >  
 > > 3.  I'm also wondering how I might be able to learn more about the petsc ilu
 > > capability.  My impression is that it does ilu(k) and I have tried it with
 > > k>0 but am wondering if one of the options might allow it to do ilut and
 > > whether as k gets big whether ilu(k) approximates lu.  I currently do not
 > > understand the petsc ilu well enough to know how much extra fill I get as I
 > > increase k and where that extra fill might be located for the case of a band
 > > matrix that one gets from discretization on a regular 2d mesh.
 > > 
 > > We do not do ilu(dt). Its complicated, and we determined that it was not worth
 > > the effort. You can get that from Hypre is you want. Certainly, for big enough
 > > k, ilu(k) is lu but its a slow way to do it.
 > > 
 > >    Matt
 > >  
 > > Thanks,
 > > 
 > > Dave
 > > 
 > > 
 > > 
 > > -- 
 > > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
 > > -- Norbert Wiener
 >