Parallel direct solvers with PFLOTRAN

[Richard Tran Mills]

Peter and Glenn,

You had asked about using parallel direct solvers with PFLOTRAN.  If you 
install a package such as mumps or superlu_dist with the corresponding PETSc 
interfaces, you can easily try using a direct solver using command line 
options such as the following:

-flow_mat_type mpiaij -flow_ksp_type preonly -flow_pc_type lu 
-flow_pc_factor_mat_solver_package mumps

You'll need to pull and update to include the push I made tonight that allows 
you to specify the matrix type on the command line.  (Things default to BAIJ, 
which isn't supported for the direct solvers.)

The above looks pretty messy but we can worry about making a cleaner way to do 
this if this turns out to be something that you might want to do on a routine 
basis.

I have tested PFLOTRAN with both MUMPS and SuperLU_dist on a Linux workstation 
using the very simple 'TAO/100_10_10' problem in the examples directory.  This 
works fine.  I was having some problems getting things to work properly on 
Jaguar (no surprise) and wasn't quite able to get things resolved before the 
machine went down for most of today.  I will try again tomorrow.

In your own experiments, I would start with trying out MUMPS, as anecdotal 
evidence suggests that it is fairly robust.

--Richard