[petsc-users] Solving a sequence of linear systems stored on disk with MUMPS

[Smith, Barry F.]

  What types of computing systems will you be doing the computations? Roughly how many MPI_ranks? 

Are the matrices all the same size? Do they have the same or different nonzero structures? Would it be possible to use the same symbolic representation for all of them and just have different numerical values?

  Clusters and large scale computing centers are notoriously terrible at IO; often IO is orders of magnitude slower than compute/memory making this type of workflow unrealistically slow. From a cost analysis point of view often just buying lots of memory might be the most  efficacious approach.

  That said, what you suggest might require only a few lines of code (though determining where to put them is the tricky part) depending on the MUMPS interface for saving a filer to disk. What we would do is keep the PETSc wrapper that lives around the MUMPS matrix Mat_MUMPS but using the MUMPS API save the information in the DMUMPS_STRUC_C id; and then reload it when needed.

  The user level API could be something like 

  MatMumpsSaveToDisk(Mat) and MatMumpsLoadFromDisk(Mat) they would just money with DMUMPS_STRUC_C id; item.


  Barry


> On Jul 23, 2019, at 9:24 AM, Thibaut Appel via petsc-users <petsc-users at mcs.anl.gov> wrote:
> 
> Dear PETSc users,
> 
> I need to solve several linear systems successively, with LU factorization, as part of an iterative process in my Fortran application code.
> 
> The process would solve M systems (A_m)(x_m,i) = (b_m,i) for m=1,M at each iteration i, but computing the LU factorization of A_m only once.
> The RHSs (b_m,i+1) are computed from all the different (x_m,i) and all depend upon each other.
> 
> The way I envisage to perform that is to use MUMPS to compute, successively, each of the LU factorizations (m) in parallel and store the factors on disk, creating/assembling/destroying the matrices A_m on the go.
> Then whenever needed, read the factors in parallel to solve the systems. Since version 5.2, MUMPS has a save/restore feature that allows that, see http://mumps.enseeiht.fr/doc/userguide_5.2.1.pdf p.20, 24 and 58.
> 
> In its current state, the PETSc/MUMPS interface does not incorporate that feature. I'm an advanced Fortran programmer but not in C so I don't think I would do an amazing job having a go inside src/mat/impls/aij/mpi/mumps/mumps.c.
> 
> I was picturing something like creating as many KSP objects as linear systems to be solved, with some sort of flag to force the storage of LU factors on disk after the first call to KSPSolve. Then keep calling KSPSolve as many times as needed.
> 
> Would you support such a feature?
> 
> Thanks for your support,
> 
> Thibaut