[petsc-users] SLEPc eigensolver that uses minimal memory and finds ALL eigenvalues of a real symmetric sparse matrix in reasonable time

[Barry Smith]

On Aug 8, 2011, at 2:14 AM, Shitij Bhargava wrote:

> Thank you Jed. That was indeed the problem. I installed a separate MPI for PETSc/SLEPc, but was running my program with a default, already installed one.
> 
> Now, I have a different question. What I want to do is this:
> 
> 1. Only 1 process, say root, calculates the matrix in SeqAIJ format
> 2. Then root creates the EPS context, eps and initializes,sets parameters, problem type,etc. properly
> 3. After this the root process broadcasts this eps object to other processes
> 4. I use EPSSolve to solve for eigenvalues (all process together in cooperation resulting in memory distribution)
> 5. I get the results from root

   We do have an undocumented routine MatDistribute_MPIAIJ(MPI_Comm comm,Mat gmat,PetscInt m,MatReuse reuse,Mat *inmat) in src/mat/impls/aij/mpi/mpiaij.c that will take a SeqAIJ matrix and distribute it over a larger MPI communicator. 

   Note that you cannot create the EPS context etc on a the root process and then broadcast the object but once the matrix is distributed you can simple create the EPS context etc on the parallel communicator where the matrix is and run with that. 

   Barry

> 
> is this possible ? I am not able to broadcast the EPS object, because it is not an MPI_DataType. Is there any PETSc/SLEPc function for this ? I am avoiding using MPIAIJ because that will mean making many changes in the existing code, including the numerous write(*,*) statements (i would have to convert them to PetscPrint in FORTRAN or something like that). 
> So I want a single process to handle matrix generation and assembly, but want to solve the eigenproblem in parallel by different processes. Running the subroutine EPSSolve in parallel and hence distribute memory is the only reason why I want to use MPI.
> 
> Thanks a lot !!
> 
> Shitij
> 
> On 8 August 2011 11:05, Jed Brown <jedbrown at mcs.anl.gov> wrote:
> On Mon, Aug 8, 2011 at 00:29, Shitij Bhargava <shitij.cse at gmail.com> wrote:
> I ran it with:
> 
> mpirun -np 2 ./slepcEigenMPI -eps_monitor
> 
> I didnt do exactly what you said, because the matrix generation part in the actual program is quite time consuming itself. But I assume what I am doing is equivalent to what you meant to do? Also, I put MPD as PETSC_DECIDE, because I didnt know what to put it for this matrix dimension.
> 
> This is the output I get: (part of the output)
> MATRIX ASSMEBLY DONE !!!!!!!! 
> 
> MATRIX ASSMEBLY DONE !!!!!!!! 
> 
>   1 EPS nconv=98 first unconverged value (error) 1490.88 (1.73958730e-05)
>   1 EPS nconv=98 first unconverged value (error) 1490.88 (1.73958730e-05)
>   2 EPS nconv=282 first unconverged value (error) 3.04636e-27 (2.49532175e-04)
>   2 EPS nconv=282 first unconverged value (error) 3.04636e-27 (2.49532175e-04)
> 
> The most likely case is that you have more than one MPI implementation installed and that you are running with a different implementation than you built with. Compare the outputs:
> 
> $ ldd ./slepcEigenMPI
> $ which mpirun
>