[petsc-users] Way to remove zero entries from an assembled matrix
Leidy Catherine Ramirez Villalba
leidy-catherine.ramirez-villalba at ec-nantes.fr
Wed Dec 7 02:49:09 CST 2016
Hi Barry,
Thanks for your reply!
I must say that I still do not master still the different solvers and options, so my problem might be due to a wrong formulation for the solver, or the final state of the matrix is not good, even if I verify that is assembled and the exit looks quite ok. Following the whole thing:
After assembling, manually and using petsc, I compare the matrices using a ASCII file (PETSC_VIEWER_ASCII_MATLAB option), and there i see no difference but the zero in some rows
1664 560 -1.6995706110114395e+06 | 1664 560 -1.6995706110114395e+06
1664 1660 -1.7462298274040222e-10 | 1664 1660 -1.7462298274040222e-10
1664 1661 6.6919512730949128e+05 | 1664 1661 6.6919512730949128e+05
1664 1663 0.0000000000000000e+00 | 1664 1664 5.4708211855417928e+06
1664 1664 5.4707211855417928e+06 | -------------------------------------------------------------------------------------------------------
1664 4600 -1.7462298274040222e-10 | 1664 4600 -1.7462298274040222e-10
then, I solve both systems using MUMPS solve, with same rhs vector and finally I compare the output vector Ur.
Below how I solve the system:
! ---------------------------------------SOLVING 1: manual matrix: mpiA ----------------------------------------------------------------
call KSPCreate(PETSC_COMM_WORLD,kspSolver,petscIerr)
call KSPSetOperators(kspSolver,mpiA,mpiA,petscIerr)
call KSPSetType(kspSolver, KSPPREONLY, petscIerr)
call KSPSetFromOptions(kspSolver,petscIerr)
call KSPGetPC(kspSolver,precond,petscIerr)
call PCSetType(precond,PCLU,petscIerr)
call PCFactorSetMatSolverPackage(precond,MATSOLVERMUMPS,petscIerr)
call PCFactorSetUpMatSolverPackage(precond,petscIerr)
call PCSetFromOptions(precond,petscIerr)
relativeTol=1.e-15
absoluteTol=1.e-15
divergenceTol=1.e4
maxIter=200
call KSPSetTolerances(kspSolver,relativeTol,absoluteTol,divergenceTol,maxIter,petscIerr)
call KSPSolve(kspSolver,mpiVecRhs,mpiVecLocSolution,petscIerr)
call KSPGetSolution(kspSolver,mpiVecLocSolution,petscIerr)
call VecScatterCreateToAll(mpiVecLocSolution,vecScatterEnv,mpiVecGlobSolution,petscIerr)
call VecScatterBegin(vecScatterEnv,mpiVecLocSolution,mpiVecGlobSolution,INSERT_VALUES,SCATTER_FORWARD,petscIerr)
call VecScatterEnd(vecScatterEnv,mpiVecLocSolution,mpiVecGlobSolution,INSERT_VALUES,SCATTER_FORWARD,petscIerr)
call VecGetArrayF90(mpiVecGlobSolution,pVecSolution,petscIerr)
Ur(1:ddl) = pVecSolution(:)
call VecRestoreArrayF90(mpiVecLocSolution,pVecSolution,petscIerr)
! -------------------------------------------------------------------------------------------------------
For the assembled matrix I have tried the same sequence, but:
1. changing the system and preconditioner specified matrix for the solver:
call KSPSetOperators(kspSolver,mpiMassMatrix,mpiMassMatrix,petscIerr)
There, I'm getting infinite for the solution for the petsc assembled matrix and the proper solution for the manual assembled one.
%Vec Object:mpiVecLocSolution 1 MPI processes | %Vec Object:mpiVecLocSolution 1 MPI processes
% type: seq | % type: seq
mpiVecLocSolution = [ | mpiVecLocSolution = [
inf | -2.2389524508816294e-20
inf | -1.5265035169220699e-20
inf | 0.0000000000000000e+00
2. I thought it was due to the extra zeros, then I tried no passing the same matrix for the preconditioner argument.
call KSPSetOperators(kspSolver,mpiMassMatrix,PETSC_NULL_OBJECT,petscIerr)
There I have the errors listed below and a solution vector full of zeros for the petsc assembled matrix.
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: You can only call this routine after the matrix object has been provided to the solver, for example with KSPSetOperators() or SNESSetJacobian()
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.4, unknown
[0]PETSC ERROR: ./aravanti on a arch-linux2-c-debug named localhost.localdomain by catherine Wed Dec 7 09:36:39 2016
[0]PETSC ERROR: Configure options --download-mumps --with-mpi-dir=/home/catherine/local/intel2016/ --download-scalapack --download-parmetis --download-metis --download-fblaslapack
[0]PETSC ERROR: #1 PCFactorSetUpMatSolverPackage_Factor() line 15 in /home/catherine/petsc/src/ksp/pc/impls/factor/factimpl.c
[0]PETSC ERROR: #2 PCFactorSetUpMatSolverPackage() line 26 in /home/catherine/petsc/src/ksp/pc/impls/factor/factor.c
solving the system
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: Not for unassembled matrix
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.4, unknown
[0]PETSC ERROR: ./aravanti on a arch-linux2-c-debug named localhost.localdomain by catherine Wed Dec 7 09:36:39 2016
[0]PETSC ERROR: Configure options --download-mumps --with-mpi-dir=/home/catherine/local/intel2016/ --download-scalapack --download-parmetis --download-metis --download-fblaslapack
[0]PETSC ERROR: #3 MatGetOrdering() line 189 in /home/catherine/petsc/src/mat/order/sorder.c
[0]PETSC ERROR: #4 PCSetUp_LU() line 125 in /home/catherine/petsc/src/ksp/pc/impls/factor/lu/lu.c
[0]PETSC ERROR: #5 PCSetUp() line 968 in /home/catherine/petsc/src/ksp/pc/interface/precon.c
[0]PETSC ERROR: #6 KSPSetUp() line 390 in /home/catherine/petsc/src/ksp/ksp/interface/itfunc.c
[0]PETSC ERROR: #7 KSPSolve() line 599 in /home/catherine/petsc/src/ksp/ksp/interface/itfunc.c
3. Or simply the default option (direct solver LU if i'm not mistaken)
call KSPCreate(PETSC_COMM_WORLD,kspSolver,petscIerr)
call KSPSetOperators(kspSolver,mpiMassMatrix,mpiMassMatrix,petscIerr)
call KSPSolve(kspSolver,mpiVecRhs,mpiVecLocSolution,petscIerr)
and then again I have an error and an exit of zeros, and a closer answer for the matrix assembled manually.
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: Matrix is missing diagonal entry 5955
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.4, unknown
[0]PETSC ERROR: ./aravanti on a arch-linux2-c-debug named localhost.localdomain by catherine Wed Dec 7 09:26:29 2016
[0]PETSC ERROR: Configure options --download-mumps --with-mpi-dir=/home/catherine/local/intel2016/ --download-scalapack --download-parmetis --download-metis --download-fblaslapack
[0]PETSC ERROR: #1 MatILUFactorSymbolic_SeqAIJ() line 1733 in /home/catherine/petsc/src/mat/impls/aij/seq/aijfact.c
[0]PETSC ERROR: #2 MatILUFactorSymbolic() line 6579 in /home/catherine/petsc/src/mat/interface/matrix.c
[0]PETSC ERROR: #3 PCSetUp_ILU() line 213 in /home/catherine/petsc/src/ksp/pc/impls/factor/ilu/ilu.c
[0]PETSC ERROR: #4 PCSetUp() line 968 in /home/catherine/petsc/src/ksp/pc/interface/precon.c
[0]PETSC ERROR: #5 KSPSetUp() line 390 in /home/catherine/petsc/src/ksp/ksp/interface/itfunc.c
[0]PETSC ERROR: #6 KSPSolve() line 599 in /home/catherine/petsc/src/ksp/ksp/interface/itfunc.c
I hope information is clear enough.
Thanks again,
Catherine
----- Mail original -----
De: "Barry Smith" <bsmith at mcs.anl.gov>
À: "Leidy Catherine Ramirez Villalba" <leidy-catherine.ramirez-villalba at ec-nantes.fr>
Cc: petsc-users at mcs.anl.gov
Envoyé: Mardi 6 Décembre 2016 18:45:52
Objet: Re: [petsc-users] Way to remove zero entries from an assembled matrix
> On Dec 6, 2016, at 11:25 AM, Leidy Catherine Ramirez Villalba <leidy-catherine.ramirez-villalba at ec-nantes.fr> wrote:
>
> Hello PETSc team:
>
> I'm doing the parallelization of the assembling of a system, previously assembled in a serial way (manual), but solved using PETSc in parallel.
> Therefore I have the old assembled matrix to compare with the one assembled with PETSc.
>
> While doing the assembling of the matrix, I avoid the zero entries using the option 'MAT_IGNORE_ZERO_ENTRIES' in MatSetOption, however the final matrix has zero values (or almost) due to the addition of non zero elements.
> Below the example of added values:
>
> 1664 i 1663 j -165509.423650377
> 1664 i 1663 j 165509.423650377
> 1664 i 1663 j -165509.423650377
> 1664 i 1663 j 165509.423650377
>
> Due to this difference between the two matrices I'm not able to get the solution for the matrix assembled with petsc.
Catherine,
The "extra" zeros should not prevent getting the solution to the system. The computed solution may be a little different but if you use an iterative method with smaller and smaller tolerances the solutions should converge to the same value. With direct solvers the solutions should be very similar.
Could you please provide more details about not getting the solution for the parallel system? Send output of -ksp_monitor_true_residual for example
Barry
>
> Therefore I wonder if there is a way to remove zero entries from an assembled matrix, or which solver to use so that zeros will not be a problem.
>
> Thanks in advance,
> Catherine
>
>
>
>
More information about the petsc-users
mailing list