[petsc-users] About Preconditioner and MUMPS
Matthew Knepley
knepley at gmail.com
Wed Dec 7 07:38:36 CST 2022
On Wed, Dec 7, 2022 at 8:15 AM 김성익 <ksi2443 at gmail.com> wrote:
> Following your comments,
> I used below command
> mpirun -np 4 ./app -ksp_type preonly -pc_type mpi
> -mpi_linear_solver_server -mpi_pc_type lu -mpi_pc_factor_mat_solver_type
> mumps -mpi_mat_mumps_icntl_7 5 -mpi_ksp_view
>
>
> so the output is as below
>
> KSP Object: (mpi_) 1 MPI process
> type: gmres
> restart=30, using Classical (unmodified) Gram-Schmidt
> Orthogonalization with no iterative refinement
> happy breakdown tolerance 1e-30
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using PRECONDITIONED norm type for convergence test
> PC Object: (mpi_) 1 MPI process
> type: lu
> out-of-place factorization
> tolerance for zero pivot 2.22045e-14
> matrix ordering: external
> factor fill ratio given 0., needed 0.
> Factored matrix follows:
> Mat Object: (mpi_) 1 MPI process
> type: mumps
> rows=192, cols=192
> package used to perform factorization: mumps
> total: nonzeros=17334, allocated nonzeros=17334
> MUMPS run parameters:
> Use -mpi_ksp_view ::ascii_info_detail to display information
> for all processes
> RINFOG(1) (global estimated flops for the elimination after
> analysis): 949441.
> RINFOG(2) (global estimated flops for the assembly after
> factorization): 18774.
> RINFOG(3) (global estimated flops for the elimination after
> factorization): 949441.
> (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant):
> (0.,0.)*(2^0)
> INFOG(3) (estimated real workspace for factors on all
> processors after analysis): 17334
> INFOG(4) (estimated integer workspace for factors on all
> processors after analysis): 1724
> INFOG(5) (estimated maximum front size in the complete
> tree): 96
> INFOG(6) (number of nodes in the complete tree): 16
> INFOG(7) (ordering option effectively used after analysis): 5
> INFOG(8) (structural symmetry in percent of the permuted
> matrix after analysis): 100
> INFOG(9) (total real/complex workspace to store the matrix
> factors after factorization): 17334
> INFOG(10) (total integer space store the matrix factors
> after factorization): 1724
> INFOG(11) (order of largest frontal matrix after
> factorization): 96
> INFOG(12) (number of off-diagonal pivots): 0
> INFOG(13) (number of delayed pivots after factorization): 0
> INFOG(14) (number of memory compress after factorization): 0
> INFOG(15) (number of steps of iterative refinement after
> solution): 0
> INFOG(16) (estimated size (in MB) of all MUMPS internal data
> for factorization after analysis: value on the most memory consuming
> processor): 1
> INFOG(17) (estimated size of all MUMPS internal data for
> factorization after analysis: sum over all processors): 1
> INFOG(18) (size of all MUMPS internal data allocated during
> factorization: value on the most memory consuming processor): 1
> INFOG(19) (size of all MUMPS internal data allocated during
> factorization: sum over all processors): 1
> INFOG(20) (estimated number of entries in the factors): 17334
> INFOG(21) (size in MB of memory effectively used during
> factorization - value on the most memory consuming processor): 1
> INFOG(22) (size in MB of memory effectively used during
> factorization - sum over all processors): 1
> INFOG(23) (after analysis: value of ICNTL(6) effectively
> used): 0
> INFOG(24) (after analysis: value of ICNTL(12) effectively
> used): 1
> INFOG(25) (after factorization: number of pivots modified by
> static pivoting): 0
> INFOG(28) (after factorization: number of null pivots
> encountered): 0
> INFOG(29) (after factorization: effective number of entries
> in the factors (sum over all processors)): 17334
> INFOG(30, 31) (after solution: size in Mbytes of memory used
> during solution phase): 0, 0
> INFOG(32) (after analysis: type of analysis done): 1
> INFOG(33) (value used for ICNTL(8)): 7
> INFOG(34) (exponent of the determinant if determinant is
> requested): 0
> INFOG(35) (after factorization: number of entries taking
> into account BLR factor compression - sum over all processors): 17334
> INFOG(36) (after analysis: estimated size of all MUMPS
> internal data for running BLR in-core - value on the most memory consuming
> processor): 0
> INFOG(37) (after analysis: estimated size of all MUMPS
> internal data for running BLR in-core - sum over all processors): 0
> INFOG(38) (after analysis: estimated size of all MUMPS
> internal data for running BLR out-of-core - value on the most memory
> consuming processor): 0
> INFOG(39) (after analysis: estimated size of all MUMPS
> internal data for running BLR out-of-core - sum over all processors): 0
> linear system matrix = precond matrix:
> Mat Object: 1 MPI process
> type: seqaij
> rows=192, cols=192
> total: nonzeros=9000, allocated nonzeros=36864
> total number of mallocs used during MatSetValues calls=0
> using I-node routines: found 64 nodes, limit used is 5
>
> Is it correct that I successfully computed with mumps by using
> metis(icntl_7 5)?
>
Yes, first you know it is MUMPS
Mat Object: (mpi_) 1 MPI process
type: mumps
Second you know METIS was used
INFOG(7) (ordering option effectively used after analysis): 5
I would not expect much speedup on 2 processes with such a small matrix
rows=192, cols=192
Thanks
Matt
> Thanks,
> Hyung Kim
>
> 2022년 12월 7일 (수) 오후 8:41, Matthew Knepley <knepley at gmail.com>님이 작성:
>
>> On Wed, Dec 7, 2022 at 6:15 AM 김성익 <ksi2443 at gmail.com> wrote:
>>
>>> I think I don't understand the meaning of
>>> -pc_type mpi
>>>
>>
>> This option says to use the PCMPI preconditioner. This allows you to
>> parallelize the
>> solver in what is otherwise a serial code.
>>
>>
>>> -mpi_pc_type lu
>>>
>>
>> This tells the underlying solver in PCMPI to use the LU preconditioner.
>>
>>
>>> What's the exact meaning of -pc_type mpi and -mpi_pc_type lu??
>>> Is this difference coming from 'mpi_linear_solver_server' option??
>>>
>>
>> Please use -ksp_view as I asked to look at the entire solver. Send it
>> anytime you mail about solver questions.
>>
>> Thanks
>>
>> Matt
>>
>>
>>> Thanks,
>>> Hyung Kim
>>>
>>> 2022년 12월 7일 (수) 오후 8:05, Matthew Knepley <knepley at gmail.com>님이 작성:
>>>
>>>> On Wed, Dec 7, 2022 at 5:13 AM 김성익 <ksi2443 at gmail.com> wrote:
>>>>
>>>>> I want to use METIS for ordering.
>>>>> I heard the MUMPS has good performance with METIS ordering.
>>>>>
>>>>> However there are some wonder things.
>>>>> 1. With option "-mpi_linear_solver_server -ksp_type preonly -pc_type
>>>>> mpi -mpi_pc_type lu " the MUMPS solving is slower than with option
>>>>> "-mpi_linear_solver_server -pc_type mpi -ksp_type preonly".
>>>>> Why does this result happen?
>>>>>
>>>>
>>>> You are probably not using MUMPS. Always always always use -ksp_view to
>>>> see exactly what solver you are using.
>>>>
>>>>
>>>>> 2. (MPIRUN case (actually, mpi_linear_solver_server case))) In my
>>>>> code, there is already has "PetscCall(PCSetType(pc,PCLU))" . However, to
>>>>> use METIS by using "-mpi_mat_mumps_icntl_7 5" I must append this option
>>>>> "-mpi_pc_type pu".
>>>>> If I don't apply "-mpi_pc_type lu", the metis option
>>>>> ("-mpi_mat_mumps_icntl_7 5"). Can I get some information about this?
>>>>>
>>>>
>>>> Again, it seems like the solver configuration is not what you think it
>>>> is.
>>>>
>>>> Thanks,
>>>>
>>>> Matt
>>>>
>>>>
>>>>> Thanks,
>>>>> Hyung Kim
>>>>>
>>>>> 2022년 12월 7일 (수) 오전 12:24, Barry Smith <bsmith at petsc.dev>님이 작성:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Dec 6, 2022, at 5:15 AM, 김성익 <ksi2443 at gmail.com> wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>>
>>>>>> I have some questions about pc and mumps_icntl.
>>>>>>
>>>>>> 1. What’s the difference between adopt preconditioner by code
>>>>>> (for example, PetscCall(PCSetType(pc,PCLU)) and option -pc_type lu??
>>>>>> And also, What’s the priority between code pcsettype and option
>>>>>> -pc_type ??
>>>>>>
>>>>>> 2. When I tried to use METIS in MUMPS, I adopted metis by option
>>>>>> (for example, -mat_mumps_icntl_7 5). In this situation, it is impossible to
>>>>>> use metis without pc_type lu. However, in my case pc type lu makes the
>>>>>> performance poor. So I don’t want to use lu preconditioner. How can I do
>>>>>> this?
>>>>>>
>>>>>> The package MUMPS has an option to use metis in its ordering
>>>>>> process which can be turned on as indicated while using MUMPS. Most
>>>>>> preconditioners that PETSc can use do not use metis for any purpose hence
>>>>>> there is no option to turn on its use. For what purpose do you wish to use
>>>>>> metis? Partitioning, ordering, ?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Hyung Kim
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>>
>>>> https://www.cse.buffalo.edu/~knepley/
>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>
>>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> <http://www.cse.buffalo.edu/~knepley/>
>>
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20221207/705d57e9/attachment-0001.html>
More information about the petsc-users
mailing list