[petsc-users] Question on incomplete factorization level and fill
Danyang Su
danyang.su at gmail.com
Wed May 24 14:28:51 CDT 2017
Hi Hong,
Awesome. Thanks for testing the case. I will try your options for the
code and get back to you later.
Regards,
Danyang
On 17-05-24 12:21 PM, Hong wrote:
> Danyang :
> I tested your data.
> Your matrices encountered zero pivots, e.g.
> petsc/src/ksp/ksp/examples/tutorials (master)
> $ mpiexec -n 24 ./ex10 -f0 a_react_in_2.bin -rhs b_react_in_2.bin
> -ksp_monitor -ksp_error_if_not_converged
>
> [15]PETSC ERROR: Zero pivot in LU factorization:
> http://www.mcs.anl.gov/petsc/documentation/faq.html#zeropivot
> [15]PETSC ERROR: Zero pivot row 1249 value 2.05808e-14 tolerance
> 2.22045e-14
> ...
>
> Adding option '-sub_pc_factor_shift_type nonzero', I got
> mpiexec -n 24 ./ex10 -f0 a_react_in_2.bin -rhs b_react_in_2.bin
> -ksp_monitor -ksp_error_if_not_converged -sub_pc_factor_shift_type
> nonzero -mat_view ascii::ascii_info
>
> Mat Object: 24 MPI processes
> type: mpiaij
> rows=450000, cols=450000
> total: nonzeros=6991400, allocated nonzeros=6991400
> total number of mallocs used during MatSetValues calls =0
> not using I-node (on process 0) routines
> 0 KSP Residual norm 5.849777711755e+01
> 1 KSP Residual norm 6.824179430230e-01
> 2 KSP Residual norm 3.994483555787e-02
> 3 KSP Residual norm 6.085841461433e-03
> 4 KSP Residual norm 8.876162583511e-04
> 5 KSP Residual norm 9.407780665278e-05
> Number of iterations = 5
> Residual norm 0.00542891
>
> Hong
>
> Hi Matt,
>
> Yes. The matrix is 450000x450000 sparse. The hypre takes hundreds
> of iterates, not for all but in most of the timesteps. The matrix
> is not well conditioned, with nonzero entries range from 1.0e-29
> to 1.0e2. I also made double check if there is anything wrong in
> the parallel version, however, the matrix is the same with
> sequential version except some round error which is relatively
> very small. Usually for those not well conditioned matrix, direct
> solver should be faster than iterative solver, right? But when I
> use the sequential iterative solver with ILU prec developed almost
> 20 years go by others, the solver converge fast with appropriate
> factorization level. In other words, when I use 24 processor using
> hypre, the speed is almost the same as as the old sequential
> iterative solver using 1 processor.
>
> I use most of the default configuration for the general case with
> pretty good speedup. And I am not sure if I miss something for
> this problem.
>
> Thanks,
>
> Danyang
>
>
> On 17-05-24 11:12 AM, Matthew Knepley wrote:
>> On Wed, May 24, 2017 at 12:50 PM, Danyang Su
>> <danyang.su at gmail.com <mailto:danyang.su at gmail.com>> wrote:
>>
>> Hi Matthew and Barry,
>>
>> Thanks for the quick response.
>>
>> I also tried superlu and mumps, both work but it is about
>> four times slower than ILU(dt) prec through hypre, with 24
>> processors I have tested.
>>
>> You mean the total time is 4x? And you are taking hundreds of
>> iterates? That seems hard to believe, unless you are dropping
>> a huge number of elements.
>>
>> When I look into the convergence information, the method
>> using ILU(dt) still takes 200 to 3000 linear iterations for
>> each newton iteration. One reason is this equation is hard to
>> solve. As for the general cases, the same method works
>> awesome and get very good speedup.
>>
>> I do not understand what you mean here.
>>
>> I also doubt if I use hypre correctly for this case. Is there
>> anyway to check this problem, or is it possible to increase
>> the factorization level through hypre?
>>
>> I don't know.
>>
>> Matt
>>
>> Thanks,
>>
>> Danyang
>>
>>
>> On 17-05-24 04:59 AM, Matthew Knepley wrote:
>>> On Wed, May 24, 2017 at 2:21 AM, Danyang Su
>>> <danyang.su at gmail.com <mailto:danyang.su at gmail.com>> wrote:
>>>
>>> Dear All,
>>>
>>> I use PCFactorSetLevels for ILU and PCFactorSetFill for
>>> other preconditioning in my code to help solve the
>>> problems that the default option is hard to solve.
>>> However, I found the latter one, PCFactorSetFill does
>>> not take effect for my problem. The matrices and rhs as
>>> well as the solutions are attached from the link below.
>>> I obtain the solution using hypre preconditioner and it
>>> takes 7 and 38 iterations for matrix 1 and matrix 2.
>>> However, if I use other preconditioner, the solver just
>>> failed at the first matrix. I have tested this matrix
>>> using the native sequential solver (not PETSc) with ILU
>>> preconditioning. If I set the incomplete factorization
>>> level to 0, this sequential solver will take more than
>>> 100 iterations. If I increase the factorization level to
>>> 1 or more, it just takes several iterations. This remind
>>> me that the PC factor for this matrices should be
>>> increased. However, when I tried it in PETSc, it just
>>> does not work.
>>>
>>> Matrix and rhs can be obtained from the link below.
>>>
>>> https://eilinator.eos.ubc.ca:8443/index.php/s/CalUcq9CMeblk4R
>>> <https://eilinator.eos.ubc.ca:8443/index.php/s/CalUcq9CMeblk4R>
>>>
>>> Would anyone help to check if you can make this work by
>>> increasing the PC factor level or fill?
>>>
>>>
>>> We have ILU(k) supported in serial. However ILU(dt) which
>>> takes a tolerance only works through Hypre
>>>
>>> http://www.mcs.anl.gov/petsc/documentation/linearsolvertable.html
>>> <http://www.mcs.anl.gov/petsc/documentation/linearsolvertable.html>
>>>
>>> I recommend you try SuperLU or MUMPS, which can both be
>>> downloaded automatically by configure, and
>>> do a full sparse LU.
>>>
>>> Thanks,
>>>
>>> Matt
>>>
>>> Thanks and regards,
>>>
>>> Danyang
>>>
>>>
>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin
>>> their experiments is infinitely more interesting than any
>>> results to which their experiments lead.
>>> -- Norbert Wiener
>>>
>>> http://www.caam.rice.edu/~mk51/
>>> <http://www.caam.rice.edu/%7Emk51/>
>>
>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to
>> which their experiments lead.
>> -- Norbert Wiener
>>
>> http://www.caam.rice.edu/~mk51/ <http://www.caam.rice.edu/%7Emk51/>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170524/f7a2d84e/attachment.html>
More information about the petsc-users
mailing list