[petsc-users] Strange GAMG performance for mixed FE formulation

Fri Mar 4 09:38:56 CST 2016

Time to solution went from 100 seconds to 30 seconds once i used 10 graphs.
Using 20 graphs started to increase in time slightly

On Fri, Mar 4, 2016 at 8:35 AM, Justin Chang <jychang48 at gmail.com> wrote:

> You're right. This is what I have:
>
> [0] PCSetUp_*GAMG*(): level 0) N=48000, n data rows=1, n data cols=1,
> nnz/row (ave)=9, np=1
>
> [0] PC*GAMG*FilterGraph():  55.7114% nnz after filtering, with threshold
> 0., 8.79533 nnz ave. (N=48000)
>
> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 1 of 1 to square
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 6672 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.954700e+00
> min=1.040410e-02 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 1) N=6672, n data cols=1, nnz/row (ave)=623, 1
> active pes
>
> [0] PC*GAMG*FilterGraph():  3.40099% nnz after filtering, with threshold
> 0., 623.135 nnz ave. (N=6672)
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 724 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.313339e+00
> min=2.474586e-02 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 2) N=724, n data cols=1, nnz/row (ave)=724, 1
> active pes
>
> [0] PC*GAMG*FilterGraph():  9.82914% nnz after filtering, with threshold
> 0., 724. nnz ave. (N=724)
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 37 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=2.011784e+00
> min=2.759552e-01 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 3) N=37, n data cols=1, nnz/row (ave)=37, 1 active
> pes
>
> [0] PCSetUp_*GAMG*(): 4 levels, grid complexity = 12.0928
>
> [0] PCSetUp_*GAMG*(): level 0) N=48000, n data rows=1, n data cols=1,
> nnz/row (ave)=9, np=1
>
> [0] PC*GAMG*FilterGraph():  55.7114% nnz after filtering, with threshold
> 0., 8.79533 nnz ave. (N=48000)
>
> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 1 of 1 to square
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 6672 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.954700e+00
> min=1.040410e-02 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 1) N=6672, n data cols=1, nnz/row (ave)=623, 1
> active pes
>
> [0] PC*GAMG*FilterGraph():  3.40099% nnz after filtering, with threshold
> 0., 623.135 nnz ave. (N=6672)
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 724 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.313339e+00
> min=2.474586e-02 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 2) N=724, n data cols=1, nnz/row (ave)=724, 1
> active pes
>
> [0] PC*GAMG*FilterGraph():  9.82914% nnz after filtering, with threshold
> 0., 724. nnz ave. (N=724)
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 37 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=2.011784e+00
> min=2.759552e-01 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 3) N=37, n data cols=1, nnz/row (ave)=37, 1 active
> pes
>
> [0] PCSetUp_*GAMG*(): 4 levels, grid complexity = 12.0928
>
> [0] PCSetUp_*GAMG*(): level 0) N=162000, n data rows=1, n data cols=1,
> nnz/row (ave)=9, np=1
>
> [0] PC*GAMG*FilterGraph():  55.6621% nnz after filtering, with threshold
> 0., 8.863 nnz ave. (N=162000)
>
> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 1 of 1 to square
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 22085 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.955376e+00
> min=8.260696e-03 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 1) N=22085, n data cols=1, nnz/row (ave)=704, 1
> active pes
>
> [0] PC*GAMG*FilterGraph():  3.1314% nnz after filtering, with threshold
> 0., 704.128 nnz ave. (N=22085)
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 2283 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.311291e+00
> min=1.484874e-02 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 2) N=2283, n data cols=1, nnz/row (ave)=2283, 1
> active pes
>
> [0] PC*GAMG*FilterGraph():  3.64497% nnz after filtering, with threshold
> 0., 2283. nnz ave. (N=2283)
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 97 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=2.043254e+00
> min=1.321528e-01 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 3) N=97, n data cols=1, nnz/row (ave)=97, 1 active
> pes
>
> [0] PC*GAMG*FilterGraph():  66.8403% nnz after filtering, with threshold
> 0., 97. nnz ave. (N=97)
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 5 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.653762e+00
> min=4.460582e-01 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 4) N=5, n data cols=1, nnz/row (ave)=5, 1 active pes
>
> [0] PCSetUp_*GAMG*(): 5 levels, grid complexity = 15.4673
>
> [0] PCSetUp_*GAMG*(): level 0) N=162000, n data rows=1, n data cols=1,
> nnz/row (ave)=9, np=1
>
> [0] PC*GAMG*FilterGraph():  55.6621% nnz after filtering, with threshold
> 0., 8.863 nnz ave. (N=162000)
>
> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 1 of 10 to square
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 22085 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.955376e+00
> min=8.260696e-03 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 1) N=22085, n data cols=1, nnz/row (ave)=704, 1
> active pes
>
> [0] PC*GAMG*FilterGraph():  3.1314% nnz after filtering, with threshold
> 0., 704.128 nnz ave. (N=22085)
>
> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 2 of 10 to square
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 545 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.311291e+00
> min=1.484874e-02 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 2) N=545, n data cols=1, nnz/row (ave)=545, 1
> active pes
>
> [0] PC*GAMG*FilterGraph():  7.55997% nnz after filtering, with threshold
> 0., 545. nnz ave. (N=545)
>
> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 3 of 10 to square
>
> [0] PC*GAMG*Prolongator_AGG(): New grid 11 nodes
>
> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.368729e+00
> min=1.563750e-01 PC=jacobi
>
> [0] PCSetUp_*GAMG*(): 3) N=11, n data cols=1, nnz/row (ave)=11, 1 active
> pes
>
> [0] PCSetUp_*GAMG*(): 4 levels, grid complexity = 12.0376
>
> On Fri, Mar 4, 2016 at 8:31 AM, Lawrence Mitchell <
> lawrence.mitchell at imperial.ac.uk> wrote:
>
>>
>> > On 4 Mar 2016, at 15:24, Justin Chang <jychang48 at gmail.com> wrote:
>> >
>> > So with -pc_gamg_square_graph 10 I get the following:
>>
>> Because you're using gamg inside the fieldsplit, I think you need:
>>
>> -fieldsplit_1_pc_gamg_square_graph 10
>>
>>
>>
>> > [0] PCSetUp_GAMG(): level 0) N=48000, n data rows=1, n data cols=1,
>> nnz/row (ave)=9, np=1
>> > [0] PCGAMGFilterGraph():       55.7114% nnz after filtering, with
>> threshold 0., 8.79533 nnz ave. (N=48000)
>> > [0] PCGAMGCoarsen_AGG(): Square Graph on level 1 of 1 to square
>>                                                  ^^^^^
>>
>> Cheers,
>>
>> Lawrence
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160304/2ddc4788/attachment-0001.html>