[petsc-users] Strange GAMG performance for mixed FE formulation

Mark Adams mfadams at lbl.gov
Fri Mar 4 16:36:27 CST 2016 

And it looks like you have a well behaved Laplacian here (M-matrix) so I
would guess 'richardson' would be faster as the smoother, instead of
'chebyshev'.

On Fri, Mar 4, 2016 at 5:04 PM, Mark Adams <mfadams at lbl.gov> wrote:

> You seem to have 3 of one type of solve that is give 'square_graph 1':
>
> 0] PC*GAMG*Coarsen_AGG(): Square Graph on level 1 of 1 to square
>
> This has 9 nnz-row and 44% are zero:
>
> [0] PC*GAMG*FilterGraph():   55.7114% nnz after filtering, with threshold
> 0., 8.79533 nnz ave.
>
> So you  want to use a _negative_ threshold.  This will keep zeros in the
> graph and help it to coarsen faster.  And you could try adding more levels
> to square the graph.
>
> The second type of solve has the 'square_graph 10'.  It looks like the
> first solve.  It should also use a negative threshold also.
>
> ML has a default of zero for the threshold, but it seems to keep zeros
> whereas GAMG does not.
>
> Mark
>
>
> On Fri, Mar 4, 2016 at 10:38 AM, Justin Chang <jychang48 at gmail.com> wrote:
>
>> Time to solution went from 100 seconds to 30 seconds once i used 10
>> graphs. Using 20 graphs started to increase in time slightly
>>
>> On Fri, Mar 4, 2016 at 8:35 AM, Justin Chang <jychang48 at gmail.com> wrote:
>>
>>> You're right. This is what I have:
>>>
>>> [0] PCSetUp_*GAMG*(): level 0) N=48000, n data rows=1, n data cols=1,
>>> nnz/row (ave)=9, np=1
>>>
>>> [0] PC*GAMG*FilterGraph():  55.7114% nnz after filtering, with
>>> threshold 0., 8.79533 nnz ave. (N=48000)
>>>
>>> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 1 of 1 to square
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 6672 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.954700e+00
>>> min=1.040410e-02 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 1) N=6672, n data cols=1, nnz/row (ave)=623, 1
>>> active pes
>>>
>>> [0] PC*GAMG*FilterGraph():  3.40099% nnz after filtering, with
>>> threshold 0., 623.135 nnz ave. (N=6672)
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 724 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.313339e+00
>>> min=2.474586e-02 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 2) N=724, n data cols=1, nnz/row (ave)=724, 1
>>> active pes
>>>
>>> [0] PC*GAMG*FilterGraph():  9.82914% nnz after filtering, with
>>> threshold 0., 724. nnz ave. (N=724)
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 37 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=2.011784e+00
>>> min=2.759552e-01 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 3) N=37, n data cols=1, nnz/row (ave)=37, 1
>>> active pes
>>>
>>> [0] PCSetUp_*GAMG*(): 4 levels, grid complexity = 12.0928
>>>
>>> [0] PCSetUp_*GAMG*(): level 0) N=48000, n data rows=1, n data cols=1,
>>> nnz/row (ave)=9, np=1
>>>
>>> [0] PC*GAMG*FilterGraph():  55.7114% nnz after filtering, with
>>> threshold 0., 8.79533 nnz ave. (N=48000)
>>>
>>> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 1 of 1 to square
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 6672 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.954700e+00
>>> min=1.040410e-02 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 1) N=6672, n data cols=1, nnz/row (ave)=623, 1
>>> active pes
>>>
>>> [0] PC*GAMG*FilterGraph():  3.40099% nnz after filtering, with
>>> threshold 0., 623.135 nnz ave. (N=6672)
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 724 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.313339e+00
>>> min=2.474586e-02 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 2) N=724, n data cols=1, nnz/row (ave)=724, 1
>>> active pes
>>>
>>> [0] PC*GAMG*FilterGraph():  9.82914% nnz after filtering, with
>>> threshold 0., 724. nnz ave. (N=724)
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 37 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=2.011784e+00
>>> min=2.759552e-01 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 3) N=37, n data cols=1, nnz/row (ave)=37, 1
>>> active pes
>>>
>>> [0] PCSetUp_*GAMG*(): 4 levels, grid complexity = 12.0928
>>>
>>> [0] PCSetUp_*GAMG*(): level 0) N=162000, n data rows=1, n data cols=1,
>>> nnz/row (ave)=9, np=1
>>>
>>> [0] PC*GAMG*FilterGraph():  55.6621% nnz after filtering, with
>>> threshold 0., 8.863 nnz ave. (N=162000)
>>>
>>> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 1 of 1 to square
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 22085 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.955376e+00
>>> min=8.260696e-03 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 1) N=22085, n data cols=1, nnz/row (ave)=704, 1
>>> active pes
>>>
>>> [0] PC*GAMG*FilterGraph():  3.1314% nnz after filtering, with threshold
>>> 0., 704.128 nnz ave. (N=22085)
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 2283 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.311291e+00
>>> min=1.484874e-02 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 2) N=2283, n data cols=1, nnz/row (ave)=2283, 1
>>> active pes
>>>
>>> [0] PC*GAMG*FilterGraph():  3.64497% nnz after filtering, with
>>> threshold 0., 2283. nnz ave. (N=2283)
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 97 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=2.043254e+00
>>> min=1.321528e-01 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 3) N=97, n data cols=1, nnz/row (ave)=97, 1
>>> active pes
>>>
>>> [0] PC*GAMG*FilterGraph():  66.8403% nnz after filtering, with
>>> threshold 0., 97. nnz ave. (N=97)
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 5 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.653762e+00
>>> min=4.460582e-01 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 4) N=5, n data cols=1, nnz/row (ave)=5, 1 active
>>> pes
>>>
>>> [0] PCSetUp_*GAMG*(): 5 levels, grid complexity = 15.4673
>>>
>>> [0] PCSetUp_*GAMG*(): level 0) N=162000, n data rows=1, n data cols=1,
>>> nnz/row (ave)=9, np=1
>>>
>>> [0] PC*GAMG*FilterGraph():  55.6621% nnz after filtering, with
>>> threshold 0., 8.863 nnz ave. (N=162000)
>>>
>>> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 1 of 10 to square
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 22085 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.955376e+00
>>> min=8.260696e-03 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 1) N=22085, n data cols=1, nnz/row (ave)=704, 1
>>> active pes
>>>
>>> [0] PC*GAMG*FilterGraph():  3.1314% nnz after filtering, with threshold
>>> 0., 704.128 nnz ave. (N=22085)
>>>
>>> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 2 of 10 to square
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 545 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.311291e+00
>>> min=1.484874e-02 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 2) N=545, n data cols=1, nnz/row (ave)=545, 1
>>> active pes
>>>
>>> [0] PC*GAMG*FilterGraph():  7.55997% nnz after filtering, with
>>> threshold 0., 545. nnz ave. (N=545)
>>>
>>> [0] PC*GAMG*Coarsen_AGG(): Square Graph on level 3 of 10 to square
>>>
>>> [0] PC*GAMG*Prolongator_AGG(): New grid 11 nodes
>>>
>>> [0] PC*GAMG*OptProlongator_AGG(): Smooth P0: max eigen=1.368729e+00
>>> min=1.563750e-01 PC=jacobi
>>>
>>> [0] PCSetUp_*GAMG*(): 3) N=11, n data cols=1, nnz/row (ave)=11, 1
>>> active pes
>>>
>>> [0] PCSetUp_*GAMG*(): 4 levels, grid complexity = 12.0376
>>>
>>> On Fri, Mar 4, 2016 at 8:31 AM, Lawrence Mitchell <
>>> lawrence.mitchell at imperial.ac.uk> wrote:
>>>
>>>>
>>>> > On 4 Mar 2016, at 15:24, Justin Chang <jychang48 at gmail.com> wrote:
>>>> >
>>>> > So with -pc_gamg_square_graph 10 I get the following:
>>>>
>>>> Because you're using gamg inside the fieldsplit, I think you need:
>>>>
>>>> -fieldsplit_1_pc_gamg_square_graph 10
>>>>
>>>>
>>>>
>>>> > [0] PCSetUp_GAMG(): level 0) N=48000, n data rows=1, n data cols=1,
>>>> nnz/row (ave)=9, np=1
>>>> > [0] PCGAMGFilterGraph():       55.7114% nnz after filtering, with
>>>> threshold 0., 8.79533 nnz ave. (N=48000)
>>>> > [0] PCGAMGCoarsen_AGG(): Square Graph on level 1 of 1 to square
>>>>                                                  ^^^^^
>>>>
>>>> Cheers,
>>>>
>>>> Lawrence
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160304/46d1fa25/attachment-0001.html>

More information about the petsc-users mailing list