[petsc-users] GAMG scalability for serendipity 20 nodes hexahedra

[TARDIEU Nicolas]

Thank you very much for your answer, Mark.
Do you think it is worth it, to play around with aggregation variants? Plain aggregation "à la Notay" for instance.

Nicolas
 



     

De : mfadams at lbl.gov <mfadams at lbl.gov>
Envoyé : mercredi 26 juin 2019 22:37
À : TARDIEU Nicolas
Cc : PETSc users list
Objet : Re: [petsc-users] GAMG scalability for serendipity 20 nodes hexahedra
  

I get growth with Q2 elements also. I've never seen anyone report scaling of high order elements with generic AMG.


First, discretizations are very important for AMG solver. All optimal solvers really. I've never looked at serendipity elements. It might be a good idea to try Q2 as well.


SNES ex56 is 3D elasticity on a cube with tensor elements. Below are parameters that I have been using. I see some evidence that more smoothing steps (-mg_levels_ksp_max_it N) helps "scaling" but not necessarily solve time.


An example of what I see, running ex56 with -cells 8,12,16  -max_conv_its 5 and the below params I get these iteration counts: 19, 20, 31, 31, 38.


My guess is that you need higher order interpolation for higher order elements and when you add a new level you get an increase in condition number (ie, it is not an optimal MG method). But, the original smoothed aggregation paper did have high order discretizations  their theory said it was still optimal, as I recall.


Mark


-log_view
-max_conv_its 5
-petscspace_degree 2
-snes_max_it 2
-ksp_max_it 100
-ksp_type cg
-ksp_rtol 1.e-11
-ksp_atol 1.e-71
-ksp_norm_type unpreconditioned
-snes_rtol 1.e-10
-pc_type gamg
-pc_gamg_type agg
-pc_gamg_agg_nsmooths 1
-pc_gamg_coarse_eq_limit 1000
-pc_gamg_process_eq_limit 200
-pc_gamg_reuse_interpolation true
-ksp_converged_reason
-snes_monitor_short
-ksp_monitor_short
-snes_converged_reason
-use_mat_nearnullspace true
-mg_levels_ksp_max_it 4
-mg_levels_ksp_type chebyshev
-mg_levels_esteig_ksp_type cg
-gamg_est_ksp_type cg
-gamg_est_ksp_max_it 10
-mg_levels_esteig_ksp_max_it 10
-mg_levels_ksp_chebyshev_esteig 0,0.05,0,1.05
-mg_levels_pc_type jacobi
-petscpartitioner_type simple
-mat_block_size 3
-matptap_via scalable
-run_type 1
-pc_gamg_repartition false
-pc_gamg_threshold 0.0
-pc_gamg_threshold_scale .25
-pc_gamg_square_graph 1
-check_pointer_intensity 0
-snes_type ksponly
-ex56_dm_view
-options_left




 


On Wed, Jun 26, 2019 at 8:21 AM TARDIEU Nicolas via petsc-users <petsc-users at mcs.anl.gov> wrote:
 
Dear PETSc team,


I have run a simple weak scalability test based on canonical 3D elasticity problem : a cube, meshed with 8 nodes hexaedra, clamped on one of its face and submited to a pressure load on the opposite face. 
I am using the FGMRES ksp with GAMG as preconditioner. I have set the rigid body modes using MatNullSpaceCreateRigidBody and its works like a charm. The solver exhibit a perfect scalability until 800 cores (I haven't tested with more cores). The ksp always  converges in 11 or 12 iterations. Let me emphasize that I use GAMG default options.



Nevertheless, if I switch to a quadratic mesh with 20 nodes serendipity hexaedra, the weak scalability deteriorates. For instance the number of iteration for the ksp increases from 20 iterations for the smallest problem to 30 for the biggest. 
Here is my question : I wonder what is the right tuning for GAMG to recover the same weak scalability as in the linear case? I apologize if this is a stupid question...





 
I  look forward to reading you,  
Nicolas

  

Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute  diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses  copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.
Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________
This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure,  either whole or partial, is prohibited except formal approval.
If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.
E-mail communication cannot be guaranteed to be timely secure, error or virus-free.
       



Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.