[petsc-users] Bad memory scaling with PETSc 3.10

[Myriam Peyrounette]

Hi Matt,

I plotted the memory scalings using different threshold values. The two
scalings are slightly translated (from -22 to -88 mB) but this gain is
neglectable. The 3.6-scaling keeps being robust while the 3.10-scaling
deteriorates.

Do you have any other suggestion?

Thanks

Myriam

Le 03/02/19 à 02:27, Matthew Knepley a écrit :
> On Fri, Mar 1, 2019 at 10:53 AM Myriam Peyrounette via petsc-users
> <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>> wrote:
>
>     Hi,
>
>     I used to run my code with PETSc 3.6. Since I upgraded the PETSc
>     version
>     to 3.10, this code has a bad memory scaling.
>
>     To report this issue, I took the PETSc script ex42.c and slightly
>     modified it so that the KSP and PC configurations are the same as
>     in my
>     code. In particular, I use a "personnalised" multi-grid method. The
>     modifications are indicated by the keyword "TopBridge" in the attached
>     scripts.
>
>     To plot the memory (weak) scaling, I ran four calculations for each
>     script with increasing problem sizes and computations cores:
>
>     1. 100,000 elts on 4 cores
>     2. 1 million elts on 40 cores
>     3. 10 millions elts on 400 cores
>     4. 100 millions elts on 4,000 cores
>
>     The resulting graph is also attached. The scaling using PETSc 3.10
>     clearly deteriorates for large cases, while the one using PETSc 3.6 is
>     robust.
>
>     After a few tests, I found that the scaling is mostly sensitive to the
>     use of the AMG method for the coarse grid (line 1780 in
>     main_ex42_petsc36.cc). In particular, the performance strongly
>     deteriorates when commenting lines 1777 to 1790 (in
>     main_ex42_petsc36.cc).
>
>     Do you have any idea of what changed between version 3.6 and version
>     3.10 that may imply such degradation?
>
>
> I believe the default values for PCGAMG changed between versions. It
> sounds like the coarsening rate
> is not great enough, so that these grids are too large. This can be
> set using:
>
>   https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCGAMGSetThreshold.html
>
> There is some explanation of this effect on that page. Let us know if
> setting this does not correct the situation.
>
>   Thanks,
>
>      Matt
>  
>
>     Let me know if you need further information.
>
>     Best,
>
>     Myriam Peyrounette
>
>
>     -- 
>     Myriam Peyrounette
>     CNRS/IDRIS - HLST
>     --
>
>
>
> -- 
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/%7Eknepley/>

-- 
Myriam Peyrounette
CNRS/IDRIS - HLST
--

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190305/10b3efce/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2975 bytes
Desc: Signature cryptographique S/MIME
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190305/10b3efce/attachment-0001.p7s>