[petsc-users] Bad memory scaling with PETSc 3.10

[Myriam Peyrounette]

Hi,

I used to run my code with PETSc 3.6. Since I upgraded the PETSc version
to 3.10, this code has a bad memory scaling.

To report this issue, I took the PETSc script ex42.c and slightly
modified it so that the KSP and PC configurations are the same as in my
code. In particular, I use a "personnalised" multi-grid method. The
modifications are indicated by the keyword "TopBridge" in the attached
scripts.

To plot the memory (weak) scaling, I ran four calculations for each
script with increasing problem sizes and computations cores:

1. 100,000 elts on 4 cores
2. 1 million elts on 40 cores
3. 10 millions elts on 400 cores
4. 100 millions elts on 4,000 cores

The resulting graph is also attached. The scaling using PETSc 3.10
clearly deteriorates for large cases, while the one using PETSc 3.6 is
robust.

After a few tests, I found that the scaling is mostly sensitive to the
use of the AMG method for the coarse grid (line 1780 in
main_ex42_petsc36.cc). In particular, the performance strongly
deteriorates when commenting lines 1777 to 1790 (in main_ex42_petsc36.cc).

Do you have any idea of what changed between version 3.6 and version
3.10 that may imply such degradation?

Let me know if you need further information.

Best,

Myriam Peyrounette


-- 
Myriam Peyrounette
CNRS/IDRIS - HLST
--

-------------- next part --------------
A non-text attachment was scrubbed...
Name: main_ex42_petsc36.cc
Type: text/x-c++src
Size: 89014 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190301/56bb3a09/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: main_ex42_petsc310.cc
Type: text/x-c++src
Size: 89260 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190301/56bb3a09/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: scaling_ex42.png
Type: image/png
Size: 24427 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190301/56bb3a09/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2975 bytes
Desc: Signature cryptographique S/MIME
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190301/56bb3a09/attachment-0001.p7s>