[petsc-users] Bad memory scaling with PETSc 3.10

[Zhang, Hong]

Myriam,
Thanks for the plot. '-mat_freeintermediatedatastructures' should not affect solution. It releases almost half of memory in C=PtAP if C is not reused.
Hong

On Wed, Apr 10, 2019 at 7:21 AM Mark Adams <mfadams at lbl.gov<mailto:mfadams at lbl.gov>> wrote:
This looks like it might be noisy data. I'd make sure you run each size on the same set of nodes and you might run each job twice (A,B,A,B) in a job script.

On Wed, Apr 10, 2019 at 8:12 AM Myriam Peyrounette via petsc-users <petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>> wrote:

Here is the time weak scaling from the same study. The 3.10.2 version seems to be much more stable with regard to the execution time. But not necessarily faster for "large scale" simulations (problem size = 1e8).

I didn't use -mat_freeintermediatedatastructures. I tested it this morning and the solver diverges when using this option (KSPReason -3).

Myriam

Le 04/09/19 à 17:23, Zhang, Hong a écrit :
Myriam,
Do you have 'execution time scalability' plot? Did you use '-mat_freeintermediatedatastructures' for PETSc 3.10.2?
We made several computing optimizations on MatPtAP(), which might trade memory for speed. It would be helpful to see a complete comparison.
Hong

On Tue, Apr 9, 2019 at 7:43 AM Myriam Peyrounette via petsc-users <petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>> wrote:
Hi,

in my first mail, I provided a memory scaling concerning the PETSc
example #42. You'll find attached the main files used (one for PETSc
3.6.4, one for PETSc 3.10.2), and the corresponding memory scaling.

In the main files, I modified the solver/preconditioner, so that it
corresponds to my problem. You'll find the modifications by searching
the keyword "TopBridge". In particular, I use GAMG.

Note that the example is about solving Stokes equation, so using GAMG
may not be adapted. However, the memory gap appears and that's the
point. No matter if the results are correct.

Are these scripts useful for you? Let me know.

Thanks,

Myriam


Le 04/04/19 à 00:09, Jed Brown a écrit :
> Myriam Peyrounette via petsc-users <petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>> writes:
>
>> Hi all,
>>
>> for your information, you'll find attached the comparison of the weak
>> memory scalings when using :
>>
>> - PETSc 3.6.4 (reference)
>> - PETSc 3.10.4 without specific options
>> - PETSc 3.10.4 with the three scalability options you mentionned
>>
>> Using the scalability options does improve the memory scaling. However,
>> the 3.6 version still has a better one...
> Yes, this still looks significant.  Is this an effect we can still
> reproduce with a PETSc example and/or using a memory profiler (such as
> massif or gperftools)?  I think it's important for us to narrow down
> what causes this difference (looks like almost 2x on your 1e8 problem
> size) so we can fix.

--
Myriam Peyrounette
CNRS/IDRIS - HLST
--



--
Myriam Peyrounette
CNRS/IDRIS - HLST
--

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190410/c0bee0ca/attachment.html>