[petsc-users] Bad memory scaling with PETSc 3.10

[Myriam Peyrounette]

Oh sorry, I missed that. That's great!

Thanks,

Myriam


Le 05/29/19 à 16:55, Zhang, Hong a écrit :
> Myriam:
> This branch is merged to master.
> Thanks for your work and patience. It helps us a lot. The graphs are
> very nice :-)
>
> We plan to re-organise the APIs of mat-mat opts, make them easier for
> users.
> Hong
>
>     Hi,
>
>     Do you have any idea when Barry's fix
>     (https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff)
>     will be released? I can see it has been merged to the "next"
>     branch. Does it mean it will be soon available on master?
>
>     +for your information, I plotted a summary of the scalings of
>     interest (memory and time):
>     - using petsc-3.10.2 (ref "bad" scaling)
>     - using petsc-3.6.4 (ref "good" scaling)
>     - using commit d330a26 + Barry's fix and different algorithms
>     (none, scalable, allatonce, allatonce_merged)
>
>     Best regards,
>
>     Myriam
>
>
>     Le 05/13/19 à 17:20, Fande Kong a écrit :
>>     Hi Myriam,
>>
>>     Thanks for your report back.
>>
>>     On Mon, May 13, 2019 at 2:01 AM Myriam Peyrounette
>>     <myriam.peyrounette at idris.fr
>>     <mailto:myriam.peyrounette at idris.fr>> wrote:
>>
>>         Hi all,
>>
>>         I tried with 3.11.1 version and Barry's fix. The good scaling
>>         is back!
>>         See the green curve in the plot attached. It is even better
>>         than PETSc
>>         3.6! And it runs faster (10-15s instead of 200-300s with 3.6).
>>
>>
>>     We are glad your issue was resolved here. 
>>      
>>
>>
>>         So you were right. It seems that not all the PtAPs used the
>>         scalable
>>         version.
>>
>>         I was a bit confused about the options to set... I used the
>>         options:
>>         -matptap_via scalable and -mat_freeintermediatedatastructures
>>         1. Do you
>>         think it would be even better with allatonce?
>>
>>
>>     "scalable" and "allatonce" correspond to different algorithms
>>     respectively. ``allatonce" should be using less memory than
>>     "scalable". The "allatonce" algorithm  would be a good
>>     alternative if your application is memory sensitive and the
>>     problem size is large. 
>>     We are definitely curious about the memory usage of ``allatonce"
>>     in your test cases but don't feel obligated to do these tests
>>     since your concern were resolved now. In case you are also
>>     interested in how our new algorithms perform, I post petsc
>>     options here that are used to 
>>     choose these algorithms:
>>
>>     algorithm 1: ``allatonce" 
>>
>>     -matptap_via allatonce
>>     -mat_freeintermediatedatastructures 1
>>
>>     algorithm 2: ``allatonce_merged" 
>>
>>     -matptap_via allatonce_merged
>>     -mat_freeintermediatedatastructures 1
>>
>>
>>     Again, thanks for your report that help us improve PETSc.
>>
>>     Fande,
>>      
>>
>>
>>         It is unfortunate that this fix can't be merged with the
>>         master branch.
>>         But the patch works well and I can consider the issue as
>>         solved now.
>>
>>         Thanks a lot for your time!
>>
>>         Myriam
>>
>>
>>         Le 05/04/19 à 06:54, Smith, Barry F. a écrit :
>>         >    Hmm, I had already fixed this, I think,
>>         >
>>         >   
>>         https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff
>>         >
>>         >    but unfortunately our backlog of pull requests kept it
>>         out of master. We are (well Satish and Jed) working on a new
>>         CI infrastructure that will hopefully be more stable than the
>>         current CI that we are using.
>>         >
>>         >    Fande,
>>         >       Sorry you had to spend time on this.
>>         >
>>         >
>>         >    Barry
>>         >
>>         >
>>         >
>>         >> On May 3, 2019, at 11:20 PM, Fande Kong via petsc-users
>>         <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>> wrote:
>>         >>
>>         >> Hi Myriam,
>>         >>
>>         >> I run the example you attached earlier with "-mx 48 -my 48
>>         -mz 48 -levels 3 -ksp_view  -matptap_via allatonce -log_view ". 
>>         >>
>>         >> There are six PtAPs. Two of them are sill using the
>>         nonscalable version of the algorithm (this might explain why
>>         the memory still exponentially increases) even though we have
>>         asked PETSc to use the ``allatonce" algorithm. This is
>>         happening because MATMAIJ does not honor the petsc option,
>>         instead, it uses the default setting of MPIAIJ.  I have a fix
>>         at
>>         https://bitbucket.org/petsc/petsc/pull-requests/1623/choose-algorithms-in/diff.
>>         The PR should fix the issue.
>>         >>
>>         >> Thanks again for your report,
>>         >>
>>         >> Fande,
>>         >>
>>         >> 
>>
>>         -- 
>>         Myriam Peyrounette
>>         CNRS/IDRIS - HLST
>>         --
>>
>>
>
>     -- 
>     Myriam Peyrounette
>     CNRS/IDRIS - HLST
>     --
>

-- 
Myriam Peyrounette
CNRS/IDRIS - HLST
--

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190529/63bc3022/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2975 bytes
Desc: Signature cryptographique S/MIME
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190529/63bc3022/attachment-0001.p7s>