[petsc-users] Bad memory scaling with PETSc 3.10

[Fande Kong]

Hi Myriam,

Thanks for your report back.

On Mon, May 13, 2019 at 2:01 AM Myriam Peyrounette <
myriam.peyrounette at idris.fr> wrote:

> Hi all,
>
> I tried with 3.11.1 version and Barry's fix. The good scaling is back!
> See the green curve in the plot attached. It is even better than PETSc
> 3.6! And it runs faster (10-15s instead of 200-300s with 3.6).
>

We are glad your issue was resolved here.


>
> So you were right. It seems that not all the PtAPs used the scalable
> version.
>
> I was a bit confused about the options to set... I used the options:
> -matptap_via scalable and -mat_freeintermediatedatastructures 1. Do you
> think it would be even better with allatonce?
>

"scalable" and "allatonce" correspond to different algorithms respectively.
``allatonce" should be using less memory than "scalable". The "allatonce"
algorithm  would be a good alternative if your application is memory
sensitive and the problem size is large.
We are definitely curious about the memory usage of ``allatonce" in your
test cases but don't feel obligated to do these tests since your concern
were resolved now. In case you are also interested in how our new
algorithms perform, I post petsc options here that are used to
choose these algorithms:

algorithm 1: ``allatonce"

-matptap_via allatonce
-mat_freeintermediatedatastructures 1

algorithm 2: ``allatonce_merged"

-matptap_via allatonce_merged
-mat_freeintermediatedatastructures 1


Again, thanks for your report that help us improve PETSc.

Fande,


>
> It is unfortunate that this fix can't be merged with the master branch.
> But the patch works well and I can consider the issue as solved now.
>
> Thanks a lot for your time!
>
> Myriam
>
>
> Le 05/04/19 à 06:54, Smith, Barry F. a écrit :
> >    Hmm, I had already fixed this, I think,
> >
> >
> https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff
> >
> >    but unfortunately our backlog of pull requests kept it out of master.
> We are (well Satish and Jed) working on a new CI infrastructure that will
> hopefully be more stable than the current CI that we are using.
> >
> >    Fande,
> >       Sorry you had to spend time on this.
> >
> >
> >    Barry
> >
> >
> >
> >> On May 3, 2019, at 11:20 PM, Fande Kong via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
> >>
> >> Hi Myriam,
> >>
> >> I run the example you attached earlier with "-mx 48 -my 48 -mz 48
> -levels 3 -ksp_view  -matptap_via allatonce -log_view ".
> >>
> >> There are six PtAPs. Two of them are sill using the nonscalable version
> of the algorithm (this might explain why the memory still exponentially
> increases) even though we have asked PETSc to use the ``allatonce"
> algorithm. This is happening because MATMAIJ does not honor the petsc
> option, instead, it uses the default setting of MPIAIJ.  I have a fix at
> https://bitbucket.org/petsc/petsc/pull-requests/1623/choose-algorithms-in/diff.
> The PR should fix the issue.
> >>
> >> Thanks again for your report,
> >>
> >> Fande,
> >>
> >>
>
> --
> Myriam Peyrounette
> CNRS/IDRIS - HLST
> --
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190513/a2bbd984/attachment.html>