<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div dir="ltr">
<div dir="ltr">Myriam:<br>
</div>
<div>This branch is merged to master.</div>
<div>Thanks for your work and patience. It helps us a lot. The graphs are very nice :-)</div>
<div><br>
</div>
<div>We plan to re-organise the APIs of mat-mat opts, make them easier for users.</div>
<div>Hong</div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>Do you have any idea when Barry's fix (<a class="gmail-m_-8191395118479287483moz-txt-link-freetext" href="https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff" target="_blank">https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff</a>)
will be released? I can see it has been merged to the "next" branch. Does it mean it will be soon available on master?
<br>
</p>
<p>+for your information, I plotted a summary of the scalings of interest (memory and time):<br>
- using petsc-3.10.2 (ref "bad" scaling)<br>
- using petsc-3.6.4 (ref "good" scaling)<br>
- using commit d330a26 + Barry's fix and different algorithms (none, scalable, allatonce, allatonce_merged)</p>
<p>Best regards,</p>
<p>Myriam<br>
</p>
<br>
<div class="gmail-m_-8191395118479287483moz-cite-prefix">Le 05/13/19 à 17:20, Fande Kong a écrit :<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>Hi Myriam,</div>
<div><br>
</div>
<div>Thanks for your report back.</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, May 13, 2019 at 2:01 AM Myriam Peyrounette <<a href="mailto:myriam.peyrounette@idris.fr" target="_blank">myriam.peyrounette@idris.fr</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Hi all,<br>
<br>
I tried with 3.11.1 version and Barry's fix. The good scaling is back!<br>
See the green curve in the plot attached. It is even better than PETSc<br>
3.6! And it runs faster (10-15s instead of 200-300s with 3.6).<br>
</blockquote>
<div><br>
</div>
<div>We are glad your issue was resolved here. </div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
So you were right. It seems that not all the PtAPs used the scalable<br>
version.<br>
<br>
I was a bit confused about the options to set... I used the options:<br>
-matptap_via scalable and -mat_freeintermediatedatastructures 1. Do you<br>
think it would be even better with allatonce?<br>
</blockquote>
<div><br>
</div>
<div>"scalable" and "allatonce" correspond to different algorithms respectively. ``allatonce" should be using less memory than "scalable". The "allatonce" algorithm would be a good alternative if your application is memory sensitive and the problem size is
large. </div>
<div>We are definitely curious about the memory usage of ``allatonce" in your test cases but don't feel obligated to do these tests since your concern were resolved now. In case you are also interested in how our new algorithms perform, I post petsc options
here that are used to </div>
<div>choose these algorithms:</div>
<div><br>
</div>
<div>algorithm 1: ``allatonce" </div>
<div><br>
</div>
<div>-matptap_via allatonce<br>
</div>
<div>-mat_freeintermediatedatastructures 1<br>
</div>
<div><br>
</div>
<div>algorithm 2: ``allatonce_merged" <br>
</div>
<div><br>
</div>
<div>
<div>-matptap_via allatonce_merged<br>
</div>
<div>-mat_freeintermediatedatastructures 1</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Again, thanks for your report that help us improve PETSc.</div>
<div><br>
</div>
<div>Fande,</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
It is unfortunate that this fix can't be merged with the master branch.<br>
But the patch works well and I can consider the issue as solved now.<br>
<br>
Thanks a lot for your time!<br>
<br>
Myriam<br>
<br>
<br>
Le 05/04/19 à 06:54, Smith, Barry F. a écrit :<br>
> Hmm, I had already fixed this, I think, <br>
><br>
> <a href="https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff" rel="noreferrer" target="_blank">
https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff</a><br>
><br>
> but unfortunately our backlog of pull requests kept it out of master. We are (well Satish and Jed) working on a new CI infrastructure that will hopefully be more stable than the current CI that we are using.<br>
><br>
> Fande,<br>
> Sorry you had to spend time on this. <br>
><br>
><br>
> Barry<br>
><br>
><br>
><br>
>> On May 3, 2019, at 11:20 PM, Fande Kong via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
>><br>
>> Hi Myriam,<br>
>><br>
>> I run the example you attached earlier with "-mx 48 -my 48 -mz 48 -levels 3 -ksp_view -matptap_via allatonce -log_view ".
<br>
>><br>
>> There are six PtAPs. Two of them are sill using the nonscalable version of the algorithm (this might explain why the memory still exponentially increases) even though we have asked PETSc to use the ``allatonce" algorithm. This is happening because MATMAIJ
does not honor the petsc option, instead, it uses the default setting of MPIAIJ. I have a fix at
<a href="https://bitbucket.org/petsc/petsc/pull-requests/1623/choose-algorithms-in/diff" rel="noreferrer" target="_blank">
https://bitbucket.org/petsc/petsc/pull-requests/1623/choose-algorithms-in/diff</a>. The PR should fix the issue.<br>
>><br>
>> Thanks again for your report,<br>
>><br>
>> Fande,<br>
>><br>
>> <br>
<br>
-- <br>
Myriam Peyrounette<br>
CNRS/IDRIS - HLST<br>
--<br>
<br>
<br>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<pre class="gmail-m_-8191395118479287483moz-signature" cols="72">--
Myriam Peyrounette
CNRS/IDRIS - HLST
--
</pre>
</div>
</blockquote>
</div>
</div>
</body>
</html>