<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>Do you have any idea when Barry's fix
(<a class="moz-txt-link-freetext" href="https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff">https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff</a>)
will be released? I can see it has been merged to the "next"
branch. Does it mean it will be soon available on master? <br>
</p>
<p>+for your information, I plotted a summary of the scalings of
interest (memory and time):<br>
- using petsc-3.10.2 (ref "bad" scaling)<br>
- using petsc-3.6.4 (ref "good" scaling)<br>
- using commit d330a26 + Barry's fix and different algorithms
(none, scalable, allatonce, allatonce_merged)</p>
<p>Best regards,</p>
<p>Myriam<br>
</p>
<br>
<div class="moz-cite-prefix">Le 05/13/19 à 17:20, Fande Kong a
écrit :<br>
</div>
<blockquote type="cite"
cite="mid:CAN5Wd-KMuSCHUQ8AhGNT3zFLiudz+NMZoe8RxjB8wo11gHZH6w@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>Hi Myriam,</div>
<div><br>
</div>
<div>Thanks for your report back.</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, May
13, 2019 at 2:01 AM Myriam Peyrounette <<a
href="mailto:myriam.peyrounette@idris.fr"
moz-do-not-send="true">myriam.peyrounette@idris.fr</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Hi
all,<br>
<br>
I tried with 3.11.1 version and Barry's fix.
The good scaling is back!<br>
See the green curve in the plot attached. It
is even better than PETSc<br>
3.6! And it runs faster (10-15s instead of
200-300s with 3.6).<br>
</blockquote>
<div><br>
</div>
<div>We are glad your issue was resolved
here. </div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br>
So you were right. It seems that not all the
PtAPs used the scalable<br>
version.<br>
<br>
I was a bit confused about the options to
set... I used the options:<br>
-matptap_via scalable and
-mat_freeintermediatedatastructures 1. Do
you<br>
think it would be even better with
allatonce?<br>
</blockquote>
<div><br>
</div>
<div>"scalable" and "allatonce" correspond to
different algorithms respectively.
``allatonce" should be using less memory
than "scalable". The "allatonce" algorithm
would be a good alternative if your
application is memory sensitive and the
problem size is large. </div>
<div>We are definitely curious about the
memory usage of ``allatonce" in your test
cases but don't feel obligated to do these
tests since your concern were resolved now.
In case you are also interested in how our
new algorithms perform, I post petsc options
here that are used to </div>
<div>choose these algorithms:</div>
<div><br>
</div>
<div>algorithm 1: ``allatonce" </div>
<div><br>
</div>
<div>-matptap_via allatonce<br>
</div>
<div>-mat_freeintermediatedatastructures 1<br>
</div>
<div><br>
</div>
<div>algorithm 2: ``allatonce_merged" <br>
</div>
<div><br>
</div>
<div>
<div>-matptap_via allatonce_merged<br>
</div>
<div>-mat_freeintermediatedatastructures 1</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Again, thanks for your report that help
us improve PETSc.</div>
<div><br>
</div>
<div>Fande,</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br>
It is unfortunate that this fix can't be
merged with the master branch.<br>
But the patch works well and I can consider
the issue as solved now.<br>
<br>
Thanks a lot for your time!<br>
<br>
Myriam<br>
<br>
<br>
Le 05/04/19 à 06:54, Smith, Barry F. a
écrit :<br>
> Hmm, I had already fixed this, I
think, <br>
><br>
> <a
href="https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://bitbucket.org/petsc/petsc/pull-requests/1606/change-handling-of-matptap_mpiaij_mpimaij/diff</a><br>
><br>
> but unfortunately our backlog of
pull requests kept it out of master. We are
(well Satish and Jed) working on a new CI
infrastructure that will hopefully be more
stable than the current CI that we are
using.<br>
><br>
> Fande,<br>
> Sorry you had to spend time on
this. <br>
><br>
><br>
> Barry<br>
><br>
><br>
><br>
>> On May 3, 2019, at 11:20 PM, Fande
Kong via petsc-users <<a
href="mailto:petsc-users@mcs.anl.gov"
target="_blank" moz-do-not-send="true">petsc-users@mcs.anl.gov</a>>
wrote:<br>
>><br>
>> Hi Myriam,<br>
>><br>
>> I run the example you attached
earlier with "-mx 48 -my 48 -mz 48 -levels 3
-ksp_view -matptap_via allatonce -log_view
". <br>
>><br>
>> There are six PtAPs. Two of them
are sill using the nonscalable version of
the algorithm (this might explain why the
memory still exponentially increases) even
though we have asked PETSc to use the
``allatonce" algorithm. This is happening
because MATMAIJ does not honor the petsc
option, instead, it uses the default setting
of MPIAIJ. I have a fix at <a
href="https://bitbucket.org/petsc/petsc/pull-requests/1623/choose-algorithms-in/diff"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://bitbucket.org/petsc/petsc/pull-requests/1623/choose-algorithms-in/diff</a>.
The PR should fix the issue.<br>
>><br>
>> Thanks again for your report,<br>
>><br>
>> Fande,<br>
>><br>
>> <br>
<br>
-- <br>
Myriam Peyrounette<br>
CNRS/IDRIS - HLST<br>
--<br>
<br>
<br>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Myriam Peyrounette
CNRS/IDRIS - HLST
--
</pre>
</body>
</html>