<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Eric :</div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
thanks... but I just realized, that into petsc 3.5.4 (which I used for this calculus) the default *was* the scalable version... :/<br></blockquote><div>We used it as default in the beginning, then switched to 'nonscalable' as default. We fixed some bugs in the latest release for the scalable version. More testings would help though.</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
So I prepared a patch (see attachment) to help debug, but it will be based on 3.7.2. We may use it to extract information to do a better bug report. Do you think there is enough PetscInfo added to resolve this issue?<br></blockquote><div> </div><div>Your patch is to add a set of PetscInfo(). We use PetscInfo for optimization, e.g.,</div><div>line 321 of mpimatmatmult.c:</div><div> PetscInfo3(Cmpi,"Reallocs %D; Fill ratio: given %g needed %g....);<br></div><div><br></div><div>You want to display following info for debugging support:</div><div>am, pN, lnnz, lMem, apnz_max,...</div><div><br></div><div>When there is a malloc() failure, PETSc would indicate the line of the code, so user would know which parameter causes the problem. Adding so many PetscInfo() for these parameters is not good for code maintenance, in my opinion.<br></div><div><br></div><div>For your problem with <span style="font-size:12.8px">1.7 billion variables, you should use 'scalable' version.</span></div><div><span style="font-size:12.8px">Let us know if it fails. </span></div><div><br></div><div>Hong</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span class=""><br>
On 21/06/16 09:36 AM, <a href="mailto:hong@aspiritech.org" target="_blank">hong@aspiritech.org</a> wrote:<br>
</span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span class="">
Eric:<br>
The nonscalable implementation is robust, and faster for small to medium<br>
size problems, thus we set it as the default. You can switch with option<br>
'-matmatmult_via scalable', which requires estimate of nonzeros A*B.<br>
The estimate was buggy, not well-tested. If you encounter any problem,<br>
let us know.<br>
<br>
Hong<br>
<br>
On Mon, Jun 20, 2016 at 10:49 PM, Eric Chamberland<br>
<<a href="mailto:Eric.Chamberland@giref.ulaval.ca" target="_blank">Eric.Chamberland@giref.ulaval.ca</a><br></span><span class="">
<mailto:<a href="mailto:Eric.Chamberland@giref.ulaval.ca" target="_blank">Eric.Chamberland@giref.ulaval.ca</a>>> wrote:<br>
<br>
<br>
<br>
Le 2016-06-20 23:37, Barry Smith a écrit :<br>
<br>
On Jun 20, 2016, at 10:32 PM, Eric Chamberland<br>
<<a href="mailto:Eric.Chamberland@giref.ulaval.ca" target="_blank">Eric.Chamberland@giref.ulaval.ca</a><br></span><span class="">
<mailto:<a href="mailto:Eric.Chamberland@giref.ulaval.ca" target="_blank">Eric.Chamberland@giref.ulaval.ca</a>>> wrote:<br>
<br>
ok, but what about -matmatmult_via scalable?<br>
<br>
Both should work. It just may be one is faster or slower<br>
than the other depending on the problem size.<br>
<br>
ok, digging further, I found this into blaming the code:<br>
<br>
0fc8cf34 (Hong Zhang 2013-06-27 14:04:58 -0500 696) /* same<br>
as MatMatMultSymbolic_MPIAIJ_MPIAIJ_nonscalable(), except using<br>
LLCondensed to avoid O(BN) memory requirement */<br>
<br>
But the commit comment says:<br>
...<br>
rename MatMatMultSymbolic_MPIAIJ_MPIAIJ -><br>
MatMatMultSymbolic_MPIAIJ_MPIAIJ_nonscalable (non-default)<br>
<br>
But it *is* the default... since another commit:<br>
<br>
commit 0d3441ae8a080c728abf17e90308c510e39e951b<br></span>
Author: Hong Zhang <<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a> <mailto:<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>>><span class=""><br>
Date: Mon Aug 24 16:40:35 2015 -0500<br>
<br>
add MatPtAPxxx_MPIAIJ_MPIAIJ_new<br>
<br>
which changed the behaviour programmed in 0fc8cf34. Is it normal?<br>
<br>
Eric<br>
<br>
<br>
<br>
</span></blockquote>
<br>
</blockquote></div><br></div></div>