<div dir="ltr">On Thu, Mar 21, 2013 at 10:08 AM, Karl Rupp <span dir="ltr"><<a href="mailto:rupp@mcs.anl.gov" target="_blank">rupp@mcs.anl.gov</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<div class="im"><br>
<br>
> Yes, I want to preserve the behavior Barry described in 2), i.e.<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
only error if the user specified an allocation pattern explicitly.<br>
I'm only questioning the way it is currently implemented:<br>
<br>
If the number of nonzeros is specified as PETSC_DECIDE to<br></div>
MatSeqAIJSetPreallocation___<u></u>SeqAIJ(),<br>
it does not set MAT_NEW_NONZERO_ALLOCATION___<u></u>ERR. If the number of<div class="im"><br>
nonzeros is larger than zero, it does. So far, so good.<br>
<br>
<br>
If nnz is nonnegative, yes.<br>
</div></blockquote>
<br>
right, it's actually nz >= 0.<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
MatMPIAIJSetPreallocation___<u></u>MPIAIJ(), however, first checks for the<div class="im"><br>
number of nonzeros. If it is PETSC_DECIDE, it sets defaults and then<br></div>
calls MatSeqAIJSetPreallocation___<u></u>SeqAIJ() with the defaults.<br>
Because MatSeqAIJSetPreallocation___<u></u>SeqAIJ() no longer sees<br>
PETSC_DECIDE, it sets MAT_NEW_NONZERO_ALLOCATION___<u></u>ERR, which is<br>
then reset right after in MatMPIAIJSetPreallocation___<u></u>MPIAIJ(). The<div class="im"><br>
CUSP-implementation does not reset the flag and thus leads to<br>
errors, even though the implementation of the CUSP-Preallocation<br>
routine looks perfectly fine. I rather want to fix the hack in<br></div>
MatMPIAIJSetPreallocation___<u></u>MPIAIJ() than to propagate it to the GPU<div class="im"><br>
implementations as well.<br>
<br>
<br>
I thought perhaps you were going to leave it to the _SeqAIJ parts to<br>
store the nonew state. If the _MPIAIJ part resolves PETSC_DECIDE<br>
independently from the _SeqAIJ part, then they have to *always* coincide<br>
exactly, otherwise we'll have very confusing bugs. You can likely<br>
forward PETSC_DECIDE down to the _SeqAIJ parts, then query a->nonew and<br>
b->nonew to set the mpiaij->nonew flag. (I'm not sure it's consistent<br>
now, but we want it to be consistent.)<br>
</div></blockquote>
<br>
There is no mpiaij->nonew flag in Mat_MPIAIJ, this is all handled in the two diagonal/off-processor matrices of type Mat_SeqAIJ. I don't want to change this ;-)<br>
I just want to resolve the way way a->nonew and b->nonew is set, so that it does not require a hack in MatMPIAIJSetPreallocation_<u></u>MPIAIJ().<br>
<br>
A closer look showed three options:<br>
a) just forward PETSC_DECIDE to MatSeqAIJSetPreallocation_<u></u>SeqAIJ(), don't modify the nonew-flags from _MPIAIJ(). This almost fixes the problem, the only drawback is that the number of nonzeros on the off-processor blocks is then also 5 instead of 2 when using PETSC_DECIDE.<br>
</blockquote><div><br></div><div style>I vote a). No new function, gets rid of a stupid optimization that does no one any good, and is the simplest.</div><div style><br></div><div style> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
b) Allow to pass default values to MatSeqAIJSetPreallocation_<u></u>SeqAIJ().<br>
The cause of the issue is that MatSeqAIJSetPreallocation_<u></u>SeqAIJ() does two things: It interprets PETSC_DECIDE and it does the preallocation. If we split these two tasks into two functions, i.e.<br>
<br>
- MatSeqAIJSetPreallocation_<u></u>SeqAIJ() for interpreting PETSC_DECIDE and dealing with nonew, then calling<br>
- MatSeqAIJSetPreallocation_<u></u>SeqAIJAlloc() for the actual preallocation<br>
<br>
we can directly call MatSeqAIJSetPreallocation_<u></u>SeqAIJAlloc() from the _MPIAIJ() part and cleanly set MAT_NEW_NONZERO_ALLOCATION_ERR for the on- and off-processor matrices.<br>
<br>
c) Improve the existing hack by querying the state of a->nonew and b->nonew before calling MatSeqAIJSetPreallocation_<u></u>SeqAIJ() and after that reset a user-defined state if necessary.<br>
<br>
I'll create a pull request for b), as it is consistent with the current behavior and appears to be the cleanest solution to this.<br>
<br>
Best regards,<br>
Karli<br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener
</div></div>