<div dir="ltr">On Thu, Mar 21, 2013 at 7:50 AM, Karl Rupp <span dir="ltr"><<a href="mailto:rupp@mcs.anl.gov" target="_blank">rupp@mcs.anl.gov</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Jed,<div><div class="h5"><br>
<br>
> As the comment noted, this is quite a hack and actually breaks the<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
current implementation for GPU matrices. I think that the problem<br>
can be completely resolved by not checking for PETSC_DEFAULT and<br>
PETSC_DECIDE in the MPI preallocation above, but just forward d_nz<br>
and o_nz to the sequential preallocation routine. This way, the<br>
logic remains in MatSeqAIJSetPreallocation(), where<br>
MAT_NEW_NONZERO_ALLOCATION_ERR is only set if the user provided an<br>
explicit preallocation scheme.<br>
<br>
<br>
Hmm, the 'nonew' member is accessed in the MATMPIAIJ so we have to<br>
either change the code to avoid ever using it from the top level or we<br>
need to propagate the setting back to the parent. See also this discussion:<br>
<br>
<a href="http://lists.mcs.anl.gov/pipermail/petsc-dev/2012-January/007121.html" target="_blank">http://lists.mcs.anl.gov/<u></u>pipermail/petsc-dev/2012-<u></u>January/007121.html</a><br>
<br>
<a href="https://bitbucket.org/petsc/petsc/commits/7827cd58" target="_blank">https://bitbucket.org/petsc/<u></u>petsc/commits/7827cd58</a><br>
</blockquote>
<br></div></div>
Yes, I want to preserve the behavior Barry described in 2), i.e. only error if the user specified an allocation pattern explicitly. I'm only questioning the way it is currently implemented:<br>
<br>
If the number of nonzeros is specified as PETSC_DECIDE to<br>
MatSeqAIJSetPreallocation_<u></u>SeqAIJ(),<br>
it does not set MAT_NEW_NONZERO_ALLOCATION_<u></u>ERR. If the number of nonzeros is larger than zero, it does. So far, so good.<br></blockquote><div><br></div><div style>If nnz is nonnegative, yes.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
MatMPIAIJSetPreallocation_<u></u>MPIAIJ(), however, first checks for the number of nonzeros. If it is PETSC_DECIDE, it sets defaults and then calls MatSeqAIJSetPreallocation_<u></u>SeqAIJ() with the defaults. Because MatSeqAIJSetPreallocation_<u></u>SeqAIJ() no longer sees PETSC_DECIDE, it sets MAT_NEW_NONZERO_ALLOCATION_<u></u>ERR, which is then reset right after in MatMPIAIJSetPreallocation_<u></u>MPIAIJ(). The CUSP-implementation does not reset the flag and thus leads to errors, even though the implementation of the CUSP-Preallocation routine looks perfectly fine. I rather want to fix the hack in MatMPIAIJSetPreallocation_<u></u>MPIAIJ() than to propagate it to the GPU implementations as well.<br>
</blockquote><div><br></div><div style>I thought perhaps you were going to leave it to the _SeqAIJ parts to store the nonew state. If the _MPIAIJ part resolves PETSC_DECIDE independently from the _SeqAIJ part, then they have to *always* coincide exactly, otherwise we'll have very confusing bugs. You can likely forward PETSC_DECIDE down to the _SeqAIJ parts, then query a->nonew and b->nonew to set the mpiaij->nonew flag. (I'm not sure it's consistent now, but we want it to be consistent.)</div>
</div></div></div>