Matrix Allocation

[Barry Smith]

   I found the cause, from MatAssemblyEnd_SeqAIJ()

   ierr = MatMarkDiagonal_SeqAIJ(A);CHKERRQ(ierr);
   ierr = PetscInfo4(A,"Matrix size: %D X %D; storage space: %D  
unneeded,%D used\n",m,A->cmap.n,fshift,a->nz);CHKERRQ(ierr);
   ierr = PetscInfo1(A,"Number of mallocs during MatSetValues() is %D 
\n",a->reallocs);CHKERRQ(ierr);
   ierr = PetscInfo1(A,"Maximum nonzeros in any row is %D 
\n",rmax);CHKERRQ(ierr);

   a->reallocs          = 0;
   A->info.nz_unneeded  = (double)fshift;
   a->rmax              = rmax;


   It prints the correct value and then sets the value to zero!

   Presumably this is done to get an accurate count of new  
reallocations if the user does a bunch more
MatSetValues().  But this is silly.

    How to fix this? We could simply remove the zeroing and have this  
count be cumulative over all the previous
MatSetValues/MatAssembly... calls. Or introduce two counters, a  
cumulative and recent counter. I would suggest
going with the simple fix.

    Barry





On Jun 30, 2008, at 3:15 PM, Andrew Colombi wrote:

>> 1) Could you be calling MatGetInfo() with the wrong matrix?
>
> Nice try, but I only have one matrix ;-)  I call it A.  We get along.
>
>> 2) Can you use -info to check the numbers?
>> 3) Can you try MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, 1)?
>
> Yes and yes.  In fact, I even wrote a small test app just for this
> purpose which is revealing some interesting behavior.  Results:
> Additional memory is being malloced; however, MatGetInfo does not
> appear to be reporting this.  Here I've grepped the output for
> "malloc":
>
> $ rjq ./make -info | grep malloc
> [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.
> [1] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()  
> is 0
> [0] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.
> [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()  
> is 1
> [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()  
> is 0
> [1] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()  
> is 1
> [ 1]    mallocs 0 memory 64
> [ 0]    mallocs 0 memory 64
>
> The initial messages are from -info, and the last two are from my
> MatGetInfo snippet.  Clearly two mallocs are happening that are not
> being reported.  Maybe this is reason: those mallocs don't happen in
> MatAssembly_MPIAIJ?  The reason for this is that each processor only
> computes the rows it's reponsible for.
>
> Thoughts?
> -Andrew
>
> On Mon, Jun 30, 2008 at 1:38 PM, Matthew Knepley <knepley at gmail.com>  
> wrote:
>> 1) Could you be calling MatGetInfo() with the wrong matrix?
>>
>> 2) Can you use -info to check the numbers?
>>
>> 3) Can you try MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, 1)?
>>
>>    Matt
>>
>> On Mon, Jun 30, 2008 at 12:25 PM, Andrew Colombi  
>> <acolombi at gmail.com> wrote:
>>> I am a little confused by some behavior.
>>>
>>> I've been trying to understand some poor performance numbers and I
>>> noticed that if I doubled the preallocation numbers (presently I'm
>>> setting d_nz and o_nz to constants, no effort has been made to get  
>>> the
>>> numbers _right_) performance improved dramatically (about 20X).
>>>
>>> This isn't so surprising except for the report from MatGetInfo.   
>>> Even
>>> with the lower preallocation numbers I'm getting 0 in info.mallocs  
>>> for
>>>
>>> MatInfo info;
>>> MatGetInfo(pc_A, MAT_LOCAL, &info); // done and reported on each
>>> machine individually
>>>
>>> So, my question is.... well.. why?  If I'm seeing 0 mallocs before
>>> doubling prealloation shouldn't that mean I've preallocated enough?
>>> Or are their some switches I need to use to enable malloc counting?
>>> Also you can see (according to the same call to MetGetInfo) I'm
>>> wasting a lot of memory:
>>>
>>> // after doubling preallocation
>>> nz_alloc 6.2704e+07 nz_used 1.9125e+07 nz_unneed 4.3579e+07
>>>
>>> (these are print outs of info.nz_allocated, info.nz_used and  
>>> info.nzunneeded).
>>>
>>> Any thoughts?  For now memory is not a bottleneck, so I guess I'll  
>>> be
>>> satisfied with guessing big numbers for d_nz and o_nz.  Still, I  
>>> spent
>>> a lot of time scratching my head since guessing higher numbers  
>>> didn't
>>> seem likely to have an effect.
>>>
>>> Thanks,
>>> -Andrew
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which
>> their experiments lead.
>> -- Norbert Wiener
>>
>>
>