[petsc-users] Mat preallocation for adaptive grid

[Matthew Knepley]

On Sat, Jun 11, 2022 at 8:32 PM Samuel Estes <samuelestes91 at gmail.com>
wrote:

> Hello,
>
> My question concerns preallocation for Mats in adaptive FEM problems. When
> the grid refines, I destroy the old matrix and create a new one of the
> appropriate (larger size). When the grid “un-refines” I just use the same
> (extra large) matrix and pad the extra unused diagonal entries with 1’s.
> The problem comes in with the preallocation. I use the MatPreallocator,
> MatPreallocatorPreallocate() paradigm which requires a specific sparsity
> pattern. When the grid un-refines, although the total number of nonzeros
> allocated is (most likely) more than sufficient, the particular sparsity
> pattern changes which leads to mallocs in the MatSetValues routines and
> obviously I would like to avoid this.
>
> One obvious solution is just to destroy and recreate the matrix any time
> the grid changes, even if it gets smaller. By just using a new matrix every
> time, I would avoid this problem although at the cost of having to rebuild
> the matrix more often than necessary. This is the simplest solution from a
> programming perspective and probably the one I will go with.
>
> I'm just curious if there's an alternative that you would recommend?
> Basically what I would like to do is to just change the sparsity pattern
> that is created in the MatPreallocatorPreallocate() routine. I'm not sure
> how it works under the hood, but in principle, it should be possible to
> keep the memory allocated for the Mat values and just assign them new
> column numbers and potentially add new nonzeros as well. Is there a
> convenient way of doing this? One thought I had was to just fill in the
> MatPreallocator object with the new sparsity pattern of the coarser mesh
> and then call the MatPreallocatorPreallocate() routine again with the new
> MatPreallocator matrix. I'm just not sure how exactly that would work since
> it would have already been called for the FEM matrix for the previous,
> finer grid.
>
> Finally, does this really matter? I imagine the bottleneck (assuming good
> preallocation) is in the solver so maybe it doesn't make much difference
> whether or not I reuse the old matrix. In that case, going with option 1
> and simply destroying and recreating the matrix would be the way to go just
> to save myself some time.
>
> I hope that my question is clear. If not, please let me know and I will
> clarify. I am very curious if there's a convenient solution for the second
> option I mentioned to recycle the allocated memory and redo the sparsity
> pattern.
>

I have not run any tests of this kind of thing, so I cannot say
definitively.

I can say that I consider the reuse of memory a problem to be solved at
allocation time. You would hope that a good malloc system would give
you back the same memory you just freed when getting rid of the prior
matrix, so you would get the speedup you want using your approach.

Second, I think the allocation cost is likely to pale in comparison to the
cost of writing the matrix itself (passing all those indices and values
through
the memory bus), and so reuse of the memory is not that important (I think).

  Thanks,

      Matt


> Thanks!
>
> Sam
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220611/620cd7cb/attachment.html>