[petsc-users] Matrix reuse with changing structure

[Smith, Kevin R.]

The use case is dynamic overset mesh coupling. I can guarantee that all matrices will fit since I have all the information I need to do so at initialization. The component meshes do not themselves change, only the overset assembly changes. I have to over-allocate memory to ensure the matrix always fits to cover the range of motion.

I did figure out a way to avoid deleting the matrix every time I solve in my system, so that provides some level of an optimization if PETSc doesn’t support restructuring sparse matrices out of the box. With this approach I can also avoid over-allocating memory. However, my use case is unlikely to hit the memory limits, even if I have to over-subscribe the preallocation to cover the entire range of motion. So I think there will still be a clear benefit to doing the allocation once, and avoiding the reallocation each time structure changes.

Not sure if this figure will get filtered out by the mailing list, but this shows a basic overset matrix structure. The green portions are the overset coupling regions, which may move around the matrix as the body moves. The number of coupling coefficients has a maximum N which is known, so I can preallocate to cover the entire range of motion. Hope that helps describe the use case.

[cid:image001.png at 01D17F7E.0D243AA0]

From: Matthew Knepley [mailto:knepley at gmail.com]
Sent: Wednesday, March 16, 2016 10:37 AM
To: Smith, Kevin R.
Cc: Barry Smith; petsc-users at mcs.anl.gov
Subject: Re: [petsc-users] Matrix reuse with changing structure

On Wed, Mar 16, 2016 at 8:23 AM, Smith, Kevin R. <Kevin.R.Smith at jhuapl.edu<mailto:Kevin.R.Smith at jhuapl.edu>> wrote:
Barry - I need to avoid dynamic allocation because it results in too much of a slowdown.
Matt - The MatSetType thing did not work for some reason. It seems like PETSc wants me to set the preallocation after calling this method (suggested by the error below). I'm trying to reuse the original block of preallocated memory for the sparse matrix and blow away the structure each time. This would let me avoid repeated deallocate/allocate calls throughout my simulation.

We need to understand the use case better. How can you guarantee that your new matrix will
fit in the old matrix memroy? If you cannot, then we have to reallocate anyway.

  Thanks,

    Matt

PETSc Function: MatGetOwnershipRange
 PETSc File: /share/maul-data/smithkr1/src/petsc-3.6.0/src/mat/interface/matrix.c
 PETSc Line: 6289
 PETSc Error Code: 73
 PETSc Error reference: http://www.mcs.anl.gov/petsc/petsc-dev/include/petscerror.h.html
 PETSc Message: Must call MatXXXSetPreallocation() or MatSetUp() on argument 1 "mat" before MatGetOwnershipRange()

Kind regards,
 Kevin

-----Original Message-----
From: Barry Smith [mailto:bsmith at mcs.anl.gov<mailto:bsmith at mcs.anl.gov>]
Sent: Tuesday, March 15, 2016 2:02 PM
To: Smith, Kevin R.
Cc: Matthew Knepley; petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] Matrix reuse with changing structure


> On Mar 15, 2016, at 12:56 PM, Smith, Kevin R. <Kevin.R.Smith at jhuapl.edu<mailto:Kevin.R.Smith at jhuapl.edu>> wrote:
>
>
> Barry - Yeah, I suspected it was doing this. In my original implementation I would get allocation errors.

   You need to call a MatSetOption() to tell the matrix that it is allowed to allocate new nonzeros.


>
> Matt - Thanks, I will try this solution out.
>
> Thanks for your help,
>
>  Kevin
>
> -----Original Message-----
> From: Barry Smith [mailto:bsmith at mcs.anl.gov<mailto:bsmith at mcs.anl.gov>]
> Sent: Tuesday, March 15, 2016 1:48 PM
> To: Matthew Knepley
> Cc: Smith, Kevin R.; petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>
> Subject: Re: [petsc-users] Matrix reuse with changing structure
>
>
>> On Mar 15, 2016, at 12:28 PM, Matthew Knepley <knepley at gmail.com<mailto:knepley at gmail.com>> wrote:
>>
>> On Tue, Mar 15, 2016 at 12:24 PM, Smith, Kevin R. <Kevin.R.Smith at jhuapl.edu<mailto:Kevin.R.Smith at jhuapl.edu>> wrote:
>> Hello,
>>
>>
>>
>> Is it possible to reuse a sparse matrix and not reuse its non-zero structure?   I have a matrix whose sparse structure changes each time. I’m hoping to avoid destroying and allocating new matrices each time the structure changes.
>
>   Sure you can just add new nonzero locations at a later time. But it won't take out any current entries even if they are zero. So effectively the nonzero structure grows over time.
>
>   Barry
>
>>
>>
>> Hmm, I can't find a toplevel API that does this (it would be something like MatReset()). You can get this effect using
>>
>>  MatSetType(A, MATSHELL)
>>  MatSetType(A, <type you had before>)
>>
>> A little messy but it will work.
>>
>>  Thanks,
>>
>>     Matt
>>
>>
>>
>> Thanks,
>>
>>  Kevin
>>
>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>> -- Norbert Wiener
>



--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160316/b65a9df6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 394372 bytes
Desc: image001.png
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160316/b65a9df6/attachment-0001.png>