[petsc-users] PETSc with mesh refinement

[Barry Smith]

  The TS/SNES/KSP/PC allocate various work vectors and matrices which they keep around for additional solvers. Thus if you want to change the size of your problem and/or number of degrees of freedom per process you need to do one of 2 things

   1) destroy the old TS/SNES/ or KSP and create a new one each time you change your vectors to a different size or

   2) call TSReset/SNESReset/or KSPReset each time you change your vectors, this causes all the internal work vectors and matrices to be freed so they can get replaced but keeps all the solver options etc.

   2) is preferred to 1.

   Barry

> On Sep 3, 2017, at 1:44 PM, Matthew Knepley <knepley at gmail.com> wrote:
> 
> On Sun, Sep 3, 2017 at 1:02 PM, Zou, Ling <ling.zou at inl.gov> wrote:
> Hi All,
> 
> I wonder if there is a good example on how PETSc should work with mesh refinement. I have done some online search/PETSc-user email search, but did not find a good one.
> 
> Here is my case. Currently, assuming NO mesh refinement (so I know exactly the problem size, mesh connectivity, non-zero pattern of matrices, etc.)
> In a transient simulation, my PETSc code looks like:
> 
> ========================================================================
> SetupVectors(); // unknown vec, old unknown, etc.
> SetupResFunc(); // residual function, basically pointing to the discretized PDE
> SetupMatrices(); // allocate memory for preconditioning matrix based on non-zero pattern (so it is mesh dependent)
>                            // I also use finite differencing to get preconditioning matrix, so there is another MatFDColoring object associated with the matrix
> for (int step = 0; step < max_step; step++)
> {
>   SNESSolve();
> }
> ========================================================================
> 
> This setup works perfectly for my problems.
> (I understand it is low efficient to use FD to get the preconditioning matrix, but it is not a priority to me at this moment).
> 
> However, now I would like to do some mesh refinement, so the previous setup would not work, because the problem size, non-zero pattern of the matrix and MatFDColoring change with mesh size (and connectivity).
> I suppose my code should be changed like this:
> 
> ========================================================================
> SetupVectors();
> SetupResFunc();
> SetupMatrices();
> 
> for (int step = 0; step < max_step; step++)
> {
>   MeshRefinement(); // I will manage my mesh
> 
>   AdjustVectors();   // resize vector
>   AdjustMatrices(); // resize matrix, re-allocate memory, adjust non-zero pattern, etc.
> 
>   SNESSolve();
> }
> ========================================================================
> 
> The issue is that I understand one should not re-allocate memory to both PETSc vector and matrix dynamically during simulation, which has been discussed here:
> https://scicomp.stackexchange.com/questions/373/is-it-possible-to-dynamically-resize-a-sparse-matrix-in-the-petsc-library
> 
> So, the final questions are:
> 1) Any good example on PETSc working with mesh refinement?
> 2) Any suggestion on my proposed code structure?
> 
> 1) No
> 
> 2) Yes. We advocate using the DM to describe your data layout. This allows all Vec and Mats to be created automatically, and
> also tells the solver (SNESSetDM, etc.) the sizes and potential decomposition.
> 
> We are working on a mesh adaptivity interface which will allow adaptivity to be used with all DM problems without any extra
> code. We should have some examples with this soon.
> 
>   Thanks,
> 
>      Matt
>  
> Best,
> 
> Ling
> 
> 
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
> 
> http://www.caam.rice.edu/~mk51/