[petsc-users] pcsetup

[Barry Smith]

> On Jul 21, 2020, at 5:11 PM, Alex Fleeter <luis.saturday at gmail.com> wrote:
> 
> Hi:
> 
> I guess I could rephrase my question like this. I am using MATNEST.
> 
> When I updated the matrix, I did assembly only for the sub-matrices. Then when I call the KSP solver, I can see that the pcsetup was called only in the first time the solver gets called. 
> 
> I tried to add MatAssembly for the big nest matrix as a second test. Then, when I called my solver, I could see that the pcsetup gets called.
> 
> I assume that there is a state flag in the MatNest object that tells the solver if it needs to update the preconditioner. After an assembly routine gets called, this flag will be activated. Could you confirm that?

   PCSetUp() has 

  ierr = PetscObjectStateGet((PetscObject)pc->pmat,&matstate);CHKERRQ(ierr);
  ierr = MatGetNonzeroState(pc->pmat,&matnonzerostate);CHKERRQ(ierr);
  if (!pc->setupcalled) {
    ierr     = PetscInfo(pc,"Setting up PC for first time\n");CHKERRQ(ierr);
    pc->flag = DIFFERENT_NONZERO_PATTERN;
  } else if (matstate == pc->matstate) {
    ierr = PetscInfo(pc,"Leaving PC with identical preconditioner since operator is unchanged\n");CHKERRQ(ierr);
    PetscFunctionReturn(0);

  so it rebuilds the preconditioner if the PetscObjectState of the matrix increased since the last application of the preconditioner. 

  Calling MatAssemblyBegin/End on a matrix always increases the matrix state (even if you have not changed the matrix entries). This is why

"MatAssembly for the big nest matrix as a second test. Then, when I called my solver, I could see that the pcsetup gets called."

  MatNest however is fundamentally broken because if you change entries in the sub-matrices you used to build the MatNest, the MatNest state is not increased hence any call to PCSetUp() with the MatNest as an argument does not trigger a new PCSetUp(). The author of MatNest knows about this flaw but has not bothered to fix it. 

  Thus every time you change values in the submatrices you must also call MatAssemblyBegin/End on the nest matrix (as you discovered) so the preconditioner  THAT DIRECTLY USES the MatNest knows it needs to rebuild.

   Now I don't understand completely the code you have below and why or if there is a problem. The two KSP newctx->upper and newctx->lower do not know about the MatNest or use the MatNest state,  they only know about the sub matrices and the sub matrix states.  So if you change entries in the sub matrices then KSPSolve(newctx->upper) will trigger rebuilding the preconditioner inside it. That is you get the behavior you expect.  In other words blockpc_apply() will rebuild the two interior PCs each time you change the sub matrices. In other words for your custom PCShell you do not need to call MatAssemblyBegin/End on the nest matrix since the two interior PCs know directly about changes to the sub matrices, since their input is the sub matrices not the nest matrix.

  Do you not get them rebuilding the preconditioner? If you run in the debugger and put a break point in PCSetUp() you should see those two interior PCs getting rebuilt exactly when they should be.

  I hope this is not confusing: a KSP/PC that takes a MatNest argument will not rebuild the preconditioner that depends on the MatNest argument if you don't call MatAssemblyBegin/End on the MatNest. But any KSP/PC that depend on individual submatrices will get rebuilt correctly since they use the state of the individual sub matrices.


  Barry





> 
> The problem is that I am trying to implement a shell preconditioner, which gets the sub-matrices in the setup routine like the following
>   MatNestGetSubMat(A, 0, 0, &newctx->submat[0]);
>   MatNestGetSubMat(A, 0, 1, &newctx->submat[1]);
>   MatNestGetSubMat(A, 1, 0, &newctx->submat[2]);
>   MatNestGetSubMat(A, 1, 1, &newctx->submat[3]);
> 
>   KSPSetOperators(newctx->upper, newctx->submat[0], newctx->submat[0]);
>   KSPSetOperators(newctx->lower, newctx->submat[3], newctx->submat[3]);
> 
> In the Apply stage, I do
> 
> PetscErrorCode blockpc_apply(PC pc, Vec x, Vec y)
> {
>   std::cout<<"APPLY PC\n";
>   blockpc * newctx;
>   PCShellGetContext(pc,(void**)&newctx);
> 
>   Vec x_v, x_p, y_v, y_p;
>   VecGetSubVector(x, newctx->is[0], &x_v);
>   VecGetSubVector(x, newctx->is[1], &x_p);
>   VecGetSubVector(y, newctx->is[0], &y_v);
>   VecGetSubVector(y, newctx->is[1], &y_p);
> 
>   KSPSolve( newctx -> upper, x_v, y_v );
>   KSPSolve( newctx -> lower, x_p, y_p );
> 
>   VecRestoreSubVector(x, newctx->is[0], &x_v);
>   VecRestoreSubVector(x, newctx->is[1], &x_p);
>   VecRestoreSubVector(y, newctx->is[0], &y_v);
>   VecRestoreSubVector(y, newctx->is[1], &y_p);
> 
>   return 0;
> }
> 
> I want to know if I shall call an assembly for the big nest matrix so that the pcsetup can be called everytime the solver is dealing with a new matrix.
> 
> Thanks,
> 
> 
> 
> 
> 
> 
> On Tue, Jul 21, 2020 at 12:44 PM Alex Fleeter <luis.saturday at gmail.com <mailto:luis.saturday at gmail.com>> wrote:
> Hi:
> 
> I want to ask under what circumstance will trigger a call for pc setup. 
> 
> I call KSPSolve to solve with the same Mat object with different entry values each time. I can see that the pc setup is only called at the beginning of the first solve. 
> 
> I tried to read the implementation, but quickly get lost...

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200721/87443a6b/attachment.html>