[petsc-dev] DMKSP and pcfieldsplit

[Matthew Knepley]

On Thu, Feb 21, 2019 at 10:18 AM Lawrence Mitchell via petsc-dev <
petsc-dev at mcs.anl.gov> wrote:

> Dear petsc-dev,
>
> 95dbaa6faf01fdfd99114b7c9e5668e4b2aa754d
>
> which does this:
>
> commit 95dbaa6faf01fdfd99114b7c9e5668e4b2aa754d
> Author: Matthew G. Knepley <knepley at gmail.com>
> Date:   Fri Oct 26 14:12:14 2018 -0400
>
>     PC: FieldSplit must copy DMKSP routines to subDM
>
> diff --git a/src/ksp/pc/impls/fieldsplit/fieldsplit.c
> b/src/ksp/pc/impls/fieldsplit/fieldsplit.c
> index b6a84ea342..f43f831204 100644
> --- a/src/ksp/pc/impls/fieldsplit/fieldsplit.c
> +++ b/src/ksp/pc/impls/fieldsplit/fieldsplit.c
> @@ -386,6 +386,13 @@ static PetscErrorCode PCFieldSplitSetDefaults(PC pc)
>            ierr = PetscObjectSetOptionsPrefix((PetscObject)(dms[i]),
> prefix);CHKERRQ(ierr);
>            ierr = KSPSetDM(ilink->ksp, dms[i]);CHKERRQ(ierr);
>            ierr = KSPSetDMActive(ilink->ksp, PETSC_FALSE);CHKERRQ(ierr);
> +          {
> +            PetscErrorCode (*func)(KSP,Mat,Mat,void*);
> +            void            *ctx;
> +
> +            ierr = DMKSPGetComputeOperators(pc->dm, &func,
> &ctx);CHKERRQ(ierr);
> +            ierr = DMKSPSetComputeOperators(dms[i],  func,
> ctx);CHKERRQ(ierr);
> +          }
>            ierr =
> PetscObjectIncrementTabLevel((PetscObject)dms[i],(PetscObject)ilink->ksp,0);CHKERRQ
>            ierr = DMDestroy(&dms[i]);CHKERRQ(ierr);
>          }
>
> Breaks many uses of PCMG and fieldsplit for us.
>
> I would like to try and understand how best to fix it. My understanding of
> the way this stuff works is that when I say:
>
> KSPSetComputeOperators
>
> I provide a callback function (which I control), and some user-defined
> context (which I also control).
>
> What this code is doing (which previously it did not), is to transfer this
> callback and context inside PCFieldsplit from a DM defining the full
> problem onto a DM from DMCreateFieldDecomposition (i.e. only part of the
> problem).
>
> This seems like it is tremendously fragile, because PETSc has no way of
> knowing that the context for the full DM (which is just a void *) is valid
> for the subdm. In particular, in my case, I think it turns out not to be.
>

Here is why I did this. I wanted this to work


https://bitbucket.org/petsc/petsc/src/4dbc1805575afffed4e440f1353fcfccbc893081/src/snes/examples/tutorials/ex62.c#lines-1062

The DMKSP is initialized by SNES to have a function which says "get your
matrix from SNES". Without this, the subproblem could
not make the matrix. Now I changed the description of exactly what matrix
it is making and stored that in the subDM.


> What is the right way to think about fixing this? I think that if the DM
> can create a field decomposition, it should also be able to transfer user
> callbacks from the parent to the subdms.
>
> That way, the DM (IOW me) can control how that happens.
>

Yes, that is what I thought I was doing above. Preserving the callbacks
that had been set in the original DM.
Maybe clarify what you think should happen.


> Or do I have the way this is supposed to work all wrong. Is the
> ComputeOperators callback I set meant to be completely agnostic, and it
> should get all the problem-specific information from the DM?
>

I am doing this, but it does not have to be the only way things work. What
I do not understand is how this change breaks your stuff.
If you were setting the DMKSP compute callback, it should just override
what I am doing above. If not, why does me setting it here
screw it up?

  Thanks,

    Matt


> Cheers,
>
> Lawrence



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20190221/7ddc0207/attachment.html>