<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Sun, Mar 5, 2017 at 6:02 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
> On Mar 5, 2017, at 11:18 AM, Jed Brown <<a href="mailto:jed@jedbrown.org">jed@jedbrown.org</a>> wrote:<br>
><br>
> Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> writes:<br>
><br>
>> I've looked at the code again and I'm afraid that just "adding a second DM" is not trivial. The DMKSP/SNES/TS stuff is all built around a single DM and has stuff like ->dmoriginal in it (do we have two dmoriginal with all its complications?).<br>
>><br>
>> Perhaps the entire KSPSetComputeOperators() model was misguided and<br>
>> I should have stuck with the "you fill up all the needed<br>
>> information for the levels of PCMG" before you do the multigrid<br>
>> setup instead of providing callbacks that can fill it them up on<br>
>> the fly "as needed".<br>
><br>
> How would this work when the PCMG is deeply nested and you want to<br>
> rediscretize on coarse grids?<br>
<br>
So, for example, a PCFIELDSPLIT with a PCMG on one of the fields.<br>
<br>
Good point, but I don't think anyone can actually do this now without having Jed hold our hand and write more obscure hook code. (And any design that requires the knowledge of a single guru is not a good design).<br>
<br>
PCFieldSplitSetDefaults() does have the possibility of calling DMCreateFieldDecomposition() or DMCreateSubDM() but, for example, neither DMCreateSubDM_DA() nor DMCreateFieldDecomposition_DA(<wbr>) know anything about DMKSP so the subDM that would be associated with the PCMG cannot do rediscretize since it won't know what functions to use, in addition the subdomain doesn't get the "field" of the "current solution" that needs to be linearized around. It looks like DMCreateFieldDecomposition() and or DMCreateSubDM() need have hooks added for them, (there are currently subdomain, refine, and coarsen hooks, but none for "fields"). Even with the addition of the hooks there is no API for the user to provide a "linearize on a particular field" to SNES to be passed down through the hooks; so that has to added. What about linearization on a particular field of a particular field for when PCFIELDSPLIT is used inside a PCFIELDSPLIT, etc, how are we going to support that generally?<br>
<br>
Backing up to the case that is supported, PCMG, note that SNESSetUpMatrices() has the code<br>
{<br>
KSP ksp;<br>
ierr = SNESGetKSP(snes,&ksp);CHKERRQ(<wbr>ierr);<br>
ierr = KSPSetComputeOperators(ksp,<wbr>KSPComputeOperators_SNES,snes)<wbr>;CHKERRQ(ierr);<br>
ierr = DMCoarsenHookAdd(snes >dm,DMCoarsenHook_SNESVecSol,<wbr>DMRestrictHook_SNESVecSol,<wbr>snes);CHKERRQ(ierr);<br>
}<br>
<br>
so SNES essentially knows about PCMG already; true it doesn't loop over the levels directly and compute the "current solution", that is handled by the DMRestrictHook_SNESVecSol, but is it really so different? It is in the snes.c file. SNES knows about the PCMG pattern! Are we going to have to "teach" SNES the same kind of thing for the other more complicated situations like a PCFIELDSPLIT with a PCMG, with tons of specific hooks for the different situations?<br>
<br>
I find the use of the hooks and DMGetDMKSPWrite() stuff already confusing and yet we still only do a very simple case with it. How can we manage the complexity once we use it more generally?<br>
<br>
<br>
Perhaps the solution is to deprecate the KSPSetOperators, SNESSetFunction, SNESSetJacobian, TSSetIFunction etc and instead directly use DMKSPSetComputeOperators(); you need to remind me again why you need to have a DMKSP instead of just putting the function pointers into the DM? Would this vastly simplify things?<br></blockquote><div><br></div><div>The DMSNES and friends are the parts that are solver specific, since a given DM can be associated with several solvers.That is why the resolution-dependent information can go there. You could have stuck all that info in the solver I guess, but perhaps Jed has a good reason not to do that (maybe to make it easier to pass around, or maybe extensible to different DMs).</div><div><br></div><div>I am not against deprecation there, but we have to preserve an easy way for people to just feed there crap in, meaning no creating a DMSHELL or other setup.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Perhaps we need to revisit what DM is? Looking at the struct _p_DM is rather scary.<br>
<br>
<br>
<br>
<br>
><br>
>> We could possibly throwout all the coarsen/refine hook stuff and<br>
>> the DMKSP construct.<br>
><br>
> I don't think the coarsen or refine stuff is specific to KSP? How would<br>
> you intend to do coarse levels for FAS as a nonlinear preconditioner?<br>
><br>
>> Essentially we'd only be requiring SNESComputeJacobian to know<br>
>> about the PCMG API to provide the coarser grid operators. The<br>
>> coarsen/refine hook stuff seems to be there only to hide from the<br>
>> SNESComputeJacobian the PCMG API. At lot of complication for what<br>
>> real benefit besides our programming egos?<br>
><br>
> Why just PCMG, not every possible nesting of PCMG inside other PCs?<br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>