[petsc-dev] Saving monitor information

[Barry Smith]

  Yucko. I hate the added complexity of two types of monitors. Since the DM can always be obtained from the solver why can't the current API be good enough. Just in the first line of the monitor you do KSP/SNES/TSGetDM() and now you have access to the DM and can do anything you want. Seems terrible to add to the API just to avoid that single call?

  Barry

> On Jan 7, 2015, at 2:04 PM, Matthew Knepley <knepley at gmail.com> wrote:
> 
> On Tue, Jan 6, 2015 at 3:18 PM, Matthew Knepley <knepley at gmail.com> wrote:
> On Tue, Jan 6, 2015 at 3:11 PM, Lukasz Kaczmarczyk <Lukasz.Kaczmarczyk at glasgow.ac.uk> wrote:
> Thanks for swift and in depth response.
> 
> I agree monitors, in particular for KSP, should be set with the solver. KSPMonitor is more solver not problem dependent. Moreover building hierarchy of DMs in general users will pick appropriate linear solver for each level and then will attach monitor. At this point all information is need. Trying to make it more general will make everything look more complex without obvious advantage.
> 
> I am not sure, but in my opinion SNES and TS are different case in this context.  For SNES/TS user will more likely to monitor residual and some other physical quantities, f.e. load factor, energy, ect.. What to quantities monitor it more depends on physics of the problem rather than on type of SNES/TS solver. It will be more convenient to set monitors for SNES/TS with DM, together with RHS and Jacobins, since user don't have to know in advance what line searcher or time integration scheme will be used.
> 
> In my case, user add fields with appropriate approximation spaces (l2,h1,hdiv,hcurl) which are span on the mesh. In addition elements are defined, i.e. bilinear form and linear form b(u,v) = f(v). Definition of approximation number and type of field, and number of finite elements defines algebraic problem. This is enough for discreet problem to attach dofs to mesh entities (vertices, edges, faces, volumes), number them and build adjacency matrix.  In my case approx. spaces are hierarchical which allow to build composition of problems.
> 
> User is responsible to implement particular physical problem,  i.e. evaluate values at integration points of finite element. Usually at that time he knows already what is needed to be monitored in SNES and TS. The complexities related dofs numeration, problem partitioning or solver problem  ect. are managed by my library, mesh database and petsc.  So main user interface is DM  in which complexities related to discretisation, mesh and algebra are linked.
> 
> Taking that I like idea of DM it wraps three components, Mesh<->Approximation<->Agebra. User at the end interested in physical problem and he implements equation to evaluate and to calculate jacobian and matrices . Since what he monitor in SMES is more very likely physical this should be controlled in DM.
> 
> I see your point. Jed introduced similar "physics" monitors in TS ex11. Maybe we should differentiate between these kinds of
> monitors, which are really about the physics, and solver monitors. The physics monitors should probably be executed in the
> step hooks for SNES or TS. I will think about it and make a proposal.
> 
> How about we introduce another set of monitors, DM monitors,
> 
>   PetscErrorCode myMonitor(DM dm, Vec u, PetscObject solver, void *monCtx)
> 
> which is called by a solver when that DM is attached. We can have policy flags that
> determine what solvers are active for the DM. Of course, there are still some problems
> with exactly what vector comes in for different solvers, and its ugly to have PetscObject
> there, but I could not think of a better way to do it, and I think its functional. We could
> leave out the solver, but I think its the first thing people would ask for, and have to shove
> in their context.
> 
>    Matt
>  
>   Matt
>  
> Kind regards,
> Lukasz
> 
> 
> 
> 
> 
> > On 6 Jan 2015, at 18:52, Barry Smith <bsmith at mcs.anl.gov> wrote:
> >
> >
> >   In the simple case where one has access to the solver object (say a KSP, could be SNES or TS) one can simply write two routines MySetMonitors() and MyClearMonitors() that set (or clear) one or more monitors onto the solver and the user calls   MySetMonitor(ksp) and MyClearMonitor() when they want some monitoring.
> >
> >   Since you know this, I assume your question is really about the more difficult case where you have a collection of solver objects inside a PCMG/PCFIELDSPLIT/... composition and want to be able to trigger monitoring of some KSP solvers inside but you don't have straightforward access to the individual KSPs? If the PCMG/PCFIELDSPLIT... composition was constructed from a "base" DM and sub-DMs and/or refine/coarsen DMs that are obtained from that base DM then you are suggesting attaching the monitor information (somehow) to the base DM, it gets propagated to the other DMs and then to each appropriate KSP? Now we see that your suggestion actually has two distinct parts
> >
> > 1) ability to attach monitors to a DM and then have the KSP/SNES/TS grab them from the DM
> >
> > 2) ability to attach "stuff" to a DM with enough additional information (logic) so that the "parts of" that "stuff" gets conveyed to subDMs and/or refine/coarsen DMs when (if) they are created, recursively. For example
> >
> >   DMSetStuff(DM,stuff,"if DM is refined give stuff to newly created DM")
> >   DMSetStuff(DM,stuff,"if DM is refined give stuff to newly created DM and then if newly created DM is split then give stuff to the newly created split DM")
> >   etc etc
> >
> > My thoughts.
> >
> >  1)You are correct we don't have a good way programatic way of getting stuff into the inner KSPs in nested solvers. This is made difficult by the facts that a) different decompositions can happen based on runtime options etc. b) one doesn't know the actual decomposition in advance, c) the "tree" of solvers created happens "lazily" and it is indeterminant when all the solvers in the tree are actually all created and then setup and then used.
> >
> >  2) You are correct that in theory (though not yet in practice) one can provide a "base" DM that then gets sliced and diced and refined and coarsened to create a tree of DMs that corresponds to the tree of solvers with one DM for each solver.
> >
> >  An alternative to what you suggest of attaching the stuff and logic for passing it down through the DMs would be to attach the stuff to solver (KSP/SNES/TS) and have the stuff distributed to the newly created sub solvers. For example:
> >
> >   KSP/PCSetStuff(KSP,stuff,"if PC is fieldsplit give stuff to first/second/whatever split created KSP")
> >   KSP/PCSetStuff(KSP,stuff,"if PC is fieldsplit give stuff to first/second/whatever split created KSP if resulting PC is MG give stuff to levels KSP/PC created")
> >   etc etc
> >
> >  My quick opinion: For something like monitors I like the idea of attaching to the solvers not the DMs since the DMs don't know or care about monitors. One could also do specialized convergence tests etc the same way. Note that I tried to do this function evaluations etc but Jed added the horrible horrible DMKSP crapola to store the function evaluation stuff as it was passed down, I never groked completely why using the KSP directly was impossible.
> >
> >  Eventually we need to figure out programatic ways of interacting with the "tree" of solvers in composed solvers; by the command line and prefixes we can do a fairly decent job with the options database. If we allow putting "stuff" and functions in to the options database we could use the options database also as the "programatic" way of interacting with the "tree" of solvers" but ....  One could also imagine using XML or JSON etc as the way of getting info into the various levels of the tree of solvers; I believe Trilinos tries to do this (hence a good reason not to waste time on that approach).
> >
> > Barry
> >
> >
> >
> >
> >
> >> On Jan 6, 2015, at 12:10 PM, Matthew Knepley <knepley at gmail.com> wrote:
> >>
> >> Should we have a mechanism to save and restore monitor information, so that monitors
> >> can be dynamically set when solvers are associated with other objects?
> >>
> >> For example, you could associate a monitor with a DM, and then if you call
> >>
> >>  SolverSetDM()
> >>
> >> the monitors would be set into the solver. They would also have to be removed
> >> if SetDM() was called again.
> >>
> >> Is this desirable?
> >>
> >>   Matt
> >>
> >> --
> >> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> >> -- Norbert Wiener
> >
> 
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener