[petsc-dev] Saving monitor information

[Matthew Knepley]

On Wed, Jan 7, 2015 at 2:27 PM, Lukasz Kaczmarczyk <
Lukasz.Kaczmarczyk at glasgow.ac.uk> wrote:

>
> > On 7 Jan 2015, at 20:04, Matthew Knepley <knepley at gmail.com> wrote:
> >
> > On Tue, Jan 6, 2015 at 3:18 PM, Matthew Knepley <knepley at gmail.com>
> wrote:
> > On Tue, Jan 6, 2015 at 3:11 PM, Lukasz Kaczmarczyk <
> Lukasz.Kaczmarczyk at glasgow.ac.uk> wrote:
> > Thanks for swift and in depth response.
> >
> > I agree monitors, in particular for KSP, should be set with the solver.
> KSPMonitor is more solver not problem dependent. Moreover building
> hierarchy of DMs in general users will pick appropriate linear solver for
> each level and then will attach monitor. At this point all information is
> need. Trying to make it more general will make everything look more complex
> without obvious advantage.
> >
> > I am not sure, but in my opinion SNES and TS are different case in this
> context.  For SNES/TS user will more likely to monitor residual and some
> other physical quantities, f.e. load factor, energy, ect.. What to
> quantities monitor it more depends on physics of the problem rather than on
> type of SNES/TS solver. It will be more convenient to set monitors for
> SNES/TS with DM, together with RHS and Jacobins, since user don't have to
> know in advance what line searcher or time integration scheme will be used.
> >
> > In my case, user add fields with appropriate approximation spaces
> (l2,h1,hdiv,hcurl) which are span on the mesh. In addition elements are
> defined, i.e. bilinear form and linear form b(u,v) = f(v). Definition of
> approximation number and type of field, and number of finite elements
> defines algebraic problem. This is enough for discreet problem to attach
> dofs to mesh entities (vertices, edges, faces, volumes), number them and
> build adjacency matrix.  In my case approx. spaces are hierarchical which
> allow to build composition of problems.
> >
> > User is responsible to implement particular physical problem,  i.e.
> evaluate values at integration points of finite element. Usually at that
> time he knows already what is needed to be monitored in SNES and TS. The
> complexities related dofs numeration, problem partitioning or solver
> problem  ect. are managed by my library, mesh database and petsc.  So main
> user interface is DM  in which complexities related to discretisation, mesh
> and algebra are linked.
> >
> > Taking that I like idea of DM it wraps three components,
> Mesh<->Approximation<->Agebra. User at the end interested in physical
> problem and he implements equation to evaluate and to calculate jacobian
> and matrices . Since what he monitor in SMES is more very likely physical
> this should be controlled in DM.
> >
> > I see your point. Jed introduced similar "physics" monitors in TS ex11.
> Maybe we should differentiate between these kinds of
> > monitors, which are really about the physics, and solver monitors. The
> physics monitors should probably be executed in the
> > step hooks for SNES or TS. I will think about it and make a proposal.
> >
> > How about we introduce another set of monitors, DM monitors,
> >
> >   PetscErrorCode myMonitor(DM dm, Vec u, PetscObject solver, void
> *monCtx)
> >
> > which is called by a solver when that DM is attached. We can have policy
> flags that
> > determine what solvers are active for the DM. Of course, there are still
> some problems
> > with exactly what vector comes in for different solvers, and its ugly to
> have PetscObject
> > there, but I could not think of a better way to do it, and I think its
> functional. We could
> > leave out the solver, but I think its the first thing people would ask
> for, and have to shove
> > in their context.
>
> Since this will be monitor for DM, and will be only about physical
> problem, it should be without passing solver object,
>
> PetscErrorCode mySNESMonitor(DM dm, Vec u,  void *monCtx)
> PetscErrorCode myTSMonitor(DM dm, Vec u,Vec u_t,  void *monCtx)
>
> I was thinking that monitor will be in addition to solver monitor. It will
> be called form solver after solver monitor, not as a replacement of it.
>

It will not replace it, but I think people would want some metadata, like
iteration number, for their output.


> No need for policy flag to determine solver. Once physics is defined and
> problem is quasi-static or dynamic/time dependent, user defining DM already
> know which monitor should be set, for SNES or DM. I imagine that it will
> make this simpler to implement in petsc and easier for users.
>

I do not agree. For example, if I have a monitor which checks a
conservation law, I may want to enable in
an inner SNES or KSP loop to see where conservation is being destroyed.

   Matt


> I note that this discussion will apply as well for convergence criteria.
> This need to be resolved as well in the future.
>
> Lukasz
>
> >
> > > On 6 Jan 2015, at 18:52, Barry Smith <bsmith at mcs.anl.gov> wrote:
> > >
> > >
> > >   In the simple case where one has access to the solver object (say a
> KSP, could be SNES or TS) one can simply write two routines MySetMonitors()
> and MyClearMonitors() that set (or clear) one or more monitors onto the
> solver and the user calls   MySetMonitor(ksp) and MyClearMonitor() when
> they want some monitoring.
> > >
> > >   Since you know this, I assume your question is really about the more
> difficult case where you have a collection of solver objects inside a
> PCMG/PCFIELDSPLIT/... composition and want to be able to trigger monitoring
> of some KSP solvers inside but you don't have straightforward access to the
> individual KSPs? If the PCMG/PCFIELDSPLIT... composition was constructed
> from a "base" DM and sub-DMs and/or refine/coarsen DMs that are obtained
> from that base DM then you are suggesting attaching the monitor information
> (somehow) to the base DM, it gets propagated to the other DMs and then to
> each appropriate KSP? Now we see that your suggestion actually has two
> distinct parts
> > >
> > > 1) ability to attach monitors to a DM and then have the KSP/SNES/TS
> grab them from the DM
> > >
> > > 2) ability to attach "stuff" to a DM with enough additional
> information (logic) so that the "parts of" that "stuff" gets conveyed to
> subDMs and/or refine/coarsen DMs when (if) they are created, recursively.
> For example
> > >
> > >   DMSetStuff(DM,stuff,"if DM is refined give stuff to newly created
> DM")
> > >   DMSetStuff(DM,stuff,"if DM is refined give stuff to newly created DM
> and then if newly created DM is split then give stuff to the newly created
> split DM")
> > >   etc etc
> > >
> > > My thoughts.
> > >
> > >  1)You are correct we don't have a good way programatic way of getting
> stuff into the inner KSPs in nested solvers. This is made difficult by the
> facts that a) different decompositions can happen based on runtime options
> etc. b) one doesn't know the actual decomposition in advance, c) the "tree"
> of solvers created happens "lazily" and it is indeterminant when all the
> solvers in the tree are actually all created and then setup and then used.
> > >
> > >  2) You are correct that in theory (though not yet in practice) one
> can provide a "base" DM that then gets sliced and diced and refined and
> coarsened to create a tree of DMs that corresponds to the tree of solvers
> with one DM for each solver.
> > >
> > >  An alternative to what you suggest of attaching the stuff and logic
> for passing it down through the DMs would be to attach the stuff to solver
> (KSP/SNES/TS) and have the stuff distributed to the newly created sub
> solvers. For example:
> > >
> > >   KSP/PCSetStuff(KSP,stuff,"if PC is fieldsplit give stuff to
> first/second/whatever split created KSP")
> > >   KSP/PCSetStuff(KSP,stuff,"if PC is fieldsplit give stuff to
> first/second/whatever split created KSP if resulting PC is MG give stuff to
> levels KSP/PC created")
> > >   etc etc
> > >
> > >  My quick opinion: For something like monitors I like the idea of
> attaching to the solvers not the DMs since the DMs don't know or care about
> monitors. One could also do specialized convergence tests etc the same way.
> Note that I tried to do this function evaluations etc but Jed added the
> horrible horrible DMKSP crapola to store the function evaluation stuff as
> it was passed down, I never groked completely why using the KSP directly
> was impossible.
> > >
> > >  Eventually we need to figure out programatic ways of interacting with
> the "tree" of solvers in composed solvers; by the command line and prefixes
> we can do a fairly decent job with the options database. If we allow
> putting "stuff" and functions in to the options database we could use the
> options database also as the "programatic" way of interacting with the
> "tree" of solvers" but ....  One could also imagine using XML or JSON etc
> as the way of getting info into the various levels of the tree of solvers;
> I believe Trilinos tries to do this (hence a good reason not to waste time
> on that approach).
> > >
> > > Barry
> > >
> > >
> > >
> > >
> > >
> > >> On Jan 6, 2015, at 12:10 PM, Matthew Knepley <knepley at gmail.com>
> wrote:
> > >>
> > >> Should we have a mechanism to save and restore monitor information,
> so that monitors
> > >> can be dynamically set when solvers are associated with other objects?
> > >>
> > >> For example, you could associate a monitor with a DM, and then if you
> call
> > >>
> > >>  SolverSetDM()
> > >>
> > >> the monitors would be set into the solver. They would also have to be
> removed
> > >> if SetDM() was called again.
> > >>
> > >> Is this desirable?
> > >>
> > >>   Matt
> > >>
> > >> --
> > >> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> > >> -- Norbert Wiener
> > >
> >
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> > -- Norbert Wiener
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> > -- Norbert Wiener
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20150107/c8cbdff8/attachment.html>