[petsc-dev] DMMG without libpetscsnes?

[Barry Smith]

On Jan 7, 2010, at 2:05 PM, Jed Brown wrote:

> On Thu, 7 Jan 2010 08:34:40 -0600, Barry Smith <bsmith at mcs.anl.gov>  
> wrote:
>>   I would not like to see them split up again. Rather I would like to
>> see the linear part of DMMG put into KSP/PC/DMMG somehow (not exactly
>> sure how) so it can be properly nested and composed just like other
>> preconditioners. I'm sure you are doing something slightly cumbersome
>> now to put your DMMG inside the Schur fieldsplit.
>
> Yes, and I don't like it, but it's acceptable because this thing is an
> application instead of a library (the required functionality is
> sufficiently well-defined that it doesn't need to be nicely  
> composable).
>
>>    Here is how I see it: DMMG's job is to "fill up" the appropriate
>> fields in the KSP, PC, MG objects, then when solving those fields are
>> used. I would like (somehow) the KSP, PC, MG objects to have the
>> mechanism that you can provide a DM to them and they can then "fill
>> up" their fields. But it is important we get this right and
>> compossible so I have hesitated to ever try it. The starting point
>> might be having an (optional) KSPSetDM() that would propagate the DM
>> into the PC and subPCs and subsubPCs etc where they would be used to
>> fill the slots if provided.
>
> So it seems to me that the real difficulty is in mesh sequencing  
> (not a
> big deal for linear problems), else PCMGSetDM seems like the natural
> thing.

    Yes we would lose that functionality. But maybe ok since the  
Cascadic multigrid is pretty much mesh sequencing.
We could avoid supporting adaptive mesh refinement for linear problems  
(linear problems are silly anyways :-)).


>
> Looking at the DMMG code more, I see that DMMGSetKSP has
>
>      ierr = KSPSetFromOptions(dmmg[i]->ksp);CHKERRQ(ierr);
>
>      /*  if the multigrid is being run with Galerkin then these  
> matrices do not need to be created except on the finest level
>          we do not take advantage of this because it may be that  
> Galerkin has not yet been selected for the KSP object
>          These are also used if grid sequencing is selected for the  
> linear problem. We should probably turn off grid sequencing
>          for the linear problem */
>
> but it is used later in the same function (before user code has a  
> chance
> to get in with PCMGSetGalerkin)
>
>  if (func) {
>    if (galerkin) {
>      ierr = (*func)(dmmg[nlevels-1],dmmg[nlevels-1]- 
> >J,dmmg[nlevels-1]->B);CHKERRQ(ierr);
>    } else {
>      for (i=0; i<nlevels; i++) {
>        ierr = (*func)(dmmg[i],dmmg[i]->J,dmmg[i]->B);CHKERRQ(ierr);
>      }
>    }
>  }
>
> so if the user does PCMGSetGalerkin instead of -pc_mg_galerkin, the
> series of matrices will be assembled twice as separate matrices (and  
> gdb
> tells me that references to both sets of matrices are retained for the
> life of the DMMG, which could be a lot of memory).

    Yes this is a known problem (at least known by me :-) that will  
need to be resolved with the next approach.

    Barry

>
>
> If we were to go with PCMGSetDM, I think the construction of coarse
> operators should to be postponed until PCSetUp (but the KSPs can be
> created early so they can be played with).
>
> Jed