DA question
Matthew Knepley
knepley at gmail.com
Mon Sep 11 13:27:05 CDT 2006
On 9/11/06, Matt Funk <mafunk at nmsu.edu> wrote:
>
> Hi,
>
> i had a general question. I have a simple rectangular domain on which i
> evaluate a pde. I am using PETSCs DA object. To set the right hand side of
> the linear system i need to evaluate the laplacian before solving the
> system.
> To do that i need to access the ghostnodes.
>
> So, i was wondering if the standard procedure is to access the (global)
> vectors i need is as follows:
>
> DACreateLocalVector(...) //create temporary (local) vector with ghostnodes
> DAGlobalToLocalBegin
> DAGlobalToLocalEnd //map global to local vector
> DAVecGetArray //get work array
>
> ... do my thing ...
>
> DAVecRestoreArray
> DALocalToGlobalBegin
> DALocalToGlobalEnd
> VecDestroy //destroy temporary local vector
>
> or if there is another (better?!) way ...
You probably only need DALocalToGlobal() which discards the ghosts.
I guess what bothers me (and i don't know if it should) is that i need to
> create the local temporary vectors every time step. What i mean is that
> the
> global vector already resides distributed on the different procs, so does
> DACreateLocalVector(...) allocate all that memory again or does it just
> allocate memory for the ghostnodes?
Well, you should use DAGetLocalVector(). It does duplicate the local
storage,
but if you want the nice data layout, this is necessary. Unless you are
running
an explicit solver with extreme memory limitations, I wouldn't worry.
Matt
thanks
> mat
>
>
> On Sunday 10 September 2006 17:50, Barry Smith wrote:
> > Jens,
> >
> > You are correct; functionality to do this does not exist in DMMG. So
> > you have to write it yourself as you have, let us know if you have any
> > trouble with it.
> >
> > Barry
> >
> > On Sun, 10 Sep 2006, jens.madsen at risoe.dk wrote:
> > > Hi again
> > >
> > > My linear problem Ax = b. In my case the matrix A is timepdependent.
> In
> > > my KSP code I calculate A and call KSPSetOperators() in each timestep.
> > > When using DMMG I have not been able to find such a functionallity ? I
> am
> > > using galerkin matrices on all coarser MG levels. Here is the code
> that I
> > > am currently using
> > >
> > > /*Matrix is time dependent*/
> > > if(para.PolEq==GLOBAL)
> > > {
> > > ierr =
> > > MGComputePolaMatrix(*dmmg,dmmg[para.MG_levels-1]->J,dmmg[
> para.MG_levels-1
> > >]->B); /*ierr =
> > > MatView(dmmg[DMMGGetLevels(dmmg)-1]->B,PETSC_VIEWER_STDOUT_WORLD);*/
> for
> > > (i=DMMGGetLevels(dmmg)-2; i>-1; i--)
> > > {
> > > if (dmmg[i]->galerkin)
> > > {
> > >
> > >
> MatPtAP(dmmg[i+1]->B,dmmg[i+1]->R,MAT_REUSE_MATRIX,1.0,&dmmg[i]->B);
> > > if (!dmmg[i]->J)
> > > {
> > > dmmg[i]->J = dmmg[i]->B;
> > > }
> > > }
> > > ierr =
> > >
> KSPSetOperators(dmmg[i]->ksp,dmmg[i]->B,dmmg[i]->B,SAME_NONZERO_PATTERN);
> > >CHKERRQ(ierr); }
> > > }
> > > else
> > > {
> > > ierr =
> > >
> KSPSetOperators(dmmg[i]->ksp,dmmg[i]->B,dmmg[i]->B,SAME_PRECONDITIONER);C
> > >HKERRQ(ierr); }
> > > ierr = DMMGSolve(dmmg);CHKERRQ(ierr);
> > >
> > > I this the way to do it or have missed some functionallity in DMMG ?
> > >
> > > Cheers Jens
> > >
> > > -----Original Message-----
> > > From: Barry Smith [mailto:bsmith at mcs.anl.gov]
> > > Sent: Thu 8/31/2006 10:12 PM
> > > To: jens.madsen at risoe.dk
> > > Subject: Re: SV: SV: SV: [PETSC #14613] DMMG question; more stupid
> danish
> > > questions :-)
> > >
> > >
> > > Yes, one of the arguments to DACreate is which directions you want
> > > periodicity.
> > >
> > > Barry
> > >
> > > On Thu, 31 Aug 2006, jens.madsen at risoe.dk wrote:
> > >> Ok I had not figured that out. Think I misunderstood a response from
> you
> > >> that you mailed in the winter :-)
> > >>
> > >> I am now trying to implement DMMG in my code. Just one question: is
> DMMG
> > >> able to handle periodic boundary conditions ? I think I have made it
> > >> work but I have not performed intensive testing ....
> > >>
> > >> thx Jens :-)
> > >>
> > >> ________________________________
> > >>
> > >> Fra: Barry Smith [mailto:bsmith at mcs.anl.gov]
> > >> Sendt: on 23-08-2006 00:06
> > >> Til: jens.madsen at risoe.dk
> > >> Cc: petsc-maint at mcs.anl.gov
> > >> Emne: Re: SV: SV: [PETSC #14613] DMMG question
> > >>
> > >>
> > >>
> > >>
> > >> Actually DMMG does not always start at the coarsest level.
> > >> It will only do that if you pass in the -dmmg_grid_sequence option.
> > >> You can run with -pc_mg_type multiplicative -pc_mg_cycles 1 for
> > >> V cycle and -pc_mg_cycles 2 for W cycles.
> > >>
> > >> There may be other reasons you might not be able to use DMMG like
> > >> it only works on logically rectangular grids, etc.
> > >>
> > >> The only example for the mg directly is
> > >> src/ksp/ksp/examples/tests/ex19.c it is terribly ugly.
> > >>
> > >>
> > >> Barry
> > >>
> > >> It is a different ex19.c
> > >>
> > >> On Tue, 22 Aug 2006 jens.madsen at risoe.dk wrote:
> > >>> Hi again
> > >>>
> > >>> As far as I can see petsc/src/snes/examples/tutorials/ex19.c uses
> DMMG
> > >>> ?
> > >>>
> > >>> I am trying to use the preconditioner itself. I do not want to use
> DMMG
> > >>> because DMMG always starts at the coarsest grid in order to improve
> the
> > >>> initial guess. I want to start on the finest grid in each timestep
> and
> > >>> do a V or W...
> > >>>
> > >>>
> > >>>
> > >>> Kind regards
> > >>>
> > >>>
> > >>>
> > >>> JEns
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> ________________________________
> > >>>
> > >>> Fra: Hong Zhang [mailto:petsc-maint at mcs.anl.gov]
> > >>> Sendt: lø 19-08-2006 03:04
> > >>> Til: jens.madsen at risoe.dk
> > >>> Cc: petsc-maint at mcs.anl.gov
> > >>> Emne: RE: SV: [PETSC #14613] DMMG question
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> See
> > >>> ~petsc/src/snes/examples/tutorials/ex19.c
> > >>>
> > >>> Hong
> > >>>
> > >>> On Fri, 18 Aug 2006 jens.madsen at risoe.dk wrote:
> > >>>> Hi again
> > >>>>
> > >>>> First I will thank you guys for developing such a great product.
> Since
> > >>>> i wrote the mails below I have developed a code simulating 2D
> > >>>> plasmaphysics. At the moment I am using the Krylov subspace
> methods,
> > >>>> KSP. Now that this is runnning I would like to see which
> improvements
> > >>>> a MG precoditioner would give. Before starting doing so I would ask
> > >>>> you guys whether you have any examples ? The documentation on the
> PCMG
> > >>>> is a bit sparse (:-))
> > >>>>
> > >>>> cheers Jens
> > >>>>
> > >>>> PS. I will not forget to site you @:-)
> > >>>>
> > >>>>
> > >>>> -----Original Message-----
> > >>>> From: Barry Smith [mailto:petsc-maint at mcs.anl.gov]
> > >>>> Sent: Thu 3/30/2006 2:35 AM
> > >>>> To: jens.madsen at risoe.dk
> > >>>> Cc: petsc-maint at mcs.anl.gov
> > >>>> Subject: Re: SV: [PETSC #14613] DMMG question
> > >>>>
> > >>>> On Wed, 29 Mar 2006, jens.madsen at risoe.dk wrote:
> > >>>>> Hi Barry and Matt
> > >>>>>
> > >>>>> Thank you for you quick response :-)
> > >>>>>
> > >>>>> I have actually tried those command line options that you suggest.
> It
> > >>>>> is probably me not using the right terminology; I am not that
> > >>>>> experienced. I try to rephrase my problem.
> > >>>>>
> > >>>>> As far as I understand the full multigrid always starts on the
> > >>>>> coarsest grid G_0. On the coarsest grid you calculate an
> approximate
> > >>>>> solution v_0 (using jacobi, GaussSeiddel, Krylov etc). This
> > >>>>> approximate solution is interpolated to a fine grid and used as an
> > >>>>> initial guess on this finer grid G_1. Now you make a few iterative
> > >>>>> sweeps on G_1 smoothening out the high k modes and get v_1.
> Restrict
> > >>>>> this approximate solution to G_0. Relax on G_0. Correct the
> solution
> > >>>>> v_1 and use this as an initial guess on an even finer grid G_2
> etc.
> > >>>>> etc. In other words it combines "nested iteration" and "coarse
> grid
> > >>>>> correction".
> > >>>>
> > >>>> You are correct this is exactly traditional full multigrid. The
> > >>>> "PETSc full multigrid" is slightly different. We start with a right
> > >>>> hand side (and initial guess) on the finest grid, restrict the
> > >>>> residual to the coarsest grid and then start up the grids with
> nested
> > >>>> iteration AND coarse grid correction. Thus unlike "traditional"
> full
> > >>>> multigrid you only need to define your problem on the finest grid.
> > >>>>
> > >>>> We've found that this usually works better than just using regular
> V
> > >>>> or W cycles. (BTW: PETSc full multigrid can, of course, use either
> V
> > >>>> or W cycles.
> > >>>>
> > >>>>> My "algorithm" is as follows:
> > >>>>>
> > >>>>> 1) apply initial conditions to Density,n, and Temperature,T.
> > >>>>> 2) find \phi solving a Poisson like equation using a multigrid
> > >>>>> scheme. Use \phi from previous timestep as an initial guess. n and
> T
> > >>>>> are variables in this equation. 3) Step n and T forward in time
> using
> > >>>>> the "stiffly stable" time stepping scheme. This is to be done
> using a
> > >>>>> pre-LU-factorized matrix. 4) goto 2)
> > >>>>>
> > >>>>> I would like to make a V (or W) cycle starting on the finest grid
> > >>>>> instead. On the finest grid I would like to apply boundary
> > >>>>> conditions, provide the previous time step as an initial guess and
> > >>>>> use "coarse grid correction" only. My own code is (probably) full
> of
> > >>>>> errors :-) so I have tried running the multigrid examples under
> KSP
> > >>>>> and SNES with -pc_type richardson and/or, -pc_mg_type additive
> and/or
> > >>>>> -ksp_type preonly etc.What I can see using -ksp/snes_view is that
> the
> > >>>>> starting point is always the coarsest grid with dimensions given
> by
> > >>>>> DACreateNd ? The next MG-Grids are always finer. Can I make DMMG
> > >>>>> start on the finest grid with dimensions given in DACreateNd ?
> > >>>>
> > >>>> You need to create a DA with a coarser size so that after it is
> > >>>> refined the number of times it gives you the grid you want. That is
> if
> > >>>> you want 5 grid points with two levels you would create a DA with 3
> > >>>> grid points and pass that into the DMMG. Sorry there is no way to
> > >>>> start with a DA on the finest (but you get the same effect by
> starting
> > >>>> with a coarser DA.
> > >>>>
> > >>>> Barry
> > >>>>
> > >>>>> Cheers and thanks Jens
> > >>>>>
> > >>>>> ________________________________
> > >>>>>
> > >>>>> Fra: Barry Smith [mailto:petsc-maint at mcs.anl.gov]
> > >>>>> Sendt: on 29-03-2006 17:20
> > >>>>> Til: jens.madsen at risoe.dk
> > >>>>> Cc: petsc-maint at mcs.anl.gov
> > >>>>> Emne: Re: [PETSC #14613] DMMG question
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> Jens,
> > >>>>>
> > >>>>> YOu can access any of the low level PCMG options from
> > >>>>> DMMG. For example, -pc_type richardson gives you "multigrid
> > >>>>> as a solve". -pc_mg_type additive gives you additive
> > >>>>> -pc_mg_type multiplicative gives you standard v or w cycel
> > >>>>> -pc_mg_type full gives "full" multigrid". -pg_mg_cycles 2
> > >>>>> gives W cycle. -pc_mg_levels_pc_type sor gives SOR as the smoother
> > >>>>> etc etc etc. Run with -help to see all the choices for the parts
> > >>>>> of the multigrid process.
> > >>>>>
> > >>>>> Barry
> > >>>>>
> > >>>>> As Matt noted, using the PCMG directly requires YOU provide
> > >>>>> a mesh infrastructure that manages the meshes, thus it is not
> > >>>>> realistic for us to provide this whole infrastructure in an
> > >>>>> example. We currently only provide the full infrastructure
> > >>>>> for structured grids (in DMMG).
> > >>>>>
> > >>>>> On Wed, 29 Mar 2006, jens.madsen at risoe.dk wrote:
> > >>>>>> Hi Petsc :-)
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> I am trying to use Petsc for solving plasma fluid equations. Is
> it
> > >>>>>> possible to use the DMMG with multiplicative or additive
> multigrid
> > >>>>>> schemes ? Also can I use multigrid as a solver not only as a
> > >>>>>> preconditioner in DMMG ?
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Also I cannot find any examples using the "low-level" PCMG
> > >>>>>> multigrid interface ? Only a testprogram in the PC/TEST directory
> ?
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Cheers Jens Madsen
>
>
--
"Failure has a thousand explanations. Success doesn't need one" -- Sir Alec
Guiness
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20060911/b8f7a62d/attachment.htm>
More information about the petsc-users
mailing list