On 9/11/06, <b class="gmail_sendername">Matt Funk</b> <<a href="mailto:mafunk@nmsu.edu">mafunk@nmsu.edu</a>> wrote:<div><span class="gmail_quote"></span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi,<br><br>i had a general question. I have a simple rectangular domain on which i<br>evaluate a pde. I am using PETSCs DA object. To set the right hand side of<br>the linear system i need to evaluate the laplacian before solving the system.
<br>To do that i need to access the ghostnodes.<br><br>So, i was wondering if the standard procedure is to access the (global)<br>vectors i need is as follows:<br><br>DACreateLocalVector(...) //create temporary (local) vector with ghostnodes
<br>DAGlobalToLocalBegin<br>DAGlobalToLocalEnd //map global to local vector<br>DAVecGetArray //get work array<br><br>... do my thing ...<br><br>DAVecRestoreArray<br>DALocalToGlobalBegin<br>DALocalToGlobalEnd<br>VecDestroy //destroy temporary local vector
<br><br>or if there is another (better?!) way ...</blockquote><div><br>
You probably only need DALocalToGlobal() which discards the ghosts. <br>
</div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">I guess what bothers me (and i don't know if it should) is that i need to<br>create the local temporary vectors every time step. What i mean is that the
<br>global vector already resides distributed on the different procs, so does<br>DACreateLocalVector(...) allocate all that memory again or does it just<br>allocate memory for the ghostnodes?</blockquote><div><br>
Well, you should use DAGetLocalVector(). It does duplicate the local storage,<br>
but if you want the nice data layout, this is necessary. Unless you are running<br>
an explicit solver with extreme memory limitations, I wouldn't worry.<br>
<br>
Matt <br>
</div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">thanks<br>mat<br><br><br>On Sunday 10 September 2006 17:50, Barry Smith wrote:<br>
> Jens,<br>><br>> You are correct; functionality to do this does not exist in DMMG. So<br>> you have to write it yourself as you have, let us know if you have any<br>> trouble with it.<br>><br>> Barry
<br>><br>> On Sun, 10 Sep 2006, <a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk</a> wrote:<br>> > Hi again<br>> ><br>> > My linear problem Ax = b. In my case the matrix A is timepdependent. In
<br>> > my KSP code I calculate A and call KSPSetOperators() in each timestep.<br>> > When using DMMG I have not been able to find such a functionallity ? I am<br>> > using galerkin matrices on all coarser MG levels. Here is the code that I
<br>> > am currently using<br>> ><br>> > /*Matrix is time dependent*/<br>>
>
if(para.PolEq==GLOBAL)<br>>
>
{<br>>
>
ierr =<br>> > MGComputePolaMatrix(*dmmg,dmmg[para.MG_levels-1]->J,dmmg[para.MG_levels-1<br>> >]->B); /*ierr =<br>> > MatView(dmmg[DMMGGetLevels(dmmg)-1]->B,PETSC_VIEWER_STDOUT_WORLD);*/ for<br>
> > (i=DMMGGetLevels(dmmg)-2; i>-1; i--)<br>>
>
{<br>>
>
if (dmmg[i]->galerkin)<br>>
>
{<br>> ><br>>
>
MatPtAP(dmmg[i+1]->B,dmmg[i+1]->R,MAT_REUSE_MATRIX,1.0,&dmmg[i]->B);<br>> > if (!dmmg[i]->J)<br>>
>
{<br>>
>
dmmg[i]->J = dmmg[i]->B;<br>>
>
}<br>>
>
}<br>>
>
ierr =<br>> > KSPSetOperators(dmmg[i]->ksp,dmmg[i]->B,dmmg[i]->B,SAME_NONZERO_PATTERN);<br>> >CHKERRQ(ierr); }<br>> > }<br>> > else<br>> > {<br>
>
>
ierr =<br>> > KSPSetOperators(dmmg[i]->ksp,dmmg[i]->B,dmmg[i]->B,SAME_PRECONDITIONER);C<br>> >HKERRQ(ierr); }<br>>
>
ierr = DMMGSolve(dmmg);CHKERRQ(ierr);<br>> ><br>> > I this the way to do it or have missed some functionallity in DMMG ?<br>> ><br>> > Cheers Jens<br>> ><br>> > -----Original Message-----
<br>> > From: Barry Smith [mailto:<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>]<br>> > Sent: Thu 8/31/2006 10:12 PM<br>> > To: <a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk</a>
<br>> > Subject: Re: SV: SV: SV: [PETSC #14613] DMMG question; more stupid danish<br>> > questions :-)<br>> ><br>> ><br>> > Yes, one of the arguments to DACreate is which directions you want
<br>> > periodicity.<br>> ><br>> > Barry<br>> ><br>> > On Thu, 31 Aug 2006, <a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk</a> wrote:<br>> >> Ok I had not figured that out. Think I misunderstood a response from you
<br>> >> that you mailed in the winter :-)<br>> >><br>> >> I am now trying to implement DMMG in my code. Just one question: is DMMG<br>> >> able to handle periodic boundary conditions ? I think I have made it
<br>> >> work but I have not performed intensive testing ....<br>> >><br>> >> thx Jens :-)<br>> >><br>> >> ________________________________<br>> >><br>> >> Fra: Barry Smith [mailto:
<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>]<br>> >> Sendt: on 23-08-2006 00:06<br>> >> Til: <a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk</a><br>> >> Cc: <a href="mailto:petsc-maint@mcs.anl.gov">
petsc-maint@mcs.anl.gov</a><br>> >> Emne: Re: SV: SV: [PETSC #14613] DMMG question<br>> >><br>> >><br>> >><br>> >><br>> >> Actually DMMG does not always start at the coarsest level.
<br>> >> It will only do that if you pass in the -dmmg_grid_sequence option.<br>> >> You can run with -pc_mg_type multiplicative -pc_mg_cycles 1 for<br>> >> V cycle and -pc_mg_cycles 2 for W cycles.
<br>> >><br>> >> There may be other reasons you might not be able to use DMMG like<br>> >> it only works on logically rectangular grids, etc.<br>> >><br>> >> The only example for the mg directly is
<br>> >> src/ksp/ksp/examples/tests/ex19.c it is terribly ugly.<br>> >><br>> >><br>> >> Barry<br>> >><br>> >> It is a different ex19.c<br>> >><br>> >> On Tue, 22 Aug 2006
<a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk</a> wrote:<br>> >>> Hi again<br>> >>><br>> >>> As far as I can see petsc/src/snes/examples/tutorials/ex19.c uses DMMG<br>> >>> ?
<br>> >>><br>> >>> I am trying to use the preconditioner itself. I do not want to use DMMG<br>> >>> because DMMG always starts at the coarsest grid in order to improve the<br>> >>> initial guess. I want to start on the finest grid in each timestep and
<br>> >>> do a V or W...<br>> >>><br>> >>><br>> >>><br>> >>> Kind regards<br>> >>><br>> >>><br>> >>><br>> >>> JEns
<br>> >>><br>> >>><br>> >>><br>> >>><br>> >>><br>> >>><br>> >>> ________________________________<br>> >>><br>> >>> Fra: Hong Zhang [mailto:
<a href="mailto:petsc-maint@mcs.anl.gov">petsc-maint@mcs.anl.gov</a>]<br>> >>> Sendt: lø 19-08-2006 03:04<br>> >>> Til: <a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk</a><br>> >>> Cc:
<a href="mailto:petsc-maint@mcs.anl.gov">petsc-maint@mcs.anl.gov</a><br>> >>> Emne: RE: SV: [PETSC #14613] DMMG question<br>> >>><br>> >>><br>> >>><br>> >>><br>> >>> See
<br>> >>> ~petsc/src/snes/examples/tutorials/ex19.c<br>> >>><br>> >>> Hong<br>> >>><br>> >>> On Fri, 18 Aug 2006 <a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk
</a> wrote:<br>> >>>> Hi again<br>> >>>><br>> >>>> First I will thank you guys for developing such a great product. Since<br>> >>>> i wrote the mails below I have developed a code simulating 2D
<br>> >>>> plasmaphysics. At the moment I am using the Krylov subspace methods,<br>> >>>> KSP. Now that this is runnning I would like to see which improvements<br>> >>>> a MG precoditioner would give. Before starting doing so I would ask
<br>> >>>> you guys whether you have any examples ? The documentation on the PCMG<br>> >>>> is a bit sparse (:-))<br>> >>>><br>> >>>> cheers Jens<br>> >>>>
<br>> >>>> PS. I will not forget to site you @:-)<br>> >>>><br>> >>>><br>> >>>> -----Original Message-----<br>> >>>> From: Barry Smith [mailto:<a href="mailto:petsc-maint@mcs.anl.gov">
petsc-maint@mcs.anl.gov</a>]<br>> >>>> Sent: Thu 3/30/2006 2:35 AM<br>> >>>> To: <a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk</a><br>> >>>> Cc: <a href="mailto:petsc-maint@mcs.anl.gov">
petsc-maint@mcs.anl.gov</a><br>> >>>> Subject: Re: SV: [PETSC #14613] DMMG question<br>> >>>><br>> >>>> On Wed, 29 Mar 2006, <a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk
</a> wrote:<br>> >>>>> Hi Barry and Matt<br>> >>>>><br>> >>>>> Thank you for you quick response :-)<br>> >>>>><br>> >>>>> I have actually tried those command line options that you suggest. It
<br>> >>>>> is probably me not using the right terminology; I am not that<br>> >>>>> experienced. I try to rephrase my problem.<br>> >>>>><br>> >>>>> As far as I understand the full multigrid always starts on the
<br>> >>>>> coarsest grid G_0. On the coarsest grid you calculate an approximate<br>> >>>>> solution v_0 (using jacobi, GaussSeiddel, Krylov etc). This<br>> >>>>> approximate solution is interpolated to a fine grid and used as an
<br>> >>>>> initial guess on this finer grid G_1. Now you make a few iterative<br>> >>>>> sweeps on G_1 smoothening out the high k modes and get v_1. Restrict<br>> >>>>> this approximate solution to G_0. Relax on G_0. Correct the solution
<br>> >>>>> v_1 and use this as an initial guess on an even finer grid G_2 etc.<br>> >>>>> etc. In other words it combines "nested iteration" and "coarse grid<br>> >>>>> correction".
<br>> >>>><br>> >>>> You are correct this is exactly traditional full multigrid. The<br>> >>>> "PETSc full multigrid" is slightly different. We start with a right
<br>> >>>> hand side (and initial guess) on the finest grid, restrict the<br>> >>>> residual to the coarsest grid and then start up the grids with nested<br>> >>>> iteration AND coarse grid correction. Thus unlike "traditional" full
<br>> >>>> multigrid you only need to define your problem on the finest grid.<br>> >>>><br>> >>>> We've found that this usually works better than just using regular V<br>> >>>> or W cycles. (BTW: PETSc full multigrid can, of course, use either V
<br>> >>>> or W cycles.<br>> >>>><br>> >>>>> My "algorithm" is as follows:<br>> >>>>><br>> >>>>> 1) apply initial conditions to Density,n, and Temperature,T.
<br>> >>>>> 2) find \phi solving a Poisson like equation using a multigrid<br>> >>>>> scheme. Use \phi from previous timestep as an initial guess. n and T<br>> >>>>> are variables in this equation. 3) Step n and T forward in time using
<br>> >>>>> the "stiffly stable" time stepping scheme. This is to be done using a<br>> >>>>> pre-LU-factorized matrix. 4) goto 2)<br>> >>>>><br>> >>>>> I would like to make a V (or W) cycle starting on the finest grid
<br>> >>>>> instead. On the finest grid I would like to apply boundary<br>> >>>>> conditions, provide the previous time step as an initial guess and<br>> >>>>> use "coarse grid correction" only. My own code is (probably) full of
<br>> >>>>> errors :-) so I have tried running the multigrid examples under KSP<br>> >>>>> and SNES with -pc_type richardson and/or, -pc_mg_type additive and/or<br>> >>>>> -ksp_type preonly
etc.What I can see using -ksp/snes_view is that the<br>> >>>>> starting point is always the coarsest grid with dimensions given by<br>> >>>>> DACreateNd ? The next MG-Grids are always finer. Can I make DMMG
<br>> >>>>> start on the finest grid with dimensions given in DACreateNd ?<br>> >>>><br>> >>>> You need to create a DA with a coarser size so that after it is<br>> >>>> refined the number of times it gives you the grid you want. That is if
<br>> >>>> you want 5 grid points with two levels you would create a DA with 3<br>> >>>> grid points and pass that into the DMMG. Sorry there is no way to<br>> >>>> start with a DA on the finest (but you get the same effect by starting
<br>> >>>> with a coarser DA.<br>> >>>><br>> >>>> Barry<br>> >>>><br>> >>>>> Cheers and thanks Jens<br>> >>>>><br>> >>>>> ________________________________
<br>> >>>>><br>> >>>>> Fra: Barry Smith [mailto:<a href="mailto:petsc-maint@mcs.anl.gov">petsc-maint@mcs.anl.gov</a>]<br>> >>>>> Sendt: on 29-03-2006 17:20<br>> >>>>> Til:
<a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk</a><br>> >>>>> Cc: <a href="mailto:petsc-maint@mcs.anl.gov">petsc-maint@mcs.anl.gov</a><br>> >>>>> Emne: Re: [PETSC #14613] DMMG question
<br>> >>>>><br>> >>>>><br>> >>>>><br>> >>>>><br>> >>>>> Jens,<br>> >>>>><br>> >>>>> YOu can access any of the low level PCMG options from
<br>> >>>>> DMMG. For example, -pc_type richardson gives you "multigrid<br>> >>>>> as a solve". -pc_mg_type additive gives you additive<br>> >>>>> -pc_mg_type multiplicative gives you standard v or w cycel
<br>> >>>>> -pc_mg_type full gives "full" multigrid". -pg_mg_cycles 2<br>> >>>>> gives W cycle. -pc_mg_levels_pc_type sor gives SOR as the smoother<br>> >>>>> etc etc etc. Run with -help to see all the choices for the parts
<br>> >>>>> of the multigrid process.<br>> >>>>><br>> >>>>> Barry<br>> >>>>><br>> >>>>> As Matt noted, using the PCMG directly requires YOU provide
<br>> >>>>> a mesh infrastructure that manages the meshes, thus it is not<br>> >>>>> realistic for us to provide this whole infrastructure in an<br>> >>>>> example. We currently only provide the full infrastructure
<br>> >>>>> for structured grids (in DMMG).<br>> >>>>><br>> >>>>> On Wed, 29 Mar 2006, <a href="mailto:jens.madsen@risoe.dk">jens.madsen@risoe.dk</a> wrote:<br>> >>>>>> Hi Petsc :-)
<br>> >>>>>><br>> >>>>>><br>> >>>>>><br>> >>>>>> I am trying to use Petsc for solving plasma fluid equations. Is it<br>> >>>>>> possible to use the DMMG with multiplicative or additive multigrid
<br>> >>>>>> schemes ? Also can I use multigrid as a solver not only as a<br>> >>>>>> preconditioner in DMMG ?<br>> >>>>>><br>> >>>>>><br>
> >>>>>><br>> >>>>>> Also I cannot find any examples using the "low-level" PCMG<br>> >>>>>> multigrid interface ? Only a testprogram in the PC/TEST directory ?
<br>> >>>>>><br>> >>>>>><br>> >>>>>><br>> >>>>>> Cheers Jens Madsen<br><br></blockquote></div><br><br clear="all"><br>-- <br>"Failure has a thousand explanations. Success doesn't need one" -- Sir Alec Guiness