<html><head></head><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:16px"><div id="yui_3_16_0_1_1468907431186_45006">Thanks. </div><div id="yui_3_16_0_1_1468907431186_45006"><br></div><div id="yui_3_16_0_1_1468907431186_45006">KSPSetOperators() allows to precondition A^h with M^h. </div><div id="yui_3_16_0_1_1468907431186_45006">This is lovely and great as it allows to implement the shifted Laplace </div><div id="yui_3_16_0_1_1468907431186_45006">preconditioner for the Helmholtz equation. </div><div id="yui_3_16_0_1_1468907431186_45006"><br></div><div id="yui_3_16_0_1_1468907431186_45006">Recently I managed to implement shifted Laplace using the DMDA </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">infrastructure in 2D. This implementation avoids having to construct </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">the hierarchy in Matlab as we did previously. </div><div id="yui_3_16_0_1_1468907431186_45006"><br></div><div id="yui_3_16_0_1_1468907431186_45006">In next stage we would like to precondition A^H with M^H on a sequence </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">of coarser grids. This is what Calandra does on two levels and what we do</div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">on multiple levels. </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr"><br></div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">We currently have an implement in which we construct the hierarchy on A^h </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">and M^h in Matlab, we read the hierarchy in PETSc, traverse the hierarchy and </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">do SetOperators and do a lot more of dark magic and witch craft by combining </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">preconditioners in a additive and multiplicative fashion. </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr"><br></div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">It would be lovely to obtain a more readable piece of code. </div><div id="yui_3_16_0_1_1468907431186_45006"><br></div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">I am not sure what kind of additional callbacks I need. My first guess here </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">would be a multilevel extension of SetOperators allowing to define M^H </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">a preconditioner for A^H on a sequence of coarser levels. But I currently </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">fail to oversee the whole matter. </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr"><br></div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">An alternative is to build a fragile code on top of DMDA first and get back </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">to you with more informed guesses on what kind of call backs I precisely need. </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">I think I prefer to go with this option. </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr"><br></div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">Does this sound reasonable? </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr"><br></div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr">Domenico. </div><div id="yui_3_16_0_1_1468907431186_45006" dir="ltr"><br></div><div class="qtdSeparateBR"><br><br></div><div class="yahoo_quoted" id="yui_3_16_0_1_1468907431186_45734" style="display: block;"> <div style="font-family: times new roman, new york, times, serif; font-size: 16px;" id="yui_3_16_0_1_1468907431186_45733"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;" id="yui_3_16_0_1_1468907431186_45732"> <div dir="ltr" id="yui_3_16_0_1_1468907431186_45731"> <font size="2" face="Arial" id="yui_3_16_0_1_1468907431186_46249"> <hr size="1" id="yui_3_16_0_1_1468907431186_46248"> <b><span style="font-weight:bold;">From:</span></b> Barry Smith <bsmith@mcs.anl.gov><br> <b><span style="font-weight: bold;">To:</span></b> domenico lahaye <domenico_lahaye@yahoo.com> <br><b><span style="font-weight: bold;">Cc:</span></b> PETSc Users List <petsc-users@mcs.anl.gov><br> <b><span style="font-weight: bold;">Sent:</span></b> Thursday, July 21, 2016 5:49 AM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [petsc-users] Regarding ksp ex42 - Citations<br> </font> </div> <div class="y_msg_container" id="yui_3_16_0_1_1468907431186_47473"><br><br clear="none">> On Jul 18, 2016, at 1:41 AM, domenico lahaye <<a shape="rect" ymailto="mailto:domenico_lahaye@yahoo.com" href="mailto:domenico_lahaye@yahoo.com">domenico_lahaye@yahoo.com</a>> wrote:<br clear="none">> <br clear="none">> Dear Matthew, <br clear="none">> <br clear="none">> I would like to place the FormJacobian statement in ex25.c in such a way that I can view <br clear="none">> the result on the different levels. Can you please point me to an example? <br clear="none">> <br clear="none">> I would like to do above with Galerkin coarsening as well. So yes, I do expect that I will need the <br clear="none">> hooks attached to the different MG levels. I appreciate more pointers here as well. <br clear="none"><br clear="none"> The thing is some parts of the solver may not be constructed on each level until the actual solve is performed so it may not be possible to view/change things before the solve starts. You can try calling KSPSetUp() and then do as Matt suggested<br clear="none"><br clear="none">"You can always call<br clear="none"><br clear="none"> <a shape="rect" href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCMGGetSmoother.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCMGGetSmoother.html</a><br clear="none"><br clear="none">and then<br clear="none"><br clear="none"> <a shape="rect" href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPGetPC.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/KSP/KSPGetPC.html</a><br clear="none"><br clear="none">and then<br clear="none"><br clear="none"> <a shape="rect" href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCGetOperators.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCGetOperators.html</a><br clear="none"><br clear="none">I would caution you against this, since it is very fragile in the code."<br clear="none"><br clear="none"> When using SNES there is really no good time to call KSPSetUp() and then access the PCMGGetSmoother(). This is why PETSc is designed around callbacks, so rather than having you look over MG levels and get some object and modify it, you provide callbacks that SNES or KSP calls at the appropriate time with a single object and then your callback function does what you want it to do. If there are additional callbacks you think we should add please let us know.<br clear="none"><br clear="none"> Barry<div class="yqt4511557646" id="yqtfd39419"><br clear="none"><br clear="none"><br clear="none"><br clear="none"><br clear="none">> <br clear="none"><br clear="none">> Thanks, Domenico. <br clear="none">> <br clear="none">> <br clear="none">> From: Matthew Knepley <<a shape="rect" ymailto="mailto:knepley@gmail.com" href="mailto:knepley@gmail.com">knepley@gmail.com</a>><br clear="none">> <br clear="none">> <br clear="none">> To: domenico lahaye <<a shape="rect" ymailto="mailto:domenico_lahaye@yahoo.com" href="mailto:domenico_lahaye@yahoo.com">domenico_lahaye@yahoo.com</a>> <br clear="none">> Cc: PETSc Users List <<a shape="rect" ymailto="mailto:petsc-users@mcs.anl.gov" href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>><br clear="none">> Sent: Monday, July 18, 2016 8:16 AM<br clear="none">> Subject: Re: [petsc-users] Regarding ksp ex42 - Citations<br clear="none">> <br clear="none">> On Mon, Jul 18, 2016 at 12:59 AM, domenico lahaye <<a shape="rect" ymailto="mailto:domenico_lahaye@yahoo.com" href="mailto:domenico_lahaye@yahoo.com">domenico_lahaye@yahoo.com</a>> wrote:<br clear="none">> Thanks for all the pointers. <br clear="none">> <br clear="none">> I am happy to switch to ksp/examples/tutorials/ex25.c in a first instance as you suggest.<br clear="none">> <br clear="none">> I am still stuck with the same issue as before though. I am trying to extract the hierarchy <br clear="none">> of coarser grid matrices and the intergrid transfer operators from the DMDA data structure. I would <br clear="none">> like to modify these operators and define a multigrid cycle with the modified operators. <br clear="none">> <br clear="none">> Given A^h (Helmholtz) and M^h (shifted Laplace), I would like to define a multigrid cycle involving <br clear="none">> both A^H and M^H. Can I rely on the multilevel DMDA structure to construct A^H and M^H for me <br clear="none">> in a set-up phase, plug them into a user-defined context, and plug them back out in a solve phase? <br clear="none">> <br clear="none">> If you are not using -pc_mg_galerkin, then the FormJacobian is called separately on each level to rediscretize the operator.<br clear="none">> The only thing that changes is the DMDA that is passed to the call. If you need more information, there are hooks to<br clear="none">> attach different contexts to each MG level. Do you need this?<br clear="none">> <br clear="none">> Thanks,<br clear="none">> <br clear="none">> Matt<br clear="none">> <br clear="none">> Thanks, Domenico. <br clear="none">> <br clear="none">> <br clear="none">> From: Matthew Knepley <<a shape="rect" ymailto="mailto:knepley@gmail.com" href="mailto:knepley@gmail.com">knepley@gmail.com</a>><br clear="none">> To: Barry Smith <<a shape="rect" ymailto="mailto:bsmith@mcs.anl.gov" href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> <br clear="none">> Cc: domenico lahaye <<a shape="rect" ymailto="mailto:domenico_lahaye@yahoo.com" href="mailto:domenico_lahaye@yahoo.com">domenico_lahaye@yahoo.com</a>>; "<a shape="rect" ymailto="mailto:petsc-users@mcs.anl.gov" href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>" <<a shape="rect" ymailto="mailto:petsc-users@mcs.anl.gov" href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>><br clear="none">> Sent: Sunday, July 17, 2016 2:29 PM<br clear="none">> Subject: Re: [petsc-users] Regarding ksp ex42 - Citations<br clear="none">> <br clear="none">> On Sat, Jul 16, 2016 at 10:11 PM, Barry Smith <<a shape="rect" ymailto="mailto:bsmith@mcs.anl.gov" href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br clear="none">> <br clear="none">> > On Jul 14, 2016, at 12:21 PM, domenico lahaye <<a shape="rect" ymailto="mailto:domenico_lahaye@yahoo.com" href="mailto:domenico_lahaye@yahoo.com">domenico_lahaye@yahoo.com</a>> wrote:<br clear="none">> ><br clear="none">> > Dear PETSc team,<br clear="none">> ><br clear="none">> > 1) I am looking into ks/examples/tutorials/ex42.c I am still new to the DMDA structure<br clear="none">> > and likely not giving it as much time as it deserves. However, I do not see immediately<br clear="none">> > what function is responsible for calling PCMGSetSmoother and PCMGSetResidual.<br clear="none">> ><br clear="none">> > I tried to call PCMGGetCoarseSolve(pc, &kcpc) and subsequently<br clear="none">> > KSPGetOperators (kspc, ... ) to check how the coarse grid operator is defined<br clear="none">> > after calling DMCoarsenHierarchy, but that failed.<br clear="none">> ><br clear="none">> > I am solving Helmholtz with shifted Laplace, and managed to exploit DMDA to perform<br clear="none">> > a multigrid solve on the preconditioner. In a next stage I want to implement the deflation<br clear="none">> > using DMDA as well.<br clear="none">> ><br clear="none">> > 2) On <a shape="rect" href="http://www.mcs.anl.gov/petsc/documentation/referencing.html" target="_blank">http://www.mcs.anl.gov/petsc/documentation/referencing.html </a>I see<br clear="none">> ><br clear="none">> > @Misc{petsc-web-page,<br clear="none">> > author = {Satish Balay and Shrirang Abhyankar and Mark~F. Adams and Jed Brown and Peter Brune<br clear="none">> > and Kris Buschelman and Lisandro Dalcin and Victor Eijkhout and William~D. Gropp<br clear="none">> > and Dinesh Kaushik and Matthew~G. Knepley<br clear="none">> > and Lois Curfman McInnes and Karl Rupp and Barry~F. Smith<br clear="none">> > and Stefano Zampini and Hong Zhang and Hong Zhang},<br clear="none">> > title = {{PETS}c {W}eb page},<br clear="none">> > url = {<a shape="rect" href="http://www.mcs.anl.gov/petsc" target="_blank">http://www.mcs.anl.gov/petsc</a>},<br clear="none">> > howpublished = {\url{<a shape="rect" href="http://www.mcs.anl.gov/petsc" target="_blank">http://www.mcs.anl.gov/petsc</a>}},<br clear="none">> > year = {2016}<br clear="none">> > }<br clear="none">> ><br clear="none">> ><br clear="none">> ><br clear="none">> > Is the last author mentioned twice intentionally?<br clear="none">> ><br clear="none">> > 3) On <a shape="rect" href="http://www.mcs.anl.gov/petsc/publications/petscapps-bib.html#OpenFOAM%202.2.1" target="_blank">http://www.mcs.anl.gov/petsc/publications/petscapps-bib.html#OpenFOAM%202.2.1 </a>I see<br clear="none">> ><br clear="none">> > @misc{OpenFOAM<br clear="none">> > ,<br clear="none">> ><br clear="none">> ><br clear="none">> > title = "OpenFOAM",<br clear="none">> ><br clear="none">> > howpublished = "\url{<a shape="rect" href="http://www.openfoam.com/" target="_blank">http://www.openfoam.com</a>}",<br clear="none">> ><br clear="none">> > url = {<a shape="rect" href="http://www.openfoam.com/" target="_blank">http://www.openfoam.com</a>},<br clear="none">> ><br clear="none">> > note = "OpenFOAM is a free, open source CFD software package. It allows PETSc linear algebra and solvers to be used underneath.",<br clear="none">> ><br clear="none">> > key = "OpenFOAM 2.2.1"<br clear="none">> ><br clear="none">> > }<br clear="none">> ><br clear="none">> ><br clear="none">> > Do you have more information on the use of PETSc within OpenFoam?<br clear="none">> <br clear="none">> Very good question. It seems that this citation is wrong or no longer valid; I have removed it from the PETSc repository. I could find no mention of PETSc usage in the OpenFoam and its third party packages. I think we should not have been listing this citation.<br clear="none">> <br clear="none">> This suggests that people are using it with OpenFOAM: <a shape="rect" href="http://powerlab.fsb.hr/ped/kturbo/OpenFOAM/slides/PatersonNuTTS2009.pdf" target="_blank">http://powerlab.fsb.hr/ped/kturbo/OpenFOAM/slides/PatersonNuTTS2009.pdf</a><br clear="none">> <br clear="none">> In fact, they use PETSc in the dynamic overset grid implementation for OpenFOAM, which I think is an approved extension:<br clear="none">> <br clear="none">> <a shape="rect" href="http://web.student.chalmers.se/groups/ofw5/Abstracts/DavidBogerAbstractOFW5.pdf" target="_blank">http://web.student.chalmers.se/groups/ofw5/Abstracts/DavidBogerAbstractOFW5.pdf</a><br clear="none">> <br clear="none">> Matt<br clear="none">> <br clear="none">> <br clear="none">> Barry<br clear="none">> <br clear="none">> ><br clear="none">> > 4) @matt in response to a question he raised in Vienna<br clear="none">> ><br clear="none">> > MIPSE is a BEM solver. Details are on:<br clear="none">> > <a shape="rect" href="http://www.g2elab.grenoble-inp.fr/plateforms/mipse-modeling-of-interconnected-power-systems-632862.kjsp?RH=G2ELAB_R-MAGE" target="_blank">http://www.g2elab.grenoble-inp.fr/plateforms/mipse-modeling-of-interconnected-power-systems-632862.kjsp?RH=G2ELAB_R-MAGE</a><br clear="none">> ><br clear="none">> > Cheers, Domenico Lahaye.<br clear="none">> <br clear="none">> ><br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> -- <br clear="none">> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br clear="none">> -- Norbert Wiener<br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> <br clear="none">> -- <br clear="none">> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br clear="none">> -- Norbert Wiener<br clear="none">> <br clear="none">> <br clear="none"></div><br><br></div> </div> </div> </div></div></body></html>