<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">I guess that only one memory allocation is done, but basically this is what I’m asking as well.</div><div class="">Is this correct? I’d be more confident if one of the developers confirmed this.</div><div class=""><br class=""></div><div class="">Lorenzo</div><div class=""><br class=""></div><br class=""><div><blockquote type="cite" class=""><div class="">On 20 Aug 2015, at 14:00, Aulisa, Eugenio <<a href="mailto:eugenio.aulisa@ttu.edu" class="">eugenio.aulisa@ttu.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; direction: ltr; font-family: Tahoma; font-size: 10pt;" class="">Thanks Lorenzo<br class=""><br class="">If I well understood what you say is<br class=""><br class="">Set up PCASM for the smoother_down[i] and then<br class="">feed it up to smoother_up[i]<br class=""><br class="">Does this assure that only one<span class="Apple-converted-space"> </span><br class="">memory allocation is done for PCASM?<br class=""><br class=""><br class="">Eugenio<div style="font-family: 'Times New Roman'; font-size: 16px;" class=""><hr tabindex="-1" class=""><div id="divRpF704458" style="direction: ltr;" class=""><font face="Tahoma" size="2" class=""><b class="">From:</b><span class="Apple-converted-space"> </span>Lorenzo Alessio Botti [<a href="mailto:lorenzoalessiobotti@gmail.com" class="">lorenzoalessiobotti@gmail.com</a>]<br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>Thursday, August 20, 2015 5:29 AM<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" class="">petsc-users@mcs.anl.gov</a><br class=""><b class="">Cc:</b><span class="Apple-converted-space"> </span>Aulisa, Eugenio<br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>GMRES -> PCMG -> PCASM pre- post- smoother<br class=""></font><br class=""></div><div class=""></div><div class="">I tried to achieve this behaviour getting all the smothers and setting the same preconditioner to the down and up smoother on the same level.<div class=""><br class=""></div><div class=""><br class=""></div><div class=""><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> smoothers.resize(nLevels+<span class="" style="color: rgb(206, 121, 36);">1</span>);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> smoothers_up.resize(nLevels);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> <span class="Apple-converted-space"> </span><span class="" style="color: rgb(175, 173, 36);">for</span><span class="Apple-converted-space"> </span>(PetscInt i =<span class="Apple-converted-space"> </span><span class="" style="color: rgb(206, 121, 36);">0</span>; i < nLevels; i++)</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> {</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> PCMGGetSmootherDown(M_pc,nLevels-i,&(smoothers[i]));</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPSetInitialGuessNonzero(smoothers[i],PETSC_TRUE);<span class="Apple-converted-space"> </span><span class="" style="color: rgb(52, 187, 199);">// for full and wCicle</span></div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> PCMGGetSmootherUp(M_pc,nLevels-i,&(smoothers_up[i]));</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> }</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> PCMGSetNumberSmoothDown(M_pc,<span class="" style="color: rgb(206, 121, 36);">1</span>);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> PCMGSetNumberSmoothUp(M_pc,<span class="" style="color: rgb(206, 121, 36);">1</span>);</div><div class=""><br class=""></div><div class="">… set coarse solver options here</div><div class=""><br class=""></div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> <span class="" style="color: rgb(175, 173, 36);">for</span><span class="Apple-converted-space"> </span>(PetscInt i =<span class="Apple-converted-space"> </span><span class="" style="color: rgb(206, 121, 36);">0</span>; i < nLevels; i++)</div><div class=""><span class="" style="color: rgb(245, 245, 245); font-family: Menlo; font-size: 14px; background-color: rgb(0, 0, 0);"> </span><span class="" style="color: rgb(245, 245, 245); font-family: Menlo; font-size: 14px; background-color: rgb(0, 0, 0);">{</span></div><div class=""><span class="" style="color: rgb(245, 245, 245); font-family: Menlo; font-size: 14px; background-color: rgb(0, 0, 0);"> PC pc;</span></div><div class=""><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPSetType(smoothers[i], KSPGMRES);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPGetPC(smoothers[i], &pc);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPSetPCSide(smoothers[i], PC_RIGHT);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> PCSetType(pc, PCASM);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> PCFactorSetPivotInBlocks(pc, PETSC_TRUE);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> PCFactorSetAllowDiagonalFill(pc);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> PCFactorSetReuseFill(pc, PETSC_TRUE);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> PCFactorSetReuseOrdering(pc, PETSC_TRUE);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPSetType(smoothers_up[i], KSPGMRES);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPSetPC(smoothers_up[i], pc);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPSetPCSide(smoothers_up[i], PC_RIGHT);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPSetConvergenceTest(smoothers[i],KSPConvergedSkip,<span class="" style="color: rgb(206, 121, 36);">NULL</span>,<span class="" style="color: rgb(206, 121, 36);">NULL</span>);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPSetConvergenceTest(smoothers_up[i],KSPConvergedSkip,<span class="" style="color: rgb(206, 121, 36);">NULL</span>,<span class="" style="color: rgb(206, 121, 36);">NULL</span>);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPSetNormType(smoothers[i],KSP_NORM_NONE);</div><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> KSPSetNormType(smoothers_up[i],KSP_NORM_NONE);</div><div class=""><div style="margin: 0px; font-size: 14px; font-family: Menlo; color: rgb(245, 245, 245); background-color: rgb(0, 0, 0);" class=""> }</div></div></div><div class=""><br class=""></div><div class="">Is this correct?</div><div class="">Note moreover that for Full Multigrid and W cicles to work as expected I need to add the KSPSetInitialGuessNonZero option.</div><div class=""><br class=""></div><div class="">Bests</div><div class="">Lorenzo</div><div class=""><br class=""></div><blockquote type="cite" class="">Message: 4<br class="">Date: Thu, 20 Aug 2015 02:37:13 -0500<br class="">From: Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" class="" target="_blank">bsmith@mcs.anl.gov</a>><br class="">To: "Aulisa, Eugenio" <<a href="mailto:eugenio.aulisa@ttu.edu" class="" target="_blank">eugenio.aulisa@ttu.edu</a>><br class="">Cc: "<a href="mailto:petsc-users@mcs.anl.gov" class="" target="_blank">petsc-users@mcs.anl.gov</a>" <<a href="mailto:petsc-users@mcs.anl.gov" class="" target="_blank">petsc-users@mcs.anl.gov</a>><br class="">Subject: Re: [petsc-users] GMRES -> PCMG -> PCASM pre- post- smoother<br class="">Message-ID: <<a href="mailto:1CF3ABE1-214C-4BBC-B8FF-93416EC26EFC@mcs.anl.gov" class="" target="_blank">1CF3ABE1-214C-4BBC-B8FF-93416EC26EFC@mcs.anl.gov</a>><br class="">Content-Type: text/plain; charset="us-ascii"<br class=""><br class=""><br class=""> What you describe is not the expected behavior. I expected exactly the result that you expected.<br class=""><br class="">Do you perhaps have some PETSc options around that may be changing the post-smoother? On the command line or in the file petscrc or in the environmental variable PETSC_OPTIONS? Can you send us some code that we could run that reproduces the problem?<br class=""><br class=""> Barry<br class=""><br class=""><blockquote type="cite" class="">On Aug 19, 2015, at 9:26 PM, Aulisa, Eugenio <<a href="mailto:eugenio.aulisa@ttu.edu" class="" target="_blank">eugenio.aulisa@ttu.edu</a>> wrote:<br class=""><br class="">Hi,<br class=""><br class="">I am solving an iteration of<br class=""><br class="">GMRES -> PCMG -> PCASM <br class=""><br class="">where I build my particular ASM domain decomposition.<br class=""><br class="">In setting the PCMG I would like at each level <br class="">to use the same pre- and post-smoother<br class="">and for this reason I am using <br class="">...<br class="">PCMGGetSmoother ( pcMG, level , &subksp );<br class=""><br class="">to extract and set at each level the ksp object.<br class=""><br class="">In setting PCASM then I use<br class="">...<br class="">KSPGetPC ( subksp, &subpc );<br class="">PCSetType ( subpc, PCASM );<br class="">...<br class="">and then set my own decomposition<br class="">...<br class="">PCASMSetLocalSubdomains(subpc,_is_loc_idx.size(),&_is_ovl[0],&_is_loc[0]);<br class="">...<br class=""><br class="">Now everything compiles, and runs with no memory leakage, <br class="">but I do not get the expected convergence.<br class=""><br class="">When I checked the output of -ksp_view, I saw something that puzzled me:<br class="">at each level >0, while in the MG pre-smoother the ASM domain decomposition<br class="">is the one that I set, for example with 4 processes I get<br class=""><br class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><br class=""></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote>...<br class="">Down solver (pre-smoother) on level 2 -------------------------------<br class=""> KSP Object: (level-2) 4 MPI processes<br class=""> type: gmres<br class=""> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br class=""> GMRES: happy breakdown tolerance 1e-30<br class=""> maximum iterations=1<br class=""> using preconditioner applied to right hand side for initial guess<br class=""> tolerances: relative=1e-12, absolute=1e-20, divergence=1e+50<br class=""> left preconditioning<br class=""> using nonzero initial guess<br class=""> using NONE norm type for convergence test<br class=""> PC Object: (level-2) 4 MPI processes<br class=""> type: asm<br class=""> Additive Schwarz: total subdomain blocks = 198, amount of overlap = 0<br class=""> Additive Schwarz: restriction/interpolation type - RESTRICT<br class=""> [0] number of local blocks = 52<br class=""> [1] number of local blocks = 48<br class=""> [2] number of local blocks = 48<br class=""> [3] number of local blocks = 50<br class=""> Local solve info for each block is in the following KSP and PC objects:<br class=""> - - - - - - - - - - - - - - - - - -<br class="">...<br class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><br class=""></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><br class=""><br class="">in the post-smoother I have the default ASM decomposition with overlapping 1:<br class=""><br class=""><br class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><br class=""></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote>...<br class="">Up solver (post-smoother) on level 2 -------------------------------<br class=""> KSP Object: (level-2) 4 MPI processes<br class=""> type: gmres<br class=""> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br class=""> GMRES: happy breakdown tolerance 1e-30<br class=""> maximum iterations=2<br class=""> tolerances: relative=1e-12, absolute=1e-20, divergence=1e+50<br class=""> left preconditioning<br class=""> using nonzero initial guess<br class=""> using NONE norm type for convergence test<br class=""> PC Object: (level-2) 4 MPI processes<br class=""> type: asm<br class=""> Additive Schwarz: total subdomain blocks = 4, amount of overlap = 1<br class=""> Additive Schwarz: restriction/interpolation type - RESTRICT<br class=""> Local solve is same for all blocks, in the following KSP and PC objects:<br class=""> KSP Object: (level-2sub_) 1 MPI processes<br class=""> type: preonly<br class=""> maximum iterations=10000, initial guess is zero<br class=""> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br class=""> left preconditioning<br class="">...<br class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><blockquote type="cite" class=""><br class=""></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote>%%%%%%%%%%%%%%%%%%%%%%%%<br class=""><br class="">So it seams that by using<br class=""><br class="">PCMGGetSmoother ( pcMG, level , &subksp );<br class=""><br class="">I was capable to set both the pre- and post- smoothers to be PCASM<br class="">but everything I did after that applied only to the<br class="">pre-smoother, while the post-smoother got the default PCASM options.<br class=""><br class="">I know that I can use<br class="">PCMGGetSmootherDown and PCMGGetSmootherUp, but that would <br class="">probably double the memory allocation and the computational time in the ASM.<br class=""><br class="">Is there any way I can just use PCMGGetSmoother <br class="">and use the same PCASM in the pre- and post- smoother?<br class=""><br class="">I hope I was clear enough.<br class=""><br class="">Thanks a lot for your help,<br class="">Eugenio</blockquote></blockquote></div></div></div></div></div></blockquote></div><br class=""></body></html>