<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style id="owaParaStyle" type="text/css">P {margin-top:0;margin-bottom:0;}</style>
</head>
<body ocsi="0" fpstyle="1" class="" style="word-wrap:break-word">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">Thanks Lorenzo<br>
<br>
If I well understood what you say is<br>
<br>
Set up PCASM for the smoother_down[i] and then<br>
feed it up to smoother_up[i]<br>
<br>
Does this assure that only one <br>
memory allocation is done for PCASM?<br>
<br>
<br>
Eugenio
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div style="direction: ltr;" id="divRpF704458"><font face="Tahoma" size="2" color="#000000"><b>From:</b> Lorenzo Alessio Botti [lorenzoalessiobotti@gmail.com]<br>
<b>Sent:</b> Thursday, August 20, 2015 5:29 AM<br>
<b>To:</b> petsc-users@mcs.anl.gov<br>
<b>Cc:</b> Aulisa, Eugenio<br>
<b>Subject:</b> GMRES -> PCMG -> PCASM pre- post- smoother<br>
</font><br>
</div>
<div></div>
<div>I tried to achieve this behaviour getting all the smothers and setting the same preconditioner to the down and up smoother on the same level.
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
smoothers.resize(nLevels+<span class="" style="color:#ce7924">1</span>);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
smoothers_up.resize(nLevels);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
<span class="" style="color:#afad24">for</span> (PetscInt i = <span class="" style="color:#ce7924">
0</span>; i < nLevels; i++)</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
{</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
PCMGGetSmootherDown(M_pc,nLevels-i,&(smoothers[i]));</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPSetInitialGuessNonzero(smoothers[i],PETSC_TRUE); <span class="" style="color:#34bbc7">
// for full and wCicle</span></p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
PCMGGetSmootherUp(M_pc,nLevels-i,&(smoothers_up[i]));</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
}</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
PCMGSetNumberSmoothDown(M_pc,<span class="" style="color:#ce7924">1</span>);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
PCMGSetNumberSmoothUp(M_pc,<span class="" style="color:#ce7924">1</span>);</p>
<div class=""><br class="">
</div>
<div class="">… set coarse solver options here</div>
<div class=""><br class="">
</div>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
<span class="" style="color:#afad24">for</span> (PetscInt i = <span class="" style="color:#ce7924">
0</span>; i < nLevels; i++)</p>
<div class=""><span class="" style="color:rgb(245,245,245); font-family:Menlo; font-size:14px; background-color:rgb(0,0,0)"> </span><span class="" style="color:rgb(245,245,245); font-family:Menlo; font-size:14px; background-color:rgb(0,0,0)">{</span></div>
<div class=""><span class="" style="color:rgb(245,245,245); font-family:Menlo; font-size:14px; background-color:rgb(0,0,0)"> PC pc;</span></div>
<div class="">
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPSetType(smoothers[i], KSPGMRES);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPGetPC(smoothers[i], &pc);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPSetPCSide(smoothers[i], PC_RIGHT);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
PCSetType(pc, PCASM);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
PCFactorSetPivotInBlocks(pc, PETSC_TRUE);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
PCFactorSetAllowDiagonalFill(pc);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
PCFactorSetReuseFill(pc, PETSC_TRUE);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
PCFactorSetReuseOrdering(pc, PETSC_TRUE);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPSetType(smoothers_up[i], KSPGMRES);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPSetPC(smoothers_up[i], pc);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPSetPCSide(smoothers_up[i], PC_RIGHT);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPSetConvergenceTest(smoothers[i],KSPConvergedSkip,<span class="" style="color:#ce7924">NULL</span>,<span class="" style="color:#ce7924">NULL</span>);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPSetConvergenceTest(smoothers_up[i],KSPConvergedSkip,<span class="" style="color:#ce7924">NULL</span>,<span class="" style="color:#ce7924">NULL</span>);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPSetNormType(smoothers[i],KSP_NORM_NONE);</p>
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
KSPSetNormType(smoothers_up[i],KSP_NORM_NONE);</p>
<div class="">
<p class="" style="margin:0px; font-size:14px; font-family:Menlo; color:rgb(245,245,245); background-color:rgb(0,0,0)">
}</p>
</div>
</div>
<div class=""><br class="">
</div>
<div class="">Is this correct?</div>
<div class="">Note moreover that for Full Multigrid and W cicles to work as expected I need to add the KSPSetInitialGuessNonZero option.</div>
<div class=""><br class="">
</div>
<div class="">Bests</div>
<div class="">Lorenzo</div>
<div class=""><br class="">
</div>
<blockquote type="cite" class="">Message: 4<br class="">
Date: Thu, 20 Aug 2015 02:37:13 -0500<br class="">
From: Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" class="" target="_blank">bsmith@mcs.anl.gov</a>><br class="">
To: "Aulisa, Eugenio" <<a href="mailto:eugenio.aulisa@ttu.edu" class="" target="_blank">eugenio.aulisa@ttu.edu</a>><br class="">
Cc: "<a href="mailto:petsc-users@mcs.anl.gov" class="" target="_blank">petsc-users@mcs.anl.gov</a>" <<a href="mailto:petsc-users@mcs.anl.gov" class="" target="_blank">petsc-users@mcs.anl.gov</a>><br class="">
Subject: Re: [petsc-users] GMRES -> PCMG -> PCASM pre- post- smoother<br class="">
Message-ID: <<a href="mailto:1CF3ABE1-214C-4BBC-B8FF-93416EC26EFC@mcs.anl.gov" class="" target="_blank">1CF3ABE1-214C-4BBC-B8FF-93416EC26EFC@mcs.anl.gov</a>><br class="">
Content-Type: text/plain; charset="us-ascii"<br class="">
<br class="">
<br class="">
What you describe is not the expected behavior. I expected exactly the result that you expected.<br class="">
<br class="">
Do you perhaps have some PETSc options around that may be changing the post-smoother? On the command line or in the file petscrc or in the environmental variable PETSC_OPTIONS? Can you send us some code that we could run that reproduces the problem?<br class="">
<br class="">
Barry<br class="">
<br class="">
<blockquote type="cite" class="">On Aug 19, 2015, at 9:26 PM, Aulisa, Eugenio <<a href="mailto:eugenio.aulisa@ttu.edu" class="" target="_blank">eugenio.aulisa@ttu.edu</a>> wrote:<br class="">
<br class="">
Hi,<br class="">
<br class="">
I am solving an iteration of<br class="">
<br class="">
GMRES -> PCMG -> PCASM <br class="">
<br class="">
where I build my particular ASM domain decomposition.<br class="">
<br class="">
In setting the PCMG I would like at each level <br class="">
to use the same pre- and post-smoother<br class="">
and for this reason I am using <br class="">
...<br class="">
PCMGGetSmoother ( pcMG, level , &subksp );<br class="">
<br class="">
to extract and set at each level the ksp object.<br class="">
<br class="">
In setting PCASM then I use<br class="">
...<br class="">
KSPGetPC ( subksp, &subpc );<br class="">
PCSetType ( subpc, PCASM );<br class="">
...<br class="">
and then set my own decomposition<br class="">
...<br class="">
PCASMSetLocalSubdomains(subpc,_is_loc_idx.size(),&_is_ovl[0],&_is_loc[0]);<br class="">
...<br class="">
<br class="">
Now everything compiles, and runs with no memory leakage, <br class="">
but I do not get the expected convergence.<br class="">
<br class="">
When I checked the output of -ksp_view, I saw something that puzzled me:<br class="">
at each level >0, while in the MG pre-smoother the ASM domain decomposition<br class="">
is the one that I set, for example with 4 processes I get<br class="">
<br class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class=""><br class="">
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
...<br class="">
Down solver (pre-smoother) on level 2 -------------------------------<br class="">
KSP Object: (level-2) 4 MPI processes<br class="">
type: gmres<br class="">
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br class="">
GMRES: happy breakdown tolerance 1e-30<br class="">
maximum iterations=1<br class="">
using preconditioner applied to right hand side for initial guess<br class="">
tolerances: relative=1e-12, absolute=1e-20, divergence=1e+50<br class="">
left preconditioning<br class="">
using nonzero initial guess<br class="">
using NONE norm type for convergence test<br class="">
PC Object: (level-2) 4 MPI processes<br class="">
type: asm<br class="">
Additive Schwarz: total subdomain blocks = 198, amount of overlap = 0<br class="">
Additive Schwarz: restriction/interpolation type - RESTRICT<br class="">
[0] number of local blocks = 52<br class="">
[1] number of local blocks = 48<br class="">
[2] number of local blocks = 48<br class="">
[3] number of local blocks = 50<br class="">
Local solve info for each block is in the following KSP and PC objects:<br class="">
- - - - - - - - - - - - - - - - - -<br class="">
...<br class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class=""><br class="">
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<br class="">
<br class="">
in the post-smoother I have the default ASM decomposition with overlapping 1:<br class="">
<br class="">
<br class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class=""><br class="">
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
...<br class="">
Up solver (post-smoother) on level 2 -------------------------------<br class="">
KSP Object: (level-2) 4 MPI processes<br class="">
type: gmres<br class="">
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br class="">
GMRES: happy breakdown tolerance 1e-30<br class="">
maximum iterations=2<br class="">
tolerances: relative=1e-12, absolute=1e-20, divergence=1e+50<br class="">
left preconditioning<br class="">
using nonzero initial guess<br class="">
using NONE norm type for convergence test<br class="">
PC Object: (level-2) 4 MPI processes<br class="">
type: asm<br class="">
Additive Schwarz: total subdomain blocks = 4, amount of overlap = 1<br class="">
Additive Schwarz: restriction/interpolation type - RESTRICT<br class="">
Local solve is same for all blocks, in the following KSP and PC objects:<br class="">
KSP Object: (level-2sub_) 1 MPI processes<br class="">
type: preonly<br class="">
maximum iterations=10000, initial guess is zero<br class="">
tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br class="">
left preconditioning<br class="">
...<br class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">
<blockquote type="cite" class=""><br class="">
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
%%%%%%%%%%%%%%%%%%%%%%%%<br class="">
<br class="">
So it seams that by using<br class="">
<br class="">
PCMGGetSmoother ( pcMG, level , &subksp );<br class="">
<br class="">
I was capable to set both the pre- and post- smoothers to be PCASM<br class="">
but everything I did after that applied only to the<br class="">
pre-smoother, while the post-smoother got the default PCASM options.<br class="">
<br class="">
I know that I can use<br class="">
PCMGGetSmootherDown and PCMGGetSmootherUp, but that would <br class="">
probably double the memory allocation and the computational time in the ASM.<br class="">
<br class="">
Is there any way I can just use PCMGGetSmoother <br class="">
and use the same PCASM in the pre- and post- smoother?<br class="">
<br class="">
I hope I was clear enough.<br class="">
<br class="">
Thanks a lot for your help,<br class="">
Eugenio<br class="">
<br class="">
<br class="">
</blockquote>
</blockquote>
</div>
</div>
</div>
</div>
</body>
</html>