<div dir="ltr"><div dir="ltr">On Mon, May 16, 2022 at 9:59 AM Loic Gouarin <<a href="mailto:loic.gouarin@polytechnique.edu">loic.gouarin@polytechnique.edu</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Le 16/05/2022 à 21:54, Barry Smith a
écrit :<br></p>
<blockquote type="cite">
<br>
<div><br>
<blockquote type="cite">
<div>On May 16, 2022, at 3:50 PM, Loic Gouarin <<a href="mailto:loic.gouarin@polytechnique.edu" target="_blank">loic.gouarin@polytechnique.edu</a>>
wrote:</div>
<br>
<div>
<div bgcolor="#FFFFFF">
<p>Thanks Barry for your quick reply.<br>
</p>
<div>Le 16/05/2022 à 21:41, Barry
Smith a écrit :<br>
</div>
<blockquote type="cite">
<div><br>
</div>
Loic,
<div><br>
</div>
<div> From your code it looks like you are
using a DM. Is it a DMDA or a shell DM? If it is a
DMDA then the process is intended to be pretty
straightforward. PCTELESCOPE should create a new DMDA
that has the same properties as the coarse grid DMDA
but lives on a smaller number of ranks. From this you
can just provide multiple levels of coarsening of the
DMDA to produce the smaller multigrid problems. <br>
</div>
</blockquote>
It's a DMDA. So what you mean is to take the KSP of
PCTELESCOPE, get the DM which is the same of the coarse
grid and build all the levels as previously for the levels
of PCTELESCOPE ?<br>
</div>
</div>
</blockquote>
<div><br>
</div>
Yes, I think you should be able to pull out of the coarse
level PC (on the subset of ranks) a DM that and then setup
coarse levels from that. But you need to look at an example that
that uses telescope and probably even the telescope code itself
to see all the details. I don't know them anymore.</div>
</blockquote>
<p>I looked at the source code but It didn't help. I will try to
explain more clearly my experiments to understand what I make
wrong and you can help more easily.</p>
<p></p></div></blockquote><div>Hi Loic!</div><div><br></div><div>Here is my guess. When TELESCOPE redistributes the coarse grid DMDA, it creates a new DMDA and copies over all the information it can see. However, I think you may have state that is attached to the old DMDA that is not getting redistributed. If this is true, I can help you write a hook that redistributes that state when TELESCOPE maps between distributions.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><p>Thanks,</p>
<p>Loic<br>
</p>
<blockquote type="cite">
<div><br>
</div>
<div> Barry</div>
<div><br>
<blockquote type="cite">
<div>
<div bgcolor="#FFFFFF">
<blockquote type="cite">
<div><br>
</div>
<div> Can you let us know more details of
what exactly goes wrong with attempts? It is likely
straightforward to get things working, perhaps due to
our unclear documentation.</div>
</blockquote>
<p>I essentially have wrong state or NULL
matrices into the PCTELESCOPE.</p>
<p>Loic<br>
</p>
<blockquote type="cite">
<div><br>
</div>
<div> Barry</div>
<div><br>
<div><br>
<blockquote type="cite">
<div>On May 16, 2022, at 3:28 PM, Loic
Gouarin <<a href="mailto:loic.gouarin@polytechnique.edu" target="_blank">loic.gouarin@polytechnique.edu</a>>
wrote:</div>
<br>
<div>
<div bgcolor="#FFFFFF">
<p>Hello,</p>
<p>I could have posted my message on
the user list but it seems to me that it's
more in the petsc development part. Don't
hesitate to tell me if I'm wrong.<br>
I am developing a code called cafes that
studies fluid-particle interactions in a
Stokes fluid. For that, we implemented the
whole solver in matrix free on a cartesian
grid. The preconditioner is naturally a
geometrical multigrid for the velocity. The
set of matrix free for each level is set by
hand (I worked with Matthew on this a long
time ago now!). <br>
I would have liked to implement telescope on
this part. I started it but I didn't get out
of it. So I would like to have your help.<br>
What I tried to do was to describe my
classical multigrid up to a level N1 and
then take the coarse solver and apply the
telescope preconditioner to it up to a level
N2. I don't know if this is the best way but
it seemed the most intuitive. The problem is
that I could not get the coarse matrix
because the setup is not done yet (I use a
DMKSPSetComputeOperators for the matrices).
<br>
</p>
<p>The construction of the matrix
free for each level is here: <a href="https://github.com/gouarin/cafes/blob/master/cafes/problem/stokes.hpp#L59-L88" target="_blank">https://github.com/gouarin/cafes/blob/master/cafes/problem/stokes.hpp#L59-L88</a></p>
<p>The description of the Stokes
preconditioner is described here : <a href="https://github.com/gouarin/cafes/blob/master/cafes/problem/stokes.hpp#L109-L162" target="_blank">https://github.com/gouarin/cafes/blob/master/cafes/problem/stokes.hpp#L109-L162</a></p>
<p>I have tried multiple
implementations without success. For me, the
start is to add at line 105 something like</p>
<div style="color:rgb(212,212,212);background-color:rgb(30,30,30);font-family:Menlo,Monaco,"Courier New",monospace;font-weight:normal;font-size:12px;line-height:18px;white-space:pre-wrap"><div><span style="color:rgb(87,166,74)">KSP coarse;</span></div><div><span style="color:rgb(87,166,74)">ierr = PCMGGetCoarseSolve(pc_i, &coarse);CHKERRQ(ierr);</span></div><span style="color:rgb(87,166,74)">
</span></div>
<div style="color:rgb(212,212,212);background-color:rgb(30,30,30);font-family:Menlo,Monaco,"Courier New",monospace;font-weight:normal;font-size:12px;line-height:18px;white-space:pre-wrap"><span style="color:rgb(87,166,74)">PC pc_coarse;</span><div><span style="color:rgb(87,166,74)">ierr = KSPGetPC(coarse, &pc_coarse);CHKERRQ(ierr);</span></div><div><span style="color:rgb(87,166,74)">ierr = PCSetType(pc_coarse, PCTELESCOPE);CHKERRQ(ierr);</span></div></div>
<div><span style="color:rgb(87,166,74)"><br>
</span></div>
<div>Then, to create the several
matrices of the levels of TELESCOPE, I
thought using the same process as before
with DMKSPSetComputeOperators. But it
doesn't work...<br>
</div>
<div>Could you tell me how I can make
it work ?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Loic<br>
</div>
<div><br>
</div>
<div><span style="color:rgb(87,166,74)"></span></div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
</blockquote>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>