<p>I would be inclined to put the complexity in MatAIJ, but hopefully have it reuse the special-purpose comm routines.</p>
<div class="gmail_quote">On Jun 9, 2012 8:47 PM, "Barry Smith" <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
On Jun 9, 2012, at 8:26 PM, Jed Brown wrote:<br>
<br>
> I think he wants to manage the messages at a higher level and call a matrix kernel for the partial sweeps.<br>
><br>
> I would prefer to write the communication primitives so that GS for each matrix format is simple.<br>
<br>
I am not sure if this is the right model. But it is an alternative approach.<br>
<br>
Do you envision PCApply_SOR() managing all the stages of MPI sends and receives? Or a new PCApply_PGS() managing that process.<br>
<br>
I admit I was thinking the Mat is responsible for its various SORs, but other alternatives as you propose seem reasonable.<br>
<br>
From the user prospective I think it is better not bot have both PCSOR and PCSPGS but to have options for PCSOR to make it it support PGS. How it is handled internally with complicated PCApply_SOR or complicated MatSOR_MPIAIJ() etc is more a detail I am less concerned about.<br>
<br>
<br>
Barry<br>
<br>
<br>
><br>
> On Jun 9, 2012 8:21 PM, "Barry Smith" <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
><br>
> On Jun 9, 2012, at 8:18 PM, Jed Brown wrote:<br>
><br>
> > He is going to alternate between smoothing some points and sending messages.<br>
><br>
> Fine but that is all INSIDE a single SOR sweep? So I was wrong to say PCSORSetIS() maps to MatSORSetIS() it is PCSORSSetISs() maps to MatSORSetISs() he won't be calling MatSOR() for each piece but ONCE for the whole process (yes the MatSOR_SeqAIJ() is more complicated in this case.<br>
><br>
> Barry<br>
><br>
> ><br>
> > On Jun 9, 2012 8:07 PM, "Barry Smith" <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
> ><br>
> > On Jun 9, 2012, at 8:01 PM, Jed Brown wrote:<br>
> ><br>
> > > Parallel Gauss-Seidel.<br>
> ><br>
> > But if you know in advance the IS that you are providing (that determines the order of the nodes smoothed) then why would you change it the next iteration? That is, if you are providing the IS then it is in no way asynchronous so that the fact that it is "parallel" Gauss-Seidel doesn't affect the ordering. Hence I consider your response humorous but non responsive :-)<br>
> ><br>
> > Barry<br>
> ><br>
> ><br>
> ><br>
> ><br>
> > ><br>
> > > On Jun 9, 2012 7:56 PM, "Barry Smith" <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
> > ><br>
> > > On Jun 9, 2012, at 7:47 PM, Jed Brown wrote:<br>
> > ><br>
> > > > Fine, but I think Mark is going to change the IS every time MatSOR is called.<br>
> > ><br>
> > > Surely not. What kind of weird-ass algorithm would that be?<br>
> > ><br>
> > > Barry<br>
> > ><br>
> > > > Either will work, but a separate call is awkward if it's not useful to be persistent.<br>
> > > ><br>
> > > > On Jun 9, 2012 7:45 PM, "Barry Smith" <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
> > > ><br>
> > > > On Jun 9, 2012, at 6:51 PM, Jed Brown wrote:<br>
> > > ><br>
> > > > > On Sat, Jun 9, 2012 at 6:43 PM, Mark F. Adams <<a href="mailto:mark.adams@columbia.edu">mark.adams@columbia.edu</a>> wrote:<br>
> > > > > 1) I need a G-S kernel that takes an IS of indices to process and a flag to process them in forward or reverse order. How should I proceed to do this. Should I just clone sor?<br>
> > > > ><br>
> > > > > You are going to have several of these index sets? You could have a PCSORSetIS(). Probably need to add a MatOp for MatSORIS(). Barry might have other ideas.<br>
> > > ><br>
> > > > PCSORSetIS() would then go down to MatSORSetIS() and then the call to MatSOR() would using the IS ordering if provided, otherwise use the default natural ordering?<br>
> > > ><br>
> > > > I don't see a need to add a MatSORIS().<br>
> > > ><br>
> > > > Barry<br>
> > > ><br>
> > > ><br>
> > > > ><br>
> > > > > 2) I don't want to use Richardson iterations for G-S. Should I make a G-S KPS method? I don't want to take a residual in the iterator (KSP) and if symmetric G-S is requested then it should drive this I think.<br>
> > > > ><br>
> > > > > Look at PCApplyRichardson_SOR().<br>
> > > > ><br>
> > > > > SOR does two sweeps in each application; I'm not wild about that because a good way to run G-S in a V(1,1) cycle is to do a forward sweep in pre smoothing and a backward sweep in post smoothing.<br>
> > > > ><br>
> > > > > Well, MatSOR() has this flag MatSORType that can specify forward and reverse. You have one PC for the down-smoother and another for the up-smoother, then configure one to be a forward sweep and the other to be reverse.<br>
> > > ><br>
> > ><br>
> ><br>
><br>
<br>
</blockquote></div>